4K and 8K ultra-high-definition field manufacturing method based on embedded technology

Abstract

The invention discloses a 4K and 8K ultra-high-definition field production method based on an embedded technology, and relates to the technical field of image production. Video signals are divided into a plurality of data packets by adopting a channel fragmentation technology, and are transmitted in parallel by adopting a plurality of transmission channels, so that the bandwidth is fully utilized, the bottleneck of a single channel is avoided, and the production efficiency is improved. The bandwidth dynamic scheduling mechanism can dynamically adjust the channel bandwidth according to real-time network conditions, improves the utilization rate of network resources, greatly reduces the bandwidth pressure, improves the transmission efficiency, avoids signal congestion, introduces a PTP1588 global time synchronization protocol, keeps microsecond-level synchronization of all cameras and signal sources when transmitting a large amount of 4K or 8K data, and improves the transmission efficiency. According to the method, time offset among video sources of multiple cameras is avoided, each angle and different signal streams of final video output can keep time sequence consistency, the consistency and coherence of visual effects are guaranteed, and the signal synchronization precision can be greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image production, and in particular to a 4K and 8K ultra-high definition on-site production method based on embedded technology. Background Art

[0002] In the live production of 4K and 8K ultra-high-definition videos, the synchronous processing of multi-channel video transmission and embedded control systems is a key link, which directly affects production efficiency, signal quality and system stability. Since 4K and 8K videos have extremely high resolution and frame rate, the amount of data generated is very large. The data volume of 4K video can reach hundreds of megabits per second, and 8K video can even exceed 1Gbps. In live production, real-time video streams from multiple cameras need to be processed simultaneously, so the requirements for transmission bandwidth and switching speed are extremely high.

[0003] When dealing with multi-channel 4K / 8K signals, the single-channel bandwidth of traditional SDI or HDMI interfaces is often insufficient, especially when transmitting uncompressed signals, which will cause video delays or signal quality degradation. In addition, video streams from different angles shot by multiple cameras need to be processed synchronously. If there is a delay between the signal sources, the final output video will be inconsistent, affecting the overall visual effect. Traditional SDI / HDMI interfaces are also prone to signal attenuation during long-distance transmission, and cannot meet the needs of on-site production in large studios or sports events.

[0004] Current solutions usually rely on multi-channel SDI (such as 12GSDI) or HDMI (such as HDMI2.1) technology. Some solutions respond by increasing the interface bandwidth and the number of links. However, when the transmission distance is too long, it is often necessary to use a relay amplifier or optical fiber transmission to reduce signal attenuation. Although increasing the number of channels or interface bandwidth can improve system performance to a certain extent, this method has limited scalability when processing 8K signals. Existing interfaces and transmission protocols are gradually unable to support higher resolutions and frame rates. At the same time, the introduction of a large number of additional hardware devices greatly increases the complexity of the system and increases the cost of debugging and maintenance. Therefore, a 4K and 8K ultra-high-definition live production method based on embedded technology is urgently needed to solve such problems. Summary of the invention

[0005] In view of the above existing problems, the present invention is proposed.

[0006] The present invention provides a 4K and 8K ultra-high-definition live production method based on embedded technology to solve the problem that it is difficult to meet the transmission bandwidth, signal synchronization and real-time control requirements in a live production environment.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0008] The present invention provides a 4K and 8K ultra-high definition on-site production method based on embedded technology, which comprises:

[0009] Step S1, video data acquisition,

[0010] Use 4K or 8K cameras that support high dynamic range (HDR) and wide color gamut (WCG) to collect video data. The cameras used have intelligent frame rate adjustment functions, which automatically adjust the frame rate according to the dynamics of the scene, and reduce the frame rate in static scenes. The video data is initially compressed and converted into a format.

[0011] Step S2, multi-channel signal transmission,

[0012] Based on the video data collected and preprocessed in step S1, the channel slicing technology is used to divide the video signal into multiple data packets, and the data packets are transmitted to the on-site production server in parallel through different transmission channels. The bandwidth dynamic scheduling mechanism is introduced here to automatically adjust the bandwidth resources according to real-time needs.

[0013] Step S3, real-time monitoring and real-time processing,

[0014] After the video signal reaches the on-site production server, all channel video signals are processed in real time;

[0015] Step S4, dynamic video encoding and compression,

[0016] After the video signal is processed in real time in step S3, it is encoded and compressed.

[0017] Introduce AI algorithm to analyze the complexity of each frame, dynamically adjust the compression rate, use lower compression rate for complex pictures and higher compression rate for simple pictures, and output the encoded video signal;

[0018] Step S5, on-site production,

[0019] Perform color adjustment, scene optimization and editing on the encoded video signal to output the final video signal.

[0020] Furthermore, in step S2, the global time synchronization protocol PTP1 588 is introduced.

[0021] Furthermore, in step S2, the multi-channel signal transmission method is:

[0022] Divide the video signal into multiple data packets, let D total Indicates the amount of raw video data collected from the camera, N frames Indicates the number of frames per second, B frame Indicates the amount of data per frame of video, S i represents the bandwidth of channel i,

[0023] The amount of single-frame video data is expressed as: Divide the video signal into N channels channels, and the bandwidth of each channel is expressed by the bandwidth scheduling formula:

[0024] Here, S i is the bandwidth of the ith channel, that is, the amount of data that each channel needs to carry during parallel transmission,

[0025] The dynamic bandwidth scheduling mechanism is introduced, and the bandwidth scheduling function is defined as:

[0026] Among them, S i (t) is the bandwidth of the ith channel at time t, S max is the maximum bandwidth limit, E current (t) is the current network load, E max is the maximum carrying capacity of the network, and α is the scheduling coefficient.

[0027] Furthermore, in step S2, the global time synchronization protocol PTP1 588 is introduced to ensure the time accuracy between different signal sources: Assume T master (t) represents the timestamp of the master clock at time t, T slave (t) represents the timestamp of the slave clock at time t, ΔT(t) is the time offset between the master and slave clocks at time t, and the time synchronization algorithm is used to ensure that ΔT(t) = T master (t)-T slave (t), where, ideally, ΔT(t)→0, that is, the time deviation between the master and slave clocks should be zero or close to zero.

[0028] The clock adjustment process introduced by the PTP1588 protocol is:

[0029] T slave (t+δt)=T slave (t)+ΔT(t), where δt represents the time update interval,

[0030] After channel slicing, each video signal is assigned to a different channel for transmission, and combined with the PTP1588 protocol, the time synchronization of all signal sources is performed. The transmission delay of each channel is expressed as:

[0031] Among them, L i is the transmission delay of channel i, which is determined by the channel bandwidth and time synchronization offset. △T(t) represents the time offset between the master and slave clocks.

[0032] Furthermore, in step S3, the real-time processing includes:

[0033] Build an AI-driven image analysis system. Use the AI-driven image analysis system to analyze overexposure, underexposure, or inaccurate focus problems that may occur in multiple images, and provide real-time prompts.

[0034] At the same time, the AI-driven image analysis system automatically adjusts picture parameters and dynamically allocates computing resources based on the complexity of the picture and the resource usage of each video stream.

[0035] Furthermore, in step S3, the real-time monitoring and real-time processing method is:

[0036] For exposure analysis, assume that each frame image is I(x, y), where x and y are the coordinates of the image pixel, the brightness value of the pixel is expressed as L(x, y), and the average brightness μ is defined as L :

[0037] Among them, N x and N y are the horizontal and vertical pixel numbers of the image, respectively, L(x, y) is the brightness value of the pixel point (x, y),

[0038] Define exposure bias E exp :

[0039] Among them, T L is the ideal exposure brightness value, ranging from 128 to 255, E exp Indicates the exposure deviation of the current frame image. If E exp If it is too large, it means that the image is overexposed or underexposed.

[0040] For focus analysis, assume that the gradient value G(x, y) of the image I(x, y) is:

[0041] in, and are the gradients of the image in the x and y directions, respectively.

[0042] Calculate the focus clarity F of the image based on the gradient value focus :

[0043] When F focus When the value is high, the image is well focused. focus If the value is low, it means that the image is out of focus or blurred. G(x, y) represents the gradient value at the image pixel (x, y), that is, the clarity of the point. focus Indicates the overall focus clarity of the image, used to determine whether there is any inaccurate focus.

[0044] Furthermore, in step S3, the real-time monitoring and real-time processing method further includes:

[0045] Based on the exposure deviation E exp and focus clarity F focus , the brightness and focus of the image are automatically adjusted, and the adjusted image brightness L′(x, y) is:

[0046] L′(x, y)=L(x, y)+k L ·(T L -μ L ), where k L is the brightness adjustment coefficient, L′(x, y) represents the adjusted pixel brightness, T F Indicates the threshold of focus sharpness,

[0047] If F focus Below the set threshold T F , the camera is notified to refocus, or the image is further post-processed and enhanced.

[0048] The image texture complexity is used to measure the complexity, and computing resources are dynamically allocated to convert the image texture complexity C complex Defined as:

[0049] Among them, C complex Represents the image texture complexity, which measures the complexity of image details. and are the second-order derivatives of the image in the x and y directions,

[0050] The resource allocation calculation function is expressed as:

[0051] R(t)=R base +k R ·C complex (t), where R(t) represents the computing resources allocated to the image processing at time t, and R base is the basic computing resource usage, k R is the resource adjustment factor.

[0052] Furthermore, step S4 adopts a distributed coding architecture to distribute the coding task to multiple parallel processing units.

[0053] Furthermore, in step S4, the dynamic video encoding and compression method is:

[0054] Define how to calculate the complexity C of each frame frame , and dynamically adjust the compression rate based on complexity. The complexity of each frame is measured by the gradient change and texture complexity of the image. A highly complex image contains more details and texture changes, so a lower compression rate is required to maintain video quality.

[0055] Assume that the frame image I(x, y) has a resolution of N x ×N y A two-dimensional matrix where x and y are the coordinates of the image pixels and the gradient change is expressed as:

[0056]

[0057] Calculate the average gradient change μ for each frame G :

[0058]

[0059] Based on this, the picture complexity C frame Defined as:

[0060] Among them, G(x, y) represents the gradient value at the coordinate (x, y), μ G Indicates the average gradient change of the image, that is, the degree of change of the overall picture, C frame Indicates the complexity of the picture, measuring the details and texture complexity of each frame, N x , N y Indicates the number of horizontal and vertical pixels of the image.

[0061] Based on the screen complexity C frame , define the compression ratio R comp Adjust the formula, complex pictures use lower compression rate, and simple pictures use higher compression rate, compression rate R comp The calculation formula is:

[0062] R comp (C frame )=R base -k comp ·(C frame -C threshold ), where R comp (C frame ) represents the compression rate for each frame of the image, which is dynamically adjusted with the complexity of the picture. base is the basic compression rate, used for simpler images, k comp is the compression ratio adjustment factor, which determines the sensitivity of the compression ratio to changes in complexity. threshold It is the threshold of picture complexity. Pictures exceeding this threshold use a lower compression rate.

[0063] Furthermore, in step S4, the dynamic video encoding and compression method further includes:

[0064] The distributed encoding architecture is used to distribute the encoding task to multiple parallel processing units. Assume that the total video data volume is D total , divide the data into Nnodes processing units, each unit i is responsible for processing the amount of data D i for:

[0065] Among them, D total Represents the total amount of video data, N nodes represents the number of parallel processing units, D i represents the amount of data that the i-th processing unit is responsible for processing,

[0066] The amount of data is allocated according to the computing power Pi of each processing unit. Assume that the total computing power of the processing unit is P total , then the data allocation of the i-th processing unit is:

[0067] Among them, P i represents the computing power of the i-th processing unit, P total represents the total computing power of all processing units, D i represents the amount of data allocated to the i-th processing unit,

[0068] Assume that the time for each processing unit to process data is T i , then the total time for parallel processing is T total Limited by the slowest node:

[0069] Among them, T i represents the time required for the i-th processing unit to process the task, D i represents the amount of data allocated to the i-th processing unit, P i represents the computing power of the i-th processing unit, T total represents the total processing time of the entire distributed system,

[0070] By analyzing the complexity of each frame of the image through AI algorithms and dynamically adjusting the compression rate, combined with parallel task allocation and load balancing in the distributed coding architecture, the coding efficiency is greatly improved and the balance between the quality of the output video and bandwidth occupancy is ensured.

[0071] The beneficial effects of the present invention are:

[0072] The present invention adopts channel slicing technology to divide the video signal into multiple data packets, and adopts multiple transmission channels for parallel transmission, so as to make full use of the bandwidth and avoid the bottleneck of single channel. The bandwidth dynamic scheduling mechanism dynamically adjusts the channel bandwidth according to the real-time network conditions, improves the utilization rate of network resources, and greatly reduces the bandwidth pressure when transmitting large amounts of 4K or 8K data, improves the transmission efficiency and avoids signal congestion.

[0073] The present invention introduces the PTP1588 global time synchronization protocol, so that all cameras and signal sources maintain microsecond synchronization, avoiding time offset between multi-camera video sources. Finally, various angles and different signal streams of the video output can maintain timing consistency, ensuring the consistency and coherence of the visual effects, and greatly improving the accuracy of signal synchronization.

[0074] In the present invention, the AI-driven image analysis system can automatically identify overexposure, underexposure and inaccurate focus problems, and provide adjustment suggestions in real time, reducing reliance on manual adjustment, improving the efficiency and quality of on-site production, and solving the scheduling bottleneck problem of traditional systems in multi-tasking processing by automatically analyzing the complexity of the picture and dynamically allocating computing resources.

[0075] In the present invention, the AI ​​algorithm dynamically adjusts the compression rate and can flexibly adjust the compression rate according to the complexity of each frame of the image, effectively reducing the data load while maintaining high-quality video output, reducing dependence on interfaces and transmission protocols, and distributing the encoding tasks to multiple parallel processing units through a distributed encoding architecture, greatly improving the encoding speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0077] Figure 1 This is a flow chart of the 4K and 8K ultra-high-definition live production method based on embedded technology of the present invention. DETAILED DESCRIPTION

[0078] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0079] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0080] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0081] Example 1, reference Figure 1 , this embodiment provides a 4K and 8K ultra-high-definition live production method based on embedded technology, including the following steps:

[0082] Step S1, video data acquisition,

[0083] Use 4K or 8K cameras that support high dynamic range (HDR) and wide color gamut (WCG) to collect video data. The cameras used have intelligent frame rate adjustment functions, which automatically adjust the frame rate according to the dynamics of the scene, and reduce the frame rate in static scenes. The video data is initially compressed and converted into a format.

[0084] Specifically, intelligent frame rate adjustment and high dynamic range (HDR) and wide color gamut (WCG) camera technologies are used to adapt the frame rate and reduce redundant data.

[0085] Step S2, multi-channel signal transmission,

[0086] Based on the video data collected and preprocessed in step S1, the channel slicing technology is used to divide the video signal into multiple data packets, and the data packets are transmitted to the on-site production server in parallel through different transmission channels. The bandwidth dynamic scheduling mechanism is introduced here to automatically adjust the bandwidth resources according to real-time needs.

[0087] In step S2, the multi-channel signal transmission method is:

[0088] Divide the video signal into multiple data packets, let D total Indicates the amount of raw video data collected from the camera, N frames Indicates the number of frames per second, B frame Indicates the amount of data per frame of video, S i represents the bandwidth of channel i,

[0089] The amount of single-frame video data is expressed as: Divide the video signal into N channels channels, and the bandwidth of each channel is expressed by the bandwidth scheduling formula:

[0090] Here, S i is the bandwidth of the ith channel, that is, the amount of data that each channel needs to carry during parallel transmission,

[0091] The dynamic bandwidth scheduling mechanism is introduced, and the bandwidth scheduling function is defined as:

[0092] Among them, S i (t) is the bandwidth of the ith channel at time t, S max is the maximum bandwidth limit, E current(t) is the current network load, E max is the maximum carrying capacity of the network, α is the scheduling coefficient,

[0093] In step S2, the global time synchronization protocol PTP1 588 is introduced to ensure the time accuracy between different signal sources: Assume T master (t) represents the timestamp of the master clock at time t, T slave (t) represents the timestamp of the slave clock at time t, ΔT(t) is the time offset between the master and slave clocks at time t, and the time synchronization algorithm is used to ensure that ΔT(t) = T master (t)-T slave (t), where, ideally, ΔT(t)→0, that is, the time deviation between the master and slave clocks should be zero or close to zero.

[0094] The clock adjustment process introduced by the PTP1588 protocol is:

[0095] T slave (t+δt)=T slave (t)+ΔT(t), where δt represents the time update interval,

[0096] After channel slicing, each video signal is assigned to a different channel for transmission, and combined with the PTP1588 protocol, the time synchronization of all signal sources is performed. The transmission delay of each channel is expressed as:

[0097] Among them, L i is the transmission delay of channel i, which is determined by the channel bandwidth and time synchronization offset. ΔT(t) represents the time offset between the master and slave clocks;

[0098] Specifically, in the process of multi-channel signal transmission, channel slicing technology is combined with the bandwidth dynamic scheduling mechanism to divide the video signal into multiple small data packets for parallel transmission on different channels. The bandwidth resources of each channel are adjusted according to the real-time network conditions to avoid bandwidth bottlenecks, thereby improving the flexibility and efficiency of transmission. The introduction of the global time synchronization protocol PTP1 588 ensures the time synchronization between multiple camera signal sources and avoids picture misalignment and delay.

[0099] Embedded control systems are used in video production to coordinate the work of multiple cameras, switchers, encoders and other equipment. Especially in 4K / 8K video production, a large number of concurrent tasks must be efficiently processed, such as signal synchronization, video stream switching and real-time monitoring. However, traditional embedded systems have limited processing capabilities. Faced with the multi-task processing requirements of ultra-high-definition video, task scheduling delays or processing bottlenecks often occur, resulting in untimely system response. This problem is more obvious in multi-camera switching or synchronous operations. Traditional solutions are difficult to achieve microsecond-level synchronization, resulting in slight time differences between different video sources in high-frame rate video production, affecting the overall effect.

[0100] Traditional embedded control solutions usually manage tasks through time slice rotation or fixed priority scheduling, such as switching camera signals, adjusting video encoders, or controlling transmission equipment. In addition, video synchronization usually relies on global clocks or signal synchronization lines. When these methods are used in 4K / 8K video production, they are not real-time enough and are prone to task blocking, affecting system stability and performance. Therefore, the following solutions are introduced to optimize them.

[0101] Step S3, real-time monitoring and real-time processing,

[0102] After the video signal reaches the on-site production server, all channel video signals are processed in real time;

[0103] In step S3, the real-time processing includes:

[0104] Build an AI-driven image analysis system. Use the AI-driven image analysis system to analyze overexposure, underexposure, or inaccurate focus problems that may occur in multiple images, and provide real-time prompts.

[0105] At the same time, the AI-driven image analysis system automatically adjusts the picture parameters and dynamically allocates computing resources according to the complexity of the picture and the resource usage of each video stream.

[0106] In step S3, the real-time monitoring and real-time processing method is:

[0107] For exposure analysis, assume that each frame image is I(x, y), where x and y are the coordinates of the image pixel, the brightness value of the pixel is expressed as L(x, y), and the average brightness μ is defined as L :

[0108] Among them, N x and N y are the horizontal and vertical pixel numbers of the image, respectively, L(x, y) is the brightness value of the pixel point (x, y),

[0109] Define exposure bias E exp :

[0110] Among them, T L is the ideal exposure brightness value, ranging from 128 to 255, E exp Indicates the exposure deviation of the current frame image. If E exp If it is too large, it means that the image is overexposed or underexposed.

[0111] For focus analysis, assume that the gradient value G(x, y) of the image I(x, y) is:

[0112] in, and are the gradients of the image in the x and y directions, respectively.

[0113] Calculate the focus clarity F of the image based on the gradient value focus :

[0114] When F focus When the value is high, the image is well focused. focus If the value is low, it means that the image is out of focus or blurred. G(x, y) represents the gradient value at the image pixel (x, y), that is, the clarity of the point. focus Indicates the overall focus clarity of the image, used to determine whether there is inaccurate focus.

[0115] Based on the exposure deviation E exp and focus clarity F focus , the brightness and focus of the image are automatically adjusted, and the adjusted image brightness L′(x, y) is:

[0116] L′(x, y)=L(x, y)+k L ·(T L -μ L ), where k L is the brightness adjustment coefficient, L′(x, y) represents the adjusted pixel brightness, T F Indicates the threshold of focus sharpness,

[0117] If F focus Below the set threshold T F , the camera is notified to refocus, or the image is further post-processed and enhanced.

[0118] The image texture complexity is used to measure the complexity, and computing resources are dynamically allocated to convert the image texture complexity C complex Defined as:

[0119] Among them, C complex Represents the image texture complexity, which measures the complexity of image details. and are the second-order derivatives of the image in the x and y directions,

[0120] The resource allocation calculation function is expressed as:

[0121] R(t)=R base +k R ·C complex (t), where R(t) represents the computing resources allocated to the image processing at time t, and R base is the basic computing resource usage, k R is the resource adjustment factor,

[0122] Specifically, in the real-time monitoring and processing stages, AI-driven image analysis has greatly improved the intelligence level of on-site production. It can analyze the exposure and focus issues of the picture in real time, automatically adjust the picture parameters, dynamically allocate computing resources, and reduce processing delays.

[0123] Step S4, dynamic video encoding and compression,

[0124] After the video signal is processed in real time in step S3, it is encoded and compressed.

[0125] Introduce AI algorithm to analyze the complexity of each frame, dynamically adjust the compression rate, use lower compression rate for complex pictures and higher compression rate for simple pictures, and output the encoded video signal;

[0126] Step S4 adopts a distributed coding architecture to distribute the coding task to multiple parallel processing units.

[0127] In step S4, the dynamic video encoding and compression method is:

[0128] Define how to calculate the complexity C of each frame frame , and dynamically adjust the compression rate based on complexity. The complexity of each frame is measured by the gradient change and texture complexity of the image. A highly complex image contains more details and texture changes, so a lower compression rate is required to maintain video quality.

[0129] Assume that the frame image I(x, y) has a resolution of N x ×N y A two-dimensional matrix where x and y are the coordinates of the image pixels and the gradient change is expressed as:

[0130]

[0131] Calculate the average gradient change μ for each frame G :

[0132]

[0133] Based on this, the picture complexity C frame Defined as:

[0134] Among them, G(x, y) represents the gradient value at the coordinate (x, y), μ G Indicates the average gradient change of the image, that is, the degree of change of the overall picture, C frame Indicates the complexity of the picture, measuring the details and texture complexity of each frame, N x , N y Indicates the number of horizontal and vertical pixels of the image.

[0135] Based on the screen complexity C frame , define the compression ratio R comp Adjust the formula, complex pictures use lower compression rate, and simple pictures use higher compression rate, compression rate R comp The calculation formula is:

[0136] R comp (c frame )=R base -k comp ·(C frame -c threshold ), where R comp (C frame ) represents the compression rate for each frame of the image, which is dynamically adjusted with the complexity of the picture. base is the basic compression rate, used for simpler images, k comp is the compression ratio adjustment factor, which determines the sensitivity of the compression ratio to changes in complexity. threshold It is the threshold of picture complexity. Pictures exceeding this threshold use a lower compression rate.

[0137] The distributed encoding architecture is used to distribute the encoding task to multiple parallel processing units. Assume that the total video data volume is D total , divide the data into N nodes processing units, each unit i is responsible for processing the amount of data D i for:

[0138] Among them, D total Represents the total amount of video data, N nodes represents the number of parallel processing units, D i represents the amount of data that the i-th processing unit is responsible for processing,

[0139] According to the computing power P of each processing unit i Assume that the total computing power of the processing unit is P total , then the data allocation of the i-th processing unit is:

[0140] Among them, P i represents the computing power of the i-th processing unit, P total represents the total computing power of all processing units, D i represents the amount of data allocated to the i-th processing unit,

[0141] Assume that the time for each processing unit to process data is T i , then the total time for parallel processing is T total Limited by the slowest node:

[0142] Among them, T i represents the time required for the i-th processing unit to process the task, D i represents the amount of data allocated to the i-th processing unit, P i represents the computing power of the i-th processing unit, T total represents the total processing time of the entire distributed system,

[0143] The AI ​​algorithm analyzes the complexity of each frame and dynamically adjusts the compression rate. Combined with the parallel task allocation and load balancing in the distributed coding architecture, the coding efficiency is greatly improved and the balance between the quality of the output video and the bandwidth usage is guaranteed.

[0144] Specifically, the complexity of each frame is analyzed through AI algorithm, and the compression rate is dynamically adjusted according to the complexity. Combined with the distributed coding architecture, high-speed processing of video encoding can be achieved.

[0145] Step S5, on-site production,

[0146] Perform color adjustment, scene optimization and editing on the encoded video signal to output the final video signal.

[0147] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A 4K and 8K ultra-high definition live production method based on embedded technology, characterized in that: include, Step S1, video data acquisition, using a 4K or 8K camera that supports high dynamic range HDR and wide color gamut WCG to acquire video data, and preliminarily compress and convert the video data into a format. Step S2, multi-channel signal transmission, based on the video data collected and pre-processed in step S1, the video signal is divided into multiple data packets using channel slicing technology, and the data packets are transmitted in parallel to the on-site production server via different transmission channels. A bandwidth dynamic scheduling mechanism is introduced here to automatically adjust bandwidth resources according to real-time needs. Step S3, real-time monitoring and real-time processing, after the video signal arrives at the on-site production server, all channel video signals are processed in real time; Step S4, dynamic video encoding and compression, after the video signal is processed in real time in step S3, it is encoded and compressed, an AI algorithm is introduced to analyze the complexity of each frame, and the compression rate is dynamically adjusted. A lower compression rate is used for a complex picture and a higher compression rate is used for a simple picture, and the encoded video signal is output; Step S5, on-site production, color adjustment, scene optimization and editing are performed on the encoded video signal to output the final video signal.

2. According to the method of claim 1, the method is characterized in that: In step S2, the global time synchronization protocol PTP1588 is introduced.

3. According to claim 2, a 4K and 8K ultra-high-definition live production method based on embedded technology is characterized in that: In step S2, the multi-channel signal transmission method is: Divide the video signal into multiple data packets, let D total Indicates the amount of raw video data collected from the camera, N frames Indicates the number of frames per second, B frame Indicates the amount of data per frame of video, S i represents the bandwidth of channel i, The amount of single-frame video data is expressed as: Divide the video signal into N channels channels, and the bandwidth of each channel is expressed by the bandwidth scheduling formula: Here, S i is the bandwidth of the ith channel, that is, the amount of data that each channel needs to carry during parallel transmission, The dynamic bandwidth scheduling mechanism is introduced, and the bandwidth scheduling function is defined as: Among them, S i (t) is the bandwidth of the ith channel at time t, S max is the maximum bandwidth limit, E current (t) is the current network load, E max is the maximum carrying capacity of the network, and α is the scheduling coefficient.

4. According to claim 3, a 4K and 8K ultra-high definition live production method based on embedded technology is characterized in that: In step S2, the global time synchronization protocol PTP1588 is introduced: Assume T master (t) represents the timestamp of the master clock at time t, T slave (t) represents the timestamp of the slave clock at time t, ΔT(t) is the time offset between the master and slave clocks at time t, and the time synchronization algorithm is used to ensure that ΔT(t) = T master (t)-T slave (t), where, ideally, ΔT(t)→0, that is, the time deviation between the master and slave clocks should be zero or close to zero. The clock adjustment process introduced by the PTP1588 protocol is: T slave (t+δt)=T slave (t)+ΔT(t), where δt represents the time update interval, After channel slicing, each video signal is assigned to a different channel for transmission, and combined with the PTP1588 protocol, the time synchronization of all signal sources is performed. The transmission delay of each channel is expressed as: Among them, L i is the transmission delay of channel i, which is determined by the channel bandwidth and time synchronization offset. ΔT(t) represents the time offset between the master and slave clocks.

5. The method for 4K and 8K ultra-high definition live production based on embedded technology according to claim 4, characterized in that: In step S3, the real-time processing includes: Build an AI-driven image analysis system. Use the AI-driven image analysis system to analyze overexposure, underexposure, or inaccurate focus problems that may occur in multiple images, and provide real-time prompts. At the same time, the AI-driven image analysis system automatically adjusts picture parameters and dynamically allocates computing resources based on the complexity of the picture and the resource usage of each video stream.

6. The method for 4K and 8K ultra-high definition live production based on embedded technology according to claim 5, characterized in that: In step S3, the real-time monitoring and real-time processing method is: For exposure analysis, assume that each frame image is I(x,y), where x and y are the coordinates of the image pixel, the brightness value of the pixel is represented by L(x,y), and the average brightness μ is defined as L : Among them, N x and N y are the horizontal and vertical pixel numbers of the image respectively, L(x,y) is the brightness value of the pixel point (x,y), Define exposure bias E exp : Among them, T L is the ideal exposure brightness value, ranging from 128 to 255, E exp Indicates the exposure deviation of the current frame image. If E exp If it is too large, it means that the image is overexposed or underexposed. For focus analysis, assume that the gradient value G(x,y) of the image I(x,y) is: in, and are the gradients of the image in the x and y directions, respectively. Calculate the focus clarity F of the image based on the gradient value focus : When F focus When the value is high, the image is well focused. focus A lower value indicates that the image is out of focus or blurred. G(x,y) represents the gradient value at the image pixel (x,y), that is, the clarity of the point. focus Indicates the overall focus clarity of the image, used to determine whether there is any inaccurate focus.

7. The method for 4K and 8K ultra-high definition live production based on embedded technology according to claim 6, characterized in that: In step S3, the real-time monitoring and real-time processing method also includes: Based on the exposure deviation E exp and focus clarity F focus , the brightness and focus of the image are automatically adjusted, and the adjusted image brightness L′(x,y) is: L′(x,y)=L(x,y)+k L ·(T L -μ L ), where k L is the brightness adjustment coefficient, L′(x,y) represents the adjusted pixel brightness, T F Indicates the threshold of focus sharpness, If F focus Below the set threshold T F , the camera is notified to refocus, or the image is further post-processed and enhanced. The image texture complexity is used to measure the complexity, and computing resources are dynamically allocated to convert the image texture complexity C complex Defined as: Among them, C complex Represents the image texture complexity, which measures the complexity of image details. and are the second-order derivatives of the image in the x and y directions, The resource allocation calculation function is expressed as: R(t)=R base +k R ·C complex (t), where R(t) represents the computing resources allocated to the image processing at time t, and R base is the basic computing resource usage, k R is the resource adjustment factor.

8. The method for 4K and 8K ultra-high definition live production based on embedded technology according to claim 7, characterized in that: Step S4 adopts a distributed encoding architecture to distribute the encoding task to multiple parallel processing units.

9. The method for 4K and 8K ultra-high definition live production based on embedded technology according to claim 8, characterized in that: In step S4, the dynamic video encoding and compression method is: Assume the frame image I(x,y) has a resolution of N x ×N y A two-dimensional matrix where x and y are the coordinates of the image pixels and the gradient change is expressed as: Calculate the average gradient change μ for each frame G : Based on this, the picture complexity C frame Defined as: Among them, G(x,y) represents the gradient value at the coordinate (x,y), μ G Indicates the average gradient change of the image, that is, the degree of change of the overall picture, C frame Indicates the complexity of the picture, measuring the details and texture complexity of each frame, N x ,N y Indicates the number of horizontal and vertical pixels of the image. Based on the screen complexity C frame , define the compression ratio R comp Adjustment formula, compression ratio R comp The calculation formula is: R comp (C frame )=R base -k comp ·(C frame -C threshold ), where R comp (C frame ) represents the compression rate of each frame of the image, which is dynamically adjusted with the complexity of the picture. base is the basic compression rate, used for simpler images, k comp is the compression ratio adjustment factor, which determines the sensitivity of the compression ratio to changes in complexity. threshold It is the threshold of picture complexity. Pictures exceeding this threshold use a lower compression rate.

10. The method for 4K and 8K ultra-high definition live production based on embedded technology according to claim 9, characterized in that: In step S4, the dynamic video encoding and compression method further includes: The distributed coding architecture is used to distribute the coding tasks to multiple parallel processing units. Assume that the total amount of video data is D total , divide the data into N nodes processing units, each unit i is responsible for processing the amount of data D i for: Among them, D total Represents the total amount of video data, N nodes represents the number of parallel processing units, D i represents the amount of data that the i-th processing unit is responsible for processing, According to the computing power P of each processing unit i Assume that the total computing power of the processing unit is P total , then the data allocation of the i-th processing unit is: Among them, P i represents the computing power of the i-th processing unit, P total represents the total computing power of all processing units, D i represents the amount of data allocated to the i-th processing unit, Assume that the time for each processing unit to process data is T i , then the total time for parallel processing is T total Limited by the slowest node: Among them, T i represents the time required for the i-th processing unit to process the task, D i represents the amount of data allocated to the i-th processing unit, P i represents the computing power of the i-th processing unit, T total Represents the total processing time of the entire distributed system.

Images