Intelligent encoding and transmission method and transmission system for ultra-high-definition 8k video

Abstract

The present invention provides an intelligent encoding and transmission method and a transmission system for ultra-high-definition 8k video, which relates to the field of video encoding technology. The method comprises the following steps: before encoding, initializing a buffer for a video stream and setting a target bit rate; obtaining a SATD value and a frame type of a current frame in the video stream, and inputting the saturation, SATD value, frame type and set target bit rate of a target buffer corresponding to the current frame into a pre-trained reinforcement learning model, and outputting a first quantization step size of the current frame; during encoding, obtaining a second quantization step size of each coding unit in the current frame, and calculating a final quantization step size of each coding unit in combination with a first quantization step size of the current frame; quantizing the corresponding coding unit in the current frame using the final quantization step size, and then performing entropy encoding to generate a bit stream, which is stored in the target buffer; and transmitting the stored bit stream data to a receiving end. The present invention introduces an intelligent learning mechanism and comprehensive consideration of multiple factors to make the encoding and transmission process more efficient and flexible.

Description

Technical Field

[0001] The present invention relates to the field of video coding technology, and in particular to an intelligent coding and transmission method and a transmission system for ultra-high-definition 8K video. Background Art

[0002] With the rapid development of video technology, ultra-high definition (UHD) video, especially 8K video, has become the mainstream direction of future video transmission. With its extremely high resolution (7680x4320 pixels) and delicate picture quality, 8K video brings unprecedented visual experience to viewers. However, this high-quality visual enjoyment is also accompanied by huge data volume challenges, which puts higher requirements on video encoding, transmission and storage technology.

[0003] Existing technology is insufficient

[0004] In traditional video coding and transmission methods, fixed quantization step sizes or dynamic quantization step size adjustment strategies based on simple rules are usually adopted. These methods have obvious shortcomings when processing ultra-high-definition 8K videos:

[0005] Fixed quantization step size: The fixed quantization step size cannot be flexibly adjusted according to the complexity of the video content and changes in network transmission conditions, which can easily lead to large fluctuations in video quality, especially when the network bandwidth is limited or the video content is complex and changeable.

[0006] Simple rules: Quantization step adjustment strategies based on simple rules often fail to fully consider multiple factors such as video content characteristics, buffer status, and network conditions, making it difficult to achieve an optimal balance between video quality and transmission efficiency.

[0007] Lack of intelligence: Traditional methods lack intelligent learning mechanisms and are unable to learn and optimize the selection strategy of quantization steps from large amounts of data, making it difficult to adapt to complex and changing practical application scenarios.

[0008] Therefore, it is necessary to provide an intelligent encoding and transmission method and transmission system for ultra-high-definition 8K video to solve the above technical problems. Summary of the invention

[0009] In order to solve the above technical problems, the present invention provides an intelligent coding and transmission method and transmission system for ultra-high-definition 8K video. By introducing an intelligent learning mechanism and comprehensive consideration of multiple factors, the coding and transmission process is made more efficient and flexible.

[0010] The present invention provides an intelligent coding and transmission method for ultra-high-definition 8K video, and the coding and transmission method comprises the following steps:

[0011] Before encoding, initialize the buffer for the video stream and set the target bitrate;

[0012] Obtaining the SATD value and frame type of the current frame in the video stream, and taking the saturation, SATD value, frame type and set target bit rate of the target buffer corresponding to the current frame as input, outputting the first quantization step of the current frame through the pre-trained reinforcement learning model, wherein the current frame is any frame in the video stream;

[0013] During encoding, the second quantization step size of each coding unit in the current frame is obtained, and combined with the first quantization step size of the current frame, the final quantization step size of each coding unit is calculated;

[0014] quantizing the corresponding coding unit in the current frame using the final quantization step size, performing entropy coding on the quantized result to generate a bit stream, and storing the generated bit stream in a target buffer;

[0015] The stored bit stream data is packaged into data packets and sent to the receiving end to achieve encoded transmission of the video stream.

[0016] Preferably, obtaining the second quantization step size of each coding unit in the current frame includes:

[0017] A multi-scale feature map of the current frame is obtained, and a second quantization step size of each coding unit in the current frame is predicted based on the multi-scale feature map of the current frame.

[0018] Preferably, obtaining the second quantization step size of each coding unit in the current frame includes:

[0019] The second quantization step size of each coding unit in the current frame is calculated according to the saturation of the target buffer and the coding quantization step size of the coding unit at the corresponding position in the previous frame.

[0020] Preferably, the calculation formula of the final quantization step size is:

[0021]

[0022] in, Indicates Frame The final quantization step size of the coding unit, Indicates Frame The second quantization step size of the coding unit, Indicates The first quantization step size of the frame, and are the weights of the second quantization step size and the first quantization step size, respectively, and .

[0023] Preferably, the training process of the reinforcement learning model is:

[0024] Collect ultra-high-definition 8k video samples as a training data set, and pre-process each video sample, wherein the pre-processing includes frame extraction and feature extraction;

[0025] Defining a state space for reinforcement learning, wherein the state space includes a SATD value of a current frame, a frame type, a saturation of a target buffer, and a set target bit rate;

[0026] Defining an action space for reinforcement learning, wherein the action space is a range of values ​​for a quantized step size;

[0027] Design a reward function to comprehensively evaluate the encoded video quality SSIM indicator, buffer status, and bitrate usage;

[0028] Initialize the parameters of the reinforcement learning model, and use the Q-learning learning algorithm to train the model. During the training process, continuously iterate and update the parameters of the reinforcement learning model to find the optimal quantization step strategy through trial and error learning, so that the reinforcement learning model can dynamically adjust the quantization step according to different inputs.

[0029] Preferably, acquiring a multi-scale feature map of the current frame, and predicting a second quantization step size of each coding unit in the current frame based on the multi-scale feature map of the current frame, comprises:

[0030] Extracting a multi-scale feature map of the current frame, wherein the multi-scale feature map includes brightness, color distribution, texture complexity, and motion vector of the current frame;

[0031] Based on the extracted multi-scale feature map, a pre-trained deep learning model is used to predict the second quantization step size of each coding unit in the current frame, wherein the deep learning model is trained through video data and learns to allocate quantization steps according to different attributes of the feature map.

[0032] Preferably, calculating the second quantization step size of each coding unit in the current frame according to the saturation of the target buffer and the coding quantization step size of the coding unit at the corresponding position in the previous frame includes:

[0033] Monitor and obtain the saturation of the current target buffer, wherein the saturation reflects the ratio of the remaining space in the buffer to the total capacity;

[0034] For each coding unit in the current frame, find and obtain the coding quantization step size of the coding unit at the corresponding position in the previous frame;

[0035] According to the saturation adjustment strategy of the current target buffer, combined with the obtained encoding quantization step and saturation adjustment strategy of the previous frame, the second quantization step of each coding unit in the current frame is calculated, wherein the saturation adjustment strategy is:

[0036] If the buffer saturation is high, the second quantization step size tends to be reduced to reduce the amount of data. Otherwise, the second quantization step size is appropriately increased to balance the coding efficiency and video quality.

[0037] Preferably, the calculation formula of the second quantization step size is:

[0038]

[0039] in, represents the second quantization step size, Indicates the encoding quantization step size of the corresponding position encoding unit in the previous frame, represents the adjustment factor, Indicates the saturation of the current target buffer. Indicates the threshold value.

[0040] Preferably, the target bit rate is also dynamically adjusted, specifically:

[0041] During the encoding process, the saturation of the target buffer and the bandwidth fluctuation of the network transmission are monitored in real time;

[0042] Dynamically adjust the preset target bit rate according to the saturation of the target buffer and the bandwidth fluctuation value to balance the smoothness of video transmission and video quality;

[0043] Among them, when the target buffer saturation is lower than the preset saturation threshold and the network bandwidth is higher than the preset bandwidth threshold, the target bit rate is increased; conversely, when the target buffer saturation is not lower than the preset saturation threshold or the network bandwidth is not higher than the preset bandwidth threshold, the target bit rate is reduced.

[0044] The present invention provides an intelligent coding transmission system for ultra-high-definition 8K video, which is used to perform an intelligent coding transmission method for ultra-high-definition 8K video. The coding transmission system includes:

[0045] An initialization module is used to initialize the buffer and set the target bit rate for the video stream before encoding;

[0046] a first quantization step acquisition module, used to acquire the SATD value and frame type of the current frame in the video stream, and take the saturation, SATD value, frame type and set target bit rate of the target buffer corresponding to the current frame as input, and output the first quantization step of the current frame through a pre-trained reinforcement learning model, wherein the current frame is any frame in the video stream;

[0047] The final quantization step length calculation module is used to obtain the second quantization step length of each coding unit in the current frame during encoding, and calculate the final quantization step length of each coding unit in combination with the first quantization step length of the current frame;

[0048] A video encoding module, configured to quantize the corresponding coding unit in the current frame using the final quantization step size, perform entropy encoding on the quantized result to generate a bit stream, and store the generated bit stream in a target buffer;

[0049] The video transmission module is used to package the stored bit stream data into data packets and send them to the receiving end to achieve encoded transmission of the video stream.

[0050] Compared with the related art, the intelligent encoding and transmission method and transmission system of ultra-high-definition 8k video provided by the present invention have the following beneficial effects:

[0051] The present invention introduces a pre-trained reinforcement learning model, and dynamically outputs an optimal quantization step size according to the SATD value of a video frame, the frame type, the saturation of a target buffer, and a set target bit rate.

[0052] The present invention uses a multi-scale feature map to predict the second quantization step size of each coding unit in the current frame, and calculates the final quantization step size in combination with the first quantization step size, thereby improving the accuracy and flexibility of the quantization step size.

[0053] The present invention monitors the saturation of the target buffer and the bandwidth fluctuation value of the network transmission in real time, dynamically adjusts the target bit rate, and ensures the fluency and quality of video transmission.

[0054] The present invention trains the model through the Q-learning reinforcement learning algorithm, so that it can dynamically adjust the quantization step strategy according to different inputs to achieve the optimal balance between video quality and transmission efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 A flowchart of an intelligent encoding and transmission method for ultra-high-definition 8k video provided by the present invention;

[0056] Figure 2 This is a module structure diagram of an ultra-high-definition 8K video intelligent encoding and transmission system provided by the present invention. DETAILED DESCRIPTION

[0057] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only the parts related to the present invention, rather than all structures, are shown in the accompanying drawings. In addition, the embodiments of the present invention and the features in the embodiments may be combined with each other without conflict.

[0058] It should also be noted that, for ease of description, only the parts related to the present invention, but not all of the contents, are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe the operations (or steps) as sequential processes, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but it can also have additional steps not included in the accompanying drawings. The process can correspond to methods, functions, procedures, subroutines, subprograms, etc.

[0059] Embodiment 1

[0060] The present invention provides an intelligent encoding and transmission method for ultra-high-definition 8k video, referring to Figure 1 As shown, the encoding transmission method includes the following steps:

[0061] S1: Before encoding, initialize the buffer for the video stream and set the target bitrate.

[0062] In the process of video encoding and transmission, the initialization stage is crucial. This stage involves the initial configuration of the encoding process, including initializing the buffer and setting the target bit rate. For ultra-high-definition 8K video, due to its high resolution and large amount of data, reasonable initialization settings before encoding can significantly affect the quality and efficiency of subsequent encoding. The initialization of the buffer determines the temporary storage location of the encoded video stream data, and the setting of the target bit rate directly affects the degree of video compression and the quality of the final video.

[0063] Specifically, before encoding, a buffer needs to be initialized to store the encoded video data. This buffer acts like a temporary data warehouse, temporarily storing the encoded video frame data during the encoding process until the data is packaged and transmitted to the receiver over the network. When initializing the buffer, its size needs to be determined, which depends on the expected video length, bit rate, and possible network conditions. If the buffer is too large, it may waste resources, and if it is too small, it may cause data overflow.

[0064] The target bitrate refers to the average bitrate planned to be used when encoding a video. A higher bitrate means better video quality, but it also produces a larger file size, which is not suitable for bandwidth-constrained network environments. When setting the target bitrate, you need to consider the video content type (such as animation, movies, live broadcasts, etc.), the expected viewing device (such as TV, computer, mobile device, etc.), and the current network environment.

[0065] In the present invention, the setting of the target bit rate is a dynamic process, which can be adjusted according to the buffer saturation and network bandwidth fluctuations monitored during the encoding process. This dynamic adjustment mechanism helps to maintain the smoothness of video transmission while ensuring that the quality of the video is not affected as much as possible.

[0066] Specifically, during the encoding process, it is necessary to monitor the saturation of the target buffer and the bandwidth fluctuation value of the network transmission in real time. According to the saturation of the target buffer and the bandwidth fluctuation value, the preset target bit rate is dynamically adjusted to balance the smoothness of video transmission and video quality. The specific adjustment rules are as follows:

[0067] When the target buffer saturation is below the preset saturation threshold and the network bandwidth is above the preset bandwidth threshold, the target bit rate is increased. This can improve video quality when network conditions are good.

[0068] On the contrary, when the target buffer saturation is not lower than the preset saturation threshold, or the network bandwidth is not higher than the preset bandwidth threshold, the target bit rate is reduced. This can reduce the amount of data when the network condition is poor or the buffer is close to full, avoiding buffer overflow or network congestion.

[0069] S2: Obtain the SATD value and frame type of the current frame in the video stream, and take the saturation, SATD value, frame type and set target bit rate of the target buffer corresponding to the current frame as input, and output the first quantization step of the current frame through the pre-trained reinforcement learning model, where the current frame is any frame in the video stream.

[0070] During the video encoding process, the choice of quantization step size directly affects the compression effect and final quality of the video. For ultra-high-definition 8K video, due to its high resolution, a reasonable choice of quantization step size is particularly important for maintaining video quality while reducing the amount of data. Traditional encoding methods usually use a fixed quantization step size or a simple adaptive quantization step size, which may not be able to fully cope with the complexity and variability of video content. Therefore, the present invention proposes a method for dynamically adjusting the quantization step size through a reinforcement learning model to adapt to the specific characteristics of each frame in the video stream, thereby achieving the purpose of optimizing video quality and coding efficiency.

[0071] In step S2, the SATD (Sum of Absolute TransformedDifferences) value and frame type of the current frame in the video stream are obtained, and the saturation, SATD value, frame type and set target bit rate of the target buffer corresponding to the current frame are taken as input, and the first quantization step size of the current frame is output through the pre-trained reinforcement learning model.

[0072] The specific implementation steps are as follows:

[0073] Extract the SATD value and frame type (such as I frame, P frame, B frame) of the current frame from the video stream. The SATD value is an indicator that measures the amount of change within a frame and can be used to evaluate the complexity of the current frame.

[0074] Monitor the saturation of the target buffer in real time to understand the available space in the current buffer.

[0075] The SATD value of the current frame, the frame type, the saturation of the target buffer, and the preset target bit rate are sent as input to the pre-trained reinforcement learning model.

[0076] The reinforcement learning model outputs the first quantization step of the current frame based on the input information. This quantization step reflects a preliminary estimate of the overall encoding quality of the current frame.

[0077] Among them, the training process of the reinforcement learning model is:

[0078] Ultra-high-definition 8k video samples are collected as training data sets, and each video sample is preprocessed, where the preprocessing includes frame extraction and feature extraction.

[0079] In this embodiment, in order to train the reinforcement learning model, a large number of ultra-high-definition 8K video samples need to be collected as a training data set. These video samples should cover a variety of scenes and content to ensure that the model can be generalized to a variety of video types. For example, videos can be obtained from different sources, including but not limited to movies, TV series, natural scenery, sports events, etc., to cover a wide range of visual features and dynamic ranges.

[0080] The preprocessing steps include frame extraction and feature extraction:

[0081] Frame extraction: Extract multiple frames from each video sample, especially key frames (such as I frames) and other types of frames (such as P frames, B frames). These frames will be used for subsequent feature extraction and model training.

[0082] Feature extraction: Useful features are extracted from the extracted frames, such as SATD value (Sum of Absolute Transformed Differences), frame type, brightness, color distribution, texture complexity, motion vector, etc. These features will be used as input to the reinforcement learning model.

[0083] Define the state space of reinforcement learning, where the state space includes the SATD value of the current frame, the frame type, the saturation of the target buffer, and the set target bit rate.

[0084] The state space defines the state of the environment in which the reinforcement learning model is located at each decision point. In this embodiment, the state space includes the following aspects:

[0085] SATD value of the current frame: measures the complexity of the current frame.

[0086] Frame type: whether the current frame is an I frame, P frame or B frame.

[0087] Saturation of the target buffer: reflects the ratio of the amount of data in the current buffer to the total capacity.

[0088] Set target bitrate: The target bitrate that guides the encoding process.

[0089] Define the action space of reinforcement learning, where the action space is the range of values ​​of the quantization step size.

[0090] In this embodiment, the action space defines all possible actions that the reinforcement learning model can take. In this case, the action space is the range of values ​​of the quantization step size. The quantization step size determines the accuracy of video encoding. A larger quantization step size will result in lower video quality but higher compression rate, and vice versa.

[0091] Design a reward function to comprehensively evaluate the encoded video quality SSIM indicator, buffer status, and bitrate usage.

[0092] In this embodiment, the reward function is used to evaluate the performance of the model after taking a certain action. The reward function should comprehensively consider the following factors:

[0093] Encoded video quality (SSIM index): Evaluates the subjective visual quality of the video.

[0094] Buffer status: Ensure that the buffer does not overflow due to excessive data, nor waste resources due to excessive free space.

[0095] Bitrate usage: Ensures that the actual bitrate used matches the preset target bitrate to avoid video quality degradation or network congestion caused by excessively high or low bitrate usage.

[0096] Initialize the parameters of the reinforcement learning model, and use the Q-learning algorithm to train the model. During the training process, continuously iterate and update the parameters of the reinforcement learning model to find the optimal quantization step strategy through trial and error learning, so that the reinforcement learning model can dynamically adjust the quantization step according to different inputs.

[0097] Before training begins, the parameters of the reinforcement learning model need to be initialized. These parameters include, but are not limited to, model weights and biases. The purpose of initializing the parameters is to provide a starting point for the model to begin the learning process.

[0098] The Q-learning algorithm is used for model training. Q-learning is a value-based reinforcement learning method that determines the best action for the next step by learning the value of the state-action pair (i.e., Q value). During the training process, the model learns through continuous trial and error, gradually updating the Q value table to find the optimal quantization step strategy. The specific steps include:

[0099] At each decision point, an action is chosen based on the current state.

[0100] Perform the chosen action and observe the next state and the immediate reward obtained.

[0101] Update the Q-value of the current state-action pair based on the immediate reward and the expected reward of the next state.

[0102] The above process is repeated until the model converges to a stable policy.

[0103] During the training process, the model will continuously iterate and update its parameters, and find the optimal quantization step strategy through trial and error learning. Each iteration will adjust the model parameters according to the actions taken and the results obtained in the current state, so that the model can dynamically adjust the quantization step according to different inputs to achieve the purpose of optimizing video encoding quality.

[0104] S3: During encoding, the second quantization step size of each coding unit in the current frame is obtained, and combined with the first quantization step size of the current frame, a final quantization step size of each coding unit is calculated.

[0105] In this embodiment, there are two ways to obtain the second quantization step size, which are described in detail below:

[0106] A first method for acquiring the second quantization step size is: predicting the second quantization step size of each coding unit in the current frame based on the multi-scale feature map of the current frame.

[0107] In this manner, the method for obtaining the second quantization step size specifically includes the following steps:

[0108] First, a multi-scale feature map of the current frame is extracted. The multi-scale feature map includes the brightness, color distribution, texture complexity and motion vector of the current frame.

[0109] In this embodiment, the specific method of extracting the multi-scale feature map includes:

[0110] A convolutional neural network (CNN) is used to extract feature maps of different scales, which reflect different levels of information, including low-level edge detection, mid-level texture features, and high-level content recognition.

[0111] The original frame is processed at multiple scales to obtain features at different scales by including but not limited to downsampling or using filters of different sizes.

[0112] Secondly, based on the extracted multi-scale feature map, a pre-trained deep learning model is used to predict the second quantization step of each coding unit in the current frame, where the deep learning model is trained through video data and learns to assign quantization steps according to different attributes of the feature map.

[0113] In this embodiment, a multi-scale feature map is input into a pre-trained deep learning model, and the model learns the relationship between features through an internal multi-layer neural network structure.

[0114] The model outputs the second quantization step size corresponding to each coding unit. These quantization step sizes are dynamically adjusted according to the content characteristics of the current frame to achieve the best compression effect and video quality.

[0115] Through the above steps, the second quantization step size of each coding unit can be predicted based on the multi-scale feature map of the current frame. The advantage of this method is that it can be finely adjusted according to the specific content of the current frame, thereby improving coding efficiency and video quality.

[0116] The second method for obtaining the second quantization step size is: calculating the second quantization step size of each coding unit in the current frame according to the saturation of the target buffer and the coding quantization step size of the coding unit at the corresponding position in the previous frame.

[0117] In this manner, the method for obtaining the second quantization step size specifically includes the following steps:

[0118] First, the saturation of the current target buffer is monitored and obtained, where the saturation reflects the ratio of the remaining space in the buffer to the total capacity.

[0119] During the encoding process, the state of the target buffer is continuously monitored, and the currently used buffer space and total capacity are recorded.

[0120] The current saturation of the buffer is calculated using the following formula:

[0121]

[0122] For example, if the total capacity of the buffer is 100 MB and 70 MB is currently used, the saturation is 70%.

[0123] A saturation threshold is preset. When the saturation of the buffer exceeds this threshold, it indicates that the buffer is close to being full and measures need to be taken to reduce the amount of data.

[0124] Secondly, for each coding unit in the current frame, the coding quantization step size of the coding unit at the corresponding position in the previous frame is searched and obtained.

[0125] In order to obtain the second quantization step size of each coding unit in the current frame, it is necessary to refer to the coding quantization step size of the coding unit at the same position in the previous frame. The specific steps are as follows:

[0126] Locate the coding units of the previous frame: For each coding unit (such as macroblock or coding tree unit) in the current frame, determine its corresponding position in the previous frame.

[0127] Get the quantization step size of the previous frame: Get the quantization step size used when encoding the corresponding coding unit in the previous frame.

[0128] Record quantization step size: record the quantization step size of the previous frame as the reference quantization step size of the corresponding coding unit in the current frame.

[0129] Finally, according to the saturation adjustment strategy of the current target buffer, combined with the obtained encoding quantization step size and saturation adjustment strategy of the previous frame, the second quantization step size of each coding unit in the current frame is calculated, where the saturation adjustment strategy is:

[0130] If the buffer saturation is high, the second quantization step size tends to be reduced to reduce the amount of data. Otherwise, the second quantization step size is appropriately increased to balance the coding efficiency and video quality.

[0131] In this embodiment, according to the saturation adjustment strategy of the current target buffer, combined with the encoding quantization step of the previous frame, the second quantization step of each coding unit in the current frame is calculated. The specific steps are as follows:

[0132] Formulate adjustment strategy: Formulate adjustment strategy according to the current buffer saturation. If the buffer saturation is high, the second quantization step size tends to be reduced to reduce the amount of data; otherwise, the second quantization step size is appropriately increased to balance coding efficiency and video quality.

[0133] Calculate the second quantization step size: Use the following formula to calculate the second quantization step size of each coding unit in the current frame.

[0134]

[0135] in, represents the second quantization step size, Indicates the encoding quantization step size of the corresponding position encoding unit in the previous frame, represents the adjustment factor, Indicates the saturation of the current target buffer. Indicates the threshold value.

[0136] Adjustment Factor The calculation of can be based on the following rules:

[0137] If the current buffer is saturated (higher than a preset threshold), then <1, to reduce the second quantization step size.

[0138] If the current buffer is less saturated (below a preset threshold), then >1 to increase the second quantization step size.

[0139] Finally, the calculated second quantization step size is applied to each coding unit in the current frame to ensure that good coding results can be achieved under different buffer saturation levels.

[0140] After obtaining the first quantization step size and the second quantization step size, the final quantization step size is calculated by combining the two according to the following formula. The specific calculation formula is:

[0141]

[0142] in, Indicates Frame The final quantization step size of the coding unit, Indicates Frame The second quantization step size of the coding unit, Indicates The first quantization step size of the frame, and are the weights of the second quantization step size and the first quantization step size, respectively, and .

[0143] S4: quantize the corresponding coding unit in the current frame using the final quantization step size, perform entropy coding on the quantized result to generate a bit stream, and store the generated bit stream in the target buffer.

[0144] In this embodiment, each coding unit in the current frame is quantized using the final quantization step size. The quantization process converts the original pixel value into fewer discrete levels, thereby reducing the amount of data. The larger the quantization step size, the coarser the converted data, the higher the compression ratio, but the video quality will decrease.

[0145] Perform entropy coding on the quantized data. Entropy coding is a lossless compression method used to further reduce the amount of data without losing information. Entropy coding methods include but are not limited to Huffman Coding and Arithmetic Coding. These methods use the frequency of symbol occurrence to assign code lengths, with shorter codes for frequently occurring symbols and longer codes for less frequently occurring symbols, thereby achieving compression.

[0146] Convert the entropy-encoded data into a bitstream format. The bitstream contains all the encoding information of the current frame, including the quantized data and other necessary metadata (such as frame header information and quantization parameters).

[0147] The generated bitstream is stored in the previously initialized target buffer. This buffer is used as a temporary storage space to temporarily store the encoded data until the data is packaged and transmitted to the receiver over the network.

[0148] S5: Pack the stored bit stream data into data packets and send them to the receiving end to achieve encoded transmission of the video stream.

[0149] In this embodiment, the stored bit stream data is read from the target buffer and packaged into a data packet. A data packet usually contains one or more continuous bit stream segments and necessary control information (such as a sequence number and a synchronization flag) to ensure that the data packets can be correctly sequenced and reassembled during transmission.

[0150] The packaged data packets are sent to the receiving end through the network. During the sending process, factors such as network bandwidth and latency need to be considered to ensure that data can be transmitted efficiently and reliably. If the network bandwidth fluctuates greatly, the sending rate needs to be dynamically adjusted according to the current network conditions to prevent data packet loss or excessive latency.

[0151] Use a protocol suitable for video transmission, including but not limited to TCP / IP or UDP, to send data packets. TCP provides reliable transmission services, but may cause higher latency; UDP provides faster transmission speeds, but has poorer reliability. Depending on the application scenario, you can choose a suitable transmission protocol.

[0152] After receiving the data packet, the receiver will send a confirmation message back to the sender to confirm that the data packet has been successfully received. The sender can determine whether the data packet needs to be retransmitted based on the received confirmation message, thereby ensuring the integrity of the data.

[0153] The working principle of the intelligent encoding and transmission method of ultra-high-definition 8k video provided by the present invention is as follows:

[0154] First, initialize the buffer and target bit rate: Before encoding begins, the system first initializes a buffer for storing encoded data and sets an initial target bit rate. This target bit rate is preset based on factors such as network conditions, device performance, and user needs, and is used to guide the subsequent encoding process.

[0155] Collect training data and train reinforcement learning models: A large number of ultra-high-definition 8K video samples are collected as training data sets, and these videos are preprocessed (such as frame extraction, feature extraction, etc.) to train a reinforcement learning model. The model can learn how to dynamically adjust the quantization step size according to the characteristics of the current video frame (such as SATD value, frame type, buffer saturation, etc.) to optimize coding efficiency and video quality.

[0156] Secondly, during the encoding process, key information such as the SATD value (Sum of Absolute Transformed Differences, an indicator for measuring the amount of change within a frame), frame type (such as I frame, P frame, B frame) and saturation of the target buffer of the current video frame are first obtained.

[0157] The features of the current frame are used as input, and the first quantization step of the current frame is predicted by the pre-trained reinforcement learning model. This quantization step is a preliminary estimate of the encoding quality based on a global perspective.

[0158] The second quantization step size of each coding unit (such as macroblock or coding tree unit) in the current frame is predicted in two ways. One way is based on the multi-scale feature map of the current frame (including brightness, color distribution, texture complexity and motion vector, etc.) and predicted by a deep learning model; the other way is to calculate it based on the saturation of the target buffer and the coding quantization step size of the coding unit at the corresponding position in the previous frame.

[0159] The first quantization step size and the second quantization step size are combined and weighted to obtain the final quantization step size of each coding unit. This step comprehensively considers global and local coding requirements to achieve more refined quantization control.

[0160] Each coding unit in the current frame is quantized using the final quantization step size, and the quantized results are entropy encoded to generate a bitstream. These bitstreams are then stored in the target buffer.

[0161] Next, the bit stream data stored in the buffer is packaged into data packets and sent to the receiving end through the network. During the sending process, the sending rate is dynamically adjusted according to the fluctuation of the network bandwidth to ensure the smoothness of data transmission.

[0162] Finally, during the encoding and transmission process, the saturation of the target buffer and the fluctuation of the network bandwidth are monitored in real time, and the preset target bit rate is dynamically adjusted based on this information. When the buffer saturation is low and the network bandwidth is sufficient, the target bit rate is increased to improve the video quality; otherwise, the target bit rate is reduced to avoid buffer overflow or network congestion.

[0163] In summary, the method of the present invention realizes high-quality, low-latency encoding and transmission of video streams by combining technical means such as reinforcement learning, multi-scale feature analysis and dynamic bit rate adjustment.

[0164] Embodiment 2

[0165] The present invention provides an ultra-high-definition 8K video intelligent coding transmission system for executing an ultra-high-definition 8K video intelligent coding transmission method, referring to Figure 2 As shown, the coding transmission system includes:

[0166] The initialization module 100 is used to initialize the buffer and set the target bit rate for the video stream before encoding.

[0167] The first quantization step acquisition module 200 is used to obtain the SATD value and frame type of the current frame in the video stream, and take the saturation, SATD value, frame type and set target bit rate of the target buffer corresponding to the current frame as input, and output the first quantization step of the current frame through a pre-trained reinforcement learning model, wherein the current frame is any frame in the video stream.

[0168] The final quantization step calculation module 300 is used to obtain the second quantization step of each coding unit in the current frame during encoding, and calculate the final quantization step of each coding unit in combination with the first quantization step of the current frame.

[0169] The video encoding module 400 is used to quantize the corresponding coding unit in the current frame using the final quantization step size, perform entropy encoding on the quantized result to generate a bit stream, and store the generated bit stream in the target buffer.

[0170] The video transmission module 500 is used to package the stored bit stream data into data packets and send them to the receiving end to achieve coded transmission of the video stream.

[0171] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0172] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an one-time programmable read-only memory (OTPROM), an electronically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0173] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

Claims

1. An intelligent encoding and transmission method for ultra-high-definition 8k video, characterized in that: The coding transmission method comprises the following steps: Before encoding, initialize the buffer for the video stream and set the target bitrate; The target bit rate is a dynamic value, and the dynamic adjustment step includes: During the encoding process, the saturation of the target buffer and the bandwidth fluctuation of the network transmission are monitored in real time; Dynamically adjust the preset target bit rate according to the saturation of the target buffer and the bandwidth fluctuation value to balance the smoothness of video transmission and video quality; When the target buffer saturation is lower than a preset saturation threshold and the network bandwidth is higher than a preset bandwidth threshold, the target bit rate is increased; otherwise, when the target buffer saturation is not lower than a preset saturation threshold or the network bandwidth is not higher than a preset bandwidth threshold, the target bit rate is reduced; Obtaining the SATD value and frame type of the current frame in the video stream, and taking the saturation, SATD value, frame type and set target bit rate of the target buffer corresponding to the current frame as input, outputting the first quantization step of the current frame through the pre-trained reinforcement learning model, wherein the current frame is any frame in the video stream; During encoding, the second quantization step size of each coding unit in the current frame is obtained, and combined with the first quantization step size of the current frame, the final quantization step size of each coding unit is calculated; The second quantization step size is obtained as follows: Obtain a multi-scale feature map of the current frame, and predict a second quantization step size of each coding unit in the current frame based on the multi-scale feature map of the current frame; Alternatively, the second quantization step size of each coding unit in the current frame is calculated according to the saturation of the target buffer and the coding quantization step size of the coding unit at the corresponding position in the previous frame; quantizing the corresponding coding unit in the current frame using the final quantization step size, performing entropy coding on the quantized result to generate a bit stream, and storing the generated bit stream in a target buffer; The stored bit stream data is packaged into data packets and sent to the receiving end to achieve encoded transmission of the video stream.

2. According to claim 1, the intelligent encoding and transmission method of ultra-high-definition 8k video is characterized in that: The calculation formula of the final quantization step size is: q′(i,j)=α·q(i,j)+β·q(i) Wherein, q′(i, j) represents the final quantization step of the j-th coding unit in the i-th frame, q(i, j) represents the second quantization step of the j-th coding unit in the i-th frame, q(i) represents the first quantization step of the i-th frame, α and β are the weights of the second quantization step and the first quantization step respectively, and α+β=1.

3. The intelligent encoding and transmission method of ultra-high-definition 8k video according to claim 1 is characterized in that: The training process of the reinforcement learning model is: Collect ultra-high-definition 8k video samples as a training data set, and pre-process each video sample, wherein the pre-processing includes frame extraction and feature extraction; Defining a state space for reinforcement learning, wherein the state space includes a SATD value of a current frame, a frame type, a saturation of a target buffer, and a set target bit rate; Defining an action space for reinforcement learning, wherein the action space is a range of values ​​for a quantized step size; Design a reward function to comprehensively evaluate the encoded video quality SSIM indicator, buffer status, and bitrate usage; Initialize the parameters of the reinforcement learning model, and use the Q-learning learning algorithm to train the model. During the training process, continuously iterate and update the parameters of the reinforcement learning model to find the optimal quantization step strategy through trial and error learning, so that the reinforcement learning model can dynamically adjust the quantization step according to different inputs.

4. The intelligent encoding and transmission method of ultra-high-definition 8k video according to claim 2 is characterized in that: The acquiring a multi-scale feature map of the current frame, and predicting a second quantization step size of each coding unit in the current frame based on the multi-scale feature map of the current frame, includes: Extracting a multi-scale feature map of the current frame, wherein the multi-scale feature map includes brightness, color distribution, texture complexity, and motion vector of the current frame; Based on the extracted multi-scale feature map, a pre-trained deep learning model is used to predict the second quantization step size of each coding unit in the current frame, wherein the deep learning model is trained through video data and learns to allocate quantization steps according to different attributes of the feature map.

5. The intelligent encoding and transmission method of ultra-high-definition 8k video according to claim 4 is characterized in that: The step of calculating the second quantization step of each coding unit in the current frame according to the saturation of the target buffer and the coding quantization step of the coding unit at the corresponding position in the previous frame includes: Monitor and obtain the saturation of the current target buffer, wherein the saturation reflects the ratio of the remaining space in the buffer to the total capacity; For each coding unit in the current frame, find and obtain the coding quantization step size of the coding unit at the corresponding position in the previous frame; According to the saturation adjustment strategy of the current target buffer, combined with the obtained encoding quantization step and saturation adjustment strategy of the previous frame, the second quantization step of each coding unit in the current frame is calculated, wherein the saturation adjustment strategy is: If the buffer saturation is high, the second quantization step size tends to be reduced to reduce the amount of data. Otherwise, the second quantization step size is appropriately increased to balance the coding efficiency and video quality.

6. The intelligent encoding and transmission method of ultra-high-definition 8k video according to claim 5 is characterized in that: The calculation formula of the second quantization step size is: q(i, j) = q·(1+γ·(AB)) Among them, q(i, j) represents the second quantization step, q represents the encoding quantization step of the corresponding position encoding unit in the previous frame, γ represents the adjustment factor, A represents the saturation of the current target buffer, and B represents the threshold.

7. An ultra-high-definition 8K video intelligent coding transmission system, used to execute an ultra-high-definition 8K video intelligent coding transmission method according to any one of claims 1 to 6, characterized in that: The coding transmission system comprises: An initialization module is used to initialize the buffer and set the target bit rate for the video stream before encoding; a first quantization step acquisition module, used to acquire the SATD value and frame type of the current frame in the video stream, and take the saturation, SATD value, frame type and set target bit rate of the target buffer corresponding to the current frame as input, and output the first quantization step of the current frame through a pre-trained reinforcement learning model, wherein the current frame is any frame in the video stream; The final quantization step length calculation module is used to obtain the second quantization step length of each coding unit in the current frame during encoding, and calculate the final quantization step length of each coding unit in combination with the first quantization step length of the current frame; A video encoding module, configured to quantize the corresponding coding unit in the current frame using the final quantization step size, perform entropy encoding on the quantized result to generate a bit stream, and store the generated bit stream in a target buffer; The video transmission module is used to package the stored bit stream data into data packets and send them to the receiving end to achieve encoded transmission of the video stream.

Images