Audio and video media transmission method and system

Abstract

The invention discloses an audio and video media transmission method and system, and relates to the field of data transmission, and the method comprises the steps: obtaining video frame data; performing motion analysis on the video frame data to obtain a first region, a second region and a third region; coding the video frame data of the first region by adopting an inter-frame prediction coding mode; coding the video frame data of the second area by adopting a motion compensation coding mode; coding the video frame data of the third region by adopting an intra-frame prediction coding mode; coding the motion vector of the second region to obtain fourth coded data; performing code rate control and bit stream encapsulation according to the first coded data, the second coded data and the third coded data to obtain a video code stream; and combining the video code stream and the fourth coded data to obtain coded video transmission data, and transmitting the coded video transmission data. Aiming at inflexible code rate control in video coding in the prior art, the invention improves the code rate control flexibility.

Description

Technical Field

[0001] The present application relates to the field of data transmission, and in particular to an audio and video media transmission method and system. Background Art

[0002] With the popularization of video applications and the diversification of network environments, video encoding and transmission technologies are facing new challenges and demands. In the fields of video communication, video surveillance, interactive entertainment, etc., users have put forward higher requirements for video quality, real-time performance, and smoothness. However, due to the differences in network bandwidth, device capabilities, and usage scenarios, how to achieve efficient, flexible, and adaptable video encoding and transmission in a complex and changing environment has become a key issue that needs to be solved urgently.

[0003] Traditional video coding schemes, such as H.264 / AVC, H.265 / HEVC, etc., mainly adopt a block-based hybrid coding framework to achieve compressed representation of video sequences through modules such as intra-frame prediction, inter-frame prediction, transform quantization, and entropy coding. These schemes have made great progress in coding efficiency and reconstruction quality, but they also have some limitations. First, the existing schemes lack adaptability to video content, do not fully consider the characteristics of different video scenes, and it is difficult to adjust coding parameters and strategies in a targeted manner. Secondly, motion estimation and compensation, as key links, have high requirements for computing and storage resources, which affects the real-time performance of encoding. Furthermore, the rate control strategy is relatively simple, lacks flexible bit allocation and priority control mechanisms, and is difficult to adapt to dynamically changing network conditions and user needs.

[0004] Specifically, in applications such as video surveillance and drone aerial photography, video images usually contain areas with different characteristics, such as static backgrounds, slow-moving targets, and fast-moving targets. Traditional coding schemes use a unified coding mode and parameters, which makes it difficult to balance the coding efficiency and reconstruction quality of each area, resulting in problems such as loss of details in background areas and obvious blur and noise in moving areas. In real-time interactive scenarios such as video conferencing and distance education, network bandwidth and latency conditions often fluctuate greatly. Fixed bitrate control strategies cannot dynamically adjust coding parameters and transmission strategies according to network conditions, which can easily cause quality degradation problems such as video freezes, frame drops, and blurs, affecting user experience.

[0005] In addition, with the development of new technologies such as 5G and the Internet of Things, video applications are becoming more diversified and personalized. Different users and devices have different concerns and perception requirements for video content. The traditional "one-size-fits-all" encoding scheme lacks pertinence and flexibility and cannot meet differentiated service needs. For example, in live sports events, different users may be interested in different areas such as players, stadiums, and spectators. Fixed encoding strategies make it difficult to balance the clarity and smoothness of each area, affecting the user's viewing experience. Summary of the invention

[0006] In view of the problem of inflexible bit rate control in video encoding in the prior art, the present application provides an audio and video media transmission method and system, which divides the video content into a background area, a slow motion area and a fast motion area according to motion characteristics, and adopts adaptive coding modes for different areas, using inter-frame prediction coding for the background area, motion compensation coding for the slow motion area, and intra-frame prediction coding for the fast motion area, and separately encodes the motion vector of the slow motion area; finally, the encoded data and motion vector data of different areas are merged and transmitted, thereby improving the flexibility of bit rate control.

[0007] The purpose of this application is achieved through the following technical solutions.

[0008] One aspect of the present application provides an audio and video media transmission method, including: S1, obtaining video frame data; S2, performing motion analysis on the video frame data to obtain a first area, a second area and a third area; the first area represents a background area, the second area represents a smooth motion area, and the third area represents a fast motion area; S3, encoding the video frame data of the first area using an inter-frame prediction coding mode to obtain first coded data; S4, encoding the video frame data of the second area using a motion compensation coding mode to obtain second coded data and a motion vector of the second area; S5, encoding the video frame data of the third area using an intra-frame prediction coding mode to obtain third coded data; S6, encoding the motion vector of the second area to obtain fourth coded data; S7, performing bit rate control and bit stream encapsulation according to the first coded data, the second coded data and the third coded data to obtain a video code stream; S8, merging the video code stream and the fourth coded data to obtain coded video transmission data, and transmitting the coded video transmission data.

[0009] Further, S2, performing motion analysis on the video frame data to obtain the first area, the second area and the third area, including: S21, dividing the video frame data into a plurality of blocks; S22, searching the reference frame for the block with the highest matching degree with the current block by minimum absolute difference to obtain a candidate motion vector; S23, selecting the best motion vector from the candidate motion vectors ; S24, divide the corresponding blocks whose optimal motion vector is less than the first threshold into the first area, representing the background area; divide the corresponding blocks whose optimal motion vector is greater than the first threshold and less than the second threshold into the second area, representing the smooth motion area; divide the corresponding blocks whose optimal motion vector is greater than the second threshold into the third area, representing the fast motion area.

[0010] Among them, the reference frame refers to the encoded and reconstructed video frame used to predict the current coding block during the inter-frame prediction coding process. The reference frame is stored in the reference frame buffer, which usually contains multiple encoded frames before the current frame. In this application, the reference frame can be selected in the following ways: Time distance: The encoded frame closest to the current encoded frame in time is preferred as the reference frame. The closer the time distance, the better the motion compensation effect. Frame type: Select the reference frame according to the coding type of the reference frame (such as I frame, P frame, B frame). Usually, I frame is a key frame and can be decoded independently; P frame refers to the previous I frame or P frame; B frame can bidirectionally refer to the previous and next I frame or P frame. Motion characteristics: Select the reference frame according to the motion characteristics of the video content. For areas with slower movement, a reference frame with a longer time distance can be selected; for areas with faster movement, a reference frame with a closer time distance can be selected. Coding efficiency: Select the reference frame that can achieve the best coding efficiency (such as the minimum rate distortion cost). By evaluating the coding efficiency of different reference frames, the optimal reference frame combination is selected. Multiple reference frames: In order to improve the prediction accuracy, multiple reference frames can be selected for combined prediction. The final prediction block is generated by weighted averaging or adaptive selection of multiple reference frames. Number of reference frames: The number of reference frames needs to balance coding efficiency and computational complexity. The more reference frames there are, the higher the prediction accuracy may be, but the search computation amount will also be greater. The appropriate number of reference frames is usually determined based on the specific application scenario.

[0011] On the one hand, the second area is a smooth motion area and is encoded through a motion compensation coding mode. In motion compensation coding, the motion vector plays a vital role and is used to describe the position offset between the current block and the predicted block in the reference frame. However, directly transmitting the motion vector will occupy a large bandwidth overhead. Therefore, by separately encoding the motion vector of the second area to obtain the fourth encoded data, the amount of motion vector data can be significantly reduced and the efficiency of motion compensation coding can be improved. Common motion vector coding methods include predictive coding, entropy coding, etc. By utilizing the correlation of motion vectors in the spatial domain and the temporal domain, the bit rate of the motion vector can be greatly compressed.

[0012] On the other hand, the video frame data of the second region is encoded using a motion compensation coding mode to obtain second coded data and a motion vector of the second region. The second coded data mainly contains residual information after motion compensation, that is, the difference between the current block and the predicted block. The fourth coded data specifically represents the motion vector information of the second region. This separate coding method of motion information and residual information is conducive to subsequent bit rate control and transmission optimization. For example, the transmission priority and quality level of motion vector and residual data can be flexibly adjusted according to the network conditions and the receiving end capabilities, so as to obtain the best reconstruction effect under limited bandwidth.

[0013] Finally, encoding and transmitting the video code stream and the fourth coded data separately can improve the robustness and adaptability of the transmission. The video code stream carries the residual data of each area after encoding, corresponding to the rate control and bitstream encapsulation process of S7. The fourth coded data is optimized for the motion vector of the second area, with higher compression efficiency and flexibility. In the actual transmission process, the video code stream and the fourth coded data can adopt different transmission strategies and protection mechanisms to cope with changes in the network environment. For example, when the network is congested or packet loss is serious, the integrity of the video code stream can be prioritized at the expense of a certain degree of motion vector accuracy; when the network conditions are good, the transmission quality of the motion vector can be improved to obtain a more accurate motion compensation effect.

[0014] By merging the video code stream and the fourth coded data for transmission, the receiving end can flexibly select a decoding strategy based on the actual receiving situation. Ideally, the receiving end can simultaneously receive and decode the video code stream and the fourth coded data to obtain a complete video reconstruction effect. However, when bit errors or packet loss occur in network transmission, the receiving end can take adaptive error concealment measures based on the received data. For example, when the fourth coded data is lost or damaged, the receiving end can use the residual information in the video code stream and the motion vectors of adjacent blocks to estimate the lost motion vectors through interpolation or prediction, thereby maintaining a certain video quality. This fault-tolerant mechanism at the decoding end can effectively reduce the impact of transmission errors on the video experience.

[0015] Further, S23, the best motion vector is selected from the candidate motion vectors , including: according to each candidate motion vector, calculating the distortion metric between the corresponding motion compensation prediction block and the current block ; According to each candidate motion vector, calculate the corresponding encoding bit rate : ,in Indicates the coding bit rate of the motion vector, which is related to the probability distribution of the motion vector and the entropy coding method; The coding bit rate of the motion compensation residual is related to the residual energy and the transform coding method; the estimation of the coding bit rate is introduced, including the coding bit rate of the motion vector itself and the coding bit rate of the motion compensation residual. The coding bit rate of the motion vector is related to its probability distribution and entropy coding method, usually using entropy coding technology such as exponential Golomb coding or context adaptive binary arithmetic coding (CABAC). The coding bit rate of the motion compensation residual is related to the residual energy and the transform coding method. The larger the residual energy, the higher the coding bit rate; using DCT transform or wavelet transform can concentrate the residual energy and improve coding efficiency.

[0016] Constructing rate-distortion optimization objective function : , where λ is the Lagrange multiplier, which controls the weight distribution of rate-distortion optimization; the objective function of rate-distortion optimization is constructed, and the distortion metric and the encoding bit rate are weighted and summed in the form of Lagrange multipliers. The Lagrange multiplier λ controls the weight distribution of rate-distortion optimization. The larger λ is, the more it tends to reduce the encoding bit rate; the smaller λ is, the more it tends to reduce the distortion metric. The selection of λ needs to be adaptively adjusted according to factors such as the encoder's target bit rate and the complexity of the video content to achieve the best rate-distortion balance.

[0017] Solve the minimum value of the rate-distortion optimization objective function and obtain the optimal motion vector : ,in, Indicates that among all candidate motion vectors, select the one that makes the objective function The motion vector that reaches the minimum value is taken as the optimal motion vector. The solution process of rate-distortion optimization is given, that is, among all candidate motion vectors, the motion vector that makes the objective function reach the minimum value is selected as the optimal motion vector. This is a discrete optimization problem, which can be solved by exhaustive search or heuristic search. In order to reduce the computational complexity, fast search algorithms such as three-step search, diamond search, orthogonal search, etc. are usually used to first locate the approximate range with coarse granularity, and then perform local optimization with fine granularity, and finally obtain the global optimal or suboptimal motion vector.

[0018] Furthermore, ,in, Indicates is the distortion measure between the candidate prediction block of the motion vector and the current block, represents the pixel value of the current block at pixel position (x, y), Indicates The candidate prediction block for the motion vector is at pixel position The pixel value at Represents the weight factor of the current block at the pixel position (x, y), which is used to reflect the contribution of the pixel at this position to the distortion metric. By introducing the weight factor , the distortion metric formula can better adapt to the content characteristics of the video frame and the visual characteristics of the human eye. When calculating the distortion metric of the candidate prediction block and the current block, the contribution of pixels of different positions and types is distinguished, so as to more accurately evaluate the matching degree of the prediction block. The reasonable setting of weight factors helps to improve the accuracy of motion estimation, reduce prediction errors, and ultimately improve the performance and quality of video encoding. For areas with faster motion speed and larger changes in motion direction, their weights can be appropriately increased, because these areas are more difficult to predict and require more attention; for areas with slower motion speed and smaller changes in motion direction, their weights can be appropriately reduced, because these areas are relatively simple to predict and less sensitive to distortion.

[0019] Further, S3, the video frame data of the first area is encoded using the inter-frame prediction coding mode to obtain the first encoded data; S31, one or more previously encoded frames are selected as reference frames in the reference frame buffer, and the reference frame buffer is used to store the reconstructed images of several frames encoded previously; S32, the block that best matches the current block is searched in the reference frame as the reference block of the current block; the residual between the current block and the reference block is calculated to obtain the residual block; S33, DST (Discrete Sine Transform) is performed on the residual block to obtain the transformation coefficient matrix: the residual block is divided into several sub-blocks, each of which has a size of 4×4; for each 4×4 sub-block, a 4×4 DST matrix is ​​used to perform a two-dimensional transformation to obtain the corresponding transformation coefficient sub-block; all the transformation coefficient sub-blocks are spliced ​​to obtain the transformation coefficient matrix; S34, the quantization parameter QP (Quantization Parameter) quantizes the transform coefficient matrix to obtain a quantization coefficient matrix: according to the rate-distortion optimization criterion, determines the optimal quantization parameter QP in the inter-frame prediction coding mode; uses the selected quantization parameter QP to quantize each coefficient in the transform coefficient matrix to obtain a quantization coefficient matrix; S35, vertically scans the quantization coefficient matrix to obtain a coefficient symbol sequence; starting from the lower left corner of the quantization coefficient matrix, scans each column of the quantization coefficient matrix in turn in a 45-degree direction from bottom to top and from left to right in a Z-shaped order to obtain a column vector sequence; reverses and transposes the column vector sequence to obtain a coefficient symbol sequence; S36, run-length encodes the coefficient symbol sequence to obtain a sequence of run pairs; counts the non-zero coefficients in the coefficient symbol sequence and the trailing zero coefficients thereof to obtain (run, level) pairs, where the run represents the number of trailing zero coefficients and the level represents the value of the non-zero coefficient; arranges all (run, level) pairs to form a sequence; S37, entropy encodes the sequence of (run, level) pairs to obtain first encoded data. According to the probability distribution of (run, level) pairs, a run-level joint coding table is constructed; using the run-level joint coding table, a sequence of (run, level) pairs is variable-length encoded; and the encoded bit stream is output as the first encoded data.

[0020] Further, S4, the video frame data of the second area is encoded using a motion compensation coding mode, including: S41, using motion estimation, searching for a block that best matches the current block in the reference frame to obtain a motion vector; S42, using motion compensation, generating a prediction block at a corresponding position in the reference frame according to the motion vector; S43, calculating the residual between the current block and the prediction block to obtain a residual block; S44, transforming, quantizing and entropy encoding the residual block to obtain second encoded data; S45, entropy encoding the motion vector to obtain the motion vector of the second area.

[0021] Further, S45, entropy encoding the motion vector to obtain the motion vector of the second area includes: screening the motion vector to obtain the horizontal component and the vertical component ; Perform credibility check on motion vectors and remove unreliable motion vectors; Perform redundancy check on credible motion vectors and remove redundant motion vectors; Normalize the filtered motion vectors to obtain a normalized horizontal component and the vertical component ; Using spatial and temporal correlation, select prediction values ​​from the coded motion vectors of adjacent blocks and adjacent frames; select the median value as the spatial prediction value among the coded motion vectors of adjacent blocks; select the median value as the temporal prediction value among the coded motion vectors of the co-located blocks of adjacent frames; calculate the weighted average value based on the spatial prediction value and the temporal prediction value as the comprehensive prediction value of the motion vector; calculate the horizontal component separately and the vertical component The difference between the predicted value and the predicted value is the prediction residual and ;calculate The difference between the horizontal component of the comprehensive prediction value and the horizontal prediction residual is obtained ;calculate The vertical prediction residual is obtained by comparing the vertical component of the comprehensive prediction value ; Calculate the prediction residuals and The statistical histogram of , and the corresponding probability distribution characteristics are obtained; and Calculate the numerical range histogram respectively and count the residual frequency in each numerical interval; and Calculate the sign distribution histograms respectively and count the frequencies of positive and negative residuals; normalize the value range histogram and the sign distribution histogram to obtain the probability distribution curve; calculate the mean, variance, skewness, kurtosis and other characteristic parameters according to the probability distribution curve; set the order of Hadamard transform according to the probability distribution characteristics, and use the Hadamard transform after setting the order to predict the residuals. and Perform a two-dimensional transformation to obtain a transformation coefficient matrix; according to The probability distribution characteristics of the horizontal transformation are set to ;according to The probability distribution characteristics of the vertical transformation are set to ;use The Hadamard transformation matrix of Perform horizontal transformation and use The Hadamard transformation matrix of Perform vertical transformation to obtain a transformation coefficient matrix;

[0022] Quantize and encode the transform coefficient matrix using context adaptive variable length coding CAVLC to obtain a quantized coefficient stream; construct multiple coding tables of CAVLC, including Total Zeros table, Run Before table, Level table, etc.; scan the transform coefficient matrix to obtain a coefficient symbol sequence; perform run-length encoding on the coefficient symbol sequence to obtain a (run, level) pair; according to the context information of the (run, level) pair, search and output the corresponding codeword in the corresponding coding table to obtain a quantized coefficient stream;

[0023] The quantized coefficient stream is packaged and multiplexed to obtain the motion vector of the second region. and Encapsulate the quantized coefficient stream, add synchronization header and length information, so that the decoder can accurately locate and separate the data of each component; and The quantized coefficient stream is multiplexed to interleave the data of the two components together to form complete motion vector information.

[0024] Further, S5, the video frame data of the third area is encoded using the intra-frame prediction coding mode to obtain third encoded data: S51, according to the encoded blocks adjacent to the current block, the spatial prediction direction and the prediction mode are calculated: the grayscale mean and variance of the encoded blocks on the left side of the current block are calculated; the grayscale mean and variance of the encoded blocks above the current block are calculated; according to the comparison of the grayscale mean and variance, the spatial prediction direction is determined to be the horizontal direction or the vertical direction; in the determined spatial prediction direction, according to the grayscale value distribution characteristics of the adjacent encoded blocks, the optimal prediction mode is selected from multiple prediction modes; S52, according to the spatial prediction direction and the prediction mode, a prediction block of the current block is generated: if the spatial prediction direction is In the horizontal direction, the boundary pixel values ​​of the coded blocks on the left side of the current block are used to generate a prediction block according to the selected prediction mode; if the spatial prediction direction is the vertical direction, the boundary pixel values ​​of the coded blocks above the current block are used to generate a prediction block according to the selected prediction mode; S53, calculate the residual between the current block and the prediction block to obtain a residual block: perform point-by-point difference between the pixel values ​​of the current block and the pixel values ​​of the prediction block to obtain a difference matrix; perform weighted quantization on the difference matrix to obtain a quantized residual block; S54, transform, quantize and entropy encode the residual block to obtain third encoded data: perform DCT transform on the quantized residual block: divide the quantized residual block into several 4×4 sub-blocks; perform a two-dimensional DCT transform on each 4×4 sub-block using a 4×4 DCT transform matrix to obtain a transform coefficient matrix; scale and truncate the transform coefficient matrix to obtain a scaled transform coefficient matrix;

[0025] The scaled transform coefficient matrix is ​​quantized: according to the characteristics of the intra-frame prediction coding mode, the range and step size of the quantization parameter QP are preset; each coefficient in the transform coefficient matrix is ​​quantized using the selected quantization parameter QP to obtain a quantization coefficient matrix; entropy coding is performed on the quantization coefficient matrix: a zigzag scan is performed on the quantization coefficient matrix to obtain a coefficient sequence; run-length coding and equal-length coding are performed on the coefficient sequence; CABAC entropy coding is performed on the coding result to obtain third coded data.

[0026] Further, S6, encoding the motion vector of the second region to obtain fourth encoded data, including: S61, according to the horizontal motion vector component of the second region and the vertical motion vector component , respectively select the coded motion vectors of the following areas as prediction values, and calculate the weighted average value as the comprehensive prediction value: select the coded motion vectors of the spatial adjacent blocks such as the left, above, upper right, and lower left of the current block as the spatial prediction value; select the coded motion vectors of the co-located blocks of the current block in the temporal adjacent frames such as the previous frame and the previous two frames as the temporal prediction value; calculate the weighted average value based on the spatial prediction value and the temporal prediction value as and The comprehensive prediction value of , where the weight coefficient is adaptively determined according to the spatiotemporal distance between adjacent blocks and adjacent frames; combining the spatial domain prediction value and the temporal domain prediction value, we can get and The prediction accuracy is expected to be further improved by using the comprehensive prediction value of . At the same time, the adaptive weighted average strategy is adopted to dynamically adjust the weight coefficient according to the temporal and spatial distances between adjacent blocks and adjacent frames, so that the prediction value can be more in line with the actual motion trend of the current block. S62, respectively calculate the horizontal motion vector components and the vertical motion vector component The difference between its comprehensive prediction value is used to obtain the motion vector prediction residual and ; S63, respectively calculate the motion vector prediction residual and The numerical range histogram and the symbol distribution histogram of are obtained, and the following processing is performed to obtain its probability distribution characteristics: the numerical range histogram and the symbol distribution histogram are normalized to eliminate the scale difference; the statistical characteristic parameters such as mean, variance, skewness, and kurtosis are extracted from the normalized histogram to characterize and The probability distribution characteristics of the residuals; the numerical range histogram and the sign distribution histogram reflect the distribution characteristics of the residuals from the perspectives of amplitude and positive and negative signs respectively.

[0027] S64, according to and The probability distribution characteristics of the horizontal and vertical transforms are set by setting the Hadamard transform order, and the motion vector prediction residual is calculated by using the two-dimensional Hadamard transform of the set order. and Transform and get and The corresponding transformation coefficient matrix: According to The probability distribution characteristics of the horizontal transformation are adaptively set to the Hadamard transformation order. ;according to The probability distribution characteristics of the vertical transformation are adaptively set to the Hadamard transform order. ;use The one-dimensional Hadamard transform of To transform horizontally, use The one-dimensional Hadamard transform of Performing a vertical transformation, we get and The two-dimensional transformation coefficient matrix of ; when the probability distribution of residual samples is more concentrated, the transformation order can be reduced accordingly; conversely, when the probability distribution of residual samples is more dispersed, the transformation order needs to be appropriately increased to match the sparse characteristics of the residual transform domain. use One-dimensional row transformation of order use The one-dimensional column transformation of order can be obtained through the two-dimensional separable Hadamard transformation. and The corresponding transformation coefficient matrix. Hadamard transform is an orthogonal transform based on rectangular wave function. The coefficients are all ±1, and the calculation only involves addition and subtraction operations. It has the advantages of fast, reversible, and integer precision.

[0028] S65, construct multiple CAVLC coding tables and and The transform coefficient matrix is ​​coded as follows to generate a motion vector quantization coefficient stream and : Perform a zigzag scan on the transform coefficient matrix to convert the two-dimensional matrix into a one-dimensional coefficient sequence; perform run-length coding on the coefficient sequence to obtain the run length and the non-zero coefficient level; perform entropy coding using the constructed CAVLC coding table according to the joint probability distribution of the run length and the non-zero coefficient level to obtain and quantized coefficient stream; first, the two-dimensional transform coefficient matrix is ​​rearranged into a one-dimensional coefficient sequence through zigzag scanning. Then, the coefficient sequence is run-length encoded to obtain two important properties: run length and non-zero coefficient level. The run length represents the number of zero-valued coefficients between two non-zero coefficients, and the non-zero coefficient level represents the numerical value of the non-zero coefficient. Finally, according to the probability distribution of the run length and the non-zero coefficient level in their respective dimensions, a series of pre-constructed CAVLC coding tables are used to perform joint entropy coding to obtain and The corresponding quantized coefficient stream. The coding table is generated based on the statistical laws of massive training data and can effectively match the probability distribution model of the transform coefficients.

[0029] Among them, the CAVLC coding table (Context-Adaptive Variable Length Coding Table) is an adaptive variable length coding table for entropy coding, which is mainly used to encode transform coefficients. In this application, multiple CAVLC coding tables are constructed according to the probability distribution characteristics of motion vector prediction residuals. Different coding tables can be designed for different residual distribution characteristics to optimize coding efficiency. By selecting the coding table that best suits the current residual distribution, the number of coding bits can be reduced and the compression performance can be improved. The construction of the AVLC coding table is usually based on a large amount of statistical data and offline training. By analyzing the statistical characteristics of the transform coefficients, the optimal coding table structure and codeword allocation are designed. In the actual encoding process, the encoder dynamically selects the appropriate coding table based on the context information and residual distribution of the transform coefficients, performs entropy encoding on the transform coefficients, and generates the final encoded bit stream.

[0030] S66, encapsulate the motion vector quantization coefficient stream to obtain the fourth encoded data: encapsulate the motion vector quantization coefficient stream, add necessary header information, tail information and synchronization information; multiplex the encapsulated motion vector quantization coefficient stream using time division multiplexing or code division multiplexing technology to obtain the fourth encoded data; the multiplexing operation can effectively improve the transmission efficiency and save bit rate overhead.

[0031] Another aspect of the present application also provides an audio and video media transmission system for executing an audio and video media transmission method of the present application.

[0032] Compared with the prior art, the advantages of this application are:

[0033] The motion analysis process accurately evaluates the motion complexity of the video content by calculating the pixel differences and motion vector sizes between adjacent frames. Background areas, slow motion areas, and fast motion areas have different spatiotemporal correlations and redundancy characteristics, so different strategies need to be adopted when controlling the bit rate. For example, the background area can be allocated less bit rate, the slow motion area needs to take into account the coding overhead of motion vectors and residuals, and the fast motion area needs to ensure the fidelity of texture details. Content-adaptive bit rate control through motion analysis can maximize the subjective visual quality of each area and improve encoding performance when the total bit rate is limited.

[0034] Inter-frame prediction coding eliminates redundant information of static background by referring to the temporal correlation between frames; motion compensation coding eliminates the displacement redundancy of slowly moving objects by motion estimation and motion compensation; intra-frame prediction coding effectively compresses the coding bit rate of fast-moving areas by using the spatial correlation of pixel values. Selecting the optimal coding mode for different areas can achieve the minimum distortion when the bit rate is limited, thus achieving efficient bit allocation and bit rate control.

[0035] Motion vector and residual are two key elements in motion compensation coding, which represent motion information and prediction error respectively. Traditional coding schemes usually encode motion vector and residual together, which makes it difficult to achieve flexible bit rate allocation. However, this application adopts a separate coding strategy, which can adjust the bit allocation ratio of motion vector and residual according to actual needs. For example, when transmitting at a low bit rate, the coding accuracy of the motion vector can be appropriately reduced, and more bit rate can be allocated to the residual signal; when transmitting at a high bit rate, the coding accuracy of the motion vector can be improved to obtain a more accurate motion compensation effect. Flexible bit rate control helps to obtain the best visual experience when bandwidth is limited.

[0036] Rate-distortion optimization is a typical motion estimation criterion that seeks the optimal motion vector by weighing the two factors of motion vector coding bit rate and prediction residual energy. Based on rate-distortion optimization, this application further utilizes the probability distribution characteristics of motion vectors, adaptively sets the order of Hadamard transform and CAVLC entropy coding table, thereby reducing the coding redundancy of motion vectors. Since motion vectors usually present a centrally symmetric Laplace distribution, targeted design of coding parameters and entropy coding tables can obtain a more compact bitstream, reduce coding distortion at the same bit rate, and improve motion compensation performance.

[0037] Traditional rate control schemes are usually independent of the encoding process, making it difficult to fully utilize the characteristics of video content, and easily leading to a disconnect between control decisions and the current encoding situation. This application feeds back the motion analysis results to the rate control module, dynamically allocates encoding parameters based on the spatiotemporal characteristics of the background area, the smooth motion area, and the fast motion area, thereby improving the content adaptability of the rate control; at the same time, the encoding end and the transmission end collaborate to optimize, and through the hierarchical transmission and scheduling of the video bitstream and motion vector, the decoding quality of important information is maximized in the event of network fluctuations and packet loss. Closed-loop control and dynamic optimization strategies effectively improve the robustness and real-time performance of the system, and meet the requirements of low-latency, high-quality transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The present application will be further described in the form of exemplary embodiments, which will be described in detail by the accompanying drawings. These embodiments are not restrictive, and in these embodiments, the same number represents the same structure, wherein:

[0039] Figure 1 is an exemplary flow chart of an audio and video media transmission method according to some embodiments of the present application;

[0040] Figure 2 is an exemplary flow chart of dividing a video frame according to some embodiments of the present application;

[0041] Figure 3 is an exemplary flow chart of calculating first coded data according to some embodiments of the present application;

[0042] Figure 4 is an exemplary flow chart of calculating the second encoded data according to some embodiments of the present application;

[0043] Figure 5 It is an exemplary flowchart for calculating the fourth encoded data according to some embodiments of the present application. DETAILED DESCRIPTION

[0044] The method and system provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0045] like Figure 1As shown, video frame data is obtained; motion analysis is performed on the video frame data to obtain a first area, a second area and a third area; the first area represents a background area, the second area represents a smooth motion area, and the third area represents a fast motion area; the video frame data of the first area is encoded in an inter-frame prediction coding mode to obtain first encoded data; the video frame data of the second area is encoded in a motion compensation coding mode to obtain second encoded data and a motion vector of the second area; the video frame data of the third area is encoded in an intra-frame prediction coding mode to obtain third encoded data; the motion vector of the second area is encoded to obtain fourth encoded data; based on the first encoded data, the second encoded data and the third encoded data, bit rate control and bit stream encapsulation are performed to obtain a video code stream; the video code stream and the fourth encoded data are merged to obtain encoded video transmission data, and the encoded video transmission data is transmitted.

[0046] Specifically, S1 obtains video frame data; receives video source data, which can come from digital cameras, video capture cards, video files and other sources. According to the format of the video source (such as YUV, RGB, etc.), parse the frame structure and encoding method of the video data to determine the starting position and length of each video frame. Split the video source data according to the frame boundaries, extract the data of a single video frame, and form a frame data sequence. Store the extracted video frame data in a memory buffer to form a frame buffer queue. The frame buffer queue uses a first-in-first-out (FIFO) method to store and access frame data in the time order of the frames. Define the frame data access interface, such as , get the data of the specified video frame according to the frame number. The frame data access interface reads the required frame data from the frame buffer queue or long-term frame buffer and passes it to the requested encoding module.

[0047] like Figure 2 As shown, S2, motion analysis is performed on the video frame data to obtain a first area, a second area and a third area; the first area represents a background area, the second area represents an area where the motion is less than a threshold, and the third area represents an area where the motion is greater than the threshold, including: S21, first, the input video frame data is divided into multiple blocks as basic units for motion estimation and compensation. The size of the block can be adaptively adjusted according to factors such as video content complexity and coding efficiency. Common block sizes include 16x16, 32x32, etc. The smaller the block, the higher the accuracy of motion estimation, but the greater the computational complexity; the larger the block, the lower the computational complexity, but the accuracy of motion estimation will also decrease. Therefore, a trade-off needs to be made between accuracy and complexity.

[0048] S22, for each block, a motion estimation method based on block matching is used to search the block that best matches the current block in the reference frame through the minimum absolute difference (MAD) criterion. Specifically, with the current block as the center, within the search window of the reference frame, the MAD value between the current block and the candidate block is calculated. The smaller the MAD value, the higher the similarity between the two blocks. The search process can use a full search or a fast search algorithm, such as a three-step search, a diamond search, etc., to reduce the amount of calculation. The displacement vector of the best matching block obtained by the search is the candidate motion vector.

[0049] S23, after obtaining the candidate motion vectors, the rate-distortion optimization (RDO) criterion is used to select the optimal motion vector. The RDO criterion comprehensively considers the two factors of reconstruction quality and coding overhead, and controls the weights of the two through Lagrange multipliers. Specifically, for each candidate motion vector, the distortion metric of the corresponding motion compensation prediction block and the current block is calculated. , The distortion metric uses the weighted mean square error (WMSE), which is the sum of the squares of the pixel differences between the predicted block and the current block and multiplies them by a weight factor. The weight factor reflects the importance of different pixel positions to visual perception and can be set adaptively based on factors such as human visual characteristics and gaze points. WMSE takes into account the correlation of pixels in a spatial neighborhood and is more consistent with subjective visual quality.

[0050] For each candidate motion vector, estimate the corresponding encoding bit rate , The coding bit rate includes two parts: motion vector and motion compensation residual. The coding bit rate of motion vector is related to its probability distribution and entropy coding method, and context-adaptive variable length coding or arithmetic coding can be used. The coding bit rate of residual is related to its energy size and transform coding method, and DCT, DST and other transform methods can be used, and the quantized bit rate can be estimated based on the quantization parameter.

[0051] Constructing rate-distortion optimization objective function , , the distortion measure and encoding bitrate Weighted summation, where the weight is controlled by the Lagrange multiplier λ. The value of λ can be adaptively adjusted according to the current bitrate constraint and image quality requirements, taking a larger value at a low bitrate and a smaller value at a high bitrate to balance the weight of distortion and bitrate.

[0052] Solving the objective function The minimum value of , .because is a discrete function that can be solved by exhaustive search or heuristic search. Although exhaustive search is computationally intensive, it can ensure that the global optimal solution is found; heuristic search, such as three-step search, finds suboptimal solutions through iterative approximation, with less computational complexity.

[0053] After obtaining the optimal motion vector, the video content is divided into three regions according to its size. Specifically, two motion vector thresholds th1 and th2 are set, and the blocks with the optimal motion vector less than th1 are divided into the background region, the blocks with the optimal motion vector greater than th1 and less than th2 are divided into the gentle motion region, and the blocks with the optimal motion vector greater than th2 are divided into the fast motion region. The selection of the threshold can be adaptively adjusted according to factors such as the characteristics of the video content and the bit rate constraint to balance the proportion of each region and the coding efficiency.

[0054] When dividing video content into background area, slow motion area and fast motion area, the selection of thresholds th1 and th2 is crucial to balance the proportion of each area and coding efficiency. The value range of the threshold needs to be adaptively adjusted according to factors such as video content characteristics, bit rate constraints, and target applications to obtain optimal performance.

[0055] Specifically, the threshold th1 is used to distinguish between background areas and motion areas, reflecting the degree of stillness of the video content. For still or slowly moving backgrounds, such as landscapes and buildings, th1 can take a smaller value, such as 1 to 3 pixels. In this way, most still or quasi-still blocks can be divided into background areas, reducing the amount of motion estimation and compensation calculations. For backgrounds with certain movements, such as crowds and traffic, th1 can be appropriately increased, such as 3 to 5 pixels. This can avoid dividing some small movements into background areas and improve the accuracy of motion compensation. For video content with more complex movements, such as sports events and action movies, th1 can be further increased, such as 5 to 8 pixels. This can better distinguish between background and motion areas, providing a more accurate reference for subsequent encoding strategies.

[0056] As for the threshold th2, it is used to distinguish between the slow motion area and the fast motion area, reflecting the motion intensity of the video content. For video content dominated by slow motion, such as news broadcasts, interviews and dialogues, th2 can take a smaller value, such as 5 to 10 pixels. In this way, most of the motion can be divided into the slow motion area, reducing the proportion of the fast motion area and improving the coding efficiency. For video content containing medium-intensity motion, such as TV series, variety shows, etc., th2 can be appropriately increased, such as 10 to 20 pixels. In this way, it is possible to better distinguish between slow motion and fast motion, and allocate appropriate coding resources to different areas. For video content dominated by fast motion, such as racing, extreme sports, etc., th2 can be further increased, such as 20 to 30 pixels or more. In this way, most of the motion can be divided into the fast motion area, improving the accuracy of motion compensation, and at the same time allocating more coding resources to the fast motion area to ensure the reconstruction quality.

[0057] This application makes full use of the spatiotemporal correlation of video signals, removes temporal redundancy between blocks through motion compensation, and takes into account coding overhead and reconstruction quality, to achieve optimal performance under limited bit rates. In addition, the video content is divided into different areas through thresholding, which provides an important basis for subsequent adaptive coding. Different coding strategies and parameter settings can be used for different areas to achieve more refined bit rate control and quality optimization.

[0058] like Figure 3 As shown, S3, after acquiring the video frame data, for the first area divided into the background area, the present application adopts the inter-frame prediction coding mode for encoding. Specifically, S31, one or more previously encoded frames are selected as reference frames in the reference frame buffer. The reference frame buffer is used to store the reconstructed images of several frames encoded previously, that is, the pictures after encoding and decoding. The selection of the reference frame can be determined based on factors such as time distance and similarity. Common strategies include forward reference, backward reference and bidirectional reference.

[0059] S32, in the selected reference frame, search for the block that best matches the current block through a block matching algorithm as the reference block of the current block. Common block matching algorithms include full search algorithm, three-step search algorithm, diamond search algorithm, etc. The matching process measures the similarity between the current block and the candidate block through indicators such as the minimum absolute sum of differences (SAD) or the minimum mean square error (MSE), and selects the best candidate block as the reference block. Calculate the residual between the current block and the reference block to obtain a residual block. The residual block represents the pixel difference between the current block and the reference block, reflecting the error of inter-frame prediction.

[0060] S33, perform DST (Discrete Sine Transform) on the residual block to obtain the transform coefficient matrix: divide the residual block into several 4×4 sub-blocks, and perform DST transform on each sub-block. For each 4×4 sub-block, use the 4×4 DST matrix to perform a two-dimensional transform to obtain the corresponding transform coefficient sub-block. The DST transform has a strong energy concentration and can concentrate the energy of the residual signal on the low-frequency coefficients, which is beneficial to subsequent quantization and encoding. All transform coefficient sub-blocks are spliced ​​to obtain a complete transform coefficient matrix.

[0061] S34, quantize the transform coefficient matrix using the quantization parameter QP (Quantization Parameter) to obtain the quantization coefficient matrix: determine the optimal quantization parameter QP in the inter-frame prediction coding mode according to the rate-distortion optimization criterion. Rate-distortion optimization selects the quantization parameter that can obtain the best reconstruction quality by weighing the bit rate and distortion. Quantize each coefficient in the transform coefficient matrix using the selected quantization parameter QP to obtain the quantization coefficient matrix. The quantization process maps the transform coefficient to a limited quantization level by dividing by the quantization step size and rounding to an integer, thereby achieving data compression.

[0062] S35, vertically scan the quantization coefficient matrix to obtain a coefficient symbol sequence; starting from the lower left corner of the quantization coefficient matrix, scan each column of the quantization coefficient matrix in a 45-degree direction from bottom to top and from left to right in a zigzag order to obtain a column vector sequence. Reverse and transpose the column vector sequence to obtain a coefficient symbol sequence. The sequence represents the sign and amplitude information of the quantization coefficient.

[0063] S36, run-length coding is performed on the coefficient symbol sequence to obtain a sequence of run-length pairs; non-zero coefficients in the coefficient symbol sequence and their trailing zero coefficients are counted to obtain (run, level) pairs. The run represents the number of zero coefficients that appear consecutively after the current non-zero coefficient, and the level represents the value of the non-zero coefficient. All (run, level) pairs are arranged in order to form a sequence of run-length pairs. Run-length coding can effectively compress the zero value in the quantized coefficient and reduce data redundancy.

[0064] S37, entropy encoding is performed on the sequence of (run, level) pairs to obtain the first coded data. According to the probability distribution of (run, level) pairs on the training set, a run-level joint coding table is constructed. The coding table assigns the optimal codeword to each (run, level) pair according to the statistical characteristics of the run and level. Using the run-level joint coding table, variable-length coding is performed on the run-pair sequence. Variable-length coding assigns codewords according to the frequency of occurrence of symbols, and shorter codewords are assigned to symbols with high frequency, and longer codewords are assigned to symbols with low frequency, thereby achieving further compression of data. The encoded bit stream is output as the first coded data to complete the inter-frame prediction coding process of the background area. The present application takes into account temporal redundancy and spatial redundancy, adaptive adjustment of quantization parameters, and dynamic optimization of coding tables, which can significantly improve data compression efficiency and reduce the bit rate of the background area while ensuring reconstruction quality.

[0065] like Figure 4 As shown, S4, after obtaining the video frame data, for the second area divided into the smooth motion area, the present application adopts the motion compensation coding mode for coding, S41, using motion estimation, searching the block that best matches the current block in the reference frame to obtain the motion vector. Motion estimation uses a block matching algorithm, such as a full search algorithm, a three-step search algorithm, etc., to find the block that is most similar to the current block in the reference frame, and its displacement is the motion vector. S42, using motion compensation, according to the motion vector, a prediction block is generated at the corresponding position in the reference frame. For the current coding block, its corresponding motion vector is obtained by motion estimation (such as S4). The motion vector indicates the position offset of the most matching block of the current block in the reference frame. According to the motion vector, the block that best matches the current block is found in the reference frame and used as the prediction block. The size of the prediction block is the same as that of the current block, and its position is obtained by adding the motion vector to the position of the current block. The prediction block in the reference frame is copied to the position of the current block to form a prediction block after motion compensation. The prediction block represents the best match of the current block in the reference frame, which is used for subsequent residual calculation and coding. The purpose of motion compensation is to use the temporal correlation between frames to predict the current block by using the coded blocks in the reference frame, thereby reducing the amount of data that needs to be encoded.

[0066] Among them, the quantization parameter QP (Quantization Parameter) is a key parameter that controls the quantization process of video encoding. It is used to adjust the trade-off between the bit rate and quality of the bitstream generated by the encoder. By adjusting the QP value, the bit rate of the bitstream generated by the encoder can be controlled. The larger the QP value, the smaller the amplitude of the coefficient after quantization, the fewer bits generated, and the lower the bit rate. Conversely, the smaller the QP value, the larger the amplitude of the coefficient after quantization, the more bits generated, and the higher the bit rate. In intra-frame coding, different QP values ​​can be assigned to different macroblocks or coding units (such as CTU). By locally adjusting the QP value, adaptive quantization control can be performed on different areas according to the complexity and importance of the image content.

[0067] S43, calculate the residual between the current block and the predicted block to obtain a residual block. Perform pixel-by-pixel difference calculation between the current block and the predicted block obtained by motion compensation to obtain a residual block. The residual block reflects the difference information between the current block and the predicted block. The pixel value of the residual block represents the pixel difference between the current block and the predicted block at the corresponding position. The closer the value in the residual block is to zero, the higher the match between the predicted block and the current block, and the higher the accuracy of motion compensation. The size of the residual block is the same as that of the current block, and the dynamic range of its pixel value is usually smaller than the original pixel value, which is beneficial to the subsequent transformation and quantization process. By calculating the residual block, the redundant information after motion compensation can be eliminated, and the amount of data to be encoded can be further reduced.

[0068] S44, transform, quantize and entropy encode the residual block to obtain second coded data. The residual block is transformed from the spatial domain to the frequency domain. Commonly used transformation methods include DCT (discrete cosine transform), DST (discrete sine transform), etc. The transformation process decomposes the residual block into a series of frequency coefficients through orthogonal transformation. The low-frequency coefficient represents the smooth area of ​​the residual block, and the high-frequency coefficient represents the detail area of ​​the residual block. The transformed frequency domain coefficient has the characteristic of concentrated energy, that is, most of the energy is concentrated on a few low-frequency coefficients, and the amplitude of the high-frequency coefficient is small. This is conducive to the subsequent quantization and entropy encoding process. The transformed frequency domain coefficient is quantized and mapped to a limited quantization level. The quantization process converts the frequency domain coefficient into a quantized coefficient by dividing by the quantization step size and rounding. The size of the quantization step size determines the quantization accuracy and compression rate. The larger the quantization step size, the smaller the amount of data after quantization, but the reconstruction quality will also decrease. The quantization step size can be adaptively adjusted according to the bit rate control and quality requirements. The quantized coefficients are entropy encoded and converted into a bit stream. Common entropy coding methods include variable length coding (such as Huffman coding) and arithmetic coding. Entropy coding achieves data compression by analyzing the probability distribution of symbols, assigning shorter codes to symbols with high probability and longer codes to symbols with low probability. The bit stream obtained after entropy coding is called the second coded data, which represents the compressed data after motion compensation and residual coding.

[0069] S45, entropy coding the motion vector to obtain the motion vector of the second region. Screen the motion vector to obtain the horizontal component and the vertical component Through credibility detection, unreliable motion vectors are eliminated; through redundancy detection, redundant motion vectors are eliminated. The filtered motion vectors are normalized to obtain the normalized horizontal component. and the vertical component .

[0070] Using spatial and temporal correlation, the prediction value is selected from the coded motion vectors of adjacent blocks and adjacent frames. Among the coded motion vectors of adjacent blocks, the median is selected as the spatial prediction value; among the coded motion vectors of the co-located blocks of adjacent frames, the median is selected as the temporal prediction value. The weighted average is calculated based on the spatial prediction value and the temporal prediction value as the comprehensive prediction value of the motion vector. The horizontal component is calculated separately. and the vertical component The difference between the predicted value and the predicted value is the prediction residual and .calculate The difference between the horizontal component of the comprehensive prediction value and the horizontal prediction residual is obtained ;calculate The vertical prediction residual is obtained by comparing the vertical component of the comprehensive prediction value .

[0071] Calculate prediction residuals and The statistical histogram of can be used to obtain the corresponding probability distribution characteristics. and Calculate the numerical range histogram respectively and count the residual frequency in each numerical interval; and Calculate the sign distribution histograms respectively and count the frequencies of positive and negative residuals. Normalize the value range histogram and the sign distribution histogram to obtain the probability distribution curve. Calculate the characteristic parameters such as mean, variance, skewness, and kurtosis based on the probability distribution curve.

[0072] According to the probability distribution characteristics, the order of Hadamard transform is set, and the prediction residual is calculated using the Hadamard transform after the order is set. and Perform a two-dimensional transformation to obtain a transformation coefficient matrix. The probability distribution characteristics of the horizontal transformation are set to ;according to The probability distribution characteristics of the vertical transformation are set to . use The Hadamard transformation matrix of Perform horizontal transformation and use The Hadamard transformation matrix of Perform vertical transformation to obtain the transformation coefficient matrix.

[0073] In the prediction residual and After performing the Hadamard transform, a transform coefficient matrix is ​​obtained. In order to further compress the data, the present application adopts context adaptive variable length coding (CAVLC) to quantize and encode the transform coefficient matrix, and uses CAVLC to quantize and encode the transform coefficient matrix to obtain a quantized coefficient stream. CAVLC is an efficient entropy coding method that uses the context information of the transform coefficient to adaptively select the optimal coding table to achieve data compression. Specifically, multiple coding tables of CAVLC are constructed, including Total Zeros table, Run Before table, Level table, etc. The Total Zeros table is used to encode the number of trailing zero coefficients in the transform coefficient; the Run Before table is used to encode the zero run length in front of the non-zero coefficient; and the Level table is used to encode the value of the non-zero coefficient. These coding tables are pre-constructed according to the statistical characteristics of the transform coefficients to optimize the coding efficiency. Scan the transform coefficient matrix to obtain a coefficient symbol sequence. CAVLC uses a zigzag scanning method, starting from the upper left corner of the transform coefficient matrix, and scanning each coefficient in turn in order from low frequency to high frequency to obtain a coefficient symbol sequence. The coefficient symbol sequence is run-length encoded to obtain a (run, level) pair. Run-length coding converts the coefficient symbol sequence into the form of (run, level) pairs by counting the number of zero coefficients between non-zero coefficients. The run represents the number of zero coefficients that appear continuously before the current non-zero coefficient, and the level represents the value of the non-zero coefficient. Run-length coding can effectively compress the zero value in the transform coefficient. According to the context information of the (run, level) pair, the corresponding codeword is searched and output in the corresponding coding table to obtain a quantized coefficient stream. For each (run, level) pair, CAVLC searches for the optimal codeword in the corresponding coding table according to its context information, such as the level of adjacent coefficients, the total number of zeros, etc. By adaptively selecting the coding table, CAVLC can make full use of the statistical characteristics of the transform coefficients and improve the coding efficiency. After performing Hadamard transform on the prediction residual, the present application uses CAVLC to quantize and encode the transform coefficient matrix, and by adaptively selecting the coding table, makes full use of the statistical characteristics of the transform coefficients to improve the coding efficiency.

[0074] The quantized coefficient stream is packaged and multiplexed to obtain the motion vector of the second region. Since the motion vector contains a horizontal component and the vertical component The two parts need to encapsulate and multiplex their quantized coefficient streams so that the decoder can accurately locate and separate the data of each component. and The quantized coefficient streams are encapsulated separately, and synchronization headers and length information are added. The synchronization header is used to identify the starting position of each component, and the length information is used to indicate the data length of each component. Through encapsulation, the decoder can accurately locate and extract the quantized coefficient stream of each component based on the synchronization header and length information. and The quantized coefficient stream is multiplexed, and the data of the two components are interwoven together to form complete motion vector information. Multiplexing forms a unified data stream by interweaving the data of different components according to certain rules. Common multiplexing methods include time division multiplexing and frequency division multiplexing. Through multiplexing, the horizontal and vertical components of the motion vector can be efficiently transmitted and stored. The quantized coefficient stream is encapsulated and multiplexed, and a synchronization header and length information are added to achieve efficient transmission and storage of the horizontal and vertical components of the motion vector. This scheme combines transform coding and entropy coding, and comprehensively considers data encapsulation and multiplexing. While reducing the amount of data, it ensures the integrity and decodability of the motion vector and improves the coding performance of the second area.

[0075] This application adopts a motion compensation coding mode for the gently moving areas in the video sequence, eliminates temporal redundancy through motion estimation and compensation, and eliminates spatial redundancy through transformation, quantization and entropy coding. In the encoding process of motion vectors, spatial and temporal correlations are used for prediction, and the order of Hadamard transform is adaptively set according to the probability distribution characteristics of the prediction residual to improve the transformation efficiency. CAVLC is used to quantize and encode the transform coefficients, and the horizontal and vertical components of the motion vector are encapsulated and multiplexed. This scheme makes full use of motion information and residual information, combined with adaptive transformation and context coding, effectively improves the coding efficiency of gently moving areas and reduces the bit rate while ensuring the reconstruction quality.

[0076] S5, after obtaining the video frame data, for the third area divided into the fast motion area, the present application adopts the intra-frame prediction coding mode for coding: S51, according to the encoded blocks adjacent to the current block, calculate the spatial prediction direction and prediction mode. Calculate the grayscale mean and variance of the encoded block on the left side of the current block. The grayscale mean is obtained by averaging the pixel values ​​of the adjacent blocks on the left; the variance is obtained by calculating the sum of the squares of the differences between the pixel values ​​and the mean. The grayscale mean reflects the brightness information of the left block, and the variance reflects the texture complexity of the left block. Calculate the grayscale mean and variance of the encoded block above the current block. Similar to the left block, the grayscale mean and variance are obtained by averaging the pixel values ​​of the upper adjacent blocks, reflecting the brightness and texture characteristics of the upper block. According to the comparison of the size of the grayscale mean and variance, determine whether the spatial prediction direction is horizontal or vertical. By comparing the grayscale mean and variance of the left block and the upper block, it is determined in which direction the current block is more correlated with the adjacent block. If the grayscale mean and variance of the left block are closer to the current block, the horizontal direction is selected as the spatial prediction direction; otherwise, the vertical direction is selected. In the determined spatial prediction direction, the optimal prediction mode is selected from multiple prediction modes based on the grayscale value distribution characteristics of the adjacent encoded blocks. For the adjacent blocks in the selected direction, the distribution characteristics of their grayscale values, such as gradient direction, edge structure, etc., are analyzed, and the mode that best matches the current block is selected from the predefined multiple prediction modes as the optimal prediction mode. Common intra-frame prediction modes include DC mode, plane mode, angle mode, etc.

[0077] S52, based on the spatial prediction direction and the prediction mode, a prediction block of the current block is generated. If the spatial prediction direction is horizontal, the boundary pixel value of the coded block on the left side of the current block is used to generate the prediction block according to the selected prediction mode. The right boundary pixel of the left block is used as a reference, and the prediction value of the current block is generated according to the interpolation rule defined in the prediction mode. If the spatial prediction direction is vertical, the boundary pixel value of the coded block above the current block is used to generate the prediction block according to the selected prediction mode. The bottom boundary pixel of the upper block is used as a reference, and the prediction value of the current block is generated according to the interpolation rule defined in the prediction mode.

[0078] S53, calculate the residual between the current block and the predicted block to obtain a residual block, and perform point-by-point difference between the pixel values ​​of the current block and the pixel values ​​of the predicted block to obtain a difference matrix. The residual reflects the degree of difference between the current block and the predicted block. The difference matrix is ​​weighted quantized to obtain a quantized residual block. The residual matrix is ​​quantized to an integer value by multiplying by the weight matrix and dividing by the quantization step size to reduce the data range.

[0079] S54, transform, quantize and entropy encode the residual block to obtain third coded data, and perform DCT transform on the quantized residual block. Divide the quantized residual block into several 4×4 sub-blocks, apply a 4×4 DCT transform matrix to each sub-block, and convert the spatial domain residual into frequency domain coefficients. DCT transform has the characteristic of concentrated energy, and can compress the residual information into low-frequency coefficients. Scale and truncate the transform coefficient matrix. By multiplying by the scale factor and truncating to an integer, the DCT coefficient is corrected so that its dynamic range is adapted to the quantizer. Quantize the scaled transform coefficient matrix. According to the characteristics of the intra-frame coding mode, the range and step size of the quantization parameter QP are preset, and the transform coefficient is quantized to an integer value by dividing by the quantization step size and rounding to achieve data compression. Entropy encoding is performed on the quantized coefficient matrix. The quantized coefficient matrix is ​​converted into a coefficient sequence by zigzag scanning, and then the coefficient sequence is run-length encoded and equal-length encoded, and finally compressed and encoded by CABAC (context adaptive binary arithmetic coding) entropy coding technology to obtain the third coded data.

[0080] This application targets fast-moving areas in video sequences, adopts an intra-frame prediction coding mode, fully utilizes spatial correlation, and achieves data compression by combining prediction and residual coding. During the prediction process, the spatial prediction direction is selected based on the grayscale mean and variance of adjacent coded blocks, and the optimal mode is selected from multiple prediction modes to improve prediction accuracy. During the residual coding process, the residual data is further compressed through a series of operations such as DCT transformation, scale scaling, quantization, and entropy coding. This scheme comprehensively considers the characteristics of fast-moving areas, and through adaptive prediction direction and mode selection, as well as efficient transformation and entropy coding technology, it significantly reduces the bit rate of the third area while ensuring the reconstruction quality, thereby improving the coding efficiency.

[0081] like Figure 5 As shown, S6, after obtaining the motion vector of the second region, in order to further compress the data volume, the present application encodes the motion vector to obtain fourth encoded data: S61, according to the horizontal motion vector component of the second region and the vertical motion vector component , respectively select the encoded motion vectors of the following areas as prediction values, and calculate the weighted average as the comprehensive prediction value: select the encoded motion vectors of the spatially adjacent blocks on the left, above, upper right, and lower left of the current block as the spatial prediction values. Using the motion vector information of the spatially adjacent blocks, the motion trend of the area around the current block can be captured to improve the prediction accuracy. Select the encoded motion vector of the co-located block of the current block in the temporal adjacent frames such as the previous frame and the previous two frames as the temporal prediction value. Using the motion vector information of the temporal adjacent frames, the motion continuity of the current block on the time axis can be captured to improve the prediction accuracy. Based on the spatial prediction value and the temporal prediction value, calculate the weighted average as and The weight coefficient is adaptively determined according to the temporal and spatial distances of adjacent blocks and adjacent frames. Combining the spatial domain prediction value and the temporal domain prediction value, we can get and The prediction accuracy is expected to be further improved. At the same time, the adaptive weighted average strategy is adopted to dynamically adjust the weight coefficient according to the temporal and spatial distances between adjacent blocks and adjacent frames, so that the prediction value can be more in line with the actual motion trend of the current block.

[0082] S62, respectively calculating the horizontal motion vector components and the vertical motion vector component The difference between its comprehensive prediction value is used to obtain the motion vector prediction residual and By taking the difference, the redundant information of the motion vector can be eliminated and the coding overhead can be reduced. Specifically, the horizontal motion vector component is calculated Its comprehensive prediction value The difference between the horizontal motion vector prediction residual is obtained. . ,in, Represents the horizontal motion vector component of the current block, Represents the comprehensive prediction value calculated based on the motion vectors of adjacent blocks in the spatial and temporal domains. By taking the difference, the Zhongyu The same parts, only keep Unique information, namely the residual This can remove the redundancy of motion vectors in the horizontal direction and reduce the amount of data that needs to be encoded.

[0083] Calculate the vertical motion vector component Its comprehensive prediction value The difference between the vertical motion vector prediction residual is obtained. . ,in, Represents the vertical motion vector component of the current block, Represents the comprehensive prediction value calculated based on the motion vectors of adjacent blocks in the spatial and temporal domains. Similar to the horizontal direction, by taking the difference, the Zhongyu The same parts, only keep Unique information, namely the residual This can remove the redundancy of the motion vector in the vertical direction and reduce the amount of data that needs to be encoded. and As the object of subsequent encoding, replace the original motion vector component and .

[0084] Through and Making a prediction and calculating the residual between it and the predicted value can significantly reduce the amplitude and dynamic range of the data to be encoded. and Contains only and The unpredicted part of , usually has a smaller value and fewer non-zero values, and is more suitable for transform and entropy coding. and Subsequent transformation, quantization and entropy coding are performed to generate the final motion vector coding data. and Transformation (such as Hadamard transformation) can further improve the energy concentration of the residual and compress the information into a few transformation coefficients. By quantizing the transformed coefficients, the coefficients can be mapped to a limited quantization level to achieve data compression. By entropy coding the quantized coefficients (such as CAVLC), the optimal coding codeword can be assigned according to the probability distribution of the coefficients to further compress the data. In the motion vector coding process of the present application, the motion vector prediction residual is obtained by calculating the difference between the motion vector component and its comprehensive prediction value, and the residual is used to replace the original motion vector component for subsequent coding. This approach can make full use of the correlation of the motion vector in the time domain and the spatial domain, eliminate redundant information through prediction, and only encode the residual, thereby significantly reducing the amount of encoded data and reducing the bit rate overhead. At the same time, the residual usually has better statistical characteristics, which is more conducive to transformation and entropy coding, and further improves the compression efficiency.

[0085] S63, respectively calculating motion vector prediction residuals and The numerical range histogram and the sign distribution histogram of are obtained, and the following processing is performed to obtain the probability distribution characteristics: the numerical range histogram and the sign distribution histogram are normalized to eliminate the scale difference. Normalization processing can map the histograms of different residual samples to the same scale, which is convenient for subsequent probability distribution analysis. Statistical characteristic parameters such as mean, variance, skewness, and kurtosis are extracted from the normalized histogram to characterize and The probability distribution characteristics of the residuals. The value range histogram reflects the distribution of the residual amplitude, and the sign distribution histogram reflects the distribution of the positive and negative signs of the residuals. By extracting these statistical features, the probability distribution law of the residuals can be fully described. S64, according to and The probability distribution characteristics of the horizontal and vertical transforms are set by setting the Hadamard transform order, and the motion vector prediction residual is calculated by using the two-dimensional Hadamard transform of the set order. and Transform and get and The corresponding transform coefficient matrix.

[0086] Specifically, according to The probability distribution characteristics of the horizontal transformation are adaptively set to the Hadamard transformation order. .analyze The probability distribution of the residual samples is calculated, and the statistical characteristic parameters such as mean, variance, skewness, and kurtosis are calculated to characterize the concentration and dispersion of the residual samples. When the probability distribution of the residual samples is more concentrated (such as the mean is close to zero, the variance is small, the skewness and kurtosis are large), it means that the residual energy is mainly concentrated on a few values. At this time, the transformation order can be reduced accordingly. A lower transform order is sufficient to capture the main features of the residual, reducing computational complexity and coding overhead. When the probability distribution of the residual samples is more dispersed (such as the mean deviates from zero, the variance is larger, and the skewness and kurtosis are smaller), it means that the residual energy is more evenly distributed on multiple values. In this case, the transform order needs to be appropriately increased. A higher transform order can better match the sparse characteristics of the residual transform domain, improve transform efficiency and compression performance. By setting appropriate thresholds and mapping relationships, the optimal horizontal transform order is adaptively determined based on the statistical characteristic parameters of the residual distribution. .

[0087] according to The probability distribution characteristics of the vertical transformation are adaptively set to the Hadamard transform order. Similar to the horizontal transformation, the analysis The probability distribution of the residual sample is calculated, and its statistical characteristic parameters such as mean, variance, skewness, and kurtosis are calculated to characterize the concentration and dispersion of the residual sample. According to the statistical characteristic parameters of the residual distribution, the optimal vertical transformation order is adaptively determined. The order of vertical transformation is the same as that of horizontal transformation. When the probability distribution of residual samples is more concentrated, the corresponding ; When the probability distribution of residual samples is more dispersed, appropriately increase By setting appropriate thresholds and mapping relationships, the optimal vertical transformation order is adaptively determined according to the statistical characteristic parameters of the residual distribution. .

[0088] use The one-dimensional Hadamard transform of order is used to transform resx horizontally, using The one-dimensional Hadamard transform of Performing a vertical transformation, we get and The two-dimensional transformation coefficient matrix of conduct One-dimensional row transform of order. According to The value of , select the Hadamard transformation matrix of the corresponding order, Transform each row of to obtain the transformed row vector. conduct One-dimensional column transformation of order. According to The value of , select the Hadamard transformation matrix of the corresponding order, Transform each column of to obtain the transformed column vector. Combine the transformed row vector and column vector into a two-dimensional matrix to obtain and The transformation coefficient matrix after the two-dimensional separable Hadamard transformation.

[0089] Hadamard transform is an orthogonal transform based on rectangular wave function. The coefficients of the transform matrix are all ±1. The transform process only involves addition and subtraction operations, which has the advantages of fast, reversible, and integer precision. Compared with other transforms such as DCT and DWT, Hadamard transform has lower computational complexity and is more suitable for hardware implementation. The transformed coefficient matrix is ​​used as the object of subsequent quantization and entropy coding to generate the final motion vector coded data. The transform coefficient matrix is ​​quantized to map the transform coefficients to a limited quantization level to achieve data compression. The quantization parameter can be adaptively adjusted according to the rate-distortion optimization criterion. The quantized coefficient matrix is ​​entropy coded, such as CAVLC or CABAC, and the optimal coding codeword is assigned according to the probability distribution of the coefficients to further compress the data. The bit stream data obtained by entropy coding is packaged and encapsulated to form the final motion vector coded data for transmission or storage.

[0090] The application proposes an adaptive Hadamard transform method for motion vector prediction residuals resx and resy. By analyzing the probability distribution characteristics of the residuals, the Hadamard transform orders of the horizontal transform and the vertical transform are adaptively set, and the residuals are subjected to a two-dimensional separable Hadamard transform using the set order to obtain a transform coefficient matrix. The method can dynamically adjust the transform order according to the statistical characteristics of the residuals, reduce the order when the residual distribution is concentrated, and reduce the computational overhead; increase the order when the residual distribution is dispersed to improve the transform efficiency. At the same time, the combination of one-dimensional row transform and one-dimensional column transform is adopted to achieve fast decomposition of the two-dimensional transform and reduce the computational complexity. The Hadamard transform uses ±1 as the coefficient and only involves addition and subtraction operations, which has the advantages of hardware friendliness and integer precision. The transformed coefficient matrix is ​​quantized and entropy encoded to generate the final motion vector encoding data. The scheme fully matches the probability distribution characteristics of the residuals through adaptive transformation, and combines quantization and entropy coding to significantly improve the compression coding efficiency of the motion vector.

[0091] S65, in the motion vector prediction residual and After performing the adaptive Hadamard transform, the corresponding transform coefficient matrix is ​​obtained. In order to further compress the data, this application constructs multiple CAVLC (context adaptive variable length coding) coding tables and performs a series of coding processes on the transform coefficient matrix to finally generate a motion vector quantization coefficient stream and .

[0092] The transform coefficient matrix is ​​scanned in a zigzag pattern to convert the two-dimensional matrix into a one-dimensional coefficient sequence. Starting from the upper left corner of the transform coefficient matrix, each coefficient in the matrix is ​​scanned in turn along a zigzag path to form a one-dimensional coefficient sequence. The zigzag scanning path is as follows: first scan a coefficient to the right, then scan a coefficient to the lower left, then scan a coefficient to the upper right, and then continue to scan a coefficient to the right, alternating in this way until the entire matrix is ​​scanned. Zigzag scanning can improve the spatial correlation of transform coefficients. Due to the energy concentration characteristic of the Hadamard transform, the transformed coefficients usually have a larger amplitude near the upper left corner of the matrix and are close to zero near the lower right corner. Zigzag scanning can gather these correlated coefficients together to form a continuous coefficient sequence, which is beneficial to subsequent entropy coding.

[0093] Run-length coding is performed on the coefficient sequence to obtain the run length and non-zero coefficient level. Traverse the one-dimensional coefficient sequence, count the number of consecutive zero-valued coefficients, and obtain the run length (Run Length). The run length represents the number of zero-valued coefficients between two non-zero coefficients. For each non-zero coefficient, the corresponding level (Level) is determined according to its numerical value. The level of a non-zero coefficient represents the interval range to which its absolute value belongs. The run length and the non-zero coefficient level are combined into a (run length, level) pair to form a run-coded symbol stream. Run-length coding can make full use of the sparsity of the transform coefficients. After the Hadamard transform, the values ​​of most coefficients will tend to zero, and only a few coefficients have significant non-zero values. Run-length coding can greatly reduce the number of symbols that need to be encoded and reduce coding redundancy by recording the number of consecutive zero-valued coefficients.

[0094] According to the joint probability distribution of run length and non-zero coefficient level, entropy coding is performed using the constructed CAVLC coding table to obtain the quantized coefficient streams of mvx and mvy. According to the statistical distribution of run length and non-zero coefficient level in the transform coefficients, multiple CAVLC coding tables are pre-constructed. Each coding table provides the optimal encoding codeword for different run lengths and level ranges. For each (run length, level) pair, the corresponding encoding codeword is searched in the corresponding CAVLC coding table according to the run length and level range to which it belongs, and output to the quantized coefficient stream. CAVLC is an adaptive variable length coding technology. It dynamically adjusts the coding table by statistically analyzing the probability distribution of transform coefficients, so that high-probability symbols obtain shorter encoding codewords, while low-probability symbols obtain longer encoding codewords, thereby optimizing entropy coding.

[0095] By constructing multiple coding tables, CAVLC can adapt to transform coefficients with different distribution characteristics, select the most matching coding mode, improve entropy coding efficiency, and reduce coding overhead. and The transform coefficient matrix is ​​subjected to the above encoding process respectively, and finally the corresponding motion vector quantization coefficient stream is obtained. and .right The transform coefficient matrix is ​​subjected to zigzag scanning, run-length coding and CAVLC entropy coding to generate a horizontal motion vector quantization coefficient stream .right The transform coefficient matrix is ​​subjected to zigzag scanning, run-length coding and CAVLC entropy coding to generate a vertical motion vector quantization coefficient stream . and The compressed data representing the horizontal motion vector and vertical motion vector after quantization and entropy coding can be used for transmission or storage.

[0096] This application uses CAVLC coding technology to efficiently compress the transform coefficient matrix after performing adaptive Hadamard transform on the motion vector prediction residual. The spatial correlation of the transform coefficients is improved by zigzag scanning, the sparsity of the transform coefficients is fully utilized by run-length coding, and multiple CAVLC coding tables are constructed to match different probability distribution characteristics, thereby achieving optimal entropy coding of the motion vector quantization coefficient stream. This scheme comprehensively considers the characteristics of transformation, quantization and entropy coding, and through adaptive coding mode selection, it significantly improves the compression efficiency of the motion vector and reduces the transmission and storage overhead.

[0097] S66, encapsulate the motion vector quantization coefficient stream to obtain fourth coded data: encapsulate the motion vector quantization coefficient stream, add necessary header information, tail information and synchronization information to ensure data integrity and synchronization. Multiplex the encapsulated motion vector quantization coefficient stream using time division multiplexing or code division multiplexing to obtain fourth coded data. The multiplexing operation can package data of different motion vector quantization coefficient streams together, improve transmission efficiency, and save bit rate overhead.

[0098] This application uses a series of coding technologies for the motion vectors of the second area, including motion vector prediction, residual calculation, probability distribution analysis, adaptive Hadamard transform, CAVLC entropy coding and data encapsulation, etc. By fully exploring the spatiotemporal correlation and probability distribution characteristics of the motion vectors, the transform order and coding mode are adaptively adjusted, and the data volume of the motion vectors is effectively compressed, generating efficient and compact fourth coded data. At the same time, the multiplexing technology is used to package the quantization coefficient stream, further improving the transmission efficiency and reducing the bit rate overhead. While ensuring the quality of motion vector coding, this solution significantly reduces the data volume of the code stream and improves the compression effect.

[0099] S7, performing bit rate control and bit stream encapsulation according to the first coded data, the second coded data and the third coded data to obtain a video bit stream, including: respectively counting the number of non-zero transform coefficients in the first coded data, the second coded data and the third coded data as the first coded data complexity, the second coded data complexity and the third coded data complexity; setting weight coefficients corresponding to the first coded data complexity, the second coded data complexity and the third coded data complexity; multiplying the first coded data complexity, the second coded data complexity and the third coded data complexity with the corresponding weight coefficients to obtain weighted complexity; adding each weighted complexity to obtain the overall complexity of the video frame;

[0100] Adaptively allocate the encoding bit rate according to the overall complexity of the video frame: when the overall complexity of the video frame is greater than a threshold, increase the encoding bit rate accordingly: calculate the increase in the encoding bit rate, which is proportional to the degree to which the overall complexity of the video frame exceeds the threshold; based on the original encoding bit rate, multiply it by a proportional coefficient greater than 1 according to the increase, to obtain the increased encoding bit rate;

[0101] When the overall complexity of the video frame is less than or equal to the threshold, the encoding bit rate is reduced accordingly: the reduction in the encoding bit rate is calculated, and the reduction is proportional to the degree to which the overall complexity of the video frame is lower than the threshold; based on the original encoding bit rate, the reduced encoding bit rate is obtained by multiplying it by a proportional coefficient less than 1 according to the reduction.

[0102] In the process of bit rate control, corresponding weight coefficients are set according to the visual importance of the first region, the second region and the third region; the encoding bit rate is multiplied by the weight coefficient of each region to obtain the target bit rate of each region; the rate distortion optimization model is used to adjust the quantization parameter under the constraint of the target bit rate of each region to minimize the encoding distortion, so as to obtain the first encoded data, the second encoded data and the third encoded data after bit rate control;

[0103] The first coded data, the second coded data and the third coded data after bit rate control are encapsulated into a bit stream: the frame header information of the video frame is set, including the frame number, the frame type, the quantization parameter, the motion vector accuracy, etc.; the header information of the video sequence is set, including the video resolution, the frame rate, the encoding tool setting, etc.; the trailer information of the video sequence is set, including the CRC checksum, etc.; according to the grammatical rules of the video encoding standard, the first coded data, the second coded data, the third coded data after bit rate control and the corresponding frame header information, the header information and the trailer information are merged to obtain a video code stream.

[0104] S8, merging the video code stream and the fourth coded data to obtain the coded video transmission data, and transmitting the coded video transmission data, specifically including the following steps: segmenting the video code stream to obtain multiple slices of the video code stream; segmenting the fourth coded data to obtain multiple fragments of the fourth coded data; wherein the segmentation method of the fourth coded data corresponds to the segmentation method of the video code stream, so that each video code stream slice corresponds to a fourth coded data fragment; creating a merged transmission unit, the transmission unit includes three parts: a synchronization header, a payload, and a check tail; segmenting the video code stream and the fourth coded data respectively. The segmentation granularity of the video code stream can be set according to the actual transmission requirements and network conditions. Common segmentation units include frames, slices, blocks, etc. The segmentation method of the fourth coded data needs to correspond to the segmentation method of the video code stream to ensure that each video code stream slice can find the corresponding fourth coded data fragment, which is convenient for subsequent merging and encapsulation. The synchronization header is used to identify the starting position and transmission parameters of the transmission unit, the payload is used to carry the video code stream slice and the fourth coded data fragment, and the check tail is used to detect and correct transmission errors.

[0105] Each video code stream slice and the corresponding fourth coded data segment are combined and encapsulated into the payload of a transmission unit: at the beginning of the payload, the length information of the video code stream slice and the fourth coded data segment is added; after the length information, the content of the video code stream slice and the fourth coded data segment is added in sequence; in order to facilitate the parsing of the receiving end, the length information is added at the beginning of the payload to indicate the length of the video code stream slice and the fourth coded data segment, and then their contents are stored in sequence. This length + data encapsulation format facilitates the receiving end to accurately extract and restore the video code stream and the fourth coded data.

[0106] In the synchronization header of the transmission unit, the sequence number, priority and timestamp of the transmission unit are added; the timestamp is used to indicate the decoding time and presentation time of the transmission unit; in the check tail of the transmission unit, a cyclic redundancy check (CRC) code is added to detect the integrity and correctness of the transmission unit;

[0107] According to the priority and timestamp of the transmission unit, multiple transmission units are sorted and scheduled to obtain encoded video transmission data; the encoded video transmission data is packaged into data packets through the network transmission protocol and sent to the receiving end; at the receiving end, the received data packets are reassembled and parsed according to the sequence number and timestamp of the transmission unit to obtain video code stream slices and the corresponding fourth encoded data fragment, and subsequent decoding and playback processing are performed.

Claims

1. A method for transmitting audio and video media, characterized in that: include: S1, obtain video frame data; S2, performing motion analysis on the video frame data to obtain a first region, a second region, and a third region; the first region represents a background region, the second region represents a smooth motion region, and the third region represents a fast motion region; S3, encoding the video frame data of the first region in an inter-frame prediction encoding mode to obtain first encoded data; S4, encoding the video frame data of the second region using a motion compensation encoding mode to obtain second encoded data and a motion vector of the second region; S5, encoding the video frame data of the third region by using an intra-frame prediction encoding mode to obtain third encoded data; S6, encoding the motion vector of the second area to obtain fourth encoded data; S7, performing bit rate control and bit stream encapsulation according to the first coded data, the second coded data and the third coded data to obtain a video bit stream; S8, combining the video code stream and the fourth coded data to obtain coded video transmission data, and transmitting the coded video transmission data.

2. The method for transmitting audio and video media according to claim 1, characterized in that: S2, performing motion analysis on the video frame data to obtain a first region, a second region, and a third region, including: S21, dividing the video frame data into a plurality of blocks; S22, searching the reference frame for the block with the highest matching degree with the current block by minimum absolute difference, and obtaining a candidate motion vector; S23, selecting the best motion vector from the candidate motion vectors ; S24, dividing the corresponding blocks whose optimal operation vector is smaller than the first threshold into a first area, indicating a background area; dividing corresponding blocks whose optimal motion vectors are greater than a first threshold and less than a second threshold into a second region, indicating a smooth motion region; The corresponding blocks whose optimal motion vectors are greater than the second threshold are divided into a third area, indicating a fast motion area.

3. The method for transmitting audio and video media according to claim 2, characterized in that: S23, selecting the best motion vector from the candidate motion vectors ,include: Based on each candidate motion vector, calculate the distortion metric between the corresponding motion compensated prediction block and the current block ; Based on each candidate motion vector, calculate the corresponding encoding bit rate : ， in, Indicates the coding bit rate of the motion vector, which is related to the probability distribution of the motion vector and the entropy coding method; Indicates the coding bit rate of the motion compensation residual, which is related to the energy size of the residual and the transform coding method; Constructing rate-distortion optimization objective function : ; Among them, λ is the Lagrange multiplier, which controls the weight distribution of rate-distortion optimization; Solve the minimum value of the rate-distortion optimization objective function and obtain the optimal motion vector : ， in, Indicates that among all candidate motion vectors, select the one that makes the objective function The motion vector that reaches the minimum value is taken as the optimal motion vector.

4. The audio and video media transmission method according to claim 3, characterized in that: Calculate the distortion metric between the corresponding motion compensated prediction block and the current block , through the following formula: ; in, Indicates is the distortion measure between the candidate prediction block of the motion vector and the current block, represents the pixel value of the current block at pixel position (x, y), Indicates The candidate prediction block for the motion vector is at pixel position The pixel value at Represents the weight factor of the current block at the pixel position (x, y), which is used to reflect the contribution of the pixel at this position to the distortion metric.

5. The audio and video media transmission method according to claim 2, characterized in that: S3, encoding the video frame data of the first region in an inter-frame prediction encoding mode to obtain first encoded data; S31, selecting one or more previously encoded frames as reference frames in a reference frame buffer, wherein the reference frame buffer is used to store a plurality of encoded frame reconstructed images; S32, searching the reference frame for a block that best matches the current block as a reference block of the current block; and calculating a residual between the current block and the reference block to obtain a residual block; S33, performing DST transformation on the residual block to obtain a transformation coefficient matrix; S34, quantizing the transform coefficient matrix using a quantization parameter QP to obtain a quantization coefficient matrix; S35, vertically scanning the quantization coefficient matrix to obtain a coefficient symbol sequence; S36, performing run-length coding on the coefficient symbol sequence to obtain a sequence of run-length pairs; S37, entropy encoding is performed on the sequence of run-length pairs, and a bit stream obtained by entropy encoding is used as first encoded data.

6. The method for transmitting audio and video media according to claim 2, wherein: S4, encoding the video frame data of the second area using a motion compensation encoding mode, including: S41, using motion estimation, searching for a block that best matches the current block in the reference frame to obtain a motion vector; S42, using motion compensation, generating a prediction block at a corresponding position in the reference frame according to the motion vector; S43, calculating the residual between the current block and the predicted block to obtain a residual block; S44, transforming, quantizing and entropy encoding the residual block to obtain second encoded data; S45, performing entropy coding on the motion vector to obtain the motion vector of the second region.

7. The method for transmitting audio and video media according to claim 6, characterized in that: S45, performing entropy coding on the motion vector to obtain the motion vector of the second region, including: Filter the motion vector to get the horizontal component and the vertical component ; Exploiting spatial and temporal correlations, predictors are selected from coded motion vectors of neighboring blocks and frames; Calculate the horizontal component separately and the vertical component The difference between the predicted value and the predicted value is the prediction residual and ; Calculate prediction residuals and The statistical histogram of , and the corresponding probability distribution characteristics are obtained; According to the probability distribution characteristics, the order of Hadamard transform is set, and the prediction residual is calculated using the Hadamard transform after the order is set. and Perform a two-dimensional transformation to obtain a transformation coefficient matrix; Quantizing and encoding the transform coefficient matrix using context adaptive variable length coding CAVLC to obtain a quantized coefficient stream; The quantized coefficient stream is encapsulated and multiplexed to obtain a motion vector of the second region.

8. The method for transmitting audio and video media according to claim 7, characterized in that: S6, encoding the motion vector of the second area to obtain fourth encoded data, including: S61, based on the horizontal motion vector component of the second region and the vertical motion vector component , respectively select the encoded motion vectors of adjacent blocks and adjacent frames as prediction values, and calculate the weighted average value as the comprehensive prediction value; S62, respectively calculating the horizontal motion vector components and the vertical motion vector component , and the difference between the corresponding comprehensive prediction value, to obtain the motion vector prediction residual and ; S63, respectively calculating motion vector prediction residuals and A value range histogram and a symbol distribution histogram are obtained, and a probability distribution feature is obtained according to the value range histogram and the symbol distribution histogram; S64, according to and The probability distribution characteristics of the horizontal and vertical transforms are set by setting the Hadamard transform order, and the motion vector prediction residual is calculated by using the Hadamard transform after setting the order. and Transform and get and The transformation coefficient matrix of S65, construct multiple CAVLC coding tables for and The transform coefficient matrix is ​​entropy encoded to generate a motion vector quantization coefficient stream; S66, encapsulating the motion vector quantization coefficient stream and obtaining fourth encoded data through multiplexing.

9. The audio and video media transmission method according to claim 2, characterized in that: S5, encoding the video frame data of the third region using an intra-frame prediction encoding mode, including: S51, calculating a spatial prediction direction and a prediction mode according to an encoded block adjacent to the current block; S52, generating a prediction block of the current block according to the spatial prediction direction and the prediction mode; S53, calculating the residual between the current block and the predicted block to obtain a residual block; S54, transform, quantize and entropy encode the residual block to obtain third encoded data.

10. An audio and video media transmission system, characterized in that: include: At least one processing unit; used to execute instructions to implement the audio and video media transmission method described in any one of claims 1 to 9.

Images