Audio and video media transmission method and system

Abstract

The present application discloses an audio and video media transmission method and system, which relates to the field of data transmission, including: obtaining video frame data; performing motion analysis on the video frame data to obtain a first region, a second region, and a third region; encoding the video frame data in the first region using an inter-frame prediction coding mode; encoding the video frame data in the second region using a motion compensation coding mode; encoding the video frame data in the third region using an intra-frame prediction coding mode; encoding the motion vector of the second region to obtain fourth encoded data; performing rate control and bitstream encapsulation based on the first, second, and third encoded data to obtain a video stream; merging the video stream and the fourth encoded data to obtain encoded video transmission data, and transmitting the encoded video transmission data. In view of the inflexibility of rate control in video encoding in the prior art, the present application improves the flexibility of rate control.

Description

Technical Field

[0001] The present application relates to the field of data transmission, and in particular to a method and system for transmitting audio and video media. Background Art

[0002] With the increasing prevalence of video applications and the diversification of network environments, video encoding and transmission technologies are facing new challenges and demands. In areas such as video communications, video surveillance, and interactive entertainment, users are placing higher demands on video quality, real-time performance, and smoothness. However, due to limitations in network bandwidth, device capabilities, and usage scenarios, achieving efficient, flexible, and adaptable video encoding and transmission in complex and changing environments has become a critical issue that needs to be addressed.

[0003] Traditional video coding schemes, such as H.264 / AVC and H.265 / HEVC, primarily employ a block-based hybrid coding framework, utilizing modules such as intra-frame prediction, inter-frame prediction, transform quantization, and entropy coding to achieve compressed representation of video sequences. While these schemes have achieved significant progress in coding efficiency and reconstruction quality, they also have limitations. First, existing schemes lack adaptability to video content and fail to fully consider the characteristics of different video scenes, making it difficult to tailor coding parameters and strategies. Second, motion estimation and compensation, as key components, require high computational and storage resources, hindering the real-time performance of encoding. Furthermore, relatively simple rate control strategies lack flexible bit allocation and priority control mechanisms, making them difficult to adapt to dynamically changing network conditions and user needs.

[0004] Specifically, in applications like video surveillance and drone aerial photography, video footage often includes areas with varying characteristics, such as static backgrounds, slow-moving objects, and fast-moving objects. Traditional coding schemes employ uniform coding modes and parameters, making it difficult to balance coding efficiency and reconstruction quality across all regions. This can lead to problems such as loss of detail in background areas and noticeable blurring and noise in moving areas. In real-time interactive scenarios like video conferencing and distance education, network bandwidth and latency often fluctuate significantly. Fixed bitrate control strategies cannot dynamically adjust encoding parameters and transmission strategies based on network conditions, easily causing video quality degradation such as freezes, frame drops, and blurring, impacting the user experience.

[0005] Furthermore, with the development of new technologies such as 5G and the Internet of Things, video applications are becoming increasingly diverse and personalized. Different users and devices have varying preferences and perceptions for video content. Traditional, one-size-fits-all encoding solutions lack specificity and flexibility, failing to meet differentiated service needs. For example, in live sports broadcasts, different users may be interested in different areas, such as players, the field, and the audience. Fixed encoding strategies struggle to maintain a balanced level of clarity and smoothness across all areas, impacting the user experience. Summary of the Invention

[0006] In response to the problem of inflexible bit rate control in video encoding in the prior art, the present application provides an audio and video media transmission method and system, which divides the video content into background area, smooth motion area and fast motion area according to motion characteristics, and adopts adaptive coding modes for different areas, using inter-frame prediction coding for the background area, motion compensation coding for the smooth motion area, and intra-frame prediction coding for the fast motion area, and separately encodes the motion vector of the smooth motion area; finally, the encoded data and motion vector data of different areas are merged and transmitted, thereby improving the flexibility of bit rate control.

[0007] The purpose of this application is achieved through the following technical solutions.

[0008] One aspect of the present application provides an audio and video media transmission method, including: S1, obtaining video frame data; S2, performing motion analysis on the video frame data to obtain a first area, a second area, and a third area; the first area represents a background area, the second area represents a smooth motion area, and the third area represents a fast motion area; S3, encoding the video frame data of the first area using an inter-frame prediction coding mode to obtain first coded data; S4, encoding the video frame data of the second area using a motion compensation coding mode to obtain second coded data and a motion vector of the second area; S5, encoding the video frame data of the third area using an intra-frame prediction coding mode to obtain third coded data; S6, encoding the motion vector of the second area to obtain fourth coded data; S7, performing bit rate control and bit stream encapsulation according to the first coded data, the second coded data, and the third coded data to obtain a video code stream; S8, merging the video code stream and the fourth coded data to obtain coded video transmission data, and transmitting the coded video transmission data.

[0009] Further, S2, performing motion analysis on the video frame data to obtain the first region, the second region and the third region, including: S21, dividing the video frame data into multiple blocks; S22, searching the reference frame for the block with the highest matching degree with the current block by minimum absolute difference to obtain candidate motion vectors; S23, selecting the optimal motion vector from the candidate motion vectors ; S24, divide the corresponding blocks whose optimal motion vector is less than the first threshold into the first area, representing the background area; divide the corresponding blocks whose optimal motion vector is greater than the first threshold and less than the second threshold into the second area, representing the smooth motion area; divide the corresponding blocks whose optimal motion vector is greater than the second threshold into the third area, representing the fast motion area.

[0010] A reference frame refers to a previously encoded and reconstructed video frame used to predict the current coding block during inter-frame prediction coding. Reference frames are stored in a reference frame buffer and typically contain multiple previously encoded frames preceding the current frame. In this application, reference frame selection can be based on the following criteria: Temporal distance: The previously encoded frame closest in time to the current coded frame is preferred as the reference frame. The closer the temporal distance, the better the motion compensation effect. Frame type: Reference frames are selected based on their coding type (e.g., I-frame, P-frame, B-frame). Generally, I-frames are key frames and can be decoded independently; P-frames reference previous I-frames or P-frames; and B-frames can bidirectionally reference previous and next I-frames or P-frames. Motion characteristics: Reference frames are selected based on the motion characteristics of the video content. For areas with slower motion, reference frames with a longer temporal distance can be selected; for areas with faster motion, reference frames with a closer temporal distance can be selected. Coding efficiency: Reference frames that achieve the best coding efficiency (e.g., the lowest rate-distortion penalty) are selected. The optimal reference frame combination is selected by evaluating the coding efficiency of different reference frames. Multiple reference frames: To improve prediction accuracy, multiple reference frames can be selected for combined prediction. The final prediction block is generated by weighted averaging or adaptively selecting multiple reference frames. Number of reference frames: The number of reference frames is a trade-off between coding efficiency and computational complexity. A greater number of reference frames can improve prediction accuracy, but also increases the search computational complexity. The appropriate number of reference frames is typically determined based on the specific application scenario.

[0011] On the one hand, the second area is a smooth motion area and is encoded using a motion compensation coding mode. In motion compensation coding, the motion vector plays a vital role and is used to describe the position offset between the current block and the predicted block in the reference frame. However, directly transmitting the motion vector will occupy a large bandwidth overhead. Therefore, by separately encoding the motion vector of the second area to obtain the fourth encoded data, the amount of motion vector data can be significantly reduced, and the efficiency of motion compensation coding can be improved. Common motion vector coding methods include predictive coding, entropy coding, etc. By utilizing the correlation of motion vectors in the spatial and temporal domains, the bit rate of the motion vector can be greatly compressed.

[0012] Meanwhile, the video frame data for the second region is encoded using a motion-compensated coding mode to produce second coded data and motion vectors for the second region. The second coded data primarily contains residual information after motion compensation, namely the difference between the current block and the predicted block. The fourth coded data specifically represents the motion vector information for the second region. This separate encoding of motion and residual information facilitates subsequent bitrate control and transmission optimization. For example, the transmission priority and quality level of motion vector and residual data can be flexibly adjusted based on network conditions and receiving end capabilities, achieving optimal reconstruction results within limited bandwidth.

[0013] Finally, encoding and transmitting the video stream and the fourth coded data separately can improve the robustness and adaptability of the transmission. The video stream carries the residual data after encoding each region, corresponding to the rate control and bitstream encapsulation process of S7. The fourth coded data is optimized for the motion vectors of the second region, with higher compression efficiency and flexibility. During actual transmission, the video stream and the fourth coded data can adopt different transmission strategies and protection mechanisms to cope with changes in the network environment. For example, in the case of network congestion or severe packet loss, the integrity of the video stream can be prioritized at the expense of a certain degree of motion vector accuracy. In good network conditions, the transmission quality of the motion vector can be improved to obtain a more accurate motion compensation effect.

[0014] By combining the video bitstream and the fourth coded data for transmission, the receiving end can flexibly select a decoding strategy based on the actual reception situation. Ideally, the receiving end can simultaneously receive and decode the video bitstream and the fourth coded data to achieve a complete video reconstruction effect. However, when errors or packet loss occur during network transmission, the receiving end can take adaptive error concealment measures based on the received data. For example, when the fourth coded data is lost or damaged, the receiving end can use the residual information in the video bitstream and the motion vectors of adjacent blocks to estimate the lost motion vectors through interpolation or prediction, thereby maintaining a certain level of video quality. This fault-tolerant mechanism at the decoding end can effectively reduce the impact of transmission errors on the video experience.

[0015] Further, S23, the optimal motion vector is selected from the candidate motion vectors , including: according to each candidate motion vector, calculating the distortion metric between the corresponding motion compensated prediction block and the current block ; Calculate the corresponding encoding bit rate based on each candidate motion vector : ,in Indicates the encoding bit rate of the motion vector, which is related to the probability distribution of the motion vector and the entropy coding method; The coding bit rate of the motion-compensated residual is related to the residual energy and the transform coding scheme. Coding bit rate estimation is introduced, including the coding bit rate of the motion vector itself and the coding bit rate of the motion-compensated residual. The coding bit rate of the motion vector is related to its probability distribution and the entropy coding scheme, typically using entropy coding techniques such as Exponential Golomb coding or Context-Adaptive Binary Arithmetic Coding (CABAC). The coding bit rate of the motion-compensated residual is also related to the residual energy and the transform coding scheme. The greater the residual energy, the higher the coding bit rate. Using a DCT transform or a wavelet transform can concentrate the residual energy and improve coding efficiency.

[0016] Constructing rate-distortion optimization objective function : , where λ is a Lagrange multiplier that controls the weight distribution for rate-distortion optimization. A rate-distortion optimization objective function is constructed, which weights the distortion metric and the encoding bitrate as a Lagrange multiplier. The Lagrange multiplier λ controls the weight distribution for rate-distortion optimization. A larger λ value favors lowering the encoding bitrate, while a smaller λ value favors lowering the distortion metric. The choice of λ requires adaptive adjustment based on factors such as the encoder's target bitrate and the complexity of the video content to achieve the optimal rate-distortion balance.

[0017] Solve the minimum value of the rate-distortion optimization objective function and obtain the optimal motion vector : ,in, Indicates that among all candidate motion vectors, the objective function is selected The motion vector that achieves the minimum value is considered the optimal motion vector. A rate-distortion optimization solution is presented. Specifically, among all candidate motion vectors, the motion vector that minimizes the objective function is selected as the optimal motion vector. This is a discrete optimization problem that can be solved using methods such as exhaustive search or heuristic search. To reduce computational complexity, fast search algorithms such as three-step search, diamond search, and orthogonal search are often used. These algorithms first locate the approximate range using coarse granularity, then perform local optimization using fine granularity, ultimately obtaining the globally optimal or suboptimal motion vector.

[0018] Further, ,in, Indicates is the distortion measure between the candidate prediction block of the motion vector and the current block, represents the pixel value of the current block at pixel position (x, y), Indicates The candidate prediction block for the motion vector is at pixel position The pixel value at Indicates the weight factor of the current block at the pixel position (x, y), which is used to reflect the contribution of the pixel at this position to the distortion metric. By introducing the weight factor , the distortion metric formula can better adapt to the content characteristics of the video frame and the visual characteristics of the human eye. When calculating the distortion metric between the candidate prediction block and the current block, the contribution of pixels of different positions and types is differentiated, thereby more accurately evaluating the matching degree of the prediction block. Reasonable setting of weight factors helps to improve the accuracy of motion estimation, reduce prediction errors, and ultimately improve the performance and quality of video encoding. For areas with faster motion speeds and larger changes in motion direction, their weights can be appropriately increased, because these areas are more difficult to predict and require more attention; for areas with slower motion speeds and smaller changes in motion direction, their weights can be appropriately reduced, because these areas are relatively simple to predict and less sensitive to distortion.

[0019] Furthermore, S3, the video frame data of the first area is encoded using the inter-frame prediction coding mode to obtain first encoded data; S31, one or more previously encoded frames are selected as reference frames in the reference frame buffer, and the reference frame buffer is used to store the reconstructed images of several frames encoded previously; S32, the block that best matches the current block is searched in the reference frame as the reference block of the current block; the residual between the current block and the reference block is calculated to obtain a residual block; S33, DST (Discrete Sine Transform) is performed on the residual block to obtain a transformation coefficient matrix: the residual block is divided into several sub-blocks, each sub-block has a size of 4×4; a 4×4 DST matrix is ​​used to perform a two-dimensional transformation on each 4×4 sub-block to obtain a corresponding transformation coefficient sub-block; all the transformation coefficient sub-blocks are spliced ​​to obtain a transformation coefficient matrix; S34, the quantization parameter QP (Quantization Parameter) quantizes the transform coefficient matrix to obtain a quantization coefficient matrix: according to a rate-distortion optimization criterion, determines an optimal quantization parameter QP in the inter-frame prediction coding mode; uses the selected quantization parameter QP to quantize each coefficient in the transform coefficient matrix to obtain a quantization coefficient matrix; S35, vertically scans the quantization coefficient matrix to obtain a coefficient symbol sequence; starting from the lower left corner of the quantization coefficient matrix, scans each column of the quantization coefficient matrix in a 45-degree direction from bottom to top and from left to right in a Z-shaped order to obtain a column vector sequence; reverses and transposes the column vector sequence to obtain a coefficient symbol sequence; S36, run-length encodes the coefficient symbol sequence to obtain a sequence of run pairs; counts the non-zero coefficients and their trailing zero coefficients in the coefficient symbol sequence to obtain (run, level) pairs, where the run represents the number of trailing zero coefficients and the level represents the value of the non-zero coefficient; arranges all the (run, level) pairs to form a sequence; S37, entropy encodes the sequence of (run, level) pairs to obtain first encoded data. According to the probability distribution of (run, level) pairs, a run-level joint coding table is constructed; using the run-level joint coding table, a sequence of (run, level) pairs is variable-length coded; and the encoded bit stream is output as the first coded data.

[0020] Furthermore, S4 encodes the video frame data of the second area using a motion compensation coding mode, including: S41, using motion estimation to search for a block that best matches the current block in the reference frame to obtain a motion vector; S42, using motion compensation to generate a prediction block at a corresponding position in the reference frame according to the motion vector; S43, calculating the residual between the current block and the prediction block to obtain a residual block; S44, transforming, quantizing and entropy encoding the residual block to obtain second encoded data; S45, entropy encoding the motion vector to obtain the motion vector of the second area.

[0021] Further, S45, entropy coding is performed on the motion vector to obtain the motion vector of the second region, including: screening the motion vector to obtain the horizontal component and vertical component ; Perform credibility detection on motion vectors and remove unreliable motion vectors; perform redundancy detection on credible motion vectors and remove redundant motion vectors; perform normalization on the filtered motion vectors to obtain the normalized horizontal component and vertical component ; Using spatial and temporal correlation, select prediction values ​​from the coded motion vectors of adjacent blocks and adjacent frames; select the median value as the spatial prediction value among the coded motion vectors of adjacent blocks; select the median value as the temporal prediction value among the coded motion vectors of the co-located blocks of adjacent frames; calculate the weighted average value based on the spatial prediction value and the temporal prediction value as the comprehensive prediction value of the motion vector; calculate the horizontal component separately and vertical component The difference between the predicted value and the predicted value is the prediction residual and ;calculate The difference between the horizontal component of the comprehensive prediction value and the horizontal prediction residual is obtained ;calculate The vertical prediction residual is obtained by dividing the vertical component of the comprehensive prediction value by ; Calculate the prediction residuals and The statistical histogram of , the corresponding probability distribution characteristics are obtained; and Calculate the numerical range histogram respectively and count the residual frequency in each numerical interval; and Calculate the sign distribution histograms respectively and count the frequencies of positive and negative residuals; normalize the value range histogram and the sign distribution histogram to obtain the probability distribution curve; calculate the characteristic parameters such as mean, variance, skewness, and kurtosis according to the probability distribution curve; set the order of Hadamard transform according to the probability distribution characteristics, and use the Hadamard transform after setting the order to predict the residuals. and Perform a two-dimensional transformation to obtain a transformation coefficient matrix; according to The probability distribution characteristics of the horizontal transformation are set to ;according to The probability distribution characteristics of the vertical transformation are set to ;use Order Hadamard transformation matrix pair Perform horizontal transformation and use Order Hadamard transformation matrix pair Perform vertical transformation to obtain the transformation coefficient matrix;

[0022] The transform coefficient matrix is ​​quantized and encoded using context-adaptive variable-length coding (CAVLC) to obtain a quantized coefficient stream. Multiple CAVLC coding tables are constructed, including a Total Zeros table, a Run Before table, and a Level table. The transform coefficient matrix is ​​scanned to obtain a coefficient symbol sequence. The coefficient symbol sequence is run-length encoded to obtain (run, level) pairs. Based on the context information of the (run, level) pairs, the corresponding codeword is searched and output in the corresponding coding table to obtain a quantized coefficient stream.

[0023] The quantized coefficient stream is encapsulated and multiplexed to obtain the motion vector of the second region. and Encapsulate the quantized coefficient stream, add synchronization header and length information, so that the decoder can accurately locate and separate the data of each component; and The quantized coefficient streams are multiplexed to interleave the data of the two components together to form complete motion vector information.

[0024] Further, S5, the video frame data of the third area is encoded using the intra-frame prediction coding mode to obtain third encoded data: S51, based on the encoded blocks adjacent to the current block, the spatial prediction direction and the prediction mode are calculated: the grayscale mean and variance of the encoded blocks on the left side of the current block are calculated; the grayscale mean and variance of the encoded blocks above the current block are calculated; based on the comparison of the grayscale mean and variance, the spatial prediction direction is determined to be the horizontal direction or the vertical direction; in the determined spatial prediction direction, the optimal prediction mode is selected from multiple prediction modes according to the grayscale value distribution characteristics of the adjacent encoded blocks; S52, based on the spatial prediction direction and the prediction mode, a prediction block of the current block is generated: if the spatial prediction direction is In the horizontal direction, the boundary pixel values ​​of the coded blocks on the left side of the current block are used to generate a prediction block according to the selected prediction mode; if the spatial prediction direction is the vertical direction, the boundary pixel values ​​of the coded blocks above the current block are used to generate a prediction block according to the selected prediction mode; S53, calculating the residual between the current block and the prediction block to obtain a residual block: performing point-by-point difference between the pixel values ​​of the current block and the pixel values ​​of the prediction block to obtain a difference matrix; performing weighted quantization on the difference matrix to obtain a quantized residual block; S54, transforming, quantizing and entropy encoding the residual block to obtain third encoded data: performing DCT transform on the quantized residual block: dividing the quantized residual block into a number of 4×4 sub-blocks; performing a two-dimensional DCT transform on each 4×4 sub-block using a 4×4 DCT transform matrix to obtain a transform coefficient matrix; scaling and truncating the transform coefficient matrix to obtain a scaled transform coefficient matrix;

[0025] The scaled transform coefficient matrix is ​​quantized: according to the characteristics of the intra-frame prediction coding mode, the range and step size of the quantization parameter QP are preset; each coefficient in the transform coefficient matrix is ​​quantized using the selected quantization parameter QP to obtain a quantization coefficient matrix; the quantization coefficient matrix is ​​entropy coded: the quantization coefficient matrix is ​​zigzag scanned to obtain a coefficient sequence; the coefficient sequence is run-length coded and equal-length coded; the coding result is CABAC entropy coded to obtain third coded data.

[0026] Further, S6, encoding the motion vector of the second region to obtain fourth encoded data, including: S61, according to the horizontal motion vector component of the second region and vertical motion vector components , respectively select the coded motion vectors of the following areas as prediction values, and calculate the weighted average value as the comprehensive prediction value: select the coded motion vectors of the spatial adjacent blocks such as the left, above, upper right, and lower left of the current block as the spatial prediction value; select the coded motion vectors of the co-located block of the current block in the temporal adjacent frames such as the previous frame and the previous two frames as the temporal prediction value; calculate the weighted average value as the comprehensive prediction value based on the spatial prediction value and the temporal prediction value and The comprehensive prediction value of , where the weight coefficient is adaptively determined according to the spatiotemporal distance between adjacent blocks and adjacent frames; Combining the spatial domain prediction value and the temporal domain prediction value, we can get and The prediction accuracy is expected to be further improved by using the comprehensive prediction value of . At the same time, the adaptive weighted average strategy is adopted to dynamically adjust the weight coefficient according to the temporal and spatial distance between adjacent blocks and adjacent frames, so that the prediction value can be more in line with the actual motion trend of the current block. S62, respectively calculate the horizontal motion vector components and vertical motion vector components The difference between the motion vector prediction residual and its comprehensive prediction value is obtained and ; S63, respectively calculate the motion vector prediction residual and The numerical range histogram and symbol distribution histogram of the value are obtained, and the following processing is performed to obtain its probability distribution characteristics: the numerical range histogram and symbol distribution histogram are normalized to eliminate scale differences; statistical characteristic parameters such as mean, variance, skewness, and kurtosis are extracted from the normalized histogram to characterize and The probability distribution characteristics of the residuals; the numerical range histogram and the sign distribution histogram reflect the distribution characteristics of the residuals from the perspectives of amplitude and positive and negative signs respectively.

[0027] S64, according to and The probability distribution characteristics of the horizontal and vertical transformations are set, and the Hadamard transform orders of the horizontal and vertical transformations are set, and the motion vector prediction residual is calculated using the two-dimensional Hadamard transform of the set order. and Transform and get and The corresponding transformation coefficient matrix: According to The probability distribution characteristics of the horizontal transformation are adaptively set to the Hadamard transformation order. ;according to The probability distribution characteristics of the vertical transformation are adaptively set to the Hadamard transform order. ;use One-dimensional Hadamard transform pair Perform horizontal transformation using One-dimensional Hadamard transform pair Perform vertical transformation to obtain and The two-dimensional transformation coefficient matrix of the residual sample is more concentrated, the transformation order can be reduced accordingly; on the contrary, when the probability distribution of the residual sample is more dispersed, the transformation order needs to be appropriately increased to match the sparse characteristics of the residual transform domain. use One-dimensional row transformation of order use The one-dimensional column transformation of order can be obtained by two-dimensional separable Hadamard transformation. and The corresponding transform coefficient matrix. The Hadamard transform is an orthogonal transform based on a rectangular wave function. The coefficients are all ±1, and the calculation only involves addition and subtraction. It has the advantages of being fast, reversible, and having integer precision.

[0028] S65, construct multiple CAVLC coding tables and and The transformation coefficient matrix is ​​coded as follows to generate a motion vector quantization coefficient stream and : Perform zigzag scanning on the transform coefficient matrix to convert the two-dimensional matrix into a one-dimensional coefficient sequence; perform run-length coding on the coefficient sequence to obtain the run length and non-zero coefficient level; according to the joint probability distribution of the run length and non-zero coefficient level, perform entropy coding using the constructed CAVLC coding table to obtain and quantized coefficient stream; first, the two-dimensional transform coefficient matrix is ​​rearranged into a one-dimensional coefficient sequence through zigzag scanning. Then, the coefficient sequence is run-length coded to obtain two important properties: run length and non-zero coefficient level. The run length represents the number of zero-valued coefficients between two non-zero coefficients, and the non-zero coefficient level represents the numerical value of the non-zero coefficient. Finally, according to the probability distribution of run length and non-zero coefficient level in each dimension, a series of pre-constructed CAVLC coding tables are used to perform joint entropy coding to obtain and The corresponding quantized coefficient stream. The coding table is generated based on the statistical laws of massive training data and can effectively match the probability distribution model of the transform coefficients.

[0029] Among them, the CAVLC coding table (Context-Adaptive Variable Length Coding Table) is an adaptive variable-length coding table for entropy coding, which is mainly used to encode transform coefficients. In this application, multiple CAVLC coding tables are constructed based on the probability distribution characteristics of motion vector prediction residuals. Different coding tables can be designed for different residual distribution characteristics to optimize coding efficiency. By selecting the coding table that best suits the current residual distribution, the number of coding bits can be reduced and compression performance can be improved. The construction of AVLC coding tables is usually based on a large amount of statistical data and offline training. By analyzing the statistical characteristics of transform coefficients, the optimal coding table structure and codeword allocation are designed. In the actual encoding process, the encoder dynamically selects an appropriate coding table based on the context information and residual distribution of the transform coefficients, performs entropy coding on the transform coefficients, and generates the final encoded bit stream.

[0030] S66, encapsulate the motion vector quantization coefficient stream to obtain the fourth encoded data: encapsulate the motion vector quantization coefficient stream, add necessary header information, tail information and synchronization information; multiplex the encapsulated motion vector quantization coefficient stream using time division multiplexing or code division multiplexing to obtain the fourth encoded data; the multiplexing operation can effectively improve the transmission efficiency and save the bit rate overhead.

[0031] Another aspect of the present application also provides an audio and video media transmission system for executing an audio and video media transmission method of the present application.

[0032] Compared with the existing technology, the advantages of this application are:

[0033] The motion analysis process accurately assesses the motion complexity of video content by calculating pixel differences and motion vector magnitudes between adjacent frames. Background areas, areas with slow motion, and areas with fast motion have different spatiotemporal correlations and redundancy characteristics, necessitating different rate control strategies. For example, background areas can be allocated a lower bitrate, areas with slow motion must balance the coding overhead of motion vectors and residuals, and areas with fast motion must ensure the fidelity of texture details. Content-adaptive rate control achieved through motion analysis maximizes the subjective visual quality of each area, improving encoding performance while limiting the overall bitrate.

[0034] Inter-frame prediction coding eliminates redundant information in static backgrounds by using temporal correlations between reference frames. Motion compensation coding eliminates redundant displacements in gently moving objects through motion estimation and compensation. Intra-frame prediction coding effectively reduces the encoding bit rate in rapidly moving areas by leveraging the spatial correlation of pixel values. Selecting the optimal coding mode for different regions minimizes distortion when bitrates are constrained, thereby achieving efficient bit allocation and rate control.

[0035] Motion vector and residual are two key elements in motion compensation coding, representing motion information and prediction error respectively. Traditional coding schemes usually encode motion vector and residual together, making it difficult to achieve flexible bit rate allocation. However, this application adopts a separate coding strategy, which can adjust the bit allocation ratio of motion vector and residual according to actual needs. For example, when transmitting at a low bit rate, the coding accuracy of the motion vector can be appropriately reduced to allocate more bit rate to the residual signal; when transmitting at a high bit rate, the coding accuracy of the motion vector can be improved to obtain a more accurate motion compensation effect. Flexible bit rate control helps to obtain the best visual experience when bandwidth is limited.

[0036] Rate-distortion optimization is a typical motion estimation criterion that seeks the optimal motion vector by weighing the motion vector encoding bit rate and the prediction residual energy. Building on rate-distortion optimization, this application further utilizes the probability distribution characteristics of motion vectors to adaptively set the order of the Hadamard transform and the CAVLC entropy coding table, thereby reducing the coding redundancy of the motion vector. Since motion vectors typically exhibit a centrally symmetric Laplace distribution, targeted design of coding parameters and entropy coding tables can yield a more compact bitstream, reducing coding distortion at the same bit rate and improving motion compensation performance.

[0037] Traditional rate control schemes are usually independent of the encoding process, making it difficult to fully utilize the characteristics of video content, and easily leading to a disconnect between control decisions and the current encoding situation. This application feeds back the motion analysis results to the rate control module, dynamically allocates encoding parameters based on the spatiotemporal characteristics of the background area, the smooth motion area, and the fast motion area, thereby improving the content adaptability of the rate control; at the same time, the encoding end and the transmission end collaborate in optimization, and through the hierarchical transmission and scheduling of the video bitstream and motion vector, the decoding quality of important information is maximized in the event of network fluctuations and packet loss. Closed-loop control and dynamic optimization strategies effectively improve the robustness and real-time performance of the system, meeting the requirements of low-latency, high-quality transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The present application will be further described in the form of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting, and in these embodiments, the same numbers represent the same structures, wherein:

[0039] Figure 1 is an exemplary flow chart of an audio and video media transmission method according to some embodiments of the present application;

[0040] Figure 2 is an exemplary flowchart of dividing a video frame according to some embodiments of the present application;

[0041] Figure 3 is an exemplary flowchart of calculating first coded data according to some embodiments of the present application;

[0042] Figure 4 is an exemplary flowchart of calculating the second encoded data according to some embodiments of the present application;

[0043] Figure 5 This is an exemplary flowchart of calculating the fourth encoded data according to some embodiments of the present application. DETAILED DESCRIPTION

[0044] The method and system provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0045] like Figure 1As shown, video frame data is obtained; motion analysis is performed on the video frame data to obtain a first area, a second area and a third area; the first area represents a background area, the second area represents a smooth motion area, and the third area represents a fast motion area; the video frame data of the first area is encoded using an inter-frame prediction coding mode to obtain first encoded data; the video frame data of the second area is encoded using a motion compensation coding mode to obtain second encoded data and a motion vector of the second area; the video frame data of the third area is encoded using an intra-frame prediction coding mode to obtain third encoded data; the motion vector of the second area is encoded to obtain fourth encoded data; based on the first encoded data, the second encoded data and the third encoded data, bit rate control and bit stream encapsulation are performed to obtain a video code stream; the video code stream and the fourth encoded data are merged to obtain encoded video transmission data, and the encoded video transmission data is transmitted.

[0046] Specifically, S1 obtains video frame data; receives video source data, which can come from various sources such as digital cameras, video capture cards, and video files. According to the format of the video source (such as YUV, RGB, etc.), parse the frame structure and encoding method of the video data to determine the starting position and length of each video frame. Split the video source data according to the boundaries of the frames, extract the data of a single video frame, and form a frame data sequence. Store the extracted video frame data in a memory buffer to form a frame buffer queue. The frame buffer queue uses a first-in-first-out (FIFO) method to store and access frame data in the time order of the frames. Define the frame data access interface, such as , obtains the data of a specified video frame according to the frame number. The frame data access interface reads the required frame data from the frame buffer queue or long-term frame buffer and passes it to the requested encoding module.

[0047] like Figure 2 As shown, S2 performs motion analysis on the video frame data to obtain a first region, a second region, and a third region; the first region represents the background region, the second region represents the region with motion less than a threshold, and the third region represents the region with motion greater than the threshold. The process includes: S21, first, dividing the input video frame data into multiple blocks, which serve as basic units for motion estimation and compensation. The block size can be adaptively adjusted based on factors such as video content complexity and coding efficiency. Common block sizes include 16x16 and 32x32. Smaller blocks increase motion estimation accuracy but also increase computational complexity. Larger blocks reduce computational complexity but also decrease motion estimation accuracy. Therefore, a trade-off between accuracy and complexity is necessary.

[0048] S22: For each block, a block-matching-based motion estimation method is used to search the reference frame for the block that best matches the current block using the minimum absolute difference (MAD) criterion. Specifically, within a search window in the reference frame, centered on the current block, the MAD value between the current block and the candidate block is calculated. A smaller MAD value indicates a higher similarity between the two blocks. The search process can use a full search or a fast search algorithm, such as a three-step search or diamond search, to reduce computational complexity. The displacement vector of the best matching block obtained through the search is the candidate motion vector.

[0049] S23, after obtaining the candidate motion vectors, the rate-distortion optimization (RDO) criterion is used to select the optimal motion vector. The RDO criterion comprehensively considers the reconstruction quality and coding overhead, and controls the weight of the two factors through Lagrange multipliers. Specifically, for each candidate motion vector, the distortion metric of the corresponding motion compensation prediction block and the current block is calculated. , The distortion metric uses the weighted mean square error (WMSE). This is achieved by summing the squared pixel differences between the predicted block and the current block and multiplying the sum by a weighting factor. The weighting factor reflects the importance of different pixel positions to visual perception and can be adaptively set based on factors such as human visual characteristics and gaze point. WMSE considers the correlation of pixels within a spatial neighborhood, which is more consistent with subjective visual quality.

[0050] For each candidate motion vector, estimate the corresponding encoding bit rate , The coding bit rate includes two parts: motion vectors and motion-compensated residuals. The coding bit rate of motion vectors is related to their probability distribution and entropy coding methods. Context-adaptive variable-length coding or arithmetic coding can be used. The coding bit rate of residuals is related to their energy and transform coding methods. Transforms such as DCT and DST can be used, and the quantized bit rate is estimated based on the quantization parameter.

[0051] Constructing rate-distortion optimization objective function , , the distortion metric and encoding bitrate Weighted summation is performed, with the weights controlled by the Lagrange multiplier λ. The value of λ can be adaptively adjusted based on the current bitrate constraint and image quality requirements, taking a larger value at low bitrates and a smaller value at high bitrates to balance the weights of distortion and bitrate.

[0052] Solving the objective function The minimum value of , .because is a discrete function that can be solved using either exhaustive or heuristic search methods. Exhaustive search, while computationally expensive, guarantees the global optimal solution. Heuristic search, such as the three-step search, uses iterative approximation to find suboptimal solutions, but with a lower computational cost.

[0053] After obtaining the optimal motion vector, the video content is divided into three regions based on its size. Specifically, two motion vector thresholds, th1 and th2, are set. Blocks with an optimal motion vector less than th1 are classified as the background region, blocks with an optimal motion vector greater than th1 but less than th2 are classified as the slow motion region, and blocks with an optimal motion vector greater than th2 are classified as the fast motion region. The thresholds can be adaptively adjusted based on factors such as video content characteristics and bitrate constraints to balance the proportions of each region and coding efficiency.

[0054] When segmenting video content into background, slow motion, and fast motion regions, selecting thresholds th1 and th2 is crucial for balancing the proportions of each region and encoding efficiency. The threshold range needs to be adaptively adjusted based on factors such as video content characteristics, bitrate constraints, and target applications to achieve optimal performance.

[0055] Specifically, the threshold th1 is used to distinguish between background and motion areas, reflecting the degree of stillness in the video content. For still or slowly moving backgrounds, such as landscapes and buildings, th1 can be set to a smaller value, such as 1 to 3 pixels. This classifies most still or quasi-still blocks as background areas, reducing the computational effort required for motion estimation and compensation. For backgrounds with some motion, such as crowds and traffic, th1 can be appropriately increased, such as to 3 to 5 pixels. This prevents minor motion from being classified as background, improving motion compensation accuracy. For video content with more complex motion, such as sports events and action movies, th1 can be further increased, such as to 5 to 8 pixels. This allows for better distinction between background and motion areas, providing a more accurate reference for subsequent encoding strategies.

[0056] Threshold th2 is used to distinguish between slow and fast motion areas, reflecting the motion intensity of the video content. For video content dominated by slow motion, such as news broadcasts and interviews, th2 can be set to a smaller value, such as 5 to 10 pixels. This helps to classify most of the motion as slow motion, reducing the proportion of fast motion areas and improving coding efficiency. For video content with moderate motion, such as TV dramas and variety shows, th2 can be appropriately increased, such as 10 to 20 pixels. This helps to better distinguish between slow and fast motion and allocate appropriate coding resources to different areas. For video content dominated by fast motion, such as racing and extreme sports, th2 can be further increased, such as 20 to 30 pixels or even larger. This helps to classify most of the motion as fast motion, improving motion compensation accuracy while allocating more coding resources to fast motion areas to ensure reconstruction quality.

[0057] This application fully utilizes the spatiotemporal correlation of video signals, removing temporal redundancy between blocks through motion compensation, while simultaneously considering coding overhead and reconstruction quality to achieve optimal performance within a limited bitrate. Furthermore, by dividing the video content into distinct regions through thresholding, it provides an important basis for subsequent adaptive coding, enabling the adoption of different coding strategies and parameter settings for different regions, achieving more refined bitrate control and quality optimization.

[0058] like Figure 3 As shown, in S3, after acquiring the video frame data, the present application uses an inter-frame prediction coding mode to encode the first area classified as the background area. Specifically, in S31, one or more previously encoded frames are selected from the reference frame buffer as reference frames. The reference frame buffer is used to store reconstructed images of several previously encoded frames, that is, images after encoding and decoding. The selection of reference frames can be determined based on factors such as temporal distance and similarity. Common strategies include forward reference, backward reference, and bidirectional reference.

[0059] S32: In the selected reference frame, a block matching algorithm is used to search for the block that best matches the current block. This block is used as the reference block for the current block. Common block matching algorithms include the full search algorithm, the three-step search algorithm, and the diamond search algorithm. The matching process measures the similarity between the current block and the candidate blocks using metrics such as the minimum sum of absolute differences (SAD) or minimum mean square error (MSE). The optimal candidate block is selected as the reference block. The residual between the current block and the reference block is calculated to obtain a residual block. The residual block represents the pixel difference between the current block and the reference block and reflects the error in inter-frame prediction.

[0060] S33 performs a Discrete Sine Transform (DST) on the residual block to obtain a transform coefficient matrix. The residual block is divided into several 4×4 sub-blocks, and each sub-block is subjected to a DST transform. Each 4×4 sub-block is subjected to a two-dimensional transform using a 4×4 DST matrix to obtain the corresponding transform coefficient sub-block. The DST transform has a strong energy concentration property, concentrating the residual signal's energy on low-frequency coefficients, which facilitates subsequent quantization and encoding. All transform coefficient sub-blocks are concatenated to obtain the complete transform coefficient matrix.

[0061] S34 quantizes the transform coefficient matrix using the quantization parameter (QP) to obtain a quantization coefficient matrix. Based on the rate-distortion optimization criterion, the optimal quantization parameter (QP) is determined in the inter-frame prediction coding mode. Rate-distortion optimization selects the quantization parameter that achieves the best reconstruction quality by balancing bit rate and distortion. Each coefficient in the transform coefficient matrix is ​​quantized using the selected quantization parameter (QP) to obtain a quantization coefficient matrix. The quantization process maps the transform coefficients to a limited number of quantization levels by dividing by the quantization step size and rounding to an integer, achieving data compression.

[0062] S35 , vertically scan the quantization coefficient matrix to obtain a coefficient symbol sequence. Starting from the lower left corner of the quantization coefficient matrix, scan each column of the quantization coefficient matrix in a zigzag order from bottom to top and from left to right at a 45-degree angle to obtain a column vector sequence. Reverse and transpose the column vector sequence to obtain a coefficient symbol sequence. This sequence represents the sign and amplitude information of the quantization coefficients.

[0063] S36, run-length encoding is performed on the coefficient symbol sequence to obtain a sequence of run pairs. Non-zero coefficients in the coefficient symbol sequence and their trailing zero coefficients are counted to obtain (run, level) pairs. The run represents the number of consecutive zero coefficients following the current non-zero coefficient, and the level represents the value of the non-zero coefficient. All (run, level) pairs are arranged in order to form a sequence of run pairs. Run-length encoding effectively compresses zero values ​​in quantized coefficients, reducing data redundancy.

[0064] S37, entropy coding is performed on the sequence of (run, level) pairs to obtain the first coded data. According to the probability distribution of the (run, level) pairs in the training set, a run-level joint coding table is constructed. The coding table assigns the optimal coding word to each (run, level) pair according to the statistical characteristics of the run and level. Using the run-level joint coding table, variable-length coding is performed on the run-pair sequence. Variable-length coding assigns codewords according to the frequency of occurrence of symbols, with shorter codewords assigned to symbols with high frequency and longer codewords assigned to symbols with low frequency, thereby achieving further compression of the data. The encoded bit stream is output as the first coded data to complete the inter-frame prediction coding process of the background area. The present application takes into account temporal redundancy and spatial redundancy, adaptively adjusts the quantization parameters, and dynamically optimizes the coding table. It can significantly improve data compression efficiency and reduce the bit rate of the background area while ensuring reconstruction quality.

[0065] like Figure 4 As shown, in step S4, after acquiring the video frame data, the present application employs a motion-compensated coding mode for encoding the second region divided into the smooth motion region. In step S41, motion estimation is used to search the reference frame for the block that best matches the current block and obtain a motion vector. Motion estimation uses a block matching algorithm, such as a full search algorithm or a three-step search algorithm, to find the block in the reference frame that is most similar to the current block. The displacement between the blocks is the motion vector. In step S42, motion compensation is used to generate a prediction block at the corresponding position in the reference frame based on the motion vector. For the current coding block, the corresponding motion vector is obtained through motion estimation (e.g., step S4). The motion vector indicates the position offset of the block that best matches the current block in the reference frame. Based on the motion vector, the block that best matches the current block is found in the reference frame and used as the prediction block. The prediction block has the same size as the current block, and its position is determined by adding the motion vector to the position of the current block. The prediction block in the reference frame is copied to the position of the current block to form a motion-compensated prediction block. The prediction block represents the best match for the current block in the reference frame and is used for subsequent residual calculation and encoding. The purpose of motion compensation is to use the temporal correlation between frames to predict the current block by using the coded blocks in the reference frame, thereby reducing the amount of data that needs to be coded.

[0066] The quantization parameter (QP) is a key parameter controlling the quantization process in video encoding, used to adjust the trade-off between bitrate and quality in the bitstream generated by the encoder. Adjusting the QP value controls the bitrate of the bitstream generated by the encoder. A larger QP value results in smaller coefficient amplitudes after quantization, resulting in fewer bits and a lower bitrate. Conversely, a smaller QP value results in larger coefficient amplitudes after quantization, resulting in more bits and a higher bitrate. In intra-frame coding, different QP values ​​can be assigned to different macroblocks or coding units (such as CTUs). Locally adjusting the QP value allows for adaptive quantization control of different regions based on the complexity and importance of the image content.

[0067] S43, calculate the residual between the current block and the predicted block to obtain a residual block. Perform pixel-by-pixel difference calculation on the current block and the predicted block obtained by motion compensation to obtain a residual block. The residual block reflects the difference information between the current block and the predicted block. The pixel values ​​of the residual block represent the pixel difference between the current block and the predicted block at corresponding positions. The closer the value in the residual block is to zero, the higher the match between the predicted block and the current block, and the higher the accuracy of motion compensation. The size of the residual block is the same as that of the current block, and the dynamic range of its pixel values ​​is usually smaller than the original pixel values, which is beneficial to the subsequent transformation and quantization process. By calculating the residual block, the redundant information after motion compensation can be eliminated, further reducing the amount of data to be encoded.

[0068] S44 transforms, quantizes, and entropy encodes the residual block to obtain second coded data. The residual block is transformed from the spatial domain to the frequency domain. Common transform methods include DCT (discrete cosine transform) and DST (discrete sine transform). The transform process decomposes the residual block into a series of frequency coefficients through an orthogonal transform. Low-frequency coefficients represent smooth areas of the residual block, while high-frequency coefficients represent detailed areas of the residual block. The transformed frequency-domain coefficients have concentrated energy, with most of the energy concentrated in a few low-frequency coefficients, while the high-frequency coefficients have smaller amplitudes. This facilitates subsequent quantization and entropy encoding. The transformed frequency-domain coefficients are quantized to map them to a limited number of quantization levels. The quantization process converts the frequency-domain coefficients into quantized coefficients by dividing by the quantization step size and rounding. The quantization step size determines the quantization accuracy and compression rate. A larger quantization step size results in a smaller amount of quantized data, but also reduces reconstruction quality. The quantization step size can be adaptively adjusted based on bit rate control and quality requirements. The quantized coefficients are entropy encoded to convert them into a bitstream. Common entropy coding methods include variable-length coding (such as Huffman coding) and arithmetic coding. Entropy coding achieves data compression by analyzing the probability distribution of symbols, assigning shorter codes to high-probability symbols and longer codes to low-probability symbols. The bit stream obtained after entropy coding is called the second coded data, which represents the compressed data after motion compensation and residual coding.

[0069] S45, entropy coding the motion vector to obtain the motion vector of the second region. Filter the motion vector to obtain the horizontal component and vertical component Through credibility detection, unreliable motion vectors are eliminated; through redundancy detection, redundant motion vectors are eliminated. The filtered motion vectors are normalized to obtain the normalized horizontal component. and vertical component .

[0070] Using spatial and temporal correlation, a prediction value is selected from the coded motion vectors of adjacent blocks and adjacent frames. The median of the coded motion vectors of adjacent blocks is selected as the spatial prediction value; the median of the coded motion vectors of the co-located blocks of adjacent frames is selected as the temporal prediction value. A weighted average is calculated based on the spatial prediction value and the temporal prediction value as the comprehensive prediction value of the motion vector. The horizontal component is calculated separately. and vertical component The difference between the predicted value and the predicted value is the prediction residual and .calculate The difference between the horizontal component of the comprehensive prediction value and the horizontal prediction residual is obtained ;calculate The vertical prediction residual is obtained by dividing the vertical component of the comprehensive prediction value by .

[0071] Calculate prediction residuals and The statistical histogram of , we can get the corresponding probability distribution characteristics. and Calculate the numerical range histogram respectively and count the residual frequency in each numerical interval; and Calculate the sign distribution histograms separately and count the frequencies of positive and negative residuals. Normalize the value range histogram and the sign distribution histogram to obtain a probability distribution curve. Calculate characteristic parameters such as mean, variance, skewness, and kurtosis based on the probability distribution curve.

[0072] According to the probability distribution characteristics, the order of Hadamard transform is set, and the prediction residual is calculated using the Hadamard transform after setting the order. and Perform a two-dimensional transformation to obtain the transformation coefficient matrix. The probability distribution characteristics of the horizontal transformation are set to ;according to The probability distribution characteristics of the vertical transformation are set to . use Order Hadamard transformation matrix pair Perform horizontal transformation and use Order Hadamard transformation matrix pair Perform vertical transformation to obtain the transformation coefficient matrix.

[0073] In the prediction residual and After performing the Hadamard transform, a transform coefficient matrix is ​​obtained. To further compress data, this application employs context-adaptive variable-length coding (CAVLC) to quantize and encode the transform coefficient matrix. Using CAVLC, the transform coefficient matrix is ​​quantized and encoded to produce a quantized coefficient stream. CAVLC is a highly efficient entropy coding method that leverages the contextual information of transform coefficients to adaptively select the optimal coding table to achieve data compression. Specifically, multiple CAVLC coding tables are constructed, including a Total Zeros table, a Run Before table, and a Level table. The Total Zeros table encodes the number of trailing zero coefficients in a transform coefficient; the Run Before table encodes the length of zero runs preceding non-zero coefficients; and the Level table encodes the values ​​of non-zero coefficients. These coding tables are pre-constructed based on the statistical characteristics of the transform coefficients to optimize coding efficiency. The transform coefficient matrix is ​​scanned to produce a coefficient symbol sequence. CAVLC uses a zigzag scan, starting from the upper left corner of the transform coefficient matrix and scanning each coefficient in order from low frequency to high frequency, to produce a coefficient symbol sequence. The coefficient symbol sequence is run-length encoded to produce (run, level) pairs. Run-length coding converts a coefficient symbol sequence into a (run, level) pair by counting the number of zero coefficients between non-zero coefficients. A run represents the number of consecutive zero coefficients preceding the current non-zero coefficient, and the level represents the value of the non-zero coefficient. Run-length coding effectively compresses zero values ​​in transform coefficients. Based on the contextual information of the (run, level) pair, the corresponding codeword is searched and output in the corresponding coding table to obtain a quantized coefficient stream. For each (run, level) pair, CAVLC searches the corresponding coding table for the optimal codeword based on its contextual information, such as the levels of adjacent coefficients and the total number of zeros. By adaptively selecting the coding table, CAVLC can fully utilize the statistical characteristics of the transform coefficients and improve coding efficiency. This application uses CAVLC to quantize and encode the transform coefficient matrix after performing a Hadamard transform on the prediction residual. By adaptively selecting the coding table, it fully utilizes the statistical characteristics of the transform coefficients and improves coding efficiency.

[0074] The quantized coefficient stream is encapsulated and multiplexed to obtain the motion vector of the second region. Since the motion vector contains the horizontal component and vertical component The two parts need to be packaged and multiplexed to encapsulate their quantized coefficient streams so that the decoder can accurately locate and separate the data of each component. and The quantized coefficient streams are encapsulated separately, and synchronization headers and length information are added. The synchronization header is used to identify the starting position of each component, and the length information is used to indicate the data length of each component. Through encapsulation, the decoder can accurately locate and extract the quantized coefficient stream of each component based on the synchronization header and length information. and The quantized coefficient streams are multiplexed, interweaving the data of the two components together to form complete motion vector information. Multiplexing interweaves the data of different components according to certain rules to form a unified data stream. Common multiplexing methods include time division multiplexing and frequency division multiplexing. Through multiplexing, the horizontal and vertical components of the motion vector can be efficiently transmitted and stored. The quantized coefficient stream is encapsulated and multiplexed, and synchronization headers and length information are added to achieve efficient transmission and storage of the horizontal and vertical components of the motion vector. This scheme combines transform coding and entropy coding, and comprehensively considers data encapsulation and multiplexing. While reducing the amount of data, it ensures the integrity and decodability of the motion vector, improving the coding performance of the second region.

[0075] This application uses a motion compensation coding mode for the gently moving areas in the video sequence, eliminating temporal redundancy through motion estimation and compensation, and eliminating spatial redundancy through transformation, quantization and entropy coding. In the encoding process of the motion vector, spatial and temporal correlation is used for prediction, and the order of the Hadamard transform is adaptively set according to the probability distribution characteristics of the prediction residual to improve the transformation efficiency. CAVLC is used to quantize and encode the transform coefficients, and the horizontal and vertical components of the motion vector are encapsulated and multiplexed. This scheme makes full use of motion information and residual information, combines adaptive transformation and context coding, and effectively improves the coding efficiency of the gently moving area while ensuring the reconstruction quality and reducing the bit rate.

[0076] S5. After acquiring the video frame data, the present application uses an intra-frame prediction coding mode to encode the third region classified as a fast-motion region. S51. Based on the encoded blocks adjacent to the current block, the spatial prediction direction and prediction mode are calculated. The grayscale mean and variance of the encoded block to the left of the current block are calculated. The grayscale mean is obtained by averaging the pixel values ​​of the adjacent blocks to the left, and the variance is obtained by calculating the sum of the squares of the differences between the pixel values ​​and the mean. The grayscale mean reflects the brightness information of the block to the left, and the variance reflects the texture complexity of the block to the left. The grayscale mean and variance of the encoded block above the current block are calculated. Similar to the block to the left, the grayscale mean and variance are obtained by averaging the pixel values ​​of the adjacent blocks above, reflecting the brightness and texture characteristics of the block above. Based on the comparison of the grayscale mean and variance, the spatial prediction direction is determined to be horizontal or vertical. By comparing the grayscale mean and variance of the block to the left and the block above, it is determined which direction of the adjacent block the current block has a stronger correlation with. If the grayscale mean and variance of the left block are closer to the current block, the horizontal direction is selected as the spatial prediction direction; otherwise, the vertical direction is selected. Within the determined spatial prediction direction, the optimal prediction mode is selected from multiple prediction modes based on the grayscale value distribution characteristics of adjacent coded blocks. For adjacent blocks in the selected direction, their grayscale value distribution characteristics, such as gradient direction and edge structure, are analyzed. The mode that best matches the current block is selected from multiple predefined prediction modes as the optimal prediction mode. Common intra-frame prediction modes include DC mode, planar mode, and angular mode.

[0077] S52: Generate a prediction block for the current block based on the spatial prediction direction and prediction mode. If the spatial prediction direction is horizontal, the prediction block is generated using the boundary pixel values ​​of the coded block to the left of the current block according to the selected prediction mode. The prediction value of the current block is generated using the right boundary pixel values ​​of the left block as a reference according to the interpolation rules defined in the prediction mode. If the spatial prediction direction is vertical, the prediction block is generated using the boundary pixel values ​​of the coded block above the current block according to the selected prediction mode. The prediction value of the current block is generated using the bottom boundary pixel values ​​of the block above the current block as a reference according to the interpolation rules defined in the prediction mode.

[0078] S53 calculates the residual between the current block and the predicted block to obtain a residual block. The pixel values ​​of the current block and the predicted block are subtracted point by point to obtain a difference matrix. The residual reflects the degree of difference between the current block and the predicted block. The difference matrix is ​​weighted quantized to obtain a quantized residual block. The residual matrix is ​​quantized to integer values ​​by multiplying it by the weight matrix and dividing it by the quantization step size, thus reducing the data range.

[0079] S54: Transform, quantize, and entropy encode the residual block to obtain third coded data. The quantized residual block is subjected to a DCT transform. The quantized residual block is divided into several 4×4 sub-blocks. A 4×4 DCT transform matrix is ​​applied to each sub-block to convert the spatial residual into frequency-domain coefficients. The DCT transform, with its energy-concentrating property, compresses residual information into low-frequency coefficients. The transform coefficient matrix is ​​scaled and truncated. The DCT coefficients are modified by multiplying them by a scaling factor and truncating them to integers to adapt their dynamic range to the quantizer. The scaled transform coefficient matrix is ​​quantized. Based on the characteristics of the intra-frame coding mode, a quantization parameter (QP) range and step size are preset. The transform coefficients are quantized to integer values ​​by dividing by the quantization step size and rounding to an integer to achieve data compression. Entropy encoding is performed on the quantized coefficient matrix. The quantized coefficient matrix is ​​converted into a coefficient sequence using zigzag scanning. The coefficient sequence is then run-length encoded and constant-length encoded. Finally, the coefficient sequence is compressed using CABAC (Context-Adaptive Binary Arithmetic Coding) entropy coding technology to obtain third coded data.

[0080] This application targets fast-moving areas in video sequences and adopts an intra-frame prediction coding mode, making full use of spatial correlation and achieving data compression through a combination of prediction and residual coding. During the prediction process, the spatial prediction direction is selected based on the grayscale mean and variance of adjacent coded blocks, and the optimal mode is selected from multiple prediction modes to improve prediction accuracy. During the residual coding process, the residual data is further compressed through a series of operations such as DCT transformation, scale scaling, quantization and entropy coding. This scheme comprehensively considers the characteristics of fast-moving areas, and through adaptive prediction direction and mode selection, as well as efficient transformation and entropy coding technology, it significantly reduces the bit rate of the third area while ensuring reconstruction quality, thereby improving coding efficiency.

[0081] like Figure 5 As shown, S6, after obtaining the motion vector of the second region, in order to further compress the amount of data, the present application encodes the motion vector to obtain fourth encoded data: S61, according to the horizontal motion vector component of the second region and vertical motion vector components , respectively select the encoded motion vectors of the following areas as prediction values, and calculate the weighted average value as the comprehensive prediction value: Select the encoded motion vectors of the spatially adjacent blocks on the left, above, upper right, and lower left of the current block as the spatial prediction value. By using the motion vector information of the spatially adjacent blocks, the motion trend of the area around the current block can be captured, thereby improving the prediction accuracy. Select the encoded motion vector of the co-located block of the current block in the temporal adjacent frames such as the previous frame and the previous two frames as the temporal prediction value. By using the motion vector information of the temporal adjacent frames, the motion continuity of the current block on the time axis can be captured, thereby improving the prediction accuracy. Based on the spatial prediction value and the temporal prediction value, calculate the weighted average value as and The comprehensive prediction value of , where the weight coefficient is adaptively determined according to the spatiotemporal distance between adjacent blocks and adjacent frames. Combining the spatial domain prediction value and the temporal domain prediction value, we can get and The prediction accuracy is expected to be further improved by combining the predicted values. At the same time, an adaptive weighted averaging strategy is used to dynamically adjust the weight coefficients based on the spatiotemporal distance between adjacent blocks and frames, making the predicted value more consistent with the actual motion trend of the current block.

[0082] S62, calculate the horizontal motion vector components respectively and vertical motion vector components The difference between the motion vector prediction residual and its comprehensive prediction value is obtained and By taking the difference, the redundant information of the motion vector can be eliminated and the coding overhead can be reduced. Specifically, the horizontal motion vector component is calculated. Its comprehensive prediction value The difference between the horizontal motion vector prediction residual is obtained . ,in, Represents the horizontal motion vector component of the current block, Represents the comprehensive prediction value calculated based on the motion vectors of adjacent blocks in the spatial and temporal domains. By taking the difference, the Zhongyu The same parts, only keep Unique information, namely the residual This can remove the redundancy of motion vectors in the horizontal direction and reduce the amount of data that needs to be encoded.

[0083] Calculate the vertical motion vector component Its comprehensive prediction value The difference between the vertical motion vector prediction residual is obtained . ,in, Represents the vertical motion vector component of the current block, Represents the comprehensive prediction value calculated based on the motion vectors of adjacent blocks in the spatial and temporal domains. Similar to the horizontal direction, by taking the difference, the Zhongyu The same parts, only keep Unique information, namely the residual This can remove the redundancy of the motion vector in the vertical direction and reduce the amount of data required for encoding. and As the object of subsequent encoding, it replaces the original motion vector component and .

[0084] Through and Making a prediction and calculating the residual between it and the predicted value can significantly reduce the amplitude and dynamic range of the data to be encoded. and Contains only and The unpredicted part of , usually has a smaller value and fewer non-zero values, and is more suitable for transformation and entropy coding. and Perform subsequent transformation, quantization and entropy coding to generate the final motion vector coding data. and Performing transformations (such as Hadamard transformations) can further improve the energy concentration of the residuals and compress the information into a few transformation coefficients. By quantizing the transformed coefficients, the coefficients can be mapped to a limited quantization level to achieve data compression. By performing entropy coding (such as CAVLC) on the quantized coefficients, the optimal coding codeword can be assigned according to the probability distribution of the coefficients to further compress the data. In the motion vector coding process, the present application calculates the difference between the motion vector component and its comprehensive prediction value to obtain the motion vector prediction residual, and uses the residual to replace the original motion vector component for subsequent coding. This approach can make full use of the correlation between the motion vector in the time domain and the spatial domain, eliminate redundant information through prediction, and only encode the residual, thereby significantly reducing the amount of encoded data and reducing the bit rate overhead. At the same time, the residual usually has better statistical characteristics, which is more conducive to transformation and entropy coding, further improving compression efficiency.

[0085] S63, respectively calculating motion vector prediction residuals and The numerical range histogram and the symbol distribution histogram of the residual sample are obtained by performing the following processing to obtain the probability distribution characteristics: the numerical range histogram and the symbol distribution histogram are normalized to eliminate the scale difference. The normalization processing can map the histograms of different residual samples to the same scale, which is convenient for the subsequent probability distribution analysis. The statistical characteristic parameters such as mean, variance, skewness, kurtosis, etc. are extracted from the normalized histogram to characterize the probability distribution characteristics of the residual sample. and The probability distribution characteristics of the residuals. The value range histogram reflects the distribution of the residual amplitude, and the sign distribution histogram reflects the distribution of the positive and negative signs of the residuals. By extracting these statistical features, the probability distribution law of the residuals can be fully described. S64, according to and The probability distribution characteristics of the horizontal and vertical transformations are set, and the Hadamard transform orders of the horizontal and vertical transformations are set, and the motion vector prediction residual is calculated using the two-dimensional Hadamard transform of the set order. and Transform and get and The corresponding transformation coefficient matrix.

[0086] Specifically, according to The probability distribution characteristics of the horizontal transformation are adaptively set to the Hadamard transformation order. .analyze The probability distribution of the residual sample is calculated, and its statistical characteristic parameters such as mean, variance, skewness, and kurtosis are calculated to characterize the concentration and dispersion of the residual sample. When the probability distribution of the residual sample is more concentrated (such as the mean is close to zero, the variance is small, the skewness and kurtosis are large), it means that the residual energy is mainly concentrated in a few values. At this time, the transformation order can be reduced accordingly. A lower transform order is sufficient to capture the main features of the residual, reducing computational complexity and coding overhead. When the probability distribution of the residual samples is more dispersed (such as the mean deviates from zero, the variance is larger, the skewness and kurtosis are smaller), it means that the residual energy is more evenly distributed on multiple values. In this case, it is necessary to increase the transform order appropriately. A higher transform order can better match the sparse characteristics of the residual transform domain, improve the transform efficiency and compression performance. By setting appropriate thresholds and mapping relationships, the optimal horizontal transform order is adaptively determined according to the statistical characteristic parameters of the residual distribution. .

[0087] according to The probability distribution characteristics of the vertical transformation are adaptively set to the Hadamard transform order. . Similar to the horizontal transformation, the analysis The probability distribution of the residual is calculated, and its statistical characteristic parameters such as mean, variance, skewness, and kurtosis are calculated to characterize the concentration and dispersion of the residual samples. According to the statistical characteristic parameters of the residual distribution, the optimal vertical transformation order is adaptively determined. The order of vertical transformation is the same as that of horizontal transformation. When the probability distribution of residual samples is more concentrated, the order of vertical transformation is reduced accordingly. ; When the probability distribution of residual samples is more dispersed, it is appropriate to increase By setting appropriate thresholds and mapping relationships, the optimal vertical transformation order is adaptively determined based on the statistical characteristic parameters of the residual distribution. .

[0088] use The one-dimensional Hadamard transform is used to transform resx horizontally. One-dimensional Hadamard transform pair Perform vertical transformation to obtain and The two-dimensional transformation coefficient matrix of conduct One-dimensional row transform of order. According to The value of , select the Hadamard transformation matrix of the corresponding order, Transform each row of to obtain the transformed row vector. conduct One-dimensional column transformation of order. According to The value of , select the Hadamard transformation matrix of the corresponding order, Transform each column of to obtain the transformed column vector. Combine the transformed row vector and column vector into a two-dimensional matrix to obtain and The transformation coefficient matrix after the two-dimensional separable Hadamard transformation.

[0089] The Hadamard transform is an orthogonal transform based on a rectangular wave function. The coefficients of the transform matrix are all ±1, and the transform process involves only addition and subtraction operations. This transform is fast, reversible, and achieves integer precision. Compared to other transforms such as the DCT and DWT, the Hadamard transform has lower computational complexity and is more suitable for hardware implementation. The transformed coefficient matrix is ​​then used for quantization and entropy coding to generate the final motion vector coded data. Quantization is performed on the transform coefficient matrix, mapping the transform coefficients to a limited number of quantization levels to achieve data compression. The quantization parameter can be adaptively adjusted based on a rate-distortion optimization criterion. Entropy coding, such as CAVLC or CABAC, is performed on the quantized coefficient matrix. Based on the probability distribution of the coefficients, optimal coding codewords are assigned to further compress the data. The entropy-coded bitstream data is then packaged to form the final motion vector coded data for transmission or storage.

[0090] The application proposes an adaptive Hadamard transform method for motion vector prediction residuals resx and resy. By analyzing the probability distribution characteristics of the residuals, the Hadamard transform orders for the horizontal and vertical transforms are adaptively set. The residuals are then subjected to a two-dimensional separable Hadamard transform using the set orders to obtain a transform coefficient matrix. This method dynamically adjusts the transform order based on the statistical characteristics of the residuals, reducing the order when the residual distribution is concentrated, thereby reducing computational overhead; and increasing the order when the residual distribution is dispersed, thereby improving transform efficiency. Furthermore, a combination of one-dimensional row transform and one-dimensional column transform is employed to achieve rapid decomposition of the two-dimensional transform, reducing computational complexity. The Hadamard transform uses coefficients of ±1 and involves only addition and subtraction operations, offering advantages such as hardware friendliness and integer precision. The transformed coefficient matrix undergoes quantization and entropy coding to generate the final motion vector encoded data. This scheme fully matches the probability distribution characteristics of the residuals through adaptive transforms. Combined with quantization and entropy coding, it significantly improves the compression coding efficiency of motion vectors.

[0091] S65, in the motion vector prediction residual and After performing the adaptive Hadamard transform, the corresponding transform coefficient matrix is ​​obtained. In order to further compress the data, this application constructs multiple CAVLC (Context Adaptive Variable Length Coding) coding tables and performs a series of coding processes on the transform coefficient matrix to finally generate a motion vector quantization coefficient stream. and .

[0092] The transform coefficient matrix is ​​scanned in a zigzag pattern to convert the two-dimensional matrix into a one-dimensional coefficient sequence. Starting from the upper left corner of the transform coefficient matrix, each coefficient in the matrix is ​​scanned in sequence along a zigzag path to form a one-dimensional coefficient sequence. The zigzag scanning path is as follows: first scan one coefficient to the right, then scan one coefficient to the lower left, then scan one coefficient to the upper right, and then continue to scan one coefficient to the right, alternating in this way until the entire matrix is ​​scanned. Zigzag scanning can improve the spatial correlation of transform coefficients. Due to the energy concentration characteristic of the Hadamard transform, the transformed coefficients usually have a larger amplitude near the upper left corner of the matrix and are close to zero near the lower right corner. Zigzag scanning can group these correlated coefficients together to form a continuous coefficient sequence, which is beneficial for subsequent entropy coding.

[0093] Run-length encoding is performed on the coefficient sequence to obtain the run length and non-zero coefficient level. The one-dimensional coefficient sequence is traversed, and the number of consecutive zero-valued coefficients is counted to obtain the run length. The run length represents the number of zero-valued coefficients between two non-zero coefficients. For each non-zero coefficient, the corresponding level is determined based on its numerical value. The level of a non-zero coefficient represents the range of its absolute value. The run length and non-zero coefficient level are combined into a (run length, level) pair to form a run-length encoded symbol stream. Run-length coding can fully utilize the sparsity of transform coefficients. After the Hadamard transform, the values ​​of most coefficients tend to zero, and only a few coefficients have significant non-zero values. By recording the number of consecutive zero-valued coefficients, run-length coding can greatly reduce the number of symbols required for encoding and reduce coding redundancy.

[0094] Based on the joint probability distribution of run length and non-zero coefficient level, entropy coding is performed using a constructed CAVLC coding table, resulting in the quantized coefficient streams mvx and mvy. Multiple CAVLC coding tables are pre-constructed based on the statistical distribution of run length and non-zero coefficient level in the transform coefficients. Each coding table provides the optimal encoding codeword for a different run length and level range. For each (run length, level) pair, the corresponding encoding codeword is searched in the corresponding CAVLC coding table based on the run length and level range and output to the quantized coefficient stream. CAVLC is an adaptive variable-length coding technique. It dynamically adjusts the coding table based on the statistical probability distribution of transform coefficients, ensuring that high-probability symbols receive shorter encoding codewords and low-probability symbols receive longer encoding codewords, thereby optimizing entropy coding.

[0095] By constructing multiple coding tables, CAVLC can adapt to transform coefficients with different distribution characteristics, select the most matching coding mode, improve entropy coding efficiency, and reduce coding overhead. and The transformation coefficient matrix is ​​subjected to the above encoding process respectively, and the corresponding motion vector quantization coefficient stream is finally obtained. and .right The transformation coefficient matrix is ​​zigzag scanned, run-length coded and CAVLC entropy coded to generate a horizontal motion vector quantization coefficient stream .right The transformation coefficient matrix is ​​zigzag scanned, run-length coded and CAVLC entropy coded to generate a vertical motion vector quantization coefficient stream . and The compressed data representing the horizontal motion vector and vertical motion vector after quantization and entropy coding can be used for transmission or storage.

[0096] This application uses CAVLC coding technology to efficiently compress the transform coefficient matrix after performing an adaptive Hadamard transform on the motion vector prediction residual. Zigzag scanning improves the spatial correlation of the transform coefficients, run-length coding fully exploits the sparsity of the transform coefficients, and constructs multiple CAVLC coding tables to match different probability distribution characteristics, thereby achieving optimal entropy coding of the motion vector quantized coefficient stream. This solution comprehensively considers the characteristics of transform, quantization, and entropy coding, and through adaptive coding mode selection, significantly improves the compression efficiency of motion vectors and reduces transmission and storage overhead.

[0097] S66: Encapsulate the motion vector quantization coefficient stream to obtain fourth encoded data. The motion vector quantization coefficient stream is encapsulated, and necessary header information, trailer information, and synchronization information are added to ensure data integrity and synchronization. The encapsulated motion vector quantization coefficient stream is multiplexed using time division multiplexing or code division multiplexing to obtain fourth encoded data. This multiplexing operation can package data from different motion vector quantization coefficient streams together, improving transmission efficiency and reducing bit rate overhead.

[0098] This application uses a series of coding techniques for the motion vectors in the second region, including motion vector prediction, residual calculation, probability distribution analysis, adaptive Hadamard transform, CAVLC entropy coding, and data encapsulation. By fully exploiting the spatiotemporal correlation and probability distribution characteristics of the motion vectors, the transform order and coding mode are adaptively adjusted to effectively compress the data volume of the motion vectors and generate efficient and compact fourth-code data. At the same time, multiplexing technology is used to package the quantized coefficient stream, further improving transmission efficiency and reducing bit rate overhead. While ensuring the quality of motion vector coding, this solution significantly reduces the data volume of the code stream and improves the compression effect.

[0099] S7, performing rate control and bitstream encapsulation based on the first coded data, the second coded data, and the third coded data to obtain a video stream, including: counting the number of non-zero transform coefficients in the first coded data, the second coded data, and the third coded data as the first coded data complexity, the second coded data complexity, and the third coded data complexity; setting weight coefficients corresponding to the first coded data complexity, the second coded data complexity, and the third coded data complexity; multiplying the first coded data complexity, the second coded data complexity, and the third coded data complexity by the corresponding weight coefficients to obtain weighted complexities; and adding the weighted complexities to obtain an overall complexity of the video frame;

[0100] Adaptively allocate the encoding bit rate based on the overall complexity of the video frame: When the overall complexity of the video frame exceeds a threshold, increase the encoding bit rate accordingly: Calculate the increase in the encoding bit rate, which is proportional to the degree to which the overall complexity of the video frame exceeds the threshold; Based on the original encoding bit rate, multiply it by a proportional coefficient greater than 1 based on the increase to obtain the increased encoding bit rate;

[0101] When the overall complexity of the video frame is less than or equal to the threshold, the encoding bit rate is reduced accordingly: the reduction in the encoding bit rate is calculated, and the reduction is proportional to the degree to which the overall complexity of the video frame is lower than the threshold; based on the original encoding bit rate, the reduced encoding bit rate is obtained by multiplying it by a proportional coefficient less than 1 according to the reduction.

[0102] During the rate control process, corresponding weight coefficients are set according to the visual importance of the first region, the second region, and the third region; the encoding bit rate is multiplied by the weight coefficient of each region to obtain a target bit rate for each region; and a rate-distortion optimization model is used to minimize the encoding distortion by adjusting the quantization parameter under the constraint of the target bit rate of each region to obtain the first encoded data, the second encoded data, and the third encoded data after rate control.

[0103] The first coded data, the second coded data and the third coded data after bit rate control are encapsulated into a bitstream: the frame header information of the video frame is set, including the frame number, frame type, quantization parameter, motion vector accuracy, etc.; the header information of the video sequence is set, including the video resolution, frame rate, encoding tool settings, etc.; the trailer information of the video sequence is set, including the CRC check code, etc.; according to the syntax rules of the video encoding standard, the first coded data, the second coded data, the third coded data after bit rate control and the corresponding frame header information, slice header information and trailer information are merged to obtain a video code stream.

[0104] S8. Merge the video stream and the fourth coded data to obtain encoded video transmission data, and transmit the encoded video transmission data. This specifically includes the following steps: segmenting the video stream to obtain multiple video stream slices; segmenting the fourth coded data to obtain multiple fourth coded data segments; wherein the segmentation method of the fourth coded data corresponds to the segmentation method of the video stream, so that each video stream slice corresponds to a fourth coded data segment; creating a combined transmission unit, which includes three parts: a synchronization header, a payload, and a check tail; and segmenting the video stream and the fourth coded data separately. The granularity of video stream segmentation can be set based on actual transmission requirements and network conditions. Common segmentation units include frames, slices, and blocks. The segmentation method of the fourth coded data must correspond to the segmentation method of the video stream to ensure that each video stream slice can find a corresponding fourth coded data segment, facilitating subsequent merging and encapsulation. The synchronization header is used to identify the starting position and transmission parameters of the transmission unit; the payload is used to carry the video stream slices and the fourth coded data segments; and the check tail is used to detect and correct transmission errors.

[0105] Each video stream slice and the corresponding fourth coded data segment are combined and encapsulated into a single transmission unit payload. The length information for the video stream slice and the fourth coded data segment is added at the beginning of the payload. Following this length information, the contents of the video stream slice and the fourth coded data segment are added in sequence. To facilitate parsing at the receiving end, the length information is added at the beginning of the payload to indicate the length of the video stream slice and the fourth coded data segment, followed by the contents of these segments. This length + data encapsulation format facilitates accurate extraction and restoration of the video stream and the fourth coded data at the receiving end.

[0106] The synchronization header of the transmission unit includes the transmission unit's sequence number, priority, and timestamp information. The timestamp indicates the decoding and presentation time of the transmission unit. The checksum of the transmission unit includes a cyclic redundancy check (CRC) code to check the integrity and correctness of the transmission unit.

[0107] According to the priority and timestamp of the transmission unit, multiple transmission units are sorted and scheduled to obtain encoded video transmission data; the encoded video transmission data is packaged into data packets through the network transmission protocol and sent to the receiving end; at the receiving end, the received data packets are reassembled and parsed according to the sequence number and timestamp of the transmission unit to obtain video code stream slices and the corresponding fourth encoded data fragments, and subsequent decoding and playback processing are performed.

Claims

1. A method for transmitting audio and video media, characterized in that: include: S1, obtain video frame data; S2, performing motion analysis on the video frame data to obtain a first region, a second region, and a third region; The first region represents a background region, the second region represents a smooth motion region, and the third region represents a fast motion region, including: S21, dividing the video frame data into a plurality of blocks; S22, searching for a block with the highest matching degree with the current block in a reference frame by minimum absolute difference, to obtain a candidate motion vector; S23, select the optimal motion vector mv from the candidate motion vectors opt , including: according to each candidate motion vector, calculating the distortion metric between the corresponding motion compensated prediction block and the current block Where D(mv) represents the x ,mv y ) is the distortion measure between the candidate prediction block of the motion vector and the current block, C(x,y) represents the pixel value of the current block at the pixel position (x,y), P(x+mv x ,y+mv y ) means (mv x ,mv y ) is the candidate prediction block of the motion vector at pixel position (x+mv x ,y+mv y ), W(x, y) represents the weight factor of the current block at the pixel position (x, y), which is used to reflect the contribution of the pixel at this position to the distortion metric; according to each candidate motion vector, the corresponding coding bit rate R(mv) is calculated: R(mv) = R mv (mv)+R res (mv), where R mv (mv) represents the coding bit rate of the motion vector, which is related to the probability distribution of the motion vector and the entropy coding method; R res (mv) represents the coding bit rate of the motion compensation residual, which is related to the residual energy and the transform coding method; construct the rate-distortion optimization objective function J(mv): J(mv)=D(mv)+λ×R(mv); where λ is the Lagrange multiplier, which controls the weight distribution of rate-distortion optimization; solving the minimum value of the rate-distortion optimization objective function, we get the optimal motion vector mv opt :mv opt =argminJ(mv), where argminJ(mv) represents the motion vector that minimizes the objective function J(mv) among all candidate motion vectors as the optimal motion vector; S24, dividing the corresponding blocks whose optimal motion vectors are less than a first threshold into a first region, representing a background region; dividing the corresponding blocks whose optimal motion vectors are greater than the first threshold and less than a second threshold into a second region, representing a smooth motion region; and dividing the corresponding blocks whose optimal motion vectors are greater than the second threshold into a third region, representing a fast motion region; S3, encoding the video frame data of the first region using an inter-frame prediction coding mode to obtain first coded data; S4, encoding the video frame data of the second region using a motion compensation encoding mode to obtain second encoded data and a motion vector of the second region; S5, encoding the video frame data of the third region using an intra-frame prediction encoding mode to obtain third encoded data; S6, encoding the motion vector of the second region to obtain fourth encoded data; S7, performing rate control and bit stream encapsulation according to the first coded data, the second coded data, and the third coded data to obtain a video stream; S8, combining the video code stream and the fourth coded data to obtain coded video transmission data, and transmitting the coded video transmission data.

2. The audio and video media transmission method according to claim 1, wherein: S3, encoding the video frame data of the first region using an inter-frame prediction coding mode to obtain first coded data; S31, selecting one or more previously encoded frames as reference frames in a reference frame buffer, wherein the reference frame buffer is used to store a plurality of encoded frames to reconstruct an image; S32, searching for a block in the reference frame that best matches the current block as a reference block for the current block; and calculating a residual between the current block and the reference block to obtain a residual block; S33, performing DST transformation on the residual block to obtain a transformation coefficient matrix; S34, quantizing the transform coefficient matrix using a quantization parameter QP to obtain a quantization coefficient matrix; S35, vertically scanning the quantization coefficient matrix to obtain a coefficient symbol sequence; S36, performing run-length coding on the coefficient symbol sequence to obtain a sequence of run-length pairs; S37, perform entropy coding on the sequence of run-length pairs, and use a bit stream obtained by entropy coding as first coded data.

3. The audio and video media transmission method according to claim 1, wherein: S4, encoding the video frame data of the second region using a motion compensation encoding mode, including: S41, using motion estimation, searching for a block that best matches the current block in the reference frame to obtain a motion vector; S42, using motion compensation, generating a prediction block at a corresponding position in the reference frame according to the motion vector; S43, calculating the residual between the current block and the predicted block to obtain a residual block; S44, transforming, quantizing, and entropy encoding the residual block to obtain second encoded data; S45 , performing entropy coding on the motion vector to obtain a motion vector of the second region.

4. The audio and video media transmission method according to claim 3, wherein: S45, performing entropy coding on the motion vector to obtain a motion vector of the second region, including: Filter the motion vector to get the horizontal component mv x and the vertical component mv y ; Exploiting spatial and temporal correlations, predictors are selected from coded motion vectors of neighboring blocks and frames; Calculate the horizontal component mv separately x and the vertical component mv y The difference between the predicted value and the predicted value is the prediction residual res x and res y ; Calculate the prediction residual res x and res y The statistical histogram of , and the corresponding probability distribution characteristics are obtained; According to the probability distribution characteristics, the order of Hadamard transform is set, and the prediction residual res is calculated using the Hadamard transform after setting the order. x and res y Perform a two-dimensional transformation to obtain a transformation coefficient matrix; Quantizing and encoding the transform coefficient matrix using context-adaptive variable length coding (CAVLC) to obtain a quantized coefficient stream; The quantized coefficient stream is encapsulated and multiplexed to obtain a motion vector of the second region.

5. The audio and video media transmission method according to claim 4, wherein: S6, encoding the motion vector of the second region to obtain fourth encoded data, including: S61, based on the horizontal motion vector component mv of the second region x and the vertical motion vector component mv y , respectively select the coded motion vectors of adjacent blocks and adjacent frames as prediction values, and calculate the weighted average as the comprehensive prediction value; S62, calculate the horizontal motion vector components mv respectively x and the vertical motion vector component mv y , and the difference between the corresponding comprehensive prediction value, the motion vector prediction residual res x and res y ; S63, respectively calculate the motion vector prediction residual res x and res y The numerical range histogram and the symbol distribution histogram are obtained, and the probability distribution characteristics are obtained according to the numerical range histogram and the symbol distribution histogram; S64, according to res x and res y The probability distribution characteristics of the horizontal and vertical transformations are set, and the Hadamard transform order of the horizontal and vertical transformations is set, and the motion vector prediction residual res is calculated by using the Hadamard transform after the order is set. x and res y Transform and get res x and res y The transformation coefficient matrix of S65, build multiple CAVLC coding tables, x and res y The transformation coefficient matrix is ​​entropy coded to generate a motion vector quantization coefficient stream; S66, encapsulating the motion vector quantization coefficient stream and obtaining fourth coded data through multiplexing.

6. The audio and video media transmission method according to claim 1, wherein: S5, encoding the video frame data of the third region using an intra-frame prediction coding mode, including: S51, calculating a spatial prediction direction and a prediction mode according to coded blocks adjacent to the current block; S52, generating a prediction block of the current block according to the spatial prediction direction and the prediction mode; S53, calculating the residual between the current block and the predicted block to obtain a residual block; S54: transform, quantize, and entropy encode the residual block to obtain third encoded data.

7. An audio and video media transmission system, characterized in that: include: at least one processing unit; Used to execute instructions to implement the audio and video media transmission method according to any one of claims 1 to 6.

Images