Video coding method and device, medium, computing equipment and computer program product

Abstract

The invention provides a video coding method and device, a medium, computing equipment and a computer program product, and the method comprises the steps: obtaining a target coding region and each reference region in a to-be-coded video; reconstructing the target coding region based on each reference region, and determining reconstruction loss corresponding to each reference region; based on the pixel information of the target coding region and the pixel information of each reference region, determining the pixel distribution correlation between the target coding region and each reference region; for any reference area, based on the pixel distribution correlation corresponding to any reference area, the reconstruction loss is adjusted, and the adjusted reconstruction loss corresponding to each reference area is obtained; and according to the adjusted reconstruction loss corresponding to each reference region, determining a target reference region from each reference region, and performing coding processing based on the target coding region and the target reference region. The reference image block similar to the current image block can be accurately selected, and the video coding quality is improved.

Description

Technical Field

[0001] This specification relates to the technical field of video encoding, and in particular to a video encoding method, apparatus, medium, computing device, and computer program product. Background Technology

[0002] With the rapid development of multimedia technology and the continuous upgrading of network bandwidth, video has become one of the most important data forms in internet traffic. Against this backdrop, improving the compression efficiency of video encoding technology, as a core link in the digital video industry chain, is particularly important.

[0003] Current mainstream video coding standards all employ a block-based hybrid coding framework, eliminating temporal redundancy through motion estimation and motion compensation, and spatial redundancy through transform quantization. However, existing video coding technologies, when determining the similarity between the current image block and the reference image block, often only consider the difference in the average or squared difference of pixel values ​​between the two blocks. This results in inaccurate selection of the most similar reference image block, leading to quality issues such as visual artifacts when encoding the current image block based on the reference block, severely impacting the subjective quality of the compressed video.

[0004] Therefore, improving the quality of video encoding has become a problem to be solved. Summary of the Invention

[0005] In view of the above, embodiments of this specification provide a video encoding method. One or more embodiments of this specification also relate to a video encoding apparatus, a computer-readable storage medium, a computing device, and a computer program product, to address the technical deficiencies existing in the prior art.

[0006] According to a first aspect of the embodiments of this specification, a video encoding method is provided, comprising:

[0007] Obtain the target encoding region and each reference region in the video to be encoded;

[0008] The target coding region is reconstructed based on each reference region, and the reconstruction loss corresponding to each reference region is determined.

[0009] Based on the pixel information of the target coding region and the pixel information of each reference region, the pixel distribution correlation between the target coding region and each reference region is determined.

[0010] For any reference region, based on the pixel distribution correlation corresponding to any reference region, the reconstruction loss between the target coding region and any reference region is adjusted to obtain the adjusted reconstruction loss corresponding to each reference region.

[0011] Based on the adjusted reconstruction loss corresponding to each reference region, the target reference region is determined from each reference region, and encoding processing is performed based on the target coding region and the target reference region.

[0012] According to a second aspect of the embodiments of this specification, a video encoding apparatus is provided, comprising:

[0013] The acquisition module is configured to acquire the target encoding region and each reference region in the video to be encoded.

[0014] The reconstruction module is configured to reconstruct the target coding region based on each reference region and determine the reconstruction loss corresponding to each reference region.

[0015] The analysis module is configured to determine the pixel distribution correlation between the target coding region and each reference region based on the pixel information of the target coding region and the pixel information of each reference region.

[0016] The evaluation module is configured to adjust the reconstruction loss between the target coding region and any reference region based on the pixel distribution correlation of any reference region, so as to obtain the adjusted reconstruction loss for each reference region.

[0017] The encoding module is configured to determine the target reference region from each reference region based on the adjusted reconstruction loss corresponding to each reference region, and to perform encoding processing based on the target encoding region and the target reference region.

[0018] According to a third aspect of the embodiments of this specification, a computing device is provided, comprising:

[0019] Memory and processor;

[0020] The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the video encoding method described above.

[0021] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores a computer program / instructions that, when executed by a processor, implement the steps of the video encoding method described above.

[0022] According to a fifth aspect of the embodiments of this specification, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the steps of the video encoding method described above.

[0023] In one embodiment of this specification, a target coding region and various reference regions are obtained from the video to be encoded. The target coding region is reconstructed based on each reference region, and the reconstruction loss corresponding to each reference region is determined. Based on the pixel information of the target coding region and the pixel information of each reference region, the pixel distribution correlation between the target coding region and each reference region is determined, thus completing the comparison of the pixel distribution correlation between the target coding region and each reference region. This provides data support for adjusting the reconstruction loss between the target coding region and any reference region based on the pixel distribution correlation corresponding to any reference region. For any reference region, the reconstruction loss between the target coding region and any reference region is adjusted based on the pixel distribution correlation corresponding to any reference region, resulting in the adjusted reconstruction loss corresponding to each reference region. Based on the adjusted reconstruction loss corresponding to each reference region, a target reference region is determined from each reference region, and encoding processing is performed based on the target coding region and the target reference region. This allows the pixel distribution correlation between the target coding region and any reference region to be incorporated into the measurement of their similarity based on the reconstruction loss corresponding to each reference region, achieving accurate selection of the reference region most similar to the target coding region. This solves quality problems such as visual artifacts that occur when encoding the target coding region based on reference regions. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of a video encoding process framework provided in one embodiment of this specification;

[0025] Figure 2 This is a flowchart illustrating a video encoding method provided in one embodiment of this specification;

[0026] Figure 3 This is a schematic diagram of the target coding region in a video coding method provided in one embodiment of this specification;

[0027] Figure 4 This is a schematic diagram of a reference area in a video coding method provided in one embodiment of this specification;

[0028] Figure 5 This is a schematic diagram of a reference area in another video coding method provided in one embodiment of this specification;

[0029] Figure 6 This is a schematic diagram of a reference area in another video coding method provided in one embodiment of this specification;

[0030] Figure 7 This is a schematic diagram of an MCTF filter provided in one embodiment of this specification;

[0031] Figure 8This is a schematic diagram of a pyramid search structure provided in one embodiment of this specification;

[0032] Figure 9 This is a flowchart illustrating the processing procedure of a video encoding method applied to video storage, provided in one embodiment of this specification.

[0033] Figure 10 This is a schematic diagram of the structure of a video encoding device provided in one embodiment of this specification;

[0034] Figure 11 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0035] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0036] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0037] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0038] Furthermore, it should be noted that the data involved in one or more embodiments of this specification (including but not limited to data used for analysis, stored data, displayed data, etc.) are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0039] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0040] Motion estimation: Motion estimation is the process of calculating the displacement vector between the current image block and a reference image block by analyzing pixel changes between adjacent frames in a video sequence. Motion estimation is used to eliminate temporal redundancy between video frames by matching the most similar image block in the current frame with the most similar image block in the reference frame to determine its motion direction and distance. For example, in Versatile Video Coding (VVC), a 16×16 image block in the current frame (each block is 16 pixels wide and 16 pixels high) is used to find the best-matching 16×16 block in the reference frame through motion estimation, and its motion vector (MV) is recorded.

[0041] Motion compensation: Motion compensation is a technique that uses the displacement vector obtained from motion estimation to shift an image block in a reference frame to its corresponding position in the current frame in order to predict the content of the current image block. Motion compensation reduces the amount of encoded data in the current frame by reconstructing the reference image block and is a core step in inter-frame prediction. For example, if motion estimation determines that the motion vector of the current block is (5, 3), then the block at the corresponding position is extracted from the reference frame as the predicted value of the current block.

[0042] Temporal filtering: Temporal filtering utilizes temporal correlation to filter multiple frames in a video sequence to eliminate noise or enhance compression efficiency. Temporal filtering is often combined with motion compensation (such as Motion Compensated Temporal Filtering (MCTF)) to improve video quality or compression performance through weighted averaging or adaptive filtering. For example, in MCTF, motion compensation is applied to three consecutive frames followed by weighted averaging to reduce high-frequency noise and temporal redundancy.

[0043] Pearson correlation coefficient: The Pearson correlation coefficient is a statistical indicator that measures the degree of linear correlation between two image patches, with a value ranging from -1 to 1. This coefficient is used to quantify the similarity between the current image patch and the reference image patch; the closer the value is to 1, the higher the similarity. For example, the Pearson correlation coefficient between the current 16×16 patch and the candidate patch in the reference frame is 0.92, indicating that the two are highly similar.

[0044] Encoding control: Encoding control is the process of dynamically adjusting encoding parameters (such as quantization parameters (QP), frame type, etc.) to optimize the balance between bitrate and distortion. It uses strategies such as Rate Distortion Optimization (RDO) to determine the encoding mode, ensuring a balance between video quality and compression efficiency. For example, in Lookahead, the structure of a Group of Pictures (GOP) is adaptively adjusted based on scene transition detection.

[0045] Video encoding technology is one of the core technologies for all video applications. The efficiency of video encoding determines the cost of video content production and the user experience. The process of video encoding is essentially the compression of signal data. Its essence is to significantly reduce the information redundancy in the original signal space through compact mathematical expressions and probabilistic models. Inevitably, a certain amount of signal distortion is introduced in this process in exchange for significant bandwidth savings. How to determine the efficiency ratio between the loss of signal distortion and the bandwidth savings directly determines the final efficiency of the encoding model.

[0046] refer to Figure 1 , Figure 1 A schematic diagram of a video encoding workflow framework provided in one embodiment of this specification is shown. Figure 1 The diagram illustrates the core processing steps of a modern video encoder (such as H.264 / AVC or VVC) for encoding an input image. These steps can be categorized into predictive coding, transform quantization, entropy coding, and deblocking filtering. The following is a detailed description of each step:

[0047] Input image: The image contained in the video to be encoded (e.g., in Luminance & Chrominance (YUV) format).

[0048] Encoding control: This function configures encoding parameters (such as GOP structure, quantization parameter QP, and bitrate control mode) and determines the frame type (such as I-frame, P-frame, and B-frame) for each frame. For example, if the current image is a scene transition image, the frame type of the current image is configured as I-frame; if the current image is a slowly changing video image (such as several people sitting around a table eating), the frame type of the current image is configured as B-frame; if the current image is a video image with fast-moving objects (such as athletes playing badminton on a court), the frame type of the current image is configured as P-frame.

[0049] Predictive coding includes: Intra-prediction: predicting the current block using neighboring pixels already encoded in the current frame, reducing spatial redundancy. Motion estimation: searching for the best-matching image block in the reference frame (forward / backward) and calculating the motion vector (MV) between them. Motion compensation: extracting the predicted block from the reference frame based on the MV, generating the residual (current block - predicted block) between the current block and the predicted block.

[0050] Transform coding includes forward transform, a mathematical transformation process that converts image patches from the spatial domain to the frequency domain to reduce spatial redundancy. For example, the Discrete Cosine Transform (DCT) or Discrete Sine Transform (DST) are commonly used to convert pixel values ​​into frequency domain coefficients. Quantization involves lossy compression of the transform coefficients to reduce high-frequency information (which is not sensitive to the human eye). Key parameters for quantization are the quantization factor (QP) and rate-distortion optimization (RDO). A higher QP value results in a higher compression ratio but also more significant distortion. RDO is used to dynamically adjust the QP to balance bit rate and quality.

[0051] Dequantization: Dequantization is the inverse process of restoring quantization coefficients to frequency domain coefficients. It recovers approximate frequency domain values ​​by multiplying by the quantization step size, but there is a loss of accuracy.

[0052] Inverse Transform: The inverse transform is the reverse process of converting frequency domain coefficients back to spatial domain pixel values. For example, image patches can be reconstructed by inverse discrete cosine transform (IDCT) or inverse discrete sine transform (IDST).

[0053] Entropy coding: used for further lossless compression of quantization coefficients, motion information, etc.

[0054] Bitstream: Outputs compressed data conforming to a standard format (such as H.264 Annex B), including encoding parameters, residual coefficients, motion vectors, etc.

[0055] Deblocking filtering: Deblocking filtering is a post-processing technique that eliminates discontinuous artifacts at the boundaries of coded blocks, improving subjective quality by smoothing the pixels at the block edges.

[0056] Output image: The filtered image is stored in the decoded image buffer and used as a reference frame for subsequent frames.

[0057] Motion information includes motion vectors, reference frame index, prediction mode, etc., and participates in entropy coding.

[0058] Control information: Control information is a set of parameters generated during the encoding process to guide decoding, including frame type, QP, motion vectors, etc. For example, the Slice Header in a VVC bitstream contains frame type and QP values.

[0059] Currently, mainstream video coding standards generally adopt a block-based hybrid coding framework, which uses motion estimation and motion compensation techniques to eliminate temporal redundancy and combines transform quantization to reduce spatial redundancy.

[0060] However, existing technologies have limitations in evaluating the similarity between image blocks, relying solely on simple metrics such as the average pixel value or the squared difference, resulting in insufficient accuracy in reference block matching. This inaccurate similarity assessment can easily lead to visual artifacts during the encoding process, such as block artifacts and texture distortion, significantly reducing the subjective visual quality of the reconstructed video.

[0061] With the widespread adoption of ultra-high-definition video applications and the increasing demands of users for image quality, how to overcome the limitations of existing similarity measurement methods and achieve more accurate reference block matching has become a key technical challenge that urgently needs to be addressed to improve video coding quality.

[0062] This application introduces the Pearson correlation coefficient into the similarity measurement process of the coding model, providing the coding model with the ability to perceive the credibility of the reference region. This can significantly improve the subjective quality of the reconstructed signal at a level of approximate bandwidth saving and objective quality loss. It can effectively guide the coding model to distribute distortion in visually insensitive areas and reduce the probability of visual artifacts such as ringing, bleeding, and color cast.

[0063] To address the aforementioned issues, this specification provides a video coding method in one or more embodiments. By introducing the pixel distribution correlation between the target coding region and each reference region when evaluating the reconstruction loss between the target coding region and any reference region, the method deeply mines the similarity of pixel distribution features between the target coding region and each reference region, thereby providing a credibility-aware capability for reference region discrimination. At approximately the same level of bandwidth saving and objective quality loss, the subjective quality of video coding can be significantly improved. This effectively guides the coding model to distribute distortion in visually insensitive areas, reducing the probability of visual artifacts such as ringing, bleed, and color cast.

[0064] This specification provides a video encoding method, and also relates to a video encoding apparatus, a computing device, a computer-readable storage medium, and a computer program product, which will be described in detail in the following embodiments.

[0065] See Figure 2 , Figure 2 A flowchart of a video encoding method provided in one embodiment of this specification is shown, including the following specific steps:

[0066] Step 202: Obtain the target coding region and each reference region in the video to be encoded.

[0067] The video to be encoded is the video that needs to be encoded. For example, after recording a video, if you want to store or transmit the recorded video, you need to encode the video; then the recorded video is the video to be encoded.

[0068] The target encoding region is the region that needs to be encoded. It can be an image block in any frame of the video to be encoded, such as an image block in the third frame of the video to be encoded (e.g., an 8×8 pixel block); it can also be any frame of the video to be encoded, such as the fourth frame of the video to be encoded; or it can be a region in any frame of the video to be encoded that is larger than the above image blocks. For example, the target encoding region can be a 32×32 pixel search window containing multiple 16×16 pixel image blocks.

[0069] The reference region is a candidate region used for matching and prediction of the target coding region. The size of the reference region is the same as the target coding region (both length and width are the same), and there are usually multiple reference regions. For example, if the target coding region is the 3rd frame of the video to be encoded, the reference regions could be the 1st, 2nd, 4th, and 5th frames of the video. As another example, if the target coding region exists in the 2nd frame of the video, and it is a 16×16 pixel image block, then the reference regions corresponding to the target coding region could be multiple 16×16 pixel image blocks in the 1st frame, and these multiple image blocks would be located within the same search region.

[0070] Optionally, the target coding region and each reference region can be obtained by initializing each frame of the video to be encoded using the encoder. For example, during intra-frame prediction, the neighboring pixels of the reconstructed target coding region are used as reference regions, and during inter-frame prediction, the pixel regions in the encoded reference frames are used as reference regions for the target coding region. Optionally, the target coding region and each reference region can be obtained through manually constructed instructions. This application does not limit the method for obtaining the target coding region and each reference region.

[0071] For example, when the target coding region and each reference region in the video to be encoded are obtained, and the target coding region and each reference region are in different frames, one option is to define a rectangular area of ​​fixed size (e.g., ±64 pixels) centered on the initial position of the target coding region in the reference frame as a candidate reference region. A coarse search (e.g., integer pixel steps) is performed first, followed by a fine search (1 / 2 or 1 / 4 pixel precision) near the matching point. For example, for the target coding region (x = 100, y = 80), the search window is (x: 36–164, y: 16–144). Within this window, the sum of absolute differences (SAD) or the sum of absolute transformed differences (SATD) is calculated to find the optimal reference region. When the target coding region and each reference region are in the same frame, one option is to refer to the reconstructed pixels in the top row and the left column for an N×N block. If some reference pixels are unavailable (e.g., located at frame boundaries), padding or default values ​​are used.

[0072] In step 202, the target coding region and each reference region in the video to be encoded are obtained, which provides a data foundation for subsequent reconstruction of the target coding region based on each reference region.

[0073] Step 204: Reconstruct the target coding region based on each reference region and determine the reconstruction loss corresponding to each reference region.

[0074] In the video reconstruction process, pixel information from a reference region is used to generate approximate data (i.e., "reconstructed data") for the target coding region through motion compensation or intra-frame prediction. This reduces data redundancy during encoding and improves compression efficiency. For example, if the target coding region in the current frame is a 16×16 pixel block, a preferred matching block (i.e., the target reference region) is found in the reference region (e.g., a 32×32 pixel search window) of the previous frame image, with a motion vector of (3, -2). The encoder records this motion vector and calculates the residual (the original value of each pixel in the target coding region minus the value of each pixel in the preferred matching block). The residual is then encoded after DCT transformation and quantization. The encoder's decoder extracts the prediction block from the reference frame based on the motion vector and adds the decoded residual to complete the reconstruction.

[0075] Reconstruction loss is a composite metric in video coding that considers both reconstruction quality and coding efficiency. It is typically expressed as Rate-Distortion Cost, and its formula is: J = D + λR. Here, J is the rate-distortion cost, D is the reconstruction distortion information, λ is the Lagrange multiplier, and R is the bitrate. Optionally, on resource-constrained devices, decoding complexity can also be considered as a factor in the rate-distortion cost, with the formula: J = D + λR + μC. Here, C is the decoding complexity, representing the computational resources required by the decoder to process the current coding block. The unit can be CPU cycles, processor time, power consumption, etc., and is not limited here.

[0076] For example, suppose the target coding region is C, and each reference region is B. i (i represents different reference regions) are B respectively. 1 B 2 The reconstruction distortion information corresponding to each reference region is D. 1 =180, D 2 =170, and the corresponding bitrates for each reference region are R 1 =9, R 2 =15, λ is 12.5, then the reconstruction loss corresponding to each reference region is J 1 =180 + 12.5 × 9 = 292.5, J 2 =170 + 12.5 × 9 = 282.5.

[0077] In step 204, the target coding region is reconstructed based on each reference region, and the reconstruction loss corresponding to each reference region is determined. This yields the overall cost of coding efficiency and coding quality when the target coding region is coded with reference to each reference region. This provides a data basis for subsequently adjusting the reconstruction loss corresponding to each reference region by utilizing the pixel distribution correlation between the target coding region and each reference region.

[0078] Step 206: Based on the pixel information of the target coding region and the pixel information of each reference region, determine the pixel distribution correlation between the target coding region and each reference region.

[0079] Pixel information consists of the luminance value, chrominance value, and Red, Green, and Blue (RGB) parameters of each pixel in the target coding region or each reference region. It is used to calculate the pixel distribution correlation between the target coding region and each reference region. For example, for YUV format video, pixel information may include the values ​​of the luminance component Y and the chrominance components U / V.

[0080] Pixel distribution correlation refers to the degree of consistency of pixel information between the target coding region and each reference region.

[0081] For example, suppose the target coding region is C, and each reference region is B. i (i represents different reference regions), the target coding region is C and each reference region is B. i The correlation of pixel distributions between them can be represented by the Pearson correlation coefficient, and the specific calculation process is as follows:

[0082] First, calculate the target coding region C and each reference region B. i Average pixel value: Then calculate the target coding region C and each reference region B. i Pearson correlation coefficient between Where N represents regions C and B i The number of pixels in C x,y These are the original values ​​of pixels at each position in the target encoding region C. Reference region B i The original values ​​of each pixel at each position in the target encoding region or reference region, where x and y represent the horizontal and vertical coordinates of each pixel in the target encoding region or reference region, respectively.

[0083] In step 206, the pixel distribution correlation between the target coding region and each reference region is obtained, providing a data basis for subsequent evaluation of the reconstruction loss between the target coding region and any reference region.

[0084] Step 208: For any reference region, based on the pixel distribution correlation corresponding to any reference region, adjust the reconstruction loss between the target coding region and any reference region to obtain the adjusted reconstruction loss corresponding to each reference region.

[0085] Reconstruction loss is a core indicator of the quality of a quantized coding mode. It reflects the combined characteristics of compression efficiency (bit rate) and reconstruction quality (distortion) during the compression process. By quantizing the gains and losses of compression, it guides the encoder to find the optimal solution between "as few bits as possible" and "as high quality as possible." For example, reconstruction loss is the rate-distortion reconstruction cost. Taking an H.265 encoder as an example, the calculation steps for the reconstruction loss between the block to be encoded (target coding region) and the reference block (any reference region) are as follows:

[0086] Generate multiple prediction modes: Test all possible prediction modes (such as different reference directions during intra-frame prediction, different motion vectors during inter-frame prediction, etc.).

[0087] Prediction and residual calculation: Generate a prediction block P for the block to be encoded, and calculate the residual R = the original value of each pixel in the block to be encoded - the value of each pixel in the prediction block P.

[0088] Transformation and quantization: Perform DCT transformation and quantization on the residuals to obtain the quantization coefficients q.

[0089] Rate estimation: The number of bits R required for encoding MV, encoding mode, and residual is estimated through Context-Adaptive Binary Arithmetic Coding (CABAC).

[0090] Reconstruction and distortion calculation: The quantization coefficient q is dequantized and inversely transformed to obtain the reconstruction block, and the distortion D (such as SSE) is calculated.

[0091] Cost comparison: Calculate the reconstruction loss J = D + λR for each mode, and select the encoding mode corresponding to the minimum reconstruction loss.

[0092] Based on the pixel distribution correlation corresponding to any reference region, the reconstruction loss between the target coding region and any reference region is adjusted. This adjustment corrects the overall evaluation of the reference region's quality by modifying the reconstruction loss value according to the pixel distribution correlation between the target coding region and any reference region. The adjusted reconstruction loss is the reconstruction loss after correcting the reconstruction loss between the target coding region and any reference region based on the pixel distribution correlation. For example, if the pixel distributions of the target coding region and any reference region are highly correlated, then even if the original reconstruction loss is large, that reference region may still be a good reference region because the overall pixel brightness in that reference region may shift due to factors such as illumination changes, but the pixel distribution structure in that reference region is similar to the pixel distribution structure in the target coding region. Conversely, if the pixel distribution correlation between the target coding region and any reference region is very low, then even if the reconstruction loss is small, it may be due to accidental matching and is not reliable.

[0093] For example, based on the pixel distribution correlation corresponding to any reference region, the reconstruction loss cost between the target coding region and any reference region is adjusted, and the specific calculation expression is as follows: Here, a and b are coefficients used to adjust the model. a can be determined based on the quantization coefficient q and the size of the encoding unit (width w and height h). For example, a = (w·h) / (q << 2), and can be a = 1.0, b = 0.5, etc.

[0094] In step 208, based on the pixel distribution correlation corresponding to any reference region, the reconstruction loss between the target coding region and any reference region is adjusted to obtain the adjusted reconstruction loss for each reference region. This avoids misjudging the reference region when simply using the reconstruction distortion information between the target coding region and any reference region to determine the reconstruction loss. By comprehensively considering the reconstruction distortion information and pixel distribution correlation corresponding to the reference regions, the structural relationship between the target coding region and each reference region can be further considered on the basis of pixel-level matching, achieving more intelligent reference region selection and laying the foundation for accurate determination of the target reference region from each reference region in the subsequent process.

[0095] Step 210: Based on the adjusted reconstruction loss corresponding to each reference region, determine the target reference region from each reference region, and perform encoding processing based on the target encoding region and the target reference region.

[0096] The target reference region is any reference region whose reconstruction loss with respect to the target coding region is less than a preset threshold. Alternatively, the target reference region can be the top M reference regions with the smallest reconstruction losses after sorting the reference regions with respect to the target coding region. For example, if the reference regions are B1, B2, and B3, and the target coding region is C1, and the reconstruction loss between B1 and C1 is 150, the reconstruction loss between B2 and C1 is 200, and the reconstruction loss between B3 and C1 is 220, then the target reference region is B1.

[0097] Encoding processing involves adjusting pixels in either the target coding region or the target reference region, or adjusting encoder parameters, based on the target coding region and the target reference region. For example, in MCTF filtering, when the target coding region and multiple target reference regions are determined, the pixels in each target reference region are adjusted according to the target coding region to ensure smooth video transitions and reduce block artifacts and glitches. As another example, in inter-frame prediction, motion estimation is performed based on the target reference region and the target coding region to obtain the motion vector between them. Pixels are then copied (or interpolated) from the target reference region based on the motion vector to generate the predicted value for the target coding region. The predicted value of the pixel in the target coding region is then subtracted from the original value of the pixel in the target coding region to obtain the residual. This residual is then transformed, quantized, and written into the bitstream along with the motion vector. For example, encoder parameters can be adjusted as follows: if the residual between the target reference region and the target coding region is small (high matching degree), reduce the quantization step size to retain more details (suitable for static or smooth regions); if the residual between the target reference region and the target coding region is extremely small (near lossless), increase the quantization parameter to save bit rate. If the residual between the target reference region and the target coding region is very small, prioritize Skip mode or Merge mode (directly reuse MV, saving bit rate).

[0098] For example, in the inter-frame prediction stage of video coding, the target coding region is a 16×16 pixel block in the third frame (a P-frame), and the target reference region is a 16×16 pixel block in the second frame. The coding process then involves motion estimation of the 16×16 pixel blocks in the second and third frames. Assuming the motion vector between them is (3, 5), the value of the target coding region is extracted from a position 3 pixels to the right and 5 pixels upwards from the target coding region position. Further, the residual between the original value of the pixel in the target coding region and the predicted value of the pixel in the target coding region is calculated. Finally, the residual is transformed, quantized, and written into the bitstream along with the motion vector.

[0099] Optionally, in this embodiment, the target coding region and each reference region can be located in the same frame of the video to be encoded. For example, when the image containing the target coding region is in a scene transition, i.e., when the image is an I-frame, intra-frame coding is performed on the target coding region, and each reference region is a neighboring pixel region of the target coding region. In this case, the above video coding method further includes:

[0100] The target coding region is reconstructed based on each reference region to obtain the reconstructed region corresponding to each reference region, and the reconstruction loss corresponding to each reference region is determined.

[0101] Based on the pixel information of the target coding region and the pixel information of the reconstructed regions corresponding to each reference region, the pixel distribution correlation between the target coding region and the reconstructed regions corresponding to each reference region is determined.

[0102] For any reference region, based on the pixel distribution correlation between the target coding region and the reconstructed regions corresponding to each reference region, the reconstruction loss between the target coding region and any reference region is adjusted to obtain the adjusted reconstruction loss corresponding to each reference region.

[0103] Based on the adjusted reconstruction loss corresponding to each reference region, the target reference region is determined from each reference region, and encoding processing is performed based on the target coding region and the target reference region.

[0104] In the embodiments of this specification, a target coding region and various reference regions are obtained from the video to be encoded. The target coding region is reconstructed based on each reference region, and the reconstruction loss corresponding to each reference region is determined. Based on the pixel information of the target coding region and the pixel information of each reference region, the pixel distribution correlation between the target coding region and each reference region is determined. This completes the comparison of the pixel distribution correlation between the target coding region and each reference region, providing data support for the subsequent adjustment of the reconstruction loss between the target coding region and any reference region based on the pixel distribution correlation corresponding to any reference region. For any reference region, the reconstruction loss between the target coding region and any reference region is adjusted based on the pixel distribution correlation corresponding to any reference region, resulting in the adjusted reconstruction loss corresponding to each reference region. Based on the adjusted reconstruction loss corresponding to each reference region, a target reference region is determined from each reference region, and encoding processing is performed based on the target coding region and the target reference region. This allows the pixel distribution correlation between the target coding region and any reference region to be introduced into the measurement of the similarity between the two based on the reconstruction loss corresponding to each reference region, achieving accurate selection of the reference region most similar to the target coding region. This solves the quality problems such as visual artifacts that occur when encoding the target coding region based on the reference region.

[0105] In one optional embodiment of this specification, step 204 includes the following specific steps:

[0106] The target coding region is reconstructed based on each reference region to obtain the reconstructed target coding region corresponding to each reference region.

[0107] Based on the pixel information of each reconstructed target coding region and the pixel information of the target coding region, determine the reconstruction distortion information corresponding to each reference region;

[0108] Obtain the bitrate corresponding to each reference region, and determine the reconstruction loss corresponding to each reference region based on the reconstruction distortion information and bitrate corresponding to each reference region.

[0109] Reconstructing the target coding region involves generating pixel blocks that are as similar as possible to the original target coding region using prediction techniques, in order to approximate the pixel content of the original target coding region. These prediction techniques include inter-frame prediction and intra-frame prediction. Inter-frame prediction is generated by searching for the target reference region in the reference frame (motion estimation + motion compensation), while intra-frame prediction utilizes the already encoded neighboring pixels in the current frame for prediction.

[0110] The reconstructed target coding region is the pixel region where pixel information has been altered. Reconstruction distortion information is a statistical characteristic of the visual quality degradation caused by compression during quantization encoding, including blockiness and blurriness, used in rate-distortion optimization trade-offs. For example, if the loss of high-frequency details leads to image blurring, reconstruction distortion information can record the gradient change magnitude in that region. Reconstruction distortion information reflects the amount of visual information lost during encoding; essentially, it measures the degree of deviation between the reconstruction result at the decoding end and the original content.

[0111] For example, suppose the target coding region is C, and each reference region is B. i (i represents different reference regions), reconstructing the distorted information into the target coding region C and each reference region B. i After reconstruction for reference, the target coding region C and each reference region B i Reconstruction distortion value D between i The specific calculation formula is as follows: Where N is the region C and B i The number of pixels in C x,y C' represents the original value of the pixel at that location. x,y This represents the reconstructed value of the pixel at that location, where x and y represent the horizontal and vertical coordinates of each pixel in the target coding region or reference region, respectively.

[0112] For example, the process of generating approximate blocks through inter-frame prediction (motion compensation) is as follows:

[0113] Motion estimation: Within the search window of the reference frame, find the target reference region that is most similar to the target coded region (e.g., using algorithms such as SAD / SATD). Record the motion vector (MV) to represent the displacement relationship between the target reference region and the target coded region.

[0114] Motion compensation: Generate an approximate block P by copying (or interpolating) pixels from the reference frame based on the MV.

[0115] Assuming the target coding region is located at (x = 100, y = 80), and the preferred matching block is located at (x = 103, y = 75) in the reference frame, then the motion vector MV = (3, -5), and the approximate block P is the pixel at (103, 75) in the reference frame.

[0116] The process of generating approximate blocks through intra-frame prediction is as follows:

[0117] The target reference region is predicted using the neighboring pixels (above and to the left) that have been reconstructed in the current frame. For example, if the brightness value of the pixel above the target reference region is 50 and that to the left is 60, then the approximate block generated using the average value of the neighboring pixels in the target reference region will have a total pixel value of (50+60) / 2 = 55.

[0118] Bitrate is the amount of binary data transmitted or stored per unit of time, directly determining the compression efficiency and visual quality of a video. For example, if a 10-second 1080p video (H.265) is 25MB in size, then the bitrate is (25×8) / 10 = 20Mbps.

[0119] The method for obtaining the bitrate corresponding to each reference area can be set according to the actual application scenario. For example, in network video and video conferencing scenarios, the bitrate is usually 1.5 to 4 Mbps, while in game and sports live streaming scenarios, the bitrate is usually 6 to 12 Mbps.

[0120] The method for determining the reconstruction loss of each reference region based on the reconstruction distortion information and bit rate has been introduced above and will not be repeated here.

[0121] In the embodiments of this specification, the reconstruction distortion information corresponding to each reference region is determined based on the pixel information of each reconstructed target coding region and the pixel information of the target coding region. This allows us to obtain the difference in pixel information between the target coding region after reconstruction with each reference region and before reconstruction, providing a data basis for subsequent accurate evaluation of the reconstruction loss between the target coding region and any reference region.

[0122] In one optional embodiment of this specification, step 206 includes the following specific steps:

[0123] For the first reference region, based on the pixel values ​​of each pixel in the target coding region and the pixel values ​​of each pixel in the first reference region, the covariance between the target coding region and the first reference region, as well as the first standard deviation of the target coding region and the second standard deviation of the first reference region are determined, wherein the first reference region is any reference region.

[0124] Based on the covariance, the first standard deviation, and the second standard deviation, the pixel distribution correlation between the target coding region and the first reference region is determined.

[0125] The first standard deviation represents the degree of dispersion of pixel values ​​in the target coding region.

[0126] The second standard deviation represents the degree of dispersion of pixel values ​​in the first reference region.

[0127] Covariance is the degree of consistency in the pixel value change trends of corresponding pixels between the target coding region and the first reference region.

[0128] For example, suppose the pixel value of each pixel in the target coding region X is The pixel values ​​of each pixel in the first reference region Y are Then the mean μ of the target coding region X X for The mean μ of Y in the first reference region Y for

[0129] According to the covariance formula cov(X,Y)=E|(X-μ) X )(Y-μ Y )|, we can get

[0130] According to the standard deviation formula The first standard deviation of the target coding region X can be obtained. Similarly, the second standard deviation of the first reference region is

[0131] After obtaining the covariance, first standard deviation, and second standard deviation, the pixel distribution correlation between the target coding region and the first reference region can be calculated using the Pearson correlation coefficient formula: Where ρ X,Y Let X be the pixel distribution correlation between the target coding region X and the first reference region Y. Substituting the values ​​calculated above, we can obtain... The pixel distribution correlation between the target coding region X and the first reference region Y is close to 1, indicating that the pixel distributions of the two regions are almost completely linearly correlated.

[0132] In the embodiments of this specification, by determining the covariance between the pixel values ​​of each pixel in the target coding region and the pixel values ​​of each pixel in the first reference region, the consistency of the pixel value change trends in the two regions can be determined. By determining the first standard deviation of the pixel values ​​of each pixel in the target coding region and the second standard deviation of the pixel values ​​of each pixel in the first reference region, the dispersion of the pixel values ​​in the two regions can be reflected. Finally, based on the covariance, the first standard deviation, and the second standard deviation, the pixel distribution correlation between the first reference region and the target coding region is determined, which can provide a data foundation for subsequent evaluation of the reconstruction loss between the target coding region and any reference region.

[0133] In one optional embodiment of this specification, the pixel distribution correlation between the target coding region and each reference region is applied to the rate-distortion optimization (RDO) process of video coding, and step 208 includes the following specific steps:

[0134] Obtain the quantization coefficients of the coding model and the size of the coding unit in the coding model;

[0135] Based on the quantization coefficients of the coding model and the size of the coding unit, the adjustment coefficients used to adjust the coding model are determined;

[0136] Based on the adjustment coefficient and the correlation of pixel distribution corresponding to any reference region, the reconstruction loss between the target coding region and any reference region is adjusted to obtain the adjusted reconstruction loss corresponding to each reference region.

[0137] The quantization factor (QP) of the encoding model is a key parameter controlling the compression ratio and quality. A larger QP value results in a larger quantization step size and a higher compression ratio, but also a greater loss of quality. The QP value is typically an integer; for example, in H.264 and H.265, the QP ranges from 0 to 51.

[0138] The adjustment coefficients for the coding model are dynamic weighting parameters used to intelligently adjust the influence of pixel distribution correlation on the reconstruction loss based on the current coding configuration (QP and CU sizes). For example, a and b are coefficients used to adjust the coding model. a can be valued based on the quantization coefficient q and the size of the coding unit (width w and height h), for example, a = (w·h) / (q << 2), and can take fixed values ​​such as a = 1.0 and b = 0.5.

[0139] A coding unit is the basic processing unit in video coding. For example, in H.265 / HEVC, the size of a coding unit can be 64×64 pixels, 32×32 pixels, 16×16 pixels, or 8×8 pixels. Optionally, the quantization coefficients of the coding model and the size of the coding units in the coding model can be obtained by reading the configuration information of the coding model or by monitoring the real-time coding status of the coding model.

[0140] Specifically, based on the pixel distribution correlation corresponding to any reference region, the reconstruction loss between the target coding region and any reference region is adjusted to obtain the adjusted reconstruction loss corresponding to each reference region, which is calculated using the following formula: Here, a and b are coefficients used to adjust the model. a can be determined based on the quantization coefficient q and the size of the encoding unit (width w and height h), for example, a = (w·h) / (q << 2), and can take fixed values ​​such as a = 1.0 and b = 0.5. R i Let D be the bitrate corresponding to the i-th reference region. iLet λ represent the reconstruction distortion information corresponding to the i-th reference region, λ be the Lagrange multiplier (used to adjust the weights of bit rate and distortion), and ρ be the pixel distribution correlation corresponding to the i-th reference region.

[0141] For example, assuming a = 1.0, b = 0.5, λ = 10, the pixel distribution correlation corresponding to the target reference region is 0.9, the reconstruction distortion information corresponding to the target reference region is 30, and the required reconstruction loss corresponding to the target reference region is no greater than 80, then substituting the above parameters into... The code rate corresponding to the target reference region needs to be greater than 8.2 bits. Therefore, the code rate corresponding to the target reference region needs to be set to 9 bits to ensure that the reconstruction loss between the target coding region and the target reference region is not greater than 80.

[0142] In the embodiments of this specification, by considering the similarity of pixel distribution between the target coding region and any reference region when judging reconstruction loss, the coding model is provided with the ability to perceive the credibility of the reference region. This can significantly improve the subjective quality of the reconstructed signal at a level of approximate bandwidth saving and objective quality loss, effectively guide the coding model to make decisions on coding mode and coding parameters, distribute distortion in visually insensitive areas, and reduce the probability of visual artifacts such as ringing, bleed, and color cast.

[0143] In one optional embodiment of this specification, the pixel distribution correlation between the target coding region and each reference region is applied to the Lookahead stage of video coding, that is, during the process of judging the similarity of the video to be encoded at the frame level (between different image frames). At this time, the target coding region is the target image in the video to be encoded, and each reference region is a reference image in the video to be encoded that is associated with the target image.

[0144] Based on the adjusted reconstruction loss corresponding to each reference region, a target reference region is determined from each reference region, and encoding processing is performed based on the target encoding region and the target reference region, including the following specific steps:

[0145] Based on the adjusted reconstruction loss corresponding to each reference image, the target reference image and coding parameters are determined from each reference image. The coding parameters are used to encode the target image based on the target reference image.

[0146] The target image is any frame in the video to be encoded, such as the 3rd frame. The reference image is an image from a different frame than the target image, such as the 2nd frame, the 4th frame, etc.

[0147] The target reference image is an image whose reconstruction loss with respect to the target image is less than a preset threshold. For example, a reference image whose reconstruction loss with respect to the target image is less than 0.1 is determined as the target reference image.

[0148] The encoding parameters are frame type and bitrate. Frame types include I-frames (Intra-coded Frames, keyframes), P-frames (Predictive Frames), and B-frames (Bi-directional Frames). I-frames are independent frames, not dependent on other frames, and contain complete image information; video playback or random access must begin with an I-frame. P-frames are generated through motion compensation based on the preceding frame (I-frame or P-frame). P-frames improve compression efficiency and reduce redundant information from continuous motion (such as camera panning); decoding requires first decoding the reference frame they depend on. B-frames rely on two preceding frames (I-frames or P-frames) for motion compensation; the subsequent reference frame must be obtained first, making them unsuitable for live streaming, but they can significantly improve compression efficiency, especially suitable for static or complex motion scenes.

[0149] For example, the video to be encoded is a video containing a moving scene, comprising 10 frames (frames 1-10). Assuming frame 4 is the target image, and frames 2, 3, 5, and 6 are reference images for the target image, the reconstruction loss between frames 2, 3, 5, and 6 and frame 4 is determined respectively. If the reconstruction losses for frames 2, 3, 5, and 6 are 0.8, 0.8, 0.9, and 0.9 respectively, it indicates a large reconstruction loss between frame 4 and the reference images. Frame 4 is likely at a scene transition point, therefore it cannot be referenced by its preceding or following frames. The frame type of frame 4 needs to be set to I-frame (forced scene transition), meaning frame 4 requires intra-frame coding at a high bitrate. In this case, none of the reference images associated with frame 4 can be used as its target reference image; therefore, the target reference image for frame 4 is itself.

[0150] For example, the video to be encoded is a family dinner scene, containing 10 frames (frames 1-10). Assuming frame 4 is the target image, and frames 2, 3, 5, and 6 are reference images for the target image, the reconstruction loss between frames 2, 3, 5, and 6 and frame 4 is determined respectively. Assuming the reconstruction losses for frames 2, 3, 5, and 6 are 0.1, 0.2, 0.1, and 0.2 respectively, this indicates that the reconstruction loss between frame 4 and each reference image is very small. Frame 4 can be inter-frame coded using its preceding or following frames as references. Therefore, the frame type of frame 4 needs to be set to B-frame (low motion, high compression), meaning frame 4 can be inter-frame coded, and a low bitrate should be set. In this case, since the reconstruction loss corresponding to the second frame image is less than that corresponding to the third frame image, and the reconstruction loss corresponding to the fifth frame image is less than that corresponding to the sixth frame image, determining the second and fifth frames as the target reference images for the fourth frame image can reduce the residual when performing inter-frame coding on the fourth frame image, thereby improving the compression efficiency and coding quality of the subsequent coding process, and providing a higher quality reference frame for subsequent frames.

[0151] In the embodiments of this specification, by determining the target reference image and coding parameters from each reference image based on the adjusted reconstruction loss corresponding to each reference image during the image preprocessing process (Lookahead stage) of video encoding, the frame type, reference frame and bit rate can be accurately assigned to each frame image, which can improve the compression efficiency and better coding quality of the subsequent coding process, and provide higher quality reference frames for subsequent frames, thereby improving the quality of video encoding.

[0152] In one optional embodiment of this specification, the pixel distribution correlation between the target coding region and each reference region is applied to the motion search stage of video coding, that is, to judge the similarity between image blocks in each image, so as to find the reference block most similar to the block to be encoded. At this time, the target coding region is the block to be encoded in the target image, and the reference region is the reference block of the search region in the reference image corresponding to the target image.

[0153] Based on the adjusted reconstruction loss corresponding to each reference region, a target reference region is determined from each reference region, and encoding processing is performed based on the target encoding region and the target reference region, including the following specific steps:

[0154] Based on the adjusted reconstruction loss corresponding to each reference block, the target reference block is determined from each reference block in the search area; the motion vector between the block to be encoded and the target reference block is determined; based on the motion vector, the block to be encoded and the target reference block are aligned to obtain the aligned block to be encoded and the target reference block; encoding is performed based on the aligned block to be encoded and the target reference block.

[0155] The target image is any frame in the video to be encoded, such as the third frame in the video to be encoded.

[0156] The block to be encoded is the image block in the target image that needs to be encoded, such as an image block with a size of 8×8 pixels.

[0157] The reference image is an image whose similarity to the target image is greater than a preset threshold, such as an image with a similarity greater than 0.8. A reference block is an image patch in the reference image with the same size as the block to be encoded. There can be multiple reference blocks, and these blocks may or may not intersect. The search region is the pixel region in the reference image that contains multiple reference blocks. The target reference block is the image patch among all reference blocks that is most similar to the block to be encoded, i.e., the reference block with the minimum reconstruction loss.

[0158] Aligning the block to be encoded with the target reference block means:

[0159] The motion vector between the block to be encoded and the target reference block is calculated. This motion vector is then used to "move" the target reference block to the position of the block to be encoded, maximizing the overlap of their spatial contents. Matching the spatial position of the target reference block with the block to be encoded using motion vectors leverages temporal redundancy for efficient compression. Higher alignment accuracy (e.g., sub-pixel motion estimation, multiple reference frames) results in better video coding performance. More precise alignment leads to smaller differences between the block to be encoded and the target reference block, less residual data, and higher compression efficiency. For example, if a car in the current frame moves 10 pixels to the right, MV = (10, 0) can be used to align the car in the reference frame to the current position, with the residual containing only minor noise or lighting variations. Alternatively, if the car moves 3.5 pixels (MV = (3.5, 0), a reference block can be generated using 1 / 2 pixel interpolation, achieving sub-pixel alignment.

[0160] Encoding based on the aligned block to be encoded and the target reference block is equivalent to inter-frame coding of the block to be encoded based on the target reference block.

[0161] Inter-frame coding of the block to be coded based on the target reference block can be performed in one of the following ways: (1) Residual calculation and coding: Residual generation: Calculate the residual between the block to be coded and the predicted block; Transformation and quantization: Perform DCT / DST transformation on the residual and then quantize to reduce the amount of data; Inverse quantization and inverse transformation: Recover the residual data; Predicted block generation: Use the same motion vector and reference frame to obtain the predicted block through motion compensation; Block reconstruction: Add the predicted block and the residual; Deblocking filter (optional): Apply a deblocking filter (such as H.264 Deblocking Filter) to smooth the block boundary; The reconstructed block will be stored in the decoded image buffer (DPB) as the reference frame for subsequent frames.

[0162] Indicative, for reference Figures 3 to 6 , Figure 3 This diagram illustrates a target coding region in a video coding method according to an embodiment of this specification. Figures 4 to 6 The following diagrams illustrate different reference regions in a video coding method provided in one embodiment of this specification. Figure 3 As shown:

[0163] Figure 3 For a single frame of image, Figure 4 , Figure 5 ,and Figure 6 Located in another frame of the image. For example, Figure 3 This is the second frame of the video to be encoded. Figure 4 , Figure 5 and Figure 6 This is the first frame of the video to be encoded. Figure 3 The square dashed box in the image represents the target encoding region. Figure 4 , Figure 5 and Figure 6 The diamond-shaped dashed box represents the search window (search area). The square dashed boxes within the search window represent reference areas corresponding to the target encoding area. Each reference area is located within the search window, and each reference area occupies a different position within the search window. (Reference) Figure 3 It can be seen that, Figure 6 The reference area represented by the square dashed box in the middle is... Figure 5 The target coding regions are most similar, therefore Figure 6 The reference area represented by the square dashed box in the figure is the target reference area.

[0164] In the embodiments of this specification, by determining the target reference block from each reference block in the search region based on the adjusted reconstruction loss corresponding to each reference block, the target reference block most similar to the block to be encoded can be determined from multiple reference blocks. This minimizes the residual between the block to be encoded and the predicted block when encoding the block based on the target reference block, thereby reducing the amount of residual data and improving compression efficiency and encoding quality.

[0165] The pixel distribution correlation between the target reference region and the target coding region mentioned above can be used to adjust the filtering weights of the MCTF during video coding, specifically including the following steps:

[0166] Obtain the target coding region and each reference region in the video to be encoded; based on the pixel information of the target coding region and each reference region, determine the pixel distribution correlation between the target coding region and each reference region respectively;

[0167] Based on the pixel distribution correlation between each target reference region and the target coding region, the weight coefficients of each target reference region are adjusted to obtain the adjusted weight coefficients corresponding to each target reference region.

[0168] Based on the adjusted weight coefficients corresponding to each target reference region, MCTF filtering is performed on the target coding region.

[0169] MCTF filtering is a temporal filtering process performed after aligning multiple frames through motion compensation. It can reduce temporal redundancy and enhance compression efficiency. For example, motion compensation is performed on 5 frames within a GOP, and then a weighted average is taken to generate a filtered frame.

[0170] The weight coefficients for each target reference region are the weight values ​​assigned to different spatial / temporal regions in MCTF filtering or weighted prediction, typically adaptively assigned based on region similarity. For example, in temporal filtering, the weight of the motion-compensated center frame is 0.5, and that of adjacent frames is 0.25 each.

[0171] Indicatively, for reference Figure 7 , Figure 7 A schematic diagram of an MCTF filter according to an embodiment of this specification is shown, including:

[0172] Pre-analysis coding: This is the Lookahead process in video coding, which has been described in the above examples and will not be repeated here.

[0173] 1) Motion Estimation: The motion trajectory of each pixel or block between adjacent frames is determined using a fixed-size block matching algorithm and search modes such as Full Search, Three-Step Search, and Pyramid Search, and recorded as the motion vector of the MCTF. In common encoders, to balance search accuracy and computational complexity, a pyramid structure built with different downsampling rates is generally used for the search, that is, the search is expanded from high downsampling rate to high accuracy.

[0174] Indicatively, Figure 8 A schematic diagram of a pyramid search structure provided in one embodiment of this specification is shown. (Reference) Figure 8 The pyramid search structure includes:

[0175] L0 (original layer), L1 (intermediate layer), and L2 (coarsest layer), each representing an image with a different downsampling factor, are used for hierarchical motion estimation.

[0176] For example, assuming the original frame has a resolution of 16×16 pixels, a 3-layer pyramid (L0, L1, L2) is constructed: L0 (original layer): highest resolution, 16×16 pixels (1× downsampling); L1 (middle layer): L0 is downsampled by 2× to obtain 8×8 pixels; L2 (coarsest layer): L1 is downsampled by 2× again to obtain 4×4 pixels.

[0177] Assume the true motion vector of the current block (16×16) in the reference frame is (12, 8) (i.e., 12 pixels to the right and 8 pixels down). The pyramid search process is as follows:

[0178] (1) L2 layer (4×4 resolution)

[0179] Search range: Since L2 is 4× downsampling, the original search range of ±16 pixels (if a full search requires 32×32=1024 times) is equivalent to ±4 pixels (4×4=16 times) in the L2 layer.

[0180] Search strategy: Perform a full search (or fast search) at L2 layer, assuming a preferred matching fast offset (3, 2) is found.

[0181] Mapping back to L1 layer: Since L2 is 4× downsampling, MV needs to be ×2 to get (6, 4).

[0182] (2) L1 layer (8×8 resolution)

[0183] Search Center: Centered on the previous layer MV(6,4), perform small-scale corrections (such as ±2 pixels) in the L1 layer.

[0184] Search process: Search within the range (6±2, 4±2) (5×5=25 calculations in total), assuming a better offset (7, 3) is found. Map back to L0 layer: L1 is 2× downsampling, MV needs to be ×2, resulting in (14, 6).

[0185] (3) L0 layer (16×16 resolution)

[0186] Search Center: Centered on MV(14,6) of L1 mapping, perform a fine-grained search (e.g., ±1 pixel).

[0187] Final result: The optimal match (12, 8) was found by searching within the range of (14±1, 6±1) (a total of 3×3=9 calculations), which is consistent with the actual MV.

[0188] The meanings of each step in motion estimation are as follows:

[0189] Filtered reference frame: The filtered reference frame is a reference frame that has been processed by deblocking filtering or Sample Adaptive Offset (SAO) to improve prediction quality and reduce the impact of reference frame noise on motion compensation. For example, the VVC decoder uses the deblocked I-frame as the reference frame for the B-frame.

[0190] Double reference frame: A double reference frame is a high-resolution reference frame generated through interpolation, supporting sub-pixel motion estimation. For example, bilinear interpolation is performed on the reference frame to generate reference data with 1 / 2 pixel precision.

[0191] Code block partitioning: Code block partitioning is the process of recursively dividing the largest coding unit into sub-blocks of different sizes. For example, VVC supports quadtree + multi-type tree partitioning to adapt to complex textures.

[0192] Integer pixel motion estimation: Integer pixel motion estimation is the process of searching for the best matching fast at integer pixel precision. It is usually used in the coarse search stage to provide the initial MV for subpixel search.

[0193] 1 / 2 pixel search: 1 / 2 pixel search is a process of refining motion vectors at half pixel precision, and improving motion compensation accuracy by interpolating reference frames.

[0194] Filtered reference frame: The filtered reference frame is a reference frame that has been processed by deblocking filtering or Sample Adaptive Offset (SAO) to improve prediction quality and reduce the impact of reference frame noise on motion compensation. For example, the VVC decoder uses the deblocked I-frame as the reference frame for the B-frame.

[0195] Double reference frame: A double reference frame is a high-resolution reference frame generated through interpolation, supporting sub-pixel motion estimation. For example, bilinear interpolation is performed on the reference frame to generate reference data with 1 / 2 pixel precision.

[0196] Code block partitioning: Code block partitioning is the process of recursively dividing the largest coding unit into sub-blocks of different sizes. For example, VVC supports quadtree + multi-type tree partitioning to adapt to complex textures.

[0197] Integer pixel motion estimation: Integer pixel motion estimation is the process of searching for the best matching fast at integer pixel precision. It is usually used in the coarse search stage to provide the initial MV for subpixel search.

[0198] 1 / 2 pixel search: 1 / 2 pixel search is a process of refining motion vectors at half pixel precision, and improving motion compensation accuracy by interpolating reference frames.

[0199] Temporal filtering blocks: Temporal filtering blocks are image blocks that participate in multi-frame temporal filtering. They typically require motion compensation alignment and are used for weighted fusion in MCTF. For example, a weighted average is calculated after motion compensation of 16×16 blocks from three consecutive frames.

[0200] Reference block acquisition: Reference block acquisition is the process of extracting a matching block from a reference frame based on motion vectors. For example, an 8×8 pixel block is extracted from the reference frame based on MV(2, -3) as the prediction value of the current block.

[0201] Block prediction cost: Block prediction cost is a metric that measures the difference between the predicted block and the current block, and is used for RDO decision-making.

[0202] Reference frame distance: The reference frame distance is the frame interval between the current frame and the reference frame. The larger the distance, the more difficult motion estimation is usually. For example, when a P-frame references the previous two frames, the reference frame distance of the P-frame is 2.

[0203] Reference frame weights: Reference frame weights are the weighting coefficients of each reference frame in multi-reference frame prediction, which improve prediction accuracy through weighted fusion. For example, in bidirectional prediction, the forward reference frame weight is 0.7 and the backward reference frame weight is 0.3.

[0204] Filter strength: Filter strength is a parameter that controls the intensity of deblocking filtering or temporal filtering, and is dynamically adjusted based on the quantization parameter or block boundary gradient. For example, when QP is large, the deblocking filter strength is set to 3 to eliminate obvious blocking effects.

[0205] Weighted fusion: Weighted fusion is a method that combines multiple prediction blocks according to their weights to generate the final prediction value. It is used for bidirectional prediction or MCTF. For example, in bidirectional prediction, forward and backward reference blocks are fused, with each block having a 50% weight.

[0206] Temporal filtering blocks: Temporal filtering blocks are image blocks that participate in multi-frame temporal filtering. They typically require motion compensation alignment and are used for weighted fusion in MCTF. For example, a weighted average is calculated after motion compensation of 16×16 blocks from three consecutive frames.

[0207] 2) Motion Compensation: Adjusting pixel values ​​in the current frame based on motion information, using motion vectors to locate and compensate for pixel positions within the frame, aligning homogeneous blocks in the current frame and the reference frame. Motion compensation includes:

[0208] Reference block acquisition: Reference block acquisition is the process of extracting a matching block from a reference frame based on motion vectors. For example, an 8×8 pixel block is extracted from the reference frame based on MV(2, -3) as the prediction value of the current block.

[0209] Block prediction cost: Block prediction cost is a metric that measures the difference between the predicted block and the current block, and is used for RDO decision-making.

[0210] Reference frame distance: The reference frame distance is the frame interval between the current frame and the reference frame. The larger the distance, the more difficult motion estimation is usually. For example, when a P-frame references the previous two frames, the reference frame distance of the P-frame is 2.

[0211] Reference frame weights: Reference frame weights are the weighting coefficients of each reference frame in multi-reference frame prediction, which improve prediction accuracy through weighted fusion. For example, in bidirectional prediction, the forward reference frame weight is 0.7 and the backward reference frame weight is 0.3.

[0212] Filter strength: Filter strength is a parameter that controls the intensity of deblocking filtering or temporal filtering, and is dynamically adjusted based on the quantization parameter or block boundary gradient. For example, when QP is large, the deblocking filter strength is set to 3 to eliminate obvious blocking effects.

[0213] Weighted fusion: Weighted fusion is a method that combines multiple prediction blocks according to their weights to generate the final prediction value. It is used for bidirectional prediction or MCTF. For example, in bidirectional prediction, forward and backward reference blocks are fused, with each block having a 50% weight.

[0214] 3) Temporal filtering: Weights are calculated for reference blocks in different frames using a series of similarity metrics. Bidirectional or unidirectional filtering is then used to filter the motion-compensated frames to remove short-term fluctuations and noise, thereby improving the visual quality of the video.

[0215] For example, the complete process of MCTF filtering is introduced using a 5-frame GOP as an example: 1. Input frame sequence: Assume that the GOP being processed contains 5 frames, arranged in chronological order as: F0→F1→F2→F3→F4, where F2 is the center frame to be filtered, F0 and F1 are forward reference frames, and F3 and F4 are backward reference frames. 2. Motion estimation: For each block of F2 (e.g., 16×16 or 8×8), a preferred matching block is searched in adjacent frames, and the motion vector (MV) is calculated: Forward motion estimation: The block of F2 searches for a matching block in F0 and F1 to obtain the motion vector MV. F2→F0 MV F2→F1 Backward motion estimation: The block of F2 searches for a matching block in F3 and F4 to obtain the motion vector MV. F2→F3 MV F2→F4 Example: Suppose that a block in F2 (coordinates (x, y)) has an offset of (2, 1) from the preferred matching block in F1, then MV F2→F1 = (2, 1). The preferred matching fast offset in F3 is (-1, 3), then MV F2→F3 = (-1, 3). 3. Motion Compensation: Based on the motion vector, extract the matching block from the reference frame and compensate for the motion: Forward Compensation: Extract the prediction block P of F2 from F0 and F1. F0 With PF1 Backward compensation: Extract the prediction block P of F2 from F3 and F4. F3 , with P F4 Example: For MV F2→F1 = (2, 1), extract a 16×16 block from the position (x+2, y+1) of F1 as P F1 For MV F2→F3 = (-1, 3), extract a 16×16 block from position (x-1, y+3) of F3 as P F3 4. Temporal filtering performs weighted fusion of the current block and all motion-compensated prediction blocks in F2. The formula derived for a single pixel is as follows:

[0216]

[0217] Where I0 is the original current pixel, I n For the processed pixels, I r (i) is the reference pixel, w r (i, a) represents the reference weight of the current pixel in the i-th frame among a total of a reference frames.

[0218] Specifically, the reference weights are composed as follows:

[0219]

[0220] Where s0(n) depends on the GOP configuration, s r (i, a) is related to the total number of reference frames and the reference distance:

[0221]

[0222] w a and σ w This is the corresponding correction option, which can be set according to the actual scenario, for example, set to 1.

[0223] The greater the pixel difference, the smaller the weight.

[0224] For the luminance component, the larger the QP and the larger the exponent, the stronger the filter intensity; for the chrominance component, σ... l (QP) = 3 × (QP - 10).

[0225] Weighting strategy: Center frames receive higher weights (e.g., W2 = 0.5) to ensure key information is retained after filtering. Neighboring frames receive lower weights (e.g., W1 = W3 = 0.2) to reduce noise impact. Longer-distance frames receive even lower weights (e.g., W0 = W4 = 0.05) to avoid introducing irrelevant data. Example: Assuming the target pixel value of F2 is 100, the predicted values ​​after motion compensation are: P F0 =95, P F1 =98, P F3 =102, P F4 =105. Using weights W0=0.05, W1=0.2, W2=0.5, W3=0.2, W4=0.05, the target pixel value of F2 after filtering is: 0.05×95+0.2×98+0.5×100+0.2×102+0.05×105=100.1. 5. After the output filtered frame is processed by MCTF, all blocks of F2 are filtered, generating F2. filtered This frame will be used for subsequent encoding or as a reference frame.

[0226] Specifically, based on the pixel distribution correlation between each target reference region and the target coding region, the calculation expression for adjusting the weight coefficients of each target reference region is: wr(i, a)′=wr(i, a)·ρ i Where wr(i, a) are the weight coefficients of each target reference region before adjustment, and ρ i The pixel distribution correlation between each target reference region and the target coding region is represented by wr(i, a)′, which is the adjusted weight coefficient of each target reference region.

[0227] For example, assume that w1, w2, w4, and w5 are the initial weight coefficients of the target reference region in frames 1, 2, 4, and 5, respectively, with values ​​of 0.1, 0.15, 0.15, and 0.1, respectively; and ρ1, ρ2, ρ4, and ρ5 are the pixel distribution correlations between the target reference region in frames 1, 2, 4, and 5 and the target coding region (the region to be filtered) in frame 3, respectively. The values ​​of ρ1, ρ2, and ρ3 are 0.9, 0.9, 0.8, and 0.8 respectively. Therefore, the adjusted weight coefficients for each target reference region are w1′=0.1×0.9=0.09, w2′=0.15×0.9=0.135, w4′=0.15×0.8=0.12, w5′=0.1×0.8=0.08, and w3′=1-0.09-0.135-0.12-0.08=0.575. This indicates that the pixel distribution correlation between each target reference region and the target coding region is reduced. Therefore, the filtering weight of each target reference region is reduced, while the filtering weight of the target coding region itself is increased.

[0228] The above scheme applies the pixel distribution correlation between the target coding region and each target reference region to the MCTF filtering process. This allows the weights of each target reference region to be adaptively adjusted according to the magnitude of the pixel distribution correlation between the target coding region and each target reference region during the filtering process. This not only effectively removes redundant information in the temporal domain, but also further improves the reference efficiency of temporal information during the encoding process, significantly improving the efficiency of video compression and achieving higher quality video transmission within a given bandwidth.

[0229] Optionally, before adjusting the weight coefficients of each target reference region based on the pixel distribution correlation between each target reference region and the target coding region, the following specific steps may be included:

[0230] The pixel distribution correlation between each target reference region and the target coding region is updated and adjusted to obtain the updated pixel distribution correlation.

[0231] Based on the pixel distribution correlation between each target reference region and the target coding region, the weight coefficients of each target reference region are adjusted, including:

[0232] Based on the updated correlation of pixel distributions, the weight coefficients of each target reference region are adjusted.

[0233] Updating and adjusting the pixel distribution correlation between each target reference region and the target coding region involves changing the numerical value of the pixel distribution correlation, which can be done by increasing or decreasing the value. For example, assuming the pixel distribution correlation between the target reference region and the target coding region is 0.9, the value can be increased to 0.95 or decreased to 0.85. The updated pixel distribution correlation is the adjusted pixel distribution correlation. Specifically, the formula for updating and adjusting the pixel distribution correlation between each target reference region and the target coding region is: ρ i ′=max(1-δ,min(1+δ),ρ i ·ε)), where ρ i ' represents the adjusted pixel distribution correlation between the i-th target reference region and the target coding region, ρ i Let δ be the unadjusted pixel distribution correlation between the i-th target reference region and the target coding region, and let ε be the range parameter of the pixel distribution correlation and ε be the scaling parameter of the pixel distribution correlation.

[0234] For example, assuming the unadjusted pixel distribution correlation between the second target reference region and the target coding region is 0.9, δ is 0.2, and ε is 0.5, then the adjusted pixel distribution correlation between the second target reference region and the target coding region is max(1-0.2, min(1+0.2), 0.9×0.5))=0.8.

[0235] The specific method for adjusting the weight coefficients of each target reference region based on the updated correlation of pixel distribution is the same as the method for adjusting the weight coefficients of each target reference region based on the correlation of pixel distribution, and will not be repeated here.

[0236] Before adjusting the weight coefficients of each target reference region based on the pixel distribution correlation between each target reference region and the target coding region, the above scheme updates and adjusts the pixel distribution correlation between each target reference region and the target coding region. Based on the updated pixel distribution correlation, the weight coefficients of each target reference region are adjusted. This incorporates the similarity of pixel value distribution into the derivation process of the filtering weights, making the pixel distribution correlation between the target reference region and the target coding region adaptable to different MCTF models, thereby improving the visual quality of the video signal after encoding and reconstruction.

[0237] Optionally, updating and adjusting the pixel distribution correlation between each target reference region and the target coding region may include the following specific steps:

[0238] Obtain pixel value fluctuation range information of the target coding region, and based on the pixel value fluctuation range information, update and adjust the range of pixel distribution correlation between each target reference region and the target coding region;

[0239] And / or,

[0240] Obtain the encoding precision parameters of the target encoding region, and based on the encoding precision parameters, scale and adjust the pixel distribution correlation values ​​between each target reference region and the target encoding region.

[0241] Pixel value fluctuation range information is the standard deviation or range of pixel values ​​within an image patch, used to describe the texture complexity of a local region. A larger fluctuation range indicates richer image patch details, requiring more bits to be allocated during encoding. For example, the pixel value fluctuation range is 2 in a flat region, while it is 15 in an edge region.

[0242] Encoding precision parameters refer to the accuracy of parameters (such as motion vectors and quantization step size) during video encoding, typically expressed as bit depth or decimal places. Higher encoding precision reduces distortion but increases computational complexity and bit rate. For example, the encoding precision of motion vectors in VVC is 1 / 16 of a pixel, which is better than H.265's 1 / 4 pixel.

[0243] Specifically, the calculation expression for updating and adjusting the range of pixel distribution correlation between each target reference region and the target coding region based on pixel value fluctuation amplitude information is as follows: Wherein, δ represents the pixel value fluctuation range information. To adjust the coefficients, different values ​​are assigned to σ in different MCTF models. C The variance of pixel values ​​in the target encoding region.

[0244] Specifically, based on the coding precision parameter, the calculation expression for scaling and adjusting the pixel distribution correlation value between each target reference region and the target coding region is: ε = k q +k l (Linear scaling), and (Exponential scaling). Where k q and k l It is a coefficient used to describe the coding precision, k q It depends on the quantization parameter, and can be k. q =0.0125×QP; k l Related to the reference efficiency, k is generally taken. l = bd - L, where L is the encoding level in the pyramid structure and bd is the bit width of the input signal.

[0245] The above scheme uses the coding level and bit width during encoding to scale and adjust the pixel distribution correlation value between each target reference region and the target coding region. Then, it adjusts the weight coefficient of each target reference region according to the adjusted pixel distribution correlation. This enables the pixel distribution correlation between the target reference region and the target coding region to be adapted to different MCTF models, thereby improving the visual quality of the video signal after encoding and reconstruction.

[0246] Optionally, obtaining the coding precision parameters of the target coding region may include the following specific steps:

[0247] Obtain the quantization parameter information and reference efficiency information of the target coding region. The reference efficiency information is determined based on the coding level and bit width of the target coding region.

[0248] The coding precision parameters are determined based on the quantization parameter information and the reference efficiency information.

[0249] Quantization parameters are scalar values ​​that control the quantization step size in video encoding, directly affecting the balance between bitrate and distortion. A larger QP value results in a larger quantization step size, leading to higher compression but also more noticeable distortion. For example, setting QP to 28 results in a lower bitrate but may cause blockiness; a QP of 20 results in a higher bitrate but clearer image quality.

[0250] Reference efficiency information indicates the effectiveness of a reference frame's prediction of the current frame, typically measured by the amount of data in the prediction residual or the rate-distortion cost. Higher reference efficiency indicates more accurate predictions from the reference frame and better coding performance. For example, if the amount of residual data in the current frame is reduced by 60% after motion compensation, the reference efficiency is high; conversely, if a sudden scene change causes an increase in residual data, the reference efficiency decreases.

[0251] The coding level refers to the hierarchical relationship of video frames within the Group of Pictures (GOP) structure. Higher-level frames (such as I-frames) can be decoded independently, while lower-level frames (such as B-frames) depend on the frames before and after them. For example, in VVC, four coding levels can be set, where level 0 is for I-frames and level 3 is for the lowest level, B-frames.

[0252] For example, assuming the quantization parameter information QP of the target coding region is 20, the coding level (L) of the target coding region is 2, and the bit width (bd) is 8, then the quantization parameter information kq = 0.0125 × QP = 0.25, and the reference efficiency information k l =bd-L=6. The formula for calculating the coding precision parameter is: ε1=k q +k l =6.25,

[0253] The above scheme determines the quantization parameter information and reference efficiency information of the target coding region based on the coding level and bit width corresponding to the target coding region. Based on the quantization parameter information and reference efficiency information, the coding precision parameter is determined, which can provide a data basis for adjusting the pixel distribution correlation between the target coding region and the target reference region.

[0254] The following is in conjunction with the appendix Figure 9 This document uses the video encoding method provided in this manual as an example in video storage to further explain the video encoding method. Specifically, Figure 9 This specification illustrates a flowchart of a video encoding method for video storage according to an embodiment of the present invention, including the following specific steps:

[0255] Step 902: Obtain the blocks to be encoded and reference blocks from the video to be stored. For example, in the Lookahead stage of the VVC encoder, the current frame (target image) is divided into 64×64 coding tree units (CTUs) as blocks to be encoded; a list of reference frames (L0 / L1) is determined according to the GOP structure, and a 128×128 search window corresponding to the spatial position of the current CTU is extracted from the reference frames as each reference block, where the distance between reference frames is 2 (P-frames refer to the second forward frame).

[0256] Step 904: Reconstruct the block to be encoded based on each reference block to obtain the reconstructed block to be encoded corresponding to each reference block. For example, after motion compensation of the candidate block in the reference frame, perform the following operations: 1) Interpolate the candidate block by 1 / 2 pixel to generate a reference frame twice the size; 2) Perform forward transform (DCT), quantization, and inverse quantization on the candidate block using quantization parameter QP=28; ​​3) Generate the reconstructed block through inverse IDCT transform and deblocking filtering (filter strength=2).

[0257] Step 906: Determine the reconstruction distortion information corresponding to each reference block based on the pixel information of each reconstructed block to be encoded and the pixel information of the block to be encoded. For example, calculate the mean square error between the original CTU and each reconstructed block as the reconstruction distortion information corresponding to each reference block.

[0258] Step 908: Obtain the bitrate corresponding to each reference block. Based on the reconstruction distortion information and bitrate of each reference block, determine the reconstruction loss corresponding to each reference block. For example, estimate the number of motion vector differential coding bits, reference frame index bits, and residual coefficient bitrate corresponding to each reference block using an entropy coding model, and sum them to obtain the total bitrate corresponding to each reference block. Based on the rate-distortion optimization model, the reconstruction distortion information and bitrate are weighted and summed using a Lagrange multiplier λ = 85 to calculate the reconstruction loss value corresponding to each reference block. For example, for a reference block with a reconstruction distortion of 3200 and a bitrate of 240 bits, its reconstruction loss value is 3200 + 85 × 240 = 23600.

[0259] Step 910: For any reference block, based on the pixel values ​​of each pixel in the block to be encoded and the pixel values ​​of each pixel in any reference block, determine the covariance between the block to be encoded and any reference block, as well as the first standard deviation of the block to be encoded and the second standard deviation of any reference block.

[0260] Step 912: Determine the pixel distribution correlation between the block to be encoded and any reference block based on the covariance, the first standard deviation, and the second standard deviation. For example, based on the covariance cov(X,Y) and the first standard deviation σ... X Second standard deviation σ Y This determines the pixel distribution correlation between the block to be encoded X and the first reference block Y. For example, the Pearson correlation coefficient is calculated: Assuming the threshold for strong correlation is set to 0.6, when ρ X,Y When the value is 0.85, the block to be encoded is determined to be strongly correlated with the first reference block.

[0261] Step 914: Obtain the quantization coefficients of the coding model and the size of the coding units in the coding model; based on the quantization coefficients and the size of the coding units, determine the adjustment coefficients used to adjust the coding model; based on the adjustment coefficients and the pixel distribution correlation corresponding to any reference region, adjust the reconstruction loss between the target coding region and any reference region to obtain the adjusted reconstruction loss corresponding to each reference region. For example, obtain the current quantization parameter QP = 28 and the coding unit size 32×32 pixels from the VVC coding configuration. Based on the quantization parameter and the size of the coding unit, determine the adjustment coefficients a = 1 and b = 0.5. For the reference block with pixel distribution correlation ρ = 0.8, then according to the aforementioned formula: We know that a(ρ+b)=1.3, therefore the adjusted reconstruction loss is the value of the original reconstruction loss divided by 1.3.

[0262] Step 916: Based on the adjusted reconstruction loss corresponding to each reference block, determine the target reference block from the reference blocks in the search area; determine the motion vector between the block to be encoded and the target reference block; based on the motion vector, align the block to be encoded and the target reference block to obtain the aligned block to be encoded and the target reference block; encode based on the aligned block to be encoded and the target reference block. For example, perform integer pixel motion estimation on a 32×32 block to be encoded within a range of ±64 pixels, find the optimal matching block (coordinate offset Δx=12, Δy=-5), perform a 1 / 2 pixel search to obtain the final MV=(12.5, -5.5), use bilinear interpolation to generate subpixel reference blocks, perform motion compensation and calculate residuals, and perform CABAC entropy coding after DCT transformation and quantization coefficient Zigzag scanning.

[0263] In the embodiments of this specification, a target coding region and various reference regions are obtained from the video to be encoded. The target coding region is reconstructed based on each reference region, and the reconstruction loss corresponding to each reference region is determined. Based on the pixel information of the target coding region and the pixel information of each reference region, the pixel distribution correlation between the target coding region and each reference region is determined, completing the comparison of the pixel distribution correlation between the target coding region and each reference region. This provides data support for adjusting the reconstruction loss between the target coding region and any reference region based on pixel distribution similarity. For any reference region, based on the reconstruction distortion information and pixel distribution correlation corresponding to any reference region, the reconstruction loss between the target coding region and any reference region is adjusted, resulting in the adjusted reconstruction loss for each reference region. Based on the adjusted reconstruction loss for each reference region, a target reference region is determined from each reference region, and encoding processing is performed based on the target coding region and the target reference region. This allows the pixel distribution correlation between the target coding region and any reference region to be incorporated into the similarity measurement based on the reconstruction loss corresponding to each reference region, achieving accurate selection of the reference region most similar to the target coding region. This solves quality problems such as visual artifacts that occur when encoding the target coding region based on reference regions.

[0264] Corresponding to the above method embodiments, this specification also provides embodiments of video encoding apparatus. Figure 10 A schematic diagram of a video encoding apparatus according to one embodiment of this specification is shown. Figure 10 As shown, the video encoding device 1000 includes an acquisition module 1002, a reconstruction module 1004, an analysis module 1006, an evaluation module 1008, and an encoding module 1010.

[0265] The acquisition module 1002 is configured to acquire the target coding region and each reference region in the video to be encoded; the reconstruction module 1004 is configured to reconstruct the target coding region based on each reference region and determine the reconstruction loss corresponding to each reference region; the analysis module 1006 is configured to determine the pixel distribution correlation between the target coding region and each reference region based on the pixel information of the target coding region and the pixel information of each reference region; the evaluation module 1008 is configured to adjust the reconstruction loss between the target coding region and any reference region based on the pixel distribution correlation corresponding to any reference region, and obtain the adjusted reconstruction loss corresponding to each reference region; the encoding module 1010 is configured to determine the target reference region from each reference region according to the adjusted reconstruction loss corresponding to each reference region, and perform encoding processing based on the target coding region and the target reference region.

[0266] Optionally, the reconstruction module 1004 is specifically configured to: reconstruct the target coding region based on each reference region to obtain the reconstructed target coding region corresponding to each reference region; determine the reconstruction distortion information corresponding to each reference region based on the pixel information of each reconstructed target coding region and the pixel information of the target coding region; obtain the bit rate corresponding to each reference region; and determine the reconstruction loss corresponding to each reference region based on the reconstruction distortion information and bit rate corresponding to each reference region.

[0267] Optionally, the analysis module 1006 is specifically configured to: for the first reference region, based on the pixel values ​​of each pixel in the target coding region and the pixel values ​​of each pixel in the first reference region, determine the covariance between the target coding region and the first reference region, as well as the first standard deviation of the target coding region and the second standard deviation of the first reference region, wherein the first reference region is any reference region; and determine the pixel distribution correlation between the target coding region and the first reference region respectively based on the covariance, the first standard deviation and the second standard deviation.

[0268] Optionally, the evaluation module 1008 is specifically configured to: obtain the quantization coefficients of the coding model and the size of the coding unit in the coding model; determine the adjustment coefficients for adjusting the coding model based on the quantization coefficients of the coding model and the size of the coding unit; and adjust the reconstruction loss between the target coding region and any reference region based on the correlation between the adjustment coefficients and the pixel distribution corresponding to any reference region, so as to obtain the adjusted reconstruction loss corresponding to each reference region.

[0269] Optionally, the target coding region is the target image in the video to be encoded, and each reference region is a reference image in the video to be encoded that is associated with the target image; the coding module 1010 is specifically configured to: determine the target reference image and coding parameters from each reference image according to the adjusted reconstruction loss corresponding to each reference image, wherein the coding parameters are used when encoding the target image based on the target reference image.

[0270] Optionally, the target encoding region is the block to be encoded in the target image, and the reference region is the reference block of the search region in the reference image corresponding to the target image; the encoding module 1010 is specifically configured to: determine the target reference block from each reference block in the search region according to the adjusted reconstruction loss corresponding to each reference block; determine the motion vector between the block to be encoded and the target reference block; align the block to be encoded and the target reference block based on the motion vector to obtain the aligned block to be encoded and the target reference block; and encode based on the aligned block to be encoded and the target reference block.

[0271] In the embodiments of this specification, the video encoding device acquires the target encoding region and each reference region in the video to be encoded. The target encoding region is reconstructed based on each reference region, and the reconstruction loss corresponding to each reference region is determined. Based on the pixel information of the target encoding region and each reference region, the pixel distribution correlation between the target encoding region and each reference region is determined, thus completing the comparison of the pixel distribution correlation between the target encoding region and each reference region. This provides data support for adjusting the reconstruction loss between the target encoding region and any reference region based on pixel distribution similarity. For any reference region, based on the reconstruction distortion information and pixel distribution correlation corresponding to any reference region, the reconstruction loss between the target encoding region and any reference region is adjusted to obtain the adjusted reconstruction loss corresponding to each reference region. Based on the adjusted reconstruction loss corresponding to each reference region, a target reference region is determined from each reference region, and encoding processing is performed based on the target encoding region and the target reference region. This allows the pixel distribution correlation between the target encoding region and any reference region to be incorporated into the similarity measurement based on the reconstruction loss corresponding to each reference region, achieving accurate selection of the reference region most similar to the target encoding region. This solves quality problems such as visual artifacts that occur when encoding the target encoding region based on reference regions.

[0272] The above is an illustrative scheme of a video encoding device according to this embodiment. It should be noted that the technical solution of this video encoding device and the technical solution of the video encoding method described above belong to the same concept. For details not described in detail in the technical solution of the video encoding device, please refer to the description of the technical solution of the video encoding method described above.

[0273] Figure 11 A structural block diagram of a computing device according to one embodiment of this specification is shown. The components of the computing device 1100 include, but are not limited to, a memory 1110 and a processor 1120. The processor 1120 is connected to the memory 1110 via a bus 1130, and a database 1150 is used to store data.

[0274] The computing device 1100 also includes an access device 1140, which enables the computing device 1100 to communicate via one or more networks 1160. Examples of these networks include a Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the Internet. The access device 1140 may include one or more of any type of wired or wireless network interface (e.g., a Network Interface Controller (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a Near Field Communication (NFC) interface.

[0275] In one embodiment of this specification, the aforementioned components of the computing device 1100 and Figure 11 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 11 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0276] The computing device 1100 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 1100 can also be a mobile or stationary server.

[0277] The processor 1120 is used to execute the following computer program / instructions, which, when executed by the processor, implement the steps of the video encoding method described above.

[0278] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the video encoding method described above belong to the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the video encoding method described above.

[0279] An embodiment of this specification also provides a computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the video encoding method described above.

[0280] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium belongs to the same concept as the technical solution of the video encoding method described above. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the video encoding method described above.

[0281] An embodiment of this specification also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the video encoding method described above.

[0282] The above is an illustrative scheme of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product and the technical solution of the video encoding method described above belong to the same concept. For details not described in detail in the technical solution of the computer program product, please refer to the description of the technical solution of the video encoding method described above.

[0283] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0284] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content contained in the computer-readable medium may be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0285] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0286] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0287] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A video encoding method, characterized in that, include: Obtain the target encoding region and each reference region in the video to be encoded; The target coding region is reconstructed based on each reference region, and the reconstruction loss corresponding to each reference region is determined. Based on the pixel information of the target coding region and the pixel information of each reference region, the pixel distribution correlation between the target coding region and each reference region is determined. For any reference region, based on the pixel distribution correlation corresponding to the reference region, the reconstruction loss between the target coding region and the reference region is adjusted to obtain the adjusted reconstruction loss corresponding to each reference region. Based on the adjusted reconstruction loss corresponding to each reference region, a target reference region is determined from each reference region, and encoding processing is performed based on the target encoding region and the target reference region.

2. The method according to claim 1, characterized in that, The step of reconstructing the target coding region based on each reference region and determining the reconstruction loss corresponding to each reference region includes: The target coding region is reconstructed based on each reference region to obtain the reconstructed target coding region corresponding to each reference region. Based on the pixel information of each reconstructed target coding region and the pixel information of the target coding region, the reconstruction distortion information corresponding to each reference region is determined; Obtain the bitrate corresponding to each reference region, and determine the reconstruction loss corresponding to each reference region based on the reconstruction distortion information and bitrate corresponding to each reference region.

3. The method according to claim 1, characterized in that, The step of determining the pixel distribution correlation between the target coding region and each of the reference regions based on the pixel information of the target coding region and the pixel information of each reference region includes: For the first reference region, based on the pixel values ​​of each pixel in the target coding region and the pixel values ​​of each pixel in the first reference region, the covariance between the target coding region and the first reference region, as well as the first standard deviation of the target coding region and the second standard deviation of the first reference region are determined, wherein the first reference region is any reference region. The pixel distribution correlation between the target coding region and the first reference region is determined based on the covariance, the first standard deviation, and the second standard deviation.

4. The method according to claim 1, characterized in that, The step of adjusting the reconstruction loss between the target coding region and the reference region based on the pixel distribution correlation corresponding to any reference region to obtain the adjusted reconstruction loss corresponding to each reference region includes: Obtain the quantization coefficients of the coding model and the size of the coding unit in the coding model; Based on the quantization coefficients of the coding model and the size of the coding unit, adjustment coefficients for adjusting the coding model are determined; Based on the adjustment coefficient and the pixel distribution correlation corresponding to any reference region, the reconstruction loss between the target coding region and any reference region is adjusted to obtain the adjusted reconstruction loss corresponding to each reference region.

5. The method according to any one of claims 1 to 4, characterized in that, The target encoding region is the target image in the video to be encoded, and each reference region is a reference image in the video to be encoded that is associated with the target image. The step of determining a target reference region from each reference region based on the adjusted reconstruction loss corresponding to each reference region, and performing encoding processing based on the target encoding region and the target reference region, includes: Based on the adjusted reconstruction loss corresponding to each of the reference images, a target reference image and encoding parameters are determined from each of the reference images, wherein the encoding parameters are used when encoding the target image based on the target reference image.

6. The method according to any one of claims 1 to 4, characterized in that, The target encoding region is the block to be encoded in the target image, and the reference region is the reference block of the search region in the reference image corresponding to the target image; The step of determining a target reference region from each reference region based on the adjusted reconstruction loss corresponding to each reference region, and performing encoding processing based on the target encoding region and the target reference region, includes: Based on the adjusted reconstruction loss corresponding to each of the reference blocks, the target reference block is determined from each reference block in the search region; Determine the motion vector between the block to be encoded and the target reference block; Based on the motion vector, the block to be encoded and the target reference block are aligned to obtain the aligned block to be encoded and the target reference block; Encoding is performed based on the aligned block to be encoded and the target reference block.

7. A video encoding device, characterized in that, include: The acquisition module is configured to acquire the target encoding region and each reference region in the video to be encoded. The reconstruction module is configured to reconstruct the target coding region based on each of the reference regions, and determine the reconstruction loss corresponding to each reference region; The analysis module is configured to determine the pixel distribution correlation between the target coding region and each of the reference regions based on the pixel information of the target coding region and the pixel information of each reference region. The evaluation module is configured to adjust the reconstruction loss between the target coding region and the reference region based on the pixel distribution correlation of the reference region for any reference region, so as to obtain the adjusted reconstruction loss for each reference region. The encoding module is configured to determine a target reference region from each reference region based on the adjusted reconstruction loss corresponding to each reference region, and to perform encoding processing based on the target encoding region and the target reference region.

8. A computing device, characterized in that, include: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, It stores a computer program / instructions that, when executed by a processor, implement the steps of the method described in any one of claims 1 to 6.

10. A computer program product, characterized in that, Includes a computer program / instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 6.

Images