A rate distortion optimization method and device for suppressing breathing effect

Abstract

The present invention discloses a rate-distortion optimization method for suppressing the breathing effect. When the encoding of each I frame is completed, the intra-frame coding mode information and the availability flag of the image blocks at all positions in the I frame are saved as reference data. When all P frames or B frames between a certain I frame and the next I frame are encoded, the reference data is updated. When encoding the next I frame, if the availability flag of the image block at the corresponding position in the reference data is available, the intra-frame coding mode information of the image block at the corresponding position in the reference data is introduced into the rate-distortion optimization operation of the intra-frame coding mode selection, and participates in the intra-frame coding mode selection. The present invention reduces the difference in intra-frame coding modes between adjacent I frames by introducing the intra-frame coding mode of a certain I frame into the rate-distortion optimization process of the intra-frame coding mode selection of the next I frame, thereby suppressing the breathing effect.

Description

Technical Field

[0001] The present invention relates to a video encoding technology. Background Art

[0002] Video coding refers to compressing the redundant components in video images and using as little data as possible to represent video information. Common video coding standards include HEVC (High Efficiency Video Coding, also known as H.265), AVC (Advanced Video Coding, also known as H.264), etc.

[0003] Video coding technology uses image blocks as the most basic coding unit. For example, in HEVC, the basic unit of coding is CU (Coding Unit). CU can be an image block of 64 pixels × 64 pixels, 32 pixels × 32 pixels, 16 pixels × 16 pixels, or 8 pixels × 8 pixels.

[0004] In order to remove information redundancy in the spatial and temporal domains, intra-frame coding technology and inter-frame coding technology are used when encoding the input video frames. According to the coding technology used, the encoded video frames can be divided into I frames that are encoded only using intra-frame coding technology, and P frames and B frames that are encoded using a mixture of inter-frame coding technology and intra-frame coding technology. When encoding I frames, all image blocks in the video frame must be encoded as intra-frame coding blocks (intra blocks), and no reference will be made to other encoded video frames. When encoding P frames and B frames, image blocks in the video frame can be encoded as inter-frame coding blocks (inter blocks) or as intra-frame coding blocks, and one or more other encoded video frames will be referenced. Since P frames and B frames have higher coding efficiency, and I frames have the function of blocking error propagation, video encoding generally encodes I frames periodically and encodes most of the remaining video frames as P frames or B frames, such as Figure 1 shown.

[0005] In order to maximize the video compression rate, video encoding often uses a lossy compression algorithm, which causes the difference between the encoded and reconstructed video and the original video, that is, the encoded and reconstructed video will be distorted. For lossy compression algorithms, such as HEVC and AVC, their encoding performance needs to be measured by the encoding bit rate (Rate) and the distortion (Distortion) introduced by the encoding. In the process of video encoding, the encoding bit rate and the encoding distortion are mutually constrained. Reducing the encoding bit rate often increases the encoding distortion, while reducing the encoding distortion often increases the encoding bit rate. The purpose of rate distortion optimization (RDO) is to reduce the encoding distortion as much as possible under a certain encoding bit rate, or to reduce the encoding bit rate as much as possible under a certain encoding distortion. Currently, in common video coding algorithms, rate distortion optimization methods based on Lagrange multipliers λ are used. Assuming D is the coding distortion, R is the coding bit rate, and J is the coding rate distortion cost (Rate Distortion Cost, RD Cost), the rate distortion optimization method based on the Lagrange multiplier λ can be expressed as min(J), where the min() function represents the minimum value, J = D + λ × R. Each video coding mode has its corresponding coding distortion D and coding bit rate R. By calculating the coding rate distortion cost J corresponding to each video coding mode and selecting the video coding mode with the minimum rate distortion cost for encoding, the optimal video coding performance can be obtained.

[0006] When encoding video for video surveillance and video conferencing scenes, periodic visual flickering is often observed in the encoded video stream, which is called the breathing effect. In particular, the breathing effect is more obvious in the static background area with simple texture in the scene. The generation of the breathing effect is related to the encoding mode of I frames, P frames and B frames. When P frames and B frames are encoded, the distortion pattern of the video frame is similar to the previously encoded video frame due to the use of inter-frame coding technology. When I frames are encoded, the distortion pattern of the video frame is completely unrelated to the previously encoded video frame due to the use of only intra-frame coding technology, and there is no similarity in the distortion pattern. In this way, when watching the decoded video stream, the distortion pattern of the I frame is quite different from the distortion pattern of the previous P frame and B frame, which in turn causes visual flicker in the human eye. Since I frames are generally encoded periodically during the video encoding process, this visual flicker in the human eye will occur periodically, that is, the breathing effect will occur. Encoded video streams with breathing effects will greatly affect the subjective feelings of viewers and greatly reduce the visual experience of viewers. Therefore, the breathing effect should be suppressed during video encoding.

[0007] Currently, there are some technical solutions for suppressing the breathing effect generated during the video encoding process.

[0008] One solution is to add a time domain filter during the video encoding process, perform time domain filtering on the original video frame corresponding to the I frame and the reconstructed frames of the P frame and B frame encoded before the I frame, and then perform I frame encoding on the filtered original video frame. In this solution, the distortion mode of the reconstructed frames of the P frame and B frame encoded before a certain I frame is introduced into the original video frame corresponding to the I frame through time domain filtering, reducing the difference between the original video frame corresponding to the I frame and the reconstructed frames of the P frame and B frame encoded before the I frame, thereby suppressing the breathing effect to a certain extent. Although this solution can make the original video frame corresponding to the I frame partially have the distortion mode of the P frame and B frame encoded before the I frame, it does not take into account that when the filtered original video frame is encoded as an I frame, the intra-frame coding technology itself will introduce a new distortion mode, resulting in a new distortion mode difference. In addition, when implementing in hardware, this solution needs to add a special time domain filter device in the encoding pipeline, which has a high hardware cost, and the bandwidth consumption and encoding delay will also increase accordingly.

[0009] Another solution is to perform two rounds of video encoding on the original video frame corresponding to the I frame. In the first round, the original video frame is encoded as a P frame, and then the reconstructed frame of the P frame is used as the input image for the second round of I frame encoding. In the first round of encoding, the original video frame is encoded as a P frame. Since the P frame uses inter-frame coding technology, the distortion pattern of its reconstructed frame is similar to the distortion pattern of the P frame and B frame encoded before the P frame. In the second round of encoding, the reconstructed frame of the P frame is encoded as an I frame, and the distortion pattern of the previously encoded P frame and B frame is partially retained in the encoded I frame, thereby suppressing the breathing effect to a certain extent. Similar to the previous solution, in this solution, the I frame encoding itself will introduce a new distortion pattern, resulting in a new distortion pattern difference. In addition, when implemented in hardware, this solution changes the logic of the hardware encoding pipeline because it requires two rounds of encoding of the I frame, which will increase the complexity of the hardware design and the hardware cost. Summary of the invention

[0010] The technical problem to be solved by the present invention is to propose a video encoding method which can better suppress the breathing effect, has low hardware implementation cost, and does not change the logic of the hardware encoding pipeline.

[0011] In order to solve the above technical problems, the present invention discloses a rate-distortion optimization method for suppressing the breathing effect, comprising the following steps. Step S1: When each I frame is encoded, the intra-frame coding mode information and the availability flag of the image blocks at all positions in the I frame are saved as reference data, and the availability flags of the image blocks at all positions are marked as available. Step S2: When all P frames or B frames between a certain I frame and the next I frame are encoded, the reference data is updated according to the coding mode information of the image blocks at each position in each P frame or B frame. Each time the reference data is updated according to a certain P frame or B frame, if the image block at a certain position in the P frame or B frame is encoded as an inter-frame coding block, further judgment is made according to the length of the motion vector of the image block; if any one of the horizontal component length or the vertical component length of the motion vector of the image block is greater than the judgment threshold, the availability flag of the image block at the corresponding position in the reference data is changed to unavailable; otherwise, the reference data is not updated. Each time the reference data is updated according to a certain P frame or B frame, if an image block at a certain position inside the P frame or B frame is encoded as an intra-frame coding block, further judgment is made based on the availability flag of the image block at the corresponding position in the reference data; if the availability flag of the image block at the corresponding position in the reference data is available, the intra-frame coding mode information of the image block at the corresponding position in the reference data is changed to the intra-frame coding mode information of the image block on the P frame or B frame, and the availability flag of the image block at the corresponding position in the reference data is not modified; otherwise, not only is the intra-frame coding mode information of the image block at the corresponding position in the reference data changed to the intra-frame coding mode information of the image block on the P frame or B frame, but the availability flag of the image block at the corresponding position in the reference data is also changed to available. Step S3: when encoding the next I frame, if the image block at a certain position inside the next I frame is marked as available for the image block at the corresponding position in the reference data, the intra-frame coding mode information of the image block at the corresponding position in the reference data is introduced into the rate-distortion optimization operation of the intra-frame coding mode selection of the image block of the next I frame, and participates in the intra-frame coding mode selection of the image block of the next I frame; when the encoding of the next I frame is completed, return to step S1 to update the reference data as a whole.

[0012] Furthermore, the intra-frame coding mode information of the image block at each position includes the position information of the image block, the size information of the image block, and the intra-frame prediction mode information of the image block.

[0013] Furthermore, the availability flag of the image block at each position is used to indicate whether the intra-frame coding mode information of the image block can be used in the rate-distortion optimization operation of selecting the intra-frame coding mode of the image block at the corresponding position of the next I frame.

[0014] Preferably, in the step S1, the reference data is stored in an intra-frame coding mode table.

[0015] Furthermore, in step S2, assuming that there are a total of q P frames and B frames between a certain I frame and the next I frame, the reference data is updated and determined q times in sequence according to the sequence of the q P frames and B frames within the range.

[0016] Preferably, in step S2, the determination threshold is set to 1, that is, 1 integer pixel distance.

[0017] Furthermore, in step S3, if the availability of the image block at a certain position within the next I frame is marked as unavailable for the image block at the corresponding position in the reference data, the intra-frame coding mode information of the image block at the corresponding position in the reference data is not introduced into the rate-distortion optimization operation of the intra-frame coding mode selection of the image block in the next I frame.

[0018] Optionally, in step S3, when the intra-frame coding mode information of the image block at the corresponding position in the reference data is introduced into the rate-distortion optimization operation of the intra-frame coding mode selection of the image block in the next I frame, an intra-frame coding mode with the highest prediction cost is removed from the original intra-frame coding modes of the image block in the next I frame.

[0019] Optionally, in step S3, when the intra-frame coding mode information of the image block at the corresponding position in the reference data is introduced into the rate-distortion optimization operation of the intra-frame coding mode selection of the image block in the next I frame, the rate-distortion cost corresponding to the introduced intra-frame coding mode is multiplied by different rate-distortion cost scaling factors according to the size of the image block, and then participates in the intra-frame coding mode selection of the image block in the next I frame; the rate-distortion cost scaling factor is negatively correlated with the size of the image block.

[0020] The present invention also discloses a rate-distortion optimization device for suppressing breathing effect, comprising an intra-frame coding mode storage module, an intra-frame coding mode update module and an intra-frame coding mode selection module. The intra-frame coding mode storage module is used to save the intra-frame coding mode information and the availability flag of the image blocks at all positions in the I frame as reference data when the coding of each I frame is completed, and the availability flags of the image blocks at all positions are marked as available. The intra-frame coding mode update module is used to update the reference data according to the coding mode information of the image blocks at each position in each P frame or B frame when encoding all P frames or B frames between a certain I frame and the next I frame; each time the reference data is updated according to a certain P frame or B frame, if the image block at a certain position in the P frame or B frame is encoded as an inter-frame coding block, further judgment is made according to the length of the motion vector of the image block; if any one of the horizontal component length or the vertical component length of the motion vector of the image block is greater than the judgment threshold, the availability flag of the image block at the corresponding position in the reference data is changed to unavailable; otherwise, the reference data is not updated; each time the reference data is updated according to a certain P frame or B frame within the range When the reference data is used, if an image block at a certain position inside the P frame or the B frame is encoded as an intra-frame coding block, further judgment is made based on the availability flag of the image block at the corresponding position in the reference data; if the availability flag of the image block at the corresponding position in the reference data is available, the intra-frame coding mode information of the image block at the corresponding position in the reference data is changed to the intra-frame coding mode information of the image block on the P frame or the B frame, and the availability flag of the image block at the corresponding position in the reference data is not modified; otherwise, not only is the intra-frame coding mode information of the image block at the corresponding position in the reference data changed to the intra-frame coding mode information of the image block on the P frame or the B frame, but the availability flag of the image block at the corresponding position in the reference data is also changed to available. The intra-frame coding mode selection module is used to introduce the intra-frame coding mode information of the image block at the corresponding position in the reference data into the rate-distortion optimization operation of the intra-frame coding mode selection of the image block in the next I frame when encoding the next I frame, if the availability mark of the image block at the corresponding position in the reference data is available, and participate in the intra-frame coding mode selection of the image block in the next I frame.

[0021] The technical effect achieved by the present invention is: by introducing the intra-frame coding mode of a certain I frame into the rate-distortion optimization process of selecting the intra-frame coding mode of the next I frame when specific conditions are met, the difference in intra-frame coding modes between adjacent I frames is reduced, thereby suppressing the breathing effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a schematic diagram of the distribution of I frames, P frames or B frames in the encoded video frames.

[0023] Figure 2 It is a flow chart of the rate distortion optimization method for suppressing the breathing effect proposed by the present invention.

[0024] Figure 3 It is a structural schematic diagram of the rate-distortion optimization device for suppressing the breathing effect proposed by the present invention.

[0025] Explanation of the reference numerals in the figure: 1 is an intra-frame coding mode saving module, 2 is an intra-frame coding mode updating module, and 3 is an intra-frame coding mode selecting module. DETAILED DESCRIPTION

[0026] The inventors have found through research that, for video surveillance and video conferencing scenarios, when a coding method is used in which I frames are periodically encoded and most of the remaining video frames are encoded as P frames or B frames, the I frame and the subsequent P frames and B frames have similar distortion patterns; the distortion pattern will not change significantly until the next I frame appears. Figure 1 When encoding the P frame or B frame after the Nth I frame, since the encoding of the P frame or B frame will use the inter-frame coding technology, it will directly or indirectly refer to the Nth I frame, so they have a distortion pattern similar to that of the Nth I frame. When the N+1th I frame is encoded, since it only uses the intra-frame coding technology, the intra-frame coding mode it uses may be very different from the Nth I frame, resulting in a large difference in the distortion pattern of the N+1th I frame and the Nth I frame, and also a large difference in the distortion pattern of the P frame or B frame encoded after the Nth I frame, which in turn produces a breathing effect. In other words, although the breathing effect observed by the viewer is caused by the large difference in the distortion pattern between the N+1th I frame and the previously encoded P and B frames, the root cause is that the N+1th I frame and the Nth I frame have different encoding modes.

[0027] Therefore, when encoding video surveillance and video conferencing scenes, especially when encoding static, simple-textured background areas in the scene, the difference in intra-frame coding modes between adjacent I frames is the main cause of the breathing effect. In the present invention, the intra-frame coding mode of the Nth I frame is introduced into the rate-distortion optimization process of selecting the intra-frame coding mode of the N+1th I frame to reduce the difference in intra-frame coding modes between adjacent I frames, thereby achieving suppression of the breathing effect. This is the core innovation of the present invention.

[0028] See also Figure 2 The rate-distortion optimization method for suppressing the breathing effect proposed by the present invention includes the following steps.

[0029] Step S1: When the encoding of each I frame of the video stream is completed, the intra-frame coding mode information and the availability flag of the image blocks at all positions within the I frame are saved as reference data, and the availability flags of the image blocks at all positions are marked as available. The intra-frame coding mode information of the image block at each position includes the position information of the image block, the size information of the image block, and the intra-frame prediction mode information of the image block (only one final selected intra-frame prediction mode). For example, the HEVC standard specifies 35 intra-frame prediction modes (1 DC mode, 1 planar mode, and 33 angle modes). The availability flag of the image block at each position is used to indicate whether the intra-frame coding mode information of the image block can be used in the rate-distortion optimization operation of the intra-frame coding mode selection of the image block at the corresponding position of the next I frame. Preferably, the reference data is stored in an intra-frame coding mode table.

[0030] Step S2: When encoding all P frames or B frames between the Nth (N is a positive integer) I frame and the N+1th I frame, the reference data is updated according to the coding mode information of the image block at each position within each P frame or B frame within the range. Assuming that there are a total of q P frames and B frames between the Nth I frame and the N+1th I frame, this step will perform q update judgment processes on the reference data in sequence according to the order of the q P frames and B frames within the range in the video stream, and each update judgment process is as follows.

[0031] (1) Each time the reference data is updated according to a P frame or B frame within the range, if an image block at a certain position within the P frame or B frame is encoded as an inter-frame coding block (inter block), further judgment is made based on the length of the motion vector of the image block. (1a) If either the horizontal component length or the vertical component length of the motion vector of the image block is greater than the judgment threshold, the image block is judged to be a motion image block, which means that its distortion pattern is significantly different from the distortion pattern of the image block at the corresponding position on the Nth I frame, so the availability flag of the image block at the corresponding position in the reference data is changed to unavailable. (1b) If both the horizontal component length and the vertical component length of the motion vector of the image block are less than the judgment threshold, the image block is judged to be a still image block, and the reference data is not updated. Preferably, the judgment threshold is set to 1, i.e., 1 integer pixel distance.

[0032] (2) Each time the reference data is updated according to a certain P frame or B frame within the range, if an image block at a certain position within the P frame or B frame is encoded as an intra-frame coding block (intra block), further judgment is made based on the availability flag of the image block at the corresponding position in the reference data. (2a) If the availability flag of the image block at the corresponding position in the reference data is available, the intra-frame coding mode information of the image block at the corresponding position in the reference data is changed to the intra-frame coding mode information of the image block on the P frame or B frame (actually only the intra-frame prediction mode information is modified), and the availability flag of the image block at the corresponding position in the reference data is not modified. (2b) If the availability flag of the image block at the corresponding position in the reference data is unavailable, not only is the intra-frame coding mode information of the image block at the corresponding position in the reference data changed to the intra-frame coding mode information of the image block on the P frame or B frame (actually only the intra-frame prediction mode information is modified), but the availability flag of the image block at the corresponding position in the reference data is also changed to available.

[0033] Step S3: When encoding the N+1th I frame, if the image block at a certain position within the N+1th I frame is marked as available for the image block at the corresponding position in the reference data, the intra-frame coding mode information of the image block at the corresponding position in the reference data is introduced into the rate-distortion optimization calculation of the intra-frame coding mode selection of the image block in the N+1th I frame, and participates in the intra-frame coding mode selection of the image block in the N+1th I frame. This operation can be understood as: when the prior art encodes the N+1th I frame, the intra-frame coding mode selection for the image block at a certain position only needs to calculate the rate-distortion optimization of k intra-frame coding modes, and then select the intra-frame coding mode with the minimum rate-distortion cost from the k intra-frame coding modes. When specific conditions are met, the present invention introduces the intra-frame coding mode information of the image block at the corresponding position in the reference data into the rate-distortion optimization operation of the intra-frame coding mode selection of the image block in the N+1th I frame. Therefore, it is necessary to calculate the rate-distortion optimization of k+1 intra-frame coding modes, and then select the intra-frame coding mode with the minimum rate-distortion cost from the k+1 intra-frame coding modes. If the availability mark of the image block at the corresponding position in the reference data of the image block at a certain position within the N+1th I frame is unavailable, the intra-frame coding mode information of the image block at the corresponding position in the reference data is not introduced into the rate-distortion optimization operation of the intra-frame coding mode selection of the image block in the N+1th I frame. Repeat step S3 to perform the rate-distortion optimization operation of the intra-frame coding mode selection on the image blocks at all positions within the N+1th I frame. When the encoding of the N+1th I frame is completed, return to step S1 and update the reference data as a whole to: the intra-frame coding mode information and availability flags of all image blocks at all positions within the N+1th I frame, and the availability flags of all image blocks at all positions are marked as available.

[0034] Optionally, in step S3, when the intra-frame coding mode information of the image block at the corresponding position in the reference data is introduced into the rate-distortion optimization operation of the intra-frame coding mode selection of the image block in the N+1th I frame, an intra-frame coding mode with the highest prediction cost is removed from the original k intra-frame coding modes of the image block in the N+1th I frame to keep the rate-distortion optimization operation amount of the intra-frame coding mode selection basically unchanged. The prediction cost refers to the cost of coarse-grained screening of the intra-frame prediction mode, which is also based on the rate-distortion optimization operation, but the accuracy required when calculating the rate-distortion cost is lower.

[0035] Optionally, in step S3, when the intra-frame coding mode information of the image block at the corresponding position in the reference data is introduced into the rate-distortion optimization operation for selecting the intra-frame coding mode of the image block in the N+1th I frame, the rate-distortion cost corresponding to the introduced intra-frame coding mode is multiplied by different rate-distortion cost scaling coefficients according to the size of the image block, and then participates in the intra-frame coding mode selection of the image block in the N+1th I frame. The value range of the rate-distortion cost scaling coefficient is >0 and ≤1, and the rate-distortion cost scaling coefficient is negatively correlated with the size of the image block. The larger the image block, the easier it is to produce a breathing effect. In this case, the rate-distortion cost scaling coefficient is set to a smaller value, so that the introduced intra-frame coding mode is easier to be selected to suppress the breathing effect. The smaller the image block, the less likely it is to produce a breathing effect. In this case, the rate-distortion cost scaling coefficient is set to a larger value, so that the introduced intra-frame coding mode is less likely to be selected to improve the coding efficiency. In this way, the effective suppression of the breathing effect can be achieved with a small loss in coding efficiency.

[0036] See also Figure 3 The rate-distortion optimization device for suppressing breathing effect proposed by the present invention includes an intra-frame coding mode storage module 1, an intra-frame coding mode update module 2 and an intra-frame coding mode selection module 3. Figure 3 The device shown corresponds to Figure 2 The method shown.

[0037] The intra-frame coding mode saving module 1 is used to save the intra-frame coding mode information and availability flags of all image blocks at positions within the I frame as reference data when the coding of each I frame of the video stream is completed, and the availability flags of all image blocks at all positions are marked as available.

[0038] The intra-frame coding mode updating module 2 is used to update the reference data according to the coding mode information of each image block at each position in each P frame or B frame within the range when encoding all P frames or B frames between the Nth I frame and the N+1th I frame. (1) Each time the reference data is updated according to a certain P frame or B frame within the range, if the image block at a certain position in the P frame or B frame is encoded as an inter-frame coding block, further judgment is made according to the length of the motion vector of the image block. (1a) If either the horizontal component length or the vertical component length of the motion vector of the image block is greater than the judgment threshold, the usability flag of the image block at the corresponding position in the reference data is changed to unusable. (1b) If both the horizontal component length and the vertical component length of the motion vector of the image block are less than the judgment threshold, the reference data is not updated. (2) Each time the reference data is updated according to a certain P frame or B frame within the range, if the image block at a certain position in the P frame or B frame is encoded as an intra-frame coding block, further judgment is made according to the usability flag of the image block at the corresponding position in the reference data. (2a) If the availability flag of the image block at the corresponding position in the reference data is available, the intra-frame coding mode information of the image block at the corresponding position in the reference data is changed to the intra-frame coding mode information of the image block on the P frame or B frame. (2b) If the availability flag of the image block at the corresponding position in the reference data is unavailable, not only is the intra-frame coding mode information of the image block at the corresponding position in the reference data changed to the intra-frame coding mode information of the image block on the P frame or B frame, but the availability flag of the image block at the corresponding position in the reference data is also changed to available.

[0039] The intra-frame coding mode selection module 3 is used to introduce the intra-frame coding mode information of the image block at the corresponding position in the reference data into the rate-distortion optimization operation of the intra-frame coding mode selection of the image block in the N+1th I frame when encoding the N+1th I frame, if the availability flag of the image block at a certain position in the N+1th I frame is available, and participate in the intra-frame coding mode selection of the image block in the N+1th I frame. When the encoding of the N+1th I frame is completed, the intra-frame coding mode storage module 1 updates the reference data as a whole to: the intra-frame coding mode information and availability flags of the image blocks at all positions in the N+1th I frame, and the availability flags of the image blocks at all positions are marked as available.

[0040] The rate-distortion optimization method proposed in the present invention reduces the difference in intra-frame coding modes between adjacent I frames by introducing the intra-frame coding mode of a certain I frame into the rate-distortion optimization process of selecting the intra-frame coding mode of the next I frame when specific conditions are met, thereby suppressing the breathing effect. In particular, for static background areas with simple textures in video surveillance and video conferencing scenes, the present invention has an obvious effect of suppressing the breathing effect. In addition, the present invention can also adjust the introduced intra-frame coding mode through coefficients, thereby effectively suppressing the breathing effect and avoiding a significant decrease in coding efficiency. The method proposed in the present invention does not require the introduction of additional computing units, does not require changes to the existing hardware encoding pipeline logic, occupies less additional bandwidth, and has a low hardware implementation cost.

[0041] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A rate-distortion optimization method for suppressing breathing effect, characterized in that: The method comprises the following steps: Step S1: when encoding of each I frame is completed, the intra-frame coding mode information and the usability flag of the image blocks at all positions in the I frame are saved as reference data, and the usability flags of the image blocks at all positions are marked as usable; the intra-frame coding mode information of the image blocks at each position includes the position information of the image block, the size information of the image block, and the intra-frame prediction mode information of the image block; Step S2: when encoding all P frames or B frames between a certain I frame and the next I frame, the reference data is updated according to the encoding mode information of the image block at each position in each P frame or B frame; Each time the reference data is updated according to a certain P frame or B frame, if an image block at a certain position inside the P frame or B frame is encoded as an inter-frame coding block, further judgment is made according to the length of the motion vector of the image block; if any one of the horizontal component length or the vertical component length of the motion vector of the image block is greater than the judgment threshold, the usability flag of the image block at the corresponding position in the reference data is changed to unusable; otherwise, the reference data is not updated; Each time the reference data is updated according to a certain P frame or B frame, if an image block at a certain position inside the P frame or B frame is encoded as an intra-frame coding block, further judgment is made according to the availability flag of the image block at the corresponding position in the reference data; if the availability flag of the image block at the corresponding position in the reference data is available, the intra-frame coding mode information of the image block at the corresponding position in the reference data is changed to the intra-frame coding mode information of the image block on the P frame or B frame, and the availability flag of the image block at the corresponding position in the reference data is not modified; otherwise, not only the intra-frame coding mode information of the image block at the corresponding position in the reference data is changed to the intra-frame coding mode information of the image block on the P frame or B frame, but also the availability flag of the image block at the corresponding position in the reference data is changed to available; Step S3: when encoding the next I frame, if the image block at a certain position inside the next I frame is marked as available for the image block at the corresponding position in the reference data, the intra-frame coding mode information of the image block at the corresponding position in the reference data is introduced into the rate-distortion optimization operation of the intra-frame coding mode selection of the image block of the next I frame, and participates in the intra-frame coding mode selection of the image block of the next I frame; when the encoding of the next I frame is completed, return to step S1 to update the reference data as a whole.

2. The rate-distortion optimization method for suppressing breathing effect according to claim 1, characterized in that: The availability flag of the image block at each position is used to indicate whether the intra-frame coding mode information of the image block can be used in the rate-distortion optimization operation of selecting the intra-frame coding mode of the image block at the corresponding position of the next I frame.

3. The rate-distortion optimization method for suppressing breathing effect according to claim 1, characterized in that: In the step S1, the reference data is stored in an intra-frame coding mode table.

4. The rate-distortion optimization method for suppressing breathing effect according to claim 1, characterized in that: In the step S2, assuming that there are q P frames and B frames between a certain I frame and the next I frame, the reference data is updated and determined q times in sequence according to the sequence of the q P frames and B frames.

5. The rate distortion optimization method for suppressing breathing effect according to claim 1, characterized in that: In step S2, the determination threshold is set to 1, that is, 1 integer pixel distance.

6. The rate distortion optimization method for suppressing breathing effect according to claim 1, characterized in that: In the step S3, if the availability mark of the image block at a certain position within the next I frame for the image block at the corresponding position in the reference data is unavailable, the intra-frame coding mode information of the image block at the corresponding position in the reference data is not introduced into the rate-distortion optimization operation of the intra-frame coding mode selection of the image block in the next I frame.

7. The rate distortion optimization method for suppressing breathing effect according to claim 1, characterized in that: In step S3, when the intra-frame coding mode information of the image block at the corresponding position in the reference data is introduced into the rate-distortion optimization operation of the intra-frame coding mode selection of the image block in the next I frame, an intra-frame coding mode with the highest prediction cost is removed from the original intra-frame coding modes of the image block in the next I frame.

8. The rate distortion optimization method for suppressing breathing effect according to claim 1, characterized in that: In the step S3, when the intra-frame coding mode information of the image block at the corresponding position in the reference data is introduced into the rate-distortion optimization operation of the intra-frame coding mode selection of the image block in the next I frame, the rate-distortion cost corresponding to the introduced intra-frame coding mode is multiplied by different rate-distortion cost scaling coefficients according to the size of the image block, and then participates in the intra-frame coding mode selection of the image block in the next I frame; the rate-distortion cost scaling coefficient is negatively correlated with the size of the image block.

9. A rate distortion optimization device for suppressing breathing effect, characterized in that: It includes an intra-frame coding mode saving module, an intra-frame coding mode updating module and an intra-frame coding mode selecting module; The intra-frame coding mode saving module is used to save the intra-frame coding mode information and the usability flag of the image blocks at all positions in the I frame as reference data when the coding of each I frame is completed, and the usability flags of the image blocks at all positions are marked as usable; the intra-frame coding mode information of the image blocks at each position includes the position information of the image block, the size information of the image block, and the intra-frame prediction mode information of the image block; The intra-frame coding mode updating module is used to update the reference data according to the coding mode information of the image block at each position inside each P frame or B frame when encoding all P frames or B frames between a certain I frame and the next I frame; Each time the reference data is updated according to a certain P frame or B frame, if an image block at a certain position inside the P frame or B frame is encoded as an inter-frame coding block, further judgment is made according to the length of the motion vector of the image block; if any one of the horizontal component length or the vertical component length of the motion vector of the image block is greater than the judgment threshold, the usability flag of the image block at the corresponding position in the reference data is changed to unusable; otherwise, the reference data is not updated; Each time the reference data is updated according to a certain P frame or B frame, if an image block at a certain position inside the P frame or B frame is encoded as an intra-frame coding block, further judgment is made according to the availability flag of the image block at the corresponding position in the reference data; if the availability flag of the image block at the corresponding position in the reference data is available, the intra-frame coding mode information of the image block at the corresponding position in the reference data is changed to the intra-frame coding mode information of the image block on the P frame or B frame, and the availability flag of the image block at the corresponding position in the reference data is not modified; otherwise, not only the intra-frame coding mode information of the image block at the corresponding position in the reference data is changed to the intra-frame coding mode information of the image block on the P frame or B frame, but also the availability flag of the image block at the corresponding position in the reference data is changed to available; The intra-frame coding mode selection module is used to introduce the intra-frame coding mode information of the image block at the corresponding position in the reference data into the rate-distortion optimization operation of the intra-frame coding mode selection of the image block in the next I frame when encoding the next I frame, if the availability mark of the image block at the corresponding position in the reference data is available, and participate in the intra-frame coding mode selection of the image block in the next I frame.

Images