A rate-distortion optimization method and device based on motion detection

Abstract

The present application discloses a rate-distortion optimization method based on motion detection, comprising the following steps. Step S10: Using motion detection technology to identify the motion and still features of the current image block to be encoded. Step S20: Selecting a suitable Lagrangian multiplier for the current image block to be encoded from a plurality of preset Lagrangian multipliers based on the motion and still features of the current image block to be encoded and the motion and still features of the adjacent encoded image blocks. Step S30: Multiplying the selected Lagrangian multiplier with the Lagrangian constant of the motion estimation of the current image block to be encoded, and then replacing the Lagrangian constant of the motion estimation with the multiplication result to perform rate-distortion optimization operation of the motion estimation of the current image block to be encoded. The present application improves the Lagrangian constant used in the rate-distortion optimization process through the method of motion detection, thereby improving the video coding efficiency.

Description

Technical Field

[0001] The present application relates to a high-definition digital video coding technology, and in particular to a rate-distortion optimization method and device based on motion detection, suitable for hardware implementation, and applied to a motion estimation link. Background Art

[0002] Video coding is a technology that represents video information using minimal data by compressing redundant components within video images. Common video coding standards include HEVC (High Efficiency Video Coding, also known as H.265) and AVC (Advanced Video Coding, also known as H.264). Because video coding algorithms require significant computational effort, hardware acceleration of the video encoding process using application-specific integrated circuits (ASICs) is a common industry practice to increase encoding speed.

[0003] To maximize video compression rates, common video coding algorithms often use lossy compression algorithms. This results in differences between the reconstructed video and the original video, i.e., distortion. AVC and HEVC are examples of lossy compression algorithms. For lossy compression algorithms, coding performance is measured by both the encoding bitrate and the distortion introduced by the encoding. In the video encoding process, bitrate and distortion are mutually constrained and conflicting. For example, reducing the bitrate often increases distortion, while reducing distortion often leads to an increase in bitrate. Therefore, the goal of rate-distortion optimization (RDO) is to minimize distortion at a given bitrate, or to minimize the bitrate at a given distortion. Currently, common video coding algorithms such as AVC and HEVC use rate-distortion optimization methods based on the Lagrange constant λ. Assuming D is the coding distortion, R is the coding bit rate, and J is the rate distortion cost (RD Cost), the rate distortion optimization method based on the Lagrange constant λ can be expressed as: min(J), where min is the minimum function, J = D + λ × R. In the encoder, each coding mode has its corresponding coding distortion D and coding bit rate R, so the rate distortion cost J for each coding mode can be calculated. During encoding, selecting the coding mode with the lowest rate distortion cost J achieves optimal coding performance.

[0004] In the rate-distortion optimization method based on the Lagrange constant λ, the Lagrange constant λ is generally calculated from other preset video coding parameters. In HEVC, the Lagrange constant λ used in the motion estimation process is pred The calculation method is as follows.

[0005] Among them, λ pred It is the Lagrange constant used in the motion estimation process to calculate the coding rate distortion cost corresponding to a candidate MV (Motion Vector), λ mode It is the Lagrangian constant for calculating the rate-distortion cost J corresponding to a certain coding mode during the coding process, λ pred Derived from λ mode α and W k It is a constant related to preset coding parameters such as QP used in coding, reference frame structure, number of reference frames, and coding frame type. QP is the quantization step size used in coding.

[0006] Motion estimation is to search for the best matching block for the current image block to be encoded in the current encoded frame in the encoded video frame (called the reference frame), so as to minimize the rate distortion cost. Motion estimation is one of the core technologies of video coding algorithms. Its function is to eliminate the time domain information redundancy of the video signal, thereby improving the coding efficiency. Figure 1 During motion estimation, the current block to be coded in the current coding frame is matched one by one with all blocks within the reference frame search window (i.e., the motion estimation search range) to find the optimal matching block. The relative offset between the optimal matching block in the reference frame and the block in the current coding frame that is co-located with the current block to be coded in the current coding frame is the optimal motion vector for the current block to be coded.

[0007] See also Figure 2 The existing motion estimation module 80 is divided into a motion estimation search module 81 and a motion estimation rate-distortion optimization module 82. In the motion estimation search module 81, the current image block to be encoded on the current encoding frame is matched one by one with all image blocks in the reference frame search window, and the coding distortion D and coding bit rate R corresponding to each matching image block are calculated respectively. In the motion estimation rate-distortion optimization module 82, the coding distortion D and coding bit rate R corresponding to each matching image block and the Lagrangian constant λ are used to optimize the motion estimation rate-distortion optimization module 82. pred Calculate the corresponding coding rate-distortion cost J. After traversing each matching image block in the reference frame search window, the matching image block that minimizes the rate-distortion cost J is the optimal matching block of the current image block, and its corresponding motion vector is the optimal motion vector of the current image block to be encoded. Summary of the Invention

[0008] The technical problem to be solved by this application is how to improve the rate-distortion optimization method based on the Lagrange constant λ in the motion estimation process to improve the encoding performance of the video encoder.

[0009] In existing motion estimation, the Lagrange constant λ used to calculate the coding rate distortion cost is pred It is generally calculated from the preset coding parameters used in the video coding process, such as the quantization step size QP, reference frame structure, number of reference frames, and coding frame type. These preset coding parameters are only related to the coding process and have nothing to do with the content characteristics of the video sequence to be encoded (i.e., a video stream composed of multiple time-continuous image frames shot at a certain frame rate). When these preset coding parameters are determined, the Lagrange constant λ pred is a fixed value. That is, in existing video encoders, the Lagrange constant λ used for motion estimation rate-distortion optimization is pred It is a constant that is related to the preset encoding parameters but has nothing to do with the content characteristics of the video sequence to be encoded.

[0010] In the existing hardware video encoder, it is not found that the Lagrange constant λ used in the rate-distortion optimization process of motion estimation of image blocks by detecting the motion and static features of image blocks is used. pred However, the patent applicant has found through a large number of experiments that the Lagrange constant λ used in the motion estimation process can be calculated based on the motion and static characteristics of each image block to be encoded in the video sequence. pred Multiply by different specific multipliers (called Lagrange multipliers λ mul ), followed by rate-distortion optimization, can significantly improve video coding efficiency. Therefore, before motion estimation, it is necessary to use motion detection to identify the motion and stationary features of the image block to be encoded. Based on this, specific Lagrange multipliers are selected for rate-distortion optimization to improve the encoding performance of the video encoder.

[0011] In order to solve the above technical problems, the present application proposes a rate-distortion optimization method based on motion detection, which includes the following steps. Step S10: Use motion detection technology to identify the motion and still features of the current image block to be encoded. Step S20: According to the motion and still features of the current image block to be encoded and the motion and still features of the adjacent encoded image blocks, select a suitable Lagrangian multiplier for the current image block to be encoded from multiple preset Lagrangian multipliers. Step S30: Multiply the selected Lagrangian multiplier with the Lagrangian constant of the motion estimation of the current image block to be encoded, and then replace the Lagrangian constant of the motion estimation with the multiplication result to perform the rate-distortion optimization operation of the motion estimation of the current image block to be encoded.

[0012] The step S10 further includes the following steps. Step S11: Determine the motion and stillness characteristics of each pixel in the current image block to be encoded, that is, mark each pixel in the current image block to be encoded as a motion pixel or a still pixel. Step S12: Determine the motion and stillness characteristics of each image sub-block in the current image block to be encoded, that is, mark each image sub-block in the current image block to be encoded as a motion image sub-block or a still image sub-block. Step S13: Determine the motion and stillness characteristics of the current image block to be encoded, that is, mark the current image block to be encoded as a motion image block or a still image block. This is a specific implementation of step S10, which adopts the motion detection technology of the inter-frame difference method.

[0013] Specifically, in step S11, the current image block to be encoded is first differentially calculated with the same-position image block on the reference frame to calculate the brightness difference of each pixel in the image block; then the brightness difference of each pixel is compared with the motion pixel determination threshold; if the brightness difference of the pixel is greater than the motion pixel determination threshold, the pixel is marked as a motion pixel; otherwise, the pixel is marked as a stationary pixel.

[0014] Specifically, in step S12, the current image block to be encoded is divided into multiple image sub-blocks of the same size, each image sub-block contains a number of pixels, and the number of moving pixels in each image sub-block is counted; if the number of moving pixels in the image sub-block is greater than the determination threshold of the moving image sub-block, the image sub-block is marked as a moving image sub-block; otherwise, the image sub-block is marked as a still image sub-block.

[0015] Specifically, in step S13, the number of motion image sub-blocks in the current image block to be encoded is counted; if the number of motion image sub-blocks in the current image block to be encoded is greater than the determination threshold of the motion image block, the image block is marked as a motion image block; otherwise, the image block is marked as a still image block.

[0016] In step S20, the adjacent coded image blocks refer to the left adjacent coded image blocks and the upper adjacent coded image blocks of the current image block to be coded, which are specifically divided into the following five situations. (1) If the current image block to be coded is the first image block in the upper left corner of the current image frame, then its left adjacent and upper adjacent image blocks do not exist. (2) If the current image block to be coded is located in the first row of the current image frame, but is not the first image block in the upper left corner of the current image frame, then its upper adjacent image block does not exist. (3) If the current image block to be coded is located in the first column of the current image frame, but is not the first image block in the upper left corner of the current image frame, then its left adjacent image block does not exist. (4) If the current image block to be coded is located in the rightmost column of the current image frame, but is not the first image block in the upper right corner of the current image frame, then its upper adjacent image blocks are only 2. (5) In other cases except the above 4 cases, there are 3 upper adjacent coded image blocks and only 1 left adjacent coded image block of the current image block to be coded.

[0017] Furthermore, in step S20, the plurality of preset Lagrangian multipliers λ mul Refers to three, which are the Lagrange multipliers λ of the motion image block mul-dynamic , Lagrange multiplier λ of the still image block mul-static , Lagrange multiplier λ of the mixed image block mul-mixed , and λ mul-dynamic <λ mul-mixed <λ mul-static .

[0018] Furthermore, in step S20, the selection of a suitable Lagrangian multiplier means: (1) if the current image block to be encoded is marked as a motion image block, and its left adjacent and upper adjacent encoded image blocks are both marked as motion image blocks, then the Lagrangian multiplier λ of the motion image block is selected for it; mul-dynamic (2) If the current image block to be coded is marked as a still image block, and its left adjacent and upper adjacent coded image blocks are both marked as still image blocks, then the Lagrange multiplier λ of the still image block is selected for it. mul-static (3) If the current image block to be coded is marked as a motion image block, but there is a static image block in its left adjacent or upper adjacent coded image block, the Lagrange multiplier λ of the mixed image block is selected for it. mul-mixed (4) If the current image block to be coded is marked as a still image block, but there is a moving image block in its left adjacent or upper adjacent coded image block, the Lagrange multiplier λ of the mixed image block is selected for it. mul-mixed (5) If the current image block to be encoded is marked as a motion image block, and its left adjacent image block and upper adjacent image block do not exist, then the Lagrange multiplier λ of the motion image block is selected for it. mul-dynamic(6) If the current image block to be encoded is marked as a still image block and its left adjacent image block and upper adjacent image block do not exist, then the Lagrange multiplier λ of the still image block is selected for it. mul-static .

[0019] The present application also proposes a rate-distortion optimization device based on motion detection, comprising a motion detection module and a Lagrangian multiplier selection module. The motion detection module is configured to identify the motion and stationary features of the current image block to be encoded using a motion detection method. The Lagrangian multiplier selection module selects an appropriate Lagrangian multiplier for the current image block to be encoded from a plurality of preset Lagrangian multipliers based on the motion and stationary features of the current image block to be encoded and the motion and stationary features of adjacent encoded image blocks. The selected Lagrangian multiplier is multiplied by the Lagrangian constant of the motion estimation of the current image block to be encoded, and the multiplication result is then used to replace the Lagrangian constant of the motion estimation to perform a rate-distortion optimization operation on the motion estimation of the current image block to be encoded.

[0020] The present application also proposes a motion estimation device, including the rate-distortion optimization device based on motion detection, and also including a motion estimation search module and a motion estimation rate-distortion optimization module. The motion estimation search module is used to match the current image block to be encoded on the current encoding frame with all image blocks in the reference frame search window one by one, and calculate the coding distortion D and coding bit rate R corresponding to each matching image block respectively. The motion estimation rate-distortion optimization module is used to calculate the coding distortion D and coding bit rate R corresponding to each matching image block and the Lagrangian constant λ of the motion estimation. pred With the chosen Lagrange multiplier λ mul The multiplication result of the corresponding improved coding rate distortion cost J' is calculated; J'=D+λ pred ×λ mul ×R; take the minimum value of the improved coding rate distortion cost J'.

[0021] The technical principle of this application is that for image blocks with different motion characteristics, the importance of distortion cost and encoding bit rate cost is different in the process of calculating the rate-distortion cost of motion estimation. For moving image blocks, the importance of distortion cost is higher, so a smaller Lagrange multiplier λ is set. mul-dynamic Better coding efficiency can be achieved. For still image blocks, the coding bit rate cost is more important, so a larger Lagrange multiplier λ is set. mul-static Better coding efficiency can be achieved. For mixed image blocks, the Lagrange multiplier λ mul-mixed The value of should be between the first two cases, that is, it needs to satisfy λ mul-dynamic <λ mul-mixed <λ mul-staticIn the traditional rate-distortion optimization method of motion estimation, the rate-distortion optimization calculation is not adjusted according to the motion characteristics of the image block. Therefore, the present application can improve the coding efficiency compared with the existing technology.

[0022] The present application first discovered in the experiment that during the motion estimation process, the Lagrange constant λ pred It should not be a quantity unrelated to the video content, but should be enlarged and reduced according to the motion and static characteristics of the image block, that is, it needs to be multiplied by the Lagrange multiplier λ mul , and needs to satisfy the Lagrange multiplier λ used by the motion image block mul-dynamic <Lagrange multiplier λ used in blending image blocks mul-mixed <Lagrange multiplier λ used for the still image block mul-static The specific implementation method is to determine the motion characteristics of the image block through motion detection and select a suitable Lagrange multiplier λ mul To λ pred Multiply (i.e. scale), and the Lagrange multiplier λ adapted to different motion characteristics mul The specific value of is determined in advance by a large number of experiments.

[0023] The technical effect achieved by this application is to improve the Lagrange constant used in the rate-distortion optimization process through motion detection methods, thereby improving video coding efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a schematic diagram of motion estimation.

[0025] Figure 2 It is a structural diagram of an existing motion estimation module.

[0026] Figure 3 This is a flow chart of the rate-distortion optimization method based on motion detection proposed in this application.

[0027] Figure 4 This is a structural diagram of the rate-distortion optimization device based on motion detection proposed in this application.

[0028] Figure 5 It is a flowchart of an exemplary implementation method of step S10.

[0029] Figure 6 Schematic diagram of the upper adjacent image block and the left adjacent image block.

[0030] Explanation of the reference numerals in the figure: 80 is the existing motion estimation module, 81 is the motion estimation search module, 82 and 82a are the motion estimation rate-distortion optimization modules, 90 is the motion estimation module proposed in this application, 91 is the motion detection module, and 92 is the Lagrange multiplier selection module. DETAILED DESCRIPTION

[0031] See also Figure 3 The present application proposes a rate-distortion optimization method based on motion detection, suitable for hardware implementation, and applied to the motion estimation link, which includes the following steps.

[0032] Step S10: Use motion detection technology to identify the motion and stillness features of the current image block to be encoded. Motion detection is a method for identifying whether there are moving objects in a given video image sequence. Currently, commonly used motion detection technologies include frame difference method (Frame Diff Method), background subtraction method (Background Subtraction Method), optical flow method (Optical Flow Method), Gaussian Mixture Model (GMM) method, and motion detection methods based on machine learning (Machine Learning).

[0033] Step S20: According to the motion and still characteristics of the current image block to be encoded and the motion and still characteristics of the adjacent encoded image blocks, a plurality of preset Lagrangian multipliers λ are selected. mul Select the appropriate Lagrange multiplier λ for the current image block to be encoded mul .

[0034] Step S30: Set the selected Lagrange multiplier λ mul The Lagrangian constant λ of the motion estimation of the current image block to be encoded pred Multiply, and then replace the multiplication result with the Lagrange constant λ of the motion estimation pred Then perform rate-distortion optimization operation of motion estimation of the current image block to be encoded.

[0035] See also Figure 4 The present application proposes a motion detection-based rate-distortion optimization device suitable for hardware implementation and applied to the motion estimation process, including a motion detection module 91 and a Lagrange multiplier selection module 92. These two modules, combined with the motion estimation search module 81 and the motion estimation rate-distortion optimization module 82a, constitute the motion estimation module 90 proposed in the present application.

[0036] The motion detection module 91 is used to identify the motion and stillness characteristics of the current image block to be encoded using a motion detection method. The Lagrangian multiplier selection module 92 selects an appropriate Lagrangian multiplier λ for the current image block to be encoded from a plurality of preset Lagrangian multipliers based on the motion and stillness characteristics of the current image block to be encoded and the motion and stillness characteristics of adjacent encoded image blocks. mul The motion estimation search module 81 is used to match the current image block to be coded on the current coding frame with all image blocks in the reference frame search window one by one, and calculate the coding distortion D and coding bit rate R corresponding to each matching image block. The motion estimation rate-distortion optimization module 82a is used to calculate the coding distortion D and coding bit rate R corresponding to each matching image block and the Lagrangian constant λ of the motion estimation. pred With the chosen Lagrange multiplier λ mul The multiplication result of the improved coding rate distortion cost J' is calculated. The improved rate distortion optimization method based on the Lagrange constant λ of this application can be expressed as: min(J'), where min is the minimum value function, J'=D+λ pred ×λ mul ×R.

[0037] See also Figure 5 An exemplary implementation of step S10 further includes the following steps.

[0038] Step S11: Determine the motion and stillness characteristics of each pixel in the current image block to be encoded. Specifically, a frame difference (inter-frame difference) is performed between the current image block to be encoded and the co-located image block in the reference frame, and the brightness difference of each pixel in the image block is calculated. The brightness difference of each pixel is then compared with the motion pixel determination threshold. If the brightness difference of the pixel is greater than the motion pixel determination threshold, the pixel is marked as a moving pixel; otherwise, the pixel is marked as a still pixel.

[0039] Step S12: Determine the motion and static characteristics of each image sub-block in the current image block to be encoded. After marking the motion and static characteristics of each pixel in the current image block to be encoded, dilation and erosion operations are performed on the image sub-blocks to highlight the image's motion characteristics, eliminate image noise, and reduce the impact of missed and false positives. Image dilation and erosion are two basic morphological operations, primarily used to find maximum and minimum regions in an image. Dilation is similar to "domain expansion," expanding the image's highlights or white areas, resulting in an image with a larger image than the original image's highlights. Erosion is similar to "domain erosion," reducing and thinning the image's highlights or white areas, resulting in an image with a smaller image than the original image's highlights. This step involves dividing the current image block to be encoded into multiple image sub-blocks of equal size, each containing a number of pixels, and counting the number of moving pixels in each sub-block. If the number of moving pixels in the image sub-block is greater than the determination threshold of the moving image sub-block, the image sub-block is marked as a moving image sub-block; otherwise, the image sub-block is marked as a still image sub-block.

[0040] Step S13: Determine the motion and stillness characteristics of the current image block to be encoded. Specifically, after marking the motion and stillness characteristics of each image sub-block in the current image block to be encoded, the number of moving image sub-blocks in the current image block to be encoded is counted. If the number of moving image sub-blocks in the current image block to be encoded is greater than the determination threshold for moving image blocks, the image block is marked as a moving image block; otherwise, the image block is marked as a still image block. The motion and stillness characteristics of the current image block to be encoded refer to whether the current image block to be encoded is marked as a moving image block or a still image block.

[0041] It should be noted that there are many methods for performing motion detection on image blocks and determining their motion and stillness characteristics, and they are not limited to the methods described in the three paragraphs above. For example, background subtraction, optical flow, Gaussian mixture models, and machine learning-based motion detection methods can also be used for determination.

[0042] In a typical video scene, there are generally moving foreground objects and stationary background objects. For example, in a road surveillance video scene, objects that do not move by themselves, such as roads, streetlights, and buildings, can be determined as stationary background objects, while objects that move by themselves, such as pedestrians and vehicles, can be determined as moving foreground objects. Of course, there may also be mutual conversion between moving foreground objects and stationary background objects. For example, during the video recording process, a vehicle that has been parked on the side of the road and has not moved can be determined as part of the stationary background. For example, the image blocks belonging to the moving foreground objects are referred to as moving image blocks, the image blocks belonging to the stationary background objects are referred to as stationary image blocks, and the image blocks containing both moving foreground objects and stationary background objects are referred to as mixed image blocks. The patent applicant has found through a large number of experiments that in the rate-distortion optimization operation process of motion estimation, different Lagrange multipliers λ are set for the moving image blocks, the stationary image blocks, and the mixed image blocks. mul , then λ mul and the Lagrangian constant λ of motion estimation pred After multiplication, the rate-distortion optimization operation can significantly improve the video coding efficiency. Therefore, the Lagrangian multiplier selection module will preset the Lagrangian multiplier λ suitable for the motion image block. mul-dynamic , Lagrange multiplier λ of the static image block mul-static and the Lagrange multiplier λ of the mixed image patch mul-mixed , these Lagrange multipliers λ of image blocks adapted to different motion and static characteristics mul Their specific values ​​are determined by experiments, but they need to satisfy λ mul-dynamic <λ mul-mixed <λ mul-static .

[0043] See also Figure 6 In step S20, the adjacent coded image blocks refer to the left adjacent coded image blocks and the upper adjacent coded image blocks of the current image block to be coded. The motion and static characteristics of these adjacent coded image blocks are obtained when they are coded. For the current image block to be coded, there are at most 3 upper adjacent coded image blocks and at most 1 left adjacent coded image block, which are specifically divided into the following 5 situations. (1) If the current image block to be coded is the first image block in the upper left corner of the current image frame, then its left adjacent and upper adjacent image blocks do not exist. At this time, the Lagrange multiplier λ is selected based on the motion and static characteristics of the current image block to be coded. mul (2) If the current image block to be encoded is located in the first row of the current image frame (excluding the first image block in the upper left corner of the current image frame), then the upper adjacent image block does not exist. In this case, the Lagrange multiplier λ is selected only with reference to the left adjacent image block. mul(3) If the current image block to be encoded is located in the first column of the current image frame (excluding the first image block in the upper left corner of the current image frame), then its left adjacent image block does not exist. In this case, the Lagrange multiplier λ is selected only with reference to the upper adjacent image block. mul (4) If the current image block to be encoded is located in the rightmost column of the current image frame (excluding the first image block in the upper right corner of the current image frame), then it has only two upper adjacent image blocks. (5) In cases other than the above four, the current image block to be encoded has three upper adjacent coded image blocks and only one left adjacent coded image block.

[0044] In step S20, the plurality of preset Lagrangian multipliers λ mul Refers to three, which are the Lagrange multipliers λ of the motion image block mul-dynamic , Lagrange multiplier λ of the still image block mul-static , Lagrange multiplier λ of the mixed image block mul-mixed , and λ mul-dynamic <λ mul-mixed <λ mul-static The selection of a suitable Lagrange multiplier λ mul It means: (1) If the current image block to be coded is marked as a motion image block, and its left adjacent and upper adjacent coded image blocks are both marked as motion image blocks, then the Lagrange multiplier λ of the motion image block is selected for it. mul-dynamic (2) If the current image block to be coded is marked as a still image block, and its left adjacent and upper adjacent coded image blocks are both marked as still image blocks, then the Lagrange multiplier λ of the still image block is selected for it. mul-static (3) If the current image block to be coded is marked as a motion image block, but there is a static image block in its left adjacent or upper adjacent coded image block, the Lagrange multiplier λ of the mixed image block is selected for it. mul-mixed (4) If the current image block to be coded is marked as a still image block, but there is a moving image block in its left adjacent or upper adjacent coded image block, the Lagrange multiplier λ of the mixed image block is selected for it. mul-mixed (5) If the current image block to be encoded is marked as a motion image block, and its left adjacent image block and upper adjacent image block do not exist, then the Lagrange multiplier λ of the motion image block is selected for it. mul-dynamic (6) If the current image block to be encoded is marked as a still image block and its left adjacent image block and upper adjacent image block do not exist, then the Lagrange multiplier λ of the still image block is selected for it. mul-static .

[0045] The rate-distortion optimization method for motion estimation proposed in this application can significantly improve the encoding effect of the video encoder. In order to verify the beneficial effects of this application, the present invention selected 20 YUV sequences of HEVC standard test sequences Class A to F for testing. In the test, the rate-distortion optimization method proposed in this application was used as a comparison benchmark to measure the changes in the encoding BD-rate of each YUV sequence after the rate-distortion optimization method proposed in this application was adopted. BD-rate is " Delta bit rate is used to measure the encoding efficiency of an encoder. A smaller BD-rate means fewer bits are used to achieve the same video quality, resulting in higher encoding efficiency. The experimental results are shown in Table 1 below.

[0046]

[0047]

[0048] Table 1: Changes in BD-rate for each YUV sequence after using the rate-distortion optimization method proposed in this application

[0049] As can be seen from Table 1, after applying the rate-distortion optimization method proposed in this application, the encoding BD-rate of Class A to F standard test sequences is reduced by an average of 0.70%, and the coding efficiency is effectively improved; especially for the test sequence of Class E video conferencing scenario, the coding efficiency is significantly improved.

[0050] The above are only preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. A rate-distortion optimization method based on motion detection, characterized in that: The method includes the following steps: Step S10: using motion detection technology to identify the motion and stillness features of the current image block to be encoded; Step S20: selecting an appropriate Lagrangian multiplier for the current image block to be encoded from a plurality of preset Lagrangian multipliers according to the motion and stillness characteristics of the current image block to be encoded and the motion and stillness characteristics of adjacent encoded image blocks; The plurality of preset Lagrange multipliers λ mul Refers to three, which are the Lagrange multipliers λ of the motion image block mul-dynamic , Lagrange multiplier λ of the static image block mul-static , Lagrange multiplier λ of the mixed image block mul-mixed , and λ mul-dynamic <λ mul-mixed <λ mul-static ; The selection of appropriate Lagrange multipliers refers to: (1) If the current image block to be coded is marked as a motion image block, and its left adjacent and upper adjacent coded image blocks are both marked as motion image blocks, then the Lagrange multiplier λ of the motion image block is selected for it. mul-dynamic ; (2) If the current image block to be coded is marked as a still image block, and its left adjacent and upper adjacent coded image blocks are both marked as still image blocks, then the Lagrange multiplier λ of the still image block is selected for it. mul-static ; (3) If the current image block to be coded is marked as a motion image block, but there is a static image block in its left adjacent or upper adjacent coded image block, the Lagrange multiplier λ of the mixed image block is selected for it. mul-mixed ; (4) If the current image block to be coded is marked as a still image block, but there is a moving image block in its left adjacent or upper adjacent coded image block, the Lagrange multiplier λ of the mixed image block is selected for it. mul-mixed ; (5) If the current image block to be encoded is marked as a motion image block and its left adjacent image block and upper adjacent image block do not exist, then the Lagrange multiplier λ of the motion image block is selected for it. mul-dynamic ; (6) If the current image block to be encoded is marked as a still image block and its left adjacent image block and upper adjacent image block do not exist, then the Lagrange multiplier λ of the still image block is selected for it. mul-static ; Step S30: multiplying the selected Lagrangian multiplier by the Lagrangian constant of motion estimation of the current image block to be encoded, and then replacing the Lagrangian constant of motion estimation with the multiplication result to perform rate-distortion optimization operation of motion estimation of the current image block to be encoded.

2. The rate-distortion optimization method based on motion detection according to claim 1, wherein: The step S10 further includes the following steps: Step S11: determining the motion and static characteristics of each pixel in the current image block to be encoded, that is, marking each pixel in the current image block to be encoded as a motion pixel or a static pixel; Step S12: determining the motion and stillness characteristics of each image sub-block in the current image block to be encoded, that is, marking each image sub-block in the current image block to be encoded as a motion image sub-block or a still image sub-block; Step S13: determining the motion or still characteristics of the current image block to be encoded, that is, marking the current image block to be encoded as a motion image block or a still image block.

3. The rate-distortion optimization method based on motion detection according to claim 2, wherein: In step S11, a frame difference is first performed between the current image block to be encoded and the same-position image block on the reference frame to calculate the brightness difference of each pixel in the image block; then the brightness difference of each pixel is compared with the motion pixel determination threshold; If the brightness difference of a pixel is greater than the motion pixel determination threshold, the pixel is marked as a motion pixel; otherwise, the pixel is marked as a stationary pixel.

4. The rate-distortion optimization method based on motion detection according to claim 2, wherein: In step S12, the current image block to be encoded is divided into multiple image sub-blocks of the same size, each image sub-block contains a number of pixels, and the number of moving pixels in each image sub-block is counted; if the number of moving pixels in the image sub-block is greater than the determination threshold of the moving image sub-block, the image sub-block is marked as a moving image sub-block; otherwise, the image sub-block is marked as a still image sub-block.

5. The rate-distortion optimization method based on motion detection according to claim 2, wherein: In step S13, the number of motion image sub-blocks in the current image block to be encoded is counted; if the number of motion image sub-blocks in the current image block to be encoded is greater than the determination threshold of the motion image block, the image block is marked as a motion image block; otherwise, the image block is marked as a still image block.

6. The rate-distortion optimization method based on motion detection according to claim 1, wherein: In step S20, the adjacent coded image blocks refer to the left adjacent coded image blocks and the upper adjacent coded image blocks of the current image block to be coded, which are specifically divided into the following five situations: (1) If the current image block to be encoded is the first image block in the upper left corner of the current image frame, then its left adjacent image block and upper adjacent image block do not exist; (2) If the current image block to be encoded is located in the first row of the current image frame, but is not the first image block in the upper left corner of the current image frame, then the adjacent image block does not exist; (3) If the current image block to be encoded is located in the first column of the current image frame, but is not the first image block in the upper left corner of the current image frame, then its left adjacent image block does not exist; (4) If the current image block to be encoded is located in the rightmost column of the current image frame, but is not the first image block in the upper right corner of the current image frame, then there are only two adjacent image blocks above it; (5) In cases other than the above four cases, there are three upper adjacent coded image blocks and only one left adjacent coded image block to the current image block to be coded.

7. A rate-distortion optimization device based on motion detection, characterized in that: It includes a motion detection module and a Lagrange multiplier selection module; The motion detection module is used to identify the motion and stillness features of the current image block to be encoded using a motion detection method; The Lagrangian multiplier selection module selects an appropriate Lagrangian multiplier for the current image block to be encoded from a plurality of preset Lagrangian multipliers according to the motion and stillness characteristics of the current image block to be encoded and the motion and stillness characteristics of adjacent encoded image blocks; The plurality of preset Lagrange multipliers λ mul Refers to three, which are the Lagrange multipliers λ of the motion image block mul-dynamic , Lagrange multiplier λ of the static image block mul-static , Lagrange multiplier λ of the mixed image block mul-mixed , and λ mul-dynamic <λ mul-mixed <λ mul-static ; The selection of appropriate Lagrange multipliers refers to: (1) If the current image block to be coded is marked as a motion image block, and its left adjacent and upper adjacent coded image blocks are both marked as motion image blocks, then the Lagrange multiplier λ of the motion image block is selected for it. mul-dynamic ; (2) If the current image block to be coded is marked as a still image block, and its left adjacent and upper adjacent coded image blocks are both marked as still image blocks, then the Lagrange multiplier λ of the still image block is selected for it. mul-static ; (3) If the current image block to be coded is marked as a motion image block, but there is a static image block in its left adjacent or upper adjacent coded image block, the Lagrange multiplier λ of the mixed image block is selected for it. mul-mixed ; (4) If the current image block to be coded is marked as a still image block, but there is a moving image block in its left adjacent or upper adjacent coded image block, the Lagrange multiplier λ of the mixed image block is selected for it. mul-mixed ; (5) If the current image block to be encoded is marked as a motion image block and its left adjacent image block and upper adjacent image block do not exist, then the Lagrange multiplier λ of the motion image block is selected for it. mul-dynamic ; (6) If the current image block to be encoded is marked as a still image block and its left adjacent image block and upper adjacent image block do not exist, then the Lagrange multiplier λ of the still image block is selected for it. mul-static ; The selected Lagrange multiplier is multiplied by the Lagrange constant of motion estimation of the current image block to be encoded, and then the multiplication result replaces the Lagrange constant of motion estimation and performs rate-distortion optimization operation of motion estimation of the current image block to be encoded.

8. A motion estimation device, characterized in that: The method comprises the rate-distortion optimization device based on motion detection according to claim 7, further comprising a motion estimation search module and a motion estimation rate-distortion optimization module; The motion estimation search module is used to match the current image block to be encoded on the current encoding frame with all image blocks in the reference frame search window one by one, and calculate the coding distortion D and coding bit rate R corresponding to each matching image block respectively; The motion estimation rate-distortion optimization module is used to calculate the Lagrange constant λ of the motion estimation based on the coding distortion D and coding bit rate R corresponding to each matching image block. pred With the chosen Lagrange multiplier λ mul The multiplication result of the corresponding improved coding rate distortion cost J' is calculated; J'=D+λ pred ×λ mul ×R; take the minimum value of the improved coding rate distortion cost J'.

Images