A motion estimation method and apparatus based on brightness distribution patterns

Abstract

The present invention discloses a motion estimation method based on brightness distribution pattern. The image blocks to be encoded in the current frame and all matching blocks in the reference frame are preprocessed, including: generating downsampled image blocks corresponding to all the aforementioned image blocks, and extracting brightness distribution patterns of all the aforementioned image blocks. The downsampled image blocks to be encoded are matched with all the downsampled matching blocks one by one, and the coding distortion, coding bit rate, and brightness distribution pattern difference corresponding to each downsampled matching block are calculated respectively. The corresponding rate-distortion cost is calculated according to the coding distortion, coding bit rate, and brightness distribution pattern difference corresponding to each downsampled matching block, and the downsampled matching block with the smallest rate-distortion cost is selected as the optimal matching block of the downsampled image block to be encoded. The present invention introduces the difference in brightness distribution pattern of image blocks into the matching process of coarse-grained search, thereby improving the accuracy of image block matching.

Description

Technical Field

[0001] The invention relates to a digital video encoding technology, and in particular to a motion estimation method. Background Art

[0002] Video coding refers to compressing the redundant components in video images and using as little data as possible to represent video information. Common video coding standards include AVC (Advanced Video Coding, also known as H.264), HEVC (High Efficiency Video Coding, also known as H.265), etc. Video coding technology uses image blocks as the most basic coding units. For example, in HEVC, the basic unit of coding is CU (Coding Unit). CU can be an image block of 64 pixels × 64 pixels, or 32 pixels × 32 pixels, or 16 pixels × 16 pixels, or 8 pixels × 8 pixels.

[0003] In order to maximize the video compression rate, common video coding algorithms such as AVC and HEVC are often lossy compression algorithms, which causes the difference between the encoded and reconstructed video and the original video, that is, the encoded and reconstructed video will produce distortion. For lossy compression algorithms, their coding performance needs to be measured by the coding bit rate (Rate) and the distortion (Distortion) introduced by the coding. In the video coding process, the coding bit rate and coding distortion are mutually restricted and contradictory. For example, reducing the coding bit rate often increases the coding distortion, while reducing the coding distortion often increases the coding bit rate. The purpose of rate distortion optimization (RDO) is to reduce the coding distortion as much as possible under a certain coding bit rate, or to reduce the coding bit rate as much as possible under a certain coding distortion. In common video coding algorithms such as AVC and HEVC, rate distortion optimization methods based on Lagrange constant λ are used. Assume that D is the coding distortion, R is the coding bit rate, and J is the coding rate distortion cost (Rate Distortion Cost, RD Cost), J = D + λ × R. The rate-distortion optimization method based on the Lagrange constant λ can be expressed as min(J), where min() means taking the minimum value. In a video encoder, each coding mode has its corresponding coding distortion D and coding bit rate R, so the coding rate-distortion cost J corresponding to each coding mode can be calculated. When encoding a video, the coding mode with the smallest rate-distortion cost J is selected for encoding to obtain the best coding performance.

[0004] Motion estimation is to search for the best matching block for the image block to be encoded in the current encoded video frame (called the current frame) in the encoded video frame (called the reference frame), so as to minimize the rate distortion cost. Motion estimation is one of the core technologies of video coding algorithms. Its function is to eliminate the time domain information redundancy of video signals, thereby improving coding efficiency. Figure 1 , the relative offset between the best matching block in the reference frame and the image block to be encoded in the current frame in the same position image block (image block at the same position) in the reference frame is the best motion vector (MV) of the image block to be encoded. During motion estimation, the image block to be encoded on the current frame is matched one by one with all image blocks (called matching blocks) in the reference frame search window to find the best matching block. Each matching block can also be regarded as a coding mode, so it also corresponds to different coding distortion D and coding bit rate R. Image block matching refers to calculating the coding distortion D and coding bit rate R corresponding to each matching block, and calculating its corresponding rate distortion cost J according to the Lagrangian constant λ. The matching block with the smallest rate distortion cost J is the best matching block of the image block to be encoded. The reference frame search window is the search range of motion estimation, which is determined by the position of the search center and the search range, which may include or exclude the same position image block.

[0005] In existing video encoders, especially in hardware video encoders, in order to reduce the amount of motion estimation calculations and improve the execution speed of motion estimation, motion estimation is generally designed as a combination of coarse-grained search and fine-grained search. Coarse-grained search is a downsampling (also called downsampling) search in a larger range with a preset point as the search center to obtain the optimal coarse pixel precision motion vector of the image block to be encoded. The preset point is usually set to (0, 0), that is, the upper left corner vertex of the image block to be encoded. Fine-grained search is a local full search in a smaller range with the optimal coarse pixel precision motion vector as the search center to obtain the optimal integer pixel precision motion vector of the image block to be encoded.

[0006] In traditional coarse-grained search, the downsampling process is generally implemented as follows. The image block of the original resolution is divided into one or more image sub-blocks of the same size, and each image sub-block corresponds to a downsampled pixel. Common image sub-block sizes are 2 pixels × 2 pixels, or 4 pixels × 4 pixels, or 8 pixels × 8 pixels. The average brightness of all pixels in each image sub-block is calculated as the brightness value of the pixel after the image sub-block is downsampled, and the downsampled image block is obtained from the image block in this way. For example, for an image block of the original resolution of 64 pixels × 64 pixels, if the downsampling step size in the horizontal and vertical directions is 4 pixels, the image block will be divided into 16 (rows) × 16 (columns) image sub-blocks. The size of each image sub-block is 4 pixels × 4 pixels, and each image sub-block corresponds to a downsampled pixel. The average brightness of 16 pixels in each image sub-block is calculated as the brightness value of a downsampled pixel. In this way, the image block of the original resolution of 64 pixels × 64 pixels is downsampled to an image block of 16 pixels × 16 pixels.

[0007] In the above-mentioned downsampling implementation, the brightness value of each pixel after downsampling is the average brightness value of all pixels in the corresponding image sub-block, which can reflect the average brightness of the entire image sub-block, but cannot reflect the relative difference in brightness between pixels at various positions within the image sub-block, that is, it cannot reflect the brightness distribution pattern of the image sub-block. Since the downsampling process loses the brightness distribution pattern information between pixels within the image sub-block, the coarse-grained search will search for a non-optimal downsampled matching block for the downsampled image block to be encoded during the image block matching process, resulting in inaccurate motion estimation results.

[0008] See also Figure 2 , which is an example of downsampling causing errors in optimal image patch matching during the coarse-grained search process. Figure 2In the figure, the left side is a 4×4 pixel image block to be encoded, and the right side is a matching block A and a matching block B of the same shape and size. The hollow squares represent the original resolution pixels, the patterned squares represent the downsampled pixels, and the numbers in the squares represent the brightness values ​​of the pixels. Assuming that the downsampling step lengths in the horizontal and vertical directions of the coarse-grained search are both 4 pixels, it is necessary to find the best matching block of the image block to be encoded in the matching block A and the matching block B. It can be seen that the brightness values ​​of each pixel in the image block to be encoded are exactly the same as the brightness values ​​of each pixel in the matching block A, so the coarse-grained search should select the matching block A as the best matching block. However, if the aforementioned downsampling implementation method is adopted, the pixel brightness values ​​of the downsampled image block to be encoded, the downsampled matching block A, and the downsampled matching block B are all 4. In this way, the coarse-grained search may select the downsampled matching block B as the best matching block of the downsampled image block to be encoded, and the motion estimation result is inaccurate.

[0009] In the existing motion estimation methods that combine coarse-grained search and fine-grained search, no method has been found to improve the coarse-grained search process based on the brightness distribution pattern of the image block to improve the accuracy of motion estimation. Summary of the invention

[0010] The technical problem to be solved by the present invention is to improve the coarse-grained search method to improve the accuracy of motion estimation in view of the problem of inaccurate matching of image blocks after downsampling in the existing coarse-grained search method.

[0011] In order to solve the above technical problems, the present invention proposes a motion estimation method based on brightness distribution pattern, comprising the following steps. Step S1: Preprocessing the image block to be encoded in the current frame and all matching blocks in the reference frame, including: generating downsampled image blocks corresponding to all the aforementioned image blocks, and extracting the brightness distribution pattern of all the aforementioned image blocks; the brightness distribution pattern of the image block represents the brightness distribution between pixels in different areas within each image block of the original resolution. Step S2: Match the downsampled image block to be encoded with all downsampled matching blocks one by one, and calculate the coding distortion, coding bit rate, and brightness distribution pattern difference corresponding to each downsampled matching block respectively. Step S3: Calculate the corresponding rate-distortion cost according to the coding distortion, coding bit rate, and brightness distribution pattern difference corresponding to each downsampled matching block, and select the downsampled matching block with the smallest rate-distortion cost as the optimal matching block of the downsampled image block to be encoded.

[0012] Furthermore, in the step S1, generating a downsampled image block corresponding to an image block of original resolution includes the following sub-steps. Step S11: Divide each image block of original resolution into one or more image sub-blocks of the same shape and size, the width of each image sub-block is equal to the downsampling step in the horizontal direction, and the height of each image sub-block is equal to the downsampling step in the vertical direction, so each image sub-block corresponds to a downsampled pixel. Step S12: Calculate the average brightness value of all pixels in each image sub-block as the brightness value of a pixel after the image sub-block is downsampled, and so on to obtain the downsampled image block from the image block of original resolution.

[0013] Furthermore, in step S1, extracting the brightness distribution pattern of an image block of original resolution includes the following sub-steps. Step S13: Divide each image sub-block within each image block of original resolution into multiple sub-regions of the same number, each sub-region has the same shape and size, and each sub-region corresponds to a brightness distribution pattern bit. Step S14: Calculate the brightness average of all pixels within each sub-region. Step S15: Calculate the brightness distribution pattern bit corresponding to each sub-region based on the brightness average of each sub-region and the brightness distribution determination threshold. Step S16: Arrange the brightness distribution pattern bits of all sub-regions within each image sub-block in a specific order to obtain the brightness distribution pattern of the image sub-block; the brightness distribution pattern of the image sub-block represents the brightness distribution between pixels in different regions within each image sub-block of original resolution. Step S17: Arrange the brightness distribution pattern of each image sub-block within the image block of original resolution in a specific order to obtain the brightness distribution pattern of the image block of original resolution. Steps S13 to S17 are performed at any position after step S11.

[0014] Preferably, in step S15, when the brightness average value of a sub-region is ≥ the brightness distribution determination threshold, the brightness distribution mode bit corresponding to the sub-region is 1; when the brightness average value of a sub-region is < the brightness distribution determination threshold, the brightness distribution mode bit corresponding to the sub-region is 0.

[0015] Preferably, in step S16, the specific order is in accordance with the row scanning order of each sub-region within the image sub-block, first from left to right, then from top to bottom; in step S17, the specific order is in accordance with the row scanning order of each image sub-block within the image block, first from left to right, then from top to bottom.

[0016] Furthermore, in step S2, the coding distortion corresponding to each downsampled matching block is obtained by a distortion cost calculation method.

[0017] Furthermore, in step S2, the encoding bit rate corresponding to each downsampled matching block is obtained by a bit rate cost calculation method.

[0018] Furthermore, in step S2, the brightness distribution pattern difference corresponding to each downsampled matching block is the number of brightness distribution pattern bits with different values ​​between the brightness distribution pattern of the image block to be encoded at the original resolution and the brightness distribution pattern of each matching block at the original resolution.

[0019] Further, in step S3, D represents the coding distortion corresponding to a downsampled matching block, R represents the coding bit rate corresponding to the downsampled matching block, M represents the brightness distribution pattern difference corresponding to the downsampled matching block, JJ represents the rate-distortion cost corresponding to the downsampled matching block, JJ=D+λ×R+ω×M; wherein λ represents the Lagrangian constant, ω represents the weight coefficient of the brightness distribution pattern difference, and ω>0.

[0020] The present invention also proposes a motion estimation device based on brightness distribution pattern, including a coarse-grained search preprocessing module, a coarse-grained search matching module and a coarse-grained search rate-distortion optimization module. The coarse-grained search preprocessing module is used to preprocess the image block to be encoded in the current frame and all the matching blocks in the reference frame, including: generating downsampled image blocks corresponding to all the aforementioned image blocks, extracting the brightness distribution patterns of all the aforementioned image blocks; the brightness distribution pattern of the image block represents the brightness distribution between the pixels in different regions within each image block of the original resolution. The coarse-grained search matching module is used to match the downsampled image block to be encoded with all the downsampled matching blocks one by one, and respectively calculate the coding distortion, coding bit rate, and brightness distribution pattern difference corresponding to each downsampled matching block. The coarse-grained search rate-distortion optimization module is used to calculate the corresponding new rate-distortion cost according to the coding distortion, coding bit rate, and brightness distribution pattern difference corresponding to each downsampled matching block, and select the downsampled matching block with the smallest new rate-distortion cost as the optimal matching block of the downsampled image block to be encoded.

[0021] The technical effect achieved by the present invention is as follows: the motion estimation method includes extracting the brightness distribution pattern of the image block, and introducing the difference calculation of the brightness distribution pattern of the image block into the matching process of the coarse-grained search and the rate-distortion optimization process, thereby overcoming the limitations of the traditional coarse-grained search method, improving the accuracy of image block matching, and effectively improving the accuracy of motion estimation. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a schematic diagram of motion estimation.

[0023] Figure 2This is a schematic diagram of the optimal image block matching error caused by downsampling during the coarse-grained search process.

[0024] Figure 3 It is a flowchart of the motion estimation method based on brightness distribution pattern proposed by the present invention.

[0025] Figure 4 It is a flowchart of generating an image block after downsampling an image block of an original resolution in step S1.

[0026] Figure 5 It is a schematic diagram of the process of extracting a brightness distribution pattern of an image block of original resolution in step S1.

[0027] Figure 6 is a schematic diagram of a specific example of steps S13 to S16.

[0028] Figure 7 is a schematic diagram of a specific example of step S17.

[0029] Figure 8 It is a schematic diagram showing how the image block matching accuracy is improved after the brightness distribution pattern of the image block is introduced.

[0030] Fig. 9 It is a structural schematic diagram of a motion estimation device based on brightness distribution pattern proposed by the present invention.

[0031] Explanation of the reference numerals in the figure: 1 is a coarse-grained search preprocessing module, 2 is a coarse-grained search matching module, and 3 is a coarse-grained search rate-distortion optimization module. DETAILED DESCRIPTION

[0032] See also Figure 3 The motion estimation method based on brightness distribution pattern proposed by the present invention includes the following steps.

[0033] Step S1: Preprocess the image blocks to be encoded in the current frame and all matching blocks in the reference frame (i.e., all image blocks within the reference frame search window). All the aforementioned image blocks are of original resolution and of the same size. The preprocessing includes: generating downsampled image blocks corresponding to all the aforementioned image blocks, and extracting the brightness distribution patterns of all the aforementioned image blocks. The brightness distribution pattern of the image block represents the brightness distribution (relative brightness difference) between pixels in different areas (various positions) within each image block of original resolution.

[0034] Step S2: Match the downsampled image block to be encoded with all downsampled matching blocks one by one, and calculate the coding distortion, coding bit rate, and brightness distribution pattern difference corresponding to each downsampled matching block. The brightness distribution pattern difference corresponding to each downsampled matching block refers to the difference between the "brightness distribution pattern of the image block to be encoded at the original resolution" and the "brightness distribution pattern of the matching block at the original resolution".

[0035] Step S3: Calculate the corresponding "rate-distortion cost" according to the coding distortion, coding bit rate, and brightness distribution pattern difference of each downsampled matching block. After traversing each downsampled matching block, select the downsampled matching block with the smallest rate-distortion cost as the optimal matching block of the downsampled image block to be encoded, and its corresponding motion vector as the optimal coarse pixel precision motion vector of the downsampled image block to be encoded.

[0036] See also Figure 4 In the step S1, generating a downsampled image block corresponding to an image block of original resolution includes the following sub-steps.

[0037] Step S11: Divide each image block of the original resolution into one or more image sub-blocks of the same shape and size. The width of each image sub-block is equal to the downsampling step in the horizontal direction, and the height of each image sub-block is equal to the downsampling step in the vertical direction. Therefore, each image sub-block corresponds to a downsampled pixel. Generally speaking, the downsampling step in the horizontal and vertical directions of the image block is the same and is a power of 2 (2 n (where n is a positive integer), and a common value is 2 pixels, 4 pixels, or 8 pixels. Therefore, the common size of an image sub-block is 2 pixels×2 pixels, 4 pixels×4 pixels, or 8 pixels×8 pixels.

[0038] Step S12: Calculate the average brightness value of all pixels in each image sub-block (i.e., the brightness mean value of the image sub-block) as the brightness value of a pixel after downsampling of the image sub-block, and so on to obtain the downsampled image block from the image block of the original resolution.

[0039] See also Figure 5 In the step S1, extracting the brightness distribution pattern of an image block of original resolution includes the following sub-steps.

[0040] Step S13: Divide each image sub-block within each image block of the original resolution into multiple sub-regions of the same number, each sub-region has the same shape and size, and each sub-region corresponds to a brightness distribution pattern bit. When the image sub-block is divided into sub-regions, the more sub-regions are divided, the more brightness distribution pattern bits of the image sub-blocks, and the higher the matching accuracy, but the greater the amount of calculation and storage space consumption; the fewer sub-regions are divided, the fewer brightness distribution pattern bits of the image sub-blocks, and the lower the matching accuracy, but the less the amount of calculation and storage space consumption. Preferably, each image sub-block is divided into 4 sub-regions of the same shape and size, so as to achieve a better balance between matching accuracy and calculation amount and storage space.

[0041] Step S14: Calculate the average brightness of all pixels in each sub-region.

[0042] Step S15: Based on the brightness average value and the brightness distribution determination threshold of each sub-region, the brightness distribution mode bit corresponding to each sub-region is calculated. An exemplary calculation method is: when the brightness average value of a sub-region ≥ the brightness distribution determination threshold, the brightness distribution mode bit corresponding to the sub-region is 1 (binary number); when the brightness average value of a sub-region < the brightness distribution determination threshold, the brightness distribution mode bit corresponding to the sub-region is 0 (binary number). The brightness distribution determination threshold can be set to the brightness average value of all pixels in the image sub-block where the sub-region is located, or it can be input from the outside.

[0043] Step S16: Arrange the brightness distribution pattern bits of all sub-regions within each image sub-block in a specific order to obtain the brightness distribution pattern of the image sub-block. The specific order is, for example, the row scanning order of each sub-region within the image sub-block, first from left to right, then from top to bottom. The brightness distribution pattern of the image sub-block represents the brightness distribution (relative brightness difference) between pixels in different regions (each position) within each image sub-block of the original resolution.

[0044] Step S17: Arrange the brightness distribution pattern of each image sub-block in the image block of the original resolution in a specific order to obtain the brightness distribution pattern of the image block of the original resolution. The specific order is, for example, the row scanning order of each image sub-block in the image block, first from left to right and then from top to bottom.

[0045] The steps S13 to S17 may be performed at any position after step S11 , for example, simultaneously with step S12 , or after step S12 .

[0046] See also Figure 6, which is a specific example of step S13 to step S16. First, an image sub-block B of the original resolution is divided into 4 sub-regions of the same shape and size, and these 4 sub-regions are respectively recorded as R0, R1, R2, and R3 according to the row scanning order of each sub-region within the image sub-block B. Then, the brightness average value of all pixels in each sub-region is calculated, and they are also recorded as L0, L1, L2, and L3 according to the row scanning order of each sub-region within the image sub-block B. Then, based on the brightness average value L0, L1, L2, and L3 of each sub-region R0, R1, R2, and R3 and the brightness distribution determination threshold τ, the brightness distribution mode bits B0, B1, B2, and B3 corresponding to each sub-region R0, R1, R2, and R3 are calculated. The exemplary calculation method is as follows: The brightness distribution pattern bits B0, B1, B2, and B3 arranged in sequence serve as the brightness distribution pattern of the image sub-block B.

[0047] See also Figure 7 , which is a specific example of step S17. Assume that an image block S of an original resolution is divided into 4 image sub-blocks A, B, C, and D of the same shape and size. The brightness distribution pattern of image sub-block A is A0, A1, A2, and A3. The brightness distribution pattern of image sub-block B is B0, B1, B2, and B3. The brightness distribution pattern of image sub-block C is C0, C1, C2, and C3. The brightness distribution pattern of image sub-block D is D0, D1, D2, and D3. The brightness distribution pattern of the image block S is A0, A1, A2, A3, B0, B1, B2, B3, C0, C1, C2, C3, D0, D1, D2, and D3.

[0048] In step S2, the coding distortion corresponding to each downsampled matching block can be obtained by a distortion cost calculation method. Common distortion cost calculation methods include SAD (Sum of Absolute Difference) algorithm, SATD (Sum of Absolute Transformed Difference) algorithm, SSD (Sum of Squared Differences) algorithm, etc., which will not be described here.

[0049] In step S2, the encoding bit rate corresponding to each downsampled matching block can be obtained by a bit rate cost calculation method. Common bit rate calculation methods include exponential-Golomb coding, etc., which will not be described here.

[0050] In step S2, the brightness distribution pattern difference corresponding to each downsampled matching block is obtained by comparing the difference between the brightness distribution pattern of the image block to be encoded at the original resolution and each "brightness distribution pattern of the matching block at the original resolution". The image block to be encoded at the original resolution and all matching blocks at the original resolution have the same size and contain the same number of image sub-blocks. The brightness distribution pattern of each image sub-block contains the same number of brightness distribution pattern bits. Therefore, the brightness distribution pattern of the image block to be encoded at the original resolution and the brightness distribution pattern of all matching blocks at the original resolution contain the same number of brightness distribution pattern bits, and these brightness distribution pattern bits are arranged in the same order. In this way, by counting the number of brightness distribution pattern bits with different values ​​between the "brightness distribution pattern of the image block to be encoded at the original resolution" and each "brightness distribution pattern of the matching block at the original resolution", the brightness distribution pattern difference can be obtained. For example, the brightness distribution pattern of the image block to be encoded at the original resolution is (1,1,1,1), the brightness distribution pattern of the matching block A at the original resolution is (1,1,1,1), and the brightness distribution pattern of the matching block B at the original resolution is (1,0,0,1). Therefore, the brightness distribution pattern difference between the image block to be encoded at the original resolution and the matching block A at the original resolution is 0, which is the brightness distribution pattern difference corresponding to the downsampled matching block A; the brightness distribution pattern difference between the image block to be encoded at the original resolution and the matching block B at the original resolution is 2, which is the brightness distribution pattern difference corresponding to the downsampled matching block B.

[0051] See also Figure 8 ,This is an example of improving the image block matching accuracy by introducing the brightness distribution pattern of the image block into the downsampling process of the coarse-grained search. Figure 8In the figure, the left side is the image block to be encoded with a size of 4 pixels × 4 pixels, and the right side is the matching block A and matching block B of the same shape and size. The hollow squares represent the original resolution pixels, the patterned squares represent the downsampled pixels, and the numbers in the squares represent the brightness values ​​of the pixels. Assuming that the downsampling step size in the horizontal and vertical directions of the coarse-grained search is 4 pixels, each image block is only divided into an image sub-block of 4 pixels × 4 pixels. It is necessary to find the optimal matching block of the image block to be encoded in matching block A and matching block B. Assuming that the brightness distribution judgment threshold is selected as the brightness average of all pixels in each image sub-block, the brightness distribution judgment thresholds of the image block to be encoded, matching block A, and matching block B are all set to 4. The brightness distribution pattern of the image block to be encoded is (1,1,1,1), the brightness distribution pattern of matching block A is (1,1,1,1), and the brightness distribution pattern of matching block B is (1,0,0,1). From this example, it can be seen that the pixel brightness values ​​of the downsampled image block to be encoded, the downsampled matching block A, and the downsampled matching block B are exactly the same, but the brightness distribution patterns of the three image blocks of original resolution are different. Among them, there is no difference in the brightness distribution pattern between the image block to be encoded at the original resolution and the matching block A at the original resolution, while the brightness distribution pattern between the image block to be encoded at the original resolution and the matching block B at the original resolution differs by 2 bits. Therefore, the present invention can accurately determine that the optimal matching block of the downsampled image block to be encoded is the downsampled matching block A, rather than the downsampled matching block B, after combining the brightness distribution pattern of the image block, which ensures the accuracy of the motion estimation result.

[0052] In step S3, assuming that D represents the coding distortion corresponding to a downsampled matching block, R represents the coding bit rate corresponding to the downsampled matching block, M represents the brightness distribution pattern difference corresponding to the downsampled matching block, and JJ represents the rate-distortion cost corresponding to the downsampled matching block, an exemplary calculation method is JJ=D+λ×R+ω×M. Wherein, λ represents the Lagrangian constant, ω represents the weight coefficient of the brightness distribution pattern difference, and ω>0.

[0053] See also Fig. 9 The motion estimation device based on brightness distribution pattern proposed by the present invention includes a coarse-grained search preprocessing module 1, a coarse-grained search matching module 2 and a coarse-grained search rate-distortion optimization module 3. Fig. 9 The device shown corresponds to Figure 3 The method shown.

[0054] The coarse-grained search preprocessing module 1 is used to preprocess the image blocks to be encoded in the current frame and all matching blocks in the reference frame, including: generating downsampled image blocks corresponding to all the aforementioned image blocks, and extracting the brightness distribution patterns of all the aforementioned image blocks. The brightness distribution pattern of the image block characterizes the distribution of brightness between pixels in different areas within each image block of original resolution. Furthermore, the coarse-grained search preprocessing module 1 includes a downsampling unit and a brightness distribution pattern extraction unit. The downsampling unit is used to generate downsampled image blocks corresponding to each of the aforementioned image blocks of original resolution. The brightness distribution pattern extraction unit is used to extract the brightness distribution pattern of each of the aforementioned image blocks of original resolution.

[0055] The coarse-grained search and matching module 2 is used to match the downsampled image block to be encoded with all the downsampled matching blocks one by one, and respectively calculate the coding distortion, coding bit rate, and brightness distribution pattern difference corresponding to each downsampled matching block. The brightness distribution pattern difference corresponding to each downsampled matching block refers to the difference between the brightness distribution pattern of the image block to be encoded at the original resolution and the brightness distribution pattern of the matching block at the original resolution.

[0056] The coarse-grained search rate-distortion optimization module 3 is used to calculate the corresponding rate-distortion cost of each downsampled matching block according to the coding distortion, coding bit rate, and brightness distribution pattern difference corresponding to each downsampled matching block, and select the downsampled matching block with the smallest rate-distortion cost as the optimal matching block of the downsampled image block to be encoded.

[0057] The present invention proposes a motion estimation method based on brightness distribution pattern, which includes extracting the brightness distribution pattern of an image block, introducing the difference calculation of the brightness distribution pattern of the image block into the matching process and rate-distortion optimization process of a coarse-grained search, overcoming the limitations of traditional coarse-grained search methods, improving the accuracy of image block matching in coarse-grained search, and effectively improving the accuracy of motion estimation.

[0058] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A motion estimation method based on brightness distribution pattern, characterized in that: The method comprises the following steps: Step S1: preprocessing the image block to be encoded in the current frame and all matching blocks in the reference frame, including: generating downsampled image blocks corresponding to the image block to be encoded and all matching blocks, and extracting brightness distribution patterns of the image block to be encoded and all matching blocks; the brightness distribution pattern of the image block represents the brightness distribution between pixels in different areas within each image block of the original resolution; Step S2: Match the downsampled image block to be encoded with all downsampled matching blocks one by one, and calculate the coding distortion, coding bit rate, and brightness distribution pattern difference corresponding to each downsampled matching block respectively; the brightness distribution pattern difference corresponding to each downsampled matching block refers to the difference between "the brightness distribution pattern of the image block to be encoded at the original resolution" and "the brightness distribution pattern of the matching block at the original resolution"; Step S3: Calculate the corresponding rate-distortion cost of each downsampled matching block according to the coding distortion, coding bit rate, and brightness distribution pattern difference corresponding to each downsampled matching block, and select the downsampled matching block with the smallest rate-distortion cost as the optimal matching block of the downsampled image block to be encoded.

2. The motion estimation method based on brightness distribution pattern according to claim 1, characterized in that: In the step S1, generating a downsampled image block corresponding to an image block with an original resolution includes the following sub-steps: Step S11: Divide each image block of the original resolution into one or more image sub-blocks of the same shape and size, wherein the width of each image sub-block is equal to the downsampling step length in the horizontal direction, and the height of each image sub-block is equal to the downsampling step length in the vertical direction, so that each image sub-block corresponds to a downsampled pixel point; Step S12: Calculate the average brightness of all pixels in each image sub-block as the brightness value of a pixel after downsampling of the image sub-block, and so on to obtain the downsampled image block from the image block with the original resolution.

3. The motion estimation method based on brightness distribution pattern according to claim 2, characterized in that: In the step S1, extracting the brightness distribution pattern of an image block of original resolution includes the following sub-steps: Step S13: Divide each image sub-block within each image block of the original resolution into a plurality of sub-regions of the same number, each sub-region having the same shape and size, and each sub-region corresponding to a brightness distribution mode bit; Step S14: Calculate the average brightness of all pixels in each sub-area; Step S15: Calculate the brightness distribution mode bit corresponding to each sub-region based on the brightness average value of each sub-region and the brightness distribution determination threshold; Step S16: arranging the brightness distribution pattern bits of all sub-regions within each image sub-block in order to obtain the brightness distribution pattern of the image sub-block; the brightness distribution pattern of the image sub-block represents the brightness distribution between pixels in different regions within each image sub-block of the original resolution; Step S17: arranging the brightness distribution pattern of each image sub-block in the image block of the original resolution in order to obtain the brightness distribution pattern of the image block of the original resolution; Steps S13 to S17 are performed at any position after step S11.

4. The motion estimation method based on brightness distribution pattern according to claim 3, characterized in that: In step S15, when the brightness average value of a sub-region is ≥ the brightness distribution determination threshold, the brightness distribution mode bit corresponding to the sub-region is 1; when the brightness average value of a sub-region is < the brightness distribution determination threshold, the brightness distribution mode bit corresponding to the sub-region is 0.

5. The motion estimation method based on brightness distribution pattern according to claim 3, characterized in that: In step S16, the arrangement in sequence is based on the row scanning order of each sub-region within the image sub-block, first from left to right, then from top to bottom; in step S17, the arrangement in sequence is based on the row scanning order of each image sub-block within the image block, first from left to right, then from top to bottom.

6. The motion estimation method based on brightness distribution pattern according to claim 1, characterized in that: In the step S2, the coding distortion corresponding to each downsampled matching block is obtained by a distortion cost calculation method.

7. The motion estimation method based on brightness distribution pattern according to claim 1, characterized in that: In the step S2, the encoding bit rate corresponding to each downsampled matching block is obtained by a bit rate cost calculation method.

8. The motion estimation method based on brightness distribution pattern according to claim 3, characterized in that: In step S2, the brightness distribution pattern difference corresponding to each downsampled matching block is the number of brightness distribution pattern bits with different values ​​between the brightness distribution pattern of the image block to be encoded at the original resolution and the brightness distribution pattern of each matching block at the original resolution.

9. The motion estimation method based on brightness distribution pattern according to claim 1, characterized in that: In step S3, D represents the coding distortion corresponding to a downsampled matching block, R represents the coding bit rate corresponding to the downsampled matching block, M represents the brightness distribution pattern difference corresponding to the downsampled matching block, JJ represents the rate-distortion cost corresponding to the downsampled matching block, JJ=D+λ×R+ω×M; wherein λ represents the Lagrangian constant, ω represents the weight coefficient of the brightness distribution pattern difference, and ω>0.

10. A motion estimation device based on brightness distribution pattern, characterized in that: It includes a coarse-grained search preprocessing module, a coarse-grained search matching module and a coarse-grained search rate-distortion optimization module; The coarse-grained search preprocessing module is used to preprocess the image block to be encoded in the current frame and all matching blocks in the reference frame, including: generating downsampled image blocks corresponding to the image block to be encoded and all matching blocks, and extracting brightness distribution patterns of the image block to be encoded and all matching blocks; The brightness distribution mode of the image block represents the brightness distribution between pixels in different areas within each image block of the original resolution; The coarse-grained search and matching module is used to match the downsampled image block to be encoded with all the downsampled matching blocks one by one, and respectively calculate the coding distortion, coding bit rate, and brightness distribution pattern difference corresponding to each downsampled matching block; the brightness distribution pattern difference corresponding to each downsampled matching block refers to the difference between "the brightness distribution pattern of the image block to be encoded at the original resolution" and "the brightness distribution pattern of the matching block at the original resolution"; The coarse-grained search rate-distortion optimization module is used to calculate the corresponding rate-distortion cost of each downsampled matching block according to the coding distortion, coding bit rate, and brightness distribution pattern difference corresponding to each downsampled matching block, and select the downsampled matching block with the smallest rate-distortion cost as the optimal matching block of the downsampled image block to be encoded.

Images