Image encoding method, electronic device, and storage medium

Abstract

The present application relates to the field of video encoding technology, and specifically relates to an image encoding method, an electronic device, and a storage medium. The method includes: obtaining an image to be encoded and dividing the image to be encoded into a plurality of image blocks; searching for a target block matching each image block from a reference frame; determining a motion vector between the target block and the image block; encoding the image to be encoded based on the motion vector; and through at least N searches, in the Mth search, a first target matching point is searched from S first points to be searched, and the first starting search point of the Mth search is different from the first target matching point; based on the coordinate distance between the first target matching point and the first starting search point, a second starting search point for the (M + 1)th search is determined, where the second starting search point is different from the first target matching point and the first starting search point, and M < N. Through the above method, a more accurate motion displacement between the current image including a fast-moving object and the reference image can be obtained.

Description

Technical Field

[0001] The present invention relates to the technical field of video encoding, and in particular to an image encoding method, electronic equipment and a storage medium. Background Art

[0002] Video is an image sequence composed of multiple consecutive frames. There is similarity between images that are adjacent in time in the video, that is, there is redundant data between adjacent images. Therefore, in the process of encoding the video, multiple reference frames are selected in the video, and the images are encoded according to the residual values ​​between the reference frame and the images adjacent to the reference frame. The reference frame here can contain the data of each pixel in the image, that is, the complete image data; the residual value is used to represent the difference between the reference frame and the adjacent image.

[0003] If the object in the video is in motion, the residual value between two consecutive frames (the reference frame and the image adjacent to the reference frame) will be large, which will not effectively reduce the amount of data that the electronic device needs to process when encoding the video. Therefore, the motion displacement of the object in the two consecutive frames can be motion estimated, that is, the motion displacement of the object can be obtained, and the image adjacent to the reference frame can be motion compensated by the estimated motion displacement, so that the residual value between the compensated image adjacent to the reference frame and the reference frame becomes smaller, reducing the amount of data that the electronic device needs to process. The process of obtaining this motion displacement is called Motion Estimation (ME).

[0004] The process of motion estimation includes: dividing the current image (image to be searched) into several image blocks of equal size; then calculating the sum of absolute differences (SAD) between each image block and the corresponding candidate image block in the reference frame, selecting the image block and the candidate image block corresponding to the minimum SAD value, and the motion vector between the image block and the candidate image block is the motion displacement between the image block and the reference frame. However, when using existing image coding methods to encode videos of high-speed moving objects, the motion displacement obtained is often inaccurate. Summary of the invention

[0005] The embodiments of the present application provide an image encoding method, an electronic device, and a storage medium.

[0006] The first aspect of the present application provides an image encoding method, comprising:

[0007] Obtaining an image to be encoded, and dividing the image to be encoded into a plurality of image blocks;

[0008] Searching for a target block matching each image block from the reference frame;

[0009] Determine a motion vector between the target block and the image block;

[0010] Encoding the image to be encoded based on the motion vector;

[0011] Among them, the target block matching each image block is searched from the reference frame, including:

[0012] A reference block matching the image block is determined in the reference frame by at least N searches, where:

[0013] Determine a reference block matching the image block in the reference frame by searching at least N times, including:

[0014] In the M-th search, a first target matching point is searched from the S first points to be searched, wherein the starting point of the M-th search is a first starting search point among the S first points to be searched that is different from the first target matching point;

[0015] Based on the coordinate distance between the first target matching point and the first starting search point, a second starting search point for the M+1th search is determined, wherein the second starting search point is different from the first target matching point and the first starting search point, and M, N, S are natural numbers, and M <N。

[0016] That is, in an embodiment of the present application, the image to be encoded may be the current image, the reference frame may be the reference image, the image to be encoded and the reference frame may be two consecutive frames of images in a video of a high-speed car, for example, the reference frame may be the next frame of the image to be encoded. The image block may be a set of pixels including a certain number of pixels obtained by dividing the image to be encoded. The target block here may be an area formed by a final matching point in the reference frame, and the sum of absolute differences (SAD, Sum of AbsoluteDifference) between the pixels included in the area and the pixels included in the image block is the smallest. The motion vector here may be a vector between the final matching point and the pixel in the image block in the image to be encoded (for example, the pixel in the upper left corner of the image block), and the motion displacement between the image block of the image to be encoded and the reference frame.

[0017] In a possible implementation of the first aspect above, the S first points to be searched form a first rectangular area, wherein the first starting search point is a first corner point of the first rectangular area.

[0018] That is, in the embodiment of the present application, N here can be a natural number 4, and S can be 16. The size of the first rectangular area can be 4×7. That is, it is an area with 4 rows and 7 columns. In the first rectangular area, 16 first points to be searched are distributed in sequence, for example: in the horizontal direction, the first points to be searched can be distributed at intervals, and in the vertical direction, the first points to be searched can be distributed without intervals. The first starting search point can be the upper left corner point of the first rectangular area. And the first target matching point can be the upper right corner point of the first rectangular area.

[0019] In a possible implementation of the first aspect above, the coordinate distance includes the distance from the first target matching point along the first side of the first rectangular area to the first starting search point or the distance from the first target matching point along the second side of the first rectangular area to the first starting search point.

[0020] That is, in the embodiment of the present application, the first side length of the first rectangular area may be a side with a length of 7, and the second side length of the first rectangular area may be a side with a length of 4. The coordinate distance between the first target matching point and the first starting search point may be the distance from the first starting search point to the first target matching point along the first side length, that is, 6. The coordinate distance between the first target matching point and the first starting search point may also be the distance from the first starting search point to the first target matching point along the second side length, that is, 0.

[0021] In a possible implementation of the first aspect, determining a second starting search point for the M+1th search based on a coordinate distance between the first target matching point and the first starting search point includes:

[0022] Corresponding to the coordinate distance being less than or equal to the preset threshold, the second starting search point is located at the first position;

[0023] Corresponding to the coordinate distance being greater than the preset threshold, the second starting search point is located at the second position.

[0024] In a possible implementation of the first aspect above, a direction from the first starting search point to the first position is opposite to a direction from the first starting search point to the first target matching point.

[0025] In a possible implementation of the first aspect above, a direction from the first starting search point to the second position is the same as a direction from the first starting search point to the first target matching point.

[0026] That is, in the embodiment of the present application, the preset threshold here can be 2, the first position can be along the direction of the first side length of the first rectangular area, located on the left side of the first starting search point; the first position can be along the direction of the first side length of the first rectangular area, located on the right side of the first starting search point. For example: if the coordinate distance between the first target matching point and the first starting search point is 6, then the second starting search point is located on the right side of the first starting search point. That is, the direction from the first starting search point to the second starting search point is the same as the direction from the first starting search point to the first target matching point. For another example: if the coordinate distance between a target matching point and the first starting search point is 2, then the second starting search point is located on the left side of the first starting search point. That is, the direction from the first starting search point to the second starting search point is opposite to the direction from the first starting search point to the first target matching point.

[0027] It can be seen that in the motion estimation method, the length and width of the search area corresponding to each stage can be adjusted. For example, if the objects in the reference image and the current image move in the horizontal direction when the reference image is displayed to the current image, the length of the area formed by the points to be searched in the horizontal direction is greater than the width in the vertical direction; if the object moves in the vertical direction, the length of the area formed by the points to be searched in the horizontal direction is less than the width in the vertical direction; at the same time, when determining the starting point of the area formed by the points to be searched corresponding to the next stage, the starting point of the area formed by the points to be searched corresponding to the next stage is not located in the area formed by the points to be searched corresponding to the previous stage, avoiding the position of the points to be searched corresponding to the minimum value of the SAD value obtained by the search in the next stage being close to the search area corresponding to the previous stage. Through the motion estimation method of the embodiment of the present application, for the current image and the reference image containing objects moving at high speed, a more accurate motion displacement between the current image and the reference image can be obtained.

[0028] In a possible implementation of the first aspect, determining a second starting search point for the M+1th search based on a coordinate distance between the first target matching point and the first starting search point includes:

[0029] The second starting search point is determined by the following formula,

[0030] new_start_point_x=if(delta_x<=2)bestmv_x–5; else bestmv_x–1,

[0031] new_start_point_y=if(delta_y<=1)bestmv_y–2; else bestmv_y–1,

[0032] Among them, new_start_point_x represents the offset of the second starting search point relative to the first starting search point in the direction of the first side length, new_start_point_y represents the offset of the second starting search point relative to the first starting search point in the direction of the second side length, bestmv_x and bestmv_y represent the position of the first target matching point in the first rectangular area, delta_x represents the coordinate distance between the first target matching point and the first starting search point in the direction of the first side length, and delta_y represents the coordinate distance between the first target matching point and the first starting search point in the direction of the second side length.

[0033] That is, in the embodiment of the present application, the constants 2, 5, and 1 are all configurable. The constants 2 and 5 can change with the size of the first rectangular area. For example, if the size of the first rectangular area is 5×9, the constants 2 and 5 can be changed to 3 and 6.

[0034] In a possible implementation of the first aspect, searching for a first target matching point from S first points to be searched includes:

[0035] Traversing S first points to be searched, and forming S first comparison areas with each first point to be searched as a corner point;

[0036] Compare the pixels included in each first comparison area with the pixels included in the image block;

[0037] Corresponding to the first comparison area that meets the preset matching condition, a first to-be-searched point corresponding to the first comparison area is determined as a first target matching point.

[0038] In a possible implementation of the first aspect above, the preset matching condition includes that a sum of absolute differences (SAD) between pixels included in the first comparison area and pixels included in the image block is less than a preset threshold.

[0039] That is, in an embodiment of the present application, the size of the first comparison area here can be the same as the size of the image block, and the sum of absolute differences (SAD) between the pixels included in the first comparison area and the pixels included in the image block is calculated, and the first to-be-searched point corresponding to the first comparison area with the smallest sum of absolute differences is determined as the first target matching point.

[0040] In a possible implementation of the first aspect, the method further includes: corresponding to the second starting search point meeting a preset search condition, taking a vector difference between the image block and the first target matching point as a motion vector.

[0041] In a possible implementation of the first aspect, meeting the preset search condition includes: the second starting search point is the same as the first starting search point.

[0042] That is, in the embodiment of the present application, the motion vector here can be a vector between the first target matching point and a pixel point in the image block in the image to be encoded (for example, a pixel point at the upper left corner of the image block), or a motion displacement between the image block of the image to be encoded and the reference frame. The preset search condition here can also include: N reaches an upper limit of the number of searches, for example, the upper limit of the number of searches can be 4. That is, after 4 searches, the search ends.

[0043] A second aspect of the present application provides an image coding system, the system comprising:

[0044] An acquisition module, the acquisition module is used to acquire the image to be encoded and divide the image to be encoded into multiple image blocks;

[0045] Search module, the search module is used to search for a target block matching each image block from a reference frame;

[0046] A calculation module, the calculation module is used to determine the motion vector between the target block and the image block;

[0047] A coding module, the coding module is used to encode the image to be coded based on the motion vector;

[0048] Among them, the target block matching each image block is searched from the reference frame, including:

[0049] A reference block matching the image block is determined in the reference frame by at least N searches, where:

[0050] Determine a reference block matching the image block in the reference frame by searching at least N times, including:

[0051] In the M-th search, a first target matching point is searched from the S first points to be searched, wherein the starting point of the M-th search is a first starting search point among the S first points to be searched that is different from the first target matching point;

[0052] Based on the coordinate distance between the first target matching point and the first starting search point, a second starting search point for the M+1th search is determined, wherein the second starting search point is different from the first target matching point and the first starting search point, and M, N, S are natural numbers, and M <N。

[0053] That is, in the embodiments of the present application, the acquisition module, the search module, the calculation module and the encoding module here may be functional modules in the video encoding system.

[0054] The third aspect of the present application provides an electronic device, comprising: a memory for storing instructions executed by one or more processors of the electronic device, and a processor, which is one of the processors of the electronic device, for executing the image encoding method provided by the aforementioned first aspect.

[0055] The fourth aspect of the present application provides a readable medium, characterized in that the readable medium stores instructions, and when the instructions are executed on an electronic device, the electronic device executes the image encoding method provided by the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 The figure shows a schematic diagram of two consecutive frames of images in a video of a car traveling at high speed provided by an embodiment of the present application;

[0057] Figures 2a to 2i FIG. 1 is a schematic diagram of a pixel-based motion estimation provided by an embodiment of the present application;

[0058] Figure 3 Shown is a schematic diagram of a motion displacement provided by an embodiment of the present application;

[0059] Figures 4a to 4i FIG. 1 is a schematic diagram of a pixel-based motion estimation provided by an embodiment of the present application;

[0060] Figure 5 The figure shows a schematic diagram of an implementation flow of motion estimation provided by an embodiment of the present application;

[0061] Figure 6 FIG. 1 is a schematic diagram of determining a search point in a reference image provided by an embodiment of the present application;

[0062] Figure 7 The figure is a schematic block diagram of the structure of a computer applicable to an image encoding method provided in an embodiment of the present application;

[0063] Figure 8 The figure is a schematic block diagram of the structure of a video encoding system provided by an embodiment of the present application;

[0064] Fig. 9 The figure is a schematic block diagram of the structure of a SoC provided in an embodiment of the present application. DETAILED DESCRIPTION

[0065] In order to clearly understand the technical solution provided in the embodiment of the present application, the relevant technical terms involved in the video encoding solution provided in the embodiment of the present application are explained below.

[0066] Image block: A pixel set including a certain number of pixels obtained by dividing a frame of image. Each image block obtained by division may include N×M pixels, where N and M are natural numbers, and N and M may be the same or different. For example, an image block may include 32×32 pixels, or an image block may include 16×32 pixels.

[0067] Reference below Figure 1 , Figure 1 Two consecutive frames of images in a video of a high-speed car are shown, namely, a current image 100 and a reference image 200 (a frame image after the current image 100). It can be seen that the cars in the current image 100 and the reference image 200 are in a moving state from the display of the reference image 200 to the display of the current image 100. In the embodiment of the present application, the motion displacement of the car in the current image 100 and the reference image 200 can be estimated to obtain the motion displacement of the car as the motion compensation between the current image 100 and the reference image 200.

[0068] Then, the current image is adjusted based on motion compensation to obtain a compensated image. At this time, the residual value between the compensated image and the reference image is calculated, and the motion compensation information and the calculated residual value information are used to encode the current image. When decoding the current image, motion compensation prediction can be performed on the current image based on the decoded motion compensation information and residual value to obtain the complete image data of the current image for display.

[0069] The current image 100 and the reference image 200 here may be two consecutive frames of images. Taking the current image 100 and the reference image 200 in a video as an example, the time of acquiring the current image 100 may be later than the time of acquiring the reference image 200.

[0070] In an embodiment of the present application, an image encoding method includes: determining an image block in a current image, and determining an area to be searched in a reference image, the size of the area to be searched may be greater than the size of the image block, wherein the area to be searched may be a local area in the reference image, the position of the area to be searched may be any position in the reference image, or the area to be searched may be the entire reference image. A set of N first points to be searched (pixel points) is first determined in the area to be searched, the first points to be searched form a first area, the first area has an initial position point, and the initial position point here may be a central pixel point in the first points to be searched. A first point to be searched with a matching point with a first target, that is, a first point to be searched with a minimum SAD value is determined in the set of N first points to be searched. The minimum SAD value here indicates that the SAD value between the pixel points included in the area where the first point to be searched is located and the pixel points included in the image block of the current image is the minimum, that is, the difference between the area where the first point to be searched is located and the image block of the current image is the minimum. The area where the first point to be searched is located here may be an area formed with the first point to be searched as a corner point, and the size of the area is the same as the size of the image block of the current image.

[0071] According to the first target matching point, determine the starting point of the area formed by the next point to be searched, that is, the starting point of the area formed by the second point to be searched (that is, the second area). The size of the second area here can be smaller than the first area. For example, the first target matching point is used as the initial position point of the second area, that is, the central pixel point of the second area, that is, the first target matching point is used as the central pixel point in the set of N second points to be searched (pixel points), and the second target matching point, that is, the second point to be searched with the smallest SAD value, is determined in the set of N second points to be searched. End the search until the search condition is met. Determine the next image block in the current image, and obtain the motion displacement between the next image block and the reference image.

[0072] The search conditions here include: the initial position point of the second area is the same pixel as the initial position point of the first area, or, when the number of search stages executed meets the search threshold (for example, the search threshold can be 3 times), the search will also be terminated.

[0073] It can be understood that the SAD value here represents the sum of the absolute values ​​of the differences in grayscale values ​​between the pixels included in the area formed by the points to be searched in the reference image and the pixels included in the image block in the current image.

[0074] Repeat the above steps to determine the first point to be searched as the center point, determine a set of second points to be searched in the reference image, where the size of the second area formed by the second points to be searched is smaller than the first area, and determine a second point to be searched in the second area in the reference image, where the SAD value between the area formed by the second points to be searched and the image block in the current image is the smallest.

[0075] It can be understood that when the preset search conditions are met, the point to be searched corresponding to the area with the smallest SAD value obtained in the reference image can be used as the final matching point. The preset search conditions here may include that the number of searches reaches the search threshold, or that the point to be searched corresponding to the minimum SAD value is the center point of the area formed by the previous point to be searched. In this case, the vector between the central pixel point in the first point to be searched and the final matching point is determined as the motion displacement of the image block in the current image relative to the reference image.

[0076] Reference below Figures 2a to 2i , Figures 2a to 2i A detailed method for pixel-based motion estimation provided by an embodiment of the present application is exemplified. Figures 2a to 2i The motion estimation method shown can be divided into three stages for execution, which can also be called a three-step method, that is, the search threshold in the preset search condition can be 3.

[0077] like Figure 2a As shown, the size of the current image 100 may be 20×16, that is, the current image 100 may be composed of 20×16 pixels. The current image 100 includes an image block 101, and the size of the image block 101 may be 4×4, that is, the image block 101 may be composed of 4×4 pixels.

[0078] like Figure 2a As shown, the size of the reference image (reference frame) 200 can be Figure 2a The size of the current image 100 in the reference image 200 is the same as that of the current image 100 in the reference image 200, that is, the reference image 200 may also be composed of 20×16 pixels. It can be understood that the entire area of ​​the reference image 200 can be used as the search area; in another embodiment, Figure 2a The reference image 200 shown may also represent a local area in the entire reference image.

[0079] exist Figure 2a A first region 201 is determined in the reference image 200 shown in FIG. 1 . For example, the size of the first region 201 may be 9×9. The first region 201 may be formed by S points to be searched. Figure 2b The first area 201 shown may include 9 points to be searched. Figure 2aThe 9 points to be searched can be represented by ①. The center point of the first area 201 can be the initial position point of the motion estimation method.

[0080] The first stage search is performed on the reference image 200 based on the current image 100, such as Figure 2b As shown, a comparison area 202 is formed with the position of the to-be-searched point 203 in the first area 201 as a corner point. Figure 2b As shown, the point at the upper left corner of the comparison area 202 coincides with the point to be searched 203, and the size of the comparison area 202 is Figure 2a The size of the image blocks 101 in are the same.

[0081] calculate Figure 2b The Sum of Absolute Difference (SAD) of the search point 203 shown is calculated, that is, the SAD value between the pixel points included in the comparison area 202 located in the reference image 200 and the pixel points included in the image block 101 located in the current image 100 is calculated.

[0082] like Figure 2c As shown, the process of calculating the SAD values ​​of the pixels included in the image block 101 and the comparison area 202 can be: respectively calculate the absolute value of the difference in grayscale value between the pixel points "1" to the pixel points "16" in the image block 101 and the comparison area 202, obtain 16 numerical values, and add the 16 numerical values ​​to obtain the SAD values ​​of the pixels included in the image block 101 and the comparison area 202.

[0083] After calculating Figure 2b The SAD values ​​of the pixels included in the image block 101 and the comparison area 202 shown in FIG. 2 , that is, the SAD values ​​of the pixels included in the image block 101 and the comparison area 202 located at the point to be searched 203, as shown in FIG. Figure 2d As shown, a comparison area 202 is formed with the position of the to-be-searched point 204 in the first area 201 as a corner point. Figure 2d As shown, the point at the upper left corner of the comparison area 202 coincides with the point to be searched 204. Figure 2d The SAD value of the point to be searched 204 shown in FIG. 1 is the sum of the absolute differences between the pixels included in the comparison area 202 in the reference image 200 and the pixels included in the image block 101 in the current image 100. After the SAD values ​​corresponding to the 9 points to be searched in the first area 201 are calculated, a minimum value is determined from the SAD values ​​corresponding to the 9 points to be searched, and the point to be searched corresponding to the minimum SAD value is used as the center point of the second point to be searched in the second stage of search.

[0084] It can be understood that if the search point corresponding to the minimum value of the determined SAD value is the center point of the first area 201, the search is terminated, the next image block in the current image 100 is selected, and the motion displacement of the image block is determined in the reference image 200.

[0085] like Figure 2e As shown, the to-be-searched point corresponding to the minimum value of the SAD value may be the to-be-searched point 205. Figure 2e In the reference image 200 shown in FIG. 2 , nine second points to be searched are determined with the point to be searched 205 as the center. The nine second points to be searched form a second area 206. Figure 2e The 9 second search points can be represented by ②, and it can be seen that the size of the second area 206 is smaller than Figure 2a The size of the first area 201 is shown, for example, the size of the second area 206 may be 5×5.

[0086] The second stage search is performed on the reference image 200 based on the current image 100, such as Figure 2f As shown, a comparison area 207 is formed with the position of the to-be-searched point 208 in the second area 206 as a corner point. Figure 2f As shown, the point at the upper left corner of the comparison area 207 coincides with the point to be searched 208, and the size of the comparison area 207 is Figure 2a The size of the image block 101 in

[0087] After calculating the SAD values ​​corresponding to the 9 second search points in the second area 206 in sequence, a minimum value is determined from the SAD values ​​corresponding to the 9 second search points, and the search point corresponding to the minimum SAD value is used as the center point of the search point in the third stage search.

[0088] like Figure 2g As shown, the to-be-searched point corresponding to the minimum value of the SAD value may be the to-be-searched point 209. Figure 2g In the reference image 200 shown in FIG. 2 , nine third points to be searched are determined with the point to be searched 209 as the center. The nine third points to be searched form a third area 210. Figure 2g The 9 third search points can be represented by ③, and it can be seen that the size of the third area 210 is smaller than Figure 2a The size of the second region 206 is shown, for example, the size of the third region 210 may be 3×3.

[0089] The third stage search is performed on the reference image 200 based on the current image 100, such as Figure 2h As shown, a comparison area 211 is formed with the position of the to-be-searched point 212 in the third area 210 as a corner point. Figure 2h As shown, the point at the upper left corner of the comparison area 211 coincides with the point to be searched 212 .

[0090] After the SAD values ​​corresponding to the 9 to-be-searched points in the third area 210 are calculated in sequence, a minimum value is determined from the SAD values ​​corresponding to the 9 to-be-searched points, and the to-be-searched point corresponding to the minimum SAD value is used as the final matching point. Figure 2i As shown, the final matching point here may be the point to be searched 213 , and the vector between the point to be searched 213 and the center point 214 of the first area 201 is determined as the motion displacement between the image block 101 in the current image 100 and the reference image 200 .

[0091] It can be understood that after the motion displacement between the image block 101 in the current image 100 and the reference image 200 is determined, the motion displacement between other image blocks in the current image 100 and the reference image 200 will continue to be determined.

[0092] refer to Figure 3 , Figure 3 Shows the Figure 1 The current image 100, the reference image 200 and the Figures 2a to 2i The schematic diagram of the combined motion displacement obtained shows that Figures 2a to 2i The obtained motion displacement 303 of the car has a certain error (the length and direction between the motion displacement 303 and the motion displacement 302) compared to the motion displacement 302 between the position of the car in the reference image 200 and the position in the current image 100. This is because Figures 2a to 2i In the motion estimation method shown, the area formed by the points to be searched determined in each stage is a square, and after completing a stage of search, the points to be searched in the next stage will be determined with the pixel points obtained in the above stage as the center, and in order to improve the search efficiency, the size of the area formed by the points to be searched in the next stage will be smaller than the area formed by the points to be searched in the previous stage, so that the points to be searched in the next stage will always be around the area formed by the points to be searched corresponding to the previous stage, resulting in the position of the final matching point to be obtained falling near the area formed by the points to be searched corresponding to the first stage, that is, the final matching point obtained by motion estimation of the current image 100 and the reference image 200 is often located near the initial position point where the motion estimation method is started in the reference image 200. Therefore, for the current image and the reference image containing a high-speed moving object, after the position of the object in the current image is converted to the corresponding position in the reference image, the object is often far away from the initial position point of the motion estimation method executed on the reference image, so the use of Figures 2a to 2i The motion estimation method shown cannot obtain a relatively accurate motion displacement of the object in the reference image and the current image.

[0093] In order to solve the above problems, the embodiment of the present application proposes a pixel-based motion estimation method. In the motion estimation method, the length and width of the search area corresponding to each stage can be adjusted. For example, if the objects in the reference image and the current image move in the horizontal direction when the reference image is displayed to the current image, the length of the area formed by the points to be searched in the horizontal direction is greater than the width in the vertical direction; if the object moves in the vertical direction, the length of the area formed by the points to be searched in the horizontal direction is less than the width in the vertical direction; at the same time, when determining the starting point of the area formed by the points to be searched corresponding to the next stage, the starting point of the area formed by the points to be searched corresponding to the next stage is not located in the area formed by the points to be searched corresponding to the previous stage, so as to avoid the position of the points to be searched corresponding to the minimum value of the SAD value obtained by the search in the next stage being close to the search area corresponding to the previous stage. Through the motion estimation method of the embodiment of the present application, for the current image and the reference image containing objects moving at high speed, a more accurate motion displacement between the current image and the reference image can be obtained.

[0094] Specifically, compared to Figures 2a to 2i The motion estimation method shown in the figure, the embodiment of the present application proposes a pixel-based motion estimation method, including:

[0095] An image block is determined in the current image, and a first area formed by a first point to be searched is determined in the reference image, wherein the size of the first area can be determined according to the running direction of the current image relative to the reference image (for example, the moving direction of the object in the current image), and the position of the first area can be any position in the reference image. The first area has an initial position point, where the initial position point can be any one of all the pixel points included in the first area, for example, the central pixel point included in the first area, and the first target matching point, that is, the first point to be searched with the smallest corresponding SAD value is determined in the set of N first points to be searched that form the first area.

[0096] According to the first target matching point and the starting point of the first area, the starting point of the area formed by the next point to be searched is determined by a position formula, that is, the starting point of the second area formed by the second point to be searched.

[0097] The position formula here is used to determine the starting point of the next set of points to be searched based on the position offset of the first matching point relative to the starting point of the first area, that is, the starting point of the second area formed by N second points to be searched, so that the starting point of the second area can be located in the offset direction between the first matching point and the starting point of the first area.

[0098] It can be understood that the position formula here is only a way to determine the starting point of the area formed by the next point to be searched. In another embodiment of the present application, the position of the starting point can also be determined by a position algorithm or a position calculation model.

[0099] In the case where the starting point of the second area is the same pixel as the starting point of the first area, the search is terminated and the first target matching point is used as the final matching point. The vector between the final matching point and the initial position point of the first area is determined as the motion displacement between the image block and the reference image. Alternatively, the search is terminated when the number of search stages executed meets the search threshold (for example, the search threshold may be 4 times).

[0100] In the case where the starting point of the second region is different from the starting point of the first region, a second target matching point, that is, a second search point with the smallest SAD value is determined from the set of N second search points forming the second region. The search is terminated when the above two conditions are met. The next image block is determined in the current image, and the motion displacement between the next image block and the reference image is obtained.

[0101] It can be understood that determining the first target matching point in the set of N first points to be searched includes: forming a comparison area with the position of the first point to be searched in the reference image; then, sequentially calculating the SAD values ​​of the pixels included in the comparison area corresponding to each first point to be searched and the pixels included in the image block, and determining the first point to be searched corresponding to the comparison area with the smallest SAD value. It can be understood that the smaller the SAD value of the first point to be searched, the smaller the difference between the comparison area where the first point to be searched is located and the image block of the current image.

[0102] In an embodiment of the present application, the size of the area formed by the points to be searched is rectangular, and the longer side of the rectangle can correspond to the running direction of the object in the current image and the reference image. After the point to be searched (i.e., the matching point) corresponding to the minimum value of the SAD value is determined through each stage search, the corresponding area to be searched in the next stage is determined by the position formula instead of the point to be searched. The position formula here is used to determine the position of the area corresponding to the point to be searched in the next stage in the reference image according to the position of the point to be searched. Until the final matching point is obtained in the reference image. The vector between the starting point of the first area and the final matching point is determined as the motion displacement of the image block in the current image relative to the reference image.

[0103] Reference below Figures 4a to 4i as well as Figure 5 , Figures 4a to 4i A schematic diagram of a process of a pixel-based motion estimation method provided in an embodiment of the present application is exemplified. Figure 5The schematic diagram of the method flow chart of the pixel-based motion estimation method is exemplified. It can be understood that Figure 5 The method shown can be implemented by a processor of an electronic device for encoding a video executing related instructions, referring to Figure 5 , the display method may include the following steps:

[0104] S501: Determine an image block to be processed in a current image and a first point to be searched in a reference image.

[0105] like Figure 4a As shown, the size of the current image 300 may be 20×16, that is, the current image 300 may be composed of 20×16 pixels. The current image 300 includes an image block to be processed, that is, an image block 301. The size of the image block 301 may be 4×4, that is, the image block 301 may be composed of 4×4 pixels. Figure 4a As shown, the size of the reference image (reference frame) 400 can be Figure 4a The size of the current image 300 in , that is, the reference image 400 may also be composed of 20×16 pixels. Figure 4a In the reference image 400 shown in FIG. 1 , a first area 401 formed by S first points to be searched is determined. For example, S here can be 16. Figure 4a The 16 points to be searched can be represented by ①. The pixel point where "0" in the first area 401 is located can be the initial position point of the motion estimation method. The size of the first area 401 can be 4×7, that is, an area with 4 rows and 7 columns, that is, 7 pixels in the horizontal direction and 4 pixels in the vertical direction. It can be understood that the size of the first area 401 can also be called 7×4, which can also be used to represent an area with 4 rows and 7 columns.

[0106] It can be seen that the first area 401 here is rectangular, and the length of the first area 401 in the horizontal direction is greater than the width in the vertical direction. In the embodiment of the present application, the length and width of the first area 401 can be adjusted. For example, if the object between the current image 300 and the reference image 400 moves in the horizontal direction, the length of the first area 401 in the horizontal direction is greater than the width in the vertical direction; if the object between the current image 300 and the reference image 400 moves in the vertical direction, the length of the first area 401 in the horizontal direction is less than the width in the vertical direction. The value of the size of the first area 401 can also be other values, which are not limited here.

[0107] S502: Determine a first target matching point in a first area in a reference image according to the image block to be processed in the current image.

[0108] A first stage search is performed on the reference image 400 based on the current image 300, such as Figure 4b As shown, a comparison area 402 is formed with the position of the to-be-searched point 403 in the first area 401 as a corner point, such as Figure 4b As shown, the point at the upper left corner of the comparison area 402 coincides with the point to be searched 403, and the size of the comparison area 402 is Figure 4a The size of the image block 301 in is the same.

[0109] In an embodiment of the present application, the position of the point to be searched 403 in the first area 401 (i.e., the starting point of the first area 401) can be a position that is 3 pixels (Pixel, PX) to the left of the pixel point where "0" in the first area 401 is located and 2 pixels (Pixel, PX) upward.

[0110] After calculating the SAD values ​​corresponding to the 16 points to be searched in the first area 401 and the image block 301 in the current image 300 in turn, a minimum value is determined from the SAD values ​​corresponding to the 16 points to be searched, and the search point (first target matching point) corresponding to the minimum SAD value is determined as the second point to be searched in the second stage search.

[0111] S503: Determine a starting point of the second area according to the first target matching point in the first area.

[0112] In an embodiment of the present application, a method for determining the starting point of the area formed by the second search point in the second stage search based on the search point (first target matching point) corresponding to the minimum SAD value obtained in the previous stage can be determined using the following position formula.

[0113] new_start_point_x=if(delta_x<=2)bestmv_x–5;else bestmv_x–1;

[0114] new_start_point_y=if(delta_y<=1)bestmv_y–2; else bestmv_y–1;

[0115] Among them, new_start_point_x represents the offset of the starting point of the area formed by the points to be searched relative to the starting point of the area formed by the points to be searched in the previous stage in the x-axis direction, new_start_point_y represents the offset of the starting point of the area formed by the points to be searched relative to the starting point of the area formed by the points to be searched in the previous stage in the y-axis direction, bestmv_x represents the x-axis coordinate of the point to be searched corresponding to the minimum value of the SAD value, bestmv_y represents the y-axis coordinate of the point to be searched corresponding to the minimum value of the SAD value, delta_x represents the offset of bestmv_x relative to the starting point of the area formed by the points to be searched at this stage in the x-axis direction, delta_y represents the offset of bestmv_y relative to the starting point of the area formed by the points to be searched at this stage in the y direction.

[0116] It can be understood that delta_x<=2 is used to determine the magnitude of the offset in the x-axis direction between the first target matching point and the starting point of the area formed by the first to-be-searched point. That is, if delta_x<=2, it means that the first target matching point is closer to the starting point of the area formed by the first to-be-searched point, that is, the first target matching point is located on the left side of the area formed by the first to-be-searched point, indicating that the position of the matching block in the reference image that matches the image block in the current image should be located on the left side of the area formed by the first to-be-searched point, such as Figure 6 As shown in FIG. 6 , the first target matching point 601 is determined in the reference image 600. It can be seen that the first target matching point 601 is close to the left side of the area formed by the first to-be-searched point, that is, it is located on the left side of the center line of the y-axis direction of the area formed by the first to-be-searched point. Therefore, by using bestmv_x–5, the position of the area formed by the next stage of the to-be-searched point (such as: the second generation search point) can also be located on the left side of the area formed by the first to-be-searched point. Therefore, as Figure 6 As shown, the area 602 formed by the second search point determined by the position formula can be located on the left side of the area formed by the first to-be-searched point.

[0117] It can be understood that if delta_x>2, it means that the first target matching point is far away from the starting point of the area formed by the first to-be-searched point, that is, the first target matching point is located on the right side of the area formed by the first to-be-searched point, indicating that the position of the matching block in the reference image that matches the image block in the current image should be located on the right side of the area formed by the first to-be-searched point. Therefore, through bestmv_x-1, the position of the area formed by the next stage of the to-be-searched point (such as: the second generation search point) can also be located on the right side of the area formed by the first to-be-searched point. It can be understood that delta_y<=1 here can be used to determine the magnitude of the offset in the y-axis direction between the first target matching point and the starting point of the area formed by the first to-be-searched point.

[0118] The above constants 2, 5 and 1 can change according to the change of the size of the area formed by the first point to be searched determined in step S501. That is, as the size of the area formed by the first point to be searched is different, the constants in the position formula may also be different, which is not limited here.

[0119] In the embodiment of the present application, the values ​​in the position formula here are exemplary. In another embodiment of the present application, the values ​​in the position formula can also take any other values. The above position formula can make the starting point of the area formed by the points to be searched corresponding to the next stage not be located in the search area corresponding to the previous stage, avoiding the position of the points to be searched corresponding to the minimum value of the SAD value obtained by the next stage search being close to the search area corresponding to the previous stage.

[0120] like Figure 4c As shown, the point to be searched corresponding to the minimum value of the SAD value may be the point to be searched 404, and the bestmv_x of the point to be searched 404 in the first area 401 in the first area 401 is 7PX, and the bestmv_y is 0PX. The delta_x of the point to be searched 404 in the first area 401 relative to the point to be searched 403 (starting point) in the first area 401 is 6PX, and the delta_y is 0PX. By the above formula, it can be determined that the starting point of the area formed by the points to be searched in the second stage search, new_start_point_x, is 5PX, and new_start_point_y is 2PX. That is, the starting point of the area formed by the second point to be searched in the second stage search may be the position of the pixel point in the first area 401 where the starting point is located, moved 1 pixel to the left and 2 pixels upward, that is, the point to be searched 406.

[0121] S504: Determine whether a preset search condition is met.

[0122] In the embodiment of the present application, the preset search condition here may include: if the starting point of the area formed by the points to be searched in the next stage is the same as the starting point of the area formed by the points to be searched in the previous stage and the number of searches reaches the search threshold, then step S505 is executed to calculate the corresponding motion displacement between the image block to be processed and the reference image, and the motion estimation of the current image block of the current image in the reference image is terminated, and then the motion estimation of the next image block of the current image in the reference image is started. Otherwise, step S502 is executed to continue to determine the target matching point of the area formed by the points to be searched in the next stage.

[0123] exist Figure 4c In the reference image 400 shown in FIG. 1 , 16 more points to be searched are determined with the point to be searched 406 as the starting point. The 16 points to be searched form a second area 405. Figure 4cThe 16 points to be searched can be represented by ②, and it can be seen that the size of the second area 405 is Figure 4a The first regions 401 are shown to have the same size, for example, the second region 405 may have a size of 7×4.

[0124] A second stage search is performed on the reference image 400 based on the current image 300, such as Figure 4d As shown, a comparison area 407 is formed with the position of the to-be-searched point 406 located in the second area 405 as a corner point, such as Figure 4d As shown, the point at the upper left corner of the comparison area 407 coincides with the point to be searched 406 .

[0125] After calculating the SAD values ​​corresponding to the 16 points to be searched in the second area 405 in sequence, a minimum value is determined from the SAD values ​​corresponding to the 16 points to be searched, and the area formed by the third point to be searched for the third stage search is determined based on the point to be searched corresponding to the minimum SAD value.

[0126] In the embodiments of the present application, Figure 4e As shown, the point to be searched corresponding to the minimum value of the SAD value can be the point to be searched 408, and the bestmv_x of the point to be searched 408 in the second area 405 is 5PX, and the bestmv_y is 2PX. The delta_x of the point to be searched 408 in the second area 405 relative to the point to be searched 406 (starting point) in the second area 405 is 4PX, and the delta_y is 1PX. Through the above formula, it can be determined that the position of the starting point of the area to be searched in the third stage new_start_point_x is 3PX, and the new_start_point_y is 1PX, that is, the position of the starting point of the area to be searched in the third stage can be the position of the starting point (point to be searched 406) in the second area 405, which is moved 1 pixel to the left and 2 pixels upward, that is, the point to be searched 410.

[0127] exist Figure 4e In the reference image 400 shown in FIG. 1 , 16 more points to be searched are determined with the point to be searched 410 as the starting point. The 16 points to be searched form a third area 409. Figure 4e The 16 points to be searched can be represented by ③, and it can be seen that the size of the third area 409 is Figure 4a The first regions 401 are shown to have the same size, for example, the third region 409 may have a size of 7×4.

[0128] The third stage search is performed on the reference image 400 based on the current image 300, such as Figure 4f As shown, a comparison area 411 is formed with the position of the to-be-searched point 410 located in the third area 409 as a corner point, as shown in FIG. Figure 4fAs shown, the point at the upper left corner of the comparison area 411 coincides with the point to be searched 410 .

[0129] After calculating the SAD values ​​corresponding to the 16 points to be searched in the third area 409 in sequence, a minimum value is determined from the SAD values ​​corresponding to the 16 points to be searched, and the area formed by the third points to be searched for the third stage search is determined based on the points to be searched corresponding to the minimum SAD value.

[0130] In the embodiments of the present application, Figure 4g As shown, the point to be searched corresponding to the minimum value of the SAD value may be the point to be searched 412, and the bestmv_x of the point to be searched 412 in the third area 409 is 7PX, and the bestmv_y is 0PX. The delta_x of the point to be searched 412 in the third area 409 relative to the point to be searched 410 (starting point) in the third area 409 is 6PX, and the delta_y is 0PX. By the above formula, it can be determined that the starting point of the area formed by the fourth point to be searched in the fourth stage of search, new_start_point_x, is 5PX, and new_start_point_y is 2PX. That is, the starting point of the area formed by the point to be searched in the fourth stage of search may be the position of the point to be searched 414, which is 1 pixel to the left and 2 pixels upward of the pixel point where the starting point (point to be searched 410) is located in the third area 409.

[0131] S505: Calculate the corresponding motion displacement between the image block to be processed and the reference image.

[0132] exist Figure 4g In the reference image 400 shown in FIG. 1 , 16 more points to be searched are determined with the point to be searched 414 as the starting point. The 16 points to be searched form a fourth area 413. Figure 4g The 16 points to be searched can be represented by ④, and it can be seen that the size of the fourth area 413 is Figure 4a The first regions 401 shown have the same size, for example, the fourth region 413 may have a size of 7×4.

[0133] The fourth stage search is performed on the reference image 400 based on the current image 300, such as Figure 4h As shown, a comparison area 415 is formed with the position of the to-be-searched point 414 located in the fourth area 413 as a corner point. Figure 4h As shown, the point at the upper left corner of the comparison area 415 coincides with the point to be searched 414 .

[0134] After the SAD values ​​corresponding to the 16 points to be searched in the fourth area 413 are calculated in sequence, a minimum value is determined from the SAD values ​​corresponding to the 16 points to be searched, and the point to be searched corresponding to the minimum SAD value is used as the final matching point, that is, the matching position. Figure 4i As shown, the final matching point here can be the point to be searched 416, and the vector between the point to be searched 413 and the pixel point in the image block 301 in the current image 300 (the pixel point at the upper left corner of the image block 301) is determined as the motion displacement between the image block 301 in the current image 300 and the reference image 400.

[0135] It can be seen that in Figures 4a to 4i In the example, the sizes of the first area 401 to the fourth area 413 are the same, so that after completing each stage of search, the position of the matching point used to determine the point to be searched in the next stage of search will not fall near the matching point searched in the previous stage, that is, it is avoided that the matching point determined in the next stage of search will be closer and closer to the matching point searched in the previous stage, resulting in the final matching point determined to be located near the first area. The accuracy of motion estimation of high-speed moving objects in the reference image and the current image is improved.

[0136] It can be understood that in an embodiment of the present application, if the starting point of the area formed by the points to be searched in the next stage is the same as the starting point of the area formed by the points to be searched in the previous stage, the motion estimation of the current image block of the current image in the reference image is terminated, and then the motion estimation of the next image block of the current image in the reference image is started.

[0137] In an embodiment of the present application, after obtaining the motion displacement between an image block determined in the current image and a reference image, the image block can be motion compensated based on the motion displacement and then a residual value between the image block and the reference image can be calculated, and then the current image can be encoded with the motion compensation information and the calculated residual value information. When decoding the current image, motion compensation prediction can be performed on the current image based on the decoded motion compensation information and the residual value to obtain complete image data of the current image for display.

[0138] Figure 7 According to an embodiment of the present application, a schematic block diagram of the structure of a computer 700 applicable to a pixel-based motion estimation method is shown.

[0139] like Figure 7 As shown, the computer 700 may include a processor 710, a wireless communication module 720, a mobile communication module 730, a power module 740, an audio module 750, an interface module 760, a camera 770, a memory 780, a sensor module 790, a button 701, and a display screen 702, etc.

[0140] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the computer 700. In other embodiments of the present application, the computer 700 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0141] The processor 710 may include one or more processing units, for example, a central processing unit CPU (Central Processing Unit), a graphics processor GPU (Graphics Processing Unit), a digital signal processor DSP, a microprocessor MCU (Micro-programmed Control Unit), an AI (Artificial Intelligence) processor or a programmable logic device FPGA (Field Programmable Gate Array) and other processing modules or processing circuits. Among them, different processing units can be independent devices or integrated in one or more processors. A storage unit can be set in the processor 710, such as a memory 780 for storing instructions and data. In some embodiments, the storage unit in the processor 710 is a cache memory.

[0142] The wireless communication module 720 may include an antenna, and transmit and receive electromagnetic waves via the antenna. The wireless communication module 720 may provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication technology (NFC), infrared technology (IR), etc., applied to the computer 700. The computer 700 may communicate with the network and other devices through wireless communication technology.

[0143] The mobile communication module 730 may include, but is not limited to, an antenna, a power amplifier, a filter, a low noise amplifier (LNA), etc. The mobile communication module 730 may provide a solution for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the computer 700. The mobile communication module 730 may receive electromagnetic waves by an antenna, filter, amplify, and process the received electromagnetic waves, and transmit them to a modulation and demodulation processor for demodulation. The mobile communication module 730 may also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna. In some embodiments, at least some of the functional modules of the mobile communication module 730 may be disposed in the processor 710. In some embodiments, at least some of the functional modules of the mobile communication module 730 may be disposed in the same device as at least some of the modules of the processor 710.

[0144] In some embodiments, the mobile communication module 730 and the wireless communication module 720 of the computer 700 may also be located in the same module.

[0145] The power module 740 may include a power source, a power management component, etc. The power source may be a battery. The power management component is used to manage the charging of the power source and the supply of power to other modules.

[0146] The audio module 750 is used to convert digital audio information into a predicted audio signal output, or convert predicted audio input into a digital audio signal. The audio module 750 can also be used to encode and decode audio signals.

[0147] The interface module 760 includes an external memory interface, a universal serial bus (USB) interface, a subscriber identification module (SIM) card interface, and the like.

[0148] The camera 770 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP (Image Signal Processing) to convert it into a digital image signal. The computer 700 can realize the shooting function through the ISP, the camera 770, the video codec, the GPU (Graphic Processing Unit), the display screen 702, and the application processor.

[0149] The sensor module 790 may include a proximity light sensor, a pressure sensor, a gyro sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, and the like.

[0150] The display screen 702 is used to display a human-computer interaction interface, images, videos, etc. The display screen 702 includes a display panel.

[0151] In some embodiments, the computer 700 further includes a button 701, a motor, and an indicator. The button 701 may include a volume button, an on / off button, etc. The motor is used to make the computer 700 vibrate, for example, vibrate when the user's computer 700 is called, so as to prompt the user to answer the call of the computer 700. The indicator may include a laser indicator, a radio frequency indicator, an LED indicator, etc.

[0152] Figure 8 According to an embodiment of the present application, a structural schematic block diagram of a video encoding system 800 is shown.

[0153] like Figure 8 As shown, the system 800 may include one or more processors 804, a system control logic 808 connected to at least one of the processors 804, a system memory 812 connected to the system control logic 808, a non-volatile memory (NVM) 816 connected to the system control logic 808, and a network interface 820 connected to the system control logic 808.

[0154] In some embodiments, the processor 804 may include one or more single-core or multi-core processors. In some embodiments, the processor 804 may include any combination of a general-purpose processor and a dedicated processor (e.g., a graphics processor, an application processor, a baseband processor, etc.). The processor 804 may be configured to execute various embodiments, for example, the embodiments described with reference to the above-mentioned Figures 2-5.

[0155] In some embodiments, system control logic 808 may include any suitable interface controller to provide any suitable interface to at least one of processors 804 and / or any suitable device or component in communication with system control logic 808 .

[0156] In some embodiments, the system control logic 808 may include one or more memory controllers to provide an interface to a system memory 812. The system memory 812 may be used to load and store data and / or instructions. In some embodiments, the memory 812 of the system 800 may include any suitable volatile memory, such as a suitable dynamic random access memory (DRAM).

[0157] NVM / memory 816 may include one or more tangible, non-transitory computer storage media for storing data and / or instructions. In some embodiments, NVM / memory 816 may include any suitable non-volatile memory such as flash memory and / or any suitable non-volatile storage device, such as at least one of a HDD (Hard Disk Drive), a CD (Compact Disc) drive, and a DVD (Digital Versatile Disc) drive. NVM / memory 816 may include a portion of storage resources on the device on which system 800 is installed, or it may be accessed by the device but is not necessarily a part of the device. For example, NVM / memory 816 may be accessed over a network via network interface 820.

[0158] In particular, system memory 812 and NVM / storage 816 may include, respectively, a temporary copy and a permanent copy of instructions 824. Instructions 824 may include instructions that, when executed by at least one of processors 804, cause system 800 to implement the functionality of the encoder structure shown in FIG. 4 above. In some embodiments, instructions 824, hardware, firmware, and / or software components thereof may be additionally / alternatively located in system control logic 808, network interface 820, and / or processor 804.

[0159] The network interface 820 may include a transceiver for providing a radio interface for the system 800 to communicate with any other suitable devices (such as a front-end module, an antenna, etc.) through one or more networks. In some embodiments, the network interface 820 may be integrated with other components of the system 800. For example, the network interface 820 may be integrated with at least one of the processor 804, the system memory 812, the NVM / storage 816, and a firmware device (not shown) having instructions. When at least one of the processors 804 executes the instructions, the system 800 implements the following. Figure 5 Functions shown.

[0160] The network interface 820 may further include any suitable hardware and / or firmware to provide a multiple-input multiple-output radio interface. For example, the network interface 820 may be a network adapter, a wireless network adapter, a telephone modem and / or a wireless modem.

[0161] In one embodiment, at least one of the processors 804 may be packaged together with logic for one or more controllers of the system control logic 808 to form a system in package (SiP). In one embodiment, at least one of the processors 804 may be integrated on the same die with logic for one or more controllers of the system control logic 808 to form a system on chip (SoC).

[0162] The system 800 may further include an input / output (I / O) device 832. The I / O device 832 may include a user interface to enable a user to interact with the system 800; and a peripheral component interface design to enable peripheral components to interact with the system 800. In some embodiments, the system 800 further includes a sensor for determining at least one of an environmental condition and location information related to the system 800.

[0163] Fig. 9 According to an embodiment of the present application, a schematic block diagram of a SoC (System on Chip) 900 is shown. Fig. 9 In the , similar components have the same reference numerals. In addition, the dashed boxes are optional features of more advanced SoCs. Fig. 9 In the embodiment, SoC 900 includes: an interconnect unit 950, which is coupled to an application processor 910; a system agent unit 970; a bus controller unit 980; an integrated memory controller unit 940; a group or one or more coprocessors 920, which may include integrated graphics logic, an image processor, an audio processor, and a video processor; a static random access memory (SRAM) unit 930; and a direct memory access (DMA) unit 960. In one embodiment, the coprocessor 920 includes a special-purpose processor, such as, for example, a network or communication processor, a compression engine, a GPGPU, a high-throughput MIC processor, or an embedded processor.

[0164] The various embodiments of the mechanism disclosed in the present application can be implemented in hardware, software, firmware or a combination of these implementation methods. The embodiments of the present application can be implemented as a computer program or program code executed on a programmable system, which includes at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device and at least one output device.

[0165] Program code can be applied to input instructions to perform each function described in the present application and generate output information. Output information can be applied to one or more output devices in a known manner. For the purpose of the present application, the processing system includes any system with a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC) or a microprocessor. The program code can be implemented with a high-level programming language or an object-oriented programming language to communicate with the processing system. When necessary, the program code can also be implemented with assembly language or machine language. In fact, the mechanism described in the present application is not limited to the scope of any specific programming language. In either case, the language can be a compiled language or an interpreted language.

[0166] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, instructions may be distributed over a network or through other computer storage media. Therefore, machine storage media may include any mechanism for storing or transmitting information in a machine (e.g., computer) readable form, including, but not limited to, floppy disks, optical disks, optical disks, read-only memories (CD-ROMs), magneto-optical disks, read-only memories (ROMs), random access memories (RAMs), erasable programmable read-only memories (EPROMs), electrically erasable programmable read-only memories (EEPROMs), magnetic or optical cards, flash memory, or a tangible machine-readable memory for transmitting information (e.g., carrier waves, infrared signals, digital signals, etc.) using the Internet in electrical, optical, acoustic, or other forms of propagation signals. Therefore, machine storage media include any type of machine storage media suitable for storing or transmitting electronic instructions or information in a machine (e.g., computer) readable form.

[0167] References to "one embodiment" or "an embodiment" in the specification mean that the specific features, structures, or characteristics described in conjunction with the embodiment are included in at least one exemplary implementation or technology disclosed according to the embodiment of the present application. The appearance of the phrase "in one embodiment" in various places in the specification does not necessarily all refer to the same embodiment.

[0168] In addition, the language used in this specification has been primarily selected for readability and instructional purposes and may not be selected to describe or limit the disclosed subject matter. Therefore, the present application embodiment disclosure is intended to illustrate rather than limit the scope of the concepts discussed herein.

Claims

1. An image coding method, characterized in that , including: Obtain the image to be encoded, and divide the image to be encoded into multiple image blocks; Search for a target block that matches each of the image blocks from a reference frame; Determine a motion vector between the target block and the image block; Encode the image to be encoded based on the motion vector; Wherein, the searching for a target block that matches each of the image blocks from a reference frame includes: Determine a reference block that matches the image block in the reference frame through at least N searches, wherein, The determining a reference block that matches the image block in the reference frame through at least N searches includes: In the M-th search, search for a first target matching point from S first points to be searched, wherein the starting point of the M-th search is a first starting search point different from the first target matching point among the S first points to be searched; Determine a second starting search point for the (M + 1)-th search based on the coordinate distance between the first target matching point and the first starting search point. Among them, the S first points to be searched form a first rectangular area. In the (M + 1)-th search, the S second points to be searched form a second rectangular area. The second rectangular area has the same size as the first rectangular area, and the second starting search point is different from the first target matching point and the first starting search point. The first starting search point is the first corner point of the first rectangular area, and M, N, and S are natural numbers, M < N, and corresponding to the coordinate distance being less than or equal to a preset threshold, the second starting search point is located at a first position. Corresponding to the coordinate distance being greater than the preset threshold, the second starting search point is located at a second position, and the direction from the first starting search point to the first position is opposite to the direction from the first starting search point to the first target matching point, and the direction from the first starting search point to the second position is the same as the direction from the first starting search point to the first target matching point.

2. The method according to claim 1, characterized in that The coordinate distance includes the distance from the first target matching point along the first side length of the first rectangular area to the first starting search point or the distance from the first target matching point along the second side length of the first rectangular area to the first starting search point.

3. The method according to claim 2, characterized in that The determining a second starting search point for the (M + 1)-th search based on the coordinate distance between the first target matching point and the first starting search point includes: Determine the second starting search point through the following formula new_start_point_x = if(delta_x <= 2) bestmv_x – 5; else bestmv_x – 1, new_start_point_y = if(delta_y <= 1) bestmv_y – 2; else bestmv_y – 1, Among them, new_start_point_x represents the offset of the second starting search point relative to the first starting search point in the direction of the first side length, new_start_point_y represents the offset of the second starting search point relative to the first starting search point in the direction of the second side length, bestmv_x and bestmv_y represent the position of the first target matching point in the first rectangular area, delta_x represents the coordinate distance between the first target matching point and the first starting search point in the direction of the first side length, and delta_y represents the coordinate distance between the first target matching point and the first starting search point in the direction of the second side length.

4. The method according to claim 1, characterized in that: The step of searching for a first target matching point from the S first points to be searched includes: Traversing the S first points to be searched, and forming S first comparison areas with each of the first points to be searched as a corner point; Comparing the pixels included in each of the first comparison areas with the pixels included in the image block; Corresponding to the first comparison area that meets the preset matching condition, the first to-be-searched point corresponding to the first comparison area is determined as the first target matching point.

5. The method according to claim 4, characterized in that The preset matching condition includes that the sum of absolute differences (SAD) between the pixels included in the first comparison area and the pixels included in the image block is less than a preset threshold.

6. The method according to claim 1, characterized in that Also includes: Corresponding to the second starting search point meeting the preset search condition, the vector difference between the image block and the first target matching point is taken as the motion vector.

7. The method according to claim 6, characterized in that The meeting of the preset search condition includes: the second starting search point is the same as the first starting search point.

8. An image coding system, characterized in that , the system comprises: An acquisition module, the acquisition module is used to acquire an image to be encoded and divide the image to be encoded into a plurality of image blocks; A search module, the search module is used to search for a target block matching each of the image blocks from a reference frame; A calculation module, the calculation module is used to determine a motion vector between the target block and the image block; An encoding module, the encoding module being used to encode the image to be encoded based on the motion vector; The step of searching for a target block matching each of the image blocks from a reference frame includes: Determine a reference block matching the image block in the reference frame by searching at least N times, wherein: The step of determining, in the reference frame, a reference block matching the image block by searching at least N times comprises: In an M-th search, a first target matching point is searched from the S first points to be searched, wherein a starting point of the M-th search is a first starting search point among the S first points to be searched that is different from the first target matching point; Determine a second starting search point for the (M + 1)-th search based on the coordinate distance between the first target matching point and the first starting search point. Among them, S first points to be searched form a first rectangular area. In the (M + 1)-th search, S second points to be searched form a second rectangular area. The second rectangular area has the same size as the first rectangular area, and the second starting search point is different from the first target matching point and the first starting search point. The first starting search point is the first corner point of the first rectangular area, and M, N, and S are natural numbers, M < N. And corresponding to the coordinate distance being less than or equal to a preset threshold, the second starting search point is located at a first position. Corresponding to the coordinate distance being greater than the preset threshold, the second starting search point is located at a second position. And the direction from the first starting search point to the first position is opposite to the direction from the first starting search point to the first target matching point, and the direction from the first starting search point to the second position is the same as the direction from the first starting search point to the first target matching point.

9. An electronic device, characterized in that: Comprising: A memory for storing instructions executed by one or more processors of the electronic device, and a processor, which is one of the processors of the electronic device, for executing the image encoding method according to any one of claims 1 to 7.

10. A readable medium, characterized in that: Instructions are stored on the readable medium, and when the instructions are executed on the electronic device, the electronic device is caused to execute the image encoding method according to any one of claims 1 to 7.

Images