Image processing methods, apparatus, devices, storage media and program products

By performing multi-stage image block matching on reference image frames using an interval search method, the problems of high computational load and low efficiency in existing technologies are solved, achieving efficient image block matching that is suitable for image processing in video encoding and cloud conferencing.

CN115665424BActive Publication Date: 2026-11-17TENCENT TECHNOLOGY (SHENZHEN) CO LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202211454712.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-11-21
Publication Date
2026-11-17
Estimated Expiration
2042-11-21

AI Technical Summary

Technical Problem

Existing motion estimation methods involve high computational costs when matching image blocks, resulting in low efficiency in block search and making it difficult to balance search efficiency and matching accuracy.

Method used

An interval search method is used to perform image patch matching on reference image frames. Multi-stage search is used to improve the matching degree and reduce the amount of computation, including the condition judgment of the initial matching image patch and the subsequent K stages of search. The search area is optimized by using the target search range and step size parameters.

Benefits of technology

While maintaining a high degree of matching, it significantly improves the search efficiency of matching blocks, reduces the amount of computation, adapts to the processing speed requirements of different application scenarios, and improves the processing efficiency of video encoding and cloud conferencing.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115665424B_ABST
    Figure CN115665424B_ABST
Patent Text Reader

Abstract

This application provides an image processing method, apparatus, device, storage medium, and program product. The method includes: if an initial matching image block obtained by searching a reference image frame does not meet the matching conditions, then for the I-th search stage, where I is any positive integer less than or equal to the number of search stages K; determining the target search range parameter and the target search step size parameter for the I-th search stage; determining the I-th search image region based on the target search range parameter and the position of the intermediate matching image block determined in the (I-1)-th search stage in the reference image frame; performing image block matching search for the I-th search image region according to the search step size determined by the target search step size parameter to determine the intermediate matching image block; and determining the intermediate matching image block determined in the K-th search stage as the target matching image block that matches the image block to be processed. This application can improve the search efficiency of matching blocks while ensuring a high degree of matching.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to image processing methods, image processing apparatus, computer equipment, computer-readable storage media, and computer program products. Background Technology

[0002] Motion estimation (ME) is a technique widely used in image processing fields such as video coding. The basic idea of ​​motion estimation is to divide each frame of a video frame sequence or image sequence into many non-overlapping image blocks, and then search for matching blocks (i.e., matching image blocks) in the reference frame. The relative displacement between the image block and the corresponding matching block is the motion vector. A full search algorithm can find matching blocks corresponding to image blocks by traversing the reference frames. While this method can find matching blocks with high matching degrees, it is computationally intensive, resulting in low search efficiency. In practical applications, it is usually necessary to balance the search efficiency and matching degree of matching blocks, which is more beneficial for subsequent processing such as in video coding. Summary of the Invention

[0003] This application provides image processing methods, apparatus, devices, storage media, and program products, which can effectively improve the search efficiency of matching blocks while ensuring that the matching blocks have a high degree of matching.

[0004] On one hand, embodiments of this application provide an image processing method, the method comprising:

[0005] An interval search method is used to perform image block matching search on the reference image frame to obtain the initial matching image block of the image block to be processed. The image block to be processed is an image block in the image frame to be processed, and the image frame to be processed is associated with the reference image frame.

[0006] If the initial matching image block does not meet the matching conditions, then the number of search stages K is obtained, where K is an integer greater than 1. The number of search stages K is used to indicate the number of times the search is performed in the reference image frame based on the initial matching image block.

[0007] For the I-th search stage, the target search range parameter and target search step size parameter of the I-th search stage are determined. Based on the target search range parameter and the position of the intermediate matching image block determined in the (I-1)-th search stage in the reference image frame, the I-th search image region in the reference image frame is determined. According to the search step size determined according to the target search step size parameter, image block matching search is performed on the I-th search image region to determine the intermediate matching image block of the image block to be processed; where I is any positive integer less than or equal to K.

[0008] The intermediate matching image block of the image block to be processed determined in the Kth search phase is determined as the target matching image block in the reference image frame that matches the image block to be processed.

[0009] On one hand, embodiments of this application provide an image processing apparatus, the apparatus comprising:

[0010] The acquisition unit is used to perform image block matching search on the reference image frame using an interval search method to obtain the initial matching image block of the image block to be processed, wherein the image block to be processed is an image block in the image frame to be processed, and the image frame to be processed is associated with the reference image frame.

[0011] The processing unit is configured to obtain the number of search stages K if the initial matching image block does not meet the matching conditions, where K is an integer greater than 1, and the number of search stages K is used to indicate the number of times the search is performed in the reference image frame based on the initial matching image block.

[0012] The processing unit is further configured to, for the I search stage, determine the target search range parameter and the target search step size parameter of the I search stage; determine the I search image region in the reference image frame based on the target search range parameter and the position of the intermediate matching image block determined in the (I-1)th search stage; and perform image block matching search on the I search image region according to the search step size determined based on the target search step size parameter to determine the intermediate matching image block of the image block to be processed; wherein, I is any positive integer less than or equal to K;

[0013] The processing unit is further configured to determine the intermediate matching image block of the image block to be processed determined in the Kth search stage as the target matching image block in the reference image frame that matches the image block to be processed.

[0014] On one hand, embodiments of this application provide a computer device, including: a processor and a memory, the processor and the memory being interconnected, wherein the memory stores executable program code, and the processor is used to call the executable program code to implement the image processing method provided in embodiments of this application.

[0015] Accordingly, embodiments of this application also provide a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to implement the image processing method provided in embodiments of this application.

[0016] Accordingly, embodiments of this application also provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to implement the image processing method provided in embodiments of this application.

[0017] In this application, if the initial matching image block determined from the reference image frame using the interval search method does not meet the matching conditions, the search stage number K is obtained, and subsequent searches are performed on the reference image frame. For the I-th search stage (I is a positive integer less than or equal to K), the search range parameter and search step size parameter for this search stage are determined, and the I-th search image region is determined based on the search range parameter and the intermediate matching image block determined in the (I-1)-th search stage. The I-th search image region is then searched according to the search step size determined based on the target search step size parameter to determine the intermediate matching image block for the I-th search stage. The intermediate matching image block determined in the K-th search stage is then determined as the target matching image block. The image processing method provided in this application uses an interval search method to search the reference image frame, which can quickly determine the initial matching image block. Based on the initial matching image block, a multi-stage subsequent matching search is performed, which can obtain a target matching image block with a higher matching degree than the initial image matching block. Since the subsequent search only targets a small part of the reference image frame, it does not increase the amount of computation too much. This also makes the subsequent matching search highly efficient. Compared with the full search algorithm that traverses the reference frame, this method has less computation and higher matching block search efficiency. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of a system architecture to which the image processing method provided in the embodiments of this application applies;

[0020] Figure 2 This is a schematic flowchart of an image processing method provided in an embodiment of this application;

[0021] Figure 3a This illustrates one way to determine the parameters;

[0022] Figure 3b This demonstrates another way to determine the parameters;

[0023] Figure 3c This demonstrates another way to determine the parameters;

[0024] Figure 4 This illustrates one method for determining the search image region;

[0025] Figure 5 This demonstrates one method for searching a region of an image;

[0026] Figure 6 This is a schematic diagram of an image processing method provided in an embodiment of this application;

[0027] Figure 7 This is a schematic diagram of the structure of an image processing device provided in an embodiment of this application;

[0028] Figure 8 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0029] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0030] It should be noted that the terms "first," "second," etc., used in the embodiments of this application are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a technical feature specified with "first" or "second" may explicitly or implicitly include at least one of those features.

[0031] This application provides an image processing method that employs an interval search approach to perform image patch matching search on a reference image frame to obtain an initial matching image patch. When the initial matching image patch does not meet the matching conditions, a subsequent K-stage search is performed on the reference image frame. This effectively improves the matching degree between the matching patch and the image patch while consuming relatively little computation, thereby improving the search efficiency of the matching patch. The image processing method provided in this application can be applied to fields or scenarios involving searching for matching image patches, such as motion estimation, video coding, and cloud conferencing.

[0032] Video coding refers to the method of converting a file in one video format into another through compression technology. Video coding includes techniques such as motion compensation, motion representation, motion estimation, transform coding, and quantization coding. Among these, motion estimation is a technique that extracts motion information from a sequence of image frames. Its core idea is to search for matching image blocks with a high degree of similarity to the current image block in a reference image frame, thereby determining the motion vector between the current image block and the matching image block. The quality of the motion estimation method directly affects the efficiency of the entire video coding method. The image processing method provided in this application can improve the search efficiency of matching blocks while ensuring a high degree of matching, thus improving the efficiency of video coding.

[0033] Cloud conferencing is an efficient, convenient, and low-cost form of conferencing based on cloud computing technology. Users only need to perform simple and easy-to-use operations through an internet interface to quickly and efficiently share voice, data files, and video with teams and clients worldwide. The complex technologies involved in data transmission and processing during the meeting are handled by the cloud conferencing service provider. In cloud conferencing scenarios, video image frames need to be encoded, and the processed encoded data is then transmitted. Therefore, the efficiency of image frame processing is crucial in cloud conferencing. The image processing method provided in this application can ensure a high matching degree of matching blocks and effectively improve the matching efficiency of video image frames, thereby improving the processing efficiency of conferencing video, saving on cloud conferencing video transmission bitrate, and saving data transmission bandwidth.

[0034] The image processing method provided in this application embodiment can be applied to... Figure 1 The image processing system shown includes an image processing device 101 and a database 102. The database 102 stores a sequence of image frames to be processed, and the image processing device 101 can obtain image frame data from the database 102 to perform image block matching processing.

[0035] In feasible implementations, the image processing device 101 can be a server. This server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal device can be a smartphone, tablet, laptop, desktop computer, intelligent voice interaction device, smart home appliance, in-vehicle terminal, etc., but is not limited to these.

[0036] In feasible implementations, the image processing device 101 may be a terminal device. The terminal device may be a smart home appliance, a handheld device with wireless communication capabilities (such as a smartphone or tablet), a computing device (such as a personal computer (PC), an in-vehicle terminal, a smart voice interaction device, a wearable device, or other smart devices, but is not limited thereto).

[0037] The image processing method provided in this application embodiment can be implemented by the image processing device 101, including but not limited to the following steps:

[0038] Image processing device 101 performs image block matching search on a reference image frame using an interval search method to obtain an initial matching image block for the image block to be processed. This image block to be processed is an image block in the image frame to be processed, which is associated with the reference image frame. If the initial matching image block does not meet the matching conditions, the number of search stages K is obtained, where K is an integer greater than 1. The number of search stages K indicates the number of times the search is performed in the reference image frame based on the initial matching image block. A specific range in the reference image frame is searched in K stages. Specifically, for the I-th search stage (I is any positive integer less than or equal to K), the target search range parameter and the target search step size parameter for the I-th search stage are determined. Based on the target search range parameter and the position of the intermediate matching image block determined in the (I-1)-th search stage in the reference image frame, the I-th search image region in the reference image frame is determined. According to the search step size determined based on the target search step size parameter, image block matching search is performed on the I-th search image region to determine the intermediate matching image block of the image block to be processed. After the K-stage search is completed, the intermediate matching image block of the image block to be processed determined in the K-th search stage is determined as the target matching image block in the reference image frame that matches the image block to be processed.

[0039] The image processing method provided in this application uses an interval search method to search the reference image frame, which can reduce the computational load of image block search. When the initial matching image block does not meet the matching conditions, subsequent searches are performed based on the initial matching image block, which can further improve the matching degree between the matching image block and the image block to be processed. Since the subsequent search is performed in the region centered on the initial matching image block, the global search of the reference image frame is avoided, so the computational load of the subsequent search is small, thereby improving the search efficiency of the matching image block.

[0040] It is understood that the system architecture diagrams illustrating the image processing methods described in the embodiments of this application are for the purpose of more clearly explaining the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. Those skilled in the art will recognize that... Figure 1The number of image processing devices 101 shown is merely illustrative. Any number of servers and terminal devices can be configured according to business needs. Furthermore, as system architecture evolves and new business scenarios emerge, the technical solutions provided in this application embodiment are equally applicable to similar technical problems.

[0041] It should be noted that the reference image frames and other related data involved in the embodiments of this application are all obtained with user authorization. Furthermore, when the embodiments of this application are applied to specific products or technologies, the data used must be authorized by the user, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0042] Please see Figure 2 , Figure 2 This is a schematic flowchart illustrating an image processing method provided in an embodiment of this application. This image processing method can be implemented by the aforementioned image processing device 101, or by other image processing devices. The flowchart of the image processing method provided in this embodiment includes, but is not limited to:

[0043] S201. Use the interval search method to perform image block matching search on the reference image frame to obtain the initial matching image block of the image block to be processed.

[0044] In this embodiment, the image block to be processed is an image block within an image frame to be processed, and the image frame to be processed is associated with a reference image frame. In one embodiment, the image frame to be processed and the reference image frame may be image frames from the same video.

[0045] In one embodiment, the interval search method can be a fast-converging motion estimation algorithm. Fast-converging motion estimation algorithms mainly include three-step search algorithms, four-step search algorithms, and the Diamond Search algorithm, etc. The main method of fast motion estimation algorithms is to search for several pixels within a reference image frame according to a certain rule, and then calculate the minimum difference sum of the matching blocks corresponding to each pixel. Based on the minimum difference sum, the matching block that matches the image block is determined. However, in the application of fast motion estimation algorithms, the "local optimum" problem may occur. In practical applications, fast-converging motion estimation algorithms can consume less computation to search for matching image blocks, but the matching degree of the obtained matching image blocks may be low, and the "local optimum" problem may occur.

[0046] In this embodiment, the matching error is determined based on the initial matching image block and the image block to be processed. The matching error is used to determine whether the initial matching image block meets the matching conditions. If the initial matching image block meets the matching conditions, it is determined as the target matching image block. If the initial matching image block does not meet the matching conditions, step S202 and subsequent steps are executed.

[0047] In this embodiment, if the matching error between the initial matching image block and the image block to be processed is greater than or equal to the error threshold, it is determined that the initial matching image block does not meet the matching conditions, and step S202 and subsequent steps need to be performed. If the matching error is less than the error threshold, it indicates that the matching degree of the initial matching image block is high, and no further search is needed; the initial matching image block can be directly determined as the target matching image block. In the method provided in this embodiment, if the matching error meets the matching conditions, no further search is needed, and the initial matching image block can be used as the target matching image block, reducing the computational load of subsequent searches and improving image processing efficiency.

[0048] In one embodiment, the method for determining the matching error between the initial matching image block and the image block to be processed can be as follows: For any pixel in the image block to be processed, calculate the pixel difference between the pixel value of the pixel and the pixel value of the matching pixel in the initial matching image block, and calculate the vector difference between the position coordinates of the pixel and the position coordinates of the matching pixel in the initial matching image block, wherein the position of the matching pixel in the initial matching image block is the same as the position of the pixel in the image block to be processed. Based on the pixel difference and vector difference corresponding to each pixel in the image block to be processed, determine the pixel difference parameter and vector difference parameter of the image block to be processed; perform a weighted summation of the pixel difference parameter and the vector difference parameter to obtain the matching error between the image block to be processed and the initial matching image block.

[0049] S202. If the initial matching image patch does not meet the matching conditions, then obtain the number of search stages K.

[0050] In this embodiment, if the initial matching image block does not meet the matching conditions, a search stage number K is obtained, where K is an integer greater than 1 and less than the initial search range parameter. The search stage number K indicates the number of searches performed in the reference image frame based on the initial matching image block. The search stage number K is related to the processing speed of the processing module, and can be directly proportional to the processing speed. The processing module can refer to a processing device in an image processing apparatus used for image block matching related processing, such as a processor, video encoder, etc., of the image processing apparatus.

[0051] In one embodiment, the number of search stages K can be determined based on the processing speed of the processing module in the image processing device and the actual application scenario. For example, in the processing of live video, a faster processing speed is required, and the resolution of the live video can be lower; therefore, a larger number of search stages K can be determined to ensure faster image processing. In the encoding of video-on-demand, a higher resolution is required; therefore, a smaller number of search stages K can be determined to ensure a higher matching degree between the image blocks in the image frame to be processed and the matching image blocks determined in the reference image frame, thereby improving the resolution of the video-on-demand.

[0052] If the initial matching image block does not meet the matching conditions, it means that the matching degree of the initial matching image block is not high. Then, K search stages are performed on the reference image frame. The specific implementation of each search stage can be step S203.

[0053] S203. For the first search stage, determine the target search range parameter and the target search step size parameter for the first search stage. Based on the target search range parameter and the position of the intermediate matching image block determined in the first-1 search stage in the reference image frame, determine the first search image region in the reference image frame. Perform image block matching search on the first search image region according to the search step size determined according to the target search step size parameter to determine the intermediate matching image block of the image block to be processed.

[0054] In this embodiment, I is any positive integer less than or equal to K. The target search range parameter and target search step size parameter for the I-th search stage can be determined as follows:

[0055] When I is 1, please refer to Figure 3a The steps for determining the target search range parameters and target search step size parameters for the first search phase can be as follows: Obtain the initial search range parameters, which include initial horizontal search range parameters and initial vertical search range parameters. Determine the initial horizontal search range parameters as the target horizontal search range parameters for the first search phase, determine the initial vertical search range parameters as the target vertical search range parameters for the first search phase, and determine the target horizontal and vertical search range parameters for the first search phase as the target search range parameters for the first search phase. Then, determine the horizontal search step size parameters for the first search phase based on the initial horizontal search range parameters and the number of search phases K, determine the vertical search step size parameters for the first search phase based on the initial vertical search range parameters and the number of search phases K, and determine the horizontal and vertical search step size parameters for the first search phase as the target search step size parameters for the first search phase.

[0056] In one embodiment, the method for obtaining the initial search range parameter can be to determine the initial search range parameter based on the processing speed of the processing module and the actual application scenario. For example, in the processing of live video, a faster processing speed is required, and the resolution of the live video can be lower; therefore, a smaller initial search range parameter can be determined to enable faster image processing. The initial search range parameter is used to indicate the search range in the reference image frame, and the initial search range parameter is related to the processing speed of the processing module.

[0057] In one embodiment, the initial search range parameters include an initial horizontal search range parameter n and an initial vertical search range parameter m, and the number of search stages is K. The initial horizontal search range parameter is determined as the target horizontal search range parameter for the first search stage, and is denoted as n. The initial vertical search range parameter is determined as the target vertical search range parameter for the first search stage, and is denoted as m. The target horizontal search range parameter and the target vertical search range parameter for the first search stage are then defined as the target search range parameter for the first search stage. The horizontal search step size parameter for the first search stage is determined based on the initial horizontal search range parameter n and the number of search stages K. The vertical search step size parameter for the first search stage is determined based on the initial vertical search range parameter and the number of search stages K. The horizontal and vertical search step size parameters of the first search phase are determined as the target search step size parameters of the first search phase. Therefore, the target search range parameters and target search step size parameters of the first search phase are determined.

[0058] When I is greater than 1 and less than K, please refer to Figure 3b The steps for determining the target search range parameters and target search step size parameters for the I search phase can be as follows: Obtain the target search step size parameters and initial search range parameters for the (I-1)th search phase. The target search step size parameters include horizontal search step size parameters and vertical search step size parameters. The initial search range parameters include initial horizontal search range parameters and initial vertical search range parameters. Determine the horizontal search step size parameters of the (I-1)th search phase as the target horizontal search range parameters for the I search phase. Determine the vertical search step size parameters of the (I-1)th search phase as the target vertical search range parameters for the I search phase. Determine the target horizontal search range parameters and target vertical search range parameters of the I search phase as the target search range parameters for the I search phase. Then, determine the horizontal search step size parameters for the I search phase based on the initial horizontal search range parameters, the number of search phases K, and I. Determine the vertical search step size parameters for the I search phase based on the initial vertical search range parameters, the number of search phases K, and I. Finally, determine the horizontal search step size parameters and vertical search step size parameters of the I search phase as the target search step parameters for the I search phase.

[0059] In one embodiment, the target search step size parameter in the (I-1)th search phase includes the horizontal search step size parameter hs. i-1 and vertical search step size parameter hv i-1 The initial search range parameters include the initial horizontal search range parameter n and the initial vertical search range parameter m; therefore, the horizontal search step size parameter of the (I-1)th search stage is determined as the target horizontal search range parameter of the I search stage, which is hs. i-1 The vertical search step size parameter of the (I-1)th search stage is determined as the target vertical search range parameter of the Ith search stage, which is hv. i-1 The target horizontal search range parameter and the target vertical search range parameter of the first search stage are determined as the target search range parameter of the first search stage. The horizontal search step size parameter hs of the first search stage is determined based on the initial horizontal search range parameter n, the number of search stages K, and I. i ,Right now, The vertical search step size parameter hv for the I-th search stage is determined based on the initial vertical search range parameter m, the number of search stages K, and I. i ,Right now, in, This is to round x down.

[0060] It should be noted that when I is greater than 1 and less than K, there may be multiple I-th search stages. The process of determining the target search range parameter and the target search step size parameter in each I-th search stage can be implemented with reference to the specific steps in the above embodiments.

[0061] When I equals K, please refer to Figure 3c The steps for determining the target search range parameters and target search step size parameters for the Kth search stage can be as follows: Obtain initial search range parameters, including initial horizontal search range parameters and initial vertical search range parameters. Determine the target horizontal search range parameters for the Kth search stage based on the initial horizontal search range parameters and the search stage number K. Determine the target vertical search range parameters for the Kth search stage based on the initial vertical search range parameters and the search stage number K. Define the target horizontal and vertical search range parameters for the Kth search stage as the target search range parameters for the Kth search stage. Obtain a first set value and a second set value. Define the first set value as the horizontal search step size parameter for the Kth search stage. Define the second set value as the vertical search step size parameter for the Kth search stage. Define the horizontal and vertical search step parameters for the Kth search stage as the target search step size parameters for the Kth search stage.

[0062] In one embodiment, the initial search range parameters include an initial horizontal search range parameter n and an initial vertical search range parameter m, with a first set value of 0 and a second set value of 0. Then, the target horizontal search range parameter for the Kth search stage is determined based on the initial horizontal search range parameter n and the number of search stages K. The target vertical search range parameter m is determined based on the initial vertical search range parameter m and the number of search stages K. in, To round x down, the target horizontal search range parameter and target vertical search range parameter of the Kth search stage are defined as the target search range parameter of the Kth search stage; the first set value of 0 is defined as the horizontal search step size parameter of the Kth search stage, the second set value of 0 is defined as the vertical search step size parameter of the Kth search stage, and the horizontal and vertical search step size parameters of the Kth search stage are defined as the target search step size parameter of the Kth search stage. Since the search step size of the Kth search stage is obtained by exponentiation of the target search step size parameter of the Kth search stage with the second target value, and in the exponentiation, the second target value is the base and the target search step size parameter is the exponent, the target search step size parameter of the Kth search stage includes the horizontal search step size parameter (which is 0) and the vertical search step size parameter (which is 0), therefore, the search step size of the Kth search stage is 1.

[0063] In the method provided in this application embodiment, the larger the initial search range parameter, the larger the target search range parameter in the I search stage, thus the initial search range parameter is inversely proportional to the processing speed of the processing module; the larger the number of search stages K, the more searches are performed in the reference image frame, and the larger the search step size is in each image patch search, thus the number of search stages K is directly proportional to the processing speed of the processing module. The search range parameter and search step size parameter are adjusted for each search stage to adapt the search step size to the search range, allowing for higher image patch search efficiency in each search stage and less increase in computational load for subsequent image patch search processing.

[0064] In this embodiment of the application, please refer to Figure 4The method for determining the I-th search image region in the reference image frame based on the target search range parameters and the position of the intermediate matching image block determined in the (I-1)th search stage in the reference image frame can be as follows: Obtain a first target value, and perform an exponentiation operation between the first target value and the target horizontal search range parameter in the target search range parameters of the I-th search stage, and determine the horizontal search range of the I-th search stage based on the exponentiation result; perform an exponentiation operation between the first target value and the target vertical search range parameter in the target search parameters of the I-th search stage, and determine the vertical search range of the I-th search stage based on the exponentiation result. The I-th search image region is determined from the reference image frame based on the position of the intermediate matching image block determined in the (I-1)th search stage, the horizontal search range, and the vertical search range of the I-th search stage. The method provided in this application, based on the intermediate matching image block determined in the (I-1)th search stage, determines the search image region of the I-th search stage, effectively reducing useless image block matching processes and improving the processing speed of the image processing module.

[0065] In this embodiment of the application, the first search image region contains multiple pixels, and the length of the first search image region is greater than or equal to the horizontal search step size of the first search stage, and the width of the first search image region is greater than or equal to the vertical search step size of the first search stage.

[0066] In one embodiment, the target horizontal search range parameter in the first search stage is A, the target vertical search range parameter is B, and the first target value is 2. Then, the first target value 2 and the target horizontal search range parameter A in the first search stage are exponentially combined to obtain an exponentiation result of 2. A Based on the result of the exponentiation operation, the horizontal search range for the I-th search stage is determined to be [-2]. A ,2 A The first target value 2 is exponentially multiplied by the target vertical search range parameter B in the I-th search stage, resulting in an exponentiation of 2. B And determine the vertical search range [-2] for the I-th search stage based on the result of the exponentiation operation. B ,2 B Using the center point of the intermediate matching image patch determined in the (I-1)th search phase as the center, and based on the horizontal search range determined in the (I-2)th search phase... A ,2 A ] and vertical search range [-2 B ,2 B We can determine a length of 2*2 A Width is 2*2 B The quadrilateral region is the I-th search image region on the reference image frame.

[0067] In this embodiment of the application, please refer to Figure 5 According to the search step size determined by the target search step size parameter, image block matching search is performed on the I-th search image region. The method to determine the intermediate matching image block of the image block to be processed can be as follows: obtain the second target value, and perform an exponentiation operation on the second target value and the horizontal search step size parameter in the target search step size parameter of the I-th search stage, and determine the horizontal search step size of the I-th search stage based on the exponentiation result; perform an exponentiation operation on the second target value and the vertical search step size parameter in the target search step size parameter of the I-th search stage, and determine the vertical search step size of the I-th search stage based on the exponentiation result. The horizontal search step size and vertical search step size of the I-th search stage are determined as the search step size of the I-th search stage. The starting search pixel point of the I-th search image region is determined, and according to the starting search pixel point, candidate search pixels are selected sequentially from the I-th search image region according to the horizontal search step size and vertical search step size of the I-th search stage; for the target search pixel point, the candidate image block corresponding to the target search pixel point is determined according to the size information of the image block to be processed. The method calculates the rate-distortion cost between the candidate image block and the image block to be processed. Here, the target search pixel is any pixel among the initial search pixel and multiple selected candidate search pixels. Based on the rate-distortion cost between the candidate image blocks corresponding to each target search pixel and the image block to be processed, intermediate matching image blocks are determined from the candidate image blocks corresponding to each target search pixel. The intermediate matching image block determined based on the rate-distortion cost is generally the image block within the current search image region that best matches the image block to be processed. The method provided in this application embodiment can quickly and effectively perform image block matching processing on the image search region of the reference image frame determined in the first search stage, and obtain the intermediate matching image block that best matches the image block to be processed in the current search stage based on the rate-distortion cost, reducing redundant computation and improving the matching degree between the matching image block and the image block to be processed.

[0068] In one embodiment, the process of determining the rate-distortion cost between a candidate image block and the image block to be processed can be as follows: Obtain the grayscale value of any pixel in the pixel block to be processed, and then calculate the absolute value of the grayscale difference between that pixel and the corresponding candidate pixel in the candidate image block; sum the absolute values ​​of the grayscale differences corresponding to all pixels in the pixel block to be processed to obtain the sum of absolute differences (SAD) of the image block to be processed; finally, determine the rate-distortion cost of the candidate image block based on the obtained SAD. The smaller the rate-distortion cost, the smaller the sum of absolute differences between the candidate image block and the image block to be processed, and the higher the matching degree between the candidate image block and the image block to be processed. Therefore, the candidate image block with the smallest rate-distortion cost is selected as the intermediate matching image block in the first search stage.

[0069] In one embodiment, the horizontal search step size parameter of the first search stage is C, the vertical search step size parameter is D, and the second target value is 2. Then, the second target value 2 and the horizontal search step size parameter C of the first search stage are exponentially calculated, and the horizontal search step size of the first search stage is determined to be 2 based on the exponentiation result. C The second target value 2 and the vertical search step size parameter D of the I-th search stage are exponentially calculated, and the vertical search step size of the I-th search stage is determined to be 2 based on the exponentiation result. D The starting search pixel of the I-th search image region is determined. Based on this starting search pixel, candidate search pixels are sequentially selected from the I-th search image region according to the horizontal and vertical search step sizes of the I-th search stage. Using the target search pixel as the center pixel of the candidate image blocks, the candidate image blocks corresponding to the target search pixel are determined based on the size information of the image block to be processed. Based on the rate-distortion cost between the candidate image blocks corresponding to each target search pixel and the image block to be processed, the intermediate matching image blocks of the image block to be processed are determined from the candidate image blocks corresponding to each target search pixel.

[0070] S204. The intermediate matching image block of the image block to be processed determined in the Kth search stage is determined as the target matching image block in the reference image frame that matches the image block to be processed.

[0071] In this embodiment, since the search range of the first search stage is determined based on the intermediate matching image block of the (I-1)th search stage, when the K search stages are completed, the intermediate matching image block of the Kth search stage is obtained. This intermediate matching image block is the image block that best matches the image block to be processed within the search range corresponding to all search stages. The intermediate matching image block determined in the Kth search stage is then identified as the target matching image block in the reference image frame that matches the image block to be processed.

[0072] Based on the above embodiments, the beneficial effects of this application are as follows: Using an interval search method to perform image block matching search on the reference image frame can significantly reduce the computational load compared to a full search algorithm; further, judging the matching conditions of the initial matching image block, and performing subsequent image processing only if the initial matching image block does not meet the matching conditions, can further reduce the computational load caused by useless image processing; determining the initial search range parameter and the number of search stages K according to the processing speed of the image processing module allows the processing speed of the image processing module to adapt to the application scenario; performing multi-stage search on a specific range of the reference image frame according to the initial search range parameter and the number of search stages K can better avoid the "local optima" problem that may occur in fast-converging motion estimation algorithms, and can also improve the matching degree between the matching image block and the image block to be processed; in summary, this method significantly improves the matching degree between the matching image block and the image block to be processed with a limited increase in computational load, thereby improving the matching efficiency of the matching image block. During video transmission, using this method for video frame compression can save data transmission bitrate and bandwidth.

[0073] Please see Figure 6 This figure is a schematic diagram of an image processing method provided in an embodiment of this application. The steps of the image processing method provided in this embodiment may include, but are not limited to:

[0074] S601. Use the interval search method to perform image block matching search on the reference image frame to obtain the initial matching image block of the image block to be processed.

[0075] In this embodiment, an interval search method is used to obtain an initial matching image block on the reference image frame, which is associated with the image block to be processed. The interval search method can be a fast-converging motion estimation algorithm. Fast-converging motion estimation algorithms mainly include three-step search algorithms, four-step search algorithms, and the Diamond Search algorithm, etc.

[0076] S602. Determine whether the matching error is greater than the error threshold. If the matching judgment result is yes, then proceed to step S603. If the matching judgment result is no, then determine the initial matching image block as the current matching image block and proceed to step S607.

[0077] In this embodiment, a matching error can be determined based on the initial matched image block and the image block to be processed. This matching error represents the degree of difference between the initial matched image block and the image block to be processed. The larger the matching error, the greater the difference between the initial matched image block and the image block to be processed. If the matching judgment result is yes, and the matching error is greater than the error threshold, it indicates that the difference between the initial matched image block and the image block to be processed is too large. In this case, an image block search operation is performed again on the reference image frame based on the initial matched image block. If the matching judgment result is no, and the matching error is less than or equal to the error threshold, it indicates that the initial matched image block and the image block to be processed are well matched. There is no need to perform an image block search operation again, and the initial matched image block is determined as the current matched image block. The method provided in this embodiment can make reasonable use of the computing resources of the image processing device and effectively improve the image processing efficiency.

[0078] S603. Obtain the search stage number K and determine that the current stage number I is 1.

[0079] In this embodiment, the initial search range parameter and the number of search stages K are determined according to the processing speed of the image processing device. The initial search range parameter is used to indicate the range of the search in the reference image frame, while the number of search stages K is used to indicate the number of times the initial matching image block is searched in the reference image frame. This allows the image block search of the reference image frame to be adjusted according to different actual application scenarios, thereby improving the adaptability of the image processing method.

[0080] S604. Determine the target search range parameters and target search step size parameters for the I search phase.

[0081] In this embodiment, I is any positive integer less than or equal to K; when I is 1, the target search range parameter of the first search stage is determined according to the initial search range parameter, and the target search step size parameter of the first search stage is determined according to the initial search range parameter and the number of search stages K; when I is greater than 1 and less than K, the target search range parameter of the first search stage is determined according to the target search step size parameter of the (I-1)th search stage, and the target search step size parameter of the first search stage is determined according to the initial search range parameter, the number of search stages K and I; when I is K, the target search range parameter of the Kth search stage is determined according to the initial search range parameter and the number of search stages K, and the target search step size parameter of the Kth search stage is determined according to the set value.

[0082] S605. Perform image block search on the reference image frame according to the target search range parameter and target search step size parameter of the I search stage to obtain the intermediate matching image block of the I search stage, and increment the current stage number I by 1.

[0083] In this embodiment, the first search image region is determined based on the target search range parameter of the first search stage, and the search step size of the first search stage is determined based on the target search step size parameter of the first search stage. Image block search is performed on the reference image frame based on the first search image region and the search step size of the first search stage to determine the intermediate matching image block related to the current image block. The current stage number I is incremented by 1 to represent the number of search stages that have been completed.

[0084] S606. Determine whether the current stage number I is greater than the search stage number K. If the stage determination result is yes, then determine the intermediate matching image block of the I search step as the current matching image block and execute step S607. If the determination result is no, then execute step S604.

[0085] In this embodiment of the application, it is determined whether the current stage number I is greater than the search stage number K. If the stage determination result is yes, the current stage number I is greater than the search stage number K, indicating that the image block search of K search stages has been completed and the search for the reference image frame can be ended. If the stage determination result is no, the current stage number I is less than or equal to the search stage number K, indicating that the number of search stages completed is less than K and the search for the reference image frame needs to continue.

[0086] S607. Determine the currently matched image block as the target matched image block of the image block to be processed.

[0087] In this embodiment of the application, the current matching image block may be determined based on the initial matching image block or based on the intermediate matching image block resolved by the I-th search.

[0088] Based on the above embodiments, the beneficial effects of this application are as follows: Initial matching image blocks are judged, and image block search is performed on the reference image frame only when the initial matching image block does not meet the conditions, thereby saving computational resources; when searching for image blocks on the reference image frame, a cyclic method is used, and the target search range parameter and target search step size parameter are adjusted each time the search is performed, so that the search range is adapted to the search step size. This improves the matching degree between intermediate matching image blocks and image blocks to be processed while minimizing the computational load of the image processing module, ultimately resulting in a significant improvement in the processing efficiency of the image processing module.

[0089] Please see Figure 7 , Figure 7 This is a schematic diagram of the structure of an image processing device provided in an embodiment of this application. The image processing device provided in this embodiment includes, but is not limited to:

[0090] The acquisition unit 701 is used to perform image block matching search on the reference image frame using an interval search method to obtain the initial matching image block of the image block to be processed, wherein the image block to be processed is an image block in the image frame to be processed, and the image frame to be processed is associated with the reference image frame.

[0091] The processing unit 702 is configured to obtain the number of search stages K if the initial matching image block does not meet the matching conditions, where K is an integer greater than 1, and the number of search stages K is used to indicate the number of times the search is performed in the reference image frame based on the initial matching image block.

[0092] The processing unit 702 is further configured to, for the first search stage, determine the target search range parameter and the target search step size parameter of the first search stage; determine the first search image region in the reference image frame based on the target search range parameter and the position of the intermediate matching image block determined in the (I-1)th search stage; and perform image block matching search on the first search image region according to the search step size determined based on the target search step size parameter to determine the intermediate matching image block of the image block to be processed; wherein, I is any positive integer less than or equal to K;

[0093] The processing unit 702 is further configured to determine the intermediate matching image block of the image block to be processed determined in the Kth search stage as the target matching image block in the reference image frame that matches the image block to be processed.

[0094] In one embodiment, when I is 1, the target search range parameter is determined based on the initial search range parameter, the target search step size parameter is determined based on the initial search range parameter and the number of search stages K, and the intermediate matching image block determined in the (I-1)th search stage is the initial matching image block; when I is greater than 1 and less than K, the target search range parameter is determined based on the target search step size parameter in the (I-1)th search stage, the target search step size parameter is determined based on the initial search range parameter, the number of search stages K, and I; when I is K, the target search range parameter is determined based on the initial search range parameter and the number of search stages K.

[0095] In one embodiment, the target search range parameters for the first search stage include the target horizontal search range parameters and the target vertical search range parameters for the first search stage. When the processing unit 702 determines the first search image region in the reference image frame based on the target search range parameters and the position of the intermediate matching image block determined in the (I-1)th search stage, it is specifically configured to: obtain a first target value; perform an exponentiation operation on the first target value and the target horizontal search range parameters for the first search stage, and determine the horizontal search range for the first search stage based on the exponentiation result; perform an exponentiation operation on the first target value and the target vertical search range parameters for the first search stage, and determine the vertical search range for the first search stage based on the exponentiation result; and determine the first search image region from the reference image frame based on the position of the intermediate matching image block determined in the (I-1)th search stage, the horizontal search range, and the vertical search range for the first search stage.

[0096] In one embodiment, the target search step size parameter of the first search stage includes a horizontal search step size parameter and a vertical search step size parameter of the first search stage. The processing unit 702 is further configured to: obtain a second target value; perform an exponentiation operation on the second target value and the horizontal search step size parameter of the first search stage, and determine the horizontal search step size of the first search stage based on the exponentiation result; perform an exponentiation operation on the second target value and the vertical search step size parameter of the first search stage, and determine the vertical search step size of the first search stage based on the exponentiation result; and determine the horizontal search step size and the vertical search step size of the first search stage as the search step size of the first search stage determined according to the target search step size parameter of the first search stage.

[0097] In one embodiment, when the processing unit 702 performs image block matching search for the I-th search image region according to a search step size determined based on the target search step size parameter, and determines the intermediate matching image block of the image block to be processed, it is specifically configured to: determine the starting search pixel in the I-th search image region; select candidate search pixels sequentially from the I-th search image region according to the starting search pixel and the horizontal and vertical search step sizes of the I-th search stage; for the target search pixel, determine the candidate image block corresponding to the target search pixel according to the size information of the image block to be processed, and calculate the rate-distortion cost between the candidate image block and the image block to be processed; the target search pixel is any pixel among the starting search pixel and the selected multiple candidate search pixels; and determine the intermediate matching image block of the image block to be processed from the candidate image blocks corresponding to each target search pixel according to the rate-distortion cost between the candidate image blocks corresponding to each target search pixel and the image block to be processed.

[0098] In one embodiment, the initial search range parameters include initial horizontal search range parameters and initial vertical search range parameters. When determining the target search range parameters and target search step size parameters for the first search stage, the processing unit 702 specifically performs the following: when I is 1, it determines the initial horizontal search range parameters as the target horizontal search range parameters for the first search stage, determines the initial vertical search range parameters as the target vertical search range parameters for the first search stage, and determines the target horizontal search range parameters and target vertical search range parameters for the first search stage as the target search range parameters for the first search stage; it determines the horizontal search step size parameters for the first search stage based on the initial horizontal search range parameters and the number of search stages K, determines the vertical search step size parameters for the first search stage based on the initial vertical search range parameters and the number of search stages K, and determines the horizontal search step size parameters and vertical search step size parameters for the first search stage as the target search step size parameters for the first search stage.

[0099] In one embodiment, the target search step size parameter of the (I-1)th search stage includes a horizontal search step size parameter and a vertical search step size parameter. When determining the target search range parameter and target search step size parameter of the I-1th search stage, the processing unit 702 specifically performs the following: when I is greater than 1 and less than K, it determines the horizontal search step size parameter of the (I-1)th search stage as the target horizontal search range parameter of the I-1th search stage, determines the vertical search step size parameter of the (I-1)th search stage as the target vertical search range parameter of the I-1th search stage, and determines the target horizontal search range parameter and target vertical search range parameter of the I-1th search stage as the target search range parameter of the I-1th search stage; it determines the horizontal search step size parameter of the I-1th search stage based on the initial horizontal search range parameter, the number of search stages K, and I; it determines the vertical search step size parameter of the I-1th search stage based on the initial vertical search range parameter, the number of search stages K, and I; and determines the horizontal search step size parameter and vertical search step size parameter of the I-1th search stage as the target search step size parameter of the I-1st search stage.

[0100] In one embodiment, the initial search range parameters include initial horizontal search range parameters and initial vertical search range parameters. When determining the target search range parameters and target search step size parameters for the I-th search stage, the processing unit 702 is specifically configured to: when I is K, determine the target horizontal search range parameters for the K-th search stage based on the initial horizontal search range parameters and the number of search stages K; determine the target vertical search range parameters for the K-th search stage based on the initial vertical search range parameters and the number of search stages K; and determine the target horizontal search range parameters and target vertical search range parameters for the K-th search stage as the target search range parameters for the K-th search stage; obtain a first set value and a second set value; determine the first set value as the horizontal search step size parameter for the K-th search stage; determine the second set value as the vertical search step size parameter for the K-th search stage; and determine the horizontal search step size parameters and vertical search step size parameters for the K-th search stage as the target search step size parameters for the K-th search stage.

[0101] In one embodiment, the processing unit 702 is further configured to: determine the matching error between the initial matching image block and the image block to be processed; if the matching error between the initial matching image block and the image block to be processed is greater than or equal to an error threshold, then determine that the initial matching image block does not meet the matching condition.

[0102] In one embodiment, the initial search range parameter and the number of search stages K are associated with the processing speed of the processing module, wherein the initial search range parameter is inversely proportional to the processing speed, and the number of search stages K is directly proportional to the processing speed.

[0103] It is understood that the functions of each functional unit of the image processing apparatus provided in the embodiments of this application can be specifically implemented according to the methods in the above-described image processing method embodiments. The specific implementation process can be referred to the relevant descriptions in the above-described image processing method embodiments, and will not be repeated here.

[0104] In this embodiment, an interval search method is used to search the reference image frame. Compared with the full search algorithm that traverses the reference frame, this method has a smaller computational load and higher search efficiency for matching blocks. When performing subsequent searches, image block matching search is performed on the search area centered on the initial matching image block, which can obtain matching image blocks with a higher matching degree than the initial image matching block. Since the subsequent search is only performed on a part of the reference image frame, the computational load added by the subsequent search is small, which also effectively ensures the search efficiency of matching blocks.

[0105] Please see Figure 8 , Figure 8 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. The computer device described in this embodiment includes: a processor 801, a memory 802, and a communication interface 803. The processor 801, memory 802, and communication interface 803 can be connected via a bus or other means; this embodiment uses a bus connection as an example.

[0106] The processor 801 (or CPU, Central Processing Unit) is the computing and control core of the computer device. It can parse various instructions and process various data within the computer device. For example, the CPU can parse power-on / off commands sent by input devices and control the computer device to perform power-on / off operations. It can also transmit various interactive data between internal structures of the computer device. The memory 802 is the storage device in the computer device, used to store programs and data. It can be understood that the memory 802 here includes both the computer device's built-in memory and extended memory supported by the computer device. The memory 802 provides storage space for the computer device's operating system, which may include, but is not limited to, Android, iOS, Windows Phone, etc. This application does not limit this. The communication interface 803 may optionally include standard wired interfaces and wireless interfaces (such as Wi-Fi, mobile communication interfaces, etc.), and is controlled by the processor 801 for sending and receiving data.

[0107] In this embodiment, the processor 801 performs the following operations by running the executable program code in the memory 802:

[0108] An interval search method is used to perform image block matching search on the reference image frame to obtain the initial matching image block of the image block to be processed. The image block to be processed is an image block in the image frame to be processed, and the image frame to be processed is associated with the reference image frame.

[0109] If the initial matching image block does not meet the matching conditions, then the number of search stages K is obtained, where K is an integer greater than 1. The number of search stages K is used to indicate the number of times the search is performed in the reference image frame based on the initial matching image block.

[0110] For the I-th search stage, the target search range parameter and target search step size parameter of the I-th search stage are determined. Based on the target search range parameter and the position of the intermediate matching image block determined in the (I-1)-th search stage in the reference image frame, the I-th search image region in the reference image frame is determined. According to the search step size determined according to the target search step size parameter, image block matching search is performed on the I-th search image region to determine the intermediate matching image block of the image block to be processed; where I is any positive integer less than or equal to K.

[0111] The intermediate matching image block of the image block to be processed determined in the Kth search phase is determined as the target matching image block in the reference image frame that matches the image block to be processed.

[0112] In one embodiment, when I is 1, the target search range parameter is determined based on the initial search range parameter, the target search step size parameter is determined based on the initial search range parameter and the number of search stages K, and the intermediate matching image block determined in the (I-1)th search stage is the initial matching image block; when I is greater than 1 and less than K, the target search range parameter is determined based on the target search step size parameter in the (I-1)th search stage, the target search step size parameter is determined based on the initial search range parameter, the number of search stages K, and I; when I is K, the target search range parameter is determined based on the initial search range parameter and the number of search stages K.

[0113] In one embodiment, the target search range parameters for the first search stage include the target horizontal search range parameters and the target vertical search range parameters for the first search stage. When the processor 801 determines the first search image region in the reference image frame based on the target search range parameters and the position of the intermediate matching image block determined in the (I-1)th search stage, it specifically performs the following operations: obtaining a first target value; performing an exponentiation operation on the first target value and the target horizontal search range parameters for the first search stage, and determining the horizontal search range for the first search stage based on the exponentiation result; performing an exponentiation operation on the first target value and the target vertical search range parameters for the first search stage, and determining the vertical search range for the first search stage based on the exponentiation result; and determining the first search image region from the reference image frame based on the position of the intermediate matching image block determined in the (I-1)th search stage, the horizontal search range, and the vertical search range for the first search stage.

[0114] In one embodiment, the target search step size parameter of the first search stage includes a horizontal search step size parameter and a vertical search step size parameter of the first search stage. The processor 801 is further configured to: obtain a second target value; perform an exponentiation operation on the second target value and the horizontal search step size parameter of the first search stage, and determine the horizontal search step size of the first search stage based on the exponentiation result; perform an exponentiation operation on the second target value and the vertical search step size parameter of the first search stage, and determine the vertical search step size of the first search stage based on the exponentiation result; and determine the horizontal search step size and the vertical search step size of the first search stage as the search step size of the first search stage determined according to the target search step size parameter of the first search stage.

[0115] In one embodiment, when the processor 801 performs image block matching search for the I-th search image region according to a search step size determined based on the target search step size parameter, and determines the intermediate matching image block of the image block to be processed, it specifically performs the following: determining the starting search pixel in the I-th search image region; selecting candidate search pixels sequentially from the I-th search image region according to the starting search pixel and the horizontal and vertical search step sizes of the I-th search stage; determining the candidate image block corresponding to the target search pixel according to the size information of the image block to be processed, and calculating the rate-distortion cost between the candidate image block and the image block to be processed; the target search pixel is any one of the starting search pixel and the selected multiple candidate search pixels; and determining the intermediate matching image block of the image block to be processed from the candidate image blocks corresponding to each target search pixel according to the rate-distortion cost between the candidate image blocks corresponding to each target search pixel and the image block to be processed.

[0116] In one embodiment, the initial search range parameters include initial horizontal search range parameters and initial vertical search range parameters. When determining the target search range parameters and target search step size parameters for the first search stage, the processor 801 specifically performs the following: when I is 1, it determines the initial horizontal search range parameters as the target horizontal search range parameters for the first search stage, determines the initial vertical search range parameters as the target vertical search range parameters for the first search stage, and determines the target horizontal search range parameters and target vertical search range parameters for the first search stage as the target search range parameters for the first search stage; it determines the horizontal search step size parameters for the first search stage based on the initial horizontal search range parameters and the number of search stages K, determines the vertical search step size parameters for the first search stage based on the initial vertical search range parameters and the number of search stages K, and determines the horizontal search step size parameters and vertical search step size parameters for the first search stage as the target search step size parameters for the first search stage.

[0117] In one embodiment, the target search step size parameter of the (I-1)th search stage includes a horizontal search step size parameter and a vertical search step size parameter. When determining the target search range parameter and target search step size parameter of the I-1th search stage, the processor 801 specifically performs the following: when I is greater than 1 and less than K, it determines the horizontal search step size parameter of the (I-1)th search stage as the target horizontal search range parameter of the I-1th search stage, determines the vertical search step size parameter of the (I-1)th search stage as the target vertical search range parameter of the I-1th search stage, and determines the target horizontal search range parameter and target vertical search range parameter of the I-1th search stage as the target search range parameter of the I-1th search stage; it determines the horizontal search step size parameter of the I-1th search stage based on the initial horizontal search range parameter, the number of search stages K, and I; it determines the vertical search step size parameter of the I-1th search stage based on the initial vertical search range parameter, the number of search stages K, and I; and determines the horizontal search step size parameter and vertical search step size parameter of the I-1th search stage as the target search step size parameter of the I-1st search stage.

[0118] In one embodiment, the initial search range parameters include initial horizontal search range parameters and initial vertical search range parameters. When determining the target search range parameters and target search step size parameters for the I-th search stage, the processor 801 specifically performs the following steps: when I is K, it determines the target horizontal search range parameters for the K-th search stage based on the initial horizontal search range parameters and the number of search stages K; it determines the target vertical search range parameters for the K-th search stage based on the initial vertical search range parameters and the number of search stages K; and it determines the target horizontal search range parameters and target vertical search range parameters for the K-th search stage as the target search range parameters for the K-th search stage. It then obtains a first set value and a second set value, determines the first set value as the horizontal search step size parameter for the K-th search stage, determines the second set value as the vertical search step size parameter for the K-th search stage, and finally determines the horizontal search step size parameters and vertical search step size parameters for the K-th search stage as the target search step size parameters for the K-th search stage.

[0119] In one embodiment, the processor 801 is further configured to: determine the matching error between the initial matching image block and the image block to be processed; if the matching error between the initial matching image block and the image block to be processed is greater than or equal to an error threshold, then determine that the initial matching image block does not meet the matching condition.

[0120] In one embodiment, the initial search range parameter and the number of search stages K are associated with the processing speed of the processing module, wherein the initial search range parameter is inversely proportional to the processing speed, and the number of search stages K is directly proportional to the processing speed.

[0121] In specific implementations, the processor 801, memory 802, and communication interface 803 described in the embodiments of this application can execute the implementation of the computer device described in the image processing method provided in the embodiments of this application, or the implementation of the image processing apparatus provided in the embodiments of this application, which will not be repeated here.

[0122] In this embodiment, an interval search method is used to search the reference image frame. Compared with the full search algorithm that traverses the reference frame, this method has a smaller computational load and higher search efficiency for matching blocks. When performing subsequent searches, image block matching search is performed on the search area centered on the initial matching image block, which can obtain matching image blocks with a higher matching degree than the initial image matching block. Since the subsequent search is only performed on a part of the reference image frame, the computational load added by the subsequent search is small, which also effectively ensures the search efficiency of matching blocks.

[0123] This application also provides a computer-readable storage medium storing a computer program that, when run on a computer, enables the computer to implement the image processing method described in this application. The specific implementation method can be found in the foregoing description and will not be repeated here.

[0124] This application also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the image processing method as described in this application. Specific implementation details are provided above and will not be repeated here.

[0125] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0126] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0127] The above-disclosed embodiments are only some of the embodiments of this application, and should not be construed as limiting the scope of this application. Therefore, any equivalent changes made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. An image processing method, characterized in that, The method includes: An interval search method is used to perform image block matching search on the reference image frame to obtain the initial matching image block of the image block to be processed. The image block to be processed is an image block in the image frame to be processed, and the image frame to be processed is associated with the reference image frame. If the initial matching image block meets the matching condition, then the initial matching image block is determined as the target matching image block in the reference image frame that matches the image block to be processed; If the initial matching image block does not meet the matching conditions, then the number of search stages K is obtained, where K is an integer greater than 1. The number of search stages K is used to indicate the number of times the search is performed in the reference image frame based on the initial matching image block. For the I-th search phase, the target search range parameter and target search step size parameter for the I-th search phase are determined. Based on the target search range parameter and the position of the intermediate matching image block determined in the (I-1)-th search phase in the reference image frame, the I-th search image region in the reference image frame is determined. Following the search step size determined according to the target search step size parameter, image block matching search is performed on the I-th search image region to determine the intermediate matching image block of the image block to be processed; where I is any positive integer less than or equal to K. When I is 1, the target search range parameter is the root... The target search step size parameter is determined based on the initial search range parameter and the number of search stages K. The intermediate matching image block determined in the (I-1)th search stage is the initial matching image block. When I is greater than 1 and less than K, the target search range parameter is determined based on the target search step size parameter in the (I-1)th search stage, which is determined based on the initial search range parameter, the number of search stages K, and I. When I is K, the target search range parameter is determined based on the initial search range parameter and the number of search stages K. The intermediate matching image block of the image block to be processed determined in the Kth search phase is determined as the target matching image block in the reference image frame that matches the image block to be processed.

2. The method according to claim 1, characterized in that, The target search range parameters for the first search phase include the target horizontal search range parameters and the target vertical search range parameters for the first search phase. The step of determining the I-th search image region in the reference image frame based on the target search range parameters and the position of the intermediate matching image block determined in the (I-1)th search stage includes: Obtain a first target value, perform an exponentiation operation on the first target value and the target horizontal search range parameter of the first search stage, and determine the horizontal search range of the first search stage based on the exponentiation result; The first target value and the target vertical search range parameter of the first search stage are exponentially operated on, and the vertical search range of the first search stage is determined based on the exponentiation result. Based on the position of the intermediate matching image block determined in the reference image frame in the I-1 search phase, and the horizontal and vertical search ranges of the I search phase, the I search image region is determined from the reference image frame.

3. The method according to claim 1, characterized in that, The target search step size parameter of the first search phase includes the horizontal search step size parameter and the vertical search step size parameter of the first search phase; the method further includes: Obtain the second target value, perform an exponentiation operation on the second target value and the horizontal search step size parameter of the first search stage, and determine the horizontal search step size of the first search stage based on the exponentiation result; The second target value and the vertical search step size parameter of the first search stage are exponentially operated on, and the vertical search step size of the first search stage is determined based on the exponentiation result. The horizontal and vertical search step sizes of the first search phase are determined as the search step size of the first search phase based on the target search step size parameter of the first search phase.

4. The method according to claim 3, characterized in that, The step of performing image block matching search for the I-th search image region according to the search step size determined based on the target search step size parameter, and determining the intermediate matching image block of the image block to be processed, includes: Determine the starting search pixel in the I-th search image region, and select candidate search pixels sequentially from the I-th search image region according to the starting search pixel and the horizontal and vertical search step sizes of the I-th search stage. For the target search pixel, based on the size information of the image block to be processed, a candidate image block corresponding to the target search pixel is determined, and the rate-distortion cost between the candidate image block and the image block to be processed is calculated; the target search pixel is any one of the starting search pixel and the selected multiple candidate search pixels. Based on the rate-distortion cost between the candidate image blocks corresponding to each target search pixel and the image block to be processed, an intermediate matching image block of the image block to be processed is determined from the candidate image blocks corresponding to each target search pixel.

5. The method according to claim 1, characterized in that, The initial search range parameters include initial horizontal search range parameters and initial vertical search range parameters; Determining the target search range parameters and target search step size parameters for the first search phase includes: When I is 1, the initial horizontal search range parameter is determined as the target horizontal search range parameter of the first search stage, the initial vertical search range parameter is determined as the target vertical search range parameter of the first search stage, and the target horizontal search range parameter and the target vertical search range parameter of the first search stage are determined as the target search range parameter of the first search stage. The horizontal search step size parameter of the first search stage is determined based on the initial horizontal search range parameter and the number of search stages K. The vertical search step size parameter of the first search stage is determined based on the initial vertical search range parameter and the number of search stages K. The horizontal search step size parameter and the vertical search step size parameter of the first search stage are determined as the target search step size parameter of the first search stage.

6. The method according to claim 1, characterized in that, The initial search range parameters include initial horizontal search range parameters and initial vertical search range parameters, and the target search step size parameters of the (I-1)th search stage include horizontal search step size parameters and vertical search step size parameters. Determining the target search range parameters and target search step size parameters for the first search phase includes: When I is greater than 1 and less than K, the horizontal search step size parameter of the (I-1)th search stage is determined as the target horizontal search range parameter of the I search stage, the vertical search step size parameter of the (I-1)th search stage is determined as the target vertical search range parameter of the I search stage, and the target horizontal search range parameter and the target vertical search range parameter of the I search stage are determined as the target search range parameter of the I search stage. The horizontal search step size parameter of the I-th search stage is determined based on the initial horizontal search range parameter, the number of search stages K and I, and the vertical search step size parameter of the I-th search stage is determined based on the initial vertical search range parameter, the number of search stages K and I, and the horizontal search step size parameter and the vertical search step size parameter of the I-th search stage are determined as the target search step size parameter of the I-th search stage.

7. The method according to claim 1, characterized in that, The initial search range parameters include initial horizontal search range parameters and initial vertical search range parameters; Determining the target search range parameters and target search step size parameters for the first search phase includes: When I is K, the target horizontal search range parameter of the Kth search stage is determined according to the initial horizontal search range parameter and the number of search stages K, and the target vertical search range parameter of the Kth search stage is determined according to the initial vertical search range parameter and the number of search stages K. The target horizontal search range parameter and the target vertical search range parameter of the Kth search stage are determined as the target search range parameter of the Kth search stage. Obtain a first set value and a second set value, determine the first set value as the horizontal search step size parameter of the Kth search stage, determine the second set value as the vertical search step size parameter of the Kth search stage, and determine the horizontal search step size parameter and the vertical search step size parameter of the Kth search stage as the target search step size parameter of the Kth search stage.

8. The method according to any one of claims 1-7, characterized in that, The method further includes: Determine the matching error between the initial matching image block and the image block to be processed; If the matching error between the initial matching image block and the image block to be processed is greater than or equal to the error threshold, then it is determined that the initial matching image block does not meet the matching condition.

9. The method according to claim 1, characterized in that, The initial search range parameter and the number of search stages K are related to the processing speed of the processing module. The initial search range parameter is inversely proportional to the processing speed, and the number of search stages K is directly proportional to the processing speed.

10. An image processing apparatus, characterized in that, include: The acquisition unit is used to perform image block matching search on the reference image frame using an interval search method to obtain the initial matching image block of the image block to be processed, wherein the image block to be processed is an image block in the image frame to be processed, and the image frame to be processed is associated with the reference image frame. The processing unit is configured to determine the initial matching image block as the target matching image block in the reference image frame that matches the image block to be processed if the initial matching image block meets the matching condition; and to obtain the number of search stages K, where K is an integer greater than 1, if the initial matching image block does not meet the matching condition. The number of search stages K is used to indicate the number of times the search is performed in the reference image frame based on the initial matching image block. The processing unit is further configured to, for the I-th search stage, determine the target search range parameter and the target search step size parameter of the I-th search stage; determine the I-th search image region in the reference image frame based on the target search range parameter and the position of the intermediate matching image block determined in the (I-1)-th search stage; and perform image block matching search on the I-th search image region according to the search step size determined based on the target search step size parameter to determine the intermediate matching image block of the image block to be processed; wherein, I is any positive integer less than or equal to K, and when I is 1, the target search range... The search range parameter is determined based on the initial search range parameter, and the target search step size parameter is determined based on the initial search range parameter and the number of search stages K. The intermediate matching image block determined in the (I-1)th search stage is the initial matching image block. When I is greater than 1 and less than K, the target search range parameter is determined based on the target search step size parameter in the (I-1)th search stage, and the target search step size parameter is determined based on the initial search range parameter, the number of search stages K, and I. When I is K, the target search range parameter is determined based on the initial search range parameter and the number of search stages K. The processing unit is further configured to determine the intermediate matching image block of the image block to be processed determined in the Kth search stage as the target matching image block in the reference image frame that matches the image block to be processed.

11. A computer device, characterized in that, include: The system includes a processor, a memory, and a communication interface, which are interconnected. The memory stores executable program code, and the processor is used to call the executable program code to implement the image processing method as described in any one of claims 1-9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed on a computer, cause the computer to implement the image processing method as described in any one of claims 1-9.

13. A computer program product, characterized in that, The computer program product includes a computer program or computer instructions, which, when executed by a processor, implement the image processing method as described in any one of claims 1-9.

Citation Information

Patent Citations

  • Image coding method, electronic equipment and storage medium

    CN115118981A