Image processing method, device and computer equipment

By using the verification motion vector of the verification image block in video encoding to verify the initial motion vector, the problem of high complexity of the motion estimation method in the prior art is solved, and the efficiency and accuracy of the video encoding are improved.

CN117750016BActive Publication Date: 2024-11-19SHUXING TECH (BEIJING) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202311744977.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-12-18
Publication Date
2024-11-19
Estimated Expiration
2043-12-18

AI Technical Summary

Technical Problem

The existing motion estimation methods have high algorithm complexity and large data calculations in video encoding, resulting in low video encoding efficiency.

Method used

By determining multiple verification image blocks from video image frames, the initial motion vector is verified by using verification motion vectors, and the target motion vector is determined based on the verification results, reducing the algorithm complexity and data calculation amount.

Benefits of technology

It effectively reduces the complexity of the algorithm, reduces the amount of data calculation, improves the video encoding efficiency, and shortens the determination time of the target motion vector.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117750016B_ABST
    Figure CN117750016B_ABST
Patent Text Reader

Abstract

The embodiment of the present application provides an image processing method, apparatus and computer equipment, wherein the method includes: determining multiple verification image blocks of the current image block from a video image frame, each verification image block having the same image block size; verifying the initial motion vector of the current image block according to the verification motion vectors of the multiple verification image blocks to obtain a verification result; wherein the initial motion vector of the current image block is obtained by processing the reference image frame of the video image frame using a first image search method, and the verification motion vector of the verification image block is determined according to the target motion vector of the encoded image area in the video image frame; using the verification result to determine the target motion vector of the current image block according to the initial motion vector or the second image search method. The method provided by the embodiment of the present application can determine the target motion vector according to the verification result of the initial motion vector, thereby effectively reducing the algorithm complexity and improving the video coding efficiency.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to image processing methods, devices and computer equipment. Background Art

[0002] With the development of computer technology, more and more videos are transmitted on the network. In the process of video transmission, in order to save network resources, the video is usually encoded. In the process of video encoding, in order to reduce the amount of data, motion estimation technology is usually used; motion estimation technology refers to the technology of estimating the motion displacement between two frames of images. Motion estimation technology can effectively remove the redundancy between video frames, so that the amount of data transmitted by the video is greatly reduced.

[0003] The commonly used motion estimation method is to first search the reference image frame using an integer pixel precision search method, and then search the reference image frame using a sub-pixel precision search method to obtain the motion vector of the image block. However, this motion estimation method has high algorithm complexity and large amount of data calculation, which easily leads to low video coding efficiency. Summary of the invention

[0004] The embodiments of the present application provide an image processing method, apparatus, and computer device, which can determine a target motion vector based on a verification result of an initial motion vector, thereby effectively reducing algorithm complexity and improving video encoding efficiency.

[0005] On the one hand, an embodiment of the present application provides an image processing method, the method comprising:

[0006] Determine a plurality of verification image blocks of the current image block from the video image frame, wherein each of the verification image blocks has the same image block size;

[0007] Verifying the initial motion vector of the current image block according to the verification motion vectors of the multiple verification image blocks to obtain a verification result; wherein the initial motion vector of the current image block is obtained by processing the reference image frame of the video image frame using a first image search method, and the verification motion vector of the verification image block is determined according to the target motion vector of the encoded image area in the video image frame;

[0008] The target motion vector of the current image block is determined using the verification result according to the initial motion vector or a second image search method.

[0009] On the one hand, an embodiment of the present application provides an image processing device, the device comprising:

[0010] A determination unit, configured to determine a plurality of verification image blocks of a current image block from a video image frame, wherein each of the verification image blocks has the same image block size;

[0011] a processing unit, configured to perform verification processing on the initial motion vector of the current image block according to the verification motion vectors of the multiple verification image blocks to obtain a verification result; wherein the initial motion vector of the current image block is obtained by processing the reference image frame of the video image frame using a first image search method, and the verification motion vector of the verification image block is determined according to the target motion vector of the encoded image area in the video image frame;

[0012] The determination unit is further configured to determine the target motion vector of the current image block using the verification result according to the initial motion vector or a second image search method.

[0013] On the one hand, an embodiment of the present application provides a computer device, comprising: a processor, a communication interface and a memory, wherein the processor, the communication interface and the memory are interconnected, wherein the memory stores an executable program code, and the processor is used to call the executable program code to implement the image processing method provided in the embodiment of the present application.

[0014] Accordingly, an embodiment of the present application further provides a computer-readable storage medium, in which instructions are stored. When the computer-readable storage medium is executed on a computer, the computer implements the image processing method provided by the embodiment of the present application.

[0015] Accordingly, the embodiment of the present application further provides a computer program product, the computer program product includes a computer program or a computer instruction, and the computer program or the computer instruction is stored in a computer-readable storage medium. The processor of the computer device reads the computer program or the computer instruction from the computer-readable storage medium, and the processor executes the computer program or the computer instruction, so that the computer device implements the image processing method provided in the embodiment of the present application.

[0016] The image processing method provided by the present application can determine multiple verification image blocks of the current image block from the video image frame, and use the verification motion vectors of the multiple verification image blocks to verify the initial motion vector of the current image block; the target motion vector of the current image block can be determined according to the initial motion vector or the second image search method based on the verification result. Through the image processing method provided by the embodiment of the present application, multiple verification image blocks corresponding to the current image block can be determined, which is convenient for verifying the initial motion vector using the verification image block; the verification motion vector of the initial motion vector of the current image block can be verified using the verification motion vectors of the multiple verification image blocks, so as to determine whether the initial motion vector meets the requirements; the verification motion vector can be determined according to the target motion vector of the encoded image area in the video image frame, reducing the computational complexity and improving the overall execution efficiency of the image processing method; the target motion vector can be determined according to the initial motion vector or the second image search method based on the verification result. When the target motion vector is determined according to the initial motion vector, it is not necessary to use the second image search method for search processing, thereby effectively reducing the complexity of the algorithm and reducing the amount of data calculation. At the same time, it can also effectively shorten the time for determining the target motion vector, thereby improving the processing efficiency of video encoding. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 is a schematic diagram of the system architecture of an image processing system provided in an embodiment of the present application;

[0019] Figure 2 It is a flowchart of an image processing method provided in an embodiment of the present application;

[0020] Figure 3 is a flowchart of another image processing method provided in an embodiment of the present application;

[0021] Figure 4 is a schematic diagram of a verification image block provided in an embodiment of the present application;

[0022] Figure 5 is a schematic diagram of a matching image block provided in an embodiment of the present application;

[0023] Figure 6 is a schematic diagram of an image processing method provided in an embodiment of the present application;

[0024] Figure 7 is a structural block diagram of an image processing device provided in an embodiment of the present application;

[0025] Figure 8 It is a structural block diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0027] It should be noted that the descriptions of "first", "second", etc. involved in the embodiments of the present application are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the technical features defined as "first" or "second" may explicitly or implicitly include at least one of the features.

[0028] When performing video encoding, motion estimation technology is usually used to remove redundancy between video frames, which greatly reduces the amount of data transmitted by the video. The commonly used motion estimation method is to first search the reference image frame using an integer pixel precision search method, and then search the reference image frame using a sub-pixel precision search method to obtain the motion vector of the current image block. However, this method of determining the motion vector has a high algorithm complexity, a large amount of data calculation, and is likely to lead to low video encoding efficiency.

[0029] Based on this, an embodiment of the present application provides an image processing method, which can determine multiple verification image blocks of the current image block from a video image frame, and each verification image block has the same image block size; the initial motion vector of the current image block is verified according to the verification motion vectors of the multiple verification image blocks to obtain a verification result; wherein the initial motion vector of the current image block is obtained by processing the reference image frame of the video image frame using a first image search method, and the verification motion vector of the verification image block is determined according to the target motion vector of the encoded image area in the video image frame; the target motion vector of the current image block is determined according to the initial motion vector or the second image search method using the verification result. Through the method provided by the embodiment of the present application, the target motion vector can be determined according to the initial motion vector or the second image search method using the verification result, which can effectively reduce the algorithm complexity, reduce the amount of data calculation, and improve the video coding efficiency.

[0030] The image processing method provided in the embodiment of the present application can also be applied to the field of cloud computing. Cloud computing is a computing mode that distributes computing tasks on a resource pool composed of a large number of computers, so that various application systems can obtain computing power, storage space and information services as needed. The cloud computing resource pool mainly includes: computing devices (virtualized machines, including operating systems), storage devices, and network devices. The image processing method provided in the embodiment of the present application can be implemented by a cloud computing device: when performing video encoding, the video to be encoded can be sent to the cloud computing device. For the video image frames in the video to be encoded, the cloud computing device can determine multiple verification image blocks of the current image block from the video image frames, and use the verification motion vectors of the multiple verification image blocks to verify the initial motion vector of the current image block to obtain a verification result, and use the verification result to determine the target motion vector of the current image block according to the initial motion vector or the second image search method, thereby effectively reducing the complexity of the algorithm and improving the efficiency of video encoding.

[0031] The architecture of the image processing system provided in the embodiments of the present application will be introduced below in conjunction with the accompanying drawings.

[0032] See also Figure 1 , which is a schematic diagram of the system architecture of an image processing system provided by an embodiment of the present application, the image processing system includes a terminal device 101, an image processing server 102 and a database 103, and the image processing server 102 can exchange data with the terminal device 101 and the database 103. Among them:

[0033] The terminal device 101 can generate and send a video acquisition request, and can also receive encoded video data, and decode the encoded video data to obtain the video to be encoded, and display the video to be encoded. The terminal device 101 can be a handheld device (such as a smart phone, a tablet computer) with video decoding function and communication function, a computing device (such as a personal computer (Personal Computer, PC), a vehicle terminal, an intelligent voice interaction device, a wearable device or other intelligent device, but is not limited thereto.

[0034] The image processing server 102 can receive a video acquisition request sent by the terminal device 101, determine the video to be encoded according to the video acquisition request, and use the image processing method provided by the present application to determine the target motion vector of the image block in each video image frame, thereby obtaining the encoded video data corresponding to the video to be encoded; the encoded video data can be sent to the terminal device 101; the image processing server 102 can also obtain the search method related data of the image search method from the database 103. The image processing server 102 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0035] The database 103 is used to store search method related data of the first image search method and the second image search method, for example: search parameters of the first image search method, similarity measurement data, etc. The database 103 can be a local database in the image processing server 102, or a cloud database associated with the image processing server 102 (i.e., a database deployed in the cloud). Specifically, it can be deployed based on any one of private cloud, public cloud, hybrid cloud, edge cloud, etc., so that the cloud database focuses on different functions. For example, the database deployed in the private cloud has the user's personal equipment as the basic cloud hardware, and focuses more on serving a small number of users, while the database deployed in the public cloud is deployed based on the cloud platform provided by a third party, which allows the data stored in the database to achieve data sharing, and any user's data can be stored in the database, and any user can also use the data in the database.

[0036] The following will explain in detail Figure 1 The working principle of the image processing system shown:

[0037] The terminal device 101 can send a video acquisition request to the image processing server 102, and the image processing server 102 can determine the video to be encoded according to the received video acquisition request. The image processing server 102 can obtain the search method related data of the first image search method and the second image search method from the database 103. For any video image frame in the video to be encoded, the image processing server 102 can process the reference image frame of the video image frame using the first image search method according to the related data of the first image search method to obtain the initial motion vector of the current image block; the image processing server 102 can determine multiple verification image blocks of the current image block from the video image frame, and the image block size of each verification image block is the same; the image processing server 102 can use the verification motion vectors of the multiple verification image blocks to verify the initial motion vector of the current image block to obtain a verification result; if the verification result indicates that the verification is passed, the target motion vector can be directly determined according to the initial motion vector, and if the verification result indicates that the verification is not passed, the target motion vector can be determined according to the second image search method, thereby realizing the video encoding processing of the video to be encoded and obtaining the encoded video data. The image processing server 102 sends the encoded video data to the terminal device 101, and the terminal device 101 can perform video decoding processing on the encoded video data to obtain the video to be encoded. Through the image processing method provided in the embodiment of the present application, the initial motion vector can be verified and the target motion vector can be determined according to the verification result, which effectively reduces the calculation complexity of the algorithm, reduces the amount of data calculation, and improves the efficiency of video encoding.

[0038] It is to be understood that the schematic diagram of the image processing system described in the embodiment of the present application is to more clearly illustrate the image processing method of the embodiment of the present application, and does not constitute a limitation on the image processing method provided by the embodiment of the present application. For example, the image processing method provided by the embodiment of the present application can be executed by the image processing server 102, and can also be executed by other devices that are different from the image processing server 102 and can communicate with the terminal device 101 and the database 103. It is known to those skilled in the art that Figure 1 The number of terminal devices 101, image processing servers 102, and databases 103 is merely illustrative. Any number of devices may be configured according to business implementation requirements. Moreover, with the evolution of system architecture and the emergence of new business scenarios, the image processing method provided in the embodiment of the present application is also applicable to similar technical problems.

[0039] It should be noted that the collection and processing of relevant data (for example, video image frames, reference image frames) in this application should be strictly in accordance with the requirements of relevant laws and regulations when applied in examples, and the informed consent or separate consent of the personal information subject should be obtained, and subsequent data use and processing should be carried out within the scope of authorization of laws and regulations and the personal information subject.

[0040] See also Figure 2 , Figure 2 A schematic diagram of a flow chart of an image processing method provided in an embodiment of the present application. The image processing method can be implemented by the above-mentioned image processing server 102, or by other devices. The flow chart of the image processing method provided in an embodiment of the present application includes but is not limited to:

[0041] S201. Determine a plurality of verification image blocks of a current image block from a video image frame, wherein each of the verification image blocks has the same image block size.

[0042] In an embodiment of the present application, a video image frame is included in a video to be encoded, and the video to be encoded includes multiple video image frames. When performing video encoding processing on the video to be encoded, in order to reduce the amount of data, a motion estimation technology can be used to calculate the motion vector corresponding to each video image frame. When calculating the motion vector corresponding to the video image frame, the current image block can be determined from the video image frame, and the current image block can include one or more pixels. After determining the current image block, multiple verification image blocks can be determined based on the current image block, and the image block size of each verification image block is the same, and each verification image block can include one or more pixels, and the position of the verification image block in the video image frame is associated with the position of the current image block in the video image frame. Through the method provided in the embodiment of the present application, multiple verification image blocks can be determined, so that the initial motion vector of the current image block can be verified according to the verification motion vectors of multiple verification image blocks, and an accurate verification result can be obtained.

[0043] It should be noted that there is a verification image block among the multiple verification image blocks that partially overlaps with the current image block, that is, the pixel points of the verification image block are included in the current image block, but not in the encoded image area in the video encoding image frame; then the image area where the multiple verification image blocks are located in the video image frame can be the target image area, and the target image area is the image area corresponding to the encoded image area in the video image frame and the current image block. Determining multiple verification image blocks of the current image block from the target image area is conducive to the subsequent rapid determination of the verification motion vectors of each verification image block, which can effectively reduce the complexity of the algorithm and improve the overall efficiency of the image processing method.

[0044] S202. Verify the initial motion vector of the current image block according to the verification motion vectors of the multiple verification image blocks to obtain a verification result; wherein the initial motion vector of the current image block is obtained by processing the reference image frame of the video image frame using a first image search method, and the verification motion vector of the verification image block is determined based on the target motion vector of the encoded image area in the video image frame.

[0045] In the embodiment of the present application, the motion trend of the pixels around the current image block in the video image frame should be roughly the same as the motion trend of the current image block, that is, the motion vector of the pixels around the current image block is roughly the same as the motion vector of the current image block. Therefore, multiple verification image blocks located around the current image block can be determined, and the initial motion vector of the current image block can be verified according to the verification motion vectors of each verification image block to obtain a verification result, which can indicate to a certain extent whether the initial motion vector meets the requirements. The initial motion vector of the current image block can be obtained by processing the reference image frame of the video image frame using the first image search method. The video image frame and the reference image frame are both included in the video to be encoded, and the video image frame is associated with the reference image frame. For example, the reference image frame can be the previous image frame of the video image frame in the video to be encoded. The motion vector of each pixel in the encoded image area can be determined according to the target motion vector of the encoded image area in the video image frame; the multiple verification image blocks of the current image block are mainly located in the encoded image area in the video image frame, and the motion vector of each pixel in the encoded image area is known, then the verification motion vector of each verification image block can be directly determined according to the motion vector of each pixel in the encoded image area. Through the method provided in the embodiment of the present application, the verification processing of the initial motion vector can be implemented to obtain accurate verification results, which is conducive to the subsequent use of the verification results to determine the target motion vector. At the same time, the verification motion vector can be determined based on the target motion vector of the encoded image area, effectively ensuring the accuracy of the verification processing and the processing efficiency of the image processing method.

[0046] It should be noted that, among the multiple verification image blocks of the current image block, there is a verification image block that partially overlaps with the current image block, that is, the pixel points of the verification image block are included in the current image block but not in the encoded image area in the video image frame. At this time, the motion vector of each pixel point can be determined based on the initial motion vector of the current image block, and then the verification motion vector of the verification image block can be determined.

[0047] In one embodiment, after determining the current image block from the video image frame, the image processing method provided by the present application can also: determine the image search range from the reference image frame according to the first image search method, and perform search processing on the image search range to obtain multiple image blocks to be selected, and the image block size of the image blocks to be selected matches the image block size of the current image block; determine the image difference data between the current image block and each of the image blocks to be selected, and the image difference data includes one or more of relative difference data, conversion difference data, and difference square sum data; use the image difference data of multiple image blocks to be selected to determine the reference image block from the multiple image blocks to be selected, and perform vector calculation on the reference image block and the current image block to obtain the initial motion vector of the current image block. After determining the current image block from the video image frame, the first image search method can be used to determine the initial motion vector of the current image block: the image search range can be determined from the reference image frame according to the first image search method, and the image search range is larger than the image range corresponding to the current image block; the image search access can be searched and processed to obtain multiple image blocks to be selected, and the image block size of each image block to be selected is the same as the image block size of the current image block; the pixel values ​​of each image block to be selected can be compared with the current image block to determine the image difference data between the current image block and each image block to be selected, and the image difference data can include one or more of relative difference data, conversion difference data, and difference square sum data, wherein the relative difference data, the conversion difference data, and the difference square sum data are all similarity measurement data.

[0048] The relative difference data, namely the sum of absolute differences (SAD), can be determined based on the absolute difference between the pixel values ​​of each pixel point in the selected pixel block and the pixel value of the corresponding pixel point in the current pixel block. The sum of absolute differences can reflect the residual time domain difference between the selected image block and the current image block to a certain extent, and the calculation complexity is low. The transformed difference data, namely the sum of absolute transformed differences (SATD), is the absolute sum of coefficients after the residual is subjected to Hadamard transformation. In most cases, SATD has a better evaluation effect than SAD, but the calculation complexity of SATD is greater than that of SAD. The sum of squared difference data, namely the sum of squared differences (SSD), can be determined based on the square difference between the pixel values ​​of each pixel point in the selected image block and the pixel value of the corresponding pixel point in the current pixel block. The sum of squared differences can better reflect the difference between the two image blocks.

[0049] The image difference data of multiple image blocks to be selected can be used to determine a reference image block from the multiple image blocks to be selected, and the reference image block is the image block with the smallest difference from the current image block among the multiple image blocks to be selected. Vector calculation can be performed on the reference image block and the current image block (for example: vector calculation is performed using the position data of each pixel point of the reference image block and the position data of the corresponding pixel point in the current image block) to obtain the initial motion vector of the current image block. Through the method provided in the embodiment of the present application, multiple different difference data can be used to determine the reference image block, which ensures the similarity between the reference image block and the current image block, and can improve the accuracy of the initial motion vector to a certain extent, thereby helping to reduce the computational complexity and improve the video coding efficiency.

[0050] It should be noted that the image difference data between the current image block and each of the candidate image blocks may also include other types of similarity measurement data, such as mean absolute difference (MAD), mean squared difference (MSD), etc. The similarity measurement data included in the image difference data may also be different depending on the application requirements of video coding.

[0051] S203: Determine the target motion vector of the current image block using the verification result according to the initial motion vector or a second image search method.

[0052] In an embodiment of the present application, the verification result can be used to determine the target motion vector of the current image block according to the initial motion vector or the second image search method, and the target motion vector can be used to implement video encoding processing of the video to be encoded. When the target motion vector of the current image block is determined according to the initial motion vector, there is no need to process the reference image frame using the second image search method, which can effectively reduce the amount of calculation and improve processing efficiency. When the target motion vector of the current image block is determined according to the second image search method, the accuracy of the target motion vector can be effectively guaranteed. For multiple video image frames in the video to be encoded, the method provided by the present application can be used to determine the target motion vector, thereby reducing the amount of calculation as a whole, reducing the computational complexity, and improving the efficiency of video encoding, thereby saving network resources.

[0053] In one embodiment, the method for implementing the target motion vector of the current image block by using the verification result according to the initial motion vector or the second image search method can be: if the verification result indicates that the verification is passed, the initial motion vector is determined as the target motion vector of the current image block; if the verification result indicates that the verification is not passed, the reference image frame is processed using the second image search method to obtain the target motion vector of the current image block. If the verification result indicates that the verification is passed, it means that the initial motion vector meets the requirements, and there is no need to perform image search processing, and the initial motion vector can be directly determined as the target motion vector of the current image block; if the verification result indicates that the verification is not passed, it means that the initial motion vector does not meet the requirements, and it is necessary to use the second image search method to perform more refined processing on the reference image frame to obtain the target motion vector of the current image block. The method provided by the embodiment of the present application can effectively reduce the computational complexity, reduce the amount of data calculation, and improve the processing efficiency of video encoding.

[0054] In one embodiment, the first image search method may be an integer pixel search method, and the second image search method may be a sub-pixel search method. The existing method for determining the motion vector is: for the current image block in the video image frame, the integer pixel search method is used to determine the initial motion vector of the current image block, and then the sub-pixel search method is used to determine the target motion vector of the current image block. However, the sub-pixel search method has a high algorithm complexity and a large amount of data calculation. In some cases (for example, the resolution of the video to be encoded is low and the video image frame is blurry), the sub-pixel search method may cause errors due to overly detailed search. In the method provided in the present application, an integer pixel search method can be used to determine the initial motion vector of the current image block, and multiple verification image blocks of the current image block can be determined. The verification motion vectors of the multiple verification image blocks are used to verify the initial motion vector to obtain a verification result. If the verification result indicates that the verification is passed, the initial motion vector is directly determined as the target motion vector, which is equivalent to terminating the target motion vector determination process in advance. There is no need to perform a pixel-by-pixel search method, thereby effectively reducing the amount of data calculation, reducing the calculation complexity, and effectively avoiding errors caused by overly detailed searches, thereby ensuring the accuracy of motion vectors and improving the video encoding effect.

[0055] It should be noted that the image processing method provided in the embodiment of the present application can be applied to encoders based on different coding standards (for example: H.264, H.265 and other coding standards), so that when the encoder encodes the video, it can use the method provided in the embodiment of the present application to quickly and accurately determine the optimal motion vector, thereby effectively improving the efficiency of video encoding. For example: in the mode decision process of the encoder, it is necessary to determine the optimal motion vector of each image block in the image frame. After the image processing method provided in the embodiment of the present application is applied to the encoder, the encoder can first use the integer pixel search method to determine the initial motion vector of the image block, and then determine the multiple verification image blocks corresponding to the image block, and use the verification motion vectors of each verification image block to verify the initial motion vector. If the verification passes, the initial motion vector is determined as the optimal motion vector, otherwise the sub-pixel search method is used to determine the optimal motion vector of the image block. This method effectively reduces the computational complexity and reduces the amount of data calculation.

[0056] In addition to being applied to the motion estimation process in the mode decision of the encoder, the image processing method provided in the embodiment of the present application can also be applied to the motion estimation process in the process of pre-analysis, motion compensation time domain filtering, etc. The pre-analysis technology can eliminate the redundancy of adjacent pixels in space and time in the video, and the image processing method provided in the embodiment of the present application can be applied to the motion estimation process of the pre-analysis calculation, and the verification result of the initial motion vector is determined by the verification motion vector of each verification image block, and the best motion vector and the best matching image block are determined according to the verification result, which is conducive to improving the processing efficiency of the pre-analysis.

[0057] Motion compensated time domain filtering, also known as time domain filtering technology based on motion compensation, can eliminate noise in the video, improve the subjective feeling when watching the video, and also improve the compression effect of the video. Motion compensated time domain filtering technology usually performs weighted averaging of the current video frame and the temporally adjacent video frame (i.e., the reference video frame) to smooth the noise and achieve the effect of video denoising. When performing weighted averaging, it is necessary to use motion estimation and motion compensation technology to weighted average the best matching image block with the current image block, thereby achieving a noise reduction effect in the spatial domain and improving video coding efficiency. When determining the best matching image block, the image processing method provided in the embodiment of the present application can be used to determine multiple verification image blocks corresponding to the current image block, and the verification motion vector of each verification image block is used to verify the initial motion vector to obtain a verification result, and the target motion vector is determined according to the verification result. The matching image block corresponding to the target motion vector is the best matching image block. The method provided in the embodiment of the present application can effectively reduce the amount of data calculation, reduce the computational complexity, and improve the processing efficiency of video denoising. The image processing method provided in the embodiments of the present application is also applicable to other application scenarios that require determining the best motion vector or the best matching image block.

[0058] Through the image processing method provided in the embodiment of the present application, multiple verification image blocks of the current image block can be reasonably determined, so as to facilitate determining accurate verification results based on the multiple verification image blocks; the verification motion vector of the verification image block can be determined based on the target motion vector of the encoded image area in the video image frame, which effectively reduces the calculation complexity and improves the efficiency and accuracy of the image processing method; the verification motion vectors of the multiple verification image blocks can be used to verify the initial motion vector to obtain a verification result, thereby determining whether the initial motion vector meets the requirements; when the target motion vector is determined based on the initial motion vector according to the verification result, the target motion vector determination process can be terminated early, thereby effectively reducing the amount of data calculation, reducing the calculation complexity, and effectively avoiding errors caused by overly detailed searches, ensuring the accuracy of the motion vector, improving the effect of video encoding, and improving the processing efficiency of video encoding.

[0059] See also Figure 3 , Figure 3 A flowchart of another image processing method provided as an example in an embodiment of the present application. The image processing method can be implemented by the above-mentioned image processing server 102, or by other devices. The process of the image processing method provided in the embodiment of the present application includes but is not limited to:

[0060] S301 : Process a reference image frame of a video image frame using a first image search method to obtain an initial motion vector of a current image block.

[0061] In the embodiment of the present application, the video image frame and the reference image frame are both included in the video to be encoded, and the reference image frame can be determined from multiple image frames included in the video to be encoded according to the video image frame. The video image frame includes multiple pixels, the current image block is included in the video image frame, and the current image block may include one or more pixels. The reference image frame can be processed using the first image search method to obtain an initial motion vector of the current image block.

[0062] In one embodiment, the implementation method of determining the initial motion vector of the current image block may be: determining an image search range from a reference image frame according to a first image search method, and performing search processing on the image search range to obtain a plurality of candidate image blocks, wherein the image block size of the candidate image blocks matches the image block size of the current image block; determining image difference data between the current image block and each candidate image block, wherein the image difference data includes one or more of relative difference data, conversion difference data, and difference square sum data; determining a reference image block from the plurality of candidate image blocks using the image difference data of the plurality of candidate image blocks, and performing vector calculation on the reference image block and the current image block to obtain the initial motion vector of the current image block. After determining the current image block from the video image frame, the first image search method can be used to determine the initial motion vector of the current image block; the image search range can be determined from the reference image frame according to the first image search method, and the image search range is larger than the image range corresponding to the current image block; the image search access can be searched and processed to obtain multiple image blocks to be selected, and the image block size of each image block to be selected is the same as the image block size of the current image block; the pixel values ​​of each image block to be selected can be compared with the current image block to determine the image difference data between the current image block and each image block to be selected, and the image difference data can include one or more of relative difference data, conversion difference data, and difference square sum data.

[0063] The image difference data of multiple image blocks to be selected can be used to determine a reference image block from the multiple image blocks to be selected, and the reference image block is the image block with the smallest difference from the current image block among the multiple image blocks to be selected. Vector calculation can be performed on the reference image block and the current image block (for example: vector calculation is performed using the position data of each pixel point of the reference image block and the position data of the corresponding pixel point in the current image block) to obtain the initial motion vector of the current image block. Through the method provided in the embodiment of the present application, multiple different difference data can be used to determine the reference image block, which ensures the similarity between the reference image block and the current image block, and can improve the accuracy of the initial motion vector to a certain extent, thereby helping to reduce the computational complexity and improve the video coding efficiency.

[0064] S302: Determine a plurality of verification image blocks of the current image block from the video image frame, wherein each of the verification image blocks has the same image block size.

[0065] In an embodiment of the present application, multiple verification image blocks of the current image block can be determined from the video image frame, and the image block size of each verification image block is the same, that is, the number of pixels contained in each verification image block is the same. The image area where each verification image block is located in the video image frame is the target image area, and the target image area includes the encoded image area in the video image frame and the image area corresponding to the current image block. Among the multiple verification image blocks of the current image block, there is a verification image block that partially overlaps with the current image block. Through the method provided in the embodiment of the present application, multiple verification image blocks can be determined in a specific image area in the video image frame, which is convenient for the subsequent determination of the verification motion vector of the verification image block, and the initial motion vector is verified according to the verification motion vector to obtain an accurate verification result.

[0066] In one embodiment, determining multiple verification image blocks of a current image block from a video image frame includes: determining the verification image block size based on a comparison result between an image block size of the current image block and a size threshold; determining a first image block from the current image block using the verification image block size; determining multiple second image blocks from an encoded image region in the video image frame using a target number and the verification image block size, the sum of the number of the first image block and the number of the multiple second image blocks being the target number; and determining the first image block and the multiple second image blocks as the verification image blocks. The verification image block size can be determined according to the image block size of the current image block: the image block size of the current image block can be compared with the size threshold to obtain a comparison result. If the comparison result is that the image block size of the current image block is less than the size threshold, the verification image block size is determined to be the first size; if the comparison result is that the image block size of the current image block is greater than or equal to the size threshold, the verification image block size is determined to be the second size; for example: the size threshold is 16*16 (i.e., the image block includes 256 pixels), the first size is 8*8, and the second size is 16*16. If the image block size of the current image block (assuming it is 8*8) is less than the size threshold, the verification image block size is determined to be 8*8; if the comparison result is that the image block size of the current image block (assuming it is 32*32) is greater than the size threshold, the verification image block size is determined to be 16*16. Determining the verification image block size according to the image block size of the current image block and the size threshold can avoid a large size difference between the current image block and the verification image block, thereby causing an error. It should be noted that the verification image block size is less than or equal to the image block size of the current image block.

[0067] After determining the size of the verification image block, the verification image block size can be used to determine the first image block from the current image block, and the first image block is included in the current image block, that is, the pixel points of the first image block are included in the current image block. The target number and the verification image block size can be used to determine multiple second image blocks from the encoded image area in the video image frame, and the sum of the number of the first image block and the multiple second image blocks is the target number. The first image block and the second image block can be determined as the verification image block. Through the method provided in the embodiment of the present application, the verification image block can be accurately and reasonably determined, which is convenient for the subsequent verification processing of the initial motion vector according to the verification image block to obtain an accurate verification result, thereby effectively improving the accuracy of the algorithm and reducing the amount of data calculation.

[0068] In one embodiment, the target number of verified image blocks can be determined according to the verification parameter K, and the verification parameter can be determined according to the image block size of the current image block and the image area size of the encoded image area in the video image frame. For example: the target number of verified image blocks can be (K+1)*(K+1), K is the verification parameter, and assuming that K is 2, the target number of verified image blocks is 9, and these 9 verified image blocks include a first image block and 8 second image blocks. Through the method provided in the embodiment of the present application, the target number can be made more reasonable, which is conducive to improving the accuracy of the verification result.

[0069] See also Figure 4 , which is a schematic diagram of a verification image block provided in an embodiment of the present application. Figure 4 The area corresponding to the solid dots in the video image frame is the coded image area of ​​the video image frame, and the area corresponding to the hollow dots is the uncoded image area of ​​the video image frame. The image block size of the current image block in the video image frame is 4*5 (that is, the current image block includes 20 pixels), and the current image block is Figure 4 The image block surrounded by the dotted line in the video image frame shown. According to the comparison result between the image block size of the current image block and the size threshold, it can be determined that the verification image block size is 3*3, and the target number is 4. Then, the verification image block size can be used to determine the first image block from the current image block ( Figure 4 The image block surrounded by the solid line in the current image block is the first image block); the size of the verified image block can be used to determine three second image blocks from the encoded image area, and the three second image blocks are as follows Figure 4 As shown. The first image block and the plurality of second image blocks may be determined as verification image blocks. Through the method provided in the embodiment of the present application, the plurality of image blocks may be reasonably and accurately determined, so as to facilitate the subsequent determination of the verification result of the initial motion vector using the verification motion vectors of the plurality of verification image blocks.

[0070] It should be noted that if the current image block is the first image block in the video image frame (i.e., at this time, the video image frame does not contain the encoded image area), the video image frame can be supplemented according to the verification image block size to obtain a supplemented image frame, which includes the video image frame and multiple supplemented pixel points. The pixel values ​​of the supplemented pixel points can be fixed values ​​or determined according to the reference image frame and the video image frame. Multiple verification image blocks can be determined from the supplemented image frame using the target number data and the verification image block size data. Figure 4 The size, shape and number of the current image block and the verification image block shown are only a feasible example. In actual application, the size, shape and number of the current image block and the verification image block can be adjusted according to different application requirements.

[0071] S303. Verify the initial motion vector of the current image block according to the verification motion vectors of the multiple verification image blocks to obtain a verification result; wherein the verification motion vector of the verification image block is determined according to the target motion vector of the encoded image area in the video image frame.

[0072] In the embodiment of the present application, the motion trend of the pixels around the current image block in the video image frame should be roughly the same as the motion trend of the current image block, that is, the motion vector of the pixels around the current image block is roughly the same as the motion vector of the current image block. Therefore, multiple verification image blocks located around the current image block can be determined, and the initial motion vector of the current image block can be verified according to the verification motion vectors of each verification image block to obtain a verification result, which can indicate to a certain extent whether the initial motion vector meets the requirements. The initial motion vector of the current image block can be obtained by processing the reference image frame of the video image frame using the first image search method. The video image frame and the reference image frame are both included in the video to be encoded, and the video image frame is associated with the reference image frame. The motion vector of each pixel in the encoded image area can be determined according to the target motion vector of the encoded image area in the video image frame; the multiple verification image blocks of the current image block are mainly located in the encoded image area in the video image frame, and the motion vector of each pixel in the encoded image area is known, then the verification motion vector of each verification image block can be directly determined according to the motion vector of each pixel in the encoded image area. Through the method provided in the embodiment of the present application, the verification processing of the initial motion vector can be implemented to obtain accurate verification results, which is conducive to the subsequent use of the verification results to determine the target motion vector. At the same time, the verification motion vector can be determined based on the target motion vector of the encoded image area, effectively ensuring the accuracy of the verification processing and the processing efficiency of the image processing method.

[0073] It should be noted that, among the multiple verification image blocks of the current image block, there is a verification image block that partially overlaps with the current image block, that is, the pixel points of the verification image block are included in the current image block but not in the encoded image area in the video image frame. At this time, the motion vector of each pixel point can be determined based on the initial motion vector of the current image block, and then the verification motion vector of the verification image block can be determined.

[0074] In one embodiment, the verification process is performed on the initial motion vector of the current image block according to the verification motion vectors of multiple verification image blocks to obtain the verification result. The implementation method may be: determining a vector deviation according to the verification motion vectors of multiple verification image blocks; performing similarity calculation using each verification image block, the initial motion vector of the current image block and the reference image frame of the video image frame to obtain the target similarity; determining whether the vector deviation and the target similarity meet the preset verification conditions, and the preset verification conditions include: the vector deviation is less than the deviation threshold and the target similarity is greater than the similarity threshold; if so, determining that the verification result of the initial motion vector is a verification pass; if not, determining that the verification result of the initial motion vector is a verification failure. After determining multiple verification image blocks, the verification motion vector of the verification image block may be determined according to the target motion vector of the encoded image area in the video image frame and the initial motion vector of the current image block, and the vector deviation may be determined according to the verification motion vectors of the multiple verification image blocks. The vector deviation may reflect the degree of difference of the multiple verification motion vectors, and the vector deviation may reflect the difference in motion trends between the multiple verification image blocks. When the difference in motion trends between the multiple verification image blocks is small, the verification result obtained by performing the verification process according to the multiple verification image blocks has a higher accuracy.

[0075] The similarity can be calculated using the initial motion vectors of each verification image block, the current image block and the reference image frame of the video image frame to obtain the target similarity; the motion trends of multiple verification image blocks and the current motion block are similar, and the target similarity can indicate the similarity between the image block determined according to the initial motion vector and the verification image block, so as to reflect whether the initial motion vector can accurately reflect the motion trend of the image block; it can be determined whether the vector deviation and the target similarity meet the preset verification conditions, and the preset verification conditions may include: the vector deviation is less than the deviation threshold and the target similarity is greater than the similarity threshold. If so, it means that the motion trends between the verification image blocks are similar, and the initial motion vector of the current image block can better reflect the motion trends of the current image block and the verification image block, then the verification result of the initial motion vector can be determined to be verified; if not, it means that the initial motion vector of the current image block may not be able to better reflect the motion trends of the current image block and the verification image block, then the verification result of the initial motion vector can be determined to be verified unsuccessful, and further image search processing is required. Through the method provided in the embodiment of the present application, the initial motion vector can be verified using the verification motion vector and the reference image frame to obtain an accurate verification result, which is conducive to determining the target motion vector based on the verification result, reducing the amount of data calculation, reducing the calculation complexity, and improving the processing efficiency of video encoding.

[0076] In one embodiment, the verification motion vector of the verification image block is determined by: determining the motion vector of each pixel in a plurality of verification image blocks according to the target motion vector of the encoded image area or the initial motion vector of the current image block; for any verification image block among the plurality of verification image blocks, the motion vector with the highest frequency among the motion vectors of the pixels included in the verification image block can be determined as the verification motion vector of the verification image block, for example: the verification image block includes four pixels, and the motion vectors of the four pixels (the directions of the four motion vectors are all from the video image frame to the reference image frame) are (0,1), (0,1), (0,0.5) and (0,0.75) respectively, then (0,1) can be determined as the verification motion vector of the verification image block. In some cases, the motion vectors of each pixel included in the verification image block can be averaged, and the obtained average pixel motion vector can be determined as the verification motion vector of the verification image block. For example, the verification image block includes four pixels, and the motion vectors of these four pixels (the directions of these four motion vectors are from the video image frame to the reference image frame) are (0, 0.25), (0, 0.75), (0, 0.5) and (0, 0.5), respectively. Then, the average pixel motion vector (0, 0.5) can be determined as the verification motion vector of the verification image block. Through the method provided in the embodiment of the present application, the verification motion vector of the verification image block can be accurately determined, which facilitates the verification processing of the initial motion vector using the verification motion vector, thereby improving the accuracy of the verification result.

[0077] In one embodiment, determining a vector deviation based on verification motion vectors of multiple verification image blocks includes: determining the occurrence frequency of any verification motion vector among the verification motion vectors of the multiple verification image blocks according to the number of occurrences of any verification motion vector in the multiple verification motion vectors; determining a reference motion vector from the multiple verification motion vectors using the occurrence frequencies of each verification motion vector; and performing deviation calculation using the verification motion vectors of each verification image block and the reference motion vector to obtain a vector deviation. After determining the verification motion vectors of the multiple verification image blocks, the occurrence frequency of any verification motion vector can be determined according to the number of occurrences of the verification motion vector in the multiple verification motion vectors; and the reference motion vector can be determined from the multiple verification motion vectors using the occurrence frequencies of each verification motion vector.

[0078] For example, there are four verification image blocks, and the verification motion vectors of these four verification image blocks are (0,1), (0,1), (0,0.5) and (0,0.75), respectively. Then, the occurrence frequencies of these four verification motion vectors can be determined to be 0.5, 0.5, 0.25 and 0.25, respectively. Then, the verification motion vector with the largest occurrence frequency can be determined as the reference motion vector, and the reference motion vector can be (0,1). After determining the reference motion vector, the verification motion vectors of each verification image block and the reference motion vector can be used to calculate the deviation to obtain the vector deviation. In some cases, the calculation formula of the vector deviation can be shown as follows (1) and (2):

[0079]

[0080] M=(K+1)*(K+1) (2)

[0081] In the above formula, K represents the verification parameter, M represents the target number of verification image blocks, MVi represents the verification motion vector of the i-th verification image block, MVg represents the reference motion vector, the direction of the reference motion vector is the same as the direction of the verification motion vector of the verification image block, and D represents the vector deviation. Through the method provided in the embodiment of the present application, the vector deviation can be reasonably and accurately determined, so that the motion trend difference between each verification image block can be better evaluated, and it is also convenient to determine the verification result according to the vector deviation later.

[0082] It should be noted that, in some cases, the reference motion vector may be a fixed value, such as a zero value. When the reference motion vector is a fixed value, the amount of data calculation can be effectively reduced, the calculation complexity can be reduced, and the processing efficiency of the image processing method can be further improved.

[0083] In one embodiment, the implementation method of calculating the similarity using each verification image block, the initial motion vector of the current image block and the reference image frame of the video image frame to obtain the target similarity can be: for any verification image block among the multiple verification image blocks, use any verification image block and the initial motion vector to determine the matching image block corresponding to any verification image block from the reference image frame; wherein the motion vector corresponding to any verification image block and the matching image block matches the initial motion vector; use any verification image block and the corresponding matching image block to calculate the structural similarity to obtain the initial similarity corresponding to any verification image block; after determining the initial similarity corresponding to each verification image block, use the initial similarity corresponding to each verification image block to perform mean calculation to obtain the target similarity. When calculating the target similarity, for any verification image block among the multiple verification image blocks, use the verification image block and the initial motion vector to determine the matching image block corresponding to the verification image block from the reference image frame, and the motion vector corresponding to the matching image block and the verification image block matches the initial motion vector.

[0084] The current image block and the pixels around it have similar motion trends, that is, the motion vectors of the current image block and the pixels around it are roughly the same, and the verification image block includes some pixels in the current image block and some pixels around the current pixel block. Then, the matching image block of the verification image block can be determined from the reference image frame using the initial motion vector. If the accuracy of the initial motion vector is high (the initial motion vector can represent the motion trend of the current image block and the pixels around it), the similarity between the verification image block and the corresponding matching image block is high. Otherwise, the similarity between the verification image block and the corresponding matching image block is high. Therefore, the structural similarity calculation can be performed on the verification image block and the corresponding matching image block to obtain the initial similarity corresponding to the verification image block. After determining the initial similarities corresponding to each verification image block, the initial similarities corresponding to each verification image block can be used to perform mean calculation to obtain the target similarity.

[0085] In some cases, the calculation formula of target similarity can be shown as the following formula (3):

[0086]

[0087] In the above formula, M represents the target number of verification image blocks, and the calculation method of M is shown in the above formula (2), Si represents the initial similarity of the i-th verification image block, and S represents the target similarity. Through the method provided in the embodiment of the present application, the target similarity can be accurately determined, so that the verification result can be determined according to the target similarity, which is conducive to reducing the complexity of the algorithm and improving the processing efficiency of video encoding.

[0088] In one embodiment, the implementation method of determining the matching image block corresponding to any verification image block from the reference image frame using the any verification image block and the initial motion vector may be as follows: for any pixel point among the multiple pixel points included in any verification image block, vector calculation is performed on the original position data of any pixel point according to the initial motion vector to obtain the matching position data of any pixel point; the matching pixel point of any pixel point is determined from the reference image frame according to the matching position data of any pixel point; after determining the matching pixel points of the multiple pixel points included in any verification image block, the image block composed of the matching pixel points of the multiple pixel points is determined as the matching image block corresponding to any verification image block. The verification image block may include multiple pixel points. For any pixel point among the multiple pixel points included in the verification image block, vector calculation can be performed on the original position data of any pixel point according to the initial motion vector to obtain the matching position data of any pixel point. For example: the original position data of a pixel point in the verification image block is (1,1), the direction of the initial motion vector is from the pixel point in the video image frame to the pixel point in the reference video frame, and the magnitude of the initial motion vector is (1,1). Then, the matching position data of the pixel point can be determined as (2,2) according to the initial motion vector.

[0089] After determining the matching position data of any pixel point, the matching pixel point of any pixel point can be determined from the reference image frame based on the matching position data, and the position data of the matching pixel point in the reference image frame is the matching position data. For example: if the matching position data of a certain pixel point is (2,2), then the matching pixel point of the pixel point can be determined from the reference image frame, and the position data of the matching pixel point in the reference image frame is (2,2). After determining the matching pixel points of the multiple pixels included in the verification image block, the pixel block composed of the matching pixel points of the multiple pixels can be determined as the matching image block. Through the method provided in the embodiment of the present application, the initial motion vector can be used to determine the matching image block corresponding to the verification image block from the reference image frame, so as to facilitate the subsequent determination of the initial similarity between the verification image block and the matching image block based on the matching image block and the verification image block.

[0090] See also Figure 5 , which is a schematic diagram of a matching image block provided in an embodiment of the present application. Figure 5 The video image frame and the reference image frame are included, wherein the verification image block included in the reference image frame is as described above. Figure 4As shown; the initial motion vector of the current image block in the video image frame can be determined, and the initial motion vector is recorded as MVa. The direction of the initial motion vector is from the video image frame to the reference image frame, and the size of the initial motion vector is (1,1). For pixel point A among the multiple pixel points included in the verification image block, the original position data of the pixel point is (2,2). The original position data of pixel point A can be vector-calculated according to the initial motion vector to obtain the matching position data of pixel point A (3,3). According to the matching position data of pixel point A, the matching pixel point of pixel point A can be determined from the reference image frame, that is, pixel point B in the reference image frame (such as Figure 5 As shown, the motion vector between pixel A and pixel B is the initial motion vector MVa); the above method can be used to determine the matching pixel points of each pixel included in the verification image block, and the image block composed of the matching pixel points of each pixel can be determined as the matching image block, and the matching image block corresponding to each verification image block can be determined. Figure 5 The image blocks surrounded by solid lines in the reference image frame are the matching image blocks corresponding to the verification image blocks. Through the method provided in the embodiment of the present application, the matching image blocks corresponding to the verification image blocks can be accurately determined, which is convenient for the subsequent determination of the initial similarity based on the matching image blocks and the verification image blocks, so that the target similarity can be determined, which is conducive to reducing the computational complexity and improving the processing efficiency of video encoding.

[0091] In one embodiment, the verification image block includes multiple pixels, the matching image block corresponding to the verification image block includes multiple pixels, and the verification image block and the corresponding matching image block include the same number of pixels; the implementation method of using any verification image block and the corresponding matching image block to perform structural similarity calculation to obtain the initial similarity corresponding to any verification image block can be: calculating the pixel values ​​of the multiple pixels included in any verification image block to obtain the pixel mean and pixel standard deviation value of any verification image block; calculating the pixel values ​​of the multiple pixels included in the matching image block corresponding to any verification image block to obtain the pixel mean and pixel standard deviation value of the matching image block; determining the brightness similarity, contrast similarity and image structure similarity of any verification image block based on the pixel mean and pixel standard deviation value of any verification image block and the pixel mean and pixel standard deviation value of the matching image block; integrating the brightness similarity, contrast similarity and image structure similarity of any verification image block to obtain the initial similarity of any verification image block. The verification image block includes a plurality of pixels, and the matching image block determined according to the verification image block and the initial motion vector also includes a plurality of pixels, and the number of pixels included in the verification image block and the corresponding matching image block is the same. When determining the initial similarity between any verification image block and its corresponding matching image block, the pixel values ​​of the plurality of pixels included in the verification image block can be calculated to obtain the pixel mean and pixel standard deviation of the verification image block. The verification image block can be represented by x, and the pixel mean of the verification image block can be represented by μ x , the pixel standard deviation of the verification image block can be expressed as δ x ; The pixel values ​​of multiple pixels included in the matching image block can be calculated to obtain the pixel mean and pixel standard deviation of the matching image block. The matching image block can be represented by y, and the pixel mean of the matching image block can be represented by μ y , the pixel standard deviation of the matching image patch can be expressed as δ y .

[0092] According to the pixel mean and pixel standard deviation value of the verification image block, and the pixel mean and pixel standard deviation value of the matching image block, the brightness similarity, contrast similarity and image structure similarity can be determined, wherein the brightness similarity can reflect the similarity between the verification image block and the matching image block in the image brightness dimension, and the brightness similarity can be expressed as l(x, y); the contrast similarity can reflect the similarity between the verification image block and the matching image block in the image contrast dimension, and the contrast similarity can be expressed as c(x, y); the image structure similarity can reflect the similarity between the verification image block and the matching image block in the image structure dimension, and the image structure similarity can be expressed as s(x, y); the brightness similarity, contrast similarity and image structure similarity can be integrated and processed to obtain the initial similarity between the verification image block and the matching image block. Through the method provided in the embodiment of the present application, the initial similarity between the verification image block and the matching image block can be accurately determined, so that the verification result of the initial motion vector can be determined according to the initial similarity, which is conducive to improving the accuracy of the image processing method, reducing the computational complexity, and improving the video encoding efficiency.

[0093] In one embodiment, the structure similarity index (Structure Similarity Index Measure, SSIM) can be used as a similarity measurement indicator between the verification image block and the matching image block: the pixel mean of the verification image block can be calculated as μ x , the pixel standard deviation of the verification image block is δ x ; The pixel mean of the matching image block is μ y , the pixel standard deviation of the matching image patch is δ y , the pixel covariance between the verification image patch and the matching image patch is δ xy , the method for determining brightness similarity, contrast similarity and image structure similarity can be shown as the following equations (4), (5) and (6):

[0094]

[0095]

[0096] s(x,y)=(δ xy +C3) / (δ x *δ y +C3) (6)

[0097] Among them, l(x, y) is brightness similarity, c(x, y) is contrast similarity, s(x, y) is image structure similarity, and C1, C2, and C3 are all constants. Through the above three formulas, the brightness similarity, contrast similarity, and image structure similarity between the verification image block and the matching image block can be determined. According to the brightness similarity, contrast similarity, and image structure similarity, the implementation method of determining the similarity between the verification image block and the matching image block can be shown in the following formula (7):

[0098] SSIM(x,y)=[l(x,y)] α *[c(x,y)] β *[s(x,y)] γ (7)

[0099] Among them, α, β, and γ are all constant parameters greater than 0. Through the above formula, the SSIM similarity between the verification image block and the matching image block can be determined. The SSIM similarity takes into account the structural information of the two image blocks, can more accurately reflect the subjective similarity of the two image blocks, and has high calculation efficiency. Through the method provided in the embodiment of the present application, the initial similarity between the verification image block and the matching image block can be accurately determined, so that the target similarity can be determined based on the initial similarity, and then the verification result of the initial motion vector can be determined based on the target similarity.

[0100] S304: Determine the target motion vector of the current image block using the verification result according to the initial motion vector or a second image search method.

[0101] In an embodiment of the present application, the verification result can be used to determine the target motion vector of the current image block according to the initial motion vector or the second image search method, and the target motion vector can be used to implement video encoding processing of the video to be encoded. When the target motion vector of the current image block is determined according to the initial motion vector, there is no need to process the reference image frame using the second image search method, which can effectively reduce the amount of calculation and improve processing efficiency. When the target motion vector of the current image block is determined according to the second image search method, the accuracy of the target motion vector can be effectively guaranteed. For multiple video image frames in the video to be encoded, the method provided by the present application can be used to determine the target motion vector, thereby reducing the amount of calculation as a whole, reducing the computational complexity, and improving the efficiency of video encoding, thereby saving network resources.

[0102] In one embodiment, the method for implementing the target motion vector of the current image block by using the verification result according to the initial motion vector or the second image search method can be: if the verification result indicates that the verification is passed, the initial motion vector is determined as the target motion vector of the current image block; if the verification result indicates that the verification is not passed, the reference image frame is processed using the second image search method to obtain the target motion vector of the current image block. If the verification result indicates that the verification is passed, it means that the initial motion vector meets the requirements, and there is no need to perform image search processing, and the initial motion vector can be directly determined as the target motion vector of the current image block; if the verification result indicates that the verification is not passed, it means that the initial motion vector does not meet the requirements, and it is necessary to use the second image search method to perform more refined processing on the reference image frame to obtain the target motion vector of the current image block. The method provided by the embodiment of the present application can effectively reduce the computational complexity, reduce the amount of data calculation, and improve the processing efficiency of video encoding.

[0103] In one embodiment, if the verification result indicates that the verification fails, the reference image frame is processed using the second image search method to obtain the target motion vector of the current image block, which can be implemented as follows: interpolation processing is performed on the sub-pixel position that may be searched in the reference image frame to obtain an interpolated image frame, that is, the reference image frame is up-sampled using an interpolation method (such as bilinear interpolation or a sub-pixel motion compensation method specified in the coding standard) to obtain an image frame with a higher resolution, which allows subsequent searches to be performed at sub-pixel positions; after determining the interpolated image frame, the interpolated image frame can be searched at a sub-pixel level in a small area around the whole pixel search result (for example: traversing possible positions in a certain order, and calculating distortion values ​​and rate-distortion cost values ​​respectively); according to the sub-pixel level search results, the target image block with the lowest rate-distortion cost (that is, the highest similarity with the current image block) is determined from the sub-pixel search area, and vector calculation is performed based on the target image block and the current image block to obtain the target motion vector of the current image block. Through the method provided in the embodiment of the present application, the target motion vector can be accurately determined, which is conducive to improving the accuracy of the target motion vector.

[0104] See also Figure 6, which is a schematic diagram of an image processing method provided in an embodiment of the present application. When performing video encoding, the motion vector of the video image frame can be calculated. For the current image block of the video image frame, the reference image frame can be processed by the first image search method to obtain the initial motion vector of the current image block; multiple verification image blocks of the current image block can be determined from the video image frame, and the image block size of each verification image block is the same. The verification motion vector of each verification image block can be determined by using the target motion vector of the encoded image area in the video image frame and the initial motion vector of the current image block; the vector deviation can be determined according to the verification motion vector of each verification image block; the matching image block corresponding to each verification image block can be determined from the reference image frame by using the multiple verification image blocks and the initial motion vector, and the target similarity can be determined according to the initial similarity between the matching image block and the verification image block; it is judged whether the vector deviation and the target similarity meet the preset verification condition, and the preset verification condition can be that the vector deviation is less than the deviation threshold and the target similarity is greater than the similarity threshold; if so, it is determined that the initial motion vector verification is passed, and the initial motion vector can be determined as the target motion vector of the current image block; if not, it is determined that the initial motion vector verification is not passed, and the reference image frame can be processed by the second image search method to obtain the target motion vector of the current image block. Through the method provided in the embodiment of the present application, the target motion vector determination process can be terminated early, thereby effectively reducing the amount of data calculation, reducing the calculation complexity, and effectively avoiding errors caused by overly detailed searches, thereby ensuring the accuracy of the target motion vector and improving the video encoding effect.

[0105] Through the image processing method provided by the embodiment of the present application, a variety of different types of similarity measurement data can be used to determine the initial motion vector of the current image block, thereby improving the accuracy of the initial motion vector and reducing the amount of data calculation; multiple verification image blocks of the current image block can be reasonably determined, so as to facilitate determining accurate verification results based on multiple verification image blocks; the initial motion vector can be verified using the verification motion vector of the verification image block to obtain a verification result, thereby determining whether the initial motion vector meets the requirements; when the target motion vector is determined using the initial motion vector based on the verification result, the target motion vector determination process can be terminated early, thereby effectively reducing the amount of data calculation, reducing the calculation complexity, and effectively avoiding errors caused by overly detailed searches, thereby ensuring the accuracy of the motion vector, improving the effect of video encoding, and improving the processing efficiency of video encoding.

[0106] See also Figure 7 , Figure 7 This is a structural block diagram of an image processing device provided in an embodiment of the present application. The device includes:

[0107] A determining unit 701 is used to determine a plurality of verification image blocks of a current image block from a video image frame, wherein each of the verification image blocks has the same image block size;

[0108] The processing unit 702 is configured to perform verification processing on the initial motion vector of the current image block according to the verification motion vectors of the multiple verification image blocks to obtain a verification result; wherein the initial motion vector of the current image block is obtained by processing the reference image frame of the video image frame using a first image search method, and the verification motion vector of the verification image block is determined according to the target motion vector of the encoded image area in the video image frame;

[0109] The determining unit 701 is further configured to determine the target motion vector of the current image block using the verification result according to the initial motion vector or a second image search method.

[0110] In one embodiment, when the determination unit 701 uses the verification result to determine the target motion vector of the current image block according to the initial motion vector or the second image search method, it is specifically used to: if the verification result indicates that the verification is passed, then the initial motion vector is determined as the target motion vector of the current image block; if the verification result indicates that the verification is not passed, then the reference image frame is processed using the second image search method to obtain the target motion vector of the current image block.

[0111] In one embodiment, when determining multiple verification image blocks of a current image block from a video image frame, the determination unit 701 is specifically used to: determine the verification image block size based on a comparison result between the image block size of the current image block and a size threshold; determine a first image block from the current image block using the verification image block size; determine multiple second image blocks from an encoded image area in the video image frame using a target number and the verification image block size, the sum of the number of the first image block and the number of the multiple second image blocks being the target number; and determine the first image block and the multiple second image blocks as verification image blocks.

[0112] In one embodiment, when the processing unit 702 performs verification processing on the initial motion vector of the current image block according to the verification motion vectors of the multiple verification image blocks to obtain a verification result, it is specifically used to: determine a vector deviation according to the verification motion vectors of the multiple verification image blocks; perform similarity calculation using each of the verification image blocks, the initial motion vector of the current image block and the reference image frame of the video image frame to obtain a target similarity; determine whether the vector deviation and the target similarity meet a preset verification condition, and the preset verification condition includes: the vector deviation is less than a deviation threshold and the target similarity is greater than a similarity threshold; if so, determine that the verification result of the initial motion vector is a passed verification; if not, determine that the verification result of the initial motion vector is a failed verification.

[0113] In one embodiment, when determining the vector deviation based on the verification motion vectors of the multiple verification image blocks, the processing unit 702 is specifically used to: determine the occurrence frequency of any verification motion vector among the verification motion vectors of the multiple verification image blocks according to the number of occurrences of the any verification motion vector in the multiple verification motion vectors; determine a reference motion vector from the multiple verification motion vectors using the occurrence frequencies of each of the verification motion vectors; and perform deviation calculation using the verification motion vectors of each of the verification image blocks and the reference motion vector to obtain a vector deviation.

[0114] In one embodiment, when the processing unit 702 uses each of the verification image blocks, the initial motion vector of the current image block and the reference image frame of the video image frame to perform similarity calculation to obtain the target similarity, it is specifically used to: for any verification image block among the multiple verification image blocks, use the any verification image block and the initial motion vector to determine the matching image block corresponding to the any verification image block from the reference image frame; wherein the motion vector corresponding to the any verification image block and the matching image block matches the initial motion vector; use the any verification image block and the corresponding matching image block to perform structural similarity calculation to obtain the initial similarity corresponding to the any verification image block; after determining the initial similarities corresponding to each of the verification image blocks, use the initial similarities corresponding to each of the verification image blocks to perform mean calculation to obtain the target similarity.

[0115] In one embodiment, the verification image block includes a plurality of pixels, the matching image block corresponding to the verification image block includes a plurality of pixels, and the verification image block and the corresponding matching image block include the same number of pixels; when the processing unit 702 uses any verification image block and the corresponding matching image block to perform structural similarity calculation to obtain the initial similarity corresponding to any verification image block, it is specifically used to: calculate the pixel values ​​of the plurality of pixels included in any verification image block to obtain the pixel mean and pixel standard deviation value of any verification image block; calculate the pixel values ​​of the plurality of pixels included in the matching image block corresponding to any verification image block to obtain the pixel mean and pixel standard deviation value of the matching image block; determine the brightness similarity, contrast similarity and image structure similarity of any verification image block according to the pixel mean and pixel standard deviation value of any verification image block and the pixel mean and pixel standard deviation value of the matching image block; integrate the brightness similarity, contrast similarity and image structure similarity of any verification image block to obtain the initial similarity of any verification image block.

[0116] In one embodiment, the determination unit 701 is further used to: determine an image search range from the reference image frame according to the first image search method, and perform search processing on the image search range to obtain multiple candidate image blocks, wherein the image block size of the candidate image blocks matches the image block size of the current image block; determine image difference data between the current image block and each of the candidate image blocks, wherein the image difference data includes one or more of relative difference data, conversion difference data, and difference square sum data; determine a reference image block from the multiple candidate image blocks using the image difference data of the multiple candidate image blocks, and perform vector calculation on the reference image block and the current image block to obtain an initial motion vector of the current image block.

[0117] It is understandable that the functions of each functional unit of the image processing device of the embodiment of the present application can be specifically implemented according to the image processing method in the above-mentioned method embodiment, and its specific implementation process can refer to the relevant description in the above-mentioned image processing method embodiment, which will not be repeated here. In the embodiment of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that contains the function of the module or unit.

[0118] Through the image processing method provided by the embodiment of the present application, a variety of different types of similarity measurement data can be used to determine the initial motion vector of the current image block, thereby improving the accuracy of the initial motion vector and reducing the amount of data calculation; multiple verification image blocks of the current image block can be reasonably determined, so as to facilitate determining accurate verification results based on multiple verification image blocks; the initial motion vector can be verified using the verification motion vector of the verification image block to obtain a verification result, thereby determining whether the initial motion vector meets the requirements; when the target motion vector is determined using the initial motion vector based on the verification result, the target motion vector determination process can be terminated early, thereby effectively reducing the amount of data calculation, reducing the calculation complexity, and effectively avoiding errors caused by overly detailed searches, thereby ensuring the accuracy of the motion vector, improving the effect of video encoding, and improving the processing efficiency of video encoding.

[0119] See also Figure 8 , Figure 8 A block diagram of a computer device provided in an embodiment of the present application. The computer device described in the embodiment of the present application includes: a processor 801, a communication interface 802, and a memory 803. The processor 801, the communication interface 802, and the memory 803 may be connected via a bus or other means, and the embodiment of the present application takes the connection via a bus as an example.

[0120] Among them, the processor 801 (or CPU (Central Processing Unit)) is the computing core and control core of the computer device, which can parse various instructions in the computer device and process various data of the computer device. For example, the CPU can be used to parse the power on and off instructions sent by the user to the computer device, and control the computer device to perform power on and off operations; for another example, the CPU can transmit various interactive data between the internal structures of the computer device, and so on. The communication interface 802 can optionally include a standard wired interface, a wireless interface (such as Wi-Fi, a mobile communication interface, etc.), which is controlled by the processor 801 to send and receive data. The memory 803 (Memory) is a memory device in the computer device for storing programs and data. It can be understood that the memory 803 here can include both the built-in memory of the computer device and the extended memory supported by the computer device. The memory 803 provides a storage space, which stores the operating system of the computer device, which may include but is not limited to: Android system, iOS system, Windows Phone system, etc., and this application does not limit this.

[0121] In the embodiment of the present application, the processor 801 performs the following operations by running the executable program code in the memory 803:

[0122] Determine a plurality of verification image blocks of the current image block from the video image frame, wherein each of the verification image blocks has the same image block size;

[0123] Verifying the initial motion vector of the current image block according to the verification motion vectors of the multiple verification image blocks to obtain a verification result; wherein the initial motion vector of the current image block is obtained by processing the reference image frame of the video image frame using a first image search method, and the verification motion vector of the verification image block is determined according to the target motion vector of the encoded image area in the video image frame;

[0124] The target motion vector of the current image block is determined using the verification result according to the initial motion vector or a second image search method.

[0125] In one embodiment, when the processor 801 uses the verification result to determine the target motion vector of the current image block according to the initial motion vector or the second image search method, it is specifically used to: if the verification result indicates that the verification is passed, then the initial motion vector is determined as the target motion vector of the current image block; if the verification result indicates that the verification is not passed, then the reference image frame is processed using the second image search method to obtain the target motion vector of the current image block.

[0126] In one embodiment, when the processor 801 determines multiple verification image blocks of a current image block from a video image frame, it is specifically used to: determine the verification image block size based on a comparison result between the image block size of the current image block and a size threshold; determine a first image block from the current image block using the verification image block size; determine multiple second image blocks from an encoded image area in the video image frame using a target number and the verification image block size, the sum of the number of the first image block and the multiple second image blocks being the target number; and determine the first image block and the multiple second image blocks as verification image blocks.

[0127] In one embodiment, when the processor 801 performs verification processing on the initial motion vector of the current image block according to the verification motion vectors of the multiple verification image blocks to obtain a verification result, it is specifically used to: determine a vector deviation according to the verification motion vectors of the multiple verification image blocks; perform similarity calculation using each of the verification image blocks, the initial motion vector of the current image block and the reference image frame of the video image frame to obtain a target similarity; determine whether the vector deviation and the target similarity meet a preset verification condition, and the preset verification condition includes: the vector deviation is less than a deviation threshold and the target similarity is greater than a similarity threshold; if so, determine that the verification result of the initial motion vector is a passed verification; if not, determine that the verification result of the initial motion vector is a failed verification.

[0128] In one embodiment, when the processor 801 determines the vector deviation based on the verification motion vectors of the multiple verification image blocks, it is specifically used to: determine the occurrence frequency of any verification motion vector among the verification motion vectors of the multiple verification image blocks according to the number of occurrences of the any verification motion vector in the multiple verification motion vectors; determine a reference motion vector from the multiple verification motion vectors using the occurrence frequencies of each of the verification motion vectors; and perform deviation calculation using the verification motion vectors of each of the verification image blocks and the reference motion vector to obtain a vector deviation.

[0129] In one embodiment, when the processor 801 uses each of the verification image blocks, the initial motion vector of the current image block and the reference image frame of the video image frame to perform similarity calculation to obtain the target similarity, it is specifically used to: for any verification image block among the multiple verification image blocks, use the any verification image block and the initial motion vector to determine the matching image block corresponding to the any verification image block from the reference image frame; wherein the motion vector corresponding to the any verification image block and the matching image block matches the initial motion vector; use the any verification image block and the corresponding matching image block to perform structural similarity calculation to obtain the initial similarity corresponding to the any verification image block; after determining the initial similarities corresponding to each of the verification image blocks, use the initial similarities corresponding to each of the verification image blocks to perform mean calculation to obtain the target similarity.

[0130] In one embodiment, the verification image block includes multiple pixels, the matching image block corresponding to the verification image block includes multiple pixels, and the verification image block and the corresponding matching image block include the same number of pixels; when the processor 801 uses any verification image block and the corresponding matching image block to perform structural similarity calculation to obtain the initial similarity corresponding to any verification image block, it is specifically used to: calculate the pixel values ​​of the multiple pixels included in any verification image block to obtain the pixel mean and pixel standard deviation value of any verification image block; calculate the pixel values ​​of the multiple pixels included in the matching image block corresponding to any verification image block to obtain the pixel mean and pixel standard deviation value of the matching image block; determine the brightness similarity, contrast similarity and image structure similarity of any verification image block according to the pixel mean and pixel standard deviation value of any verification image block and the pixel mean and pixel standard deviation value of the matching image block; integrate the brightness similarity, contrast similarity and image structure similarity of any verification image block to obtain the initial similarity of any verification image block.

[0131] In one embodiment, the processor 801 is further used to: determine an image search range from the reference image frame according to the first image search method, and perform search processing on the image search range to obtain multiple candidate image blocks, wherein the image block size of the candidate image blocks matches the image block size of the current image block; determine image difference data between the current image block and each of the candidate image blocks, wherein the image difference data includes one or more of relative difference data, conversion difference data, and difference square sum data; determine a reference image block from the multiple candidate image blocks using the image difference data of the multiple candidate image blocks, and perform vector calculation on the reference image block and the current image block to obtain an initial motion vector of the current image block.

[0132] In a specific implementation, the processor 801, communication interface 802 and memory 803 described in the embodiment of the present application can execute the implementation method of the image processing server described in an image processing method provided in an embodiment of the present application, and can also execute the implementation method described in an image processing device provided in an embodiment of the present application, which will not be repeated here.

[0133] Through the image processing method provided by the embodiment of the present application, a variety of different types of similarity measurement data can be used to determine the initial motion vector of the current image block, thereby improving the accuracy of the initial motion vector and reducing the amount of data calculation; multiple verification image blocks of the current image block can be reasonably determined, so as to facilitate determining accurate verification results based on multiple verification image blocks; the initial motion vector can be verified using the verification motion vector of the verification image block to obtain a verification result, thereby determining whether the initial motion vector meets the requirements; when the target motion vector is determined using the initial motion vector based on the verification result, the target motion vector determination process can be terminated early, thereby effectively reducing the amount of data calculation, reducing the calculation complexity, and effectively avoiding errors caused by overly detailed searches, thereby ensuring the accuracy of the motion vector, improving the effect of video encoding, and improving the processing efficiency of video encoding.

[0134] The present application also provides a computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer-readable storage medium is run on a computer, the computer executes the image processing method as described in the present application. The specific implementation method can be referred to the above description, and will not be repeated here.

[0135] The embodiment of the present application also provides a computer program product, which includes a computer program or a computer instruction, and the computer program or the computer instruction is stored in a computer-readable storage medium. The processor of the computer device reads the computer program or the computer instruction from the computer-readable storage medium, and the processor executes the computer program or the computer instruction, so that the computer device performs the image processing method as described in the embodiment of the present application. The specific implementation method can be referred to the above description, which will not be repeated here.

[0136] It should be noted that, for the above-mentioned various method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0137] In the above embodiments, the description of each embodiment has its own emphasis. For the part that is not described in detail in a certain embodiment, please refer to the relevant description of other embodiments. The technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a computer, a server or a network device, etc., specifically a processor in a computer device) to perform all or part of the steps of the above methods of each embodiment of the present application. Among them, the aforementioned storage medium may include: U disk, mobile hard disk, magnetic disk, optical disk, read-only memory (English: Read-Only Memory, abbreviated: ROM) or random access memory (English: Random Access Memory, abbreviated: RAM) and other media that can store program codes.

[0138] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, it should be understood that the technical solutions recorded in the aforementioned embodiments can still be modified, or some of the technical features therein can be replaced by equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An image processing method, It is characterized in that The method comprises: Determine a plurality of verification image blocks of the current image block from the video image frame, wherein each of the verification image blocks has the same image block size; Determining a vector deviation according to verification motion vectors of the plurality of verification image blocks, wherein the verification motion vectors of the verification image blocks are determined according to target motion vectors of encoded image regions in the video image frame; For any verification image block among the multiple verification image blocks, determine a matching image block corresponding to the any verification image block from a reference image frame of the video image frame using initial motion vectors of the any verification image block and the current image block; wherein the motion vector corresponding to the any verification image block and the matching image block matches the initial motion vector, and the initial motion vector of the current image block is obtained by processing the reference image frame using a first image search method; Calculating the structural similarity using any verification image block and the corresponding matching image block to obtain an initial similarity corresponding to any verification image block; After determining the initial similarities corresponding to the verification image blocks, performing mean calculation using the initial similarities corresponding to the verification image blocks to obtain a target similarity; Determining whether the vector deviation and the target similarity meet a preset verification condition, wherein the preset verification condition includes: the vector deviation is less than a deviation threshold and the target similarity is greater than a similarity threshold; If yes, determining that the verification result of the initial motion vector is verification passed; If not, determining that the verification result of the initial motion vector is verification failure; If the verification result indicates that the verification is passed, determining the initial motion vector as the target motion vector of the current image block; If the verification result indicates that the verification fails, the reference image frame is processed using a second image search method to obtain a target motion vector of the current image block.

2. The method according to claim 1, It is characterized in that The step of determining a plurality of verification image blocks of a current image block from a video image frame comprises: Determining a verification image block size according to a comparison result between an image block size of the current image block and a size threshold; Determining a first image block from the current image block using the verification image block size; Determine a plurality of second image blocks from the encoded image region in the video image frame using the target number and the verification image block size, wherein the sum of the number of the first image block and the plurality of second image blocks is the target number; The first image block and the plurality of second image blocks are determined as verification image blocks.

3. The method according to claim 1, It is characterized in that The determining the vector deviation according to the verification motion vectors of the multiple verification image blocks comprises: For any verification motion vector among the verification motion vectors of the multiple verification image blocks, determine the occurrence frequency of the any verification motion vector according to the number of occurrences of the any verification motion vector in the multiple verification motion vectors; Determine a reference motion vector from the plurality of verification motion vectors using the occurrence frequencies of the respective verification motion vectors; Deviation calculation is performed using the verification motion vectors of each verification image block and the reference motion vector to obtain a vector deviation.

4. The method according to claim 1, It is characterized in that The verification image block includes a plurality of pixels, the matching image block corresponding to the verification image block includes a plurality of pixels, and the verification image block and the corresponding matching image block include the same number of pixels; the method of calculating the structural similarity using any of the verification image blocks and the corresponding matching image block to obtain the initial similarity corresponding to any of the verification image blocks includes: Calculating pixel values ​​of a plurality of pixel points included in any verification image block to obtain a pixel mean value and a pixel standard deviation value of any verification image block; Calculating pixel values ​​of a plurality of pixel points included in a matching image block corresponding to any one of the verification image blocks to obtain a pixel mean value and a pixel standard deviation value of the matching image block; Determining brightness similarity, contrast similarity and image structure similarity of any verification image block according to the pixel mean and pixel standard deviation value of any verification image block and the pixel mean and pixel standard deviation value of the matching image block; The brightness similarity, contrast similarity and image structure similarity of any verification image block are integrated to obtain the initial similarity of any verification image block.

5. The method according to any one of claims 1 to 4, It is characterized in that The method further comprises: Determining an image search range from the reference image frame according to the first image search method, and performing search processing on the image search range to obtain a plurality of image blocks to be selected, wherein the image block sizes of the image blocks to be selected match the image block size of the current image block; Determine image difference data between the current image block and each of the to-be-selected image blocks, wherein the image difference data includes one or more of relative difference data, conversion difference data, and difference square sum data; A reference image block is determined from the multiple image blocks to be selected by using the image difference data of the multiple image blocks to be selected, and vector calculation is performed on the reference image block and the current image block to obtain an initial motion vector of the current image block.

6. An image processing device, It is characterized in that The device comprises: a determination unit and a processing unit, and the image processing device is used to implement the image processing method according to any one of claims 1 to 5.

7. A computer device, It is characterized in that include: A processor, a communication interface and a memory, wherein the processor, the communication interface and the memory are connected to each other, wherein the memory stores an executable program code, and the processor is used to call the executable program code to implement the image processing method according to any one of claims 1 to 5.

8. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor and implementing the image processing method according to any one of claims 1 to 5.

9. A computer program product, It is characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the image processing method according to any one of claims 1 to 5 is implemented.