Video image processing method and device, computer device and storage medium

Abstract

The application relates to a video image processing method and device, computer equipment, a storage medium and a computer program product. The method determines corresponding target precision of sub-pixel motion estimation according to different encoding speed requirements. For a current pixel block, integer pixel motion estimation is first performed, and when sub-pixel motion estimation is performed, the sub-pixel motion estimation is performed in the order from low to high precision of the sub-pixel motion estimation, the target optimal position of the sub-pixel motion estimation of the current precision is obtained by taking the target optimal position determined by the previous pixel motion estimation as the center each time, and the sub-pixel motion estimation is stopped until the precision of the current sub-pixel motion estimation reaches the target precision. The method can achieve the precision of the encoding speed matching and improve the encoding efficiency, so that the sub-pixel motion estimation of different precisions can be flexibly performed according to different encoding speeds.

Description

Technical Field

[0001] This application relates to the fields of cloud technology and image processing, and in particular to a video image processing method, apparatus, computer equipment, storage medium, and computer program product. Background Technology

[0002] With the rapid development of internet technology, the demand for video applications is increasing. Video applications inevitably face the challenge of video compression. There are many methods for target video compression encoding, including HEVC and VVC. In video compression encoding, fractional pixel motion estimation is often used after integer pixel motion estimation to more accurately determine the position of the matching pixel block. In addition, it is frequently used in temporal filtering during the preprocessing stage of video encoding to find the best-matching block, effectively removing noise from the image. This saves codewords needed for compression and allows for better reference for other video frames, improving the encoder's compression efficiency.

[0003] However, current pixel-level motion estimation suffers from low efficiency. Summary of the Invention

[0004] Therefore, it is necessary to provide a video image processing method, apparatus, computer equipment, computer-readable storage medium, and computer program product that can improve efficiency in response to the above-mentioned technical problems.

[0005] Firstly, this application provides a video image processing method. The method includes:

[0006] Obtain the current pixel block of the current image frame in the video image to be encoded;

[0007] Perform integer-pixel motion estimation on the current pixel block to determine the target optimal position of the current pixel block based on the integer-pixel motion estimation;

[0008] Obtain the target encoding speed of the video application, and determine the target accuracy of the corresponding pixel-level motion estimation based on the target encoding speed;

[0009] In order of increasing pixel motion estimation accuracy, pixel motion estimation is performed with the target optimal position determined by the previous pixel motion estimation as the center, to obtain the target optimal position of the current pixel motion estimation accuracy, until the pixel motion estimation accuracy of the current pixel motion estimation reaches the target accuracy and then the pixel motion estimation stops.

[0010] The optimal position of the target, estimated using the pixel-level motion with the target precision, is taken as the motion estimation result of the current pixel block.

[0011] Secondly, this application also provides a video image processing apparatus. The apparatus includes:

[0012] The pixel block acquisition module is used to acquire the current pixel block of the current image frame in the video image to be encoded;

[0013] The integer pixel motion estimation module is used to perform integer pixel motion estimation on the current pixel block and determine the target optimal position of the current pixel block based on the integer pixel motion estimation.

[0014] The speed matching module is used to obtain the target encoding speed of the video application and determine the target accuracy of the corresponding pixel-level motion estimation based on the target encoding speed.

[0015] The pixel-by-pixel motion estimation module is used to perform pixel-by-pixel motion estimation in order of increasing pixel-by-pixel motion estimation accuracy, taking the target optimal position determined by the previous pixel motion estimation as the center, to obtain the target optimal position of the pixel-by-pixel motion estimation with the current accuracy, until the accuracy of the current pixel-by-pixel motion estimation reaches the target accuracy, and then stop pixel-by-pixel motion estimation, and use the target optimal position estimated by pixel-by-pixel motion with the target accuracy as the motion estimation result of the current pixel block.

[0016] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0017] Obtain the current pixel block of the current image frame in the video image to be encoded;

[0018] Perform integer-pixel motion estimation on the current pixel block to determine the target optimal position of the current pixel block based on the integer-pixel motion estimation;

[0019] Obtain the target encoding speed of the video application, and determine the target accuracy of the corresponding pixel-level motion estimation based on the target encoding speed;

[0020] In order of increasing pixel motion estimation accuracy, pixel motion estimation is performed with the target optimal position determined by the previous pixel motion estimation as the center, to obtain the target optimal position of the current pixel motion estimation accuracy, until the pixel motion estimation accuracy of the current pixel motion estimation reaches the target accuracy and then the pixel motion estimation stops.

[0021] The optimal position of the target, estimated using the pixel-level motion with the target precision, is taken as the motion estimation result of the current pixel block.

[0022] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0023] Obtain the current pixel block of the current image frame in the video image to be encoded;

[0024] Perform integer-pixel motion estimation on the current pixel block to determine the target optimal position of the current pixel block based on the integer-pixel motion estimation;

[0025] Obtain the target encoding speed of the video application, and determine the target accuracy of the corresponding pixel-level motion estimation based on the target encoding speed;

[0026] In order of increasing pixel motion estimation accuracy, pixel motion estimation is performed with the target optimal position determined by the previous pixel motion estimation as the center, to obtain the target optimal position of the current pixel motion estimation accuracy, until the pixel motion estimation accuracy of the current pixel motion estimation reaches the target accuracy and then the pixel motion estimation stops.

[0027] The optimal position of the target, estimated using the pixel-level motion with the target precision, is taken as the motion estimation result of the current pixel block.

[0028] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0029] Obtain the current pixel block of the current image frame in the video image to be encoded;

[0030] Perform integer-pixel motion estimation on the current pixel block to determine the target optimal position of the current pixel block based on the integer-pixel motion estimation;

[0031] Obtain the target encoding speed of the video application, and determine the target accuracy of the corresponding pixel-level motion estimation based on the target encoding speed;

[0032] In order of increasing pixel motion estimation accuracy, pixel motion estimation is performed with the target optimal position determined by the previous pixel motion estimation as the center, to obtain the target optimal position of the current pixel motion estimation accuracy, until the pixel motion estimation accuracy of the current pixel motion estimation reaches the target accuracy and then the pixel motion estimation stops.

[0033] The optimal position of the target, estimated using the pixel-level motion with the target precision, is taken as the motion estimation result of the current pixel block.

[0034] The aforementioned video image processing methods, apparatuses, computer devices, storage media, and computer program products, for different encoding speed requirements, determine the target accuracy for pixel-level motion estimation. For the current pixel block, integer-level motion estimation is performed first. Then, pixel-level motion estimation is performed in ascending order of accuracy. Each pixel-level motion estimation is centered on the target optimal position determined by the previous pixel motion estimation, obtaining the target optimal position for the current accuracy. This process continues until the accuracy of the current pixel-level motion estimation reaches the target accuracy, at which point the pixel-level motion estimation stops. By skipping higher-precision pixel-level motion estimations when the target accuracy corresponding to the encoding speed is reached, encoding efficiency can be improved while achieving accuracy matching the encoding speed. This allows for flexible pixel-level motion estimation with different accuracies based on different encoding speeds. This method can be flexibly adjusted according to different encoding speed requirements to meet business application scenarios with different encoding speeds and compression efficiencies. Attached Figure Description

[0035] Figure 1 This is an application environment diagram of a video image processing method in one embodiment;

[0036] Figure 2 This is a flowchart illustrating a video image processing method in one embodiment;

[0037] Figure 3 This is a flowchart illustrating the steps of performing sub-pixel motion estimation on the target's optimal position with the current precision, centered on the target's optimal position determined by the previous pixel motion estimation, in one embodiment.

[0038] Figure 4 This is a schematic diagram of the target search point set in another embodiment;

[0039] Figure 5 This is a flowchart illustrating a video image processing method in another embodiment;

[0040] Figure 6 This is a structural block diagram of a video image processing device in one embodiment;

[0041] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0043] The video image processing method provided in this application can be applied to business scenarios of video applications, including live video streaming, video-on-demand, and cloud gaming. Cloud gaming, also known as gaming on demand, is an online gaming technology based on cloud computing. Cloud gaming technology enables thin clients with relatively limited graphics processing and data computing capabilities to run high-quality games. In cloud gaming scenarios, the game does not run on the player's gaming terminal but on a cloud server. The cloud server renders the game scene as a video and audio stream, which is then transmitted to the player's gaming terminal via the network. The player's gaming terminal does not need powerful graphics processing and data computing capabilities; it only needs basic streaming media playback capabilities and the ability to obtain player input commands and send them to the cloud server.

[0044] The video image processing method of this application can be applied to video coding processing, such as VVC / H.266 encoders. Currently, common video coding and decoding standards are based on a block-based hybrid coding framework. Each frame in a video image is divided into square Largest Coding Units (LCUs) of the same size (e.g., 128×128, 64×64, etc.). Each LCU can be further divided into rectangular Coding Units (CUs) according to rules; and the Coding Units may be further divided into smaller Prediction Units (PUs). Specifically, the hybrid coding framework may include modules such as prediction, transformation, quantization, entropy coding, and in-loop filtering; among them, the prediction module may include intra-prediction and inter-prediction, and inter-prediction may include motion estimation and motion compensation. Because there is a strong correlation between adjacent pixels within a video frame, intra-frame prediction can eliminate spatial redundancy between adjacent pixels in video encoding and decoding technology. However, because there is also a strong similarity between adjacent frames in a video image, inter-frame prediction can eliminate temporal redundancy between adjacent frames in video encoding and decoding technology, thereby improving coding efficiency.

[0045] The video image processing method provided in this application embodiment can be applied to, for example... Figure 1The video live streaming application environment shown is illustrated. The live streaming terminal 102 and the viewing terminal 106 communicate with the server 104 via a network. A data storage system can store the data that the server 104 needs to process. The data storage system can be integrated onto the server 104, or it can be located in the cloud or on another server. The live streaming terminal 102 collects video data, encodes the video data to obtain a bitstream, and then transmits the bitstream to the server and each viewing terminal. The encoding process of the live streaming terminal, simply put, includes source video pixel data segmentation → prediction residual → DCT / integer transform → quantization → entropy coding. The prediction residual is achieved through motion estimation and motion compensation.

[0046] Motion estimation, simply put, involves finding the optimal matching block for the current block within a reference frame, representing the positional relationship between the matching block and the current block using a motion vector (MV). Motion compensation, simply put, involves calculating the difference between the matching block and the current block to obtain a residual for transmission.

[0047] The process involves the live streaming terminal acquiring the current pixel block of the current image frame in the video image to be encoded; performing integer-pixel motion estimation on the current pixel block to determine the target optimal position of the current pixel block based on the integer-pixel motion estimation; acquiring the target encoding speed of the video application and determining the target precision of the corresponding sub-pixel motion estimation based on the target encoding speed; performing sub-pixel motion estimation in ascending order of sub-pixel motion estimation precision, using the target optimal position determined by the previous pixel motion estimation as the center, to obtain the target optimal position of the current precision sub-pixel motion estimation, until the current precision of the sub-pixel motion estimation reaches the target precision, at which point the sub-pixel motion estimation stops; and using the target optimal position obtained from the sub-pixel motion estimation at the target precision as the motion estimation result of the current pixel block. The live streaming terminal 102 can be, but is not limited to, various desktop computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. The server 104 can be implemented using a standalone server or a server cluster composed of multiple servers.

[0048] In one embodiment, such as Figure 2 As shown, a video image processing method is provided, which can be applied to... Figure 1 Taking the terminal in the example, the explanation includes the following steps:

[0049] Step 202: Obtain the current pixel block of the current image frame in the video image to be encoded.

[0050] In video sequences, images exhibit strong temporal correlation, meaning adjacent frames are largely similar. Inter-frame prediction eliminates temporal redundancy, thus compressing the video. Motion compensation (MC) is a method for describing the differences between adjacent frames. Through motion compensation, the encoder can eliminate redundant information between frames by encoding the differences. Motion estimation (ME) refers to the method of searching for the most matching block position in neighboring frames for each block. The relative spatial offset between the two is called the motion vector (MV).

[0051] The basic idea of ​​motion estimation is to divide each video frame of the image sequence into many non-overlapping prediction units, which are pixel blocks. It is assumed that the displacement of all pixels in the prediction unit is the same. Then, for each prediction unit, within a certain specific search range to the video reference frame, the block most similar to the current pixel block is found according to a certain matching criterion. This block is called the matching block (i.e., the reference prediction unit). The relative position of the matching block and the current pixel block is the motion vector. During video compression, only the motion vector and residual data need to be saved to completely recover the current block. Motion estimation can remove inter-frame redundancy, which greatly reduces the number of bits transmitted in the video.

[0052] In this embodiment, the current image frame of the video frame to be encoded is divided into multiple pixel blocks so that motion estimation can be performed on each pixel block.

[0053] Step 204: Perform integer pixel motion estimation on the current pixel block to determine the optimal target position of the current pixel block based on the integer pixel motion estimation.

[0054] Motion estimation can include integer pixel motion estimation and fractional pixel motion estimation.

[0055] Specifically, pixel motion estimation is performed using reference frames. Reference frames can be pre-encoded image frames or original image frames. Since temporally adjacent reference frames are more likely to contain similar image patches, these adjacent frames are generally the primary source of reference frames. Integer pixel motion estimation is pixel motion estimation with the video frame size set to the original size. After selecting a reference frame, the matching block in integer pixel motion estimation moves in one-pixel increments. Within a certain search window in the reference frame, the block most similar to the current pixel block is found according to a certain matching criterion; this is the best matching block. The center of the best matching block is taken as the target optimal position of the current pixel block in integer pixel motion estimation. The relative displacement between the best matching block and the current pixel block is the motion vector.

[0056] Step 206: Obtain the target encoding speed of the video application, and determine the target accuracy of the corresponding pixel-level motion estimation based on the target encoding speed.

[0057] Fractional pixel motion estimation can include 1 / 2 fractional pixel motion estimation, 1 / 4 fractional pixel motion estimation, 1 / 8 fractional pixel motion estimation, and 1 / 16 fractional pixel motion estimation, and even higher precision (smallest fractional pixel value) fractional pixel motion estimation. The values ​​1 / 2, 1 / 4, 1 / 8, and 1 / 16 represent the precision of the fractional pixel motion estimation; the larger the value, the lower the precision. Specifically, 1 / 2 fractional pixel motion estimation involves magnifying the video reference frame by two times and interpolating it. Therefore, the 1 / 2 reference prediction unit (matching block) movement step size in this 1 / 2 pixel motion estimation is half a pixel, achieving more accurate search position finding than integer pixel motion estimation. 1 / 4 pixel motion estimation involves magnifying the video reference frame by four times and interpolating it. Therefore, the 1 / 4 reference prediction unit movement step size in this 1 / 4 pixel motion estimation is one-quarter of a pixel, achieving even more accurate search position finding than 1 / 2 pixel motion estimation. 1 / 8 pixel motion estimation involves magnifying the video reference frame eight times and interpolating it. Therefore, the 1 / 8 reference prediction unit in this method has a movement step of one-eighth of a pixel, achieving more precise search position finding than 1 / 4 pixel motion estimation. Similarly, 1 / 16 pixel motion estimation involves magnifying the video reference frame sixteen times and interpolating it. Therefore, the 1 / 16 reference prediction unit in this method has a movement step of one-sixteenth of a pixel, achieving even more precise search position finding than 1 / 8 pixel motion estimation.

[0058] Among them, the smaller the value of the pixel motion estimation, the higher the accuracy, the better the compression effect, and the slower the corresponding encoding speed.

[0059] Different video application scenarios often have different encoding speed requirements. For example, live streaming scenarios with high real-time requirements often require higher encoding speeds to meet these requirements. In contrast, offline transcoding scenarios can use lower encoding speeds.

[0060] Based on the relationship between pixel-level motion estimation accuracy and encoding speed, target encoding speeds can be preset for different video application scenarios, and a correspondence between the target encoding speed and the accuracy of pixel-level motion estimation can be set. For example, if a high target encoding speed is required, the corresponding pixel-level motion estimation accuracy can be set to low. Conversely, if a slow target encoding speed is required, the corresponding pixel-level motion estimation accuracy can be set to high. For instance, in a live video application scenario, the target encoding speed can be 20-30 fps, and the corresponding target accuracy of pixel-level motion estimation can be 1 / 2 pixel or 1 / 4 pixel. For a video-on-demand application scenario, the target encoding speed can be 1-5 fps, and the corresponding target accuracy of pixel-level motion estimation can be 1 / 8 pixel or 1 / 16 pixel.

[0061] It should be noted that the step numbers in this embodiment are not intended to limit the order of the steps. For example, the step of obtaining the target encoding speed of the video application and determining the target accuracy of the corresponding subpixel motion estimation based on the target encoding speed can be before or after the step of obtaining the current pixel block of the current image frame in the video image to be encoded.

[0062] Step 208: Based on the pixel motion estimation accuracy from low to high, perform pixel motion estimation with the target optimal position determined by the previous pixel motion estimation as the center, and obtain the target optimal position of the current pixel motion estimation accuracy. Stop pixel motion estimation when the accuracy of the current pixel motion estimation reaches the target accuracy.

[0063] Specifically, the order of pixel-level motion estimation accuracy from low to high corresponds to the order of pixel fraction values ​​from large to small, for example, performing pixel-level motion estimation in the order of 1 / 2, 1 / 4, 1 / 8 to 1 / 16. During each pixel-level motion estimation, the current accuracy is calculated from the optimal target position of the previous pixel-level motion estimation, resulting in the current optimal target position. Specifically, the pixel positions of corresponding fractions within a certain range centered on the optimal target position of the previous pixel-level motion estimation are obtained, and sub-pixel matching blocks corresponding to these fractions are obtained through interpolation. The encoding cost of each pixel within this range is calculated, and the point with the lowest encoding cost is taken as the current optimal target position for pixel-level motion estimation. Taking 1 / 2-pixel motion estimation as an example, the previous pixel motion estimation was an integer pixel motion estimation. For the 1 / 2 pixels within a certain range centered on the target optimal position of the integer pixel motion estimation, interpolation is performed to obtain the sub-pixel matching blocks corresponding to each 1 / 2 pixel. The encoding cost of each point (1 / 2 pixel and 1 integer pixel) within the range is calculated, and the point with the smallest encoding cost is taken as the target optimal position of the 1 / 2-pixel motion estimation. Taking 1 / 4-pixel motion estimation as an example, the previous pixel motion estimation was a 1 / 2-pixel motion estimation. For the 1 / 4 pixels within a certain range centered on the target optimal position of the 1 / 2-pixel motion estimation, interpolation is performed to obtain the sub-pixel matching blocks corresponding to each 1 / 4 pixel. The encoding cost of each point (1 / 4 pixel and 1 integer pixel) within the range is calculated, and the point with the smallest encoding cost is taken as the target optimal position of the 1 / 4-pixel motion estimation. 1 / 8-pixel motion estimation and 1 / 16-pixel motion estimation follow the same principle and will not be elaborated further here.

[0064] In this embodiment, pixel-by-pixel motion estimation is performed in ascending order of accuracy until the current pixel-by-pixel motion estimation accuracy reaches the target accuracy, at which point pixel-by-pixel motion estimation stops. That is, pixel-by-pixel motion estimation stops when the current accuracy reaches the target accuracy, and higher-precision pixel-by-pixel motion estimation ceases. For example, the target accuracy for pixel-by-pixel motion corresponding to the target encoding speed of a video application is 1 / 4 pixel. Following the ascending order of pixel-by-pixel motion estimation accuracy, 1 / 2 pixel-by-pixel motion estimation is performed centered on the target optimal position determined by integer pixel motion estimation, resulting in the target optimal position for 1 / 2 pixel-by-pixel motion estimation. Then, 1 / 4 pixel-by-pixel motion estimation is performed centered on the target optimal position determined by 1 / 2 pixel-by-pixel motion estimation, resulting in the target optimal position for 1 / 4 pixel-by-pixel motion estimation. At this point, the target accuracy for pixel-by-pixel motion estimation has been reached, and higher-precision pixel-by-pixel motion estimation stops.

[0065] Therefore, in this embodiment, at least one pixel-level motion estimation is performed, which can guarantee the encoding accuracy to a certain extent. Furthermore, pixel-level motion estimation stops when the accuracy of the current pixel-level motion estimation reaches the target accuracy; that is, when the target accuracy (pixel-level accuracy corresponding to the encoding speed) is reached, higher-precision pixel-level motion estimation is skipped, thus meeting the encoding speed requirements and improving compression efficiency. This method can be flexibly adjusted according to different encoding speed requirements to meet business application scenarios with different encoding speeds and compression efficiencies.

[0066] Step 210: Use the target's optimal position, estimated by pixel-level motion with target precision, as the motion estimation result for the current pixel block.

[0067] Specifically, the optimal matching block can be determined based on the target's optimal position estimated from pixel-level motion with target precision. Then, a prediction residual is calculated based on the difference between the optimal matching block and the current pixel block for motion compensation. Motion compensation (MC) is a method for describing the differences between adjacent frames. Through motion compensation, the encoder can eliminate redundant information between frames by encoding the differences between them. Therefore, motion compensation can obtain a residual by calculating the difference between the matching block and the current pixel block for transmission.

[0068] Taking H.265 / HEVC encoding as an example, in H.265 / HEVC, motion estimation is performed, and an already encoded image is selected as a reference image. The selected reference image is used as the best matching block for the current image's coding block, and the offset of the matching block is calibrated. This offset is called the motion vector. Based on the motion estimation results, the prediction residual is calculated according to the difference between the best matching block and the matching block, in order to perform motion compensation.

[0069] The aforementioned video image processing method, for different encoding speed requirements, determines the target accuracy for pixel-level motion estimation. For the current pixel block, integer-level motion estimation is performed first. Then, pixel-level motion estimation is performed in ascending order of accuracy. Each pixel-level motion estimation is centered on the target optimal position determined by the previous pixel motion estimation, obtaining the target optimal position for the current accuracy. This process continues until the accuracy of the current pixel-level motion estimation reaches the target accuracy, at which point the pixel-level motion estimation stops. By skipping higher-precision pixel-level motion estimations when the target accuracy corresponding to the encoding speed is reached, encoding efficiency is improved while achieving accuracy matching the encoding speed. This allows for flexible pixel-level motion estimation with different accuracies based on different encoding speeds. This method can be flexibly adjusted according to different encoding speed requirements to meet business application scenarios with different encoding speeds and compression efficiencies.

[0070] Analysis of the video images reveals that the image changes between adjacent frames are gradual and exhibit a directional trend. For example, the position of a vehicle changes depending on the camera's movement in adjacent frames. Considering the direction and speed of these changes, a skipping strategy can be employed during the pixel-by-pixel point matching process to skip certain points and improve encoding speed.

[0071] Specifically, such as Figure 3 As shown, using the target's optimal position determined by the previous pixel motion estimation as the center, pixel-by-pixel motion estimation is performed to obtain the target's optimal position with the current precision pixel-by-pixel motion estimation, including:

[0072] Step 302: Using the optimal target position determined by the previous pixel motion estimation as the center, a preset number of sub-pixel points with the current precision are used as the target search point set.

[0073] Specifically, a predetermined number of pixels with current precision, centered on the target's optimal position determined by the previous pixel motion estimation, are used as the target search point set. The specific predetermined number can be determined based on the search range of the matching block. For example, if the search range of the matching block is 3×3, then the target search point set is the 8-neighborhood pixels with current precision centered on the target's optimal position. Figure 4 As shown, the target's optimal position C is taken as the center, and the current precision sub-pixels of the 8 neighboring pixels around C are used as the target search points. Taking a current precision of 1 / 2 as an example, the target's optimal position C is taken as the center, and half of the pixels in the 8 neighboring pixels around C are used as the target search points. Taking a current precision of 1 / 4 as an example, the target's optimal position C is taken as the center, and one-quarter of the pixels in the 8 neighboring pixels around C are used as the target search points.

[0074] Step 304: Search the first part of the target search points in the target search point set according to the preset search direction of pixel arrangement, and calculate the matching cost of the first part of the target search points.

[0075] Pixel arrangement typically has two dimensions: horizontal and vertical. Preset search directions for pixel arrangement can include horizontal and vertical searches. A horizontal search searches the pixel arrangement sequentially along its horizontal axis, for example, searching the first row of pixels, then the second row, and so on. This sequential search can be from top to bottom or bottom to top. A vertical search searches the pixel arrangement vertically, for example, searching the first column of pixels, then the second column, and so on. This sequential search can be from left to right or right to left.

[0076] Typically, the search process for the best matching block includes: calculating the matching cost between the matching block corresponding to pixel I in the reference frame and the current pixel block, with the matching block corresponding to the point with the smallest cost being the best matching block. When searching the first part of the target search points in the target search point set along a preset search direction arranged by pixels, the matching cost of each target search point is calculated. Taking a horizontal search as an example, the first part of the target search points in the target search point set is searched horizontally along the pixel arrangement, and the cost of each target search point is calculated.

[0077] The first part of the target search points refers to a subset of the target search points, not all of them. For example, the first subset of search points can be determined based on a preset search direction with pixel arrangement. For instance, in a left-to-right vertical search, the target search points in the first and second columns are used as the first subset. Similarly, in a top-to-bottom horizontal search, the target search points in the first and second rows are used. In other words, the search is not performed on all target pixels, but rather on a subset of them first.

[0078] Step 306: When the matching cost in the preset search direction arranged by pixels shows an upward trend, stop estimating the remaining target search points in the target search point set, and take the point with the smallest matching cost among the first part of the target search points and the target optimal position determined by the previous pixel motion estimation as the target optimal position of the current precision pixel motion estimation.

[0079] The trend of matching cost change along the search direction in terms of pixel arrangement can be determined based on the trend of the average or minimum matching cost along the search direction. For example, as the search direction of pixel arrangement progresses, the trend of the average matching cost of each row / column pixel is taken as the trend of matching cost change along the search direction, and the trend of the minimum matching cost of each row / column pixel is taken as the trend of matching cost change along the search direction.

[0080] When the matching cost increases along the search direction, it can be assumed that the objects in the image have a tendency to change in the opposite direction of the search direction along the search direction of the pixel arrangement. If the search continues along the search direction, the matching cost of the remaining target search points will continue to increase. Therefore, the value of searching the remaining target search points is not great, so the remaining target search points can be skipped and no longer searched, thereby improving the efficiency of sub-pixel motion estimation and thus improving the overall coding efficiency.

[0081] When only the first part of the target search points are estimated, the point with the minimum matching cost between the first part of the target search points and the target optimal position determined by the previous pixel motion estimation is taken as the target optimal position of the current precision pixel motion estimation.

[0082] In this embodiment, for the target search point set, the first part of the target search points is searched according to a preset search direction based on pixel arrangement, and the matching cost of the first part of the target search points is calculated. When it is determined that the matching cost is increasing according to the search direction, it indicates that the change trend of the object in the image is opposite to the search direction, and the matching cost of the remaining target search points will continue to increase. Therefore, the remaining target search point set can be skipped, and the remaining target search points are no longer searched. On the one hand, when it is determined that the matching cost is increasing according to the search direction, the strategy of skipping the remaining target search points improves the efficiency of sub-pixel motion estimation, thereby improving the overall coding efficiency. On the other hand, the judgment principle for skipping the remaining target search points is based on the change trend of the matching cost of the target search points according to the search direction, based on the change trend of two adjacent frames. Therefore, skipping the remaining target search points does not affect the motion estimation accuracy.

[0083] Please continue reading. Figure 3 The method also includes:

[0084] Step 308: When the preset search direction of the matching cost arranged by pixels shows a non-increasing trend, search the remaining target search points in the target search point set and calculate the matching cost of the remaining target search points.

[0085] Specifically, non-increasing trends include decreasing trends and fluctuating trends. When the matching cost shows a decreasing trend in the preset search direction arranged by pixels, it indicates that the matching cost of the remaining target search points may be even lower. When the matching cost shows a fluctuating trend in the preset search direction arranged by pixels, it indicates that the matching cost of the remaining target search points is volatile. In both cases, it is impossible to accurately predict the change in the matching cost of the remaining target search points. In this situation, the search continues for the remaining target search points to obtain their matching costs.

[0086] If the matching cost decreases in the preset search direction arranged by pixels, it means that the matching cost of the remaining target search points may be smaller, and the best matching point may be among the remaining target search points. If the matching cost fluctuates in the preset search direction arranged by pixels, the location of the best matching point cannot be determined. In this case, increasing the estimation of the remaining target search points can achieve an acceleration effect with minimal misjudgment.

[0087] Step 310: Take the point with the minimum matching cost among the target search point set and the target optimal position determined by the previous pixel motion estimation as the target optimal position of the current precision pixel motion estimation.

[0088] When estimating the full set of target search points, the point with the minimum matching cost between the target search point set and the target optimal position determined by the previous pixel motion estimation is taken as the target optimal position for the current pixel motion estimation.

[0089] In this embodiment, by estimating the remaining target search points in the target search point set when the preset search direction of the matching cost arranged by pixels shows a non-increasing trend, the accuracy of pixel-by-pixel motion estimation can be improved.

[0090] In one embodiment, the method for determining the first part of target search points includes: grouping the target search point set according to the pixel arrangement direction to obtain N groups of search points; and taking at least the first two groups of search points in the preset search direction from the N groups of search points as the first part of target search points.

[0091] This involves sorting the target search point set horizontally by pixel arrangement, resulting in N groups of search points corresponding to each row of pixels; that is, one row of pixels constitutes one group of search points. Figure 4 For example, pixels (4, 0, 5) can be used as one set of search points, pixels (1, 3) as another, and pixels (6, 2, 7) as yet another. Alternatively, the target search point set can be sorted vertically by pixel arrangement to obtain N sets of search points for each column of pixels; that is, one column of pixels constitutes one set of search points. Figure 4 For example, pixels (4, 1, 6) are used as a set of search points, pixels (0, 2) are used as a set of search points, and pixels (5, 3, 7) are used as a set of search points.

[0092] Since the first set of target search points are the target search points estimated earlier, at least the first two sets of search points in the preset search direction from the N sets of search points can be used as the first set of target search points. Because the first set of target search points are at least the first two sets of search points in the preset search direction, it can be ensured that the first target search points are the points searched first in the target search point set.

[0093] In this embodiment, by grouping the target search point set according to the pixel arrangement direction, the first part of the target search points can be determined based on the search direction. Since the search direction is consistent with the pixel arrangement direction, the first part of the target search points can be accurately obtained.

[0094] At this point, when the search point with the lowest matching cost among the first part of the target search points is in the first group of the search direction, it is determined that the preset search direction with matching cost arranged by pixels shows an upward trend.

[0095] by Figure 4For example, a row of pixels is considered as a group of search points, including three groups: pixel group (4, 0, 5), pixel group (1, 3), and pixel group (6, 2, 7). If the preset search direction is from top to bottom, the target pixels in pixel group (4, 0, 5) and pixel group (1, 3) are used as the first part of the target search points. If the preset search direction is from bottom to top, the target pixels in pixel group (6, 2, 7) and pixel group (1, 3) are used as the first part of the target search points.

[0096] by Figure 4 For example, a column of pixels is considered as a group of search points, including three groups: pixel group (4, 1, 6), pixel group (0, 2), and pixel group (5, 3, 7). If the preset search direction is from left to right, the target pixels in pixel group (4, 1, 6) and pixel group (0, 2) are used as the first part of the target search points. If the preset search direction is from bottom to top, the target pixels in pixel group (5, 3, 7) and pixel group (0, 2) are used as the first part of the target search points.

[0097] Since the first part of the target search points are at least the first two groups of search points in the preset search direction among the N groups of search points, if the search point with the smallest matching cost in the first part of the target search points is in the first group in the search direction, that is, the search point with the smallest matching cost in the first part of the target search points is in the first row / column in the search direction, the matching cost of the search points in the corresponding rows / columns of other groups is relatively large. It can be determined that the matching cost in the preset search direction arranged by pixels shows an upward trend.

[0098] To facilitate comparison, the target search points for each group can be placed in a separate array. The matching cost for at least the first two pixel groups is then calculated. If the minimum matching cost among the first two pixel groups is in the first array, it indicates that the matching cost is increasing according to the preset search direction based on pixel arrangement. In this case, the estimation of other pixel groups can be skipped. For example, consider a column of pixels as a group of search points, including three groups: pixel group (4, 1, 6), pixel group (0, 2), and pixel group (5, 3, 7). If the preset search direction is from left to right, the target pixels in pixel groups (4, 1, 6) and (0, 2) are used as the first part of the target search points. When the search point with the minimum matching cost in the first part is in the first group (4, 1, 6), the search for the third pixel group (5, 3, 7) is skipped.

[0099] When the search point with the lowest matching cost in the first part of the target search points is not in the first group along the search direction, it is determined that the preset search direction with the matching cost arranged by pixels has a non-increasing trend. For example, a column of pixels is used as a group of search points, including three groups of search points: pixel group (4, 1, 6), pixel group (0, 2), and pixel group (5, 3, 7). If the preset search direction is from left to right, then the target pixels in pixel groups (4, 1, 6) and pixel group (0, 2) are used as the first part of the target search points. When the search point with the lowest matching cost in the first part of the target search points is in the second group (0, 2), the search continues for the third group of pixel groups (5, 3, 7).

[0100] In this embodiment, by determining the position of the search point with the lowest matching cost among the first part of the target search points on the target search point group, the changing trend of the preset search direction with matching cost arranged by pixels is quickly determined, and some search points are skipped.

[0101] In another embodiment, the video image processing method further includes: when the matching error between the best matching block where the optimal position of the target in the integer pixel motion estimation is located and the current pixel block is less than a threshold, taking the optimal position of the target in the integer pixel motion estimation as the motion estimation result of the current pixel block.

[0102] In other words, when the optimal target position in integer pixel motion estimation meets certain conditions, pixel-by-pixel motion estimation is not performed; instead, the optimal target position in integer pixel motion estimation is used as the motion estimation result for the current pixel block. This condition is that the matching error between the best matching block containing the optimal target position and the current pixel block is less than a threshold.

[0103] In motion estimation, parameters such as Minimum Mean Square Error (MSE), Minimum Mean Absolute Difference (MAD), and Sum of Absolute Difference (SAD) are commonly used as criteria to evaluate the similarity between two blocks. The Minimum SAD criterion is the most widely used because it does not involve multiplication or division, making it easier to implement in hardware. In this embodiment, the matching error can be the sum of absolute errors. When the sum of absolute errors between the best matching block where the target's optimal position in integer pixel motion estimation is located and the pixel block is less than a threshold, it indicates a high degree of matching. Further improving the matching accuracy then yields significantly less benefit, as expending a large amount of computation for such a small improvement in accuracy becomes very inefficient, especially for 16x16 pixel blocks. Specifically, the threshold can be 512 in the fast mode and 256 in the slow mode.

[0104] In this embodiment, during integer pixel motion estimation, the sum of absolute errors (SAD) between the best matching block and the current pixel block is calculated. The SAD is compared with a set threshold, and a decision is made based on the comparison result to skip sub-pixel motion estimation. Specifically, sub-pixel motion estimation is skipped if the SAD is less than the set threshold, thereby improving the flexibility and efficiency of pixel motion estimation.

[0105] In one embodiment, the optimal target position determined by the previous pixel motion estimation is used as the center, and the sub-pixels with the current precision in the 8-neighborhood of this center point are used as the target search point set. The target search point set is divided into three groups horizontally according to the pixel arrangement, and the search direction is from top to bottom. A video image processing method is described as follows. Figure 5 As shown, it includes:

[0106] Step 501: Obtain the current pixel block of the current image frame in the video image to be encoded.

[0107] Step 502: Perform integer pixel motion estimation on the current pixel block to determine the optimal target position of the current pixel block based on the integer pixel motion estimation.

[0108] Step 503: Determine whether the matching error between the best matching block containing the optimal target position of the integer pixel motion estimation and the current pixel image block is less than a threshold. If yes, proceed to step 504. If no, proceed to step 505.

[0109] Step 504: Use the optimal position of the target in the integer pixel motion estimation as the motion estimation result of the current pixel block.

[0110] In other words, when the matching error between the best matching block containing the optimal position of the target in the integer pixel motion estimation and the current pixel block is less than the threshold, the optimal position of the target in the integer pixel motion estimation is directly used as the motion estimation result of the current pixel block, thus skipping the sub-pixel motion estimation.

[0111] Step 505: Obtain the target encoding speed of the video application, and determine the target accuracy of the corresponding pixel-level motion estimation based on the target encoding speed.

[0112] Step 506: Using the optimal target position determined by the previous pixel motion estimation as the center, the sub-pixel points with the current precision of the 8-neighborhood of this center are used as the target search point set.

[0113] Taking the current accuracy of 1 / 2 pixel as an example, the 1 / 2 sub-pixel points in the 8 neighborhood centered on the optimal position of the target determined by the integer pixel motion estimation are used as the target search point set.

[0114] Step 507: Divide the target search point set into three groups according to pixel rows, search the first two groups of target search points, and calculate the matching cost of the first two groups of target search points.

[0115] Step 508: Determine if the search point with the lowest matching cost among the first two groups of target search points is in the first group. If yes, proceed to step 509; otherwise, proceed to step 510.

[0116] Step 509: The point with the minimum matching cost among the first two sets of target search points and the target optimal position determined by the previous pixel motion estimation is taken as the target optimal position for the current precision pixel motion estimation. After step 509, step 512 is executed.

[0117] Step 510: Match the third set of target search points to obtain the matching cost of the third set of target search points.

[0118] Step 511: Take the point with the smallest matching cost among the three sets of target search points and the target optimal position determined by the previous pixel motion estimation as the target optimal position of the current precision pixel motion estimation.

[0119] In the pixel-by-pixel estimation process, the trend of the matching cost is determined by the location of the minimum matching cost of the first two sets of target search points, which helps to determine whether some search points need to be skipped, thus improving search efficiency.

[0120] Step 512: Determine whether the accuracy of the current pixel-level motion estimation has reached the target accuracy. If yes, proceed to step 513; otherwise, proceed to step 514.

[0121] Step 513: Use the target's optimal position, estimated by pixel-level motion with target precision, as the motion estimation result for the current pixel block.

[0122] Step 514: Update the current pixel motion estimation accuracy, and after step 514, return to step 506 to perform the next level of pixel motion estimation accuracy.

[0123] For example, if the current precision is 1 / 2 pixel and the target precision is 1 / 4 pixel, and the current pixel precision has not yet reached the target precision, then step 514 is executed to update the current pixel precision to 1 / 4, and the process is returned to step 506 to perform 1 / 4 pixel motion estimation. This allows for adjusting the pixel motion estimation to a higher precision according to the encoding speed requirements, thereby improving the efficiency of pixel motion estimation.

[0124] In video coding, fractional pixel motion estimation is often used after integer pixel motion estimation to more accurately determine the position of the matching pixel block. It is also frequently used in temporal filtering during video encoding preprocessing to find the best-matching block, effectively removing noise from the image. This saves codewords needed for compression and provides better reference for other video frames, improving the encoder's compression efficiency. Traditional methods use fixed-precision searching at a few points around integer pixels. This application introduces an algorithm combining early termination skipping of multiple parts and configurable fractional pixel precision to achieve the optimal compression ratio for different speed performance requirements. Specifically:

[0125] 1. When the matching error between the best matching block containing the optimal position of the target in the integer pixel motion estimation and the current pixel block is less than a threshold, the optimal position of the target in the integer pixel motion estimation is taken as the motion estimation result of the current pixel block, thereby skipping the sub-pixel motion estimation and improving the flexibility and efficiency of pixel motion estimation.

[0126] 2. The target precision of pixel motion estimation corresponding to the encoding speed is set. After performing pixel motion estimation with the corresponding target precision, adjusting to higher precision pixel motion estimation can achieve the accuracy matching the encoding speed while improving encoding efficiency. This allows for flexible pixel motion estimation with different precisions according to different encoding speeds.

[0127] 3. During the pixel-by-pixel motion estimation process, the target search points are divided into groups. The trend of the matching cost is determined by the position of the minimum matching cost of the first two groups of target search points, which determines whether some search points need to be skipped, thus improving search efficiency.

[0128] The video image processing method of this application can adapt to the requirements of different scenarios regarding encoding speed and compression efficiency. When different encoding speeds are needed, it employs algorithms such as fractional pixel skipping, flexible configuration of fractional pixel precision, and search point skipping to meet speed requirements with minimal compression performance loss, thus achieving better savings in bandwidth and other resources. This method can be applied to fractional pixel motion estimation in the temporal filtering stage of VVC / H.266 encoders. Compared to fixed-mode algorithms, it can bring considerable bandwidth savings to offline video-on-demand scenarios; while in live interactive scenarios, it brings several times the speed improvement, meeting real-time requirements.

[0129] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0130] Based on the same inventive concept, this application also provides a video image processing apparatus for implementing the video image processing method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more video image processing apparatus embodiments provided below can be found in the limitations of the video image processing method described above, and will not be repeated here.

[0131] In one embodiment, such as Figure 6 As shown, a video image processing apparatus is provided, comprising:

[0132] The pixel block acquisition module 602 is used to acquire the current pixel block of the current image frame in the video image to be encoded;

[0133] The integer pixel motion estimation module 604 is used to perform integer pixel motion estimation on the current pixel block and determine the target optimal position of the current pixel block based on the integer pixel motion estimation.

[0134] The speed matching module 606 is used to obtain the target encoding speed of the video application and determine the target accuracy of the corresponding pixel motion estimation based on the target encoding speed.

[0135] The pixel-by-pixel motion estimation module 608 is used to perform pixel-by-pixel motion estimation in order of increasing pixel-by-pixel motion estimation accuracy, taking the target optimal position determined by the previous pixel motion estimation as the center, to obtain the target optimal position of the pixel-by-pixel motion estimation with the current accuracy, until the accuracy of the current pixel-by-pixel motion estimation reaches the target accuracy, and then stop pixel-by-pixel motion estimation, and use the target optimal position estimated by pixel-by-pixel motion with the target accuracy as the motion estimation result of the current pixel block.

[0136] The aforementioned video image processing device, for different encoding speed requirements, determines the target accuracy for pixel-level motion estimation. For the current pixel block, integer-pixel motion estimation is performed first. Then, pixel-level motion estimation is performed in ascending order of accuracy. Each pixel-level motion estimation is centered on the target optimal position determined by the previous pixel motion estimation, obtaining the target optimal position for the current accuracy. This process continues until the accuracy of the current pixel-level motion estimation reaches the target accuracy, at which point the pixel-level motion estimation stops. By skipping higher-precision pixel-level motion estimations when the target accuracy corresponding to the encoding speed is reached, encoding efficiency can be improved while achieving accuracy matching the encoding speed. This allows for flexible pixel-level motion estimation with different accuracies based on different encoding speeds. This method can be flexibly adjusted according to different encoding speed requirements to meet business application scenarios with different encoding speeds and compression efficiencies.

[0137] In one embodiment, the pixel-level motion estimation module includes:

[0138] The search point determination module is used to match a first part of the target search points in the target search point set according to a preset search direction arranged by pixels, and to obtain the matching cost of the first part of the target search points.

[0139] The search module is used to search for a first part of the target search points in the target search point set according to a preset search direction arranged by pixels, and to calculate the matching cost of the first part of the target search points.

[0140] The search output module is used to stop searching for the remaining target search points in the target search point set when the matching cost is increasing according to the preset search direction of the pixel arrangement, and to take the point with the smallest matching cost among the first part of the target search points and the target optimal position determined by the previous pixel motion estimation as the target optimal position of the current precision pixel motion estimation.

[0141] In another embodiment, the search module is further configured to search the remaining target search points in the target search point set and calculate the matching cost of the remaining target search points when the matching cost is not increasing according to the preset search direction of the pixel arrangement.

[0142] The search output module is further configured to take the point with the minimum matching cost among the target search point set and the target optimal position determined by the previous pixel motion estimation as the target optimal position of the current precision pixel motion estimation.

[0143] In another embodiment, the search point determination module is further configured to group the target search point set according to the pixel arrangement direction to obtain N groups of search points; and to take at least the first two groups of search points in the preset search direction from the N groups of search points as the first part of the target search points.

[0144] In another embodiment, the search point determination module is further configured to determine, when matching the search point with the lowest cost in the first set of the first part of the target search points in the first group in the search direction, that the matching cost is increasing according to the preset search direction of the pixel arrangement.

[0145] In another embodiment, the search point determination module is further configured to determine that the matching cost is not increasing according to the preset search direction of the pixel arrangement when the search point with the lowest matching cost among the first part of the target search points is not in the first group in the search direction.

[0146] In another embodiment, the integer pixel motion estimation module is further configured to use the integer pixel motion estimation of the target optimal position as the motion estimation result of the current pixel block when the best matching block where the target optimal position of the integer pixel motion estimation is located and the matching error of the current pixel block are less than a threshold.

[0147] Each module in the aforementioned video image processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0148] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 7As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a video image processing method. The display unit of the computer device is used to form a visually visible image. It can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0149] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0150] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the video image processing methods described in the above embodiments.

[0151] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the video image processing methods described in the above embodiments.

[0152] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the video image processing methods described in the above embodiments.

[0153] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0154] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0155] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A video image processing method, characterized in that, The method includes: Obtain the current pixel block of the current image frame in the video image to be encoded; Perform integer-pixel motion estimation on the current pixel block to determine the target optimal position of the current pixel block based on the integer-pixel motion estimation; Based on the relationship between the accuracy of pixel-level motion estimation and the encoding speed, different encoding speeds are pre-matched to different video application scenarios. Specifically, if a fast encoding speed is required, the corresponding accuracy of pixel-level motion estimation is low; if a slow encoding speed is required, the corresponding accuracy of pixel-level motion estimation is high. Obtain the target encoding speed of the video application, and determine the target accuracy of the corresponding pixel-level motion estimation based on the target encoding speed; In order of increasing pixel motion estimation accuracy, pixel motion estimation is performed with the target optimal position determined by the previous pixel motion estimation as the center, to obtain the target optimal position of the current pixel motion estimation accuracy, until the pixel motion estimation accuracy of the current pixel motion estimation reaches the target accuracy and then the pixel motion estimation stops. The optimal position of the target, estimated using pixel-level motion with the target precision, is taken as the motion estimation result of the current pixel block. Using the target's optimal position determined by the previous pixel motion estimation as the center, perform sub-pixel motion estimation to obtain the target's optimal position with the current precision sub-pixel motion estimation, including: Using the target's optimal position determined by the previous pixel motion estimation as the center, a preset number of sub-pixels with the current precision are used as the target search point set; The first part of the target search points in the target search point set is searched according to the preset search direction of the pixel arrangement, and the matching cost of the first part of the target search points is calculated. When the matching cost increases in the preset search direction of the pixel arrangement, the search for the remaining target search points in the target search point set is stopped, and the point with the smallest matching cost among the first part of the target search points and the target optimal position determined by the previous pixel motion estimation is taken as the target optimal position of the current precision pixel motion estimation.

2. The method according to claim 1, characterized in that, The method further includes: When the matching cost is not increasing according to the preset search direction of the pixel arrangement, the remaining target search points in the target search point set are searched, and the matching cost of the remaining target search points is calculated. The point with the minimum matching cost among the target search point set and the target optimal position determined by the previous pixel motion estimation is taken as the target optimal position for the current precision pixel motion estimation.

3. The method according to claim 1 or 2, characterized in that, The method further includes: The target search point set is grouped according to the pixel arrangement direction to obtain N groups of search points; The first two sets of search points in the preset search direction from the N sets of search points are taken as the first part of the target search points.

4. The method according to claim 3, characterized in that, The method for determining whether the matching cost shows an upward trend according to the preset search direction of the pixel arrangement includes: When the search point with the lowest matching cost in the first part of the target search points is in the first group in the search direction, it is determined that the matching cost is increasing in the preset search direction according to the pixel arrangement.

5. The method according to claim 4, characterized in that, The method further includes: When the search point with the lowest matching cost among the first part of the target search points is not in the first group in the search direction, it is determined that the matching cost is non-increasing in the preset search direction according to the pixel arrangement.

6. The method according to claim 1, characterized in that, The method further includes: When the matching error between the optimal matching block where the target optimal position is located in the integer pixel motion estimation and the current pixel block is less than a threshold, the target optimal position in the integer pixel motion estimation is taken as the motion estimation result of the current pixel block.

7. A video image processing apparatus, characterized in that, The device includes: The pixel block acquisition module is used to acquire the current pixel block of the current image frame in the video image to be encoded; The integer pixel motion estimation module is used to perform integer pixel motion estimation on the current pixel block and determine the target optimal position of the current pixel block based on the integer pixel motion estimation. The speed matching module is used to preset encoding speeds for different video application scenarios based on the relationship between the accuracy of pixel-level motion estimation and the encoding speed; obtain the target encoding speed of the video application, and determine the target accuracy of the corresponding pixel-level motion estimation based on the target encoding speed; wherein, if the encoding speed is required to be fast, the corresponding pixel-level motion estimation accuracy is low; if the encoding speed is required to be slow, the corresponding pixel-level motion estimation accuracy is high. The pixel-by-pixel motion estimation module is used to perform pixel-by-pixel motion estimation in order of increasing pixel-by-pixel motion estimation accuracy, taking the target optimal position determined by the previous pixel motion estimation as the center, to obtain the target optimal position of the pixel-by-pixel motion estimation with the current accuracy, until the accuracy of the current pixel-by-pixel motion estimation reaches the target accuracy, and then stop pixel-by-pixel motion estimation, and use the target optimal position estimated by pixel-by-pixel motion with the target accuracy as the motion estimation result of the current pixel block; The pixel-level motion estimation module includes: The search point determination module is used to take the target's optimal position determined by the previous pixel motion estimation as the center and use a preset number of sub-pixel points of the current precision as the target search point set. The search module is used to search for a first part of the target search points in the target search point set according to a preset search direction arranged by pixels, and to calculate the matching cost of the first part of the target search points. The search output module is used to stop searching for the remaining target search points in the target search point set when the matching cost is increasing according to the preset search direction of the pixel arrangement, and to take the point with the smallest matching cost among the first part of the target search points and the target optimal position determined by the previous pixel motion estimation as the target optimal position of the current precision pixel motion estimation.

8. The apparatus according to claim 7, characterized in that: The search module is further configured to search the remaining target search points in the target search point set and calculate the matching cost of the remaining target search points when the matching cost is not increasing according to the preset search direction of the pixel arrangement. The search output module is further configured to take the point with the minimum matching cost among the target search point set and the target optimal position determined by the previous pixel motion estimation as the target optimal position of the current precision pixel motion estimation.

9. The apparatus according to claim 7 or 8, characterized in that, The search point determination module is also used for: The target search point set is grouped according to the pixel arrangement direction to obtain N groups of search points; The first two sets of search points in the preset search direction from the N sets of search points are taken as the first part of the target search points.

10. The apparatus according to claim 9, characterized in that, The search point determination module is also used for: When the search point with the lowest matching cost in the first part of the target search points is in the first group in the search direction, it is determined that the matching cost is increasing in the preset search direction according to the pixel arrangement.

11. The apparatus according to claim 10, characterized in that, The search point determination module is also used for: When the search point with the lowest matching cost among the first part of the target search points is not in the first group in the search direction, it is determined that the matching cost is non-increasing in the preset search direction according to the pixel arrangement.

12. The apparatus according to claim 7, characterized in that, The integer pixel motion estimation module is also used for: When the matching error between the optimal matching block where the target optimal position is located in the integer pixel motion estimation and the current pixel block is less than a threshold, the target optimal position in the integer pixel motion estimation is taken as the motion estimation result of the current pixel block.

13. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

15. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images