Method of motion estimation in a coding process and related products

Abstract

The embodiment of the application provides a motion estimation method in a coding process and a related product, which are applied to a vehicle-mounted scene, and the method comprises the following steps: obtaining a to-be-coded video frame, and determining a target coding block to be coded in the to-be-coded video frame; obtaining reference motion information of an auxiliary coding block of the target coding block; determining a target reference image search range for the target coding block from M reference image lists according to image search range information indicated by the reference motion information, wherein the reference motion information of the auxiliary coding block comprises motion information of N adjacent coding blocks adjacent to the target coding block and motion information included in a reference coding block list, and M is an integer greater than 1; and performing motion estimation on the target coding block according to the target reference image search range, so that the efficiency of motion estimation in a video frame coding process can be improved.

Description

Technical Field

[0001] This application relates to the field of video coding technology, and in particular to a motion estimation method and related products in the coding process. Background Technology

[0002] In the field of video compression, motion estimation is a widely used technique in video coding and processing. Currently, when encoders perform motion estimation on coded blocks of video frames, in order to improve coding accuracy, they typically perform motion estimation within a large search range, such as searching within images corresponding to multiple reference image lists, to complete the motion estimation for that coded block. While this method significantly improves the accuracy of motion estimation during the coding process, it results in low efficiency in motion estimation of coded blocks and consumes a large amount of computing power. Summary of the Invention

[0003] This application provides a motion estimation method and related products in the encoding process, which can improve the efficiency of motion estimation in the video frame encoding process.

[0004] On one hand, embodiments of this application provide a motion estimation method during the encoding process, the method comprising:

[0005] Acquire the video frame to be encoded, and determine the target coding block to be encoded in the video frame to be encoded;

[0006] Obtain reference motion information of auxiliary coding blocks of the target coding block. The reference motion information of the auxiliary coding block includes: motion information of N adjacent coding blocks adjacent to the target coding block and motion information included in the reference coding block list, where N is an integer greater than or equal to 1.

[0007] Based on the image search range information indicated by the reference motion information, the target reference image search range is determined for the target coding block from a list of M reference images, where M is an integer greater than 1;

[0008] Motion estimation is performed on the target coded block based on the search range of the target reference image.

[0009] In one embodiment, determining the target reference image search range for the target coding block from a list of M reference images based on the reference image search range information indicated by the reference motion information includes:

[0010] Based on the image search range information indicated by the reference motion information of the auxiliary coding block, a first condition parameter is determined. The first condition parameter includes: the number and / or area of ​​the first type of auxiliary coding block. The first type of auxiliary coding block refers to the coding block in each auxiliary coding block of the target coding block that performs a reference image search on the first reference image list in the M reference image list. The first reference image list is a forward reference list.

[0011] If the first condition parameter satisfies the first filtering condition, then the first reference image list is used as the target reference image search range for the target coding block.

[0012] In one embodiment, determining the target reference image search range for the target coding block from a list of M reference images based on the reference image search range information indicated by the reference motion information includes:

[0013] Based on the image search range information indicated by the reference motion information of the auxiliary coding block, a second condition parameter is determined. The second condition parameter includes the number and / or area of ​​the second type of auxiliary coding block. The second type of auxiliary coding block refers to the coding block in each auxiliary coding block of the target coding block that performs a reference image search on the second reference image list in the M reference image list. The second reference image list is a backward reference list.

[0014] If the second condition parameter satisfies the second filtering condition, then the second reference image list is used as the target reference image search range for the target coding block.

[0015] In one embodiment, before determining the second condition parameter based on the image search range information indicated by the reference motion information of the auxiliary coding block, the method further includes:

[0016] Determine whether the first condition parameter meets the first filtering condition;

[0017] If the first condition parameter does not meet the first screening condition, then the second condition parameter is determined by executing the image search range information indicated by the reference motion information of the auxiliary coding block;

[0018] The first condition parameter includes: the number and / or area of ​​the first type of auxiliary coding blocks, where the first type of auxiliary coding block refers to the coding block in each auxiliary coding block of the target coding block that performs a reference image search on the first reference image list in the M reference image list, and the first reference image list is a forward reference list.

[0019] In one embodiment, if the first condition parameter satisfies the first filtering condition, then the first reference image list is used as the target reference image search range for the target coding block, including:

[0020] If the first condition parameter satisfies the first screening condition, then the third condition parameter is determined according to the reference image corresponding to the reference motion information of the auxiliary coding block. The third condition parameter includes the number and / or area of ​​the third type of auxiliary coding block. The third type of auxiliary coding block refers to the coding block in each auxiliary coding block of the target coding block that performs a reference image search for the target reference image in the first reference image list.

[0021] If the third condition parameter satisfies the third filtering condition, then the target reference image in the first reference image list is used as the target reference image search range for the target coding block.

[0022] In one embodiment, if the second condition parameter satisfies the second filtering condition, then the second reference image list is used as the target reference image search range for the target coding block, including:

[0023] If the second condition parameter satisfies the second filtering condition, then the fourth condition parameter is determined according to the reference image corresponding to the reference motion information of the auxiliary coding block. The fourth condition parameter includes the number and / or area of ​​the fourth type of auxiliary coding block. The fourth type of auxiliary coding block refers to the coding block in each auxiliary coding block of the target coding block that performs a reference image search for the target reference image in the second reference image list.

[0024] If the fourth condition parameter satisfies the fourth filtering condition, then the target reference image in the second reference image list is used as the target reference image search range for the target coding block.

[0025] In one embodiment, determining the target reference image search range for the target coding block from a list of M reference images based on the reference image search range information indicated by the reference motion information includes:

[0026] Based on the image search range information indicated by the reference motion information, a fifth condition parameter is determined. The fifth condition parameter includes the number and / or area of ​​the fifth type of auxiliary coding blocks. The fifth type of auxiliary coding block refers to the coding block in each auxiliary coding block of the target coding block that performs reference image search on the reference search image in the list of M reference images.

[0027] If the fifth condition parameter satisfies the fifth filtering condition, then the reference search image is used as the target reference image search range for the target coding block.

[0028] In one embodiment, before obtaining the reference motion information of the auxiliary coding block of the target coding block, the method further includes:

[0029] Obtain feature information of the target coding block, the feature information including: the time layer of the video frame to be encoded, or the feature information including: the time layer of the video frame to be encoded and the size of the target coding block;

[0030] If the feature information of the target coding block meets the triggering condition, then the process of obtaining the reference motion information of the auxiliary coding block of the target coding block is triggered.

[0031] If the feature information of the target coding block does not meet the triggering condition, then at least the first reference image list and the second reference image list in the M reference image list shall be used as the target reference image search range of the target coding block.

[0032] In one embodiment, motion estimation of the target coded block based on the target reference image search range includes:

[0033] Determine the target reference image list corresponding to the target reference image search range and the target reference image corresponding to the target reference image list;

[0034] Based on the target reference image list and its corresponding target reference image, motion estimation is performed using the first reference image search rule to obtain the first motion estimation parameters of the target coding block;

[0035] Based on the target reference image list and its corresponding target reference image, motion estimation is performed using a second reference image search rule to obtain the second motion estimation parameters of the target coding block; the second reference image search rule is different from the first reference image search rule.

[0036] Select a target motion estimation parameter for the target coding block from the first motion estimation parameter and the second motion estimation parameter.

[0037] In one embodiment, selecting target motion estimation parameters for the target coding block from the first motion estimation parameters and the second motion estimation parameters includes:

[0038] Obtain the first rate distortion corresponding to the first motion estimation parameter and the second rate distortion corresponding to the second motion estimation parameter;

[0039] If the first rate distortion is greater than the second rate distortion, then the second motion estimation parameter corresponding to the second rate distortion is determined as the target motion estimation parameter of the target coding block;

[0040] If the second rate distortion is greater than the first rate distortion, then the first motion estimation parameter corresponding to the first rate distortion is determined as the target motion estimation parameter of the target coding block.

[0041] On one hand, embodiments of this application provide a motion estimation device for the encoding process, the device comprising:

[0042] An acquisition unit is used to acquire a video frame to be encoded and to determine a target coding block to be encoded in the video frame to be encoded.

[0043] The acquisition unit is further configured to acquire reference motion information of auxiliary coding blocks of the target coding block. The reference motion information of the auxiliary coding block includes: motion information of N adjacent coding blocks adjacent to the target coding block and motion information included in the reference coding block list, where N is an integer greater than or equal to 1.

[0044] The processing unit is configured to determine the target reference image search range for the target coding block from a list of M reference images based on the image search range information indicated by the reference motion information, where M is an integer greater than 1;

[0045] The processing unit is used to perform motion estimation on the target coded block based on the search range of the target reference image.

[0046] In one embodiment, when the processing unit determines the target reference image search range for the target coding block from a list of M reference images based on the reference image search range information indicated by the reference motion information, it is specifically used for:

[0047] Based on the image search range information indicated by the reference motion information of the auxiliary coding block, a first condition parameter is determined. The first condition parameter includes: the number and / or area of ​​the first type of auxiliary coding block. The first type of auxiliary coding block refers to the coding block in each auxiliary coding block of the target coding block that performs a reference image search on the first reference image list in the M reference image list. The first reference image list is a forward reference list.

[0048] If the first condition parameter satisfies the first filtering condition, then the first reference image list is used as the target reference image search range for the target coding block.

[0049] In one embodiment, when the processing unit determines the target reference image search range for the target coding block from a list of M reference images based on the reference image search range information indicated by the reference motion information, it is specifically used for:

[0050] Based on the image search range information indicated by the reference motion information of the auxiliary coding block, a second condition parameter is determined. The second condition parameter includes the number and / or area of ​​the second type of auxiliary coding block. The second type of auxiliary coding block refers to the coding block in each auxiliary coding block of the target coding block that performs a reference image search on the second reference image list in the M reference image list. The second reference image list is a backward reference list.

[0051] If the second condition parameter satisfies the second filtering condition, then the second reference image list is used as the target reference image search range for the target coding block.

[0052] In one embodiment, before determining the second condition parameter based on the image search range information indicated by the reference motion information of the auxiliary coding block, the processing unit is further configured to:

[0053] Determine whether the first condition parameter meets the first filtering condition;

[0054] If the first condition parameter does not meet the first screening condition, then the second condition parameter is determined by executing the image search range information indicated by the reference motion information of the auxiliary coding block;

[0055] The first condition parameter includes: the number and / or area of ​​the first type of auxiliary coding blocks, where the first type of auxiliary coding block refers to the coding block in each auxiliary coding block of the target coding block that performs a reference image search on the first reference image list in the M reference image list, and the first reference image list is a forward reference list.

[0056] In one embodiment, when the processing unit uses the first reference image list as the target reference image search range for the target coding block if the first condition parameter satisfies the first filtering condition, it is specifically used for:

[0057] If the first condition parameter satisfies the first screening condition, then the third condition parameter is determined according to the reference image corresponding to the reference motion information of the auxiliary coding block. The third condition parameter includes the number and / or area of ​​the third type of auxiliary coding block. The third type of auxiliary coding block refers to the coding block in each auxiliary coding block of the target coding block that performs a reference image search for the target reference image in the first reference image list.

[0058] If the third condition parameter satisfies the third filtering condition, then the target reference image in the first reference image list is used as the target reference image search range for the target coding block.

[0059] In one embodiment, when the processing unit uses the second reference image list as the target reference image search range for the target coding block if the second condition parameter satisfies the second filtering condition, it is specifically used for:

[0060] If the second condition parameter satisfies the second filtering condition, then the fourth condition parameter is determined according to the reference image corresponding to the reference motion information of the auxiliary coding block. The fourth condition parameter includes the number and / or area of ​​the fourth type of auxiliary coding block. The fourth type of auxiliary coding block refers to the coding block in each auxiliary coding block of the target coding block that performs a reference image search for the target reference image in the second reference image list.

[0061] If the fourth condition parameter satisfies the fourth filtering condition, then the target reference image in the second reference image list is used as the target reference image search range for the target coding block.

[0062] In one embodiment, when the processing unit determines the target reference image search range for the target coding block from a list of M reference images based on the reference image search range information indicated by the reference motion information, it is specifically used for:

[0063] Based on the image search range information indicated by the reference motion information, a fifth condition parameter is determined. The fifth condition parameter includes the number and / or area of ​​the fifth type of auxiliary coding blocks. The fifth type of auxiliary coding block refers to the coding block in each auxiliary coding block of the target coding block that performs reference image search on the reference search image in the list of M reference images.

[0064] If the fifth condition parameter satisfies the fifth filtering condition, then the reference search image is used as the target reference image search range for the target coding block.

[0065] In one embodiment, before acquiring the reference motion information of the auxiliary coding block of the target coding block, the acquisition unit is further configured to: acquire feature information of the target coding block, the feature information including: the time layer of the video frame to be encoded, or the feature information including: the time layer of the video frame to be encoded and the size of the target coding block;

[0066] The processing unit is further configured to, if the feature information of the target coding block satisfies the triggering condition, trigger the execution of obtaining the reference motion information of the auxiliary coding block of the target coding block;

[0067] If the feature information of the target coding block does not meet the triggering condition, then at least the first reference image list and the second reference image list in the M reference image list shall be used as the target reference image search range of the target coding block.

[0068] In one embodiment, when the processing unit performs motion estimation on the target coded block based on the target reference image search range, it may specifically be used to:

[0069] Determine the target reference image list corresponding to the target reference image search range and the target reference image corresponding to the target reference image list;

[0070] Based on the target reference image list and its corresponding target reference image, motion estimation is performed using the first reference image search rule to obtain the first motion estimation parameters of the target coding block;

[0071] Based on the target reference image list and its corresponding target reference image, motion estimation is performed using a second reference image search rule to obtain the second motion estimation parameters of the target coding block; the second reference image search rule is different from the first reference image search rule.

[0072] Select a target motion estimation parameter for the target coding block from the first motion estimation parameter and the second motion estimation parameter.

[0073] In one embodiment, when selecting a target motion estimation parameter for the target coding block from the first motion estimation parameter and the second motion estimation parameter, the acquisition unit is used to acquire the first rate distortion corresponding to the first motion estimation parameter and the second rate distortion corresponding to the second motion estimation parameter.

[0074] The processing unit is configured to, if the first rate distortion is greater than the second rate distortion, determine the second motion estimation parameter corresponding to the second rate distortion as the target motion estimation parameter of the target coding block; and if the second rate distortion is greater than the first rate distortion, determine the first motion estimation parameter corresponding to the first rate distortion as the target motion estimation parameter of the target coding block.

[0075] On one hand, embodiments of this application provide a computer device, which includes an input device, an output device, a processor, and a computer storage medium. The processor and the computer storage medium are interconnected. The computer storage medium is used to store a computer program, and the processor is configured to call the computer program to execute the motion estimation method in the above-described encoding process.

[0076] On one hand, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, is used to implement the motion estimation method in the above-described encoding process.

[0077] On one hand, embodiments of this application provide a computer program product, which includes a computer program; the computer program is stored in a computer-readable storage medium, and when the computer program is executed by the processor of a computer device, it performs the motion estimation method described above in the encoding process.

[0078] In this embodiment, a video frame to be encoded is acquired, and a target coding block to be encoded is determined within the video frame. Reference motion information of auxiliary coding blocks for the target coding block is acquired. Based on the image search range information indicated by the reference motion information, a target reference image search range is determined from a list of M reference images, where M is an integer greater than 1. Motion estimation is performed on the target coding block based on the target reference image search range. By using the reference motion information of the auxiliary coding blocks to reduce the number of reference images or reference lists to search for the target coding block, the search of a portion of the reference image list or reference images during motion estimation is skipped, reducing computational load, lowering the time complexity of motion estimation, and improving the efficiency of motion estimation in the video frame encoding process. Attached Figure Description

[0079] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0080] Figure 1 This is a flowchart illustrating a motion estimation method in the encoding process provided in an embodiment of this application;

[0081] Figure 2a This is a schematic diagram of an adjacent coding block adjacent to a target coding block provided in an embodiment of this application;

[0082] Figure 2b This is a schematic diagram of a general video coding framework provided in an embodiment of this application;

[0083] Figure 3 This is a flowchart illustrating another motion estimation method in the encoding process provided in an embodiment of this application;

[0084] Figure 4 This is a schematic diagram of motion estimation provided in an embodiment of this application;

[0085] Figure 5 This is a schematic diagram of the structure of a motion estimation device in the encoding process provided in an embodiment of this application;

[0086] Figure 6This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0087] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0088] This application provides a motion estimation scheme during the encoding process, which offers a fast algorithm for motion estimation during encoding, accelerating motion estimation of coded blocks in video frames. The general principle of the fast algorithm involved in this motion estimation scheme is as follows: For a video frame to be encoded, motion information of adjacent coded blocks in the temporal and spatial domains, as well as motion information included in a history-based motion vector prediction list (HMVP), can be used to perform motion estimation (ME) of the target coded block in the video frame to be encoded. First, motion information of N adjacent coded blocks adjacent to the target coded block can be obtained, and / or motion information can be obtained from the history-based motion vector prediction list (HMVP), where N can be an integer greater than or equal to 1. The history-based motion vector prediction list (hereinafter referred to as the reference coded block list) includes: coded blocks encoded before the target coded block and their motion information. Then, reference motion information for auxiliary coding blocks is determined from the motion information of N adjacent coding blocks and / or the motion information included in the reference coding list. Based on the image search range information indicated by the reference motion information of the auxiliary coding blocks, a target reference image search range is determined for the target coding block from a list of M reference images; M is an integer greater than 1. The auxiliary coding blocks can refer to all or part of the coding blocks in the N adjacent coding blocks and the reference coding lists. The target reference image search range is allowed to include only the image search range corresponding to a portion of the reference image lists in the M reference image lists. Next, motion estimation is performed on the target coding block based on the target reference image search range to obtain the target motion estimation parameters of the target coding block. By using the reference motion information of the auxiliary coding blocks, the number of reference image lists to be searched for the target coding block can be reduced, skipping the search of a portion of the reference image lists during motion estimation, thereby reducing the time complexity of motion estimation and accelerating the motion estimation of the target coding block and improving video compression efficiency.

[0089] In one embodiment, this application embodiment can skip not only a portion of the reference image list during motion estimation but also a portion of the reference images themselves. First, based on the reference motion information of the auxiliary coding block, the conditional parameters of the auxiliary coding block of the reference search image can be used when performing reference image search. When these conditional parameters meet the filtering conditions, the reference search image can be used as the target reference image search range for the target coding block. The reference search image can be one or more reference images from a list of M reference images. For example, if the list of M reference images includes reference search image 1, and the conditional parameters of the auxiliary coding block of reference search image 1 in the list of M reference images meet the filtering conditions, then reference search image 1 can be used as the target reference image search range for the target coding block. That is, when performing reference image search, reference search image 1 can be searched directly to perform motion estimation for the target coding block.

[0090] When the condition parameter does not meet the filtering criteria, it means that the auxiliary coding block rarely uses the reference search image when performing motion estimation, while the target coding block can use the auxiliary coding block as a reference and may not use the reference search image to some extent. Therefore, when performing motion estimation on the target coding block, the search for the reference search image can be skipped. For example, in the example above, if the condition parameter of the auxiliary coding block using reference search image 1 from the list of M reference images does not meet the filtering criteria, it means that the auxiliary coding block does not use reference search image 1 when performing motion estimation. When performing motion estimation on the target coding block, the search for reference search image 1 can be skipped. In this way, the number of reference images searched for the target coding block can be reduced, the time complexity of motion estimation can be reduced, and the efficiency of motion estimation and video compression can be improved.

[0091] In one embodiment, this application also proposes a motion estimation parameter search and reset mechanism to ensure the stability of the fast algorithm. Before obtaining the motion information of N adjacent coding blocks and / or the motion information of a reference coding block, it can be determined whether the target coding block can use the fast algorithm. If it is determined that the target coding block can use the fast algorithm, the motion information of N adjacent coding blocks and / or the motion information of the reference coding block can be obtained. If it is determined that the target coding block cannot use the fast algorithm, the list of M reference images is used as the target reference image search range for the target coding block. The motion estimation parameter search and reset mechanism can prevent falling into local optima when performing fast motion estimation on a coding block, that is, prevent the coding blocks included in the video frame to be encoded from always using the same list of reference images or the same reference images for motion estimation, and avoid the problem that some motion estimation parameters cannot be updated when performing motion estimation on a coding block.

[0092] The motion estimation scheme provided in this application has the following beneficial effects during the encoding process: On the one hand, the target reference image search range is determined from a list of M reference images based on the motion information of N adjacent coding blocks and / or the motion information of a reference coding block. This target reference image search range can refer to a partial list of reference images and / or a partial list of reference images within the M reference image list. By reducing the number of reference images or reference images searched for the target coding block, partial lists or reference images used in motion estimation are skipped, thereby reducing the time complexity of motion estimation, accelerating motion estimation of the target coding block, and reducing computational load. On the other hand, the motion estimation parameter search and reset mechanism in motion estimation prevents falling into local optima when performing fast motion estimation on a coding block, preventing the coding blocks included in a video frame from continuously using the same list of reference images or the same reference images for motion estimation, and avoiding the problem that some motion parameters cannot be updated when performing motion estimation on a coding block.

[0093] Based on the motion estimation scheme provided in the encoding process above, please refer to Figure 1 , Figure 1This is a flowchart illustrating a motion estimation method during the encoding process provided in this application embodiment. This motion estimation method can be executed by a computer device, which can be a terminal device or a server. More specifically, the motion estimation method during the encoding process can be executed by an encoder in the computer device. The aforementioned terminal device can be a smartphone, tablet computer, laptop computer, desktop computer, intelligent voice interaction device, smart home appliance, vehicle terminal, aircraft, etc.; the aforementioned server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms, etc. The motion estimation method during the encoding process described in this embodiment includes the following steps S101-S104.

[0094] S101: Acquire a video frame to be encoded and determine the target coding block to be encoded within the video frame. The video frame to be encoded can be a frame of image captured in real time, or it can be a frame of image captured in advance. The video frame to be encoded can be any frame from live video, vehicle driving video, real-time vehicle operation video, game video, entertainment video, etc. The video frame to be encoded can include one or more coding blocks, and the size of each coding block can be the same or different. For example, the size of coding block 1 in the video frame to be encoded can be 128, the size of coding block 2 can be 64, and so on. In one embodiment, the computer device can randomly select a coding block from the coding blocks included in the video frame to be encoded as the target coding block to be encoded, or each coding block included in the video frame to be encoded has a corresponding number, and the computer device can determine all coding blocks included in the video frame to be encoded as target coding blocks to be encoded based on the number.

[0095] S102: Obtain reference motion information of the auxiliary coding block of the target coding block. The reference motion information may include: a list of reference images, reference images in the list of reference images (or reference image indices in the list of reference images), and a motion vector (MV). The motion vector refers to the displacement between the target coding block and the reference block (the reference block is a coding block that meets a similarity threshold with the target coding block).

[0096] In one embodiment, since the motion estimation parameters of adjacent coding blocks in the temporal and spatial domains are highly correlated, and Versatile Video Coding (VVC) has a merge mode that allows the current coding block to reuse motion information from adjacent coding blocks, in this embodiment, the auxiliary coding block can be N adjacent coding blocks adjacent to the target coding block. The auxiliary coding block can be all or part of the N adjacent coding blocks, where N is an integer greater than or equal to 1. Here, "adjacent" refers to being directly adjacent to the target coding block. For example... Figure 2a In this context, N is 5, and the N adjacent coded blocks that are adjacent to the target coded block can be the lower left adjacent coded block (i.e., Figure 2a A1 in the middle), the next adjacent coded block (i.e. Figure 2a A0 in the middle), the upper left adjacent coding block (i.e. Figure 2a B2 in the middle), the upper adjacent coding block (i.e. Figure 2a B0 in the middle) and the upper right adjacent coding block (i.e. Figure 2a In B1), the corresponding reference motion information of the auxiliary coding block may include the motion information of the N adjacent coding blocks that are adjacent to the target coding block.

[0097] In another embodiment, since the merge mode in General Video Coding (VVC) introduces HMVP technology to expand the applicability of the Merge mode, the Merge mode can include motion information of coding blocks that are not directly adjacent to the target coding block in space. This motion information can describe situations where moving objects occlude each other, resulting in the same object existing in non-adjacent coding blocks. Furthermore, this HMVP technology can construct and maintain a motion information queue of length 5 (i.e., construct a reference coding block list). This reference coding block list can not only record motion information available for decoding but also serve as a reference when performing motion estimation on the current coding block (i.e., the target coding block). When encoding the current coding block, motion information from the HMVP list can be added to the Merge list in reverse order until it is full. The HMVP list is updated on a first-in, first-out basis. Therefore, in this embodiment, the auxiliary coding block can be a coding block in the reference coding block list, and this auxiliary coding block can be some or all of the coding blocks in the reference coding block list. The aforementioned reference coding block list includes coding blocks that have already been encoded before the target coding block. The reference coded block list may include N adjacent coded blocks that are directly adjacent to the target coded block (here, N adjacent coded blocks refer to coded blocks that are not directly adjacent to the target coded block in space) and coded blocks that are not directly adjacent to the target coded block in space. Accordingly, the reference motion information of the auxiliary coded blocks may include the motion information included in the reference coded block list.

[0098] In another embodiment, the N adjacent coded blocks and the coded blocks in the reference coded block list can all be used as auxiliary coded blocks, so that the reference motion information based on the auxiliary coded blocks can better determine the target reference image search range of the target coded block. Accordingly, the reference motion information of the auxiliary coded blocks may include the motion information of the N adjacent coded blocks and the motion information included in the reference coded block list, where N is an integer greater than or equal to 1.

[0099] In one embodiment, see Figure 2b , Figure 2b This is a schematic diagram of the VVC encoding framework, which includes a Decode Picture Buffer (DPB). The Decode Picture Buffer maintains reference motion information about auxiliary coding blocks. The computer device can obtain the reference motion information of the auxiliary coding blocks from the Decode Picture Buffer and then execute step S103.

[0100] S103: Based on the image search range information indicated by the reference motion information, determine the target reference image search range for the target coding block from M reference image lists, where M is an integer greater than 1. The image search range information may include the reference image list and / or reference images (reference image indices) in the reference image list. The reference image list may include at least two of a first reference image list, a second reference image list, and a bidirectional reference image list (or bidirectional reference frame list). The bidirectional reference image list includes both the first and second reference image lists. The first reference image list includes multiple frames of reference images, and the second reference image list includes multiple frames of reference images. The target reference image search range may include the target reference image list and the target reference images in the target reference image list. The target reference image list may include at least either the first or second reference image list. The first reference image list may be a forward reference image list (or list0 reference image list, or forward reference list), and the second reference image list may be a backward reference image list (or list1 reference image list, or backward reference list).

[0101] The specific implementation method of step S103 will be introduced next:

[0102] (1) Since the auxiliary coding block uses a list of reference images from the M reference image list for encoding (or motion estimation), and the auxiliary coding block is related to the target coding block, the target coding block will also largely use this list of reference images from the M reference image list for encoding. In this case, the image search range information mentioned above includes the reference image list. The reference image list included in the image search range information indicated by the reference motion information of the auxiliary coding block can reduce the number of reference images in the target coding block search, thereby accelerating the motion estimation of the target coding block and reducing the coding complexity.

[0103] In one embodiment, a computer device can determine a first condition parameter based on the image search range information indicated by the reference motion information of the auxiliary coding blocks. The first condition parameter may include: the number and / or area of ​​a first type of auxiliary coding blocks. A first type of auxiliary coding block refers to a coding block in each auxiliary coding block of the target coding block that performs a reference image search on a first reference image list from a list of M reference images; the number of first type auxiliary coding blocks can be one or more. The computer device can then determine whether the first condition parameter satisfies a first filtering condition. If the first condition parameter satisfies the first filtering condition, the first reference image list is used as the target reference image search range for the target coding block; that is, the first reference image list needs to be searched during the reference image search. If the first condition parameter does not satisfy the first filtering condition, the first reference image list is not used as the target reference image search range for the target coding block; that is, the search of the first reference image list can be skipped during the reference image search. The first filtering condition may include at least one of the following: the number of first type auxiliary coding blocks is greater than a number threshold; the ratio between the area of ​​the first type of auxiliary coding blocks and the sum of the areas of all auxiliary coding blocks is greater than an area ratio threshold. The first reference image list is a forward reference list.

[0104] When the first condition parameter is the number of first-type auxiliary coding blocks and the first filtering condition is that the number of first-type auxiliary coding blocks is greater than the number threshold, the computer device can count the number of first-type auxiliary coding blocks based on the image search range information indicated by the reference motion information of the auxiliary coding blocks, and determine whether the number of first-type auxiliary coding blocks is greater than the number threshold. If the number is greater than the number threshold, it is determined that the first condition parameter meets the first filtering condition. If the number of first-type auxiliary coding blocks is less than or equal to the number threshold, it is determined that the first condition parameter does not meet the first filtering condition.

[0105] When the first condition parameter is the area of ​​the first type of auxiliary coding block, and the first filtering condition is that the ratio between the area of ​​the first type of auxiliary coding block and the sum of the areas of all auxiliary coding blocks is greater than an area ratio threshold, the computer device can determine the first type of auxiliary coding block based on the image search range information indicated by the reference motion information of the auxiliary coding block, and calculate the area of ​​the first type of auxiliary coding block. Then, it calculates the area ratio between the area of ​​the first type of auxiliary coding block and the sum of the areas of all auxiliary coding blocks, and determines whether the area ratio is greater than the area ratio threshold. If the area ratio is greater than the area ratio threshold, the first condition parameter is determined to satisfy the first filtering condition; if the area ratio is less than or equal to the area ratio threshold, the first condition parameter is determined not to satisfy the first filtering condition. The number of first type auxiliary coding blocks can be one or more. When multiple first type auxiliary coding blocks are used, the area of ​​the first type of auxiliary coding block refers to the sum of the areas of all first type auxiliary coding blocks. For example, the first screening condition is that the ratio between the area of ​​the first type of auxiliary coding block and the sum of the areas of all auxiliary coding blocks is greater than the area ratio threshold of 0.5. The number of auxiliary coding blocks is 3 (coding block 1, coding block 2 and coding block 3 respectively). The first type of auxiliary coding blocks are coding block 1 and coding block 2. The sum of the areas of coding block 1 and coding block 2 is calculated to obtain area sum 1; the sum of the areas of coding block 1, coding block 2 and coding block 3 is calculated to obtain area sum 2. Then the computer device can calculate that the area ratio between area sum 1 and area sum 2 is 0.6. This area ratio of 0.6 is greater than the area ratio threshold of 0.5, and the first condition parameter is determined to meet the first screening condition.

[0106] The quantity threshold and area ratio threshold can be determined based on the temporal layer and / or target coding block size of the video frame to be encoded. The temporal layer can be temporal layer 0 (k=0), temporal layer 1 (k=1), temporal layer 2 (k=2), temporal layer 3 (k=3), etc. In some embodiments, the number of temporal layers can vary depending on the encoder type. For example, encoder 1 might have 3 temporal layers (i.e., temporal layers 0-2), while encoder 2 might have 4 temporal layers (i.e., temporal layers 0-3). The target coding block size can be determined based on the temporal layer of the video frame to be encoded. For example, when the temporal layer is temporal layer 0, the target coding block size can be 256; when the temporal layer is temporal layer 1, the target coding block size is 128.

[0107] In one embodiment, when the quantity threshold and area ratio threshold are determined based on the time layer of the video frame to be encoded, the larger the time layer of the video frame to be encoded, the smaller the quantity threshold and area ratio threshold. For example, taking the quantity threshold as an example, the correspondence between time layers and quantity thresholds can be preset. For example, the quantity threshold corresponding to time layer 0 is 10, and the quantity threshold corresponding to time layer 5 is 2.

[0108] In another embodiment, when the quantity threshold and area ratio threshold are determined based on the size of the coded block, the larger the size of the coded block, the smaller the aforementioned quantity threshold and area ratio threshold. For example, taking the quantity threshold as an example, a correspondence between the size of the coded block and the quantity threshold can be preset. For instance, the quantity threshold corresponding to a coded block size of 64 is 10, and the quantity threshold corresponding to a coded block size of 256 is 2. The quantity threshold is obtained by retrieving the quantity threshold corresponding to the size of the target coded block from the correspondence.

[0109] In another embodiment, when the quantity threshold and area ratio threshold are determined based on the time layer of the video frame to be encoded and the size of the target coding block, the correspondence between the quantity threshold and area ratio threshold and the time layer of the video frame to be encoded and the size of the coding block can be preset. For example, if the time layer 0 of the video frame to be encoded is 0 and the size of the coding block is 256, the corresponding quantity threshold is 10; if the time layer 1 of the video frame to be encoded is 128 and the size of the coding block is 128, the corresponding quantity threshold is 10, or if the time layer 1 of the video frame to be encoded is 128 and the size of the coding block is 128, the corresponding quantity threshold is 4, and so on. In the above cases, the quantity threshold is obtained by: obtaining the time layer of the video frame to be encoded and / or obtaining the size of the target coding block, and then determining the quantity threshold or area ratio threshold from the correspondence based on the time layer and / or the size of the target coding block. For example, if the time layer of the video frame to be encoded is 1 and the size of the target coding block is 128, the quantity threshold is determined to be 4 based on the correspondence between the time layer and the size of the target coding block. That is, the first screening condition mentioned above is that the number of first-type auxiliary coding blocks is greater than the quantity threshold of 4. Similarly, the area ratio threshold can be obtained.

[0110] In one embodiment, a second condition parameter is determined based on the image search range information indicated by the reference motion information of the auxiliary coding blocks. The second condition parameter includes the number and / or area of ​​the second type of auxiliary coding blocks. The second type of auxiliary coding block refers to the coding block in each auxiliary coding block of the target coding block that performs a reference image search on the second reference image list in the M reference image lists. Then, it is determined whether the second condition parameter satisfies the second filtering condition. If it is determined that the second condition parameter satisfies the second filtering condition, the second reference image list is used as the target reference image search range of the target coding block. If it is determined that the second condition parameter does not satisfy the second filtering condition, the second reference image list is not used as the target reference image search range, that is, the search for the second reference image list is skipped when performing the reference image search. The second filtering condition may include at least one of the following: the number of the second type of auxiliary coding blocks is greater than a number threshold, and the ratio between the area of ​​the second type of auxiliary coding block and the sum of the areas of all auxiliary coding blocks is greater than an area ratio threshold. The second reference image list is a backward reference list. The number threshold and the area ratio threshold can be determined according to the time layer of the video frame to be encoded and / or the size of the target coding block. It should be noted that the specific implementation method for determining whether the second condition parameter meets the second screening condition can be found in the above-mentioned specific implementation method for determining whether the first condition parameter meets the first screening condition.

[0111] In one embodiment, before determining the second condition parameter based on the image search range information indicated by the reference motion information of the auxiliary coding block, it can be first determined whether the first condition parameter satisfies the first filtering condition. If the first condition parameter does not satisfy the first filtering condition, the second condition parameter can be determined based on the image search range information indicated by the reference motion information of the auxiliary coding block. The second condition parameter includes: the number and / or area of ​​the second type of auxiliary coding blocks. The second type of auxiliary coding block refers to the coding block in each auxiliary coding block of the target coding block that performs a reference image search on the second reference image list in the M reference image list. If the second condition parameter satisfies the second filtering condition, the second reference image list is taken as the target reference image search range of the target coding block. If the second condition parameter does not satisfy the second filtering condition, at least the first reference image list and the second reference image list in the M reference image list are taken as the target reference image search range of the target coding block.

[0112] Since the auxiliary coding block uses a reference picture (RP) from a specific reference picture in a list of M reference pictures for encoding (or motion estimation), and the auxiliary coding block is related to the target coding block, the target coding block will also largely use a reference picture from that reference picture in the list of M reference pictures for encoding. For example, if the auxiliary coding block uses reference picture 1 from the first reference picture list for encoding, the target coding block will also largely use reference picture 1 from the first reference picture list for encoding. In this case, the image search range information mentioned above can include the reference picture list and the reference pictures (or the indices of the reference pictures) included in the reference picture list. By using the reference motion information of the auxiliary coding block to indicate the image search range information including the reference picture list and the reference pictures, the number of reference picture lists and reference pictures in the target coding block can be reduced, thereby accelerating the motion estimation of the target coding block and reducing the encoding complexity.

[0113] In one embodiment, after determining the first reference image list, the reference images in the first reference image list can be filtered to further accelerate the motion estimation of the target coding block. If it is determined that the first condition parameter satisfies the first filtering condition, then the first reference image list can be used as the target reference image search range for the target coding block. Specifically, if the first condition parameter satisfies the first filtering condition, then a third condition parameter is determined based on the reference image corresponding to the reference motion information of the auxiliary coding block. The third condition parameter includes the number and / or area of ​​a third type of auxiliary coding block. The third type of auxiliary coding block refers to the coding block in each auxiliary coding block of the target coding block that performs a reference image search for the target reference image in the first reference image list. Then, it is determined whether the third condition parameter satisfies the third filtering condition. If the third condition parameter satisfies the third filtering condition, then the target reference image in the first reference image list is not used as the target reference image search range for the target coding block. That is, when performing a reference image search, the search for the target reference image in the first reference image list is skipped, and other reference images in the first reference image list are searched instead. In other words, the above-mentioned use of the first reference image list as the target reference image search range of the target coding block specifically means that the target reference images in the first reference image list are used as the target reference image search range of the target coding block, and for the target reference images in the first reference image list, multiple auxiliary coding blocks or large-area auxiliary coding blocks of the target coding block use them as the reference image search range.

[0114] The third screening criterion includes at least one of the following: the number of third-type auxiliary coding blocks is greater than a number threshold; the area ratio between the area of ​​the third-type auxiliary coding block and the sum of the areas of all auxiliary coding blocks is greater than an area ratio threshold. The number threshold and area ratio threshold can be determined by the temporal layer of the video frame to be encoded and / or the size of the target coding block. Specifically, the larger the temporal layer, the smaller the number threshold and area ratio threshold; the larger the size of the target coding block, the smaller the number threshold and area ratio threshold.

[0115] In one embodiment, when the third condition parameter is the number of third-type auxiliary coding blocks, and the third filtering condition is that the number of third-type auxiliary coding blocks is greater than a quantity threshold, the specific implementation process for determining whether the third condition parameter satisfies the third filtering condition is as follows: If the first condition parameter satisfies the first filtering condition, then the number of third-type auxiliary coding blocks is determined based on the reference image corresponding to the reference motion information of the auxiliary coding blocks, and it is determined whether the number of third-type auxiliary coding blocks is greater than the quantity threshold. If the number is greater than the quantity threshold, then it is determined that the third condition parameter satisfies the third filtering condition. If the number is determined to be less than or equal to the quantity threshold, then it is determined that the third condition parameter does not satisfy the third filtering condition.

[0116] In one embodiment, when the third condition parameter is the area of ​​the third type of auxiliary coding block, and the third filtering condition is that the area ratio between the area of ​​the third type of auxiliary coding block and the sum of the areas of all auxiliary coding blocks is greater than an area ratio threshold, the specific implementation process for determining whether the third condition parameter satisfies the third filtering condition is as follows: If the first condition parameter satisfies the first filtering condition, then based on the reference image corresponding to the reference motion information of the auxiliary coding block, the third type of auxiliary coding block is determined, and the area of ​​the third type of auxiliary coding block and the sum of the areas of all auxiliary coding blocks are calculated. Then, the area ratio between the area of ​​the third type of auxiliary coding block and the sum of the areas of all auxiliary coding blocks is calculated, and it is determined whether the area ratio is greater than the area ratio threshold. If it is determined that the area ratio is greater than the area ratio threshold, then it is determined that the third condition parameter satisfies the third filtering condition. If it is determined that the area ratio is less than or equal to the area ratio threshold, then it is determined that the third condition parameter does not satisfy the third filtering condition. The number of third type auxiliary coding blocks can be one or more. When the number of third type auxiliary coding blocks is multiple, the area of ​​the third type of auxiliary coding block refers to the sum of the areas of all third type auxiliary coding blocks. For example, the third type of auxiliary coding blocks are auxiliary coding block 1 and auxiliary coding block 2, and all auxiliary coding blocks are auxiliary coding block 1, auxiliary coding block 2, and auxiliary coding block 3. The computer device can calculate the sum of the areas of auxiliary coding blocks 1 and 2 (1), and calculate the sum of the areas of auxiliary coding blocks 1, 2, and 3 (2). Then, it calculates the area ratio between the sum of the areas (1) and the sum of the areas (2), and determines whether this area ratio is greater than an area ratio threshold.

[0117] In one embodiment, after determining the second reference image list, the reference images in the second reference image list can be filtered to further accelerate motion estimation for the target encoding. If it is determined that the second condition parameter satisfies the second filtering condition, then the second reference image list is used as the target reference image search range for the target encoding block. A specific implementation of this can be: if the second condition parameter satisfies the second filtering condition, then a fourth condition parameter is determined based on the reference image corresponding to the reference motion information of the auxiliary encoding block. The fourth condition parameter includes: the number and / or area of ​​the fourth type of auxiliary encoding blocks. The fourth type of auxiliary encoding blocks refers to the encoding blocks in each auxiliary encoding block of the target encoding block that perform reference image search on the target reference images in the second reference image list. It is then determined whether the fourth condition parameter satisfies the fourth filtering condition. If the fourth condition parameter satisfies the fourth filtering condition, then the target reference images included in the second reference image list are used as the target reference image search range for the target encoding block. If the fourth condition parameter does not satisfy the fourth filtering condition, then the target reference images included in the second reference image list are not used as the target reference image search range for the target encoding block; that is, when performing reference image search, the search for target reference images included in the second reference image list is skipped. In other words, the above-mentioned use of the second reference image list as the target reference image search range of the target coding block specifically means that the target reference images in the second reference image list are used as the target reference image search range of the target coding block, and for the target reference images in the second reference image list, multiple auxiliary coding blocks or large-area auxiliary coding blocks of the target coding block use them as the reference image search range.

[0118] It should be noted that the method for determining whether the fourth condition parameter meets the fourth screening condition can be found in the specific implementation method for determining whether the third condition parameter meets the third screening condition, as described above. The specific methods for obtaining the quantity threshold and area ratio threshold mentioned above can be found in the corresponding implementation methods, and will not be repeated here.

[0119] In one embodiment, there may be a situation where the first condition parameter does not meet the first filtering condition, and the second condition parameter does not meet the second filtering condition. In this case, it means that the target reference image search range cannot be determined for the target coding block from the list of M reference images based on the image search range information indicated by the reference motion information. In this situation, the computer device can use the first reference image list and the second reference image list from at least the list of M reference images as the target reference image search range. In some embodiments, all of the M reference image lists can be used as the target reference image search range. For example, when the M reference image lists include the first reference image list, the second reference image list, and the bidirectional reference image list, all of the first reference image list, the second reference image list, and the bidirectional reference image list can be used as the target reference image search range.

[0120] (2) Since the auxiliary coding block uses a reference image from the list of M reference images for encoding (or motion estimation), and the auxiliary coding block and the target coding block have a certain correlation, the target coding block will also largely use that reference image from the list of M reference images for encoding. For example, if the auxiliary coding block uses reference image 1 from the list of M reference images for encoding, the target coding block will also largely use reference image 1 for encoding. In this case, the image search range information mentioned above includes the reference image or the image search range information includes the image index of the reference image. By using the reference motion information of the auxiliary coding block to indicate that the image search range information includes the reference image, the number of reference images to search for in the target coding block can be reduced, thereby accelerating the motion estimation of the target coding block.

[0121] In one embodiment, when determining the target reference image search range of the target coding block, the reference search image can be directly filtered from an existing list of multiple reference images to determine the reference search image to be searched, and this reference search image can be determined as the target reference image search range of the target coding block. There is no need to distinguish between different reference lists during the filtering process. The computer device can determine a fifth condition parameter based on the image search range information indicated by the reference motion information. The fifth condition parameter includes the number and / or area of ​​fifth-class auxiliary coding blocks, where the fifth-class auxiliary coding blocks refer to the coding blocks in each auxiliary coding block of the target coding block that perform reference image searches on the reference search image. If the fifth condition parameter satisfies the fifth filtering condition, the reference search image is taken as the target reference image search range of the target coding block. Otherwise, the reference search image is not taken as the target reference image search range of the target coding block.

[0122] In one embodiment, the aforementioned fifth type of auxiliary coding block may specifically be a coding block in each auxiliary coding block of the target coding block that performs a reference image search on the reference search image in the M reference image lists. In this case, the specific implementation of determining the target reference image search range for the target coding block from the M reference image lists based on the reference image search range information indicated by the reference motion information may be as follows: Based on the image search range information indicated by the reference motion information, a fifth condition parameter is determined. The fifth condition parameter includes the number and / or area of ​​the fifth type of auxiliary coding blocks. The fifth type of auxiliary coding block refers to a coding block in each auxiliary coding block of the target coding block that performs a reference image search on the reference search image in the M reference image lists. For example, the M reference image lists include a forward reference list, a backward reference list, and a bidirectional reference list. Both the forward reference list and the bidirectional reference list include the target reference image 1. The fifth type of auxiliary coding block refers to a coding block in each auxiliary coding block that performs a reference image search on the target reference image 1. Then, it is determined whether the fifth condition parameter satisfies the fifth filtering condition. If the fifth condition parameter satisfies the fifth filtering condition, the reference search image in the list of M reference images is used as the target reference image search range for the target coding block. That is, when performing a reference image search, the reference search image can be searched directly. If the fifth condition parameter does not satisfy the fifth filtering condition, the reference search image in the list of M reference images is not used as the target reference image search range for the target coding block. That is, when performing a reference image search, the search for the reference search image is skipped, thereby reducing the number of reference images to search for the target coding block. It should be noted that when using the reference search image as the target reference image search range for the target coding block, the reference image list to which the reference search image belongs is included. This is beneficial for subsequent image searches, as it helps to know the reference image list to which the reference search image belongs. For example, if reference search image 1 belongs to reference image list 1, reference search image 1 is included in reference image list 1. The fifth filtering condition may include at least one of the following: the number of fifth-class auxiliary coding blocks is greater than a number threshold, or the area ratio between the area of ​​the fifth-class auxiliary coding block and the sum of the areas of all auxiliary coding blocks is greater than an area ratio threshold.

[0123] In one embodiment, the number of reference search images can be one or more. When there are multiple reference search images, all reference search images in the M reference image list can be used as the target reference image search range for the target coding block. For example, if there are 5 reference search images, all 5 reference search images can be used as the target reference image search range for the target coding block. When performing a reference image search, these 5 reference search images need to be searched. Alternatively, a portion of the reference search images can be randomly selected from the M reference image list as the target reference image search range for the target coding block. In this case, when performing a reference image search, the selected reference search images need to be searched, and the search of unselected reference search images is skipped. For example, if there are 10 reference search images, 9 reference search images can be randomly selected as the target reference image search range. When performing a reference image search, the 9 selected reference search images need to be searched, and the unselected reference search images are skipped.

[0124] When the fifth condition parameter is the number of fifth-class auxiliary coding blocks, and the fifth filtering condition is that the number of fifth-class auxiliary coding blocks is greater than a quantity threshold, the computer device can count the number of fifth-class auxiliary coding blocks based on the image search range information indicated by the reference motion information of the auxiliary coding blocks. Then, it determines whether the number of fifth-class auxiliary coding blocks is greater than the quantity threshold. If the number is greater than the quantity threshold, the fifth condition parameter is determined to satisfy the fifth filtering condition. If the number is less than or equal to the quantity threshold, the fifth condition parameter is determined not to satisfy the fifth filtering condition. For example, if a list of M reference images includes reference search image 1, the quantity threshold is 4, and the number of auxiliary coding blocks used for image search of reference search image 1 is 5, which is greater than the quantity threshold of 4, the computer device determines that the fifth condition parameter satisfies the fifth filtering condition.

[0125] When the fifth condition parameter is the area of ​​the fifth type of auxiliary coding block, and the fifth filtering condition is that the area ratio between the area of ​​the fifth type of auxiliary coding block and the sum of the areas of all auxiliary coding blocks is greater than the area ratio threshold, the computer device can determine the fifth type of auxiliary coding block based on the image search range information indicated by the reference motion information of the auxiliary coding block, calculate the area ratio between the area of ​​the fifth type of auxiliary coding block and the sum of the areas of all auxiliary coding blocks, and determine whether the area ratio is greater than the area ratio threshold. If the area ratio is greater than the area ratio threshold, the fifth condition parameter is determined to meet the fifth filtering condition. If the area ratio is less than or equal to the area ratio threshold, the fifth condition parameter is determined not to meet the fifth filtering condition. For example, if the area ratio threshold is 0.15, and the fifth type of auxiliary coding blocks are auxiliary coding block 1 and auxiliary coding block 2, the computer device can calculate the sum of the areas between auxiliary coding block 1 and auxiliary coding block 2 to obtain area sum 1, and calculate the sum of the areas between all auxiliary coding blocks to obtain area sum 2. Then, the area ratio between area sum 1 and area sum 2 is calculated to be 0.2. Since the area ratio of 0.2 is greater than the area ratio threshold of 0.15, the fifth condition parameter is determined to meet the fifth screening condition.

[0126] The quantity threshold and area ratio threshold can be determined based on the temporal layer of the video frame to be encoded and / or the size of the target coding block. For example, the larger the temporal layer of the video frame to be encoded, the smaller the quantity threshold and area ratio threshold; similarly, the larger the size of the target coding block, the smaller the quantity threshold and area ratio threshold. It should be noted that the specific implementation methods for obtaining the quantity threshold and area ratio threshold can be found in the corresponding sections above, and will not be repeated here.

[0127] In one embodiment, if the fifth condition parameter satisfies the fifth filtering condition, then the specific implementation of using the reference search image in the list of M reference images as the target reference image search range for the target coding block can be as follows: if the fifth condition parameter satisfies the fifth filtering condition, then the reference search image in the list of M reference images is determined as the candidate reference image search range. This candidate reference image search range includes the reference search image and the reference image list to which the reference search image belongs. Then, the computer device can determine the sixth condition parameter based on the image search range information indicated by the reference motion information. The sixth condition parameter may include the number and / or area of ​​the sixth type of auxiliary reference coding blocks; the sixth type of auxiliary... A reference coding block refers to a coding block in each auxiliary coding block of the target coding block that performs a reference image search on the first reference image list of the candidate reference image search range. The process involves determining whether the sixth condition parameter satisfies the sixth filtering condition. If the sixth condition parameter satisfies the sixth filtering condition, the first reference image list and the reference search images within it are used as the target reference image search range for the target coding block. If the sixth condition parameter does not satisfy the sixth filtering condition, the first reference image list and the reference search images within it are not used as the target reference image search range for the target coding block; that is, the search for the first reference image list is skipped during the reference image search. The sixth filtering condition may include at least one of the following: the number of sixth-type auxiliary coding blocks is greater than a quantity threshold; the ratio between the area of ​​the sixth-type auxiliary coding block and the sum of the areas of all auxiliary coding blocks is greater than an area ratio threshold. The quantity threshold and area ratio threshold can be determined by the temporal layer of the video frame to be encoded and / or the size of the target coding block. Specifically, the larger the temporal layer, the smaller the quantity threshold and area ratio threshold; the larger the size of the target coding block, the smaller the quantity threshold and area ratio threshold.

[0128] In one embodiment, when the sixth condition parameter is the number of sixth type auxiliary coding blocks and the sixth filtering condition is that the number of sixth type auxiliary coding blocks is greater than a quantity threshold, the specific implementation of determining whether the sixth condition parameter satisfies the sixth filtering condition can be as follows: the computer device can determine the number of sixth auxiliary coding blocks based on the image search range information indicated by the reference motion information, and determine whether the number of sixth auxiliary coding blocks is greater than the quantity threshold. If the number is greater than the quantity threshold, then the sixth condition parameter is determined to satisfy the sixth filtering condition; if the number is less than or equal to the quantity threshold, then the sixth condition parameter is determined not to satisfy the sixth filtering condition.

[0129] In one embodiment, when the sixth condition parameter is the area of ​​the sixth type of auxiliary coding block, the sixth filtering condition is that the ratio between the area of ​​the sixth type of auxiliary coding block and the sum of the areas of all auxiliary coding blocks is greater than the area ratio threshold. A specific implementation of determining whether the sixth condition parameter satisfies the sixth filtering condition can be as follows: The computer device can determine the sixth type of auxiliary coding block based on the image search range information indicated by the reference motion information, and calculate the area of ​​the sixth type of auxiliary coding block; then, it calculates the area ratio between the area of ​​the sixth type of auxiliary coding block and the sum of the areas of all auxiliary coding blocks, and determines whether the area ratio is greater than the area ratio threshold. If the area ratio is greater than the area ratio threshold, the sixth condition parameter is determined to satisfy the sixth filtering condition; if the area ratio is less than or equal to the area ratio threshold, the sixth condition parameter is determined not to satisfy the sixth filtering condition.

[0130] In one embodiment, if the fifth condition parameter satisfies the fifth filtering condition, then the specific implementation of using the reference search image in the list of M reference images as the target reference image search range of the target coding block can be as follows: if the fifth condition parameter satisfies the fifth filtering condition, then the reference search image in the list of M reference images is determined as the candidate reference image search range. The candidate reference image search range includes the reference search image and the reference image list to which the reference search image belongs. Then, the computer device can determine the seventh condition parameter based on the image search range information indicated by the reference motion information. The seventh condition parameter may include: the number and / or area of ​​the seventh type of auxiliary coding blocks. The seventh type of auxiliary coding block refers to the coding block in each auxiliary coding block of the target coding block that performs a reference image search on the second reference image list of the candidate reference image search range. Then, it is determined whether the seventh condition parameter satisfies the seventh screening condition. If the seventh condition parameter satisfies the seventh screening condition, the second reference image list and the reference search images in the second reference image list are used as the target reference image search range for the target coding block. If the seventh condition parameter does not satisfy the seventh screening condition, the second reference image list and the reference search images in the second reference image list are not used as the target reference image search range for the target coding block; that is, the search for the second reference image list is skipped when performing reference image search. The seventh screening condition may include at least one of the following: the number of seventh-class auxiliary coding blocks is greater than a quantity threshold; the area ratio between the area of ​​the seventh-class auxiliary coding block and the sum of the areas of all auxiliary coding blocks is greater than an area ratio threshold. The quantity threshold and area ratio threshold can be determined by the time layer of the video frame to be encoded and / or the size of the target coding block. The larger the time layer, the smaller the quantity threshold and area ratio threshold; the larger the size of the target coding block, the smaller the quantity threshold and area ratio threshold. It should be noted that the specific implementation methods for determining whether the seventh condition parameter satisfies the seventh screening condition and for obtaining the quantity threshold and area ratio threshold can be found in the corresponding embodiments described above, and will not be repeated here.

[0131] In one embodiment, if the fifth condition parameter does not meet the fifth filtering condition, the feature information of the target coding block is obtained. The feature information includes: the time layer of the video frame to be encoded, or the feature information includes: the time layer of the video frame to be encoded and the size of the target coding block. If the feature information of the target coding block meets the triggering condition, the process of obtaining the reference motion information of the auxiliary coding block of the target coding block is triggered. If the feature information of the target coding block does not meet the triggering condition, at least the first reference image list and the second reference image list in the M reference image list are used as the target reference image search range of the target coding block.

[0132] S104: Perform motion estimation on the target coded block based on the target reference image search range. The target reference image search range includes a list of target reference images and the target reference images corresponding to the list.

[0133] In one embodiment, the computer device can perform motion estimation based on a target reference image search range using a first reference image search rule to obtain first motion estimation parameters for the target coding block, and perform motion estimation based on a target reference image search range using a second reference image search rule to obtain second motion estimation parameters for the target coding block. The optimal motion estimation parameter is then determined from the first and second motion estimation parameters as the target motion estimation parameter for the target coding block. The first reference image search rule can be a translational motion estimation model (ME), and the second reference image search rule can be an affine motion estimation model (AEM). The computer device can determine the optimal motion estimation parameter from the first and second motion estimation parameters as the target motion estimation parameter for the target coding block by selecting the motion estimation parameter corresponding to the minimum rate distortion (RD) as the optimal motion estimation parameter and using it as the target motion estimation parameter for the target coding block. Compared to the VVC reference software's general video coding test model (VTM, VVC test model), which reduces the number of RD calculations during motion estimation based on predefined search templates and rules, the embodiments of this application, through the target reference image search range obtained in step S103, can reduce the number of RD calculations during motion estimation, accelerate the motion estimation of the target coding block, and also more accurately ensure the accuracy of the target motion estimation parameters of the target coding block obtained based on the target reference image search range.

[0134] In this embodiment, a video frame to be encoded is acquired, and a target coding block to be encoded is determined within the video frame. Reference motion information of auxiliary coding blocks for the target coding block is acquired. Based on the image search range information indicated by the reference motion information, a target reference image search range is determined from a list of M reference images, where M is an integer greater than 1. Motion estimation is performed on the target coding block based on the target reference image search range. By using the reference motion information of the auxiliary coding blocks to reduce the number of reference images or reference lists to search for the target coding block, the search of a portion of the reference image list or reference images during motion estimation is skipped, reducing computational load, lowering the time complexity of motion estimation, and improving the efficiency of motion estimation in the video frame encoding process.

[0135] Based on the motion estimation scheme in the above encoding process, please refer to Figure 3 , Figure 3This is a flowchart illustrating a motion estimation method during the encoding process provided in this application embodiment. The motion estimation method during the encoding process can be executed by a computer device, which can be a terminal device or a server. More specifically, the motion estimation method during the encoding process can be executed by an encoder in the computer device. The aforementioned terminal device can be a smartphone, tablet computer, laptop computer, desktop computer, intelligent voice interaction device, smart home appliance, vehicle terminal, aircraft, etc.; the aforementioned server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms, etc. The motion estimation method during the encoding process described in this embodiment includes the following steps S301-S306.

[0136] S301: Acquire the video frame to be encoded and determine the target coding block to be encoded in the video frame.

[0137] S302: Obtain the feature information of the target coding block. The feature information includes: the time layer of the video frame to be encoded, or the feature information includes: the time layer of the video frame to be encoded and the size of the target coding block. The time layer can be time layer 0, time layer 1, time layer 2, time layer 3, time layer 4 and time layer 5, etc., and the size of the target coding block can be 256, 128, 64, 32, etc.

[0138] In one embodiment, for important video frames or to prevent the target motion estimation parameters from failing to update, a trigger condition can be predefined. The computer device can determine whether the feature information of the acquired target coded block meets the trigger condition. This trigger condition is used to trigger the acquisition of reference motion information of the auxiliary coded block of the target coded block. If it is determined that the feature information of the target coded block meets the trigger condition, then fast motion estimation can be performed on the target coded block, and step S303 is executed. If it is determined that the feature information of the target coded block does not meet the trigger condition, it indicates that the video frame is an important video frame. At this time, it is not possible to determine the target reference image search range for the target coded block from the list of M reference images based on the image search range information indicated by the reference motion information to perform fast motion estimation on the target coded block, and the computer device executes step S305.

[0139] In one embodiment, when the feature information of the target coding block includes the time layer of the video frame to be encoded, the triggering condition can be that the time layer of the video frame to be encoded is greater than the time layer threshold. That is, fast motion estimation is prohibited for coding blocks included in video frames of lower time layers to ensure coding efficiency. The computer device can determine whether the time layer of the video frame to be encoded included in the feature information of the target coding block is greater than the time layer threshold. If it is determined that the time layer of the video frame to be encoded is greater than the time layer threshold, then fast motion estimation can be performed on the target coding block, i.e., step S303 is executed; if it is determined that the time layer of the video frame to be encoded is less than or equal to the time layer threshold, it means that the video frame to be encoded is an important frame, and fast motion estimation cannot be performed on the target coding block. In this case, step S305 can be executed.

[0140] In one embodiment, when the feature information of the target coding block includes the time layer of the video frame to be encoded and the size of the target coding block, the triggering condition may include the predefined coding block size corresponding to each time layer. For example, in a random access (RA) configuration, the predefined coding block size corresponding to time layer 0 is 256, the predefined coding block size corresponding to time layer 1 is 128, the predefined coding block size corresponding to time layer 2 is 128, the predefined coding block size corresponding to time layer 3 is 64, the predefined coding block size corresponding to time layer 4 is 64, and the predefined coding block size corresponding to time layer 5 is 32. It should be noted that the predefined coding block size corresponding to each time layer in this application can be set according to requirements and is not limited to the predefined coding block size corresponding to each time layer in the above example. Then, the specific process by which the computer device determines whether the feature information of the target coding block meets the triggering condition is as follows: the computer device determines the target predefined coding block size corresponding to the time layer of the video frame to be encoded from the predefined coding block sizes corresponding to each time layer based on the time layer of the video frame to be encoded. Then, it determines whether the size of the target coding block is greater than the target predefined coding block size. If the size of the target coding block is determined to be larger than the predefined target coding block size, then fast motion estimation can be performed on the target coding block and step S303 can be executed; if the size of the target coding block is determined to be smaller than or equal to the predefined target coding block size, then fast motion estimation cannot be performed on the target coding block and step S305 can be executed.

[0141] For example, if the time layer of the video frame to be encoded is 0 and the size of the target coding block is 256, the computer device can determine the predefined coding block size corresponding to time layer 0 as 128 from the predefined coding block sizes corresponding to each time layer. Then, if the computer device determines that the size of the target coding block is greater than the predefined coding block size (i.e., 256 is greater than 128), fast motion estimation can be performed on the target coding block. As another example, if the time layer of the video frame to be encoded is 0 and the size of the target coding block is 64, and the computer device determines that the size of the target coding block is smaller than the predefined coding block size (i.e., 64 is less than 128), fast motion estimation cannot be performed on the target coding block. In this case, the first and second reference image lists from at least M reference image lists can be used as the target reference image search range.

[0142] S303: If the feature information of the target coding block meets the triggering condition, then trigger the execution to obtain the reference motion information of the auxiliary coding block of the target coding block.

[0143] S304: Based on the image search range information indicated by the reference motion information, determine the target reference image search range for the target coding block from a list of M reference images, where M is an integer greater than 1.

[0144] For a detailed implementation of step S304, please refer to [link / reference]. Figure 1 The specific implementation methods of the corresponding parts will not be elaborated here.

[0145] S305: If the feature information of the target coding block does not meet the triggering condition, then at least the first reference image list and the second reference image list in the M reference image lists shall be used as the target reference image search range for the target coding block. The first reference image list may be a forward reference image list, and the second reference image list may be a backward reference image list.

[0146] In one embodiment, when the list of M reference images includes a first reference image list and a second reference image list, if it is determined that the feature information of the target coding block does not meet the triggering condition, then both the first reference image list and the second reference image list are used as the target reference image search range.

[0147] In another embodiment, when the M reference image lists include a first reference image list, a second reference image list, and a bidirectional reference image list, if it is determined that the feature information of the target coding block does not meet the triggering condition, the first reference image list and the first reference image list can be used as the target reference image search range; or if it is determined that the feature information of the target coding block does not meet the triggering condition, the first reference image list, the first reference image list, and the bidirectional reference image list can all be used as the target reference image search range.

[0148] S306: Perform motion estimation on the target coded block based on the search range of the target reference image.

[0149] In one embodiment, the specific process of motion estimation of the target coding block based on the target reference image search range can be referred to in Figure 104. The specific implementation of step S104 is as follows: The computer device determines the target reference image list corresponding to the target reference image search range and the target reference image corresponding to the target reference image list. The number of target reference images can be one or more. Then, based on the target reference image list and its corresponding target reference image, motion estimation is performed through a first reference image search rule to obtain the first motion estimation parameter of the target coding block. And based on the target reference image list and its corresponding target reference image, motion estimation is performed through a second reference image search rule to obtain the second motion estimation parameter of the target coding block. The second reference image search rule is different from the first reference image search rule. A target motion estimation parameter is selected for the target coding block from the first motion estimation parameter and the second motion estimation parameter.

[0150] Each target reference image includes one or more coded blocks. The specific implementation of obtaining the first motion estimation parameters of the target coded blocks by performing motion estimation using a first reference image search rule based on the target reference image list and its corresponding target reference images can be as follows: Using the first reference image search rule, traverse the target reference images in the target reference image list; then, using rate-distortion (RD) cost as a criterion, determine the first matching block with the minimum rate-distortion among the coded blocks included in the target reference images corresponding to the target reference image list; and determine the motion parameters corresponding to the first matching block with the minimum rate-distortion as the first motion estimation parameters of the target coded blocks. Similarly, the specific implementation of obtaining the second motion estimation parameters of the target coded blocks by performing motion estimation using a second reference image search rule based on the target reference image list and its corresponding target reference images can be as follows: Using the second reference image search rule, traverse the target reference images in the target reference image list; and using rate-distortion (RD) cost as a criterion, determine the second matching block with the minimum rate-distortion among the coded blocks included in the target reference images corresponding to the target reference image list; and determine the motion estimation parameters corresponding to the second matching block as the second motion estimation parameters of the target coded blocks. The second reference image search rule is a gradient descent solution rule. Its loss function is the minimum mean square error (MSE) of the coding block before and after motion estimation, and its initial motion estimation parameters are determined by the motion estimation parameters of the first reference image search rule.

[0151] In one embodiment, when the target reference image search range includes a first reference image list and a second reference image list from at least M reference image lists, see [link to relevant documentation]. Figure 4 , Figure 4 This is a schematic diagram illustrating motion estimation of a target coded block using a first reference image search rule. The specific implementation of the computer device performing motion estimation based on a list of target reference images and their corresponding target reference images, using the first reference image search rule to obtain the first motion estimation parameters of the target coded block, can be as follows: According to the first reference image search rule, traverse all reference images in the first and second reference image lists. Using rate-distortion as the accuracy, determine the first matching block with the minimum rate-distortion from the coded blocks included in the target reference images corresponding to the first reference image list, and the second matching block with the minimum rate-distortion from the coded blocks included in the target reference images corresponding to the second reference image list. Then, perform a weighted summation of the first and second matching blocks to obtain a bidirectional matching block. Finally, determine the motion estimation parameters corresponding to the matching block with the minimum rate-distortion from the motion estimation parameters corresponding to the first matching block, the second matching block, and the bidirectional matching block, and use these as the first motion estimation parameters of the target coded block.

[0152] In one embodiment, to select the optimal motion estimation parameter as the target motion estimation parameter for the target coding block, the rate distortion corresponding to the first motion estimation parameter and the rate distortion corresponding to the second motion estimation parameter can be compared to select the optimal motion estimation parameter. A specific implementation for selecting the target motion estimation parameter from the first and second motion estimation parameters can be: obtaining the first rate distortion corresponding to the first motion estimation parameter and the second rate distortion corresponding to the second motion estimation parameter; then comparing the first and second rate distortions; if the first rate distortion is greater than the second rate distortion, then the second motion estimation parameter corresponding to the second rate distortion is determined as the target motion estimation parameter for the target coding block; if the second rate distortion is greater than the first rate distortion, then the first motion estimation parameter corresponding to the first rate distortion is determined as the target motion estimation parameter for the target coding block. If the first and second rate distortions are the same, then both the first and second motion estimation parameters can be determined as the target motion estimation parameters for the target coding block. In some embodiments, if the first and second rate distortions are the same, then either the first or the second motion estimation parameter can be determined as the target motion estimation parameter for the target coding block.

[0153] In this embodiment, a computer device can acquire a video frame to be encoded and determine a target coding block to be encoded within that video frame. It can also acquire feature information of the target coding block, including the temporal layer of the video frame to be encoded, or the temporal layer of the video frame to be encoded and the size of the target coding block. By setting trigger conditions, it is possible to control whether rapid motion estimation is performed on the target coding block, preventing the use of the same list of reference images or reference images for motion estimation in each coding block during video frame encoding, thus avoiding falling into local optima. On the other hand, if the feature information of the target coding block meets the triggering condition, the process of obtaining the reference motion information of the auxiliary coding block of the target coding block is triggered. Based on the image search range information indicated by the reference motion information, the target reference image search range is determined for the target coding block from a list of M reference images. If the feature information of the target coding block does not meet the triggering condition, at least the first reference image list and the second reference image list in the list of M reference images are used as the target reference image search range for the target coding block. Motion estimation of the target coding block is performed based on the target reference image search range. When the feature information of the target coding block meets the triggering condition, fast motion estimation of the target coding block can be performed. By determining the target reference image search range, the amount of computation can be reduced, the motion estimation of the target coding block can be improved, and the coding efficiency can be improved.

[0154] Based on the description of the motion estimation method embodiments in the above encoding process, this application also discloses a motion estimation device in the encoding process. This motion estimation device can be a computer program (including program code) running on the aforementioned computer device. This motion estimation device in the encoding process can execute... Figure 1 and Figure 3 The method shown. Please refer to [link / reference]. Figure 5 The motion estimation device in the encoding process may include the following units:

[0155] The acquisition unit 501 is used to acquire a video frame to be encoded and determine a target coding block to be encoded in the video frame to be encoded.

[0156] The acquisition unit 501 is further configured to acquire reference motion information of auxiliary coding blocks of the target coding block. The reference motion information of the auxiliary coding block includes: motion information of N adjacent coding blocks adjacent to the target coding block and motion information included in the reference coding block list, where N is an integer greater than or equal to 1.

[0157] Processing unit 502 is configured to determine the target reference image search range for the target coding block from a list of M reference images based on the image search range information indicated by the reference motion information, where M is an integer greater than 1;

[0158] The processing unit 502 is used to perform motion estimation on the target coded block based on the search range of the target reference image.

[0159] In one embodiment, when the processing unit 502 determines the target reference image search range for the target coding block from a list of M reference images based on the reference image search range information indicated by the reference motion information, it may specifically be used to:

[0160] Based on the image search range information indicated by the reference motion information of the auxiliary coding block, a first condition parameter is determined. The first condition parameter includes: the number and / or area of ​​the first type of auxiliary coding block. The first type of auxiliary coding block refers to the coding block in each auxiliary coding block of the target coding block that performs a reference image search on the first reference image list in the M reference image list. The first reference image list is a forward reference list.

[0161] If the first condition parameter satisfies the first filtering condition, then the first reference image list is used as the target reference image search range for the target coding block.

[0162] In one embodiment, when the processing unit 502 determines the target reference image search range for the target coding block from a list of M reference images based on the reference image search range information indicated by the reference motion information, it may specifically be used to:

[0163] Based on the image search range information indicated by the reference motion information of the auxiliary coding block, a second condition parameter is determined. The second condition parameter includes the number and / or area of ​​the second type of auxiliary coding block. The second type of auxiliary coding block refers to the coding block in each auxiliary coding block of the target coding block that performs a reference image search on the second reference image list in the M reference image list. The second reference image list is a backward reference list.

[0164] If the second condition parameter satisfies the second filtering condition, then the second reference image list is used as the target reference image search range for the target coding block.

[0165] In one embodiment, before determining the second condition parameter based on the image search range information indicated by the reference motion information of the auxiliary coding block, the method is further configured to:

[0166] Determine whether the first condition parameter meets the first filtering condition;

[0167] If the first condition parameter does not meet the first screening condition, then the second condition parameter is determined by executing the image search range information indicated by the reference motion information of the auxiliary coding block;

[0168] The first condition parameter includes: the number and / or area of ​​the first type of auxiliary coding blocks, where the first type of auxiliary coding block refers to the coding block in each auxiliary coding block of the target coding block that performs a reference image search on the first reference image list in the M reference image list, and the first reference image list is a forward reference list.

[0169] In one embodiment, when the processing unit 502 uses the first reference image list as the target reference image search range for the target coding block if the first condition parameter satisfies the first filtering condition, it can be specifically used for:

[0170] If the first condition parameter satisfies the first screening condition, then the third condition parameter is determined according to the reference image corresponding to the reference motion information of the auxiliary coding block. The third condition parameter includes the number and / or area of ​​the third type of auxiliary coding block. The third type of auxiliary coding block refers to the coding block in each auxiliary coding block of the target coding block that performs a reference image search for the target reference image in the first reference image list.

[0171] If the third condition parameter satisfies the third filtering condition, then the target reference image in the first reference image list is used as the target reference image search range for the target coding block.

[0172] In one embodiment, when the processing unit 502 uses the second reference image list as the target reference image search range for the target coding block if the second condition parameter satisfies the second filtering condition, it can be specifically used for:

[0173] If the second condition parameter satisfies the second filtering condition, then the fourth condition parameter is determined according to the reference image corresponding to the reference motion information of the auxiliary coding block. The fourth condition parameter includes the number and / or area of ​​the fourth type of auxiliary coding block. The fourth type of auxiliary coding block refers to the coding block in each auxiliary coding block of the target coding block that performs a reference image search for the target reference image in the second reference image list.

[0174] If the fourth condition parameter satisfies the fourth filtering condition, then the target reference image in the second reference image list is used as the target reference image search range for the target coding block.

[0175] In one embodiment, when the processing unit 502 determines the target reference image search range for the target coding block from a list of M reference images based on the reference image search range information indicated by the reference motion information, it may specifically be used to:

[0176] Based on the image search range information indicated by the reference motion information, a fifth condition parameter is determined. The fifth condition parameter includes the number and / or area of ​​the fifth type of auxiliary coding blocks. The fifth type of auxiliary coding block refers to the coding block in each auxiliary coding block of the target coding block that performs reference image search on the reference search image in the list of M reference images.

[0177] If the fifth condition parameter satisfies the fifth filtering condition, then the reference search image is used as the target reference image search range for the target coding block.

[0178] In one embodiment, before acquiring the reference motion information of the auxiliary coding block of the target coding block, the acquisition unit 501 is further configured to: acquire the feature information of the target coding block, the feature information including: the time layer of the video frame to be encoded, or the feature information including: the time layer of the video frame to be encoded and the size of the target coding block;

[0179] The processing unit 502 is further configured to, if the feature information of the target coding block meets the triggering condition, trigger the execution of obtaining the reference motion information of the auxiliary coding block of the target coding block;

[0180] If the feature information of the target coding block does not meet the triggering condition, then at least the first reference image list and the second reference image list in the M reference image list shall be used as the target reference image search range of the target coding block.

[0181] In one embodiment, when the processing unit 502 performs motion estimation on the target coded block based on the target reference image search range, it may specifically be used to:

[0182] Determine the target reference image list corresponding to the target reference image search range and the target reference image corresponding to the target reference image list;

[0183] Based on the target reference image list and its corresponding target reference image, motion estimation is performed using the first reference image search rule to obtain the first motion estimation parameters of the target coding block;

[0184] Based on the target reference image list and its corresponding target reference image, motion estimation is performed using a second reference image search rule to obtain the second motion estimation parameters of the target coding block; the second reference image search rule is different from the first reference image search rule.

[0185] Select a target motion estimation parameter for the target coding block from the first motion estimation parameter and the second motion estimation parameter.

[0186] In one embodiment, when selecting a target motion estimation parameter for the target coding block from the first motion estimation parameter and the second motion estimation parameter, the acquisition unit 501 is used to acquire the first rate distortion corresponding to the first motion estimation parameter and the second rate distortion corresponding to the second motion estimation parameter.

[0187] The processing unit 502 is configured to, if the first rate distortion is greater than the second rate distortion, determine the second motion estimation parameter corresponding to the second rate distortion as the target motion estimation parameter of the target coding block; and if the second rate distortion is greater than the first rate distortion, determine the first motion estimation parameter corresponding to the first rate distortion as the target motion estimation parameter of the target coding block.

[0188] According to one embodiment of this application, Figure 1 and Figure 3 Each step involved in the method shown can be performed by... Figure 5 The motion estimation process shown is performed by individual units in the motion estimation device. For example, Figure 1 Steps S101-S102 shown are... Figure 5 The acquisition unit 501 shown is used to execute steps S103-S104. Figure 5 The processing unit 502 shown is used to execute this. For example, Figure 3 Steps S301-S302 shown are... Figure 5 The acquisition unit 501 shown is used to execute steps S303-S306. Figure 5 The processing unit 502 shown is used to execute this.

[0189] According to another embodiment of this application, Figure 5 The units in the motion estimation device during the encoding process shown can be individually or entirely merged into one or more other units, or some of the units can be further divided into multiple functionally smaller units. This achieves the same operation without affecting the technical effect of the embodiments of this application. The above units are based on logical function division. In practical applications, the function of one unit can be implemented by multiple units, or the function of multiple units can be implemented by one unit. In other embodiments of this application, the motion estimation device during the encoding process may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented collaboratively by multiple units.

[0190] According to another embodiment of this application, processing elements and storage elements, such as a central processing unit (CPU), random access storage medium (RAM), and read-only storage medium (ROM), can be used. For example, a general-purpose computing device such as a computer can run on a device capable of performing tasks such as... Figure 1 and Figure 3 The computer program (including program code) for each step involved in the corresponding method shown, to construct such... Figure 5 The diagram illustrates a motion estimation apparatus for the encoding process, and a motion estimation method for implementing the embodiments of this application. The computer program may be recorded on, for example, a computer-readable recording medium, loaded onto the aforementioned computer device via the same medium, and executed therein.

[0191] In this embodiment, a video frame to be encoded is acquired, and a target coding block to be encoded is determined within the video frame. Reference motion information of auxiliary coding blocks for the target coding block is acquired. Based on the image search range information indicated by the reference motion information, a target reference image search range is determined from a list of M reference images, where M is an integer greater than 1. Motion estimation is performed on the target coding block based on the target reference image search range. By using the reference motion information of the auxiliary coding blocks to reduce the number of reference images or reference lists to search for the target coding block, the search of a portion of the reference image list or reference images during motion estimation is skipped, reducing computational load, lowering the time complexity of motion estimation, and improving the efficiency of motion estimation in the video frame encoding process.

[0192] Based on the description of the motion estimation method embodiments in the above encoding process, this application also discloses a computer device. Please refer to [link to relevant documentation]. Figure 6 The computer device may include at least a processor 601, an input device 602, an output device 603, and a memory 604. The processor 601, input device 602, output device 603, and memory 604 within the computer device may be connected via a bus or other means.

[0193] The memory 604 is a memory device in a computer device used to store programs and data. It is understood that the memory 604 here can include the computer device's built-in storage medium, or it can include extended storage media supported by the computer device. The memory 604 provides storage space for the computer device's operating system. Furthermore, the computer program (including program code) is also stored in this storage space. It should be noted that the computer storage medium here can be a high-speed RAM memory; optionally, it can also be at least one computer storage medium located remotely from the aforementioned processor. The processor can be called a Central Processing Unit (CPU), which is the core and control center of the computer device, used to run the computer programs stored in the memory 604.

[0194] In one embodiment, the processor 601 can load and execute a computer program stored in the memory 604 to implement the corresponding steps of the method in the above-described embodiments concerning the motion estimation method in the encoding process; specifically, the processor 601 loads and executes the computer program stored in the memory 604 for:

[0195] Acquire the video frame to be encoded, and determine the target coding block to be encoded in the video frame to be encoded;

[0196] Obtain reference motion information of auxiliary coding blocks of the target coding block. The reference motion information of the auxiliary coding block includes: motion information of N adjacent coding blocks adjacent to the target coding block and motion information included in the reference coding block list, where N is an integer greater than or equal to 1.

[0197] Based on the image search range information indicated by the reference motion information, the target reference image search range is determined for the target coding block from a list of M reference images, where M is an integer greater than 1;

[0198] Motion estimation is performed on the target coded block based on the search range of the target reference image.

[0199] In one embodiment, when the processor 601 loads and executes the computer program stored in the memory 604 to determine the target reference image search range for the target coding block from a list of M reference images based on the reference image search range information indicated by the reference motion information, it can be specifically used for:

[0200] Based on the image search range information indicated by the reference motion information of the auxiliary coding block, a first condition parameter is determined. The first condition parameter includes: the number and / or area of ​​the first type of auxiliary coding block. The first type of auxiliary coding block refers to the coding block in each auxiliary coding block of the target coding block that performs a reference image search on the first reference image list in the M reference image list. The first reference image list is a forward reference list.

[0201] If the first condition parameter satisfies the first filtering condition, then the first reference image list is used as the target reference image search range for the target coding block.

[0202] In one embodiment, when the processor 601 loads and executes the computer program stored in the memory 604 to determine the target reference image search range for the target coding block from a list of M reference images based on the reference image search range information indicated by the reference motion information, it can be specifically used for:

[0203] Based on the image search range information indicated by the reference motion information of the auxiliary coding block, a second condition parameter is determined. The second condition parameter includes the number and / or area of ​​the second type of auxiliary coding block. The second type of auxiliary coding block refers to the coding block in each auxiliary coding block of the target coding block that performs a reference image search on the second reference image list in the M reference image list. The second reference image list is a backward reference list.

[0204] If the second condition parameter satisfies the second filtering condition, then the second reference image list is used as the target reference image search range for the target coding block.

[0205] In one embodiment, before determining the second condition parameter based on the image search range information indicated by the reference motion information of the auxiliary coding block, the processor 601 loads and executes the computer program stored in the memory 604, and is further configured to:

[0206] Determine whether the first condition parameter meets the first filtering condition;

[0207] If the first condition parameter does not meet the first screening condition, then the second condition parameter is determined by executing the image search range information indicated by the reference motion information of the auxiliary coding block;

[0208] The first condition parameter includes: the number and / or area of ​​the first type of auxiliary coding blocks, where the first type of auxiliary coding block refers to the coding block in each auxiliary coding block of the target coding block that performs a reference image search on the first reference image list in the M reference image list, and the first reference image list is a forward reference list.

[0209] In one embodiment, when the processor 601 loads and executes the computer program stored in the memory 604, and uses the first reference image list as the target reference image search range for the target coding block if the first condition parameter satisfies the first filtering condition, it can be specifically used for:

[0210] If the first condition parameter satisfies the first screening condition, then the third condition parameter is determined according to the reference image corresponding to the reference motion information of the auxiliary coding block. The third condition parameter includes the number and / or area of ​​the third type of auxiliary coding block. The third type of auxiliary coding block refers to the coding block in each auxiliary coding block of the target coding block that performs a reference image search for the target reference image in the first reference image list.

[0211] If the third condition parameter satisfies the third filtering condition, then the target reference image in the first reference image list is used as the target reference image search range for the target coding block.

[0212] In one embodiment, when the processor 601 loads and executes the computer program stored in the memory 604, and uses the second reference image list as the target reference image search range for the target coding block if the second condition parameter satisfies the second filtering condition, it can be specifically used for:

[0213] If the second condition parameter satisfies the second filtering condition, then the fourth condition parameter is determined according to the reference image corresponding to the reference motion information of the auxiliary coding block. The fourth condition parameter includes the number and / or area of ​​the fourth type of auxiliary coding block. The fourth type of auxiliary coding block refers to the coding block in each auxiliary coding block of the target coding block that performs a reference image search for the target reference image in the second reference image list.

[0214] If the fourth condition parameter satisfies the fourth filtering condition, then the target reference image in the second reference image list is used as the target reference image search range for the target coding block.

[0215] In one embodiment, when the processor 601 loads and executes the computer program stored in the memory 604 to determine the target reference image search range for the target coding block from a list of M reference images based on the reference image search range information indicated by the reference motion information, it can be specifically used for:

[0216] Based on the image search range information indicated by the reference motion information, a fifth condition parameter is determined. The fifth condition parameter includes the number and / or area of ​​the fifth type of auxiliary coding blocks. The fifth type of auxiliary coding block refers to the coding block in each auxiliary coding block of the target coding block that performs reference image search on the reference search image in the list of M reference images.

[0217] If the fifth condition parameter satisfies the fifth filtering condition, then the reference search image is used as the target reference image search range for the target coding block.

[0218] In one embodiment, before the processor 601 loads and executes the computer program stored in the memory 604 to obtain the reference motion information of the auxiliary coding block of the target coding block, it is further configured to:

[0219] Obtain feature information of the target coding block, the feature information including: the time layer of the video frame to be encoded, or the feature information including: the time layer of the video frame to be encoded and the size of the target coding block;

[0220] If the feature information of the target coding block meets the triggering condition, then the process of obtaining the reference motion information of the auxiliary coding block of the target coding block is triggered.

[0221] If the feature information of the target coding block does not meet the triggering condition, then at least the first reference image list and the second reference image list in the M reference image list shall be used as the target reference image search range of the target coding block.

[0222] In one embodiment, when the processor 601 loads and executes the computer program stored in the memory 604 to perform motion estimation on the target coded block based on the target reference image search range, it may specifically be used for:

[0223] Determine the target reference image list corresponding to the target reference image search range and the target reference image corresponding to the target reference image list;

[0224] Based on the target reference image list and its corresponding target reference image, motion estimation is performed using the first reference image search rule to obtain the first motion estimation parameters of the target coding block;

[0225] Based on the target reference image list and its corresponding target reference image, motion estimation is performed using a second reference image search rule to obtain the second motion estimation parameters of the target coding block; the second reference image search rule is different from the first reference image search rule.

[0226] Select a target motion estimation parameter for the target coding block from the first motion estimation parameter and the second motion estimation parameter.

[0227] In one embodiment, when the processor 601 loads and executes the computer program stored in the memory 604 to select target motion estimation parameters for the target coding block from the first motion estimation parameters and the second motion estimation parameters, it may specifically be used for:

[0228] Obtain the first rate distortion corresponding to the first motion estimation parameter and the second rate distortion corresponding to the second motion estimation parameter;

[0229] If the first rate distortion is greater than the second rate distortion, then the second motion estimation parameter corresponding to the second rate distortion is determined as the target motion estimation parameter of the target coding block;

[0230] If the second rate distortion is greater than the first rate distortion, then the first motion estimation parameter corresponding to the first rate distortion is determined as the target motion estimation parameter of the target coding block.

[0231] It should be understood that, in the embodiments of this application, the processor 601 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0232] In this embodiment, a video frame to be encoded is acquired, and a target coding block to be encoded is determined within the video frame. Reference motion information of auxiliary coding blocks for the target coding block is acquired. Based on the image search range information indicated by the reference motion information, a target reference image search range is determined from a list of M reference images, where M is an integer greater than 1. Motion estimation is performed on the target coding block based on the target reference image search range. By using the reference motion information of the auxiliary coding blocks to reduce the number of reference images or reference lists to search for the target coding block, the search of a portion of the reference image list or reference images during motion estimation is skipped, reducing computational load, lowering the time complexity of motion estimation, and improving the efficiency of motion estimation in the video frame encoding process.

[0233] This application provides a computer-readable storage medium storing a computer program, the computer program including program instructions, which, when executed by a processor, can perform the steps performed in all the above embodiments.

[0234] This application also provides a computer program product, which includes computer instructions stored in a computer-readable storage medium. When the computer instructions are executed by the processor of a computer device, they perform the methods described in all the above embodiments.

[0235] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0236] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Those skilled in the art will understand that implementing all or part of the processes of the above embodiments and making equivalent changes in accordance with the claims of this application are still within the scope of the invention.

Claims

1. A motion estimation method in the encoding process, characterized in that, include: Acquire the video frame to be encoded, and determine the target coding block to be encoded in the video frame to be encoded; Obtain reference motion information of auxiliary coding blocks of the target coding block. The reference motion information of the auxiliary coding block includes: motion information of N adjacent coding blocks adjacent to the target coding block and motion information included in the reference coding block list, where N is an integer greater than or equal to 1. Based on the image search range information indicated by the reference motion information, the target reference image search range is determined for the target coding block from a list of M reference images, where M is an integer greater than 1; Determine the target reference image list corresponding to the target reference image search range and the target reference image corresponding to the target reference image list; Based on the target reference image list and its corresponding target reference images, motion estimation is performed using a first reference image search rule to obtain the first motion estimation parameters of the target coding block. Then, based on the target reference image list and its corresponding target reference images, motion estimation is performed using a second reference image search rule to obtain the second motion estimation parameters of the target coding block. The second reference image search rule is different from the first reference image search rule. The optimal motion estimation parameter is determined from the first motion estimation parameter and the second motion estimation parameter and used as the target motion estimation parameter for the target coding block.

2. The method as described in claim 1, characterized in that, The step of determining the target reference image search range for the target coding block from a list of M reference images based on the reference image search range information indicated by the reference motion information includes: Based on the image search range information indicated by the reference motion information of the auxiliary coding block, a first condition parameter is determined. The first condition parameter includes: the number and / or area of ​​the first type of auxiliary coding block. The first type of auxiliary coding block refers to the coding block in each auxiliary coding block of the target coding block that performs a reference image search on the first reference image list in the M reference image list. The first reference image list is a forward reference list. If the first condition parameter satisfies the first filtering condition, then the first reference image list is used as the target reference image search range for the target coding block.

3. The method as described in claim 1, characterized in that, The step of determining the target reference image search range for the target coding block from a list of M reference images based on the reference image search range information indicated by the reference motion information includes: Based on the image search range information indicated by the reference motion information of the auxiliary coding block, a second condition parameter is determined. The second condition parameter includes the number and / or area of ​​the second type of auxiliary coding block. The second type of auxiliary coding block refers to the coding block in each auxiliary coding block of the target coding block that performs a reference image search on the second reference image list in the M reference image list. The second reference image list is a backward reference list. If the second condition parameter satisfies the second filtering condition, then the second reference image list is used as the target reference image search range for the target coding block.

4. The method as described in claim 3, characterized in that, Before determining the second condition parameter based on the image search range information indicated by the reference motion information of the auxiliary coding block, the method further includes: Determine whether the first condition parameter meets the first filtering condition; If the first condition parameter does not meet the first screening condition, then the second condition parameter is determined by executing the image search range information indicated by the reference motion information of the auxiliary coding block; The first condition parameter includes: the number and / or area of ​​the first type of auxiliary coding blocks, where the first type of auxiliary coding block refers to the coding block in each auxiliary coding block of the target coding block that performs a reference image search on the first reference image list in the M reference image list, and the first reference image list is a forward reference list.

5. The method as described in claim 2, characterized in that, The step of using the first reference image list as the target reference image search range for the target coding block if the first condition parameter satisfies the first filtering condition includes: If the first condition parameter satisfies the first screening condition, then the third condition parameter is determined according to the reference image corresponding to the reference motion information of the auxiliary coding block. The third condition parameter includes the number and / or area of ​​the third type of auxiliary coding block. The third type of auxiliary coding block refers to the coding block in each auxiliary coding block of the target coding block that performs a reference image search for the target reference image in the first reference image list. If the third condition parameter satisfies the third filtering condition, then the target reference image in the first reference image list is used as the target reference image search range for the target coding block.

6. The method as described in claim 3, characterized in that, If the second condition parameter satisfies the second filtering condition, then the second reference image list is used as the target reference image search range for the target coding block, including: If the second condition parameter satisfies the second filtering condition, then the fourth condition parameter is determined according to the reference image corresponding to the reference motion information of the auxiliary coding block. The fourth condition parameter includes the number and / or area of ​​the fourth type of auxiliary coding block. The fourth type of auxiliary coding block refers to the coding block in each auxiliary coding block of the target coding block that performs a reference image search for the target reference image in the second reference image list. If the fourth condition parameter satisfies the fourth filtering condition, then the target reference image in the second reference image list is used as the target reference image search range for the target coding block.

7. The method as described in claim 1, characterized in that, The step of determining the target reference image search range for the target coding block from a list of M reference images based on the reference image search range information indicated by the reference motion information includes: Based on the image search range information indicated by the reference motion information, a fifth condition parameter is determined. The fifth condition parameter includes the number and / or area of ​​the fifth type of auxiliary coding blocks. The fifth type of auxiliary coding block refers to the coding block in each auxiliary coding block of the target coding block that performs reference image search on the reference search image in the list of M reference images. If the fifth condition parameter satisfies the fifth filtering condition, then the reference search image is used as the target reference image search range for the target coding block.

8. The method as described in claim 1, characterized in that, Before obtaining the reference motion information of the auxiliary coding block of the target coding block, the method further includes: Obtain feature information of the target coding block, the feature information including: the time layer of the video frame to be encoded, or the feature information including: the time layer of the video frame to be encoded and the size of the target coding block; If the feature information of the target coding block meets the triggering condition, then the process of obtaining the reference motion information of the auxiliary coding block of the target coding block is triggered. If the feature information of the target coding block does not meet the triggering condition, then at least the first reference image list and the second reference image list in the M reference image list shall be used as the target reference image search range of the target coding block.

9. The method as described in claim 1, characterized in that, The step of selecting target motion estimation parameters for the target coding block from the first motion estimation parameters and the second motion estimation parameters includes: Obtain the first rate distortion corresponding to the first motion estimation parameter and the second rate distortion corresponding to the second motion estimation parameter; If the first rate distortion is greater than the second rate distortion, then the second motion estimation parameter corresponding to the second rate distortion is determined as the target motion estimation parameter of the target coding block; If the second rate distortion is greater than the first rate distortion, then the first motion estimation parameter corresponding to the first rate distortion is determined as the target motion estimation parameter of the target coding block.

10. A motion estimation device in an encoding process, characterized in that, include: An acquisition unit is used to acquire a video frame to be encoded and to determine a target coding block to be encoded in the video frame to be encoded. The acquisition unit is further configured to acquire reference motion information of auxiliary coding blocks of the target coding block. The reference motion information of the auxiliary coding block includes: motion information of N adjacent coding blocks adjacent to the target coding block and motion information included in the reference coding block list, where N is an integer greater than or equal to 1. The processing unit is configured to determine the target reference image search range for the target coding block from a list of M reference images based on the image search range information indicated by the reference motion information, where M is an integer greater than 1; The processing unit is configured to: determine a list of target reference images corresponding to the target reference image search range and a target reference image corresponding to the list of target reference images; perform motion estimation according to the list of target reference images and their corresponding target reference images using a first reference image search rule to obtain a first motion estimation parameter for the target coding block; and perform motion estimation according to the list of target reference images and their corresponding target reference images using a second reference image search rule to obtain a second motion estimation parameter for the target coding block; the second reference image search rule is different from the first reference image search rule; and determine the optimal motion estimation parameter from the first motion estimation parameter and the second motion estimation parameter as the target motion estimation parameter for the target coding block.

11. A computer device, characterized in that, include: Memory, used to store computer programs; The processor executes a computer program stored in the memory to implement the motion estimation method in the encoding process as described in any one of claims 1-9.

12. A computer storage medium, characterized in that, The computer storage medium stores a computer program, which, when executed by a processor, is used to implement the motion estimation method in the encoding process as described in any one of claims 1-9.

13. A computer program product comprising a computer program; the computer program being stored in a computer-readable storage medium, and when executed by a processor of a computer device, performing a motion estimation method in the encoding process as described in any one of claims 1-9.

Images