Inter-frame prediction method and electronic device

Abstract

This application provides an inter-frame prediction method and an electronic device. The method includes: obtaining a first motion vector corresponding to the current block; obtaining a first segmentation map of a first reference block corresponding to the current block based on the first motion vector, the first segmentation map including a first region and a second region; determining a first target region from the first region and the second region; establishing a motion vector candidate list based on the spatial distribution of the first target region in the first segmentation map, the spatial distribution describing the position of the first target region in the first segmentation map; determining a second motion vector corresponding to the current block based on the motion vector candidate list; performing motion compensation based on the target segmentation map, the first motion vector, and the second motion vector to obtain a predicted block corresponding to the current block; the target segmentation map is determined based on the first segmentation map and / or the second segmentation map, the second segmentation map being a segmentation map of the second reference block corresponding to the current block determined based on the second motion vector. This improves compression performance.

Description

Technical Field

[0001] This application relates to the field of encoding and decoding, and more particularly to an inter-frame prediction method and electronic device. Background Technology

[0002] Generally, the detail changes between adjacent frames of a video sequence are very small, meaning that adjacent frames have a strong correlation. Therefore, utilizing the correlation between adjacent frames for encoding (i.e., inter-frame coding) can improve the compression ratio of the video sequence. Inter-frame coding can be broadly divided into two processes: inter-frame prediction and encoding. The inter-frame prediction process can include motion search (also called motion estimation, i.e., determining the motion vector (MV), which describes the displacement information of the coded block in the coded frame and the reference block in the reference frame) and motion compensation (i.e., extracting the reference block (also called the prediction block) from the reference frame). The encoding process can encode the motion vector into a bitstream, or encode the motion vector and the residual block between the prediction block and the coded block into a bitstream.

[0003] Because coded blocks possess complex motion fields (i.e., multiple motion fields), the traditional method of describing a coded block using a single motion vector is no longer sufficient to accurately depict the motion fields contained within it. Therefore, a coded block can be divided into multiple regions, each described by a separate motion vector, to achieve an accurate description of the coded block containing complex motion fields. However, existing inter-frame prediction methods suffer from low compression performance. Summary of the Invention

[0004] To address the aforementioned technical problems, this application provides an inter-frame prediction method and an electronic device. In this method, motion compensation is performed based on the segmentation map of the reference block corresponding to the current block, which improves compression performance.

[0005] In a first aspect, embodiments of this application provide an inter-frame prediction method, the method comprising: firstly, obtaining a first motion vector corresponding to the current block; based on the first motion vector, obtaining a first segmentation map of a first reference block corresponding to the current block, the first segmentation map including a first region and a second region; subsequently, determining a first target region from the first region and the second region; next, establishing a motion vector candidate list based on the spatial distribution of the first target region in the first segmentation map, the spatial distribution being used to describe the position of the first target region in the first segmentation map; based on the motion vector candidate list, determining a second motion vector corresponding to the current block, the second motion vector corresponding to a third region in the current block, the third region corresponding to the first target region, the first motion vector corresponding to a fourth region in the current block, and the fourth region corresponding to a second target region in the first region and the second region; then, performing motion compensation based on the target segmentation map, the first motion vector, and the second motion vector to obtain a prediction block corresponding to the current block, the target segmentation map being determined based on the first segmentation map and / or the second segmentation map, the second segmentation map being a segmentation map of the second reference block corresponding to the current block determined based on the second motion vector.

[0006] The first segmentation map of the first reference block describes the partitioning of the foreground and background regions of the first reference block. Since the first reference block is strongly correlated with the current block, its first segmentation map can also describe the partitioning of the foreground and background regions of the current block. In other words, partitioning the first reference block using its first segmentation map can be applied to partitioning the current block. Similarly, the second segmentation map of the second reference block describes the partitioning of the foreground and background regions of the second reference block. Since the second reference block is strongly correlated with the current block, its second segmentation map can also describe the partitioning of the foreground and background regions of the current block. In other words, partitioning the second reference block using its segmentation map can be applied to partitioning the current block. Furthermore, based on the target segmentation map, the partitioning information for different motion fields of the current block can be obtained. The target segmentation map can accurately and efficiently segment motion fields, thus enabling accurate and efficient partitioning of different motion fields within the current block. This allows for inter-frame prediction based on motion vectors from different motion fields of the current block, improving the accuracy of inter-frame prediction and consequently enhancing coding quality.

[0007] Furthermore, the spatial distribution of the first target region in the first segmentation map can describe the position of the third region corresponding to the first target region in the current block. Based on the different motion field distribution information of the current block, the method of constructing the motion vector candidate list can be decided, which can improve the encoding efficiency of motion vectors for different motion fields.

[0008] In one possible approach, the first region corresponds to the foreground region of the first reference block and the foreground region of the current block, and the second region corresponds to the background region of the first reference block and the background region of the current block.

[0009] In one possible approach, the first region corresponds to the background region of the first reference block and the background region of the current block, and the second region corresponds to the foreground region of the first reference block and the foreground region of the current block; the specific settings can be configured according to requirements, and this application does not impose any restrictions on this.

[0010] In one possible approach, the third region is the background region of the current block, and the fourth region is the foreground region of the current block.

[0011] In one possible approach, the third region is the foreground region of the current block, and the fourth region is the background region of the current block.

[0012] For example, the first segmentation map of the first reference block is a block-level segmentation map, which can be a binary map.

[0013] In one possible approach, the pixel value of the first region in the first segmentation image of the first reference block is 0 (i.e., the first region is black), and the pixel value of the second region in the first segmentation image of the first reference block is 1 (i.e., the second region is white).

[0014] In one possible approach, the pixel value of the pixels in the first region of the first segmentation image of the first reference block can also be 1 (i.e., the first region is white), and the pixel value of the pixels in the second region of the first segmentation image of the first reference block can be 0 (i.e., the second region is black); this application does not limit this.

[0015] For example, the second segmentation map of the second reference block is a block-level segmentation map, which can be a binary map. It should be understood that the second segmentation map is similar to the first segmentation map, and will not be described again here.

[0016] For example, the target segmentation map has the same size as the first segmentation map (or the second segmentation map).

[0017] In one possible approach, the first region can be designated as the first target region, and the second region as the second target region.

[0018] In one possible approach, the second region can be used as the first target region, and the first region can be used as the second target region.

[0019] It should be understood that this application does not restrict whether the first region or the second region is used as the first target region. The first target region and the second target region are different regions.

[0020] According to the first aspect, the motion vector candidate list includes multiple third motion vectors corresponding to multiple adjacent blocks of the current block; the distance between the adjacent block corresponding to the i-th third motion vector in the motion vector candidate list and the third region is less than the distance between the adjacent block corresponding to the (i+1)-th third motion vector and the third region; i is a positive integer.

[0021] In this way, the third motion vector of the adjacent block that is closer to the third region can be placed at the front of the motion vector candidate list. The second motion vector of the third region in the current block is more similar to the third motion vector of the adjacent block that is closer to the third region; therefore, the optimal third motion vector is selected earlier from the motion vector candidate list, requiring fewer bytes for encoding. Compared with existing technologies that generate the motion vector list in a traditional fixed order, this application can improve the efficiency of encoding the second motion vector and save bitstream.

[0022] Furthermore, it can improve the accuracy of the determined second motion vector, thereby improving the accuracy of inter-frame prediction and thus improving coding quality.

[0023] According to the first aspect, or any implementation of the first aspect above, the second motion vector corresponding to the current block is determined based on the motion vector candidate list, including: selecting the first third motion vector in the motion vector candidate list as the second motion vector.

[0024] In this way, since the second motion vector of the third region in the current block is more similar to the third motion vector of the adjacent block that is closer to the third region, and the third motion vector of the adjacent block that is closer to the third region ranks higher in the motion vector candidate list, the accuracy of the determined second motion vector can be improved, thereby improving the accuracy of inter-frame prediction and thus improving the coding quality.

[0025] Furthermore, the earlier the third motion vector appears in the candidate list of the second motion vector, the fewer bytes are required for encoding, thus making the encoding of the second motion vector more efficient and saving bitstream.

[0026] According to the first aspect, or any implementation thereof, a motion vector candidate list is established based on the spatial distribution of the first target region in the first segmentation map. This includes: determining the spatial distribution type corresponding to the first target region from multiple preset spatial distribution types based on the spatial distribution of the first target region in the first segmentation map; determining a motion vector candidate list from a preset candidate list set based on the spatial distribution type corresponding to the first target region, the preset candidate list set including multiple preset motion vector candidate lists corresponding to multiple preset spatial distribution types, each preset motion vector candidate list including multiple neighbor block identifiers, where each neighbor block identifier represents a neighboring block at a given location of the current block; and filling the motion vector candidate list with the third motion vectors corresponding to multiple neighboring blocks of the current block according to the multiple neighbor block identifiers included in the motion vector candidate list. Thus, by selecting a motion vector candidate list from a pre-established preset candidate list set based on the spatial distribution of the first target region, a motion vector candidate list can be established accurately and quickly.

[0027] According to the first aspect, or any implementation of the first aspect above, the method further includes establishing a preset candidate list set: obtaining multiple preset segmentation maps corresponding to multiple preset image blocks, the preset segmentation maps being used to describe the partitioning of the preset image blocks, the preset segmentation maps including a fifth region and a sixth region; classifying based on the spatial distribution of the third target region in each preset segmentation map to obtain multiple preset spatial distribution types, the third target region being selected from the fifth region and the sixth region; for the target spatial distribution type among the multiple preset spatial distribution types, determining a preset motion vector candidate list corresponding to the target spatial distribution type based on the positional relationship between the seventh region in the target preset image block corresponding to the target spatial distribution type and multiple adjacent blocks of the target preset image block, the seventh region corresponding to the third target region; and establishing a preset candidate list set using the preset motion vector candidate lists corresponding to various preset spatial distribution types. In this way, through the spatial distribution of the third target region in multiple preset segmentation maps, various preset spatial distribution types can be accurately and comprehensively classified.

[0028] According to the first aspect, or any implementation of the first aspect above, for a target spatial distribution type among multiple preset spatial distribution types, based on the positional relationship between the seventh region in the target preset image block corresponding to the target spatial distribution type and multiple adjacent blocks of the target preset image block, a preset motion vector candidate list corresponding to the target spatial distribution type is determined, including: determining the distance between the seventh region in the target preset image block and multiple adjacent blocks of the target preset image block respectively; determining the arrangement order of multiple adjacent blocks of the target preset image block based on the distance between the seventh region in the target preset image block and multiple adjacent blocks of the target preset image block; and filling the neighbor block identifiers corresponding to the multiple adjacent blocks of the target preset image block into the preset motion vector candidate list corresponding to the target spatial distribution type according to the arrangement order. In this way, it can be ensured that the motion vector candidate list subsequently determined from the preset candidate list set is determined based on the positional relationship between the third region and multiple adjacent blocks of the current block, which can improve the encoding efficiency of motion vectors for different motion fields.

[0029] According to the first aspect, or any implementation of the first aspect above, in the preset motion vector candidate list corresponding to the target spatial distribution type, the distance between the neighboring block corresponding to the k-th neighboring block identifier and the seventh region in the target preset image block is less than the distance between the neighboring block corresponding to the (k+1)-th neighboring block identifier and the seventh region in the target preset image block; k is a positive integer. This ensures that in the subsequent motion vector candidate list determined from the preset candidate list set, the third motion vector of the neighboring block that is closer to the third region is ranked higher in the motion vector candidate list.

[0030] According to the first aspect, or any implementation of the first aspect above, a first target region is determined from the first region and the second region, including: determining the region with the smallest area in the first region and the second region as the first target region; wherein the second target region is the region with the largest area in the first region and the second region. This ensures that the first reference block is closer to the current block, and subsequent determination of the second motion vector and motion compensation based on the first segmentation map of the first reference block improves the accuracy of inter-frame prediction, thereby improving coding quality.

[0031] In one possible approach, the region with the largest area in the first region and the region with the smallest area in the first region and the region with the smallest area in the second region can be defined as the first target region; and the region with the smallest area in the first region and the region with the smallest area can be defined as the second target region.

[0032] In one possible approach, the region corresponding to the foreground region of the current block in the first region and the second region can be determined as the first target region; and the region corresponding to the background region of the current block in the first region and the second region can be determined as the second target region.

[0033] In one possible approach, the region corresponding to the background region of the current block in the first region and the second region can be determined as the first target region; and the region corresponding to the foreground region of the current block in the first region and the second region can be determined as the second target region.

[0034] In one possible approach, the region in which the pixel values ​​of all pixels in the first region and the second region are both 1 can be defined as the first target region; and the region in which the pixel values ​​of all pixels in the first region and the second region are both 0 can be defined as the second target region.

[0035] In one possible approach, the region in which the pixel values ​​of all pixels contained in the first region and the second region are both 0 can be defined as the first target region; and the region in which the pixel values ​​of all pixels contained in the first region and the second region are both 1 can be defined as the second target region.

[0036] It should be understood that other methods may also be used to determine the first target area and the second target area from the first area and the second area, and this application does not limit this.

[0037] According to the first aspect, or any implementation of the first aspect above, based on the first motion vector, a first segmentation map of the first reference block corresponding to the current block is obtained, including: determining the first reference block corresponding to the current block based on the first motion vector; and segmenting the first reference block to obtain the first segmentation map. This allows for a more accurate description of the foreground and background regions of the current block.

[0038] For example, the reference frame of the current block can be segmented to determine the frame segmentation map corresponding to the reference frame. Then, based on the first motion vector, the first reference block corresponding to the current block is determined from the reference frame, and the first segmentation map of the first reference block is extracted from the frame segmentation map of the reference frame. The segmentation of the first reference block is similar to the segmentation of the reference frame.

[0039] According to the first aspect, or any implementation of the first aspect above, segmenting a first reference block to obtain a first segmentation map includes: inputting the first reference block into a segmentation network, and having the segmentation network segment the first reference block to obtain the first segmentation map. The segmentation network can be a neural network or a deep learning network, and can include AI (Artificial Intelligence) segmentation networks, such as instance segmentation networks, video object segmentation networks, etc., which are not limited in this application. By employing an AI image segmentation scheme, the accuracy of segmenting different motion fields of the current block can be improved.

[0040] According to the first aspect, or any implementation of the first aspect above, the first reference block is segmented to obtain a first segmentation map, including: segmenting the first reference block based on the edge detection results of the first reference block to obtain the first segmentation map. This reduces computational load and improves the efficiency of determining the segmentation map.

[0041] According to the first aspect, or any implementation of the first aspect above, both the first motion vector and the second motion vector include integer and fractional parts. Motion compensation based on the target segmentation map, the first motion vector, and the second motion vector to obtain the prediction block corresponding to the current block includes: performing integer-pixel-level motion compensation based on the target segmentation map, the first reference block, and the second reference block to determine the integer-pixel prediction block, wherein the first reference block is determined based on the integer part of the first motion vector, and the second reference block is determined based on the integer part of the second motion vector; performing fractional-pixel-level motion compensation based on the target segmentation map, the integer-pixel prediction block, the fractional part of the first motion vector, and the fractional part of the second motion vector to obtain the prediction block corresponding to the current block. In this way, through integer-pixel-level motion compensation, boundary artifacts can be avoided at the boundaries between the region corresponding to the second target region in the first reference block and the region corresponding to the first target region in the second reference block. Furthermore, due to the inherent spatial discreteness of digital video, the block division may not coincidentally align with pixels; therefore, fractional-pixel-level motion compensation can improve the precision of the prediction block.

[0042] For example, the size of the predicted block is the same as the size of the current block.

[0043] According to the first aspect, or any implementation thereof, integer-pixel motion compensation is performed based on the target segmentation map, the first reference block, and the second reference block to determine the integer-pixel prediction block. This includes: linearly fusing the first reference block and the second reference block based on the target segmentation map to determine the integer-pixel prediction block. In this way, linear fusion can effectively reduce artifacts caused by block-level motion compensation.

[0044] According to the first aspect, or any implementation thereof, a linear fusion of a first reference block and a second reference block is performed based on the segmentation map to determine an integer-pixel prediction block. This includes: determining the pixel category corresponding to each pixel in the first reference block and the pixel category corresponding to each pixel in the second reference block based on the distance between each pixel in the target segmentation map and the segmentation line, where the segmentation line is the segmentation line between the first region and the second region in the target segmentation map; determining a first weight corresponding to each pixel in the first reference block based on the target segmentation map and the pixel category corresponding to each pixel in the first reference block; determining a second weight corresponding to each pixel in the second reference block based on the pixel category corresponding to each pixel in the target segmentation map and the second reference block; and linearly weighting the pixel values ​​of corresponding positions in the first and second reference blocks based on the first and second weights to obtain an integer-pixel prediction block. This approach fully utilizes the motion-inconsistent region segmentation information and multiple reference blocks to process pixels around the segmentation line of motion-inconsistent regions, ensuring better inter-frame prediction performance without boundary artifacts.

[0045] According to the first aspect, or any implementation of the first aspect above, pixel-level motion compensation is performed based on the target segmentation map, integer-pixel prediction blocks, fractional portions of the first motion vector, and fractional portions of the second motion vector to obtain the prediction block corresponding to the current block. This includes: dividing the integer-pixel prediction block into an eighth region and a ninth region based on the dividing lines of the first and second regions in the target segmentation map; performing pixel-level interpolation on the eighth region based on the fractional portion of the first motion vector to obtain a first pixel-level prediction block; performing pixel-level interpolation on the ninth region based on the fractional portion of the second motion vector to obtain a second pixel-level prediction block; and linearly fusing the first and second pixel-level prediction blocks based on the target segmentation map to obtain the prediction block. This allows for the precise generation of fractional-pixel predictions, improving the precision of the prediction block.

[0046] Based on the first aspect, or any of the implementations of the first aspect above.

[0047] The target segmentation image is the first segmentation image; or,

[0048] The target segmentation image is the second segmentation image; or,

[0049] The target segmentation map is the segmentation map obtained by fusing the first segmentation map and the second segmentation map.

[0050] In one possible approach, the pixel values ​​of corresponding pixels in the first and second segmentation images can be ANDed or ORed; this application does not limit this approach.

[0051] It should be noted that in the first aspect and any implementation thereof, the first motion vector corresponding to the current block is determined first; then, based on the first segmentation map determined by the first motion vector, the partitioning of the current block (i.e., the partitioning of the third and fourth regions in the current block) is determined; then, according to the first segmentation map, the second motion vector corresponding to the third region in the current block is determined, and the first motion vector is used as the motion vector of the fourth region in the current block. In other words, the second motion vector is determined based on the first motion vector.

[0052] Secondly, embodiments of this application provide an inter-frame prediction method, which includes: first, obtaining a first motion vector corresponding to the current block and obtaining a second motion vector corresponding to the current block, wherein both the first motion vector and the second motion vector include an integer part and a fractional part; next, determining a first reference block of the current block based on the integer part of the first motion vector, and determining a second reference block of the current block based on the integer part of the second motion vector; subsequently, obtaining a segmentation map, which is used to describe the partitioning of the current block; then, performing integer-pixel-level motion compensation based on the segmentation map, the first reference block, and the second reference block to determine an integer-pixel prediction block; and performing fractional-pixel-level motion compensation based on the segmentation map, the integer-pixel prediction block, the fractional part of the first motion vector, and the fractional part of the second motion vector to obtain the prediction block corresponding to the current block.

[0053] It should be noted that the second aspect can be any method other than the first aspect and any implementation thereof, to determine the motion vectors corresponding to different motion fields of the current block. In this case, the reference block determined according to the motion vectors corresponding to different motion fields can still be subjected to integer pixel-level motion compensation and sub-pixel-level motion compensation based on the segmentation map and the motion vectors of different motion fields. This can improve the precision of the prediction block and improve the coding quality.

[0054] For example, geometric partitioning can be used to divide the current block into different motion fields, and then the motion vectors corresponding to different motion fields can be determined.

[0055] According to the second aspect, performing pixel-level motion compensation based on the segmentation map, the first reference block, and the second reference block to determine the pixel prediction block includes: performing linear fusion of the first reference block and the second reference block based on the segmentation map to determine the pixel prediction block.

[0056] According to the second aspect, or any implementation of the second aspect above, the segmentation map includes a first region and a second region; based on the segmentation map, the first reference block and the second reference block are linearly fused to determine an integer pixel prediction block, including: determining the pixel category corresponding to each pixel in the first reference block and the pixel category corresponding to each pixel in the second reference block based on the distance between each pixel in the segmentation map and the segmentation line, wherein the segmentation line is the segmentation line between the first region and the second region in the segmentation map; determining the first weight corresponding to each pixel in the first reference block based on the segmentation map and the pixel category corresponding to each pixel in the first reference block; determining the second weight corresponding to each pixel in the second reference block based on the pixel category corresponding to each pixel in the segmentation map and the second reference block; and linearly weighting the pixel values ​​of corresponding positions of pixels in the first reference block and the second reference block based on the first weight and the second weight to obtain an integer pixel prediction block.

[0057] According to the second aspect, or any implementation of the second aspect above, pixel-level motion compensation is performed based on the segmentation map, the integer pixel prediction block, the fractional part of the first motion vector, and the fractional part of the second motion vector to obtain the prediction block corresponding to the current block, including: dividing the integer pixel prediction block into a third region and a fourth region based on the segmentation line of the first region and the second region in the segmentation map; performing pixel-level interpolation on the third region based on the fractional part of the first motion vector to obtain a first pixel-level prediction block; and performing pixel-level interpolation on the fourth region based on the fractional part of the second motion vector to obtain a second pixel-level prediction block; and performing linear fusion of the first pixel-level prediction block and the second pixel-level prediction block based on the segmentation map to obtain the prediction block.

[0058] According to the second aspect, or any implementation of the second aspect above, the current block is divided into a fifth region and a sixth region, and the segmentation map includes a first region corresponding to the fifth region and a second region corresponding to the sixth region; a first motion vector corresponds to the fifth region, and a second motion vector corresponds to the sixth region; obtaining the first motion vector corresponding to the current block includes: establishing a candidate list of first motion vectors based on the spatial distribution of the first region in the segmentation map, the spatial distribution being used to describe the position of the first region in the segmentation map; and determining the first motion vector corresponding to the fifth region in the current block based on the candidate list of first motion vectors.

[0059] According to the second aspect, or any implementation of the second aspect above, the current block is divided into a fifth region and a sixth region, and the segmentation map includes a first region corresponding to the fifth region and a second region corresponding to the sixth region; a first motion vector corresponds to the fifth region, and a second motion vector corresponds to the sixth region; obtaining the second motion vector corresponding to the current block includes: establishing a candidate list of second motion vectors based on the spatial distribution of the second region in the segmentation map, the spatial distribution being used to describe the position of the second region in the segmentation map; and determining the second motion vector corresponding to the sixth region in the current block based on the candidate list of second motion vectors.

[0060] According to the second aspect, or any implementation of the second aspect above, the second motion vector candidate list includes multiple third motion vectors corresponding to multiple adjacent blocks of the current block; the distance between the adjacent block corresponding to the i-th third motion vector in the motion vector candidate list and the sixth region is less than the distance between the adjacent block corresponding to the (i+1)-th third motion vector and the sixth region; i is a positive integer.

[0061] According to the second aspect, or any implementation of the second aspect above, based on the second motion vector candidate list, the second motion vector corresponding to the sixth region in the current block is determined, including: selecting the first third motion vector in the second motion vector candidate list as the second motion vector.

[0062] According to the second aspect, or any implementation of the second aspect above, a second motion vector candidate list is established based on the spatial distribution of the second region in the segmentation map, including: determining the spatial distribution type corresponding to the second region from multiple preset spatial distribution types based on the spatial distribution of the second region in the segmentation map; determining the second motion vector candidate list from a preset candidate list set based on the spatial distribution type corresponding to the second region, the preset candidate list set including multiple preset motion vector candidate lists corresponding to multiple preset spatial distribution types, the preset motion vector candidate list including multiple neighbor block identifiers, the neighbor block identifiers being used to represent the neighboring blocks at a position of the current block; and filling the second motion vector candidate list with the third motion vectors corresponding to multiple neighboring blocks of the current block according to the multiple neighbor block identifiers included in the second motion vector candidate list.

[0063] According to the second aspect, or any implementation of the second aspect above, the method further includes: the method further includes establishing a preset candidate list set: obtaining multiple preset segmentation maps corresponding to multiple preset image blocks, the preset segmentation maps being used to describe the partitioning of the preset image blocks, the preset segmentation maps including a seventh region and an eighth region; classifying the spatial distribution of target regions based on the spatial distribution of each preset segmentation map to obtain multiple preset spatial distribution types, the target regions being selected from the seventh region and the eighth region; for the target spatial distribution type among the multiple preset spatial distribution types, determining a preset motion vector candidate list corresponding to the target spatial distribution type based on the positional relationship between the target spatial distribution type and the ninth region in the target preset image block, and multiple adjacent blocks of the target preset image block, the ninth region corresponding to the third target region; and establishing a preset candidate list set using the preset motion vector candidate lists corresponding to various preset spatial distribution types.

[0064] According to the second aspect, or any implementation of the second aspect above, for a target spatial distribution type among multiple preset spatial distribution types, based on the positional relationship between the ninth region in the target preset image block corresponding to the target spatial distribution type and multiple adjacent blocks of the target preset image block, a preset motion vector candidate list corresponding to the target spatial distribution type is determined, including: determining the distance between the ninth region in the target preset image block and multiple adjacent blocks of the target preset image block respectively; determining the arrangement order of multiple adjacent blocks of the target preset image block based on the distance between the ninth region in the target preset image block and multiple adjacent blocks of the target preset image block; and filling the neighbor block identifiers corresponding to the multiple adjacent blocks of the target preset image block into the preset motion vector candidate list corresponding to the target spatial distribution type according to the arrangement order.

[0065] According to the second aspect, or any implementation of the second aspect above, in the preset motion vector candidate list corresponding to the target spatial distribution type, the distance between the neighboring block corresponding to the kth neighboring block identifier and the ninth region in the target preset image block is less than the distance between the neighboring block corresponding to the (k+1)th neighboring block identifier and the ninth region in the target preset image block; k is a positive integer.

[0066] Based on the second aspect, or any implementation of the second aspect above, obtain the segmentation map, including:

[0067] The first reference block is segmented to obtain a segmentation map; or,

[0068] The second reference block is segmented to obtain a segmentation map; or,

[0069] The first segmentation map obtained by segmenting the first reference block and the second segmentation map obtained by segmenting the second reference block are merged to obtain a segmentation map.

[0070] For example, if the current block is divided into a fifth region and a sixth region using geometric partitioning to determine the first motion vector corresponding to the fifth region and the second motion vector corresponding to the sixth region, the segmentation map can be a geometric partition map. In this case, this application does not restrict the order in which the segmentation map, the first motion vector, and the second motion vector are obtained.

[0071] According to the second aspect, or any implementation of the second aspect above, the first reference block is segmented to obtain a segmentation map, including: inputting the first reference block into a segmentation network, and having the segmentation network segment the first reference block to obtain a segmentation map.

[0072] According to the second aspect, or any implementation of the second aspect above, the first reference block is segmented to obtain a segmentation map, including: segmenting the first reference block based on the edge detection results of the first reference block to obtain a segmentation map.

[0073] The second aspect and any implementation thereof correspond to the first aspect and any implementation thereof, respectively. The technical effects of the second aspect and any implementation thereof are similar to those of the first aspect and any implementation thereof, and will not be repeated here.

[0074] Thirdly, embodiments of this application provide a bitstream generation method, which includes: encoding based on a first motion vector corresponding to the current block to obtain a first bitstream; wherein the first motion vector is determined based on the first aspect and any implementation thereof or the second aspect and any implementation thereof.

[0075] In one possible approach, the index of the first motion vector corresponding to the candidate list of motion vectors can be encoded to obtain the first bitstream.

[0076] In one possible approach, the vector difference between the first motion vector and the third motion vector corresponding to the coded block is encoded to obtain the first bitstream.

[0077] In one possible approach, the first motion vector can be encoded to obtain the first bitstream.

[0078] According to the third aspect, the method further includes: writing the region indicator corresponding to the first motion vector into the first bitstream.

[0079] For example, when the second target region is the first region in the segmentation map, and all pixels in the first region have a pixel value of 0, the region indicator can be 0. When all pixels in the first region have a pixel value of 1, the region indicator can be 1. For instance, the parameter `judge_flag` can be written into the first bitstream, and the region indicator can be written into the first bitstream as the parameter value of `judge_flag`. It should be understood that it is also possible to omit the region indicator corresponding to the first motion vector from writing it into the first bitstream. In this case, the encoding and decoding ends can pre-agree on the method for determining the first target region.

[0080] According to the third aspect, or any implementation of the third aspect above, the method further includes: encoding based on the second motion vector corresponding to the current block to obtain a second bitstream; wherein the second motion vector is determined based on the first aspect and any implementation of the first aspect or the second aspect and any implementation of the second aspect.

[0081] Fourthly, embodiments of this application provide a bitstream generation method, which includes: encoding based on a second motion vector corresponding to the current block to obtain a bitstream; wherein the second motion vector is determined based on the second aspect and any implementation thereof.

[0082] It should be noted that if the current block is divided into a geometric partition (the current block is divided into a fifth region and a sixth region) to determine the first motion vector corresponding to the fifth region and the second motion vector corresponding to the sixth region in the current block, it is not necessary to encode based on the first motion vector or the second motion vector. Instead, the partitioning information of the geometric partition can be encoded.

[0083] Fifthly, this application provides a code stream transmission method for transmitting code streams generated by the third aspect and any implementation thereof or the fourth aspect and any implementation thereof.

[0084] In a sixth aspect, embodiments of this application provide an electronic device, including: a memory and a processor, the memory being coupled to the processor; the memory storing program instructions, when executed by the processor, causing the electronic device to perform the inter-frame prediction method in the first aspect or any possible implementation of the first aspect.

[0085] The sixth aspect and any implementation thereof correspond to the first aspect and any implementation thereof, respectively. The technical effects of the sixth aspect and any implementation thereof are similar to those of the first aspect and any implementation thereof, and will not be repeated here.

[0086] In a seventh aspect, embodiments of this application provide an electronic device, including: a memory and a processor, the memory being coupled to the processor; the memory storing program instructions, when executed by the processor, causing the electronic device to perform the inter-frame prediction method in the second aspect or any possible implementation of the second aspect.

[0087] The seventh aspect and any implementation thereof correspond to the second aspect and any implementation thereof, respectively. The technical effects corresponding to the seventh aspect and any implementation thereof are similar to those corresponding to the second aspect and any implementation thereof, and will not be repeated here.

[0088] Eighthly, embodiments of this application provide a chip including one or more interface circuits and one or more processors; the interface circuits are used to receive signals from the memory of an electronic device and send signals to the processors, the signals including computer instructions stored in the memory; when the processor executes the computer instructions, it causes the electronic device to perform the inter-frame prediction method in the first aspect or any possible implementation of the first aspect.

[0089] The eighth aspect and any implementation thereof correspond to the first aspect and any implementation thereof, respectively. The technical effects corresponding to the eighth aspect and any implementation thereof are similar to those corresponding to the first aspect and any implementation thereof, and will not be repeated here.

[0090] Ninthly, embodiments of this application provide a chip including one or more interface circuits and one or more processors; the interface circuits are used to receive signals from the memory of an electronic device and send signals to the processors, the signals including computer instructions stored in the memory; when the processor executes the computer instructions, it causes the electronic device to perform the inter-frame prediction method in the second aspect or any possible implementation of the second aspect.

[0091] The ninth aspect and any implementation thereof correspond to the second aspect and any implementation thereof, respectively. The technical effects corresponding to the ninth aspect and any implementation thereof are similar to those corresponding to the second aspect and any implementation thereof, and will not be repeated here.

[0092] In a tenth aspect, embodiments of this application provide a computer-readable storage medium storing a computer program that, when run on a computer or processor, causes the computer or processor to perform the inter-frame prediction method in the first aspect or any possible implementation thereof.

[0093] The tenth aspect and any implementation thereof correspond to the first aspect and any implementation thereof, respectively. The technical effects corresponding to the tenth aspect and any implementation thereof are similar to those corresponding to the first aspect and any implementation thereof, and will not be repeated here.

[0094] Eleventhly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when run on a computer or processor, causes the computer or processor to perform inter-frame prediction in the second aspect or any possible implementation thereof.

[0095] The eleventh aspect and any implementation thereof correspond to the second aspect and any implementation thereof, respectively. The technical effects corresponding to the eleventh aspect and any implementation thereof can be found in the technical effects corresponding to the second aspect and any implementation thereof, as described above, and will not be repeated here.

[0096] In a twelfth aspect, embodiments of this application provide a computer program product, which includes a software program that, when executed by a computer or processor, causes the computer or processor to perform the inter-frame prediction method in the first aspect or any possible implementation thereof.

[0097] The twelfth aspect and any implementation thereof correspond to the first aspect and any implementation thereof, respectively. The technical effects corresponding to the twelfth aspect and any implementation thereof are the same as those corresponding to the first aspect and any implementation thereof, and will not be repeated here.

[0098] In a thirteenth aspect, embodiments of this application provide a computer program product, which includes a software program that, when executed by a computer or processor, causes the computer or processor to perform the inter-frame prediction method in the second aspect or any possible implementation thereof.

[0099] The thirteenth aspect and any implementation thereof correspond to the second aspect and any implementation thereof, respectively. The technical effects corresponding to the thirteenth aspect and any implementation thereof are as described above for the technical effects corresponding to the second aspect and any implementation thereof, and will not be repeated here.

[0100] In a fourteenth aspect, embodiments of this application provide an inter-frame prediction apparatus, the apparatus comprising:

[0101] The first motion vector acquisition module is used to acquire the first motion vector corresponding to the current block;

[0102] The segmentation map determination module is used to obtain a first segmentation map of the first reference block corresponding to the current block based on the first motion vector. The first segmentation map is used to describe the partitioning of the current block and includes a first region and a second region.

[0103] The target area determination module is used to determine a first target area from the first area and the second area;

[0104] The candidate list building module is used to build a motion vector candidate list based on the spatial distribution of the first target region in the first segmentation map. The spatial distribution is used to describe the position of the first target region in the first segmentation map.

[0105] The second motion vector determination module is used to determine the second motion vector corresponding to the current block based on the motion vector candidate list. The second motion vector corresponds to the third region in the current block, the first motion vector corresponds to the fourth region in the current block, the third region corresponds to the first target region, and the fourth region corresponds to the second target region in the first and second regions.

[0106] The motion compensation module is used to perform motion compensation based on the target segmentation map, the first motion vector, and the second motion vector to obtain the predicted block corresponding to the current block. The target segmentation map is determined based on the first segmentation map and / or the second segmentation map, and the second segmentation map is the segmentation map of the second reference block corresponding to the current block determined based on the second motion vector.

[0107] According to the fourteenth aspect, the motion vector candidate list includes multiple third motion vectors corresponding to multiple adjacent blocks of the current block; the distance between the adjacent block corresponding to the i-th third motion vector in the motion vector candidate list and the third region is less than the distance between the adjacent block corresponding to the (i+1)-th third motion vector and the third region; i is a positive integer.

[0108] According to aspect fourteen, or any implementation of aspect fourteen above, the second motion vector determination module is specifically used to select the first third motion vector in the motion vector candidate list as the second motion vector.

[0109] According to aspect fourteen, or any implementation of aspect fourteen above, the candidate list building module is specifically used to determine the spatial distribution type corresponding to the first target region from multiple preset spatial distribution types based on the spatial distribution of the first target region in the first segmentation map; based on the spatial distribution type corresponding to the first target region, determine a motion vector candidate list from a preset candidate list set, the preset candidate list set including multiple preset motion vector candidate lists corresponding to multiple preset spatial distribution types, the preset motion vector candidate list including multiple neighbor block identifiers, the neighbor block identifiers being used to represent the adjacent blocks at a position of the current block; and according to the multiple neighbor block identifiers included in the motion vector candidate list, fill the motion vector candidate list with the third motion vectors corresponding to the multiple adjacent blocks of the current block.

[0110] According to aspect fourteen, or any implementation thereof, both the first motion vector and the second motion vector include an integer part and a fractional part; the motion compensation module is specifically used to perform integer-pixel-level motion compensation based on the target segmentation map, the first reference block, and the second reference block to determine an integer-pixel prediction block, wherein the first reference block is determined based on the integer part of the first motion vector, and the second reference block is determined based on the integer part of the second motion vector; and to perform fractional-pixel-level motion compensation based on the target segmentation map, the integer-pixel prediction block, the fractional part of the first motion vector, and the fractional part of the second motion vector to obtain the prediction block corresponding to the current block.

[0111] According to aspect fourteen, or any implementation thereof, the motion compensation module is specifically used to linearly fuse the first reference block and the second reference block based on the target segmentation map to determine the integer pixel prediction block.

[0112] According to aspect fourteen, or any implementation thereof, the motion compensation module is specifically used to determine the pixel category corresponding to each pixel in the first reference block and the pixel category corresponding to each pixel in the second reference block based on the distance between each pixel in the target segmentation image and the segmentation line, wherein the segmentation line is the segmentation line between the first region and the second region in the target segmentation image; determine the first weight corresponding to each pixel in the first reference block based on the target segmentation image and the pixel category corresponding to each pixel in the first reference block; determine the second weight corresponding to each pixel in the second reference block based on the pixel category corresponding to each pixel in the target segmentation image and the second reference block; and linearly weight the pixel values ​​of corresponding pixels in the first reference block and the second reference block based on the first weight and the second weight to obtain an integer pixel prediction block.

[0113] According to aspect fourteen, or any implementation thereof, the motion compensation module is specifically used to divide the integer pixel prediction block into an eighth region and a ninth region based on the dividing line between the first region and the second region in the target segmentation map; to perform pixel interpolation on the eighth region based on the fractional part of the first motion vector to obtain a first pixel prediction block; and to perform pixel interpolation on the ninth region based on the fractional part of the second motion vector to obtain a second pixel prediction block; and to perform linear fusion of the first pixel prediction block and the second pixel prediction block based on the target segmentation map to obtain a prediction block.

[0114] According to aspect fourteen, or any implementation thereof, the inter-frame prediction apparatus further includes: a candidate list set establishment module, used to establish a preset candidate list set; wherein,

[0115] The candidate list set establishment module is specifically used to obtain multiple preset segmentation maps corresponding to multiple preset image blocks. The preset segmentation maps describe the partitioning of the preset image blocks and include a fifth region and a sixth region. Based on the spatial distribution of the third target region in each preset segmentation map, classification is performed to obtain multiple preset spatial distribution types. The third target region is selected from the fifth and sixth regions. For the target spatial distribution type among the multiple preset spatial distribution types, based on the positional relationship between the seventh region in the target preset image block corresponding to the target spatial distribution type and multiple adjacent blocks of the target preset image block, a preset motion vector candidate list corresponding to the target spatial distribution type is determined. The seventh region corresponds to the third target region. Using the preset motion vector candidate lists corresponding to various preset spatial distribution types, a preset candidate list set is established.

[0116] According to aspect fourteen, or any implementation of aspect fourteen above, the candidate list set establishment module is specifically used to determine the distance between the seventh region in the target preset image block and multiple adjacent blocks of the target preset image block; based on the distance between the seventh region in the target preset image block and multiple adjacent blocks of the target preset image block, determine the arrangement order of multiple adjacent blocks of the target preset image block; and according to the arrangement order, fill the neighbor block identifiers corresponding to the multiple adjacent blocks of the target preset image block into the preset motion vector candidate list corresponding to the target spatial distribution type.

[0117] According to aspect fourteen, or any implementation of aspect fourteen above, in the preset motion vector candidate list corresponding to the target spatial distribution type, the distance between the neighboring block corresponding to the kth neighboring block identifier and the seventh region in the target preset image block is less than the distance between the neighboring block corresponding to the (k+1)th neighboring block identifier and the seventh region in the target preset image block; k is a positive integer.

[0118] According to aspect fourteen, or any implementation of aspect fourteen above, the target region determination module is specifically used to determine the region with the smallest area in the first region and the second region as the first target region; wherein, the second target region is the region with the largest area in the first region and the second region.

[0119] According to aspect fourteen, or any implementation of aspect fourteen above, the target segmentation map is a first segmentation map; or, the target segmentation map is a second segmentation map; or, the target segmentation map is a segmentation map obtained by fusing the first segmentation map and the second segmentation map.

[0120] In a fifteenth aspect, embodiments of this application provide an inter-frame prediction apparatus, the apparatus comprising:

[0121] The motion vector acquisition module is used to acquire the first motion vector corresponding to the current block and the second motion vector corresponding to the current block. Both the first and second motion vectors include an integer part and a fractional part.

[0122] The reference block determination module is used to determine a first reference block of the current block based on the integer part of a first motion vector, and to determine a second reference block of the current block based on the integer part of a second motion vector.

[0123] The segmentation map acquisition module is used to acquire segmentation maps, which describe the partitioning of the current block.

[0124] The integer pixel motion compensation module is used to perform integer pixel-level motion compensation based on the segmentation map, the first reference block, and the second reference block, and to determine the integer pixel prediction block.

[0125] The pixel-level motion compensation module is used to perform pixel-level motion compensation based on the segmentation map, the integer pixel prediction block, the fractional part of the first motion vector, and the fractional part of the second motion vector to obtain the prediction block corresponding to the current block.

[0126] According to the fifteenth aspect, the integer pixel motion compensation module is specifically used to perform linear fusion of the first reference block and the second reference block based on the segmentation map to determine the integer pixel prediction block.

[0127] According to aspect fifteen, or any implementation thereof, the integer pixel motion compensation module is specifically used to determine the pixel category corresponding to each pixel in the first reference block and the pixel category corresponding to each pixel in the second reference block based on the distance between each pixel in the segmentation map and the segmentation line, wherein the segmentation line is the segmentation line between the first region and the second region in the segmentation map; determine the first weight corresponding to each pixel in the first reference block based on the segmentation map and the pixel category corresponding to each pixel in the first reference block; determine the second weight corresponding to each pixel in the second reference block based on the pixel category corresponding to each pixel in the segmentation map and the second reference block; and linearly weight the pixel values ​​of the corresponding pixels in the first reference block and the second reference block based on the first weight and the second weight to obtain an integer pixel prediction block.

[0128] According to aspect fifteen, or any implementation thereof, the pixel-level motion compensation module is specifically used to divide the integer pixel prediction block into a third region and a fourth region based on the dividing line between the first region and the second region in the segmentation map; to perform pixel-level interpolation on the third region based on the fractional part of the first motion vector to obtain a first pixel-level prediction block; and to perform pixel-level interpolation on the fourth region based on the fractional part of the second motion vector to obtain a second pixel-level prediction block; and to perform linear fusion of the first pixel-level prediction block and the second pixel-level prediction block based on the segmentation map to obtain a prediction block.

[0129] According to aspect 15, or any implementation thereof, the current block is divided into a fifth region and a sixth region, and the segmentation map includes a first region corresponding to the fifth region and a second region corresponding to the sixth region; a first motion vector corresponds to the fifth region, and a second motion vector corresponds to the sixth region; the motion vector acquisition module is specifically used to establish a first motion vector candidate list based on the spatial distribution of the first region in the segmentation map, the spatial distribution being used to describe the position of the first region in the segmentation map; and to determine the first motion vector corresponding to the fifth region in the current block based on the first motion vector candidate list.

[0130] According to aspect 15, or any implementation thereof, the current block is divided into a fifth region and a sixth region, and the segmentation map includes a first region corresponding to the fifth region and a second region corresponding to the sixth region; a first motion vector corresponds to the fifth region, and a second motion vector corresponds to the sixth region; the motion vector acquisition module is specifically used to establish a candidate list of second motion vectors based on the spatial distribution of the second region in the segmentation map, the spatial distribution being used to describe the position of the second region in the segmentation map; and based on the candidate list of second motion vectors, to determine the second motion vector corresponding to the sixth region in the current block.

[0131] According to aspect 15, or any implementation thereof, the second motion vector candidate list includes multiple third motion vectors corresponding to multiple adjacent blocks of the current block; the distance between the adjacent block corresponding to the i-th third motion vector in the second motion vector candidate list and the sixth region is less than the distance between the adjacent block corresponding to the (i+1)-th third motion vector and the sixth region; i is a positive integer.

[0132] According to aspect 15, or any implementation thereof, the motion vector acquisition module is specifically used to select the first third motion vector in the second motion vector candidate list as the second motion vector.

[0133] According to aspect 15, or any implementation thereof, the motion vector acquisition module is specifically used to determine the spatial distribution type corresponding to the second region from a variety of preset spatial distribution types based on the spatial distribution of the second region in the segmentation map; to determine a second motion vector candidate list from a preset candidate list set based on the spatial distribution type corresponding to the second region, the preset candidate list set including multiple preset motion vector candidate lists corresponding to multiple preset spatial distribution types, the preset motion vector candidate list including multiple neighbor block identifiers, the neighbor block identifiers being used to represent the neighboring blocks at a position of the current block; and to fill the second motion vector candidate list with the third motion vectors corresponding to the multiple neighbor blocks of the current block according to the multiple neighbor block identifiers included in the second motion vector candidate list.

[0134] According to aspect fifteen, or any implementation thereof, the inter-frame prediction apparatus further includes: a candidate list set establishment module, used to establish a preset candidate list set; wherein...

[0135] The candidate list set establishment module is specifically used to obtain multiple preset segmentation maps corresponding to multiple preset image blocks. The preset segmentation maps describe the partitioning of the preset image blocks and include a seventh region and an eighth region. Based on the spatial distribution of the target region in each preset segmentation map, classification is performed to obtain multiple preset spatial distribution types. The target region is selected from the seventh region and the eighth region. For the target spatial distribution type among the multiple preset spatial distribution types, based on the positional relationship between the target spatial distribution type and the ninth region in the target preset image block, and multiple adjacent blocks of the target preset image block, a preset motion vector candidate list corresponding to the target spatial distribution type is determined. The ninth region corresponds to the third target region. Using the preset motion vector candidate lists corresponding to various preset spatial distribution types, a preset candidate list set is established.

[0136] According to aspect 15, or any implementation thereof, the candidate list set establishment module is specifically used to determine the distance between the ninth region in the target preset image block and multiple adjacent blocks of the target preset image block; based on the distance between the ninth region in the target preset image block and multiple adjacent blocks of the target preset image block, determine the arrangement order of the multiple adjacent blocks of the target preset image block; and according to the arrangement order, fill the neighboring block identifiers corresponding to the multiple adjacent blocks of the target preset image block into the preset motion vector candidate list corresponding to the target spatial distribution type.

[0137] According to aspect 15, or any implementation of aspect 15 above, in the preset motion vector candidate list corresponding to the target spatial distribution type, the distance between the neighboring block corresponding to the kth neighboring block identifier and the ninth region in the target preset image block is less than the distance between the neighboring block corresponding to the (k+1)th neighboring block identifier and the ninth region in the target preset image block; k is a positive integer.

[0138] According to aspect 15, or any implementation thereof, the segmentation map acquisition module is specifically used to segment a first reference block to obtain a segmentation map; or to segment a second reference block to obtain a segmentation map; or to fuse the first segmentation map obtained by segmenting the first reference block and the second segmentation map obtained by segmenting the second reference block to obtain a segmentation map.

[0139] In a sixteenth aspect, embodiments of this application provide an apparatus for storing a bitstream, the apparatus comprising: a receiver and at least one storage medium, the receiver being used to receive the bitstream; at least one storage medium being used to store the bitstream; the bitstream being generated according to the third aspect and any implementation thereof or the fourth aspect and any implementation thereof.

[0140] In a seventeenth aspect, embodiments of this application provide a system for distributing bitstreams, the system comprising: at least one storage medium for storing at least one bitstream, the at least one bitstream being generated according to the third aspect and any implementation thereof or the fourth aspect and any implementation thereof; and a streaming media device for obtaining a target bitstream from the at least one storage medium and sending the target bitstream to an end-side device, wherein the streaming media device includes a content server or a content distribution server.

[0141] Eighteenthly, embodiments of this application provide an apparatus for transmitting a bitstream, the apparatus comprising: a transmitter and at least one storage medium, the at least one storage medium being used to store the bitstream, the bitstream being generated according to the third aspect and any implementation thereof or the fourth aspect and any implementation thereof; the transmitter being used to obtain the bitstream from the storage medium and transmit the bitstream to an end-side device via the transmission medium. Attached Figure Description

[0142] Figure 1a This is a schematic diagram illustrating the encoding and decoding process as an example.

[0143] Figure 1b This is a schematic diagram illustrating the inter-frame prediction process of this application;

[0144] Figure 2a This is a schematic diagram illustrating the inter-frame prediction process as an example.

[0145] Figure 2b A schematic diagram illustrating an image patch and segmentation map as an example;

[0146] Figure 2c This is a schematic diagram illustrating the inter-frame prediction process as an example.

[0147] Figure 3a This is an illustrative diagram illustrating the process of creating a preset candidate list set.

[0148] Figure 3b This is a schematic diagram illustrating a preset spatial distribution type;

[0149] Figure 3c This is an example illustration of candidate Merge locations;

[0150] Figure 4a This is a schematic diagram illustrating an example of the inter-frame prediction process;

[0151] Figure 4b This is a schematic diagram illustrating an image segmentation process as an example.

[0152] Figure 4c This is a schematic diagram illustrating the process of determining a segmentation map;

[0153] Figure 4d A schematic diagram illustrating a linear fusion process as an example;

[0154] Figure 4e A schematic diagram illustrating a linear fusion process as an example;

[0155] Figure 4f This is a schematic diagram illustrating the motion compensation process as an example.

[0156] Figure 5 This is a schematic diagram illustrating the inter-frame prediction process as an example.

[0157] Figure 6a This is a schematic diagram illustrating the inter-frame prediction process as an example.

[0158] Figure 6b A schematic diagram of geometric partitioning is shown as an example;

[0159] Figure 7 This is a schematic diagram of an exemplary inter-frame prediction device;

[0160] Figure 8 This is a schematic diagram of an exemplary inter-frame prediction device;

[0161] Figure 9 This is a schematic diagram of the structure of an exemplary device. Detailed Implementation

[0162] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0163] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0164] The terms "first" and "second," etc., used in the specification and claims of this application are used to distinguish different objects, not to describe a specific order of objects. For example, "first target object" and "second target object," etc., are used to distinguish different target objects, not to describe a specific order of target objects.

[0165] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0166] In the description of the embodiments in this application, unless otherwise stated, "multiple" means two or more. For example, multiple processing units means two or more processing units; multiple systems means two or more systems.

[0167] Figure 1a This is a schematic diagram illustrating the encoding and decoding process as an example.

[0168] Reference Figure 1a (1) The encoding process is explained.

[0169] For example, after the encoding end obtains the video to be encoded, it can divide each frame of the video into blocks to obtain blocks to be encoded. For each block to be encoded, it can be determined whether to perform traditional inter-frame prediction (i.e., use the traditional inter-frame prediction method for inter-frame prediction), or inter-frame prediction of this application (i.e., use the inter-frame prediction method of this application for inter-frame prediction), or intra-frame prediction.

[0170] When intra-frame prediction is determined, intra-frame prediction can be performed based on the reconstructed frames in the reconstructed frame buffer to determine the prediction block corresponding to the block to be coded. After obtaining the prediction block, on the one hand, the residual information between the block to be coded and the prediction block can be determined, and the residual information can be transformed, quantized, and entropy encoded to obtain the bitstream. On the other hand, the residual information obtained by quantization can be dequantized, inversely transformed, and superimposed with the prediction block to obtain the reconstructed block. Then, the reconstructed block is subjected to loop filtering, and the loop-filtered reconstructed block is concatenated to the corresponding position of the frame to be reconstructed in the reconstructed frame buffer (wherein, after concatenating the last reconstructed block of the frame to be reconstructed, the reconstructed frame can be obtained).

[0171] When it is determined that the inter-frame prediction (or traditional inter-frame prediction) is applicable, inter-frame prediction (or traditional inter-frame prediction) can be performed based on the reconstructed frames in the reconstructed frame buffer to determine the motion vector corresponding to the block to be coded and the prediction block corresponding to the block to be coded. After obtaining the prediction block, on the one hand, the residual information between the block to be coded and the prediction block can be determined, and the residual information can be transformed, quantized, and entropy encoded. The motion vector can also be entropy encoded to obtain the bitstream. On the other hand, the residual information obtained by quantization can be dequantized, inverse transformed, and the prediction blocks can be superimposed to obtain the reconstructed block. Then, the reconstructed block is subjected to loop filtering, and the loop-filtered reconstructed block is concatenated to the corresponding position of the frame to be reconstructed in the reconstructed frame buffer (wherein, after concatenating the last reconstructed block of the frame to be reconstructed, the reconstructed frame can be obtained).

[0172] Figure 1b This is a schematic diagram illustrating the inter-frame prediction process of this application as an example.

[0173] Reference Figure 1b (1) For example, in the encoding process, the inter-frame prediction of this application refers to: determining multiple motion vectors corresponding to multiple motion fields in the current block through motion search (e.g., performing motion search based on the segmentation map of the reference block corresponding to the block to be encoded); and then based on the segmentation map of the reference block corresponding to the block to be encoded, multiple motion vectors, and a reference frame (the reference frame is a reconstruction frame in the reconstruction frame buffer, Figure 2b (1) Motion compensation is performed to determine the prediction block. The segmentation map can be a binary map (where the area corresponding to the foreground region of the reference block is black (or white) and the area corresponding to the background region of the reference block is white (or black)).

[0174] For example, in the encoding process, traditional inter-frame prediction refers to: determining a block-level motion vector corresponding to the current block through motion search, and then performing motion compensation based on the block-level motion vector and the reference frame to determine the predicted block. In other words, in the traditional inter-frame prediction process, the segmentation map of the reference block corresponding to the block to be encoded is not used for motion compensation.

[0175] Reference Figure 1a (2) The decoding process is explained.

[0176] For example, after receiving the bitstream, the decoding end can perform entropy decoding on the bitstream.

[0177] If entropy decoding only yields the encoded residual information of the block to be reconstructed, but not the motion vector, then intra-frame prediction of the block to be reconstructed is necessary. On one hand, the encoded residual information of the block to be reconstructed obtained from entropy decoding can be inversely quantized and inversely transformed to obtain the residual information of the block to be reconstructed. On the other hand, intra-frame prediction can be performed based on the reconstructed frames in the reconstructed frame buffer to determine the prediction block corresponding to the block to be reconstructed; then, the residual information can be superimposed with the prediction block to obtain the reconstructed block. Next, loop filtering can be applied to the reconstructed block, and the loop-filtered reconstructed block can be concatenated to the corresponding position of the frame to be reconstructed in the reconstructed frame buffer; after concatenating the last reconstructed block of the frame to be reconstructed, the reconstructed frame can be obtained and output; the multiple output reconstructed frames can constitute the reconstructed video.

[0178] If entropy decoding yields the encoded information of the motion vector and the residual information of the block to be reconstructed, then when it is determined to be frame prediction (or traditional inter-frame prediction), on the one hand, the encoded information of the residual information of the block to be reconstructed obtained by entropy decoding can be inversely quantized and inversely transformed to obtain the residual information of the block to be reconstructed. On the other hand, based on the reconstructed frame and motion vector in the reconstructed frame buffer, inter-frame prediction (or traditional inter-frame prediction) can be performed to determine the prediction block corresponding to the block to be reconstructed. Then, the residual information can be superimposed with the prediction block to obtain the reconstructed block. Afterward, loop filtering can be applied to the reconstructed block, and the loop-filtered reconstructed block can be concatenated to the corresponding position of the frame to be reconstructed in the reconstructed frame buffer. After the last reconstructed block of the frame to be reconstructed is concatenated, the reconstructed frame can be obtained and output. The multiple output reconstructed frames can constitute the reconstructed video.

[0179] Reference Figure 1b (2) In the decoding process, the inter-frame prediction in this application refers to the prediction based on the motion vectors parsed from the bitstream and the reference frame corresponding to the block to be reconstructed. Figure 1b (2) The segmentation map of the reference block corresponding to the block to be reconstructed is used to perform motion compensation to determine the prediction block.

[0180] For example, in the decoding process, traditional inter-frame prediction refers to extracting the reference block of the block to be reconstructed from the reference frame corresponding to the block to be reconstructed based on motion vectors, and using it as the prediction block.

[0181] It should be understood that, Figure 1a This is merely an example; the coding process of this application may include more than... Figure 1a (1) The more or fewer operations shown, and the decoding process of this application may include more than Figure 1a (2) The application does not limit the number of operations shown.

[0182] It should be understood that when it is determined that inter-frame prediction (traditional inter-frame prediction or inter-frame prediction of this application) will be performed on the block to be coded, during the encoding process, it is not necessary to calculate the residual information between the predicted block and the block to be coded, nor is it necessary to encode the residual information; only the motion vector needs to be encoded. During the decoding process, after parsing the motion vector and determining the predicted block based on the motion vector, the predicted block can be used as the reconstructed block. In other words, this application does not restrict whether residual information is generated or encoded during the inter-frame coding process.

[0183] It should be noted that, referring to Figure 1a This application adds inter-frame prediction to the traditional encoding and decoding; it should be understood that the inter-frame prediction of this application can also be used to replace the traditional inter-frame prediction, and this application does not limit this.

[0184] The following describes the inter-frame prediction process of this application.

[0185] Figure 2a This is a schematic diagram illustrating an exemplary inter-frame prediction process. Figure 2a The inter-frame prediction process described in the embodiments can be an inter-frame prediction process in the encoding process or an inter-frame prediction process in the decoding process, and this application does not limit it in this way.

[0186] S201, obtain the first motion vector corresponding to the current block.

[0187] For example, when Figure 2a When the embodiment describes the inter-frame prediction process in the encoding process, the current block can be the block to be encoded in the frame to be encoded; at this time, the first motion vector corresponding to the current block can be determined based on the motion matching of the block.

[0188] For example, when Figure 2a When the embodiment describes the inter-frame prediction process in the decoding process, the current block can be the block to be reconstructed in the frame to be reconstructed; at this time, the first motion vector corresponding to the current block can be parsed from the bitstream.

[0189] S202, based on the first motion vector, obtain the first segmentation map of the first reference block corresponding to the current block. The first segmentation map is used to describe the partitioning of the current block and includes a first region and a second region.

[0190] For example, each reconstructed frame can be segmented to separate the foreground and background regions, resulting in a frame segmentation map (i.e., a frame-level segmentation map). This frame segmentation map describes the partitioning of the foreground and background regions of the reconstructed frame. The size of the frame segmentation map is the same as the size of the reconstructed frame.

[0191] For example, each reconstructed frame can be segmented upon acquisition to obtain a frame segmentation map for each reconstructed frame. Alternatively, during the inter-frame prediction process of this application, unsegmented reconstructed frames can be segmented separately to obtain a frame segmentation map for each reconstructed frame; this application does not impose any limitations on this approach.

[0192] For example, each reconstructed frame can be segmented using an AI (Artificial Intelligence) image segmentation scheme or a non-AI image segmentation scheme; this application does not limit the choice.

[0193] For example, a reference frame for the current block can be determined from one or more reconstructed frames. Then, based on the pixel coordinates of the current block (e.g., the pixel coordinates of the four vertices of a rectangle if the current block is rectangular) and a first motion vector, a first reference block corresponding to the current block can be determined from the reference frame. The size of the first reference block is the same as the size of the current block. Then, based on the pixel coordinates of the first reference block, a block segmentation map (i.e., a block-level segmentation map, hereinafter referred to as the first segmentation map) of the first reference block is extracted from the frame segmentation map corresponding to the reference frame of the current block. The size of the first segmentation map of the first reference block is the same as the size of the first reference block.

[0194] For example, the first segmentation map of the first reference block can describe the partitioning of the foreground and background regions of the first reference block. Since the first reference block is highly correlated with the current block, the first segmentation map of the first reference block can also describe the partitioning of the foreground and background regions of the current block; that is, based on the first segmentation map of the first reference block, the partitioning information of different motion fields of the current block can be obtained.

[0195] It should be understood that, alternatively, a first reference block corresponding to the current block can be determined from the reference frame of the current block based on the pixel coordinates of the current block and the first motion vector; then, a first segmentation map of the first reference block can be determined by segmenting the first reference block. The method of segmenting the first reference block is similar to the method of segmenting the reconstructed frame, and will not be elaborated here.

[0196] For example, the first segmentation map of the first reference block may include a first region and a second region.

[0197] In one possible approach, the first region corresponds to the foreground region of the first reference block and the foreground region of the current block, and the second region corresponds to the background region of the first reference block and the background region of the current block.

[0198] Figure 2b This is a schematic diagram of an image patch and a segmentation map, shown as an example.

[0199] Figure 2b (1) is a schematic diagram of the first reference block; Figure 2b In (1), the background area 1 is the basketball court floor. Figure 2b (1) The foreground area 1 is the athlete.

[0200] Figure 2b (2) is a schematic diagram of the current block; Figure 2b (2) Background area 2 is the basketball court floor. Figure 2b (2) The foreground area 2 is the athlete.

[0201] Figure 2b (3) is a schematic diagram of the first segmentation of the first reference block; Figure 2b (3) The first region in the first segmentation diagram of the first reference block corresponds to Figure 2b (1) The background region 1 of the first reference block, and the corresponding Figure 2b (2) The background area 2 of the current block. Figure 2b (3) The second region in the first segmentation diagram of the first reference block corresponds to Figure 2b (1) The foreground region 1 of the first reference block, and the corresponding Figure 2b (2) The foreground region 2 of the current block.

[0202] in, Figure 2b (3) The pixel value of the first region in the first segmentation image of the first reference block is 0 (i.e., the first region is black). Figure 2b (3) In the first segmentation image of the first reference block, the pixel value of the second region included in the second region is 1 (i.e., the second region is white). It should be understood that... Figure 2b (3) is just an example. Figure 2b(3) The pixel value of the first region in the first segmentation map of the first reference block can also be 1 (i.e., the first region is white). Figure 2b (3) The pixel value of the second region included in the first segmentation diagram of the first reference block is 0 (i.e. the second region is black); this application does not limit this.

[0203] In one possible approach, the first region corresponds to the background region of the first reference block and the background region of the current block, and the second region corresponds to the foreground region of the first reference block and the foreground region of the current block; the specific settings can be configured according to requirements, and this application does not impose any restrictions on this.

[0204] S203, determine the first target area from the first area and the second area.

[0205] For example, one region can be selected from the first region and the second region as the first target region, and the other region can be selected as the second target region.

[0206] In one possible approach, the first region can be designated as the first target region, and the second region as the second target region.

[0207] In one possible approach, the second region can be used as the first target region, and the first region can be used as the second target region.

[0208] It should be understood that this application does not restrict whether the first region or the second region is used as the first target region.

[0209] S204. Based on the spatial distribution of the first target region in the first segmentation map, establish a candidate list of motion vectors.

[0210] For example, after determining the first target region, a motion vector candidate list can be established based on the spatial distribution of the first target region in the first segmentation map. For example, the spatial distribution of the first target region in the first segmentation map, that is, the position of the first target region in the first segmentation map, can be used to describe the position of the third region corresponding to the first target region in the current block; thus, the determined motion vector candidate list is the motion vector candidate list corresponding to the third region in the current block.

[0211] For example, refer to Figure 2b If the first region is selected as the first target region, it can be determined that the first target region is located on the upper left side of the segmentation map; at this time, it can also be determined that the background region 2 (i.e. the third region) is located on the upper left side of the current block.

[0212] For example, refer to Figure 2bIf the second region is selected as the first target region, it can be determined that the first target region is located on the lower right side of the segmentation map; at this time, it can also be determined that the foreground region 2 (i.e. the third region) is located on the lower right side of the current block.

[0213] For example, the motion vector candidate list may include multiple motion vectors, which are the motion vectors of multiple adjacent blocks of the current block (hereinafter referred to as the third motion vector). The third motion vector may be the motion vector corresponding to any region (i.e., the motion field) in the adjacent blocks, and this application does not limit this.

[0214] S204, based on the motion vector candidate list, determine the second motion vector corresponding to the current block. The second motion vector corresponds to the third region in the current block, the first motion vector corresponds to the fourth region in the current block, the third region corresponds to the first target region, and the fourth region corresponds to the second target region in the first and second regions.

[0215] For example, a third motion vector can be selected from the candidate list of motion vectors according to a preset rule, and used as the second motion vector. For example, the preset rule can be set according to requirements, such as selecting the first third motion vector in the candidate list of motion vectors; or selecting the optimal third motion vector in the candidate list of motion vectors; etc., this application does not limit this.

[0216] For example, the second motion vector can be used as the motion vector corresponding to the third region in the current block; correspondingly, the first motion vector can be used as the motion vector corresponding to the fourth region in the current block, wherein the fourth region is any region in the current block other than the third region, and the fourth region corresponds to the second target region.

[0217] S205, Motion compensation is performed based on the target segmentation map, the first motion vector, and the second motion vector to obtain the prediction block corresponding to the current block.

[0218] For example, a second reference block corresponding to the current block can be determined from the reference frame of the current block based on the pixel coordinates of the current block (e.g., the pixel coordinates of the four vertices of the rectangle if the current block is rectangular) and the second motion vector. Then, a second segmentation map of the second reference block can be determined by referring to the method described above for determining the first segmentation map of the first reference block. Next, a target segmentation map is determined based on the first segmentation map and / or the second segmentation map.

[0219] In one possible approach, the target segmentation map is the first segmentation map.

[0220] In one possible approach, the target segmentation map is the second segmentation map.

[0221] In one possible approach, the target segmentation map is a segmentation map obtained by fusing the first segmentation map and the second segmentation map. For example, the pixel values ​​of corresponding pixels in the first segmentation map and the second segmentation map can be ANDed or ORed; this application does not limit this.

[0222] For example, the second segmentation map of the second reference block may include a tenth region and an eleventh region.

[0223] In one possible approach, the tenth region corresponds to the foreground region of the second reference block and the foreground region of the current block, and the eleventh region corresponds to the background region of the second reference block and the background region of the current block.

[0224] In one possible approach, the eleventh region corresponds to the background region of the second reference block and the background region of the current block, and the tenth region corresponds to the foreground region of the second reference block and the foreground region of the current block; the specific settings can be configured according to requirements, and this application does not impose any restrictions on this.

[0225] For example, the target segmentation map may include a twelfth region and a thirteenth region.

[0226] In one possible configuration, the twelfth region corresponds to the first region of the first segmentation map and the tenth region of the second segmentation map; the thirteenth region corresponds to the second region of the first segmentation map and the eleventh region of the second segmentation map.

[0227] In one possible configuration, the twelfth region corresponds to the second region of the first segmentation map and the eleventh region of the second segmentation map; the thirteenth region corresponds to the first region of the first segmentation map and the tenth region of the second segmentation map; the specific configuration can be set according to requirements, and this application does not impose any restrictions on it.

[0228] Subsequently, based on the target segmentation map, the first reference block and the second reference block corresponding to the current block can be fused to obtain the prediction block corresponding to the current block. The size of the prediction block is the same as the size of the current block.

[0229] For example, suppose the twelfth region of the target segmentation map corresponds to the first region of the first segmentation map, and the thirteenth region of the target segmentation map corresponds to the second region of the first segmentation map. In one possible approach, when the first target region is the first region, region A corresponding to the thirteenth region of the target segmentation map can be extracted from the first reference block, and region B corresponding to the twelfth region of the target segmentation map can be extracted from the second reference block; then, regions A and B are concatenated to obtain the predicted block corresponding to the current block. When the first target region is the second region, region A corresponding to the twelfth region of the target segmentation map can be extracted from the first reference block, and region B corresponding to the thirteenth region of the segmentation map can be extracted from the second reference block; then, regions A and B are concatenated to obtain the predicted block corresponding to the current block.

[0230] In one possible approach, the pixel values ​​of each pixel in the first reference block and the pixel values ​​of each pixel in the second reference block can be linearly fused based on the target segmentation map to obtain the prediction block corresponding to the current block.

[0231] It should be understood that other methods can also be used to fuse the first reference block and the second reference block based on the target segmentation map, and this application does not limit this.

[0232] Figure 2c This is a schematic diagram illustrating an exemplary inter-frame prediction process. Figure 2c In the embodiments described, the inter-frame prediction process in the encoding process is illustrated with practical examples. Among them, Figure 2c The target segmentation map in the embodiment is the first segmentation map.

[0233] Reference Figure 2c For example, a first motion vector can be determined based on block motion matching; then, a first segmentation map of the first reference block can be determined based on the first motion vector. In the first segmentation map, a first region corresponds to the background region of the current block, and a second region corresponds to the foreground region of the current block.

[0234] If the first region is defined as the first target region, and the second region as the second target region, then the two motion fields of the current block can be: region B corresponding to the first region (i.e., corresponding to the third region mentioned above), and region A corresponding to the second region (i.e., corresponding to the fourth region mentioned above). Accordingly, the first motion vector determined based on block motion matching can be used as the motion vector corresponding to region A; and based on the first segmentation map, motion matching is performed on region B to determine the second motion vector, which is the motion vector corresponding to region B. Finally, motion compensation can be performed based on the first segmentation map, the first motion vector, and the second motion vector to obtain the predicted block.

[0235] If the second region is determined as the first target region, and the first region is determined as the second target region, then the two motion fields of the current block can be: region B corresponding to the first region (i.e., the fourth region mentioned above), and region A corresponding to the second region (i.e., the third region mentioned above). Accordingly, the first motion vector determined based on block motion matching can be used as the motion vector corresponding to region B; and based on the first segmentation map, motion matching is performed on region A to determine the second motion vector, which is the motion vector corresponding to region A. Finally, motion compensation can be performed based on the first segmentation map, the first motion vector, and the second motion vector to obtain the predicted block.

[0236] It should be noted that, Figure 2cThis is only to indicate that the partitioning information of different motion fields of the current block can be obtained based on the first segmentation map; in the actual inter-frame prediction process, the operation of dividing the current block according to the first segmentation map is not performed.

[0237] On the one hand, since the first reference block and the current block are highly correlated, partitioning the first reference block using its first segmentation map can be applied to partitioning the current block; and the first segmentation map can accurately and efficiently segment different motion fields (such as foreground and background regions). Similarly, since the second reference block and the current block are highly correlated, partitioning the second reference block using its second segmentation map can be applied to partitioning the current block; and the second segmentation map can accurately and efficiently segment different motion fields (such as foreground and background regions). Therefore, the target segmentation map determined based on the first and / or second segmentation maps can accurately and efficiently partition different motion fields of the current block; thus, inter-frame prediction based on motion vectors of different motion fields of the current block can improve the accuracy of inter-frame prediction, thereby improving coding quality.

[0238] On the other hand, by determining the method for constructing the candidate list of motion vectors based on the distribution information of different motion fields in the current block, the coding efficiency of motion vectors for different motion fields can be improved.

[0239] The following section details the process of creating a candidate list of motion vectors.

[0240] For example, a preset candidate list set can be established in advance, wherein the preset candidate list set may include multiple preset motion vector candidate lists; then, based on the spatial distribution of the first target region in the segmentation map, a motion vector candidate list is selected from the preset candidate list set.

[0241] The following is an illustrative example of the process of creating a preset candidate list set.

[0242] Figure 3a This is a schematic diagram illustrating the process of establishing a preset candidate list set as an example. Figure 3a The process of establishing a preset candidate list set described in the embodiments can be the process of establishing a preset candidate list set during the encoding process or the process of establishing a preset candidate list set during the decoding process, and this application does not limit it in this way.

[0243] S301. Obtain multiple preset segmentation maps corresponding to multiple preset image blocks. The preset segmentation maps are used to describe the partitioning of the preset image blocks. The preset segmentation maps include the fifth region and the sixth region.

[0244] For example, one or more video sequences can be acquired; then, on the one hand, each image frame in the video sequence can be divided into blocks to obtain multiple preset image blocks. The sizes of these multiple preset image blocks can be the same or different, and this application does not impose any limitations on this. On the other hand, each image frame in the video sequence can be segmented to determine a frame segmentation map corresponding to each image frame in the video sequence.

[0245] Next, for each of the multiple preset image blocks, a preset segmentation map (a block-level segmentation map) can be determined from the frame segmentation map of the corresponding image frame based on the pixel coordinates of that preset image block. Each preset segmentation map has the same size as the corresponding preset image block and can be used to describe the partitioning of the foreground and background regions of the corresponding preset image block. In this way, multiple preset segmentation maps can be obtained.

[0246] For example, each preset segmentation map may include a fifth region and a sixth region. In one possible embodiment, the fifth region in each preset segmentation map corresponds to the foreground region of the preset image block corresponding to the preset segmentation map, and the sixth region corresponds to the background region of the preset image block corresponding to the preset segmentation map. In another possible embodiment, the fifth region in each preset segmentation map corresponds to the background region of the preset image block corresponding to the preset segmentation map, and the sixth region corresponds to the foreground region of the preset image block corresponding to the preset segmentation map; the specific configuration can be determined according to requirements, and this application does not impose any limitations on this.

[0247] S302. Classify the spatial distribution of the third target region based on the spatial distribution of each preset segmentation map to obtain multiple preset spatial distribution types. The third target region is selected from the fifth and sixth regions.

[0248] For example, one region can be selected from the fifth and sixth regions as the third target region, and the other region can be selected as the fourth target region.

[0249] In one possible approach, the fifth region could be designated as the third target region, and the sixth region as the fourth target region.

[0250] In one possible approach, the sixth region could be designated as the third target region, and the fifth region as the fourth target region.

[0251] Next, based on the spatial distribution of the third target region in each preset segmentation map, the spatial distribution type is classified, resulting in multiple preset spatial distribution types. For example, the spatial distribution type can be classified based on the corners covered by the third target region in the corresponding preset segmentation map.

[0252] Figure 3bThis is a schematic diagram illustrating a preset spatial distribution type as an example. Figure 3b The document shows 16 preset spatial distribution types.

[0253] Reference Figure 3b For example, i is a preset spatial distribution type identifier; i=1 indicates the first preset spatial distribution type, i=2 indicates the second preset spatial distribution type, ..., i=16 indicates the 16th preset spatial distribution type.

[0254] Reference Figure 3b The spatial distribution of the third target area is shown in the position of the small black square.

[0255] For example, the first preset spatial distribution type (i.e., i=1) indicates that the third target region is the region covering the upper left corner of the preset image block; the second preset spatial distribution type (i.e., i=2) indicates that the third target region is the region covering the upper right corner of the preset image block; the third preset spatial distribution type (i.e., i=3) indicates that the third target region is the region covering the lower left corner of the preset image block; and the fourth preset spatial distribution type (i.e., i=4) indicates that the third target region is the region covering the lower right corner of the preset image block.

[0256] For example, the fifth preset spatial distribution type (i.e., i=5) indicates that the third target region is the region in the preset image block that covers the upper left, upper right, and lower left corners; the sixth preset spatial distribution type (i.e., i=6) indicates that the third target region is the region in the preset image block that covers the upper left, upper right, and lower right corners; the seventh preset spatial distribution type (i.e., i=7) indicates that the third target region is the region in the preset image block that covers the upper left, lower right, and lower left corners; and the eighth preset spatial distribution type (i.e., i=8) indicates that the third target region is the region in the preset image block that covers the upper right, lower right, and lower left corners.

[0257] For example, the 9th preset spatial distribution type (i.e., i=9) indicates that the third target region is the region covering the upper left and lower left corners of the preset image block; the 10th preset spatial distribution type (i.e., i=10) indicates that the third target region is the region covering the upper right and lower right corners of the preset image block; the 11th preset spatial distribution type (i.e., i=11) indicates that the third target region is the region covering the lower right and lower left corners of the preset image block; and the 12th preset spatial distribution type (i.e., i=12) indicates that the third target region is the region covering the upper right and upper left corners of the preset image block.

[0258] For example, the 13th preset spatial distribution type (i.e., i=13) indicates that the third target region is the region in the preset image block that covers the upper left and lower right corners; the 13th preset spatial distribution type (i.e., i=14) indicates that the third target region is the region in the preset image block that covers the upper right and lower left corners; the 15th preset spatial distribution type (i.e., i=15) indicates that the third target region is the region in the preset image block that covers the lower right, lower left, upper right, and upper left corners; and the 16th preset spatial distribution type (i.e., i=16) indicates that the third target region is the region in the preset image block that does not cover any corner.

[0259] Then, candidate lists of preset motion vectors corresponding to various preset spatial distribution types can be established, resulting in multiple candidate lists of preset motion vectors, as described in S303:

[0260] S303. For the target spatial distribution type among multiple preset spatial distribution types, based on the positional relationship between the seventh region in the target preset image block corresponding to the target spatial distribution type and multiple adjacent blocks of the target preset image block, determine the preset motion vector candidate list corresponding to the target spatial distribution type, and the seventh region corresponds to the third target region.

[0261] S304. Establish a preset candidate list set by using preset motion vector candidate lists corresponding to various preset spatial distribution types.

[0262] For example, one preset spatial distribution type can be selected sequentially from multiple preset spatial distribution types as the target spatial distribution type. Next, a preset image block (hereinafter referred to as the target preset image block) corresponding to the target spatial distribution type can be determined. The target spatial distribution type may correspond to multiple preset image blocks, and one preset image block can be selected from these multiple preset image blocks as the target preset image block. Then, based on the positional relationship between the seventh region in the target preset image block, which corresponds to the third target region, and the adjacent blocks of the target preset image block, a preset motion vector candidate list corresponding to the target spatial distribution type can be determined.

[0263] In one possible approach, the distances between the seventh region in the target preset image block and multiple adjacent blocks of the target preset image block can be determined separately. Then, based on the distances between the seventh region in the target preset image block and multiple adjacent blocks of the target preset image block, the arrangement order of the multiple adjacent blocks of the target preset image block is determined. Subsequently, according to the arrangement order, the neighboring block identifiers corresponding to the multiple adjacent blocks of the target preset image block are filled into the preset motion vector candidate list corresponding to the target spatial distribution type.

[0264] For example, the distance between the seventh region in the target preset image block and multiple adjacent blocks of the target preset image block can be determined based on the Merge candidate locations (including spatial and temporal locations).

[0265] Figure 3c This is an example illustration of a candidate Merge location.

[0266] Reference Figure 3c For example, "Current CU" means the current block (in Figure 3a In the embodiments, B2, B1, B0, A1, A0, and T are neighbor block identifiers, which are used to represent the adjacent blocks at a certain position of the current block. Among them, B2 represents the nearest neighbor block to the upper left of the current block, B1 represents the rightmost neighbor block above the current block, B0 represents the nearest neighbor block to the upper left of the current block, B1 represents the neighbor block with the smallest left side of the current block, A0 represents the nearest neighbor block to the lower left of the current block, and T0 is the block at the corresponding position of the current block in the reconstructed frame.

[0267] Reference Figure 3b When i = 1, the third target region is the region covering the upper left corner of the preset segmentation map; correspondingly, the seventh region is the region covering the upper left corner of the target preset map. At this time, refer to... Figure 3c The distances of the seventh region to B2, B1, B0, A1, A0 and T, from closest to farthest, are: B2, B1, A1, B0, A0 and T.

[0268] Reference Figure 3b When i = 2, the third target region is located at the upper right corner of the preset image block, and correspondingly, the seventh region in the target preset image block is also located at the upper right corner of the target preset image. At this time, refer to... Figure 3c The distances of the seventh region to B2, B1, B0, A1, A0, and T, from closest to farthest, are: B0, B2, T, B1, A0, and A1. It should be understood that for... Figure 3b When i equals other values, it can also be combined with Figure 3c Similarly, the relationships between the distances of the seventh region and B2, B1, B0, A1, A0 and T are determined, which will not be elaborated here.

[0269] In one possible approach, among multiple neighboring blocks of the target preset image block, the neighboring block that is closer to the seventh region in the target preset image block is ranked higher. That is, in the preset motion vector candidate list corresponding to the target spatial distribution type, the distance between the neighboring block corresponding to the k-th neighboring block identifier and the seventh region in the target preset image block is less than the distance between the neighboring block corresponding to the (k+1)-th neighboring block identifier and the seventh region in the target preset image block; k is a positive integer.

[0270] Assuming that the neighboring block identifiers of the four nearest adjacent blocks to the seventh region are used, a preset motion vector candidate list is generated; then, multiple preset motion vector candidate lists are used to form a preset candidate list set, as shown in Table 1 below:

[0271] Table 1

[0272] Type(i) 1 / 5 2 / 6 3 / 7 4 / 8 9 10 11 12 13 14 15 16 cand[0] B2 B0 A0 T A1 B0 A0 B1 B2 B1 0 B1 cand[1] B1 B2 B2 B0 B2 T T B2 T A1 B1 A1 cand[2] A1 T T A0 A0 B1 A1 B0 B1 B0 A1 B0 cand[3] B0 B1 A1 B1 0 0 0 0 A1 A0 B0 A0

[0273] In Table 1, one column represents a list of preset motion vector candidates.

[0274] For example, column 1 is Figure 3b The first column contains the preset candidate lists of motion vectors corresponding to the first spatial distribution type (i=1) and the fifth spatial distribution type (i=5). The second column contains... Figure 3b The preset motion vector candidate list corresponding to the second spatial distribution type (i=2) and the sixth spatial distribution type (i=6). Column 3 is... Figure 3b The preset motion vector candidate lists corresponding to the third spatial distribution type (i=3) and the seventh spatial distribution type (i=7). Column 4 is... Figure 3b The preset motion vector candidate lists corresponding to the 4th spatial distribution type (i=4) and the 8th spatial distribution type (i=8). Column 5 is... Figure 3b The list of preset motion vector candidates corresponding to the 9th spatial distribution type (i=9). Column 6 is... Figure 3b The list of preset motion vector candidates corresponding to the 10th spatial distribution type (i=10). Column 7 is... Figure 3b The preset motion vector candidate list corresponding to the 11th spatial distribution type (i=11). Column 8 is... Figure 3b The preset motion vector candidate list corresponding to the 12th spatial distribution type (i=12). Column 9 is... Figure 3b The preset candidate list of motion vectors corresponding to the 13th spatial distribution type (i=13). Column 10 is... Figure 3b The preset motion vector candidate list corresponding to the 14th spatial distribution type (i=14). Column 11 is... Figure 3b The preset candidate list of motion vectors corresponding to the 15th spatial distribution type (i=15). Column 12 is... Figure 3bThe preset motion vector candidate list corresponding to the 16th spatial distribution type (i=16).

[0275] In Table 1, “0” indicates that there is no neighboring block identifier; “cand[0]”, “cand[1]”, “cand[2]” and “cand[3]” refer to elements in the preset motion vector list.

[0276] The following is based on Figure 3a The process of establishing a motion vector candidate list is described in the example of the preset candidate list set.

[0277] Figure 4a This is a schematic diagram illustrating an example of the inter-frame prediction process. Wherein, Figure 4a The inter-frame prediction process described in the embodiments can be an inter-frame prediction process during the encoding process or an inter-frame prediction process during the decoding process; this application does not limit this. Figure 4a In this embodiment, the target segmentation map is the first segmentation map.

[0278] S401, obtain the first motion vector corresponding to the current block.

[0279] For example, when the current block is the block to be encoded, it can be based on Figure 3c The Merge candidate positions are used to establish a motion vector candidate list (hereinafter referred to as the first motion vector candidate list, which is used to determine the first motion vector); then the first motion vector can be selected from the first motion vector candidate list.

[0280] For example, refer to Figure 3c The third motion vector corresponding to B0 can be perturbed to generate M1 (M1 is a positive integer) candidate motion vectors; the third motion vector corresponding to B1 can be perturbed to generate M2 (M2 is a positive integer) candidate motion vectors; the third motion vector corresponding to B2 can be perturbed to generate M3 (M3 is a positive integer) candidate motion vectors; the third motion vector corresponding to A1 can be perturbed to generate M4 (M4 is a positive integer) candidate motion vectors; the third motion vector corresponding to A0 can be perturbed to generate M5 (M2 is a positive integer) candidate motion vectors; and the third motion vector corresponding to T can be perturbed to generate M6 (M6 is a positive integer) candidate motion vectors.

[0281] Next, the first motion vector candidate list can be constructed using M1 candidate motion vectors, M2 candidate motion vectors, M3 candidate motion vectors, M4 candidate motion vectors, M5 candidate motion vectors, M6 candidate motion vectors, the third motion vector corresponding to B0, the third motion vector corresponding to B1, the third motion vector corresponding to B2, the third motion vector corresponding to A1, the third motion vector corresponding to A0, and the third motion vector corresponding to T.

[0282] In one possible approach, M1+M2+M3+M4+M5+M6=64, so the length of the first motion vector candidate list can be 70, meaning that the first motion vector candidate list can include 70 motion vectors.

[0283] For example, the optimal motion vector can be selected from the first motion vector candidate list as the first motion vector. The process of selecting the optimal motion vector from the first motion vector candidate list can be as follows: For each motion vector in the first motion vector list, a first test reference block corresponding to the current block can be determined based on the motion vector and the pixel coordinates corresponding to the current block; thus, multiple first test reference blocks can be obtained. Then, the difference information between the current block and each first test reference block is calculated, and the bitstream size corresponding to each motion vector is determined; subsequently, based on the difference information between the current block and each first test reference block, and the bitstream size corresponding to each motion vector, the optimal motion vector is selected (wherein, the smaller the bitstream corresponding to the motion vector, and the smaller the difference information between the corresponding first test reference block and the current block, the better the motion vector).

[0284] S402, based on the first motion vector, obtain the first segmentation map of the first reference block corresponding to the current block. The first segmentation map is used to describe the partitioning of the current block and includes a first region and a second region.

[0285] One possible approach is to use an AI image segmentation scheme to segment the reconstructed frames.

[0286] For example, this application employs different AI image segmentation schemes for reconstructed frames of different qualities. For instance, for high-quality reconstructed frames, an instance segmentation network is used for segmentation; for low-quality reconstructed frames, a video object segmentation network is used. The following example illustrates this using a GOP (Group of Pictures) (comprising N (N is a positive integer) reconstructed frames, with the first reconstructed frame being a high-quality frame and the second to Nth reconstructed frames being low-quality frames).

[0287] Figure 4bThis is a schematic diagram illustrating an image segmentation process as an example.

[0288] For example, firstly, the first reconstructed frame in the GOP can be input into an instance segmentation network, which segments the first reconstructed frame and outputs instance segmentation information for the first reconstructed frame (e.g., the instance segmentation information may include a mask image, where different instances (i.e., objects) correspond to different colors in the mask image of the first reconstructed frame). Figure 4b (Different instances shown in the image correspond to different grayscale values). Next, the instance segmentation information of the first reconstructed frame, as well as the first to Nth reconstructed frames, can be input into the video object segmentation network. The video object segmentation network uses the instance segmentation information of the first reconstructed frame as an inference reference (i.e., the label) for instance tracking, extracting instances from each of the first to Nth reconstructed frames, and outputting a mask image for each of the first to Nth reconstructed frames. Subsequently, the mask images of the first to Nth reconstructed frames can be binarized to obtain the frame segmentation image (also a binary image) corresponding to each of the first to Nth reconstructed frames.

[0289] For example, the mask image of each reconstructed frame from the 1st to the Nth reconstructed frame can be binarized by referring to the following formula (1):

[0290]

[0291] Among them, M i (x,y) represents the pixel value at position (x,y) in the binary image (frame segmentation image), S i (x,y) represents the pixel value of the pixel at (x,y) in the mask image, Φ I Represents an instance set.

[0292] Referring to formula (1), the pixel values ​​of the corresponding pixels of the instance in the mask image (that is, the pixels contained in the foreground area) are set to 1, and the pixel values ​​of the pixels in the mask image other than the corresponding pixels of the instance (that is, the pixels contained in the background area) are set to 0, thus obtaining a binary image.

[0293] It should be understood that the pixel values ​​of the corresponding pixels of the instance in the mask image (i.e., the pixels contained in the foreground region) can also be set to 0, and the pixel values ​​of all pixels in the mask image except the corresponding pixels of the instance (i.e., the pixels contained in the background region) can be set to 1, thus obtaining a binary image. This application does not limit this.

[0294] Video object segmentation network is a hybrid propagation and detection method based on spatiotemporal attention mechanism. It relies on the complex dynamic spatiotemporal dependencies of instances in multiple frames to capture motion fields and achieve accurate segmentation of instances in each frame. Correspondingly, the first segmentation map of the first reference block is extracted from the frame segmentation map of the reference frame of the current block, and thus the motion field of the first reference block can also be accurately segmented. The segmentation of the first reference block can be transferred to the segmentation of the current block, thereby adopting an AI image instance segmentation scheme, which can improve the accuracy of segmenting different motion fields of the current block.

[0295] One possible approach is to use a non-AI image instance segmentation scheme for segmentation.

[0296] For example, instance segmentation can be performed based on edge detection. This involves calculating the gradient of the reference frame for the current block, and then performing thresholding based on the gradient to obtain a binarized image. The binarized image is then subjected to morphological clustering to separate the foreground and background regions of the reference frame, thus obtaining a frame segmentation map of the reference frame.

[0297] It should be understood that this application does not limit the image segmentation method for reconstructing frames.

[0298] Figure 4c This is a schematic diagram illustrating the process of determining the first segmentation map. Wherein, Figure 4c The size of the first segmentation image is the same as the size of the current block. Figure 4c The first segment image has been enlarged for easier viewing.

[0299] Reference Figure 4c For example, after determining the first motion vector, the reference frame corresponding to the current block, and the frame segmentation map of the reference frame, the first reference block can be determined from the reference frame of the current block based on the pixel coordinates of the current block and the first motion vector. Then, based on the pixel coordinates of the first reference block, the block segmentation map (i.e., the first segmentation map) of the first reference block is extracted from the frame segmentation map of the reference frame.

[0300] S403, determine the first target area from the first area and the second area.

[0301] In one possible approach, the region with the smallest area in the first region and the second region can be designated as the first target region; and the region with the largest area in the first region and the second region can be designated as the second target region.

[0302] In one possible approach, the region with the largest area in the first region and the region with the smallest area in the first region and the region with the smallest area in the second region can be defined as the first target region; and the region with the smallest area in the first region and the region with the smallest area can be defined as the second target region.

[0303] In one possible approach, the region corresponding to the foreground region of the current block in the first region and the second region can be determined as the first target region; and the region corresponding to the background region of the current block in the first region and the second region can be determined as the second target region.

[0304] In one possible approach, the region corresponding to the background region of the current block in the first region and the second region can be determined as the first target region; and the region corresponding to the foreground region of the current block in the first region and the second region can be determined as the second target region.

[0305] In one possible approach, the region in which the pixel values ​​of all pixels in the first region and the second region are both 1 can be defined as the first target region; and the region in which the pixel values ​​of all pixels in the first region and the second region are both 0 can be defined as the second target region.

[0306] In one possible approach, the region in which the pixel values ​​of all pixels contained in the first region and the second region are both 0 can be defined as the first target region; and the region in which the pixel values ​​of all pixels contained in the first region and the second region are both 1 can be defined as the second target region.

[0307] It should be understood that other methods may also be used to determine the first target area and the second target area from the first area and the second area, and this application does not limit this.

[0308] For example, the process of establishing a candidate list of motion vectors based on the spatial distribution of the first target region in the first segmentation map can be referred to as follows: S404 to S406:

[0309] S404, based on the spatial distribution of the first target region in the first segmentation map, determine the spatial distribution type corresponding to the first target region from a variety of preset spatial distribution types.

[0310] For example, it can be seen from Figure 3b Among the 16 preset spatial distribution types, the spatial distribution type corresponding to the first target region is determined based on the spatial distribution of the first target region in the first segmentation map.

[0311] For example, suppose that Figure 2b If the first region in the diagram is determined as the first target region, then the spatial distribution type corresponding to the first target region is: Figure 3b The fifth preset spatial distribution type (i=5).

[0312] For example, suppose we would Figure 2b If the second region in the diagram is determined to be the first target region, then the spatial distribution type corresponding to the first target region is: Figure 3b The fourth preset spatial distribution type (i=4).

[0313] S405, based on the spatial distribution type corresponding to the first target area, determine the motion vector candidate list from the preset candidate list set. The preset candidate list set includes multiple preset motion vector candidate lists corresponding to various preset spatial distribution types.

[0314] For example, Figure 3a In the embodiments, the preset candidate list set is determined, including multiple preset motion vector candidate lists corresponding to multiple preset spatial distribution types; a motion vector candidate list (hereinafter referred to as the second motion vector candidate list, which is used to determine the second motion vector) can be determined from the preset candidate list set according to the spatial distribution type corresponding to the first target area.

[0315] For example, the spatial distribution type corresponding to the first target area is: Figure 3b The fifth preset spatial distribution type (i=5) is then used to determine the candidate list of the second motion vectors from Table 1 above, as shown in Table 2:

[0316] Table 2

[0317]

[0318] For example, the spatial distribution type corresponding to the first target area is: Figure 3b The fourth preset spatial distribution type (i=4) can be used to determine the candidate list of the second motion vectors from Table 1 above, as shown in Table 3:

[0319] Table 3

[0320]

[0321] S406, according to the multiple neighboring block identifiers contained in the motion vector candidate list, fill the motion vector candidate list with the third motion vectors corresponding to multiple neighboring blocks of the current block.

[0322] For example, the second motion vector candidate list contains multiple neighbor block identifiers. The third motion vectors corresponding to multiple neighbor blocks of the current block can be filled into the second motion vector candidate list according to the multiple neighbor block identifiers contained in the motion vector candidate list.

[0323] S407, Based on the candidate list of motion vectors, determine the second motion vector corresponding to the current block.

[0324] For example, the motion vector candidate list includes multiple third motion vectors corresponding to multiple adjacent blocks of the current block; the distance between the adjacent block corresponding to the i-th third motion vector in the motion vector candidate list and the third region is less than the distance between the adjacent block corresponding to the (i+1)-th third motion vector and the third region; i is a positive integer.

[0325] In one possible approach, the third motion vector ranked first in the candidate list of the second motion vectors, that is, the first third motion vector, can be used as the second motion vector (that is, the second motion vector corresponding to the third region in the current block, and the first motion vector is the motion vector corresponding to the fourth region in the current block).

[0326] Since the second motion vector of the third region in the current block is more similar to the third motion vector of the adjacent block that is closer to the third region, the accuracy of the determined second motion vector can be improved, and thus the inter-frame prediction can be more accurate, thereby improving the coding quality.

[0327] Furthermore, the earlier the third motion vector appears in the candidate list of the second motion vector, the fewer bytes are required for encoding, thus making the encoding of the second motion vector more efficient and saving bitstream.

[0328] In one possible approach, the optimal third motion vector can be selected from the candidate list of second motion vectors as the second motion vector. Since the third motion vectors of adjacent blocks closer to the third region are ranked higher in the motion vector candidate list, and the second motion vector of the third region in the current block is more similar to the third motion vectors of adjacent blocks closer to the third region, even if the optimal third motion vector is not the first in the second motion vector list, this application can still improve the efficiency of encoding the second motion vector and save bitstream compared to existing technologies that generate motion vector lists in a traditional fixed order.

[0329] In one possible approach, the process of selecting the optimal third motion vector from the candidate list of motion vectors (i.e., the second motion vector list) as the second motion vector can be as follows: Assume the second motion vector candidate list includes four third motion vectors: third motion vector 1, third motion vector 2, third motion vector 3, and third motion vector 4. Based on third motion vector 1 and the pixel coordinates of the current block, extract second test reference block 1 from the reference frame of the current block; based on third motion vector 2 and the pixel coordinates of the current block, extract second test reference block 2 from the reference frame of the current block; based on third motion vector 3 and the pixel coordinates of the current block, extract second test reference block 3 from the reference frame of the current block; based on third motion vector 4 and the pixel coordinates of the current block, extract second test reference block 4 from the reference frame of the current block. Next, region A corresponding to the second target region is extracted from the first reference block according to the first segmentation map; and region B1 corresponding to the first target region is extracted from the second test reference block 1 according to the first segmentation map, region B2 corresponding to the first target region is extracted from the second test reference block 2 according to the first segmentation map, region B3 corresponding to the first target region is extracted from the second test reference block 3 according to the first segmentation map, and region B4 corresponding to the first target region is extracted from the second test reference block 4 according to the first segmentation map. Subsequently, regions A and B1 are concatenated to obtain region C1; regions A and B2 are concatenated to obtain region C2; regions A and B3 are concatenated to obtain region C3; and regions A and B4 are concatenated to obtain region C4. Then, the difference information between the current block and regions C1, C2, C3, and C4 is calculated respectively; the bitstream size corresponding to the encoded third motion vector 1, third motion vector 2, third motion vector 3, and third motion vector 4 can also be determined. Then, based on the difference information between the current block and regions C1, C2, C3 and C4 respectively, and the bitstream size corresponding to the third motion vector 1, third motion vector 2, third motion vector 3 and third motion vector 4, the optimal third motion vector is selected from the third motion vector 1, third motion vector 2, third motion vector 3 and third motion vector 4 as the second motion vector.

[0330] S408, based on the integer part of the first motion vector, obtain the first reference block.

[0331] For example, the first motion vector includes an integer part and a fractional part.

[0332] It should be noted that the first reference block mentioned above is determined based on the integer part of the first motion vector and the pixel coordinates of the current block; that is, after determining the first reference block as described above, this step can directly obtain the first reference block.

[0333] S409, based on the integer part of the second motion vector, obtain the second reference block.

[0334] For example, the second motion vector includes an integer part and a fractional part. A second reference block can be determined from the reference frame of the current block based on the integer part of the second motion vector and the pixel coordinates of the current block.

[0335] For example, the process of performing motion compensation based on the first segmentation map, the first motion vector, and the second motion vector to obtain the predicted block corresponding to the current block can be as follows: S410~S411:

[0336] S410, perform integer-pixel motion compensation based on the first segmentation map, the first reference block, and the second reference block to determine an integer-pixel prediction block, wherein the first reference block is determined based on the integer part of the current block and the first motion vector, and the second reference block is determined based on the integer part of the current block and the second motion vector.

[0337] For example, a second reference block can also be determined based on the integer part of the second motion vector and the pixel coordinates of the current block.

[0338] For example, a first reference block and a second reference block can be linearly fused based on a first segmentation map to determine an integer pixel prediction block.

[0339] For example, the pixel category corresponding to each pixel in the first reference block and the pixel category corresponding to each pixel in the second reference block can be determined based on the distance between each pixel in the first segmentation image and the segmentation line; wherein, the segmentation line is the segmentation line between the first region and the second region in the first segmentation image.

[0340] In one possible approach, the pixel category corresponding to each pixel in the first segmentation image can be determined based on the distance between each pixel in the first segmentation image and the segmentation line; then, based on the pixel category corresponding to each pixel in the first segmentation image, the pixel category corresponding to each pixel in the first reference block and the pixel category corresponding to each pixel in the second reference block can be determined.

[0341] In one possible approach, edge detection operators based on first-order derivatives can be used to classify the pixels of the first segmentation image pixel by pixel to determine the pixel category corresponding to each pixel in the first segmentation image.

[0342] For example, the Prewitt operator can be introduced to calculate the gradient G0(i,j) between the current pixel and its neighboring pixels, as shown in the following formula (2):

[0343] G0(i,j)=max(S0⊙P0,S0⊙P0 T (2)

[0344] For example, S0 is a 3×3 matrix composed of the pixel value of the pixel at (i,j) in the first segmentation image and the pixel values ​​of its neighboring pixels. S0 can be represented by the following formula (3):

[0345]

[0346] Among them, M i,j M represents the pixel value at position (i,j). i-1,j-1 M represents the pixel value at position (i-1, j-1). i,j-1 M represents the pixel value at position (i,j-1). i+1,j-1 This represents the pixel value at (i+1, j-1), and so on.

[0347] For example, P0 represents the Prewitt operator, which can be represented as shown in equation (4):

[0348]

[0349] For example, based on the principle of the Prewitt operator, P0 can be extended into a 5×5 operator P1, as shown in formula (5):

[0350]

[0351] Next, operator P1 is used to calculate the gradient G1(i,j) between the current pixel and the pixel 1 pixel away from the current pixel, as shown in formula (6):

[0352] G1(i,j)=max(S1⊙P1,S1⊙P1 T (6)

[0353] Wherein, S1 is a 5×5 matrix, which consists of the pixel value of the pixel at (i,j) in the first segmentation image, the pixel value of the adjacent pixel, and the pixel value of the pixel at a distance of 1 pixel from it, as shown in formula (7):

[0354]

[0355] Among them, M i-2,j-2 M represents the pixel value at position (i-2, j-2). i+2,j+2 M represents the pixel value at position (i+2, j+2). i-2,j+2 M represents the pixel value at position (i-2, j+2). i+2,j+2 This represents the pixel value at the pixel located at (i+2, j+2), and so on.

[0356] For example, the pixel classification method is shown in formula (8):

[0357]

[0358] Referring to formula (8), when the gradient between the pixel at (i,j) and its neighboring pixels is greater than 0, the pixel at (i,j) can be determined as a first-class pixel (that is, a pixel 1 pixel away from the dividing line); when the gradient between the pixel at (i,j) and its neighboring pixels is equal to 0, and the gradient between the pixel at (i,j) and the pixel 1 pixel away is greater than 0, the pixel at (i,j) can be determined as a second-class pixel (that is, a pixel 2 pixels away from the dividing line); in other cases, the pixel at (i,j) is a third-class pixel (that is, a pixel more than 2 pixels away from the dividing line).

[0359] Furthermore, based on the calculation results of G0 and G1 above, the classification C of the pixel at (i,j) in the first segmentation image can be derived. i,j .

[0360] In one possible approach, the pixel category corresponding to each pixel in the first segmentation image can be used as the pixel category of the corresponding pixel in the first reference block, and can also be used as the pixel category of the corresponding pixel in the second reference block.

[0361] For example, for different pixel categories, corresponding weight information can be preset as shown in formula (9):

[0362]

[0363] Referring to formula (9), if the pixel at (i,j) is a pixel of type 1, then the two weights corresponding to the pixel at (i,j) are: If the pixel at (i,j) is a second-class pixel, then the two weights corresponding to the pixel at (i,j) are: If the pixel at (i,j) is a pixel of class 0, then the two weights corresponding to the pixel at (i,j) are: ω0=1, ω1=0.

[0364] It should be understood that ω0 and ω1 corresponding to different pixel categories can be set as needed, and this application does not impose any restrictions on this.

[0365] For example, a first weight corresponding to each pixel in the first reference block can be determined based on the pixel category corresponding to each pixel in the first segmentation map and the first reference block; and a second weight corresponding to each pixel in the second reference block can be determined based on the pixel category corresponding to each pixel in the first segmentation map and the second reference block.

[0366] For example, for a pixel at (i,j) in the first reference block, the first weight corresponding to the pixel at (i,j) in the first reference block can be determined as ω0 or ω1 based on the pixel value of the pixel at (i,j) in the first segmentation image, the pixel category of the pixel at (i,j) and the weight information in the above formula (9).

[0367] For example, for a pixel at (i,j) in the second reference block, the second weight corresponding to the pixel at (i,j) in the second reference block can be determined as ω0 or ω1 based on the pixel value of the pixel at (i,j) in the first segmentation image, the pixel category of the pixel at (i,j) and the weight information in the above formula (9).

[0368] For example, suppose the pixel value of the pixel corresponding to the first region in the first segmentation image is 0, and the pixel value of the pixel corresponding to the second region is 1; if the first target region is the first region, then when the pixel value of the pixel at (i,j) in the first segmentation image is 1, ω0 can be used as the first weight corresponding to the pixel at (i,j) in the first reference block, and ω1 can be used as the second weight corresponding to the pixel at (i,j) in the second reference block. When the pixel value of the pixel at (i,j) in the first segmentation image is 0, ω1 can be used as the first weight corresponding to the pixel at (i,j) in the first reference block, and ω0 can be used as the second weight corresponding to the pixel at (i,j) in the second reference block.

[0369] Accordingly, based on the first weight and the second weight, the pixel values ​​of the corresponding pixels in the first reference block and the second reference block are linearly weighted to obtain the integer pixel prediction block, as shown in the following formula (10):

[0370]

[0371] Where P3(i,j) is the pixel value of the pixel at the integer pixel prediction block (i,j), P1(i,j) is the pixel value of the pixel at the first reference block (i,j), and P2(i,j) is the pixel value of the pixel at the second reference block (i,j).

[0372] For example, suppose the pixel value of the pixel corresponding to the second region in the first segmentation image is 0, and the pixel value of the pixel corresponding to the first region is 1; if the first target region is the first region, then when the pixel value of the pixel at (i,j) in the first segmentation image is 0, ω0 can be used as the first weight corresponding to the pixel at (i,j) in the first reference block, and ω1 can be used as the second weight corresponding to the pixel at (i,j) in the second reference block. When the pixel value of the pixel at (i,j) in the first segmentation image is 1, ω1 can be used as the first weight corresponding to the pixel at (i,j) in the first reference block, and ω0 can be used as the second weight corresponding to the pixel at (i,j) in the second reference block.

[0373] Accordingly, based on the first weight and the second weight, the pixel values ​​of the corresponding pixels in the first reference block and the second reference block are linearly weighted to obtain the integer pixel prediction block, as shown in the following formula (11):

[0374]

[0375] Where P3(i,j) is the pixel value of the pixel at the integer pixel prediction block (i,j), P1(i,j) is the pixel value of the pixel at the first reference block (i,j), and P2(i,j) is the pixel value of the pixel at the second reference block (i,j).

[0376] Figure 4d This is a schematic diagram illustrating a linear fusion process as an example. Figure 4d The description describes the linear fusion process between the first and second reference blocks when pixels in the first and second reference blocks are divided into three pixel categories; for details, please refer to the above. Figure 4a The description will not be repeated here.

[0377] in, Figure 4d (1) is the first segmentation image; and the pixel value of the first region in the first segmentation image is 0, the pixel value of the second region is 1, and the first target region is the second region.

[0378] Figure 4d (2) For the first reference block and Figure 4d (1) Local region A1 corresponding to local region 1 in the first segmentation diagram, and in the second reference block, with Figure 4d (1) An example diagram of linear fusion of local region A2 corresponding to local region 1 in the first segmentation diagram.

[0379] For example, Figure 4d(2) For pixels of pixel category 0 (i.e., pixels more than 2 pixels away from the dividing line), if the pixel value of a pixel of pixel category 0 in the first segmentation image is 1, then the first weight corresponding to the white pixel (i.e., the pixel with a pixel value of 1) in local region A1 of the first reference block is ω1 = 0, and the second weight corresponding to the white pixel in local region A1 of the second reference block is ω0 = 1. Therefore, the pixel of pixel category 0 obtained by fusing the first and second reference blocks is... Figure 4d (2) White pixels.

[0380] For example, Figure 4d (2) For pixels of pixel category 0 (i.e., pixels more than 2 pixels away from the dividing line), if the pixel value of pixels of pixel category 0 in the first segmentation image is 0, then the first weight corresponding to the black pixel in local region A1 of the first reference block is ω0 = 1, and the second weight corresponding to the white pixel in local region A2 of the second reference block is ω1 = 0. Therefore, the pixel of pixel category 0 obtained by fusing the first and second reference blocks is... Figure 4d (2) Black pixels.

[0381] For example, Figure 4d (2) For pixels belonging to the second pixel category (i.e., pixels 2 pixels away from the dividing line), if the pixel value of the second pixel category in the first segmentation image is 1, then the first weight corresponding to the white pixel in local region A1 of the first reference block is ω1 = 1 / 4, and the second weight corresponding to the white pixel in local region A2 of the second reference block is ω0 = 3 / 4. Therefore, the pixel value of the second pixel category fused from the first and second reference blocks is... Figure 4d (2) Light gray pixels (gray pixels close to white pixels).

[0382] For example, Figure 4d (2) For pixels belonging to the second pixel category (i.e., pixels 2 pixels away from the dividing line), if the pixel value of the second pixel category in the first segmentation image is 0, then the first weight corresponding to the pixel in local region A1 of the first reference block that corresponds to the black pixel in local region 1 is ω0 = 3 / 4, and the second weight corresponding to the pixel in local region A2 of the second reference block that corresponds to the black pixel in local region 1 is ω1 = 1 / 4. Therefore, the pixel values ​​of the second pixel category in the first and second reference blocks are fused to obtain... Figure 4d (2) Medium-dark gray pixels (gray pixels that are close to black pixels).

[0383] For example, Figure 4d (2) For pixels belonging to the first pixel category (i.e., pixels one pixel away from the dividing line), if the pixel value of the first pixel category in the first segmentation image is 1, then the first weight corresponding to the white pixel in local region A1 of the first reference block is ω1 = 1 / 2, and the second weight corresponding to the white pixel in local region A2 of the second reference block is ω0 = 1 / 2. Therefore, the pixel values ​​of the second pixel category in the first and second reference blocks are fused to obtain... Figure 4d (2) Medium gray pixels (pixels that are close to light gray pixels and between light gray pixels, and close to light gray pixels).

[0384] For example, Figure 4d (2) For pixels belonging to the first pixel category (i.e., pixels one pixel away from the dividing line), if the pixel value of the first pixel category in the first segmentation image is 0, then the first weight corresponding to the pixel in local region A1 of the first reference block that corresponds to the black pixel in local region 1 is ω0 = 1 / 2, and the second weight corresponding to the pixel in local region A2 of the second reference block that corresponds to the black pixel in local region 1 is ω1 = 1 / 2. Therefore, the pixel of the second pixel category obtained by fusing the first and second reference blocks is... Figure 4d (2) Medium gray pixels (pixels that are close to light gray pixels and between light gray pixels, and close to dark gray pixels).

[0385] Figure 4d (3) is in the first reference block and Figure 4d (1) Local region B1 corresponding to local region 2 in the first segmentation image, and in the second reference block with Figure 4d (1) An example diagram of linear fusion of local region B2 corresponding to local region 2 in the first segmentation image. For the specific fusion process, please refer to... Figure 4d The description of (2) will not be repeated here.

[0386] It should be understood that the above only divides pixels into 3 pixel categories. Pixels can be divided into more pixel categories, that is, it is necessary to calculate the gradient between the current pixel and a pixel further away to divide the pixel into more pixel categories. Correspondingly, the preset classification weight information can also set corresponding weights for more pixel categories. The specific settings can be made according to requirements, and this application does not impose any restrictions on this. When pixels are divided into more pixel categories, the method of weighting the pixels at corresponding positions of the first reference block and the second reference block based on the first weight and the second weight to obtain the integer pixel prediction block can be referred to as the following formula (12):

[0387] P3(i,j)=(W0P1(i,j)+W1P2(i,j)+B)>>3 (12)

[0388] Wherein, W0 is the first weight matrix, including multiple first weights determined according to preset classification weight information; W1 is the second weight matrix, including multiple second weights determined according to preset classification weight information. B is a constant, which can be set according to requirements, and this application does not impose any restrictions on it.

[0389] For example, W0 + W1 = L, where L is the difference between the length of the maximum Prewitt operator and 1.

[0390] For example, suppose the pixels are divided into 5 categories: category 0 (pixels more than 4 pixels away from the dividing line), category 1 (pixels 1 pixel away from the dividing line), category 2 (pixels 2 pixels away from the dividing line), category 3 (pixels 3 pixels away from the dividing line), and category 4 (pixels 4 pixels away from the dividing line). In this case, W0 + W1 = 8, and B = 4.

[0391] Figure 4e This is a schematic diagram illustrating a linear fusion process as an example. Figure 4e The description describes the linear fusion process between the first and second reference blocks when pixels in the first and second reference blocks are divided into five pixel categories; for details, please refer to the above. Figure 4d The description will not be repeated here.

[0392] In this way, pixels that are a certain distance from the dividing line are linearly weighted to make the boundary smooth and avoid obvious seams and edge artifacts.

[0393] S411, perform pixel-level motion compensation based on the first segmentation map, the integer pixel prediction block, the fractional part of the first motion vector, and the fractional part of the second motion vector to obtain the prediction block corresponding to the current block.

[0394] Due to the inherent spatial discreteness of digital video, the block division may not happen to be aligned with pixels. Therefore, the integer pixel prediction block determined by the compensation in S410 above may not be able to predict the current block well. Therefore, this application can improve the precision of the prediction block by performing pixel interpolation on the integer pixel prediction block.

[0395] For example, the integer pixel prediction block can be divided into an eighth region and a ninth region based on the dividing line between the first region and the second region of the first segmentation map; then, pixel interpolation is performed on the eighth region based on the fractional part of the first motion vector to obtain the first pixel prediction block, and pixel interpolation is performed on the ninth region based on the fractional part of the second motion vector to obtain the second pixel prediction block; then, the first pixel prediction block and the second pixel prediction block are linearly fused based on the first segmentation map to obtain the prediction block.

[0396] For example, the dividing lines between the first and second regions of the first segmentation image can be migrated to the integer-pixel prediction block to divide the integer-pixel prediction block into an eighth and a ninth region. If the first target region is the second region, then when the eighth region corresponds to the first region in the first segmentation image and the ninth region corresponds to the second region in the first segmentation image, it can be determined that the eighth region corresponds to the first motion vector and the ninth region corresponds to the second motion vector. This application uses the example of the eighth region corresponding to the first motion vector and the ninth region corresponding to the second motion vector for illustration.

[0397] For example, the eighth region can be completed, and then the completed eighth region (with the same size as the whole-pixel prediction block) and the fractional part of the first motion vector can be input into DCTIF (Discrete Cosine Transform Interpolation Filter). DCTIF then performs sub-pixel interpolation on the eighth region based on the fractional part of the first motion vector to obtain the first sub-pixel prediction block. The first sub-pixel prediction block has the same size as the first reference block.

[0398] For example, the ninth region can be completed, and then the completed ninth region (with the same size as the whole-pixel prediction block) and the fractional part of the second motion vector can be input into the DCTIF (Discrete Cosine Transform Interpolation Filter). The DCTIF then performs sub-pixel interpolation on the ninth region based on the fractional part of the second motion vector to obtain the second sub-pixel prediction block. The second sub-pixel prediction block has the same size as the second reference block.

[0399] Next, based on the first segmentation map, the first sub-pixel prediction block and the second sub-pixel prediction block can be linearly fused to obtain a prediction block. This is similar to the linear fusion process of the first reference block and the second reference block described above, and will not be repeated here.

[0400] Figure 4f This is a schematic diagram illustrating the motion compensation process as an example. Figure 4f In this embodiment, the first region in the first segmentation diagram is the first target region, and the second region is the second target region. Region A in the first reference block corresponds to the second region in the first segmentation diagram, and region B in the second reference block corresponds to the first region in the first segmentation diagram. The first reference block is determined based on the integer part of the current block and the first motion vector, and the second reference block is determined based on the integer parts of the current block and the second motion vector.

[0401] Reference Figure 4f First, pixel-level motion compensation can be performed on the first reference block and the second reference block. Specifically, linear fusion can be performed on the first reference block and the second reference block based on the first segmentation map to determine the pixel prediction block. For details, please refer to the above description, which will not be repeated here.

[0402] Reference Figure 4f After obtaining the integer-pixel prediction block, it can be divided into an eighth region and a ninth region based on the first segmentation image. The eighth region corresponds to the second region of the first segmentation image, and the ninth region corresponds to the first region of the first segmentation image. Then, the completed eighth region and the fractional part of the first motion vector are input into DCTIF. After pixel-level interpolation, DCTIF can output the first fractional-pixel prediction block. Similarly, the completed ninth region and the fractional part of the second motion vector can be input into DCTIF. After pixel-level interpolation, DCTIF can output the second fractional-pixel prediction block.

[0403] Reference Figure 4f After obtaining the first sub-pixel prediction block and the second sub-pixel prediction block, the first sub-pixel prediction block and the second sub-pixel prediction block can be linearly fused based on the first segmentation map to determine the prediction block; for details, please refer to the above description, which will not be repeated here.

[0404] For example, one way to obtain the prediction block corresponding to the current block by performing motion compensation based on the first segmentation map, the first motion vector, and the second motion vector can be: performing subpixel interpolation on the first reference block to obtain the sixth reference block; and performing subpixel interpolation on the second reference block to obtain the seventh reference block; and then performing motion compensation on the overlapping area of ​​the sixth and seventh reference blocks based on the first segmentation map to obtain the prediction block; the details can be referred to the description above, and will not be repeated here.

[0405] For example, one way to determine the first motion vector, the first target region, and the second motion vector could be to select a motion vector Xi from a candidate list of first motion vectors, and then determine the first reference block based on motion vector Xi. Assume the list of second motion vectors includes four third motion vectors:

[0406] If the first region is taken as the first target region and the second region as the second target region, then region A corresponding to the second target region can be extracted from the first reference block according to the first segmentation map. Next, following the above method, regions C1, C2, C3, and C4 are determined; then, the difference information between the current block and regions C1, C2, C3, and C4 is calculated respectively; and the bitstream sizes corresponding to the encoded third motion vector 1, third motion vector 2, third motion vector 3, and third motion vector 4 are determined, as well as the bitstream size of the encoded motion vector Xi. In this case, for motion vector Xi, four calculation results can be determined (each calculation result includes difference information, the bitstream size of the encoded third motion vector, and the bitstream size of the encoded motion vector Xi).

[0407] If the second region is taken as the first target region and the first region is taken as the second target region, then four calculation results can also be obtained.

[0408] Assuming the first list of motion vectors includes 70 motion vectors, 560 calculation results can be obtained. Then, the optimal calculation result can be selected from these 560 results; the motion vector Xi corresponding to the optimal calculation result is taken as the first motion vector; the third motion vector corresponding to the optimal calculation result is taken as the second motion vector; and if the first target region corresponding to the optimal calculation result is a first region, then the first region can be taken as the first target region; if the first target region corresponding to the optimal calculation result is a second region, then the second region can be taken as the first target region.

[0409] It should be noted that, Figure 2a Implementation examples and Figure 4a In this embodiment, the first motion vector corresponding to the current block is first determined; then, based on the first segmentation map determined by the first motion vector, the partitioning of the current block (i.e., the partitioning of the third and fourth regions in the current block) is determined; then, according to the first segmentation map, the second motion vector corresponding to the third region in the current block is determined, and the first motion vector is used as the motion vector of the fourth region in the current block. In other words, the second motion vector is determined based on the first motion vector.

[0410] For example, this application proposes a bitstream generation method, which can encode based on a first motion vector corresponding to the current block to obtain a first bitstream. The first motion vector can be based on the above... Figure 2a Implementation examples or Figure 4a The example was generated.

[0411] For example, the region indicator corresponding to the first motion vector can be written into the first bitstream. The region in the current block corresponding to the first motion vector is the aforementioned second target region. For example, when the second target region is the first region in the first segmentation image, and all pixels in the first region have a pixel value of 0, the region indicator can be 0. When all pixels in the first region have a pixel value of 1, the region indicator can be 1. For instance, the parameter `judge_flag` can be written into the first bitstream, and the region indicator can be used as the parameter value of `judge_flag`.

[0412] It should be understood that it is also not necessary to write the region indicator corresponding to the first motion vector in the first bitstream. In this case, the encoding end and the decoding end can agree on the method of determining the first target region in advance.

[0413] For example, encoding based on the first motion vector corresponding to the current block to obtain the first bitstream can be achieved in several ways:

[0414] In one possible approach, the first motion vector can be encoded to obtain a first bitstream. After receiving the first bitstream, the decoder can parse the first motion vector from it and then execute steps S202–S206 (or S402–S411) to determine the prediction block. In this case, the decoder can proceed according to… Figure 3a The embodiments generate multiple preset spatial distribution types and preset candidate list sets, and multiple preset spatial distribution types and preset candidate list sets can also be obtained from the encoding end. This application does not limit this.

[0415] In one possible approach, the index of the first motion vector corresponding to the motion vector candidate list can be encoded to obtain the first bitstream. Here, the motion vector candidate list refers to the aforementioned first motion vector candidate list. Thus, after receiving the first bitstream, the decoding end can parse the index of the first motion vector from the first bitstream, and then search for the first motion vector from the first motion vector candidate list based on the index. Next, steps S202–S206 (or S402–S411) can be executed (where the decoding end can determine the first target region based on the region indicator parsed from the first bitstream (or a pre-agreed method for determining the first target region)) to determine the prediction block. In this case, the decoding end can proceed according to… Figure 4aThe first motion vector candidate list is generated in the embodiment, or it can be obtained from the encoding end; this application does not limit this. Furthermore, the decoding end can follow... Figure 3a The embodiments generate multiple preset spatial distribution types and preset candidate list sets, and multiple preset spatial distribution types and preset candidate list sets can also be obtained from the encoding end. This application does not limit this.

[0416] In one possible approach, the vector difference between the first motion vector and the third motion vector corresponding to the coded block is encoded to obtain the first bitstream. The coded block can be a coded block in the image frame containing the current block, or a coded block in another image frame; this application does not limit this. The third motion vector can be the motion vector corresponding to any region (i.e., the motion field) within the coded block; this application does not limit this. Thus, after receiving the first bitstream, the decoding end can parse the vector difference from the first bitstream, and then use the vector difference and the vector sum of the third motion vector of the corresponding reconstructed block as the first motion vector. Next, it can execute the above steps S202-S206 (or S402-S411) (where the decoding end can determine the first target region based on the region indicator parsed from the first bitstream (or a pre-agreed method for determining the first target region)) to determine the prediction block. In this case, the decoding end can proceed according to... Figure 3a The embodiments generate multiple preset spatial distribution types and preset candidate list sets, and multiple preset spatial distribution types and preset candidate list sets can also be obtained from the encoding end. This application does not limit this.

[0417] For example, encoding can also be performed based on the second motion vector corresponding to the current block to obtain a second bitstream. The second motion vector can be based on the above... Figure 2a Implementation examples or Figure 4a This is generated by the embodiment. In this way, after the decoding end receives the first bitstream and the second bitstream, it can parse the first motion vector from the first bitstream and the second motion vector from the second bitstream; then, it executes the above-mentioned S206 (or S410 to S411) (wherein, the decoding end can determine the first target region according to the region indication identifier parsed from the first bitstream (or the method of determining the first target region agreed upon in advance with the decoding end) to determine the prediction block.

[0418] For example, encoding based on the second motion vector corresponding to the current block to obtain the second bitstream can also include multiple methods, which can refer to the above-described multiple methods of encoding based on the first motion vector corresponding to the current block to obtain the first bitstream, and will not be repeated here.

[0419] In other words, Figure 2a Implementation examples and Figure 4aIn some embodiments, the first motion vector must be encoded, but the second motion vector may or may not be encoded, and this application does not impose any restrictions on this.

[0420] For example, this application addresses matters other than those described above. Figure 2a Implementation examples and Figure 4a In addition to the methods of the embodiments, other methods can be used to determine the motion vectors corresponding to different motion fields for the current block. Alternatively, based on the target segmentation map and the motion vectors of different motion fields, integer-pixel-level motion compensation and sub-pixel-level motion compensation can be performed on the reference block determined according to the motion vectors corresponding to different motion fields. This can improve the precision of the predicted block, thereby improving the coding quality.

[0421] Figure 5 This is a schematic diagram illustrating an exemplary inter-frame prediction process. Figure 5 The inter-frame prediction process described in the embodiments can be an inter-frame prediction process in the encoding process or an inter-frame prediction process in the decoding process, and this application does not limit it in this way.

[0422] S501, obtain the first motion vector corresponding to the current block. The first motion vector includes an integer part and a fractional part.

[0423] S502, obtain the second motion vector corresponding to the current block. The second motion vector includes an integer part and a fractional part.

[0424] S503, determine the first reference block of the current block based on the integer part of the first motion vector.

[0425] S504, determine the second reference block of the current block based on the integer part of the second motion vector.

[0426] S505, Obtain the partition map, which describes the partitioning of the current block.

[0427] S506, perform pixel-level motion compensation based on the segmentation map, the first reference block, and the second reference block to determine the pixel prediction block.

[0428] S507, pixel-level motion compensation is performed based on the segmentation map, the integer pixel prediction block, the fractional part of the first motion vector, and the fractional part of the second motion vector to obtain the prediction block corresponding to the current block.

[0429] It should be noted that, Figure 5 In this embodiment, the current block can be partitioned (e.g., into two regions), and then the motion vectors corresponding to different regions in the current block (e.g., the first motion vector and the second motion vector) can be determined. This application does not restrict whether the first motion vector or the second motion vector is obtained first. That is to say, this application does not restrict the execution order of S501 and S502.

[0430] For example, the current block can be a block to be encoded or a block to be reconstructed; this application does not limit this.

[0431] For example, in Figure 5 In the embodiments described above, the method for determining the first motion vector during the encoding process can refer to the above. Figure 4a The method for determining the first motion vector in the embodiment can also refer to the above. Figure 4a The method of using the second motion vector in the embodiments is not limited in this application. During the decoding process, the method for determining the first motion vector can be to parse the first motion vector from the bitstream, or it can refer to the above. Figure 4a The second embodiment is a motion vector method.

[0432] For example, in Figure 5 In the embodiments described above, the method for determining the second motion vector during the encoding process can refer to the above. Figure 4a The method for determining the first motion vector in the embodiment can also refer to the above. Figure 4a The method of determining the second motion vector in the embodiments described herein is not limited thereto. During the decoding process, the second motion vector can be determined by parsing it from the bitstream, or by referring to the above-described method. Figure 4a The second embodiment is a motion vector method.

[0433] For example, S501 to S507 can refer to the above. Figure 4a The description of the embodiments will not be repeated here.

[0434] Figure 6a This is a schematic diagram illustrating an exemplary inter-frame prediction process. Figure 6a The inter-frame prediction process described in the embodiments can be an inter-frame prediction process in the encoding process or an inter-frame prediction process in the decoding process, and this application does not limit it in this way.

[0435] S601 divides the current block into the fifth and sixth regions.

[0436] For example, the current block can be geometrically divided into a fifth region and a sixth region.

[0437] Figure 6b This is a schematic diagram of geometric partitioning as an example.

[0438] Figure 6b (1) Angle parameters of the dividing line shown in the example And the offset parameter ρ. Figure 6b (2) Examples of various geometric partition diagrams are shown.

[0439] S602, obtain the first motion vector corresponding to the fifth region in the current block. The first motion vector includes an integer part and a fractional part, and the first motion vector corresponds to the fifth region.

[0440] For example, during the encoding process, if the second motion vector is not determined before the first motion vector is determined, then the above procedure can be followed. Figure 4a In this embodiment, the first motion vector is determined by determining the first motion vector corresponding to the fifth region in the current block. If the second motion vector has been determined before the first motion vector is determined, then the above procedure can be followed. Figure 4a In this embodiment, the second motion vector is determined by determining the first motion vector corresponding to the fifth region in the current block.

[0441] S603, Obtain the segmentation image.

[0442] In one possible approach, the first reference block is segmented to obtain a segmentation map; or,

[0443] In one possible approach, the second reference block is segmented to obtain a segmentation map; or,

[0444] In one possible approach, a first segmentation map obtained by segmenting a first reference block and a second segmentation map obtained by segmenting a second reference block are fused to obtain a segmentation map.

[0445] For example, if the current block is divided into a geometric partition (the current block is divided into a fifth region and a sixth region) to determine the first motion vector corresponding to the fifth region and the second motion vector corresponding to the sixth region in the current block, the partition map can be a geometric partition map.

[0446] For example, the segmentation map includes a first region and a second region.

[0447] In one possible arrangement, the first region corresponds to the fifth region, and the second region corresponds to the sixth region.

[0448] In one possible arrangement, the second region corresponds to the fifth region, and the first region corresponds to the sixth region.

[0449] For example, the process of obtaining the second motion vector corresponding to the sixth region in the current block can be described in S604 to S605 below.

[0450] It should be noted that S604 to S605 are implemented based on the correspondence between the first region and the fifth region, and the correspondence between the second region and the sixth region; and based on the determination of the first motion vector corresponding to the fifth region of the current block; S604 to S605 can be referred to the above for details. Figure 4a The description will not be repeated here.

[0451] It should be understood that when the segmentation map is a geometric partition map, this application does not limit whether the first motion vector is determined before performing S604 to S605.

[0452] S604. Based on the spatial distribution of the second region in the segmentation map, establish a candidate list of second motion vectors.

[0453] S605, based on the candidate list of second motion vectors, determine the second motion vector corresponding to the sixth region in the current block.

[0454] S606, determine the first reference block of the current block based on the integer part of the first motion vector.

[0455] S607, determine the second reference block of the current block based on the integer part of the second motion vector.

[0456] S608 performs pixel-level motion compensation based on the segmentation map, the first reference block, and the second reference block to determine the pixel prediction block.

[0457] S609, pixel-level motion compensation is performed based on the segmentation map, the integer pixel prediction block, the fractional part of the first motion vector, and the fractional part of the second motion vector to obtain the prediction block corresponding to the current block.

[0458] For example, S608 to S609 can be referred to Figure 4a The descriptions in the embodiments will not be repeated here.

[0459] For example, this application proposes a bitstream generation method, which can encode based on a first motion vector corresponding to the current block and the partition information of the current block to obtain a first bitstream. The first motion vector can be based on the above... Figure 5 Implementation examples and Figure 6a This is generated using an embodiment. Thus, after receiving the bitstream, the decoding end can decode the partitioning information (such as the angle parameters and offset parameters of the dividing lines) and the first motion vector from the bitstream. Then, it can partition the current block according to the partitioning information (i.e., execute S601), and determine the fifth and sixth regions (wherein, the encoding and decoding ends can pre-agree which region is the fifth region and which is the sixth region; or the partitioning information can carry a region identifier to indicate the fifth and / or sixth regions), and determine the region corresponding to the first motion vector as the fifth region. Subsequently, it can execute the above-described S502 to S507 (or S603 to S609) to determine the prediction block.

[0460] For example, the bitstream generation method further includes: encoding based on the second motion vector corresponding to the current block to obtain a second bitstream. The second motion vector can be based on the above... Figure 5 Implementation examples and Figure 6aThis is generated by the embodiment. In this way, after the decoding end receives the first bitstream and the second bitstream, it can decode the first motion vector and partition information from the first bitstream and decode the second motion vector from the second bitstream; then, it can execute the above S503 to S507 (or S601, S603, S606 to S609) to determine the prediction block.

[0461] For example, this application also proposes a bitstream generation method, which can encode based on the second motion vector corresponding to the current block and the partition information of the current block to obtain a first bitstream. The second motion vector can be based on the above... Figure 5 Implementation examples and Figure 6a This is generated using an embodiment. Thus, after receiving the bitstream, the decoding end can decode the partitioning information (such as the angle parameters and offset parameters of the dividing lines) and the second motion vector from the bitstream. Then, it can partition the current block according to the partitioning information (i.e., execute S601), and determine the fifth and sixth regions (wherein, the encoding and decoding ends can pre-agree which region is the fifth region and which is the sixth region; or the partitioning information can carry region identifiers to indicate the fifth and / or sixth regions), and determine the region corresponding to the second motion vector as the sixth region. Subsequently, it can execute the above-described S501, S503 to S507 (or S602 to S603, S606 to S609) to determine the prediction block.

[0462] For example, the bitstream generation method further includes: encoding based on the first motion vector corresponding to the current block to obtain a second bitstream. Thus, after receiving the first and second bitstreams, the decoding end can decode the second motion vector and partition information from the first bitstream, and decode the first motion vector from the second bitstream; then, steps S503–S507 (or S601, S603, S606–S609) can be executed to determine the prediction block.

[0463] In other words, Figure 5 Implementation examples and Figure 6a In some embodiments, either one or both of the first motion vector and the second motion vector can be encoded, and this application does not limit this.

[0464] It should be noted that if the current block is divided into a geometric partition (the current block is divided into a fifth region and a sixth region) to determine the first motion vector corresponding to the fifth region and the second motion vector corresponding to the sixth region in the current block, it is not necessary to encode based on the first motion vector or the second motion vector; instead, the partitioning information of the geometric partition can be encoded. Correspondingly, this application provides a bitstream generation method that encodes the partitioning information to obtain the bitstream. In this case, after receiving the bitstream, the decoding end can parse the partitioning information from the bitstream, then partition the block to be reconstructed according to the partitioning information, and then... Figure 4a The embodiment establishes a second motion vector list by creating a first motion vector list and a second motion vector list; then, from the first motion vector list, a third motion vector at a first preset position is selected as the first motion vector, and from the second motion vector list, a third motion vector at a second preset position is selected as the second motion vector. The first and second preset positions can be pre-agreed by the decoding and encoding ends, or the first and second preset positions can be identified and written into the bitstream, allowing the decoding end to parse the first and second preset positions from the bitstream. This application does not impose any restrictions on this.

[0465] This application also provides an inter-frame prediction apparatus that can be used to perform the methods of the foregoing embodiments. Therefore, the beneficial effects it can achieve can be referred to the beneficial effects of the corresponding methods provided above, and will not be repeated here.

[0466] Figure 7 This is a schematic diagram illustrating an exemplary inter-frame prediction apparatus. (Refer to...) Figure 7 For example, an inter-frame prediction device may include:

[0467] The first motion vector acquisition module 701 is used to acquire the first motion vector corresponding to the current block;

[0468] The segmentation map determination module 702 is used to obtain a first segmentation map of the first reference block corresponding to the current block based on the first motion vector. The first segmentation map is used to describe the partitioning of the current block and includes a first region and a second region.

[0469] The target area determination module 703 is used to determine a first target area from the first area and the second area;

[0470] The candidate list building module 704 is used to build a motion vector candidate list based on the spatial distribution of the first target region in the first segmentation map. The spatial distribution is used to describe the position of the first target region in the first segmentation map.

[0471] The second motion vector determination module 705 is used to determine the second motion vector corresponding to the current block based on the motion vector candidate list. The second motion vector corresponds to the third region in the current block, the first motion vector corresponds to the fourth region in the current block, the third region corresponds to the first target region, and the fourth region corresponds to the second target region in the first and second regions.

[0472] The motion compensation module 706 is used to perform motion compensation based on the target segmentation map, the first motion vector, and the second motion vector to obtain the prediction block corresponding to the current block. The target segmentation map is determined based on the first segmentation map and / or the second segmentation map, and the second segmentation map is a segmentation map of the second reference block corresponding to the current block determined based on the second motion vector.

[0473] For example, the motion vector candidate list includes multiple third motion vectors corresponding to multiple adjacent blocks of the current block; the distance between the adjacent block corresponding to the i-th third motion vector in the motion vector candidate list and the third region is less than the distance between the adjacent block corresponding to the (i+1)-th third motion vector and the third region; i is a positive integer.

[0474] For example, the second motion vector determination module 705 is specifically used to select the first third motion vector in the motion vector candidate list as the second motion vector.

[0475] For example, the candidate list building module 704 is specifically used to determine the spatial distribution type corresponding to the first target region from a variety of preset spatial distribution types based on the spatial distribution of the first target region in the first segmentation map; to determine a motion vector candidate list from a preset candidate list set based on the spatial distribution type corresponding to the first target region, the preset candidate list set including multiple preset motion vector candidate lists corresponding to multiple preset spatial distribution types, the preset motion vector candidate list including multiple neighbor block identifiers, the neighbor block identifiers being used to represent the neighboring blocks at a certain position of the current block; and to fill the motion vector candidate list with the third motion vectors corresponding to the multiple neighboring blocks of the current block according to the multiple neighbor block identifiers included in the motion vector candidate list.

[0476] For example, both the first motion vector and the second motion vector include an integer part and a fractional part; the motion compensation module 706 is specifically used to perform integer-pixel-level motion compensation based on the target segmentation map, the first reference block and the second reference block to determine an integer-pixel prediction block, wherein the first reference block is determined based on the integer part of the first motion vector and the second reference block is determined based on the integer part of the second motion vector; and to perform fractional-pixel-level motion compensation based on the target segmentation map, the integer-pixel prediction block, the fractional part of the first motion vector and the fractional part of the second motion vector to obtain the prediction block corresponding to the current block.

[0477] For example, the motion compensation module 706 is specifically used to linearly fuse the first reference block and the second reference block based on the target segmentation map to determine the integer pixel prediction block.

[0478] For example, the motion compensation module 706 is specifically used to determine the pixel category corresponding to each pixel in the first reference block and the pixel category corresponding to each pixel in the second reference block based on the distance between each pixel in the target segmentation image and the segmentation line, wherein the segmentation line is the segmentation line between the first region and the second region in the target segmentation image; determine the first weight corresponding to each pixel in the first reference block based on the target segmentation image and the pixel category corresponding to each pixel in the first reference block; determine the second weight corresponding to each pixel in the second reference block based on the pixel category corresponding to each pixel in the target segmentation image and the second reference block; and linearly weight the pixel values ​​of corresponding pixels in the first reference block and the second reference block based on the first weight and the second weight to obtain an integer pixel prediction block.

[0479] For example, the motion compensation module 706 is specifically used to divide the integer pixel prediction block into an eighth region and a ninth region based on the dividing line between the first region and the second region in the target segmentation map; to perform pixel interpolation on the eighth region based on the fractional part of the first motion vector to obtain a first pixel prediction block; and to perform pixel interpolation on the ninth region based on the fractional part of the second motion vector to obtain a second pixel prediction block; and to perform linear fusion on the first pixel prediction block and the second pixel prediction block based on the target segmentation map to obtain a prediction block.

[0480] For example, the inter-frame prediction device further includes: a candidate list set establishment module, used to establish a preset candidate list set; wherein,

[0481] The candidate list set establishment module is specifically used to obtain multiple preset segmentation maps corresponding to multiple preset image blocks. The preset segmentation maps describe the partitioning of the preset image blocks and include a fifth region and a sixth region. Based on the spatial distribution of the third target region in each preset segmentation map, classification is performed to obtain multiple preset spatial distribution types. The third target region is selected from the fifth and sixth regions. For the target spatial distribution type among the multiple preset spatial distribution types, based on the positional relationship between the seventh region in the target preset image block corresponding to the target spatial distribution type and multiple adjacent blocks of the target preset image block, a preset motion vector candidate list corresponding to the target spatial distribution type is determined. The seventh region corresponds to the third target region. Using the preset motion vector candidate lists corresponding to various preset spatial distribution types, a preset candidate list set is established.

[0482] For example, the candidate list set building module is specifically used to determine the distance between the seventh region in the target preset image block and multiple adjacent blocks of the target preset image block; based on the distance between the seventh region in the target preset image block and multiple adjacent blocks of the target preset image block, determine the arrangement order of the multiple adjacent blocks of the target preset image block; and according to the arrangement order, fill the neighbor block identifiers corresponding to the multiple adjacent blocks of the target preset image block into the preset motion vector candidate list corresponding to the target spatial distribution type.

[0483] For example, in the preset motion vector candidate list corresponding to the target spatial distribution type, the distance between the neighboring block corresponding to the kth neighboring block identifier and the seventh region in the target preset image block is less than the distance between the neighboring block corresponding to the (k+1)th neighboring block identifier and the seventh region in the target preset image block; k is a positive integer.

[0484] For example, the target area determination module 703 is specifically used to determine the area with the smallest area in the first area and the second area as the first target area; wherein the second target area is the area with the largest area in the first area and the second area.

[0485] For example, the target segmentation map is a first segmentation map; or, the target segmentation map is a second segmentation map; or, the target segmentation map is a segmentation map obtained by fusing the first segmentation map and the second segmentation map.

[0486] This application also provides an inter-frame prediction apparatus that can be used to perform the methods of the foregoing embodiments. Therefore, the beneficial effects it can achieve can be referred to the beneficial effects of the corresponding methods provided above, and will not be repeated here.

[0487] Figure 8 This is a schematic diagram illustrating an exemplary inter-frame prediction apparatus. (Refer to...) Figure 8 For example, an inter-frame prediction device may include:

[0488] The motion vector acquisition module 801 is used to acquire the first motion vector corresponding to the current block and the second motion vector corresponding to the current block. Both the first motion vector and the second motion vector include an integer part and a fractional part.

[0489] The reference block determination module 802 is used to determine a first reference block of the current block based on the integer part of a first motion vector, and to determine a second reference block of the current block based on the integer part of a second motion vector.

[0490] The segmentation map acquisition module 803 is used to acquire a segmentation map, which describes the partitioning of the current block.

[0491] The integer pixel motion compensation module 804 is used to perform integer pixel-level motion compensation based on the segmentation map, the first reference block and the second reference block, and to determine the integer pixel prediction block.

[0492] The pixel-level motion compensation module 805 is used to perform pixel-level motion compensation based on the segmentation map, the integer pixel prediction block, the fractional part of the first motion vector, and the fractional part of the second motion vector to obtain the prediction block corresponding to the current block.

[0493] For example, the integer pixel motion compensation module 804 is specifically used to linearly fuse the first reference block and the second reference block based on the segmentation map to determine the integer pixel prediction block.

[0494] For example, the integer pixel motion compensation module 804 is specifically used to determine the pixel category corresponding to each pixel in the first reference block and the pixel category corresponding to each pixel in the second reference block based on the distance between each pixel in the segmentation map and the segmentation line, where the segmentation line is the segmentation line between the first region and the second region in the segmentation map; determine the first weight corresponding to each pixel in the first reference block based on the segmentation map and the pixel category corresponding to each pixel in the first reference block; determine the second weight corresponding to each pixel in the second reference block based on the pixel category corresponding to each pixel in the segmentation map and the second reference block; and linearly weight the pixel values ​​of the corresponding pixels in the first reference block and the second reference block based on the first weight and the second weight to obtain an integer pixel prediction block.

[0495] For example, the pixel-level motion compensation module 805 is specifically used to divide the integer pixel prediction block into a third region and a fourth region based on the dividing line between the first region and the second region in the segmentation map; to perform pixel-level interpolation on the third region based on the fractional part of the first motion vector to obtain a first pixel-level prediction block; and to perform pixel-level interpolation on the fourth region based on the fractional part of the second motion vector to obtain a second pixel-level prediction block; and to perform linear fusion on the first pixel-level prediction block and the second pixel-level prediction block based on the segmentation map to obtain a prediction block.

[0496] For example, the current block is divided into a fifth region and a sixth region, and the segmentation map includes a first region corresponding to the fifth region and a second region corresponding to the sixth region; a first motion vector corresponds to the fifth region, and a second motion vector corresponds to the sixth region; the motion vector acquisition module 801 is specifically used to establish a first motion vector candidate list based on the spatial distribution of the first region in the segmentation map, the spatial distribution being used to describe the position of the first region in the segmentation map; and based on the first motion vector candidate list, to determine the first motion vector corresponding to the fifth region in the current block.

[0497] For example, the current block is divided into a fifth region and a sixth region. The segmentation map includes a first region corresponding to the fifth region and a second region corresponding to the sixth region. A first motion vector corresponds to the fifth region, and a second motion vector corresponds to the sixth region. The motion vector acquisition module 801 is specifically used to establish a candidate list of second motion vectors based on the spatial distribution of the second region in the segmentation map. The spatial distribution is used to describe the position of the second region in the segmentation map. Based on the candidate list of second motion vectors, the second motion vector corresponding to the sixth region in the current block is determined.

[0498] For example, the second motion vector candidate list includes multiple third motion vectors corresponding to multiple adjacent blocks of the current block; the distance between the adjacent block corresponding to the i-th third motion vector in the second motion vector candidate list and the sixth region is less than the distance between the adjacent block corresponding to the (i+1)-th third motion vector and the sixth region; i is a positive integer.

[0499] For example, the motion vector acquisition module 801 is specifically used to select the first third motion vector in the second motion vector candidate list as the second motion vector.

[0500] For example, the motion vector acquisition module 801 is specifically used to determine the spatial distribution type corresponding to the second region from a variety of preset spatial distribution types based on the spatial distribution of the second region in the segmentation map; to determine a second motion vector candidate list from a preset candidate list set based on the spatial distribution type corresponding to the second region, the preset candidate list set including multiple preset motion vector candidate lists corresponding to multiple preset spatial distribution types, the preset motion vector candidate list including multiple neighbor block identifiers, the neighbor block identifiers being used to represent the neighboring blocks at a position of the current block; and to fill the second motion vector candidate list with the third motion vectors corresponding to the multiple neighbor blocks of the current block according to the multiple neighbor block identifiers included in the second motion vector candidate list.

[0501] For example, the inter-frame prediction device further includes: a candidate list set establishment module, used to establish a preset candidate list set; wherein,

[0502] The candidate list set establishment module is specifically used to obtain multiple preset segmentation maps corresponding to multiple preset image blocks. The preset segmentation maps describe the partitioning of the preset image blocks and include a seventh region and an eighth region. Based on the spatial distribution of the target region in each preset segmentation map, classification is performed to obtain multiple preset spatial distribution types. The target region is selected from the seventh region and the eighth region. For the target spatial distribution type among the multiple preset spatial distribution types, based on the positional relationship between the target spatial distribution type and the ninth region in the target preset image block, and multiple adjacent blocks of the target preset image block, a preset motion vector candidate list corresponding to the target spatial distribution type is determined. The ninth region corresponds to the third target region. Using the preset motion vector candidate lists corresponding to various preset spatial distribution types, a preset candidate list set is established.

[0503] For example, the candidate list set building module is specifically used to determine the distance between the ninth region in the target preset image block and multiple adjacent blocks of the target preset image block; based on the distance between the ninth region in the target preset image block and multiple adjacent blocks of the target preset image block, determine the arrangement order of the multiple adjacent blocks of the target preset image block; and according to the arrangement order, fill the neighbor block identifiers corresponding to the multiple adjacent blocks of the target preset image block into the preset motion vector candidate list corresponding to the target spatial distribution type.

[0504] For example, in the preset motion vector candidate list corresponding to the target spatial distribution type, the distance between the neighboring block corresponding to the kth neighboring block identifier and the ninth region in the target preset image block is less than the distance between the neighboring block corresponding to the (k+1)th neighboring block identifier and the ninth region in the target preset image block; k is a positive integer.

[0505] For example, the segmentation map acquisition module 802 is specifically used to segment the first reference block to obtain a segmentation map; or, to segment the second reference block to obtain a segmentation map; or, to fuse the first segmentation map obtained by segmenting the first reference block and the second segmentation map obtained by segmenting the second reference block to obtain a segmentation map.

[0506] In one example, Figure 9 A schematic block diagram illustrating an embodiment of the present application shows an apparatus 900. The apparatus 900 may include a processor 901 and a transceiver / transceiver pin 902, and optionally, a memory 903.

[0507] The various components of device 900 are coupled together via bus 904, which includes a data bus, a power bus, a control bus, and a status signal bus. However, for clarity, all buses are referred to as bus 904 in the figure.

[0508] Optionally, the memory 903 can be used to store instructions from the foregoing method embodiments. The processor 901 can be used to execute the instructions in the memory 903, control the receive pin to receive signals, and control the transmit pin to transmit signals.

[0509] The device 900 may be an electronic device or a chip of an electronic device in the above method embodiments.

[0510] All relevant content of each step involved in the above method embodiments can be referenced from the functional description of the corresponding functional module, and will not be repeated here.

[0511] This embodiment also provides a computer-readable storage medium storing computer instructions. When the computer instructions are executed on an electronic device, the electronic device performs the aforementioned method steps to implement the methods described in the above embodiments.

[0512] This embodiment also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned steps to implement the methods described in the above embodiments.

[0513] In addition, embodiments of this application also provide an apparatus, which may specifically be a chip, component, or module. The apparatus may include a connected processor and a memory; wherein the memory is used to store computer execution instructions, and when the apparatus is running, the processor may execute the computer execution instructions stored in the memory to cause the chip to execute the methods in the above-described method embodiments.

[0514] In this embodiment, the electronic device, computer-readable storage medium, computer program product or chip are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0515] Through the above description of the embodiments, those skilled in the art will understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0516] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another apparatus, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0517] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0518] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0519] Any content in the various embodiments of this application, as well as any content in the same embodiment, can be freely combined. Any combination of the above content is within the scope of this application.

[0520] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0521] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0522] The steps of the methods or algorithms described in conjunction with the embodiments of this application can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium well known in the art. An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0523] Those skilled in the art will recognize that the functions described in the embodiments of this application in one or more of the above examples can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer-readable storage media and communication media, wherein communication media include any medium that facilitates the transmission of a computer program from one place to another. Storage media can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0524] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An inter prediction method, characterized by, The method includes: Obtain the first motion vector corresponding to the current block; Based on the first motion vector, a first segmentation map of the first reference block corresponding to the current block is obtained. The first segmentation map is used to describe the partitioning of the current block. The first segmentation map includes a first region and a second region. A first target area is determined from the first area and the second area; Based on the spatial distribution of the first target region in the first segmentation map, a motion vector candidate list is established, wherein the spatial distribution is used to describe the position of the first target region in the first segmentation map; Based on the motion vector candidate list, a second motion vector corresponding to the current block is determined. The second motion vector corresponds to a third region in the current block, the first motion vector corresponds to a fourth region in the current block, the third region corresponds to the first target region, and the fourth region corresponds to a second target region in the first and second regions. Motion compensation is performed based on the target segmentation map, the first motion vector, and the second motion vector to obtain the predicted block corresponding to the current block. The target segmentation map is determined based on the first segmentation map and / or the second segmentation map, and the second segmentation map is a segmentation map of the second reference block corresponding to the current block determined based on the second motion vector.

2. The method of claim 1, wherein, The motion vector candidate list includes multiple third motion vectors corresponding to multiple adjacent blocks of the current block; The distance between the adjacent block corresponding to the i-th third motion vector in the motion vector candidate list and the third region is less than the distance between the adjacent block corresponding to the (i+1)-th third motion vector and the third region; i is a positive integer.

3. The method of claim 2, wherein, The step of determining the second motion vector corresponding to the current block based on the motion vector candidate list includes: The first third motion vector in the candidate list of motion vectors is selected as the second motion vector.

4. The method according to any one of claims 1 to 3, characterized in that, The step of establishing a motion vector candidate list based on the spatial distribution of the first target region in the first segmentation map includes: Based on the spatial distribution of the first target region in the first segmentation map, the spatial distribution type corresponding to the first target region is determined from a variety of preset spatial distribution types; Based on the spatial distribution type corresponding to the first target area, the motion vector candidate list is determined from a preset candidate list set. The preset candidate list set includes multiple preset motion vector candidate lists corresponding to multiple preset spatial distribution types. The preset motion vector candidate list includes multiple neighbor block identifiers, which are used to represent the neighboring blocks at a position of the current block. According to the multiple neighbor block identifiers contained in the motion vector candidate list, the third motion vectors corresponding to multiple neighboring blocks of the current block are filled into the motion vector candidate list.

5. The method according to any one of claims 1 to 4, characterized in that, Both the first motion vector and the second motion vector include an integer part and a fractional part; The motion compensation based on the target segmentation map, the first motion vector, and the second motion vector to obtain the prediction block corresponding to the current block includes: Based on the target segmentation map, the first reference block, and the second reference block, integer pixel-level motion compensation is performed to determine integer pixel prediction blocks, wherein the first reference block is determined based on the integer part of the first motion vector, and the second reference block is determined based on the integer part of the second motion vector; Pixel-level motion compensation is performed based on the target segmentation map, the integer pixel prediction block, the fractional part of the first motion vector, and the fractional part of the second motion vector to obtain the prediction block corresponding to the current block.

6. The method of claim 5, wherein, The step of performing integer-pixel-level motion compensation based on the target segmentation map, the first reference block, and the second reference block to determine integer-pixel prediction blocks includes: Based on the target segmentation map, the first reference block and the second reference block are linearly fused to determine the integer pixel prediction block.

7. The method of claim 6, wherein, The step of linearly fusing the first reference block and the second reference block based on the target segmentation map to determine the integer pixel prediction block includes: Based on the distance between each pixel in the target segmentation image and the dividing line, the pixel category corresponding to each pixel in the first reference block and the pixel category corresponding to each pixel in the second reference block are determined, and the dividing line is the dividing line between the first region and the second region in the target segmentation image; Based on the target segmentation map and the pixel category corresponding to each pixel in the first reference block, the first weight corresponding to each pixel in the first reference block is determined; Based on the target segmentation map and the pixel category corresponding to each pixel in the second reference block, the second weight corresponding to each pixel in the second reference block is determined; Based on the first weight and the second weight, the pixel values ​​of corresponding pixels in the first reference block and the second reference block are linearly weighted to obtain the integer pixel prediction block.

8. The method according to any one of claims 5 to 7, characterized in that, The step of performing pixel-level motion compensation based on the target segmentation map, the integer-pixel prediction block, the fractional portion of the first motion vector, and the fractional portion of the second motion vector to obtain the prediction block corresponding to the current block includes: Based on the dividing lines between the first and second regions in the target segmentation map, the integer pixel prediction block is divided into the eighth and ninth regions; The eighth region is interpolated pixel-wise based on the fractional portion of the first motion vector to obtain a first pixel prediction block; and the ninth region is interpolated pixel-wise based on the fractional portion of the second motion vector to obtain a second pixel prediction block. Based on the target segmentation map, the first sub-pixel prediction block and the second sub-pixel prediction block are linearly fused to obtain the prediction block.

9. The method of claim 4, wherein, The method further includes establishing the preset candidate list set: Multiple preset segmentation maps corresponding to multiple preset image blocks are obtained. The preset segmentation maps are used to describe the partitioning of the preset image blocks. The preset segmentation maps include a fifth region and a sixth region. The spatial distribution of the third target region in each preset segmentation map is classified to obtain the various preset spatial distribution types. The third target region is selected from the fifth region and the sixth region. For the target spatial distribution type among the various preset spatial distribution types, based on the positional relationship between the seventh region in the target preset image block corresponding to the target spatial distribution type and multiple adjacent blocks of the target preset image block, a preset motion vector candidate list corresponding to the target spatial distribution type is determined, wherein the seventh region corresponds to the third target region; The preset candidate list set is established by using the preset motion vector candidate lists corresponding to the various preset spatial distribution types.

10. The method of claim 9, wherein, For the target spatial distribution type among the multiple preset spatial distribution types, based on the positional relationship between the seventh region in the target preset image block corresponding to the target spatial distribution type and multiple adjacent blocks of the target preset image block, a preset motion vector candidate list corresponding to the target spatial distribution type is determined, including: Determine the distances between the seventh region in the target preset image block and multiple adjacent blocks of the target preset image block; Based on the seventh region in the target preset image block and the distance between it and multiple adjacent blocks of the target preset image block, the arrangement order of the multiple adjacent blocks of the target preset image block is determined; According to the arrangement order, the neighboring block identifiers of multiple adjacent blocks of the target preset image block are filled into the preset motion vector candidate list corresponding to the target spatial distribution type.

11. The method according to claim 9 or 10, characterized in that, In the preset motion vector candidate list corresponding to the target spatial distribution type, the distance between the neighboring block corresponding to the kth neighboring block identifier and the seventh region in the target preset image block is less than the distance between the neighboring block corresponding to the (k+1)th neighboring block identifier and the seventh region in the target preset image block; k is a positive integer.

12. The method according to any one of claims 1 to 11, characterized in that, Determining the first target region from the first region and the second region includes: The region with the smallest area among the first region and the second region is determined as the first target region; The second target area is the largest area among the first and second areas.

13. The method according to any one of claims 1 to 12, characterized in that, The target segmentation map is the first segmentation map; or... The target segmentation map is the second segmentation map; or... The target segmentation map is a segmentation map obtained by fusing the first segmentation map and the second segmentation map.

14. An inter prediction method, characterized by, The method includes: Obtain the first motion vector corresponding to the current block and obtain the second motion vector corresponding to the current block, wherein both the first motion vector and the second motion vector include an integer part and a fractional part; A first reference block of the current block is determined based on the integer part of the first motion vector, and a second reference block of the current block is determined based on the integer part of the second motion vector; Obtain a segmentation map, which is used to describe the partitioning of the current block; Based on the segmentation map, the first reference block, and the second reference block, perform pixel-level motion compensation to determine the pixel prediction block; Pixel-level motion compensation is performed based on the segmentation map, the integer pixel prediction block, the fractional part of the first motion vector, and the fractional part of the second motion vector to obtain the prediction block corresponding to the current block.

15. The method of claim 14, wherein, The step of performing integer-pixel-level motion compensation based on the segmentation map, the first reference block, and the second reference block to determine integer-pixel prediction blocks includes: Based on the segmentation map, the first reference block and the second reference block are linearly fused to determine the integer pixel prediction block.

16. The method of claim 15, wherein, The segmentation map includes a first region and a second region; The step of linearly fusing the first reference block and the second reference block based on the segmentation map to determine the integer pixel prediction block includes: Based on the distance between each pixel in the segmentation image and the segmentation line, the pixel category corresponding to each pixel in the first reference block and the pixel category corresponding to each pixel in the second reference block are determined, and the segmentation line is the segmentation line between the first region and the second region in the segmentation image; Based on the segmentation map and the pixel category corresponding to each pixel in the first reference block, the first weight corresponding to each pixel in the first reference block is determined; Based on the segmentation map and the pixel category corresponding to each pixel in the second reference block, the second weight corresponding to each pixel in the second reference block is determined; Based on the first weight and the second weight, the pixel values ​​of the corresponding pixels in the first reference block and the second reference block are linearly weighted to obtain the integer pixel prediction block.

17. The method according to any one of claims 14 to 16, characterized in that, The step of performing pixel-level motion compensation based on the segmentation map, the integer-pixel prediction block, the fractional portion of the first motion vector, and the fractional portion of the second motion vector to obtain the prediction block corresponding to the current block includes: Based on the dividing lines of the first and second regions in the segmentation map, the integer pixel prediction block is divided into the third and fourth regions; The third region is interpolated pixel-wise based on the fractional portion of the first motion vector to obtain a first pixel prediction block; and the fourth region is interpolated pixel-wise based on the fractional portion of the second motion vector to obtain a second pixel prediction block. Based on the segmentation map, the first sub-pixel prediction block and the second sub-pixel prediction block are linearly fused to obtain the prediction block.

18. The method according to any one of claims 14 to 17, characterized in that, The current block is divided into a fifth region and a sixth region. The segmentation map includes a first region corresponding to the fifth region and a second region corresponding to the sixth region. The first motion vector corresponds to the fifth region, and the second motion vector corresponds to the sixth region. The step of obtaining the first motion vector corresponding to the current block includes: Based on the spatial distribution of the first region in the segmentation map, a first motion vector candidate list is established, wherein the spatial distribution is used to describe the position of the first region in the segmentation map; Based on the first motion vector candidate list, the first motion vector corresponding to the fifth region in the current block is determined.

19. The method according to any one of claims 14 to 17, characterized in that, The current block is divided into a fifth region and a sixth region. The segmentation map includes a first region corresponding to the fifth region and a second region corresponding to the sixth region. The first motion vector corresponds to the fifth region, and the second motion vector corresponds to the sixth region. The step of obtaining the second motion vector corresponding to the current block includes: Based on the spatial distribution of the second region in the segmentation map, a second motion vector candidate list is established, wherein the spatial distribution is used to describe the position of the second region in the segmentation map; Based on the second motion vector candidate list, the second motion vector corresponding to the sixth region in the current block is determined.

20. The method of claim 19, wherein, The second motion vector candidate list includes multiple third motion vectors corresponding to multiple adjacent blocks of the current block; The distance between the adjacent block corresponding to the i-th third motion vector in the second motion vector candidate list and the sixth region is less than the distance between the adjacent block corresponding to the (i+1)-th third motion vector and the sixth region; i is a positive integer.

21. The method of claim 20, wherein, The step of determining the second motion vector corresponding to the sixth region in the current block based on the second motion vector candidate list includes: The first third motion vector in the second motion vector candidate list is selected as the second motion vector.

22. The method according to any one of claims 19 to 21, characterized in that, The step of establishing a second motion vector candidate list based on the spatial distribution of the second region in the segmentation map includes: Based on the spatial distribution of the second region in the segmentation diagram, the spatial distribution type corresponding to the second region is determined from a variety of preset spatial distribution types; Based on the spatial distribution type corresponding to the second region, the second motion vector candidate list is determined from the preset candidate list set. The preset candidate list set includes multiple preset motion vector candidate lists corresponding to multiple preset spatial distribution types. The preset motion vector candidate list includes multiple neighbor block identifiers. The neighbor block identifier is used to represent the adjacent block at a position of the current block. According to the multiple neighbor block identifiers contained in the second motion vector candidate list, the third motion vectors corresponding to multiple neighboring blocks of the current block are filled into the second motion vector candidate list.

23. The method according to claim 22, characterized in that, The method further includes: establishing the preset candidate list set. Multiple preset segmentation maps corresponding to multiple preset image blocks are obtained. The preset segmentation maps are used to describe the partitioning of the preset image blocks. The preset segmentation maps include a seventh region and an eighth region. The target regions are classified based on the spatial distribution of each preset segmentation map to obtain the various preset spatial distribution types. The target regions are selected from the seventh region and the eighth region. For the target spatial distribution type among the various preset spatial distribution types, based on the positional relationship between the target spatial distribution type and the ninth region in the target preset image block, and between the target spatial distribution type and multiple adjacent blocks of the target preset image block, a preset motion vector candidate list corresponding to the target spatial distribution type is determined, wherein the ninth region corresponds to the third target region; The preset candidate list set is established by using the preset motion vector candidate lists corresponding to the various preset spatial distribution types.

24. The method according to claim 23, characterized in that, For the target spatial distribution type among the multiple preset spatial distribution types, based on the positional relationship between the target spatial distribution type and the ninth region in the target preset image block, and between the target spatial distribution type and multiple adjacent blocks of the target preset image block, a preset motion vector candidate list corresponding to the target spatial distribution type is determined, including: Determine the distances between the ninth region in the target preset image block and multiple adjacent blocks of the target preset image block; Based on the distance between the ninth region in the target preset image block and multiple adjacent blocks of the target preset image block, the arrangement order of the multiple adjacent blocks of the target preset image block is determined; According to the arrangement order, the neighboring block identifiers of multiple adjacent blocks of the target preset image block are filled into the preset motion vector candidate list corresponding to the target spatial distribution type.

25. The method according to claim 23 or 24, characterized in that, In the preset motion vector candidate list corresponding to the target spatial distribution type, the distance between the neighboring block corresponding to the kth neighboring block identifier and the ninth region in the target preset image block is less than the distance between the neighboring block corresponding to the (k+1)th neighboring block identifier and the ninth region in the target preset image block; k is a positive integer.

26. The method according to any one of claims 14 to 25, characterized in that, The process of obtaining the segmentation map includes: The first reference block is segmented to obtain the segmentation map; or, The second reference block is segmented to obtain the segmentation map; or, The first segmentation map obtained by segmenting the first reference block and the second segmentation map obtained by segmenting the second reference block are fused together to obtain the segmentation map.

27. A method for generating a bitstream, characterized in that, The method includes: Encode the first bitstream based on the first motion vector corresponding to the current block; Wherein, the first motion vector is determined based on the method described in any one of claims 1 to 26.

28. The method according to claim 27, characterized in that, The method further includes: Write the region indicator corresponding to the first motion vector into the first bitstream.

29. The method according to claim 27 or 28, characterized in that, The method further includes: Encode the second bitstream based on the second motion vector corresponding to the current block; The second motion vector is determined based on the method described in any one of claims 1 to 26.

30. A method for generating a bitstream, characterized in that, The method includes: Encode the bitstream based on the second motion vector corresponding to the current block; The second motion vector is determined based on the method described in any one of claims 14 to 26.

31. A method for transmitting a code stream, characterized in that, The method is used to transmit the bitstream generated by the bitstream generation method according to any one of claims 27 to 30.

32. An apparatus for storing a bitstream, characterized in that, The device includes: a receiver and at least one storage medium. The receiver is used to receive the bit stream; The at least one storage medium is used to store the bit stream; The bitstream is generated according to the bitstream generation method as described in any one of claims 27-30.

33. A system for distributing bitstreams, characterized in that, The system includes: At least one storage medium for storing at least one bitstream, said at least one bitstream being generated according to the bitstream generation method of any one of claims 27-30. A streaming media device is configured to acquire a target bitstream from the at least one storage medium and send the target bitstream to an end-side device, wherein the streaming media device includes a content server or a content distribution server.

34. An apparatus for transmitting a code stream, characterized in that, The device includes: a transmitter and at least one storage medium. The at least one storage medium is used to store a bitstream, which is generated according to the bitstream generation method according to any one of claims 27-30; The transmitter is used to obtain the bitstream from the storage medium and send the bitstream to the end-side device through the transmission medium.

35. An electronic device, characterized in that, include: A memory and a processor, wherein the memory is coupled to the processor; The memory stores program instructions that, when executed by the processor, cause the electronic device to perform the inter-frame prediction method according to any one of claims 1 to 26.

36. A chip, characterized in that, The device includes one or more interface circuits and one or more processors; the interface circuits are configured to receive signals from the memory of an electronic device and send the signals to the processors, the signals including computer instructions stored in the memory; when the processor executes the computer instructions, the electronic device performs the inter-frame prediction method according to any one of claims 1 to 26.

37. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when run on a computer or processor, causes the computer or processor to perform the inter-frame prediction method as described in any one of claims 1 to 26.

38. A computer program product, characterized in that, The computer program product includes a software program that, when executed by a computer or processor, causes the steps of the method described in any one of claims 1 to 26 to be performed.

Images