Encoding method, decoding method, code stream, encoders, decoders, and storage medium

Abstract

Disclosed in the embodiments of the present application are an encoding method, a decoding method, a code stream, encoders, decoders, and a storage medium. A method comprises: determining an initial reference block of a current block, and determining a macro pixel co-located block corresponding to the initial reference block (S901); separately performing search on the basis of the initial reference block and the macro pixel co-located block to determine a target motion parameter (S902); and, on the basis of the target motion parameter, determining a predicted value of the current block (S903). Thus, the accuracy of motion vectors in light field video encoding and decoding can be improved, thereby improving the efficiency of light field video compression.

Description

Coding and decoding method, code stream, encoder, decoder and storage medium Technical Field

[0001] The embodiments of the present application relate to the field of video coding and decoding technology, and in particular to a coding and decoding method, a bit stream, an encoder, a decoder, and a storage medium. Background Art

[0002] Light field video is a video format captured by a light field camera. Due to the unique microlens array design within the light field camera, the image of a light field video appears as a series of closely spaced macropixels (or microimages), as shown in Figure 1. The circular pattern shown in Figure 1 is the macropixel, and in this example, the macropixels are closely spaced in a hexagonal pattern.

[0003] Light field videos often involve enormous amounts of data, which makes transmission and compression difficult. Therefore, video encoding technologies specifically tailored to light field videos are particularly important. However, related technologies, such as decoder-side motion vector refinement (DMVR), fail to adequately consider the light field layout, resulting in low light field video compression efficiency.

[0004] Summary of the Invention

[0005] The embodiments of the present application provide a coding and decoding method, a bit stream, an encoder, a decoder, and a storage medium, which can improve the compression efficiency of light field video.

[0006] The technical solution of the embodiment of the present application can be implemented as follows:

[0007] In a first aspect, an embodiment of the present application provides a decoding method, applied to a decoder, the method comprising:

[0008] Determining an initial reference block of the current block, and determining a macropixel co-located block corresponding to the initial reference block;

[0009] Search the initial reference block and the macro pixel co-located block respectively to determine the target motion parameters;

[0010] According to the target motion parameters, the prediction value of the current block is determined.

[0011] In a second aspect, an embodiment of the present application provides an encoding method, applied to an encoder, the method comprising:

[0012] Determining an initial reference block of the current block, and determining a macropixel co-located block corresponding to the initial reference block;

[0013] Search the initial reference block and the macro pixel co-located block respectively to determine the target motion parameters;

[0014] According to the target motion parameters, the prediction value of the current block is determined.

[0015] In a third aspect, an embodiment of the present application provides a code stream, wherein the code stream is generated by bit encoding based on information to be encoded; wherein the information to be encoded includes at least one of the following: a prediction difference value of the current block and a prediction parameter of the current block.

[0016] In a fourth aspect, an embodiment of the present application provides an encoder, comprising a first determining unit, a first searching unit, and a first predicting unit, wherein:

[0017] A first determining unit is configured to determine an initial reference block of a current block; and determine a macropixel co-located block corresponding to the initial reference block;

[0018] A first search unit is configured to search based on the initial reference block and the macropixel co-located block respectively to determine the target motion parameter;

[0019] The first prediction unit is configured to determine a prediction value of a current block according to a target motion parameter.

[0020] In a fourth aspect, an embodiment of the present application provides an encoder, comprising a first memory and a first processor, wherein:

[0021] a first memory for storing a computer program capable of running on the first processor;

[0022] The first processor is configured to execute the method according to the second aspect when running the computer program.

[0023] In a fifth aspect, an embodiment of the present application provides a decoder, comprising a second determination unit, a second search unit, and a second prediction unit, wherein:

[0024] A second determining unit is configured to determine an initial reference block of the current block; and determine a macropixel co-located block corresponding to the initial reference block;

[0025] a second search unit configured to search based on the initial reference block and the macropixel co-located block respectively to determine the target motion parameter;

[0026] The second prediction unit is configured to determine a prediction value of the current block according to the target motion parameter.

[0027] In a sixth aspect, an embodiment of the present application provides a decoder, comprising a second memory and a second processor, wherein:

[0028] a second memory for storing a computer program capable of running on the second processor;

[0029] The second processor is configured to execute the method according to the first aspect when running the computer program.

[0030] In a seventh aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by at least one processor, it implements the method as described in the first aspect or the method as described in the second aspect.

[0031] The embodiments of the present application provide a coding and decoding method, bitstream, encoder, decoder, and storage medium. Whether at the encoding or decoding end, the initial reference block of the current block and the macropixel co-located block corresponding to the initial reference block are first determined; a search is then performed based on the initial reference block and the macropixel co-located block to determine target motion parameters; and a prediction value of the current block is then determined based on the target motion parameters. In this way, because there is spatial correlation between macropixels in a light field video image, when determining the target motion parameters, the accuracy of the motion vector in the light field video coding and decoding can be improved by expanding the search range to include adjacent macropixels, thereby saving bit rate, improving the compression efficiency of the light field video, and thus improving the coding and decoding performance of the light field video. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] FIG1 is a schematic diagram of a macro pixel of a light field image;

[0033] FIG2 is a schematic diagram of a block-based hybrid coding framework;

[0034] FIG3 is a schematic diagram of motion vectors of a DMVR technique;

[0035] FIG4 is a schematic diagram of a search process of a DMVR technology;

[0036] FIG5 is a block diagram of a light field video encoding and decoding process;

[0037] FIG6 is a schematic diagram of a codec network architecture provided in an embodiment of the present application;

[0038] FIG7 is a schematic diagram of the structure of an encoder provided in an embodiment of the present application;

[0039] FIG8 is a schematic diagram of the structure of a decoder provided in an embodiment of the present application;

[0040] FIG9 is a schematic diagram of a flow chart of a decoding method provided in an embodiment of the present application;

[0041] FIG10 is a schematic diagram of a positional relationship between a first reference block and a first macropixel co-located block provided in an embodiment of the present application;

[0042] FIG11 is a schematic diagram of a first flow chart of determining target motion parameters according to an embodiment of the present application;

[0043] FIG12 is a schematic diagram of a macropixel refinement search according to an embodiment of the present application;

[0044] FIG13 is a schematic diagram of a flow chart of an encoding method provided in an embodiment of the present application;

[0045] FIG14 is a second schematic diagram of a process for determining target motion parameters according to an embodiment of the present application;

[0046] FIG15 is a schematic diagram of a refinement algorithm of an improved DMVR technology provided in an embodiment of the present application;

[0047] FIG16 is a schematic diagram of the structure of an encoder provided in an embodiment of the present application;

[0048] FIG17 is a schematic diagram of a specific hardware structure of an encoder provided in an embodiment of the present application;

[0049] FIG18 is a schematic diagram of the structure of a decoder provided in an embodiment of the present application;

[0050] FIG19 is a schematic diagram of a specific hardware structure of a decoder provided in an embodiment of the present application;

[0051] FIG20 is a schematic diagram of the composition structure of a coding and decoding system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] In order to enable a more detailed understanding of the features and technical contents of the embodiments of the present application, the implementation of the embodiments of the present application is described in detail below with reference to the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present application.

[0053] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0054] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0055] It should also be pointed out that the terms "first\second\third" involved in the embodiments of the present application are only used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0056] Before further explaining the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained first. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations:

[0057] H.264 / Advanced Video Coding (AVC);

[0058] H.265 / High Efficiency Video Coding (HEVC);

[0059] H.266 / Versatile Video Coding (VVC);

[0060] Moving Picture Expert Group (MPEG);

[0061] Lenslet Video Coding (LVC);

[0062] Decoder side Motion Vector Refinement (DMVR);

[0063] Motion Vector (MV);

[0064] Fusion technology (Merge);

[0065] Sum of Absolute Difference (SAD);

[0066] Sum of Absolute Transformed Difference (SATD);

[0067] Mean Squared Error (MSE);

[0068] Sum of Squared Error (SSE);

[0069] Mean Absolute Error (MAE).

[0070] Currently, common video codec standards all adopt a block-based hybrid coding framework. Each image, sub-image, or frame in a video is divided into square maximum coding units (LCUs) or coding tree units (CTUs) of the same size (e.g., 256×256, 128×128, 64×64, etc.). Each LCU or CTU can be divided into rectangular CUs according to a rule. Coding units may also be divided into prediction units (PUs) and transform units (TUs). Specifically, as shown in Figure 2, the hybrid coding framework includes modules such as prediction, transform / quantization, entropy coding, inverse quantization / inverse transform, and in-loop filtering. The prediction module can include intra-frame prediction and inter-frame prediction, and inter-frame prediction can include motion estimation and motion compensation. Because adjacent pixels within a video image are highly correlated, intra-frame prediction is used in video coding and decoding to eliminate spatial redundancy between adjacent pixels. Furthermore, because adjacent images in a video image have strong similarities, inter-image prediction is used in video coding and decoding to eliminate temporal redundancy between adjacent images, thereby improving coding efficiency.

[0071] The basic process of a video codec is as follows: On the encoder side, an image is divided into blocks. Intra-frame prediction or inter-frame prediction is used on the current block to generate a prediction block for the current block. The prediction block is subtracted from the original block to obtain a residual block. The residual block is transformed and quantized to obtain a quantization coefficient matrix. This quantization coefficient matrix is ​​entropy-encoded and output to the bitstream. On the decoder side, intra-frame prediction or inter-frame prediction is used on the current block to generate a prediction block for the current block. The bitstream is then parsed to obtain a quantization coefficient matrix. This quantization coefficient matrix is ​​inversely quantized and inversely transformed to obtain a residual block. The prediction block and residual block are added together to obtain a reconstructed block. The reconstructed blocks form a reconstructed image, which is then subjected to image-based or block-based loop filtering to obtain a decoded image. The encoder side also performs similar operations as the decoder side to obtain a decoded image. The decoded image can serve as a reference image for inter-frame prediction of subsequent images. Block division information, prediction, transform, quantization, entropy coding, loop filtering, and other mode or parameter information determined by the encoder are output to the bitstream if necessary. The decoding end determines the same block division information as the encoding end by parsing the bit stream and analyzing the existing information, as well as the mode information or parameter information such as prediction, transformation, quantization, entropy coding, and loop filtering, thereby ensuring that the decoded image obtained by the encoding end is the same as the decoded image obtained by the decoding end. The decoded image obtained by the encoding end is also usually called a reconstructed image. The current block can be divided into prediction units during prediction, and the current block can be divided into transformation units during transformation. The division of prediction units and transformation units can be different. The above is the basic process of the video codec under the block-based hybrid coding framework. With the development of technology, some modules or steps of the framework or process may be optimized. The embodiment of the present application is applicable to the basic process of the video codec under the block-based hybrid coding framework, but is not limited to the framework and process.

[0072] In the embodiments of the present application, the current block (CB) can be the current coding unit, the current prediction unit, or the current transform unit. Due to the need for parallel processing, the image can be divided into slices, etc. Slices in the same image can be processed in parallel, that is, there is no data dependency between them. "Frame" is a commonly used term, and it can generally be understood that a frame is an image. In addition, the frame described in the embodiments of the present application can also be replaced by an image or a slice.

[0073] As can be understood, DMVR technology is a technology in existing video coding standards. Its function is to refine the motion vector MV obtained in the merge mode of inter-frame prediction at the decoding end. The general process is: at the decoding end, the motion vectors MV0 and MV1 obtained in the merge mode (from the first reference image in reference image list 0 and the second reference image in reference image list 1, respectively) are obtained and used as the initial motion vector. Then, within a certain search range and based on certain selection criteria, a search is performed near the initial motion vector to find the more effective motion vectors MV0' and MV1', as shown in Figure 3. Here, the black-filled block represents the current block. In the first reference image, the block filled with dots represents the block pointed to by the motion vector MV0, and the block filled with diagonal lines represents the block pointed to by the more effective motion vector MV0'. In the second reference image, the block filled with dots represents the block pointed to by the motion vector MV1, and the block filled with diagonal lines represents the block pointed to by the more effective motion vector MV1'.

[0074] It is worth noting that DMVR is used for bidirectional prediction, which is why there are two motion vectors MV0 and MV1. In addition, the refinement search process is symmetrical, that is, the offsets of MV0' and MV1' relative to the initial MV0 and MV1 are also symmetrically distributed.

[0075] Here, a more common search (or refinement) process is shown in Figure 4, which specifically involves traversing 5×5 positions (i.e., a 5×5 grid search) centered on the starting point (the position indicated by MV0(1)) in sequence, and selecting the best position (i.e., the position indicated by MV0(1)') based on certain selection criteria. The black-filled dots represent the initial position (i.e., the starting point), specifically, the position indicated by MV0(1); the white-filled dots represent the search position; and the diagonally filled dots represent the refined position (i.e., the search result), specifically, the position indicated by MV0(1)'.

[0076] It is also understood that the light field video of the embodiment of the present application can be a light field video captured by a light field camera, which is currently being studied by the MPEG LVC working group.

[0077] Firstly, light field video differs from conventional camera imaging models in that a light field camera adds a microlens array in front of the imaging plane. This allows light from the same point on the object plane to be captured simultaneously by multiple microlenses, effectively capturing the same point from multiple angles simultaneously.

[0078] On the other hand, due to their unique imaging model, the visual quality of light field images differs significantly from that of traditional images. This results in compression methods used for conventional images or videos being ineffective when processing light field images or videos. The LVC group was established to address this issue, researching compression methods more suitable for light field videos.

[0079] On the other hand, the block diagram of the light field video encoding and decoding process is shown in Figure 5, which specifically includes: capturing / shooting light field video → converting data format → codec processing → converting data format → displaying light field video. Among them, the first data format conversion is used to convert the captured light field video into a data format that is compatible with codec processing, and the second data format conversion is used to convert the decoded data format into a data format that is compatible with back-end display. It should be noted that based on existing video codec tools (AVC, HEVC or VVC, etc.), the embodiments of this application mainly optimize the inter-frame prediction part of the codec (Encoder & Decoder).

[0080] In other words, light field video can be a video format captured by a light field camera. Due to the unique microlens array design inside the light field camera, the image of the light field video will present a series of closely arranged macro pixels (or micro images), as shown in Figure 1. In this example, the macro pixels are closely arranged in a hexagonal template.

[0081] Light field videos often involve enormous amounts of data, which makes transmission and compression challenging. Therefore, video encoding technologies tailored to light field videos are particularly important. The technical solution described here further refines the motion vectors obtained in merge mode, resulting in significant performance gains for general video. However, DMVR technology does not consider the specifics of light field layout, so there is still room for improvement in light field video compression.

[0082] Based on this, an embodiment of the present application provides a coding and decoding method. Whether at the encoding or decoding end, the method first determines the initial reference block of the current block and the macropixel co-located block corresponding to the initial reference block; then, searches are performed based on the initial reference block and the macropixel co-located block to determine the target motion parameters; and then, based on the target motion parameters, the predicted value of the current block is determined. In this way, due to the spatial correlation between macropixels in a light field video image, when determining the target motion parameters, the accuracy of the motion vector in the light field video coding and decoding can be improved by expanding the search range to include adjacent macropixels, thereby saving bit rate, improving the compression efficiency of the light field video, and further improving the coding and decoding performance of the light field video.

[0083] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0084] FIG6 is a schematic diagram of a codec network architecture provided in an embodiment of the present application. As shown in FIG6 , the network architecture includes one or more electronic devices 13 to 1N and a communication network 01, wherein electronic devices 13 to 1N can perform video interaction via the communication network 01. During implementation, the electronic devices can be various types of devices with codec capabilities. For example, the electronic devices can include mobile phones, tablet computers, personal computers, personal digital assistants, navigators, digital phones, video phones, televisions, sensor devices, servers, etc., and the embodiments of the present application are not limited thereto.

[0085] In an embodiment of the present application, a network architecture of a codec system including a decoding method and an encoding method is provided. The decoder or encoder in the embodiment of the present application can be the aforementioned electronic device. In other words, the electronic device in the embodiment of the present application has video encoding and decoding capabilities, and generally includes a video encoder (i.e., encoder) and a video decoder (i.e., decoder).

[0086] FIG7 is a schematic diagram of the composition structure of an encoder provided in an embodiment of the present application. As shown in FIG7 , the encoder 100 may include: a segmentation unit 101, a prediction unit 102, a first adder 107, a transform unit 108, a quantization unit 109, an inverse quantization unit 110, an inverse transform unit 111, a second adder 112, a filtering unit 113, a decoded picture buffer (DPB) unit 114, and an entropy coding unit 115. Here, the input of the encoder 100 may be a video consisting of a series of pictures or a static picture, and the output of the encoder 100 may be a bitstream (also referred to as a "codestream") used to represent a compressed version of the input video.

[0087] Among them, the segmentation unit 101 segments the picture in the input video into one or more Coding Tree Units (CTUs). The segmentation unit 101 divides the picture into multiple tiles (or tiles), and can further divide a tile into one or more bricks. Here, a tile or a brick may include one or more complete and / or partial CTUs. In addition, the segmentation unit 101 can form one or more slices, where a slice may include one or more tiles arranged in a grid order in the picture, or one or more tiles covering a rectangular area in the picture. The segmentation unit 101 can also form one or more sub-pictures, where a sub-picture may include one or more slices, tiles or bricks.

[0088] During the encoding process of encoder 100, segmentation unit 101 transmits the CTU to prediction unit 102. Generally, prediction unit 102 may be composed of block segmentation unit 103, motion estimation (ME) unit 104, motion compensation (MC) unit 105, and intra prediction unit 106. Specifically, block segmentation unit 103 iteratively uses quadtree segmentation, binary tree segmentation, and ternary tree segmentation to further divide the input CTU into smaller coding units (CUs). Prediction unit 102 may use ME unit 104 and MC unit 105 to obtain inter-prediction blocks for the CU. Intra prediction unit 106 may use various intra prediction modes, including MIP mode, to obtain intra-prediction blocks for the CU. In an example, a rate-distortion optimized motion estimation method may be used by ME unit 104 and MC unit 105 to obtain inter-prediction blocks, and a rate-distortion optimized mode determination method may be used by intra prediction unit 106 to obtain intra-prediction blocks. The prediction unit 102 outputs the prediction block of the CU, and the first adder 107 calculates the difference between the CU in the output of the segmentation unit 101 and the prediction block of the CU, i.e., the residual CU. The transform unit 108 reads the residual CU and performs one or more transform operations on the residual CU to obtain coefficients. The quantization unit 109 quantizes the coefficients and outputs the quantized coefficients (i.e., levels). The inverse quantization unit 110 performs a scaling operation on the quantized coefficients to output reconstructed coefficients. The inverse transform unit 111 performs one or more inverse transforms corresponding to the transform in the transform unit 108 and outputs the reconstructed residual. The second adder 112 calculates the reconstructed CU by adding the reconstructed residual and the prediction block of the CU from the prediction unit 102. The second adder 112 also sends its output to the prediction unit 102 for use as an intra-frame prediction reference. After all CUs in the picture or sub-picture are reconstructed, the filtering unit 113 performs loop filtering on the reconstructed picture or sub-picture. Here, the filtering unit 113 includes one or more filters, such as a deblocking filter, a sample adaptive offset (SAO) filter, an adaptive loop filter (ALF), a luminance mapping and chroma scaling (LMCS) filter, and a neural network-based filter. Alternatively, when the filtering unit 113 determines that the CU is not used as a reference for encoding other CUs, the filtering unit 113 performs loop filtering on one or more target pixels in the CU. The output of the filtering unit 113 is a decoded picture or sub-picture, which is cached to the DPB unit 114. The DPB unit 114 outputs the decoded picture or sub-picture according to the timing and control information.Here, the picture stored in the DPB unit 114 can also be used as a reference for the prediction unit 102 to perform inter-frame prediction or intra-frame prediction. Finally, the entropy coding unit 115 converts the parameters necessary for decoding the picture from the encoder 100 (such as control parameters and supplementary information, etc.) into binary form, and writes such binary form into the code stream according to the syntax structure of each data unit, that is, the encoder 100 finally outputs the code stream.

[0089] Furthermore, encoder 100 may include a first processor and a first memory storing a computer program. When the first processor reads and executes the computer program, encoder 100 reads the input video and generates a corresponding bitstream. Alternatively, encoder 100 may be a computing device comprising one or more chips. These units, implemented as integrated circuits on the chip, have similar connectivity and data exchange functions as the corresponding units in FIG7 .

[0090] Figure 8 is a schematic diagram of the structure of a decoder provided in an embodiment of the present application. As shown in Figure 8, the decoder 200 may include: a parsing unit 201, a prediction unit 202, an inverse quantization unit 205, an inverse transform unit 206, an adder 207, a filtering unit 208, and a decoded image buffer unit 209. Here, the input of the decoder 200 is a bitstream representing a compressed version of a video or a still image, and the output of the decoder 200 may be a decoded video consisting of a series of images or a decoded still image.

[0091] The input codestream to decoder 200 may be the codestream generated by encoder 100. Parsing unit 201 parses the input codestream and obtains syntax element values ​​from the input codestream. Parsing unit 201 converts the binary representation of the syntax elements into digital values ​​and sends the digital values ​​to units within decoder 200 to obtain one or more decoded pictures. Parsing unit 201 may also parse one or more syntax elements from the input codestream to display decoded pictures.

[0092] During the decoding process in decoder 200, parsing unit 201 transmits the values ​​of syntax elements and one or more variables set or determined based on the values ​​of the syntax elements, used to obtain one or more decoded pictures, to units within decoder 200. Prediction unit 202 determines a prediction block for the current decoding block (e.g., a CU). Prediction unit 202 may include motion compensation unit 203 and intra prediction unit 204. Specifically, when an inter decoding mode is indicated for decoding the current decoding block, prediction unit 202 passes relevant parameters from parsing unit 201 to motion compensation unit 203 to obtain an inter prediction block. When an intra prediction mode (including a MIP mode indicated based on a MIP mode index value) is indicated for decoding the current decoding block, prediction unit 202 passes relevant parameters from parsing unit 201 to intra prediction unit 204 to obtain an intra prediction block. Dequantization unit 205 has the same functionality as dequantization unit 110 in encoder 100. Dequantization unit 205 performs a scaling operation on the quantization coefficients (i.e., levels) from parsing unit 201 to obtain reconstructed coefficients. The inverse transform unit 206 has the same function as the inverse transform unit 111 in the encoder 100. The inverse transform unit 206 performs one or more transform operations (i.e., the inverse of the one or more transform operations performed by the inverse transform unit 111 in the encoder 100) to obtain a reconstructed residual. The adder 207 performs an addition operation on its input (the prediction block from the prediction unit 202 and the reconstructed residual from the inverse transform unit 206) to obtain a reconstructed block of the current decoded block. The reconstructed block is also sent to the prediction unit 202 to be used as a reference for other blocks encoded in the intra prediction mode.

[0093] After all CUs in the picture or sub-picture are reconstructed, the filtering unit 208 performs loop filtering on the reconstructed picture or sub-picture. The filtering unit 208 includes one or more filters, such as a deblocking filter, a sample adaptive offset filter, an adaptive loop filter, a luminance mapping and chroma scaling filter, and a neural network-based filter. Alternatively, when the filtering unit 208 determines that the reconstructed block is not used as a reference for decoding other blocks, the filtering unit 208 performs loop filtering on one or more target pixels in the reconstructed block. Here, the output of the filtering unit 208 is a decoded picture or sub-picture, which is cached to the DPB unit 209. The DPB unit 209 outputs the decoded picture or sub-picture based on timing and control information. The picture stored in the DPB unit 209 can also be used as a reference for performing inter-frame prediction or intra-frame prediction by the prediction unit 202.

[0094] Furthermore, the decoder 200 can be a second memory having a second processor and a computer program. When the first processor reads and runs the computer program, the decoder 200 reads the input bit stream and generates the corresponding decoded video. In addition, the decoder 200 can also be a computing device having one or more chips. These units implemented as integrated circuits on the chip have similar connection and data exchange functions as the corresponding units in Figure 8.

[0095] It should also be noted that, when the embodiment of the present application is applied to the encoder 100, the "current block" specifically refers to the block currently to be encoded in the light field video image (which may also be simply referred to as the "encoding block"); when the embodiment of the present application is applied to the decoder 200, the "current block" specifically refers to the block currently to be decoded in the light field video image (which may also be simply referred to as the "decoding block").

[0096] In one embodiment of the present application, FIG9 is a flowchart of a decoding method provided by the embodiment of the present application. As shown in FIG9 , the method may include:

[0097] S901: Determine an initial reference block of a current block, and determine a macropixel co-located block corresponding to the initial reference block.

[0098] It should be noted that in the embodiments of this application, the method is applied to a decoder. Specifically, based on the structure of decoder 200 shown in Figure 8, the decoding method of the embodiments of this application can be applied to the inter-frame prediction part. Here, it is mainly an optimization solution proposed for the DMVR technology of light field video coding to improve compression efficiency.

[0099] It should also be noted that, in the embodiment of the present application, determining the initial reference block of the current block may include: decoding the code stream to determine the prediction parameters of the current block; and determining the initial reference block of the current block based on the prediction parameters.

[0100] In one possible implementation, the prediction parameters may include: a motion vector, a reference image index, and an inter-frame prediction identification parameter. The reference image index indicates the reference image used by the current block; the inter-frame prediction identification parameter indicates the prediction method used to decode the current block, and the prediction methods may include: L0 prediction, L1 prediction, and bidirectional prediction.

[0101] It can be understood that in the embodiment of the present application, L0 represents the reference image in the reference image list 0, which can also be called the forward reference image, that is, the first reference image; L1 represents the reference image in the reference image list 1, which can also be called the backward reference image, that is, the second reference image.

[0102] For L0 prediction, the flag value for using reference picture list 0 for inter-frame prediction is equal to 1; meanwhile, the flag value for using reference picture list 1 for inter-frame prediction is equal to 0. For example, forward prediction.

[0103] For L1 prediction, the flag value for using reference picture list 0 for inter-frame prediction is equal to 0; meanwhile, the flag value for using reference picture list 1 for inter-frame prediction is equal to 1. For example, backward prediction.

[0104] For bidirectional prediction, the flag value for using reference picture list 0 for inter-frame prediction is equal to 1; at the same time, the flag value for using reference picture list 1 for inter-frame prediction is equal to 1. For example, forward prediction and backward prediction.

[0105] In some embodiments, when the prediction parameter includes an inter-frame prediction identification parameter, the method further includes: determining that the inter-frame prediction identification parameter may indicate that the prediction method used for decoding the current block is bidirectional prediction.

[0106] For example, if the current block uses bidirectional prediction, the inter-frame prediction flag parameter may indicate that the prediction method used for decoding the current block is bidirectional prediction. Otherwise, if the current block uses L0 prediction or L1 prediction, the inter-frame prediction flag parameter may indicate that the prediction method used for decoding the current block is unidirectional prediction. The prediction method used by the current block is set according to actual conditions and is not specifically limited in the embodiments of the present application.

[0107] In some embodiments, when the inter-frame prediction identification parameter indicates that the prediction method used for decoding the current block is bidirectional prediction, the reference image index may include a first reference image index and a second reference image index, and the motion vector may include a first initial motion vector and a second initial motion vector. Accordingly, determining the initial reference block of the current block based on the prediction parameter may include: determining the first reference block of the current block based on the first reference image indicated by the first reference image index and the first initial motion vector in the motion vector; determining the second reference block of the current block based on the second reference image indicated by the second reference image index and the second initial motion vector in the motion vector; and using the first reference block and the second reference block as the initial reference blocks of the current block.

[0108] That is to say, in an embodiment of the present application, based on information such as the motion vector, reference image index, and inter-frame prediction identification parameters obtained by decoding, if the prediction method used for decoding the current block is bidirectional prediction, then the first reference block of the current block, such as the L0 reference block, and the second reference block of the current block, such as the L1 reference block, can be determined.

[0109] In which, when the motion vector includes a first initial motion vector MV0 (for example, a forward motion vector), the first reference block (or called a "forward matching block") can be determined from the first reference image based on the first initial motion vector MV0; when the motion vector includes a second initial motion vector MV1 (for example, a backward motion vector), the second reference block (or called a "backward matching block") can be determined from the second reference image based on the second initial motion vector MV1.

[0110] In another possible implementation, the prediction parameters may include: a motion vector difference, a reference image index, and an inter-frame prediction identification parameter. The reference image index indicates the reference image used by the current block; the inter-frame prediction identification parameter indicates the prediction method used for decoding the current block, and the prediction methods may include: L0 prediction, L1 prediction, and bidirectional prediction.

[0111] In some embodiments, when the inter-frame prediction identification parameter indicates that the prediction method used to decode the current block is bidirectional prediction, the reference image index may include a first reference image index and a second reference image index, and the motion vector difference includes a first motion vector difference and a second motion vector difference. Accordingly, determining the initial reference block of the current block based on the prediction parameters may include: determining a first initial motion vector prediction value of the current block in the first reference image and a second initial motion vector prediction value of the current block in the second reference image; determining a first initial motion vector based on the first motion vector difference and the first initial motion vector prediction value in the motion vector difference, and determining the first reference block of the current block based on the first reference image and the first initial motion vector indicated by the first reference image index; determining a second initial motion vector based on the second motion vector difference and the second initial motion vector prediction value in the motion vector difference, and determining the second reference block of the current block based on the second reference image and the second initial motion vector indicated by the second reference image index; and using the first reference block and the second reference block as the initial reference blocks of the current block.

[0112] It's important to note that for motion information, complete motion vectors can be written directly into the bitstream. However, to save bitrate, motion vector differences can also be written into the bitstream. This allows the motion vector to be represented by the motion vector prediction (MVP) plus the motion vector difference (MVD), i.e., MV = MVP + MVD. The more accurate the MVP, the smaller the MVD, which reduces the overhead in the bitstream and saves bitrate.

[0113] That is to say, in an embodiment of the present application, based on information such as the motion vector difference, reference image index and inter-frame prediction identification parameters obtained by decoding, if the prediction method used for decoding the current block is bidirectional prediction, then the first initial motion vector MV0 can be obtained by adding the first motion vector difference MVD0 and the determined first initial motion vector prediction value MVP0, and then the first reference block of the current block, such as the L0 reference block, is determined from the first reference image; and, the second initial motion vector MV1 can be obtained by adding the second motion vector difference MVD1 and the determined second initial motion vector prediction value MVP1, and then the second reference block of the current block, such as the L1 reference block, is determined from the second reference image.

[0114] In another possible implementation, for merge mode, the prediction parameters may include: a motion vector index, a reference image index, and an inter-frame prediction flag parameter. The reference image index indicates the reference image used by the current block; the inter-frame prediction flag parameter indicates the prediction method used for decoding the current block, and the prediction methods may include: L0 prediction, L1 prediction, and bidirectional prediction.

[0115] In some embodiments, when the inter-frame prediction identification parameter indicates that the prediction method used for decoding the current block is bidirectional prediction, the reference image index may include a first reference image index and a second reference image index, and the motion vector index may include a first motion vector index and a second motion vector index. Accordingly, determining the initial reference block of the current block based on the prediction parameter may include: constructing a motion vector candidate list for the current block; determining a first initial motion vector based on the motion vector candidate list and the first motion vector index in the motion vector index, and determining the first reference block of the current block based on the first reference image and the first initial motion vector indicated by the first reference image index; determining a second initial motion vector based on the motion vector candidate list and the second motion vector index in the motion vector index, and determining the second reference block of the current block based on the second reference image and the second initial motion vector indicated by the second reference image index; and using the first reference block and the second reference block as the initial reference blocks of the current block.

[0116] It should be noted that, in the embodiment of the present application, the first motion vector index is used to indicate the index number of the first initial motion vector in the motion vector candidate list, and the second motion vector index is used to indicate the index number of the second initial motion vector in the motion vector candidate list.

[0117] It should also be noted that in the embodiments of the present application, a motion vector candidate list can first be constructed. Specifically, this can be constructed using information related to the decoded portion of the current block. Spatial motion information and / or temporal motion information can be used in this construction process. Furthermore, the number of candidate motion vectors in the motion vector candidate list can be two, three, four, or more, without specific limitation.

[0118] It should also be noted that, in the embodiment of the present application, the motion vector candidate list constructed by the encoding end needs to be consistent with the motion vector candidate list constructed by the decoding end, so that the decoding end can determine the first initial motion vector MV0 and the second initial motion vector MV1 through the motion vector index transmitted in the code stream.

[0119] In this way, for the decoding end, the first initial motion vector MV0 and the second initial motion vector MV1 (from the first reference image L0 and the second reference image L1, respectively) can be obtained by decoding the code stream, and then the first reference block can be determined based on the first initial motion vector MV0 and the first reference image L0, and the second reference block can be determined based on the second initial motion vector MV1 and the second reference image L1.

[0120] It's also understood that in a light field video, the corresponding light field video image will appear as a series of closely spaced macropixels. To improve encoding and decoding efficiency, the macropixel co-located blocks corresponding to each initial reference block can be further determined. It's important to note that the relative position of the initial reference block within its corresponding macropixel is identical to the relative position of the macropixel co-located blocks within its corresponding macropixel.

[0121] In some embodiments, determining the macropixel co-located block corresponding to the initial reference block may include: determining at least one first macropixel co-located block of the first reference block in the first reference image based on the first reference block; and determining at least one second macropixel co-located block of the second reference block in the second reference image based on the second reference block.

[0122] In a specific embodiment, when determining at least one first macropixel co-located block of the first reference block in the first reference image based on the first reference block, it can include: determining a first current macropixel and at least one first candidate macropixel in the first reference image, the first current macropixel including at least part of the first reference block; and determining the respective macropixel co-located blocks in the at least one first candidate macropixel based on the positional relationship of the first reference block in the first current macropixel to obtain at least one first macropixel co-located block.

[0123] It should be noted that, in the embodiment of the present application, the number of at least one first candidate macropixel may be 1, 2, 3, 4, or more. In addition, at least one first candidate macropixel may be adjacent to the first current macropixel, for example, the at least one first candidate macropixel is distributed around the first current macropixel; however, the at least one first candidate macropixel may not be adjacent to the first current macropixel, and this is not specifically limited here. It should also be noted that, in the embodiment of the present application, the macropixels may be arranged in a hexagonal shape, or according to other shape templates, such as a square, rectangle, etc., and this is not specifically limited here.

[0124] For example, as shown in FIG10 , when macropixels are arranged in a hexagon, the number of at least one first candidate macropixel can be set to 6, and these 6 first candidate macropixels are adjacent to the first current macropixel and distributed around the first current macropixel. The hexagon in the middle is the first current macropixel, and the 6 adjacent hexagons are all first candidate macropixels.

[0125] It should also be noted that, in an embodiment of the present application, the first current macropixel may include a portion or all of the first reference block, and each first candidate macropixel may include a portion or all of the corresponding first macropixel co-located block. Still taking Figure 10 as an example, the black-filled dot in the first current macropixel represents the upper left corner position of the first reference block, that is, the black-filled dot can indicate the position of the first reference block; in these six first candidate macropixels, the grid-filled dots represent the upper left corner position of the first macropixel co-located block, that is, the grid-filled dots can indicate the position of the first macropixel co-located block. In other words, in the first reference image, the relative position of the first reference block in the first current macropixel is the same as the relative position of the first macropixel co-located block in the first candidate macropixel.

[0126] In another specific embodiment, when determining at least one second macropixel co-located block of the second reference block in the second reference image based on the second reference block, it can include: determining a second current macropixel and at least one second candidate macropixel in the second reference image, the second current macropixel including at least part of the second reference block; and determining the respective macropixel co-located blocks in the at least one second candidate macropixel based on the positional relationship of the second reference block in the second current macropixel to obtain at least one second macropixel co-located block.

[0127] It should be noted that, in the embodiment of the present application, the number of the at least one second candidate macropixel may be 1, 2, 3, 4, or more. In addition, the at least one second candidate macropixel may be adjacent to the second current macropixel, for example, the at least one second candidate macropixel is distributed around the second current macropixel; however, the at least one second candidate macropixel may not be adjacent to the second current macropixel, which is not specifically limited here.

[0128] It should also be noted that, in this embodiment of the present application, the second current macropixel may include a portion or all of the second reference block, and each second candidate macropixel may include a portion or all of the corresponding second macropixel co-located block. Furthermore, in the second reference image, the relative position of the second reference block in the second current macropixel is the same as the relative position of the second macropixel co-located block in the second candidate macropixel. It should be noted that the method for determining the second macropixel co-located block in the second reference image is similar to the method for determining the first macropixel co-located block in the first reference image. See FIG. 10 for details, and no further details will be given here.

[0129] In this way, after determining the initial reference block and macropixel co-located block (including the first reference block and the first macropixel co-located block located in the first reference image and the second reference block and the second macropixel co-located block located in the second reference image), a refined search can be performed to determine better target motion parameters.

[0130] S902: Searching the initial reference block and the macropixel co-located block respectively to determine target motion parameters.

[0131] In the embodiment of the present application, FIG11 is a flow chart of a method for determining target motion parameters provided by the embodiment of the present application. As shown in FIG11 , the method may include:

[0132] S1101: Searching the preset areas of the corresponding macropixels according to the initial reference block and the macropixel co-located block to determine multiple groups of candidate motion parameters.

[0133] S1102: Calculate the cost of each of the multiple sets of candidate motion parameters to obtain multiple sets of cost results.

[0134] S1103: Determine a minimum cost result from the multiple groups of cost results, and determine a group of candidate motion parameters corresponding to the minimum cost result as target motion parameters.

[0135] It should be noted that, in the embodiment of the present application, each set of candidate motion parameters may include a first candidate motion vector and a second candidate motion vector; accordingly, the ultimately determined target motion parameters may include a first target motion vector and a second target motion vector. The first candidate motion vector and the first target motion vector are determined for the first reference image, while the second candidate motion vector and the second target motion vector are determined for the second reference image.

[0136] It should also be noted that in the embodiment of the present application, the preset area can be a 5×5 grid area, but can also be a 3×3 grid area, a 4×4 grid area, etc. In other words, there is no limitation on the number of search area locations and the selection of specific locations, and there is no specific limitation on the size of the grid area.

[0137] In some embodiments, determining multiple groups of candidate motion parameters based on searching the preset areas of the macropixels to which the initial reference block and the macropixel co-located block belong respectively may include: searching in a first preset area of ​​the macropixels to which the initial reference block belongs, with the initial reference block as the center, to determine at least one group of first candidate motion parameters; searching in a second preset area of ​​the macropixels to which the macropixel co-located block belongs, with the macropixel co-located block as the center, to determine at least one group of second candidate motion parameters; and determining the at least one group of first candidate motion parameters and the at least one group of second candidate motion parameters as multiple groups of candidate motion parameters.

[0138] Here, the first preset area and the second preset area can be the same or different. For example, the first preset area can be a 5×5 grid area, and the second preset area can also be a 5×5 grid area; then, with the initial reference block and the macropixel co-located block as the center, a 5×5 grid search is performed respectively, and then the position indicated by the optimal motion parameter (i.e., the target motion parameter) is determined, which is the result of the refined search.

[0139] For example, assuming that the number of macropixel co-located blocks is 6, then for at least one group of first candidate motion parameters, there can be specifically 5×5 groups; for at least one group of second candidate motion parameters, there can be specifically 6×5×5 groups; that is, for multiple groups of candidate motion parameters, there can be a total of (1+6)×5×5 groups of candidate motion parameters, and then the best group is selected from so many groups of candidate motion parameters, that is, the group of candidate motion parameters corresponding to the minimum cost result is determined as the target motion parameters.

[0140] In a specific embodiment, searching in a first preset area of ​​the macropixels to which the initial reference block belongs with an initial reference block as the center to determine at least one set of first candidate motion parameters may include: searching in a first preset area of ​​the first current macropixel with the first reference block as the center to determine at least one first candidate motion vector; and determining at least one second candidate motion vector corresponding to the second reference block based on the at least one first candidate motion vector and the first preset area of ​​the second current macropixel with the second reference block as the center; and determining at least one set of first candidate motion parameters based on the at least one first candidate motion vector and the at least one second candidate motion vector.

[0141] In the embodiment of the present application, the first offset between the first initial motion vector and the first candidate motion vector and the second offset between the second initial motion vector and the second candidate motion vector are symmetrical. In other words, the refinement search process is symmetrical, i.e., the first offset between the position corresponding to the first reference block and the position indicated by the first candidate motion vector and the second offset between the position corresponding to the second reference block and the position indicated by the second candidate motion vector are symmetrically distributed.

[0142] It should also be noted that, in the embodiment of the present application, for a set of candidate motion parameters, the first candidate motion vector and the second candidate motion vector appear in pairs. For example, in the first reference image, the first candidate motion vector MV0' can be a first initial motion vector MV0 shifted by MV diff Then in the second reference image, the second candidate motion vector MV1' can be a moving -MV based on the second initial motion vector MV1 diff If the first preset area is a 5×5 grid area, then 5×5 groups of first candidate motion parameters can be searched out.

[0143] In another specific embodiment, determining at least one set of second candidate motion parameters by searching a second preset area of ​​macropixels to which the macropixel co-located block belongs, with the macropixel co-located block as the center, may include: determining at least one first candidate motion vector by searching a second preset area of ​​first candidate macropixels to which the first candidate macropixels belong, with the first macropixel co-located block as the center; and determining at least one second candidate motion vector corresponding to the second macropixel co-located block based on the at least one first candidate motion vector and the second preset area of ​​second candidate macropixels to which the second macropixel co-located block is the center; and determining at least one set of second candidate motion parameters based on the at least one first candidate motion vector and the at least one second candidate motion vector.

[0144] It should be noted that in the embodiment of the present application, the refined search process here is symmetrical, that is, the first offset between the corresponding position of the first macropixel co-located block and the position indicated by the first candidate motion vector and the second offset between the corresponding position of the second macropixel co-located block and the position indicated by the second candidate motion vector are symmetrically distributed.

[0145] It should also be noted that in the embodiment of the present application, the search process is similar to the search process centered on the initial reference block. If the second preset area is a 5×5 grid area and the number of macropixel co-located blocks is 6, then 6×5×5 groups of second candidate motion parameters can be searched here.

[0146] In this way, during the refinement search, the scope of the refinement operation can be expanded to include adjacent macropixels. After searching for at least one set of first candidate motion parameters and at least one set of second candidate motion parameters, multiple sets of candidate motion parameters can be obtained. Costs are then calculated for each of these multiple sets of candidate motion parameters to obtain multiple sets of cost results.

[0147] In one possible implementation, cost calculation is performed on multiple groups of candidate motion parameters respectively to obtain multiple groups of cost results, including: determining a first candidate block corresponding to a first candidate motion vector and a second candidate block corresponding to a second candidate motion vector based on each group of candidate motion parameters; calculating the cost results of the first candidate block and the second candidate block according to preset criteria to obtain a cost result corresponding to each group of candidate motion parameters.

[0148] In an embodiment of the present application, the preset criteria may include at least one of the following: the sum of absolute errors SAD, the sum of absolute transformed errors SATD, the mean square error MSE, the sum of squared errors SSE, and the mean absolute error MAE.

[0149] Exemplarily, the preset criterion takes the sum of absolute differences SAD as an example. First, the first candidate block pointed to by the first candidate motion vector MV0' and the second candidate block pointed to by the second candidate motion vector MV1' are determined. Then, the SAD between the first candidate block and the second candidate block is calculated. Finally, a set of MV0' and MV1' with the smallest SAD value is selected as the target motion vector, that is, the final refined search result.

[0150] In another possible implementation, cost calculation is performed on multiple groups of candidate motion parameters respectively to obtain multiple groups of cost results, which may include: determining the first candidate block corresponding to the first candidate motion vector and the second candidate block corresponding to the second candidate motion vector based on each group of candidate motion parameters, and performing weighted calculation based on the first candidate block and the second candidate block to determine the candidate prediction block; performing weighted calculation based on the first reference block and the second reference block to determine the initial prediction block; calculating the cost results of the candidate prediction blocks and the initial prediction blocks according to preset criteria to obtain the cost results corresponding to each group of candidate motion parameters.

[0151] In an embodiment of the present application, the preset criteria may include at least one of the following: the sum of absolute errors SAD, the sum of absolute transformed errors SATD, the mean square error MSE, the sum of squared errors SSE, and the mean absolute error MAE.

[0152] Exemplarily, the preset criterion takes the sum of absolute differences SAD as an example, first determining the first candidate block pointed to by the first candidate motion vector MV0' and the second candidate block pointed to by the second candidate motion vector MV1', then performing a weighted combination of the first candidate block and the second candidate block to obtain a candidate prediction block, and then performing a weighted combination based on the first reference block pointed to by the first initial motion vector MV0 and the second reference block pointed to by the second initial motion vector MV1 to obtain the initial prediction block, if the SAD value between the initial prediction block and the candidate prediction block is the smallest, then the MV0' and MV1' corresponding to the candidate prediction block can be used as the target motion vector, that is, the final refined search result.

[0153] It should also be noted that the two possible implementations described above are merely two common examples, and other possible implementations are also possible. Furthermore, the selection criteria for the refined search may be other than the sum of absolute deviations (SAD), the sum of absolute transformed deviations (SATD), the mean square error (MSE), the sum of squared errors (SSE), and the mean absolute error (MAE), and are not specifically limited here.

[0154] For example, as shown in FIG12 , macropixels are arranged in a hexagonal shape, and the search range in each macropixel can be a 5×5 grid area. Taking the first reference image as an example, the middle macropixel is the first current macropixel, and the six adjacent macropixels are the first candidate macropixels. Among them, the dots filled with black can indicate the position of the first reference block (i.e., the initial position), the dots filled with grid can indicate the position of the first macropixel co-located block (i.e., the macropixel co-located position), and the dots filled with white represent the search position. In this way, the position of the first reference block and the position of the first macropixel co-located block are respectively centered, and a 5×5 grid search is performed, and then the best refined position is selected from the (1+6)×5×5 search positions, that is, the position indicated by the target motion vector MV0(1)'. In FIG12 , the dots filled with slashes can indicate the position indicated by the target motion vector MV0(1)', which is also the final search result.

[0155] S903: Determine a prediction value of the current block according to the target motion parameter.

[0156] It should be noted that after obtaining the target motion parameters, since the prediction method used for decoding the current block is bidirectional prediction, the target motion parameters may include a first target motion vector and a second target motion vector. In some embodiments, determining a prediction value for the current block based on the target motion parameters may include: determining a first prediction block for the current block based on the first target motion vector; determining a second prediction block for the current block based on the second target motion vector; and determining a prediction value for the current block based on the first prediction block and the second prediction block.

[0157] In an embodiment of the present application, the first prediction block of the current block is determined according to the first target motion vector, and the block pointed to by the first target motion vector in the first reference image may be used as the first prediction block; similarly, the second prediction block of the current block is determined according to the second target motion vector, and the block pointed to by the second target motion vector in the second reference image may be used as the second prediction block.

[0158] In some embodiments, determining the prediction value of the current block according to the first prediction block and the second prediction block may include: performing a weighted calculation according to the first prediction block and the second prediction block to determine the prediction value of the current block.

[0159] For example, assuming that the first prediction block can be represented by Pred1, the second prediction block can be represented by Pred2, the weight corresponding to the first prediction block is set to w1, and the weight corresponding to the second prediction block is set to w2; then the prediction value Pred[x][y] of the current block is calculated as follows:

[0160] Pred[x][y]=w1×Pred1+w2×Pred2, wherein the sizes of w1 and w2 can be preset constant values, or can be determined according to the size parameters of the current block, or can even be determined according to other methods, which are not specifically limited here.

[0161] In some embodiments, the method further includes: decoding the code stream to determine a prediction difference value of the current block; and determining a reconstructed value of the current block according to the prediction difference value of the current block and the prediction value of the current block.

[0162] It should be noted that in an embodiment of the present application, determining the reconstruction value of the current block based on the prediction difference value of the current block and the prediction value of the current block may include: performing an addition operation on the prediction difference value of the current block and the prediction value of the current block to determine the reconstruction value of the current block.

[0163] That is, in the embodiment of the present application, after obtaining the prediction value of the current block, the prediction difference value (ie, prediction residual) obtained by decoding can be combined to restore the reconstructed value of the current block.

[0164] This embodiment provides a decoding method that determines an initial reference block for a current block and a macropixel co-located block corresponding to the initial reference block; performs searches based on the initial reference block and the macropixel co-located block to determine target motion parameters; and determines a prediction value for the current block based on the target motion parameters. Because macropixels in a light field video image have spatial correlation, when determining the target motion parameters, the accuracy of motion vectors in light field video encoding and decoding can be improved by expanding the search range to include adjacent macropixels, thereby saving bitrate, improving compression efficiency, and ultimately enhancing light field video encoding and decoding performance.

[0165] In another embodiment of the present application, FIG13 is a flow chart of an encoding method provided in an embodiment of the present application. As shown in FIG13 , the method may include:

[0166] S1301: Determine an initial reference block of a current block, and determine a macropixel co-located block corresponding to the initial reference block.

[0167] It should be noted that in the embodiments of this application, the method is applied to an encoder. Specifically, based on the structure of encoder 100 shown in Figure 7, the encoding method of the embodiments of this application can be applied to the inter-frame prediction part. Here, it is mainly an optimization scheme proposed for the DMVR technology of light field video coding to improve compression efficiency.

[0168] It should also be noted that in the embodiment of the present application, to determine the initial reference block for the current block, the encoder can perform a motion search. Specifically, the encoder uses block matching to find the best matching block for the current block in the reference image, i.e., the initial reference block. The initial motion vector is a vector pointing from the current block to the initial reference block.

[0169] It should also be noted that in the embodiments of the present application, prediction parameters for the current block are first required. In some embodiments, the prediction reference includes at least an inter-frame prediction identification parameter. The inter-frame prediction identification parameter indicates the prediction method used to encode the current block, which can include: L0 prediction, L1 prediction, and bidirectional prediction.

[0170] It can be understood that in the embodiment of the present application, L0 represents the reference image in the reference image list 0, which can also be called the forward reference image, that is, the first reference image; L1 represents the reference image in the reference image list 1, which can also be called the backward reference image, that is, the second reference image.

[0171] For L0 prediction, the flag value for using reference picture list 0 for inter-frame prediction is equal to 1; meanwhile, the flag value for using reference picture list 1 for inter-frame prediction is equal to 0. For example, forward prediction.

[0172] For L1 prediction, the flag value for using reference picture list 0 for inter-frame prediction is equal to 0; meanwhile, the flag value for using reference picture list 1 for inter-frame prediction is equal to 1. For example, backward prediction.

[0173] For bidirectional prediction, the flag value for using reference picture list 0 for inter-frame prediction is equal to 1; at the same time, the flag value for using reference picture list 1 for inter-frame prediction is equal to 1. For example, forward prediction and backward prediction.

[0174] In some embodiments, when the prediction parameter includes an inter-frame prediction identification parameter, the method further includes: determining that the inter-frame prediction identification parameter may indicate that the prediction method used to encode the current block is bidirectional prediction.

[0175] For example, if the current block uses bidirectional prediction, the inter-frame prediction flag parameter may indicate that the prediction method used to encode the current block is bidirectional prediction. Otherwise, if the current block uses L0 prediction or L1 prediction, the inter-frame prediction flag parameter may indicate that the prediction method used to encode the current block is unidirectional prediction. The prediction method used by the current block is set according to actual conditions and is not specifically limited in the embodiments of the present application.

[0176] When the inter-frame prediction identification parameter indicates that the prediction method used to encode the current block is bidirectional prediction, the initial reference block at this time may include a first reference block and a second reference block. Specifically, the current block can find the first reference block in the first reference image through block matching, and the vector pointing from the current block to the first reference block can be called the first initial motion vector. The current block can find the second reference block in the second reference image through block matching, and the vector pointing from the current block to the second reference block can be called the second initial motion vector.

[0177] In one possible implementation, the prediction parameters may include: a motion vector, a reference image index, and an inter-frame prediction identification parameter. The reference image index indicates the reference image used by the current block, and the inter-frame prediction identification parameter indicates that the prediction method used to encode the current block is bidirectional prediction. In this case, the initial reference block includes a first reference block and a second reference block. In some embodiments, determining the prediction parameters of the current block may also include: determining a first initial motion vector in a first reference image and a first reference image index for indicating the first reference image based on the first reference block; and determining a second initial motion vector in a second reference image and a second reference image index for indicating the second reference image based on the second reference block.

[0178] In an embodiment of the present application, for prediction parameters, the motion vector may include a first initial motion vector and a second initial motion vector, and the reference image index may include a first reference image index and a second reference image index. Here, the first reference image index may indicate the first reference image in reference image list 0, and the second reference image index may indicate the second reference image in reference image list 1; and the first initial motion vector represents a vector pointing from the current block to the first reference block in the first reference image, and the second initial motion vector represents a vector pointing from the current block to the second reference block in the second reference image.

[0179] In some embodiments, the method further includes: determining prediction parameters for the current block; encoding the prediction parameters for the current block, and writing the resulting coded bits into a bitstream. In this way, a subsequent decoder can determine, based on information such as the decoded motion vector, reference image index, and inter-frame prediction identification parameter, a first reference block for the current block, such as the L0 reference block, and a second reference block for the current block, such as the L1 reference block, if the prediction method used for decoding the current block is bidirectional prediction.

[0180] In which, when the motion vector includes a first initial motion vector MV0 (for example, a forward motion vector), the first reference block (or called a "forward matching block") can be determined from the first reference image based on the first initial motion vector MV0; when the motion vector includes a second initial motion vector MV1 (for example, a backward motion vector), the second reference block (or called a "backward matching block") can be determined from the second reference image based on the second initial motion vector MV1.

[0181] In another possible implementation, the prediction parameters may include: a motion vector difference, a reference image index, and an inter-frame prediction identification parameter. The reference image index indicates the reference image used by the current block, and the inter-frame prediction identification parameter indicates that the prediction method used to encode the current block is bidirectional prediction. In this case, the initial reference block includes a first reference block and a second reference block. In some embodiments, determining the prediction parameters of the current block may also include: determining a first initial motion vector in the first reference image and a first reference image index for indicating the first reference image based on the first reference block; and determining a first initial motion vector prediction value of the current block in the first reference image, and determining a first motion vector difference based on the first initial motion vector and the first initial motion vector prediction value; determining a second initial motion vector in the second reference image and a second reference image index for indicating the second reference image based on the second reference block; and determining a second initial motion vector prediction value of the current block in the second reference image, and determining a second motion vector difference based on the second initial motion vector and the second initial motion vector prediction value.

[0182] In an embodiment of the present application, for the prediction parameters, the motion vector difference includes a first motion vector difference and a second motion vector difference, and the reference image index includes a first reference image index and a second reference image index. Here, the first reference image index may indicate the first reference image in reference image list 0, and the second reference image index may indicate the second reference image in reference image list 1; and the first motion vector difference represents the difference between the first initial motion vector and the first initial motion vector prediction value, the first initial motion vector representing the vector of the current block pointing to the first reference block in the first reference image; the second motion vector difference represents the difference between the second initial motion vector and the second initial motion vector prediction value, the second initial motion vector representing the vector of the current block pointing to the second reference block in the second reference image.

[0183] In some embodiments, the method further includes: determining prediction parameters for the current block; encoding the prediction parameters for the current block, and writing the resulting coded bits into the bitstream. That is, for motion information, the complete motion vector can be written directly into the bitstream, but to save bitrate, the motion vector difference can also be written into the bitstream. The motion vector difference MVD is equal to the difference between the initial motion vector MV and the motion vector prediction value MVP, i.e., MVD = MV - MVP. The more accurate the MVP, the smaller the MVD, which results in less overhead in the bitstream, thereby saving bitrate.

[0184] In this way, the subsequent decoding end can, based on the information such as the motion vector difference, reference image index and inter-frame prediction identification parameters obtained by decoding, if the prediction method used for decoding the current block is bidirectional prediction, then the first initial motion vector MV0 can be obtained by adding the first motion vector difference MVD0 and the determined first initial motion vector prediction value MVP0, and then the first reference block of the current block, such as the L0 reference block, is determined from the first reference image; and the second initial motion vector MV1 can be obtained by adding the second motion vector difference MVD1 and the determined second initial motion vector prediction value MVP1, and then the second reference block of the current block, such as the L1 reference block, is determined from the second reference image.

[0185] In another possible implementation, for the merge mode, the prediction parameters may include: a motion vector index, a reference image index, and an inter-frame prediction identification parameter. The reference image index indicates the reference image used by the current block, and the inter-frame prediction identification parameter indicates that the prediction method used to encode the current block is bidirectional prediction. In this case, the initial reference block includes a first reference block and a second reference block. In some embodiments, determining the prediction parameters of the current block may also include: constructing a motion vector candidate list for the current block; determining, based on the first reference block, a first initial motion vector in the first reference image and a first reference image index for indicating the first reference image, and determining a first motion vector index of the first initial motion vector in the motion vector candidate list; determining, based on the second reference block, a second initial motion vector in the second reference image and a second reference image index for indicating the second reference image, and determining a second motion vector index of the second initial motion vector in the motion vector candidate list.

[0186] In an embodiment of the present application, for prediction parameters, the motion vector index includes a first motion vector index and a second motion vector index, and the reference image index includes a first reference image index and a second reference image index. Here, the first reference image index may indicate the first reference image in reference image list 0, and the second reference image index may indicate the second reference image in reference image list 1; and the first motion vector index represents the index number of the first initial motion vector in the motion vector candidate list, the first initial motion vector representing the vector of the current block pointing to the first reference block in the first reference image; the second motion vector index represents the index number of the second initial motion vector in the motion vector candidate list, the second initial motion vector representing the vector of the current block pointing to the second reference block in the second reference image.

[0187] It should also be noted that in the embodiments of the present application, a motion vector candidate list can first be constructed. Specifically, this can be constructed using information related to the previously encoded portion of the current block. Spatial motion information and / or temporal motion information can be used in this construction process. Furthermore, the number of candidate motion vectors in the motion vector candidate list can be two, three, four, or more, without specific limitation herein.

[0188] It should also be noted that, in the embodiment of the present application, the motion vector candidate list constructed by the encoding end needs to be consistent with the motion vector candidate list constructed by the decoding end, so that the decoding end can determine the first initial motion vector MV0 and the second initial motion vector MV1 through the motion vector index transmitted in the code stream.

[0189] In some embodiments, the method further includes: determining prediction parameters for the current block; encoding the prediction parameters for the current block, and writing the resulting coded bits into a bitstream. In this way, a subsequent decoder can obtain a first initial motion vector MV0 and a second initial motion vector MV1 (derived from the first reference image L0 and the second reference image L1, respectively) by decoding the bitstream. A first reference block can then be determined based on the first initial motion vector MV0 and the first reference image L0, and a second reference block can be determined based on the second initial motion vector MV1 and the second reference image L1.

[0190] It's also understood that in a light field video, the corresponding light field video image will appear as a series of closely spaced macropixels. To improve encoding and decoding efficiency, the macropixel co-located blocks corresponding to each initial reference block can be further determined. It's important to note that the relative position of the initial reference block within its corresponding macropixel is identical to the relative position of the macropixel co-located blocks within its corresponding macropixel.

[0191] In some embodiments, determining the macropixel co-located block corresponding to the initial reference block may include: determining at least one first macropixel co-located block of the first reference block in the first reference image based on the first reference block; and determining at least one second macropixel co-located block of the second reference block in the second reference image based on the second reference block.

[0192] In a specific embodiment, when determining at least one first macropixel co-located block of the first reference block in the first reference image based on the first reference block, it can include: determining a first current macropixel and at least one first candidate macropixel in the first reference image, the first current macropixel including at least part of the first reference block; and determining the respective macropixel co-located blocks in the at least one first candidate macropixel based on the positional relationship of the first reference block in the first current macropixel to obtain at least one first macropixel co-located block.

[0193] It should be noted that, in the embodiment of the present application, the number of at least one first candidate macropixel may be 1, 2, 3, 4, or more. In addition, at least one first candidate macropixel may be adjacent to the first current macropixel, for example, the at least one first candidate macropixel is distributed around the first current macropixel; however, the at least one first candidate macropixel may not be adjacent to the first current macropixel, and this is not specifically limited here. It should also be noted that, in the embodiment of the present application, the macropixels may be arranged in a hexagonal shape, or according to other shape templates, such as a square, rectangle, etc., and this is not specifically limited here.

[0194] For example, still using FIG. 10 as an example, when macropixels are arranged in a hexagon, the number of at least one first candidate macropixel can be set to 6, and these 6 first candidate macropixels are adjacent to the first current macropixel and distributed around the first current macropixel. The hexagon in the middle is the first current macropixel, and the 6 adjacent hexagons are all first candidate macropixels.

[0195] It should also be noted that, in an embodiment of the present application, the first current macropixel may include a portion or all of the first reference block, and each first candidate macropixel may include a portion or all of the corresponding first macropixel co-located block. Still taking Figure 10 as an example, the black-filled dot in the first current macropixel represents the upper left corner position of the first reference block, that is, the black-filled dot can indicate the position of the first reference block; in these six first candidate macropixels, the grid-filled dots represent the upper left corner position of the first macropixel co-located block, that is, the grid-filled dots can indicate the position of the first macropixel co-located block. In other words, in the first reference image, the relative position of the first reference block in the first current macropixel is the same as the relative position of the first macropixel co-located block in the first candidate macropixel.

[0196] In another specific embodiment, when determining at least one second macropixel co-located block of the second reference block in the second reference image based on the second reference block, it can include: determining a second current macropixel and at least one second candidate macropixel in the second reference image, the second current macropixel including at least part of the second reference block; and determining the respective macropixel co-located blocks in the at least one second candidate macropixel based on the positional relationship of the second reference block in the second current macropixel to obtain at least one second macropixel co-located block.

[0197] It should be noted that, in the embodiment of the present application, the number of the at least one second candidate macropixel may be 1, 2, 3, 4, or more. In addition, the at least one second candidate macropixel may be adjacent to the second current macropixel, for example, the at least one second candidate macropixel is distributed around the second current macropixel; however, the at least one second candidate macropixel may not be adjacent to the second current macropixel, which is not specifically limited here.

[0198] It should also be noted that, in this embodiment of the present application, the second current macropixel may include a portion or all of the second reference block, and each second candidate macropixel may include a portion or all of the corresponding second macropixel co-located block. Furthermore, in the second reference image, the relative position of the second reference block in the second current macropixel is the same as the relative position of the second macropixel co-located block in the second candidate macropixel. It should be noted that the method for determining the second macropixel co-located block in the second reference image is similar to the method for determining the first macropixel co-located block in the first reference image. See FIG. 10 for details, and no further details will be given here.

[0199] In this way, after determining the initial reference block and macropixel co-located block (including the first reference block and the first macropixel co-located block located in the first reference image and the second reference block and the second macropixel co-located block located in the second reference image), a refined search can be performed to determine better target motion parameters.

[0200] S1302: Search the initial reference block and the macropixel co-located block respectively to determine target motion parameters.

[0201] It should be noted that, in an embodiment of the present application, searching based on the initial reference block and the macropixel co-located block respectively to determine the target motion parameters may include: searching based on the initial reference block and the macropixel co-located block in preset areas of the corresponding macropixels to determine multiple groups of candidate motion parameters; performing cost calculations on the multiple groups of candidate motion parameters to obtain multiple groups of cost results; determining the minimum cost result from the multiple groups of cost results, and determining the group of candidate motion parameters corresponding to the minimum cost result as the target motion parameters.

[0202] It should also be noted that, in the embodiment of the present application, each set of candidate motion parameters may include a first candidate motion vector and a second candidate motion vector; accordingly, the ultimately determined target motion parameters may include a first target motion vector and a second target motion vector. The first candidate motion vector and the first target motion vector are determined for the first reference image, while the second candidate motion vector and the second target motion vector are determined for the second reference image.

[0203] It should also be noted that in the embodiment of the present application, the preset area can be a 5×5 grid area, but can also be a 3×3 grid area, a 4×4 grid area, etc. In other words, there is no limitation on the number of search area locations and the selection of specific locations, and there is no specific limitation on the size of the grid area.

[0204] In the embodiment of the present application, for determining target motion parameters, FIG14 is a second flow chart of determining target motion parameters provided by the embodiment of the present application. As shown in FIG14 , the method may include:

[0205] S1401: With an initial reference block as the center, search in a first preset area of ​​macro pixels to which the initial reference block belongs to determine at least one set of first candidate motion parameters.

[0206] S1402: With the macropixel co-located block as the center, search in a second preset area of ​​macropixels belonging to the macropixel co-located block to determine at least one set of second candidate motion parameters.

[0207] S1403: Perform cost calculation on multiple groups of candidate motion parameters respectively to obtain multiple groups of cost results; wherein the multiple groups of candidate motion parameters include at least one group of first candidate motion parameters and at least one group of second candidate motion parameters.

[0208] S1404: Determine a minimum cost result from the multiple groups of cost results, and determine a group of candidate motion parameters corresponding to the minimum cost result as target motion parameters.

[0209] It should be noted that, in an embodiment of the present application, for multiple groups of candidate motion parameters, during the detailed search process, a search can be performed in a first preset area of ​​macropixels to which the initial reference block belongs, with the initial reference block as the center, to determine at least one group of first candidate motion parameters; and a search can be performed in a second preset area of ​​macropixels to which the macropixel co-located block belongs, with the macropixel co-located block as the center, to determine at least one group of second candidate motion parameters; and then the at least one group of first candidate motion parameters and the at least one group of second candidate motion parameters are determined as multiple groups of candidate motion parameters.

[0210] It should also be noted that in the embodiments of the present application, the first preset area and the second preset area can be the same or different. For example, the first preset area can be a 5×5 grid area, and the second preset area can also be a 5×5 grid area; then, a 5×5 grid search is performed with the initial reference block and the macropixel co-located block as the center, and the position indicated by the optimal motion parameter (i.e., the target motion parameter) is determined from the search, which is the result of the refined search.

[0211] For example, assuming that the number of macropixel co-located blocks is 6, then for at least one group of first candidate motion parameters, there can be specifically 5×5 groups; for at least one group of second candidate motion parameters, there can be specifically 6×5×5 groups; that is, for multiple groups of candidate motion parameters, there can be a total of (1+6)×5×5 groups of candidate motion parameters, and then the best group is selected from so many groups of candidate motion parameters, that is, the group of candidate motion parameters corresponding to the minimum cost result is determined as the target motion parameters.

[0212] In a specific embodiment, searching in a first preset area of ​​the macropixels to which the initial reference block belongs with an initial reference block as the center to determine at least one set of first candidate motion parameters may include: searching in a first preset area of ​​the first current macropixel with the first reference block as the center to determine at least one first candidate motion vector; and determining at least one second candidate motion vector corresponding to the second reference block based on the at least one first candidate motion vector and the first preset area of ​​the second current macropixel with the second reference block as the center; and determining at least one set of first candidate motion parameters based on the at least one first candidate motion vector and the at least one second candidate motion vector.

[0213] In the embodiment of the present application, the first offset between the first initial motion vector and the first candidate motion vector and the second offset between the second initial motion vector and the second candidate motion vector are symmetrical. In other words, the refinement search process is symmetrical, i.e., the first offset between the position corresponding to the first reference block and the position indicated by the first candidate motion vector and the second offset between the position corresponding to the second reference block and the position indicated by the second candidate motion vector are symmetrically distributed.

[0214] It should also be noted that, in the embodiment of the present application, for a set of candidate motion parameters, the first candidate motion vector and the second candidate motion vector appear in pairs. For example, in the first reference image, the first candidate motion vector MV0' can be a first initial motion vector MV0 shifted by MV diff Then in the second reference image, the second candidate motion vector MV1' can be a moving -MV based on the second initial motion vector MV1 diffIf the first preset area is a 5×5 grid area, then 5×5 groups of first candidate motion parameters can be searched out.

[0215] In another specific embodiment, determining at least one set of second candidate motion parameters by searching a second preset area of ​​macropixels to which the macropixel co-located block belongs, with the macropixel co-located block as the center, may include: determining at least one first candidate motion vector by searching a second preset area of ​​first candidate macropixels to which the first candidate macropixels belong, with the first macropixel co-located block as the center; and determining at least one second candidate motion vector corresponding to the second macropixel co-located block based on the at least one first candidate motion vector and the second preset area of ​​second candidate macropixels to which the second macropixel co-located block is the center; and determining at least one set of second candidate motion parameters based on the at least one first candidate motion vector and the at least one second candidate motion vector.

[0216] It should be noted that in the embodiment of the present application, the refined search process here is symmetrical, that is, the first offset between the corresponding position of the first macropixel co-located block and the position indicated by the first candidate motion vector and the second offset between the corresponding position of the second macropixel co-located block and the position indicated by the second candidate motion vector are symmetrically distributed.

[0217] It should also be noted that in the embodiment of the present application, the search process is similar to the search process centered on the initial reference block. If the second preset area is a 5×5 grid area and the number of macropixel co-located blocks is 6, then 6×5×5 groups of second candidate motion parameters can be searched here.

[0218] In this way, during the refinement search, the scope of the refinement operation can be expanded to include adjacent macropixels. After searching for at least one set of first candidate motion parameters and at least one set of second candidate motion parameters, multiple sets of candidate motion parameters can be obtained. Costs are then calculated for each of these multiple sets of candidate motion parameters to obtain multiple sets of cost results.

[0219] In one possible implementation, cost calculation is performed on multiple groups of candidate motion parameters respectively to obtain multiple groups of cost results, including: determining a first candidate block corresponding to a first candidate motion vector and a second candidate block corresponding to a second candidate motion vector based on each group of candidate motion parameters; calculating the cost results of the first candidate block and the second candidate block according to preset criteria to obtain a cost result corresponding to each group of candidate motion parameters.

[0220] In an embodiment of the present application, the preset criteria may include at least one of the following: the sum of absolute errors SAD, the sum of absolute transformed errors SATD, the mean square error MSE, the sum of squared errors SSE, and the mean absolute error MAE.

[0221] Exemplarily, the preset criterion takes the sum of absolute differences SAD as an example. First, the first candidate block pointed to by the first candidate motion vector MV0' and the second candidate block pointed to by the second candidate motion vector MV1' are determined. Then, the SAD between the first candidate block and the second candidate block is calculated. Finally, a set of MV0' and MV1' with the smallest SAD value is selected as the target motion vector, that is, the final refined search result.

[0222] In another possible implementation, cost calculation is performed on multiple groups of candidate motion parameters respectively to obtain multiple groups of cost results, which may include: determining the first candidate block corresponding to the first candidate motion vector and the second candidate block corresponding to the second candidate motion vector based on each group of candidate motion parameters, and performing weighted calculation based on the first candidate block and the second candidate block to determine the candidate prediction block; performing weighted calculation based on the first reference block and the second reference block to determine the initial prediction block; calculating the cost results of the candidate prediction blocks and the initial prediction blocks according to preset criteria to obtain the cost results corresponding to each group of candidate motion parameters.

[0223] In an embodiment of the present application, the preset criteria may include at least one of the following: the sum of absolute errors SAD, the sum of absolute transformed errors SATD, the mean square error MSE, the sum of squared errors SSE, and the mean absolute error MAE.

[0224] Exemplarily, the preset criterion takes the sum of absolute differences SAD as an example, first determining the first candidate block pointed to by the first candidate motion vector MV0' and the second candidate block pointed to by the second candidate motion vector MV1', then performing a weighted combination of the first candidate block and the second candidate block to obtain a candidate prediction block, and then performing a weighted combination based on the first reference block pointed to by the first initial motion vector MV0 and the second reference block pointed to by the second initial motion vector MV1 to obtain the initial prediction block, if the SAD value between the initial prediction block and the candidate prediction block is the smallest, then the MV0' and MV1' corresponding to the candidate prediction block can be used as the target motion vector, that is, the final refined search result.

[0225] It should also be noted that the two possible implementations described above are merely two common examples, and other possible implementations are also possible. Furthermore, the selection criteria for the refined search may be other than the sum of absolute deviations (SAD), the sum of absolute transformed deviations (SATD), the mean square error (MSE), the sum of squared errors (SSE), and the mean absolute error (MAE), and are not specifically limited here.

[0226] For example, taking FIG12 as an example, the macro pixels are arranged in a hexagonal shape, and the search range in each macro pixel can be a 5×5 grid area. Taking the first reference image as an example, the middle macro pixel is the first current macro pixel, and the 6 adjacent macro pixels are the first candidate macro pixels. Among them, the dots filled with black can indicate the position of the first reference block (i.e., the initial position), the dots filled with grid can indicate the position of the first macro pixel co-location block (i.e., the macro pixel co-location position), and the dots filled with white represent the search position. In this way, the position of the first reference block and the position of the first macro pixel co-location block are respectively centered, and a 5×5 grid search is performed, and then the best refined position is selected from the (1+6)×5×5 search positions, that is, the position indicated by the target motion vector MV0(1)'. In FIG12, the dots filled with slashes can indicate the position indicated by the target motion vector MV0(1)', which is also the final search result.

[0227] S1303: Determine the prediction value of the current block according to the target motion parameter.

[0228] It should be noted that after obtaining the target motion parameters, since the prediction method used to encode the current block is bidirectional prediction, the target motion parameters may include a first target motion vector and a second target motion vector. In some embodiments, determining a prediction value for the current block based on the target motion parameters may include: determining a first prediction block for the current block based on the first target motion vector; determining a second prediction block for the current block based on the second target motion vector; and determining a prediction value for the current block based on the first prediction block and the second prediction block.

[0229] In an embodiment of the present application, the first prediction block of the current block is determined according to the first target motion vector, and the block pointed to by the first target motion vector in the first reference image may be used as the first prediction block; similarly, the second prediction block of the current block is determined according to the second target motion vector, and the block pointed to by the second target motion vector in the second reference image may be used as the second prediction block.

[0230] In some embodiments, determining the prediction value of the current block according to the first prediction block and the second prediction block may include: performing a weighted calculation according to the first prediction block and the second prediction block to determine the prediction value of the current block.

[0231] For example, assuming that the first prediction block can be represented by Pred1, the second prediction block can be represented by Pred2, the weight corresponding to the first prediction block is set to w1, and the weight corresponding to the second prediction block is set to w2; then the prediction value Pred[x][y] of the current block is calculated as follows:

[0232] Pred[x][y]=w1×Pred1+w2×Pred2, wherein the sizes of w1 and w2 can be preset constant values, or can be determined according to the size parameters of the current block, or can even be determined according to other methods, which are not specifically limited here.

[0233] Furthermore, after determining the prediction value of the current block, in some embodiments, referring to FIG. 13 , the method further includes:

[0234] S1304: Determine a prediction difference value of the current block according to the prediction value of the current block.

[0235] S1305: Encode the prediction difference of the current block and write the obtained coded bits into the bitstream.

[0236] It should be noted that, in an embodiment of the present application, determining the prediction difference value of the current block based on the prediction value of the current block may include: determining the initial value of the current block; and determining the prediction difference value of the current block based on the initial value of the current block and the prediction value of the current block.

[0237] In a specific embodiment, determining the prediction difference value of the current block may specifically include performing a subtraction operation on the initial value of the current block and the prediction value of the current block to determine the prediction difference value of the current block.

[0238] That is, in the embodiment of the present application, after obtaining the predicted value of the current block, the predicted value can be subtracted from the initial value of the current block to obtain the predicted difference value of the current block (i.e., the prediction residual). The predicted difference value of the current block is then encoded into the bitstream. This allows a subsequent decoder to obtain the predicted difference value of the current block by decoding the bitstream and thereby recover the reconstructed value of the current block.

[0239] In another embodiment of the present application, the embodiment of the present application further provides a code stream, which can be generated by bit encoding based on the information to be encoded; wherein the information to be encoded includes at least one of the following: a prediction difference value of the current block and a prediction parameter of the current block.

[0240] It should be noted that, in the embodiment of the present application, for the prediction parameters of the current block, the prediction parameters may include a motion vector, a reference image index, and an inter-frame prediction identification parameter, that is, the motion vector, the reference image index, and the inter-frame prediction identification parameter may be encoded into the bitstream; or, the prediction parameters may include a motion vector difference, a reference image index, and an inter-frame prediction identification parameter, that is, the motion vector difference, the reference image index, and the inter-frame prediction identification parameter may be encoded into the bitstream; or, the prediction parameters may include a motion vector index, a reference image index, and an inter-frame prediction identification parameter, that is, the motion vector index, the reference image index, and the inter-frame prediction identification parameter may be encoded into the bitstream. The content of the prediction parameters is not specifically limited here.

[0241] This embodiment provides an encoding method that determines an initial reference block for a current block and a macropixel co-located block corresponding to the initial reference block; performs searches based on the initial reference block and the macropixel co-located block to determine target motion parameters; and determines a prediction value for the current block based on the target motion parameters. Because macropixels in a light field video image have spatial correlation, when determining the target motion parameters, the accuracy of motion vectors in light field video encoding and decoding can be improved by expanding the search range to include adjacent macropixels, thereby saving bitrate, improving compression efficiency, and ultimately enhancing light field video encoding and decoding performance.

[0242] In another embodiment of the present application, an improved DMVR technology solution for light field video coding is proposed. Generally speaking, there is a strong correlation between macropixels in light field video images. Therefore, when refining motion vectors, the scope of the refinement operation can be expanded to include adjacent macropixels. Based on this, the specific process of the improved solution is as follows:

[0243] (1) Obtain the initial motion vectors MV0 and MV1 obtained in merge mode at the decoding end (from reference picture list 0 and reference picture list 1, respectively);

[0244] (2) Refined search based on macropixel co-located blocks: The position indicated by MV0(1) is used as the initial position, and the corresponding macropixel co-located block position is obtained from this (the relative positions of the initial position and the macropixel co-located block position in each macropixel are the same). Then, a 5×5 grid search is performed with the initial position and the macropixel co-located block position as the center, and the best position is selected as the position indicated by MV0(1)', which is the result of the refined search. Specifically, as shown in Figure 15, the refined algorithm of the improved DMVR technology is shown here.

[0245] (3) It should be noted that in FIG15 , the best one needs to be selected from (1+6)×5×5 positions, but the number of search positions and the selection of positions are not limited. That is, the number of macropixel co-located blocks is not necessarily 6, and the grid search is not necessarily 5×5, but can also be 3×3, etc. The macropixels in FIG15 are arranged in a hexagonal shape, but can also be arranged according to other shape templates, such as a square, etc. The selection criteria used for the refined search are also not limited. The following are two common selection criteria:

[0246] i. The initial prediction block is obtained by weighted combination of the blocks represented by the positions indicated by MV0 and MV1. Then, the candidate prediction block is obtained by weighted combination of the blocks represented by the positions indicated by MV0' and MV1'. If the SAD value between the initial prediction block and the candidate prediction block is the smallest, the MV0' and MV1' corresponding to the candidate prediction block are the final refined search results;

[0247] ii. Calculate the SAD between the blocks represented by the positions pointed to by MV0' and MV1'. The MV0' and MV1' with the smallest SAD value are the final refined search results.

[0248] In the embodiments of the present application, the specific implementation of the aforementioned embodiments is described in detail through the above embodiments. It can be seen that according to the technical solutions of the aforementioned embodiments, this improved solution takes into account the spatial correlation of light field images, which is beneficial to improving the accuracy of motion vectors in light field video encoding, thereby improving the compression efficiency of light field videos, and further improving the encoding and decoding performance of light field videos.

[0249] In yet another embodiment of the present application, based on the same inventive concept as the aforementioned embodiment, FIG16 is a schematic diagram of the structure of an encoder provided by the embodiment of the present application. As shown in FIG16 , the encoder 160 may include a first determination unit 1601, a first search unit 1602, and a first prediction unit 1603, wherein: the first determination unit 1601 is configured to determine an initial reference block for the current block; and determine a macropixel co-located block corresponding to the initial reference block;

[0250] A first search unit 1602 is configured to search based on the initial reference block and the macropixel co-located block respectively to determine the target motion parameter;

[0251] The first prediction unit 1603 is configured to determine a prediction value of the current block according to the target motion parameter.

[0252] In some embodiments, referring to FIG. 16 , the encoder 160 may further include an encoding unit 1604, wherein: the first determination unit 1601 is further configured to determine a prediction difference value of the current block based on a prediction value of the current block; and the encoding unit 1604 is configured to perform encoding processing on the prediction difference value of the current block and write the obtained encoding bits into the bitstream.

[0253] In some embodiments, the first determining unit 1601 is further configured to determine an initial value of the current block; and determine a prediction difference value of the current block according to the initial value of the current block and the prediction value of the current block.

[0254] In some embodiments, the first determining unit 1601 is further configured to determine prediction parameters of the current block; the encoding unit 1604 is further configured to encode the prediction parameters of the current block and write the obtained coded bits into the bitstream.

[0255] In some embodiments, the prediction parameter includes an inter-frame prediction identification parameter; the first determining unit 1601 is further configured to determine whether the inter-frame prediction identification parameter indicates that the prediction method used to encode the current block is bidirectional prediction.

[0256] In some embodiments, the initial reference block includes a first reference block and a second reference block, and the prediction parameters also include a motion vector and a reference image index; the first determination unit 1601 is further configured to determine, based on the first reference block, a first initial motion vector in the first reference image and a first reference image index for indicating the first reference image; based on the second reference block, determine a second initial motion vector in the second reference image and a second reference image index for indicating the second reference image; wherein the motion vector includes a first initial motion vector and a second initial motion vector, and the reference image index includes a first reference image index and a second reference image index.

[0257] In some embodiments, the initial reference block includes a first reference block and a second reference block, and the prediction parameters also include a motion vector difference and a reference image index; the first determination unit 1601 is further configured to determine, based on the first reference block, a first initial motion vector in the first reference image and a first reference image index for indicating the first reference image; and determine a first initial motion vector prediction value of the current block in the first reference image, and determine a first motion vector difference based on the first initial motion vector and the first initial motion vector prediction value; determine a second initial motion vector in the second reference image and a second reference image index for indicating the second reference image based on the second reference block; and determine a second initial motion vector prediction value of the current block in the second reference image, and determine a second motion vector difference based on the second initial motion vector and the second initial motion vector prediction value; wherein, the motion vector difference includes a first motion vector difference and a second motion vector difference, and the reference image index includes a first reference image index and a second reference image index.

[0258] In some embodiments, the initial reference block includes a first reference block and a second reference block, and the prediction parameters also include a motion vector index and a reference image index; the first determination unit 1601 is also configured to construct a motion vector candidate list for the current block; based on the first reference block, determine the first initial motion vector in the first reference image and the first reference image index for indicating the first reference image, and determine the first motion vector index of the first initial motion vector in the motion vector candidate list; based on the second reference block, determine the second initial motion vector in the second reference image and the second reference image index for indicating the second reference image, and determine the second motion vector index of the second initial motion vector in the motion vector candidate list; wherein the motion vector index includes the first motion vector index and the second motion vector index, and the reference image index includes the first reference image index and the second reference image index.

[0259] In some embodiments, the first determination unit 1601 is further configured to determine, based on the first reference block, at least one first macropixel co-located block of the first reference block in the first reference image; and to determine, based on the second reference block, at least one second macropixel co-located block of the second reference block in the second reference image.

[0260] In some embodiments, the first determination unit 1601 is further configured to determine a first current macropixel and at least one first candidate macropixel in the first reference image, where the first current macropixel includes at least part of the first reference block; and determine the respective macropixel co-located blocks in the at least one first candidate macropixel based on the positional relationship of the first reference block in the first current macropixel to obtain at least one first macropixel co-located block.

[0261] In some embodiments, the first determination unit 1601 is further configured to determine a second current macropixel and at least one second candidate macropixel in a second reference image, where the second current macropixel includes at least part of the second reference block; and determine the respective macropixel co-located blocks in at least one second candidate macropixel based on the positional relationship of the second reference block in the second current macropixel to obtain at least one second macropixel co-located block.

[0262] In some embodiments, referring to FIG16 , the encoder 160 may further include a first selection unit 1605, wherein: the first search unit 1602 is further configured to search in preset areas of the macropixels to which it belongs according to the initial reference block and the macropixel co-located block, and determine multiple groups of candidate motion parameters, wherein each group of candidate motion parameters includes a first candidate motion vector and a second candidate motion vector; the first selection unit 1605 is configured to perform cost calculations on the multiple groups of candidate motion parameters respectively, and obtain multiple groups of cost results; determine a minimum cost result from the multiple groups of cost results, and determine a group of candidate motion parameters corresponding to the minimum cost result as the target motion parameters.

[0263] In some embodiments, the first search unit 1602 is further configured to search a first preset area of ​​macropixels to which the initial reference block belongs, with the initial reference block as the center, to determine at least one set of first candidate motion parameters; to search a second preset area of ​​macropixels to which the macropixel co-located block belongs, with the macropixel co-located block as the center, to determine at least one set of second candidate motion parameters; and to determine at least one set of first candidate motion parameters and at least one set of second candidate motion parameters as multiple sets of candidate motion parameters.

[0264] In some embodiments, the first search unit 1602 is further configured to search in a first preset area of ​​the first current macropixel with the first reference block as the center to determine at least one first candidate motion vector; and determine at least one second candidate motion vector corresponding to the second reference block based on the at least one first candidate motion vector and the first preset area of ​​the second current macropixel with the second reference block as the center; and determine at least one set of first candidate motion parameters based on the at least one first candidate motion vector and the at least one second candidate motion vector.

[0265] In some embodiments, a first offset between the first initial motion vector and the first candidate motion vector and a second offset between the second initial motion vector and the second candidate motion vector have a symmetrical relationship.

[0266] In some embodiments, the first selection unit 1605 is further configured to determine, based on each group of candidate motion parameters, a first candidate block corresponding to the first candidate motion vector and a second candidate block corresponding to the second candidate motion vector; calculate the cost results of the first candidate block and the second candidate block according to preset criteria to obtain the cost results corresponding to each group of candidate motion parameters.

[0267] In some embodiments, the first selection unit 1605 is further configured to determine, based on each group of candidate motion parameters, a first candidate block corresponding to the first candidate motion vector and a second candidate block corresponding to the second candidate motion vector, and determine the candidate prediction block by performing weighted calculation based on the first candidate block and the second candidate block; determine the initial prediction block by performing weighted calculation based on the first reference block and the second reference block; calculate the cost results of the candidate prediction blocks and the initial prediction blocks according to preset criteria to obtain the cost results corresponding to each group of candidate motion parameters.

[0268] In some embodiments, the preset criterion includes at least one of the following: the sum of absolute errors (SAD), the sum of absolute transformed errors (SATD), the mean square error (MSE), the sum of squared errors (SSE), and the mean absolute error (MAE).

[0269] In some embodiments, the target motion parameters include a first target motion vector and a second target motion vector; the first determination unit 1601 is further configured to determine a first prediction block of the current block based on the first target motion vector; determine a second prediction block of the current block based on the second target motion vector; and determine a prediction value of the current block based on the first prediction block and the second prediction block.

[0270] In some embodiments, the first determining unit 1601 is further configured to perform weighted calculation based on the first prediction block and the second prediction block to determine the prediction value of the current block.

[0271] It is understandable that in the embodiments of the present application, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and of course it can also be a module, or it can be non-modular. Moreover, the various components in this embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into a single unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional modules.

[0272] If the integrated unit is implemented as a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, or the portion that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the method described in this embodiment. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0273] Therefore, an embodiment of the present application provides a computer-readable storage medium, which is applied to the encoder 160. The computer-readable storage medium stores a computer program, and when the computer program is executed by the first processor, it implements the method described in any one of the aforementioned embodiments.

[0274] Based on the composition of the above-mentioned encoder 160 and the computer-readable storage medium, Figure 17 is a schematic diagram of the specific hardware structure of an encoder provided in an embodiment of the present application. As shown in Figure 17, the encoder 160 may include: a first communication interface 1701, a first memory 1702 and a first processor 1703; each component is coupled together through a first bus system 1704. It can be understood that the first bus system 1704 is used to realize the connection and communication between these components. In addition to the data bus, the first bus system 1704 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, various buses are labeled as the first bus system 1704 in Figure 17. Among them, the first communication interface 1701 is used to receive and send signals in the process of sending and receiving information between other external network elements;

[0275] A first memory 1702 is used to store computer programs that can be run on the first processor 1703;

[0276] The first processor 1703 is configured to, when running the computer program, execute:

[0277] Determine an initial reference block of the current block and a macropixel co-located block corresponding to the initial reference block; perform searches based on the initial reference block and the macropixel co-located block to determine target motion parameters; and determine a prediction value of the current block based on the target motion parameters.

[0278] It is understood that the first memory 1702 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DRRAM). The first memory 1702 of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0279] The first processor 1703 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits or software instructions in the first processor 1703. The above-mentioned first processor 1703 can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The various methods, steps, and logic block diagrams disclosed in the embodiments of this application can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the first memory 1702 , and the first processor 1703 reads the information in the first memory 1702 and completes the steps of the above method in combination with its hardware.

[0280] It is to be understood that these embodiments described in the present application can be implemented with hardware, software, firmware, middleware, microcode or its combination.For hardware implementation, the processing unit can be implemented in one or more application specific integrated circuits (Application Specific Integrated Circuits, ASIC), digital signal processor (Digital Signal Processing, DSP), digital signal processing equipment (DSP Device, DSPD), programmable logic device (Programmable Logic Device, PLD), field programmable gate array (Field-Programmable Gate Array, FPGA), general-purpose processor, controller, microcontroller, microprocessor, other electronic units for performing functions described in the present application or its combination.For software implementation, the technology described in the present application can be realized by the module (such as process, function etc.) that performs functions described in the present application. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0281] Optionally, as another embodiment, the first processor 1703 is further configured to execute any one of the methods described in the foregoing embodiments when running the computer program.

[0282] This embodiment provides an encoder. In this encoder, due to the spatial correlation between macropixels in a light field video image, when determining target motion parameters, the search range involved in the operation is expanded to between adjacent macropixels. This can improve the accuracy of motion vectors in light field video encoding and decoding, thereby saving bit rate, improving the compression efficiency of light field videos, and further improving the encoding and decoding performance of light field videos.

[0283] In yet another embodiment of the present application, based on the same inventive concept as the aforementioned embodiment, FIG18 is a schematic diagram of the structure of a decoder provided by the embodiment of the present application. As shown in FIG18 , the decoder 180 may include a second determination unit 1801, a second search unit 1802, and a second prediction unit 1803, wherein: the second determination unit 1801 is configured to determine an initial reference block for the current block; and determine a macropixel co-located block corresponding to the initial reference block;

[0284] The second search unit 1802 is configured to search the initial reference block and the macropixel co-located block respectively to determine the target motion parameter;

[0285] The second prediction unit 1803 is configured to determine a prediction value of the current block according to the target motion parameter.

[0286] In some embodiments, referring to FIG18 , the decoder 180 may further include a decoding unit 1804 configured to decode the bitstream and determine prediction parameters of the current block;

[0287] The second determining unit 1801 is further configured to determine an initial reference block of the current block according to the prediction parameters.

[0288] In some embodiments, the prediction parameter includes an inter-frame prediction identification parameter; the second determining unit 1801 is further configured to determine that the inter-frame prediction identification parameter indicates that the prediction method used for decoding the current block is bidirectional prediction.

[0289] In some embodiments, the prediction parameters also include a motion vector and a reference image index, and the reference image index includes a first reference image index and a second reference image index; the second determination unit 1801 is further configured to determine the first reference block of the current block based on the first reference image indicated by the first reference image index and the first initial motion vector in the motion vector; determine the second reference block of the current block based on the second reference image indicated by the second reference image index and the second initial motion vector in the motion vector; and use the first reference block and the second reference block as the initial reference blocks of the current block.

[0290] In some embodiments, the prediction parameters also include a motion vector difference and a reference image index, and the reference image index includes a first reference image index and a second reference image index; the second determination unit 1801 is also configured to determine a first initial motion vector prediction value of the current block in the first reference image and a second initial motion vector prediction value of the current block in the second reference image; determine the first initial motion vector based on the first motion vector difference and the first initial motion vector prediction value in the motion vector difference, and determine the first reference block of the current block based on the first reference image and the first initial motion vector indicated by the first reference image index; determine the second initial motion vector based on the second motion vector difference and the second initial motion vector prediction value in the motion vector difference, and determine the second reference block of the current block based on the second reference image and the second initial motion vector indicated by the second reference image index; and use the first reference block and the second reference block as the initial reference blocks of the current block.

[0291] In some embodiments, the prediction parameters also include a motion vector index and a reference image index, and the reference image index includes a first reference image index and a second reference image index; the second determination unit 1801 is also configured to construct a motion vector candidate list for the current block; determine the first initial motion vector based on the motion vector candidate list and the first motion vector index in the motion vector index, and determine the first reference block of the current block based on the first reference image indicated by the first reference image index and the first initial motion vector; determine the second initial motion vector based on the motion vector candidate list and the second motion vector index in the motion vector index, and determine the second reference block of the current block based on the second reference image indicated by the second reference image index and the second initial motion vector; use the first reference block and the second reference block as the initial reference blocks of the current block.

[0292] In some embodiments, the second determination unit 1801 is further configured to determine, based on the first reference block, at least one first macropixel co-located block of the first reference block in the first reference image; and to determine, based on the second reference block, at least one second macropixel co-located block of the second reference block in the second reference image.

[0293] In some embodiments, the second determination unit 1801 is further configured to determine a first current macropixel and at least one first candidate macropixel in the first reference image, where the first current macropixel includes at least part of the first reference block; and determine the respective macropixel co-located blocks in the at least one first candidate macropixel based on the positional relationship of the first reference block in the first current macropixel to obtain at least one first macropixel co-located block.

[0294] In some embodiments, the second determination unit 1801 is further configured to determine a second current macropixel and at least one second candidate macropixel in a second reference image, where the second current macropixel includes at least part of the second reference block; and determine the respective macropixel co-located blocks in at least one second candidate macropixel based on the positional relationship of the second reference block in the second current macropixel to obtain at least one second macropixel co-located block.

[0295] In some embodiments, referring to FIG18 , the decoder 180 may further include a second selection unit 1805, wherein: the second search unit 1802 is further configured to search in preset areas of the macropixels to which it belongs according to the initial reference block and the macropixel co-located block, and determine multiple groups of candidate motion parameters, wherein each group of candidate motion parameters includes a first candidate motion vector and a second candidate motion vector; the second selection unit 1805 is configured to perform cost calculations on the multiple groups of candidate motion parameters respectively, and obtain multiple groups of cost results; determine the minimum cost result from the multiple groups of cost results, and determine a group of candidate motion parameters corresponding to the minimum cost result as the target motion parameters.

[0296] In some embodiments, the second search unit 1802 is further configured to search a first preset area of ​​macropixels to which the initial reference block belongs, with the initial reference block as the center, to determine at least one set of first candidate motion parameters; to search a second preset area of ​​macropixels to which the macropixel co-located block belongs, with the macropixel co-located block as the center, to determine at least one set of second candidate motion parameters; and to determine at least one set of first candidate motion parameters and at least one set of second candidate motion parameters as multiple sets of candidate motion parameters.

[0297] In some embodiments, the second search unit 1802 is further configured to search in a first preset area of ​​the first current macropixel with the first reference block as the center to determine at least one first candidate motion vector; and determine at least one second candidate motion vector corresponding to the second reference block based on the at least one first candidate motion vector and the first preset area of ​​the second current macropixel with the second reference block as the center; and determine at least one set of first candidate motion parameters based on the at least one first candidate motion vector and the at least one second candidate motion vector.

[0298] In some embodiments, a first offset between the first initial motion vector and the first candidate motion vector and a second offset between the second initial motion vector and the second candidate motion vector have a symmetrical relationship.

[0299] In some embodiments, the second selection unit 1805 is further configured to determine, based on each group of candidate motion parameters, a first candidate block corresponding to the first candidate motion vector and a second candidate block corresponding to the second candidate motion vector; and calculate the cost results of the first candidate block and the second candidate block according to preset criteria to obtain the cost results corresponding to each group of candidate motion parameters.

[0300] In some embodiments, the second selection unit 1805 is further configured to determine, based on each group of candidate motion parameters, a first candidate block corresponding to the first candidate motion vector and a second candidate block corresponding to the second candidate motion vector, and determine the candidate prediction block by performing weighted calculation based on the first candidate block and the second candidate block; determine the initial prediction block by performing weighted calculation based on the first reference block and the second reference block; calculate the cost results of the candidate prediction blocks and the initial prediction blocks according to preset criteria to obtain the cost results corresponding to each group of candidate motion parameters.

[0301] In some embodiments, the preset criterion includes at least one of the following: the sum of absolute errors (SAD), the sum of absolute transformed errors (SATD), the mean square error (MSE), the sum of squared errors (SSE), and the mean absolute error (MAE).

[0302] In some embodiments, the target motion parameters include a first target motion vector and a second target motion vector; the second determination unit 1801 is further configured to determine a first prediction block of the current block based on the first target motion vector; determine a second prediction block of the current block based on the second target motion vector; and determine a prediction value of the current block based on the first prediction block and the second prediction block.

[0303] In some embodiments, the second determining unit 1801 is further configured to perform weighted calculation based on the first prediction block and the second prediction block to determine the prediction value of the current block.

[0304] In some embodiments, the decoding unit 1804 is further configured to decode the code stream and determine the prediction difference value of the current block; the second determination unit 1801 is further configured to determine the reconstructed value of the current block based on the prediction difference value of the current block and the prediction value of the current block.

[0305] It is understood that in this embodiment, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and can also be a module or a non-modular system. Furthermore, the various components in this embodiment can be integrated into a single processing unit, or each unit can exist physically separately, or two or more units can be integrated into a single unit. The aforementioned integrated units can be implemented in the form of hardware or software functional modules.

[0306] If the integrated unit is implemented as a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, this embodiment provides a computer-readable storage medium for use in decoder 180. The computer-readable storage medium stores a computer program that, when executed by a second processor, implements any of the methods described in the aforementioned embodiments.

[0307] Based on the composition of the above-mentioned decoder 180 and the computer-readable storage medium, Figure 19 is a schematic diagram of the specific hardware structure of a decoder provided in an embodiment of the present application. As shown in Figure 19, the decoder 180 may include: a second communication interface 1901, a second memory 1902 and a second processor 1903; each component is coupled together through a second bus system 1904. It can be understood that the second bus system 1904 is used to realize the connection and communication between these components. In addition to the data bus, the second bus system 1904 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, various buses are labeled as the second bus system 1904 in Figure 19. Among them, the second communication interface 1901 is used to receive and send signals in the process of sending and receiving information between other external network elements;

[0308] The second memory 1902 is used to store computer programs that can be run on the second processor 1903;

[0309] The second processor 1903 is configured to, when running the computer program, execute:

[0310] Determine an initial reference block of the current block and a macropixel co-located block corresponding to the initial reference block; perform searches based on the initial reference block and the macropixel co-located block to determine target motion parameters; and determine a prediction value of the current block based on the target motion parameters.

[0311] Optionally, as another embodiment, the second processor 1903 is further configured to execute any one of the methods described in the foregoing embodiments when running the computer program.

[0312] It can be understood that the hardware functions of the second memory 1902 are similar to those of the first memory 1702, and the hardware functions of the second processor 1903 are similar to those of the first processor 1703; they will not be described in detail here.

[0313] This embodiment provides a decoder. In this decoder, due to the spatial correlation between macropixels in a light field video image, when determining target motion parameters, the search range involved in the operation is expanded to include adjacent macropixels. This can improve the accuracy of motion vectors in light field video encoding and decoding, thereby improving the compression efficiency of the light field video and further improving the encoding and decoding performance of the light field video.

[0314] In yet another embodiment of the present application, FIG20 is a schematic diagram of the composition structure of a coding and decoding system provided in an embodiment of the present application. As shown in FIG20 , the coding and decoding system 2000 may include an encoder 2001 and a decoder 2002 .

[0315] In the embodiment of the present application, the encoder 2001 may be the encoder described in any one of the aforementioned embodiments, and the decoder 2002 may be the decoder described in any one of the aforementioned embodiments.

[0316] It should be noted that, in this application, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0317] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0318] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0319] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0320] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0321] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims. Industrial Applicability

[0322] In the embodiments of the present application, both at the encoding and decoding ends, the initial reference block of the current block and the macropixel co-located block corresponding to the initial reference block are first determined; then, searches are performed based on the initial reference block and the macropixel co-located block to determine target motion parameters; and then, based on the target motion parameters, a prediction value for the current block is determined. In this way, because there is spatial correlation between macropixels in a light field video image, when determining the target motion parameters, the accuracy of motion vectors in light field video encoding and decoding can be improved by expanding the search range to include adjacent macropixels, thereby saving bit rate, improving the compression efficiency of light field videos, and thus enhancing the encoding and decoding performance of light field videos.

Claims

1. A decoding method, applied to a decoder, the method comprising: Determine an initial reference block of the current block, and determine a macropixel co-located block corresponding to the initial reference block; Searching respectively according to the initial reference block and the macro-pixel co-located block to determine target motion parameters; A prediction value of the current block is determined according to the target motion parameter.

2. The method according to claim 1, wherein: The determining of the initial reference block of the current block comprises: Decoding a bitstream to determine a prediction parameter of the current block; An initial reference block of the current block is determined according to the prediction parameters.

3. The method according to claim 2, wherein: The prediction parameters include inter-frame prediction identification parameters; the method further includes: It is determined that the inter-frame prediction identification parameter indicates that the prediction method used for decoding the current block is bidirectional prediction.

4. The method according to claim 3, wherein: The prediction parameters further include a motion vector and a reference image index, wherein the reference image index includes a first reference image index and a second reference image index; and determining the initial reference block of the current block according to the prediction parameters includes: Determine a first reference block of the current block according to a first reference image indicated by the first reference image index and a first initial motion vector in the motion vectors; Determine a second reference block of the current block according to a second reference image indicated by the second reference image index and a second initial motion vector in the motion vector; The first reference block and the second reference block are used as initial reference blocks of the current block.

5. The method according to claim 3, wherein: The prediction parameters further include a motion vector difference and a reference image index, wherein the reference image index includes a first reference image index and a second reference image index; The step of determining the initial reference block of the current block according to the prediction parameter comprises: Determine a first initial motion vector prediction value of the current block in the first reference image and a second initial motion vector prediction value of the current block in the second reference image; Determine a first initial motion vector according to a first motion vector difference in the motion vector difference and the first initial motion vector prediction value, and determine a first reference block of the current block according to a first reference image indicated by the first reference image index and the first initial motion vector; Determine a second initial motion vector according to a second motion vector difference in the motion vector difference and the second initial motion vector prediction value, and determine a second reference block of the current block according to a second reference image indicated by the second reference image index and the second initial motion vector; The first reference block and the second reference block are used as initial reference blocks of the current block.

6. The method according to claim 3, wherein: The prediction parameters further include a motion vector index and a reference image index, wherein the reference image index includes a first reference image index and a second reference image index; and determining the initial reference block of the current block according to the prediction parameters includes: Constructing a motion vector candidate list of the current block; Determine a first initial motion vector according to the motion vector candidate list and a first motion vector index in the motion vector index, and determine a first reference block of the current block according to a first reference image indicated by the first reference image index and the first initial motion vector; Determine a second initial motion vector according to the motion vector candidate list and a second motion vector index in the motion vector index, and determine a second reference block of the current block according to a second reference image indicated by the second reference image index and the second initial motion vector; The first reference block and the second reference block are used as initial reference blocks of the current block.

7. The method according to any one of claims 4 to 6, wherein: The determining of the macro pixel co-located block corresponding to the initial reference block comprises: Determine, according to the first reference block, at least one first macropixel co-located block of the first reference block in the first reference image; At least one second macropixel co-located block of the second reference block in the second reference image is determined according to the second reference block.

8. The method according to claim 7, wherein: The determining, according to the first reference block, at least one first macropixel co-located block of the first reference block in the first reference image comprises: Determine a first current macropixel and at least one first candidate macropixel in the first reference image, wherein the first current macropixel includes at least a portion of the first reference block; Determine the position of each of the at least one first candidate macropixel according to the position relationship of the first reference block in the first current macropixel to obtain the at least one first macropixel co-located block; The determining, according to the second reference block, at least one second macropixel co-located block of the second reference block in the second reference image comprises: Determine a second current macropixel and at least one second candidate macropixel in the second reference image, wherein the second current macropixel includes at least a portion of the second reference block; According to the positional relationship of the second reference block in the second current macropixel, respective macropixel co-located blocks in the at least one second candidate macropixel are determined to obtain the at least one second macropixel co-located block.

9. The method according to claim 8, wherein: The step of searching respectively according to the initial reference block and the macro pixel co-located block to determine the target motion parameter comprises: Searching the preset areas of the corresponding macropixels according to the initial reference block and the macropixel co-located block respectively to determine a plurality of groups of candidate motion parameters, wherein each group of candidate motion parameters includes a first candidate motion vector and a second candidate motion vector; Performing cost calculations on the multiple groups of candidate motion parameters respectively to obtain multiple groups of cost results; A minimum cost result is determined from the multiple groups of cost results, and a group of candidate motion parameters corresponding to the minimum cost result is determined as the target motion parameters.

10. The method according to claim 9, wherein: The step of searching respectively in preset areas of the corresponding macropixels according to the initial reference block and the macropixel co-located block to determine multiple groups of candidate motion parameters includes: Taking the initial reference block as the center, searching in a first preset area of ​​macro pixels to which the initial reference block belongs to determine at least one set of first candidate motion parameters; Taking the macro pixel co-located block as the center, searching in a second preset area of ​​macro pixels to which the macro pixel co-located block belongs to determine at least one set of second candidate motion parameters; The at least one set of first candidate motion parameters and the at least one set of second candidate motion parameters are determined as the plurality of sets of candidate motion parameters.

11. The method according to claim 10, wherein: The searching in a first preset area of ​​macro pixels to which the initial reference block belongs to determine at least one set of first candidate motion parameters includes: Taking the first reference block as the center, searching in a first preset area of ​​the first current macropixel to determine at least one first candidate motion vector; and determining at least one second candidate motion vector corresponding to the second reference block according to the at least one first candidate motion vector and the first preset area of ​​the second current macropixel taking the second reference block as the center; The at least one set of first candidate motion parameters is determined based on the at least one first candidate motion vector and the at least one second candidate motion vector.

12. The method according to claim 11, wherein: A first offset between the first initial motion vector and the first candidate motion vector and a second offset between the second initial motion vector and the second candidate motion vector have a symmetrical relationship.

13. The method according to claim 9, wherein: The cost calculation is performed on the multiple groups of candidate motion parameters respectively to obtain multiple groups of cost results, including: Based on each group of candidate motion parameters, determine a first candidate block corresponding to the first candidate motion vector and a second candidate block corresponding to the second candidate motion vector; The cost results of the first candidate block and the second candidate block are calculated according to a preset criterion to obtain the cost results corresponding to each group of candidate motion parameters.

14. The method according to claim 9, wherein: The cost calculation is performed on the multiple groups of candidate motion parameters respectively to obtain multiple groups of cost results, including: Based on each group of candidate motion parameters, determine a first candidate block corresponding to the first candidate motion vector and a second candidate block corresponding to the second candidate motion vector, and perform weighted calculation based on the first candidate block and the second candidate block to determine a candidate prediction block; Performing weighted calculation based on the first reference block and the second reference block to determine an initial prediction block; The cost results of the candidate prediction blocks and the initial prediction blocks are calculated according to a preset criterion to obtain the cost results corresponding to each group of candidate motion parameters.

15. The method according to claim 13 or 14, wherein: The preset criteria include at least one of the following: the sum of absolute errors SAD, the sum of absolute transformed errors SATD, the mean square error MSE, the sum of squared errors SSE, and the mean absolute error MAE.

16. The method according to any one of claims 1 to 15, wherein: The target motion parameters include a first target motion vector and a second target motion vector; and determining the prediction value of the current block according to the target motion parameters includes: Determining a first prediction block of the current block according to the first target motion vector; Determining a second prediction block of the current block according to the second target motion vector; A prediction value of the current block is determined according to the first prediction block and the second prediction block.

17. The method according to claim 16, wherein: The step of determining the prediction value of the current block according to the first prediction block and the second prediction block includes: A prediction value of the current block is determined by performing a weighted calculation based on the first prediction block and the second prediction block.

18. The method according to any one of claims 1 to 15, wherein: The method further comprises: Decoding a bitstream to determine a prediction difference value of the current block; A reconstructed value of the current block is determined according to the prediction difference value of the current block and the prediction value of the current block.

19. A coding method, applied to an encoder, the method comprising: Determine an initial reference block of the current block, and determine a macropixel co-located block corresponding to the initial reference block; Searching respectively according to the initial reference block and the macro-pixel co-located block to determine target motion parameters; A prediction value of the current block is determined according to the target motion parameter.

20. The method according to claim 19, wherein: The method further comprises: Determining a prediction difference value of the current block according to the prediction value of the current block; The prediction difference of the current block is coded, and the obtained coded bits are written into a bit stream.

21. The method according to claim 20, wherein: The step of determining the prediction difference value of the current block according to the prediction value of the current block includes: Determining an initial value of the current block; A prediction difference value of the current block is determined according to the initial value of the current block and the prediction value of the current block.

22. The method according to claim 19, wherein: The method further comprises: Determining prediction parameters of the current block; The prediction parameters of the current block are coded, and the obtained coded bits are written into a bit stream.

23. The method according to claim 22, wherein: The prediction parameter includes an inter-frame prediction identification parameter; and determining the prediction parameter of the current block includes: It is determined that the inter-frame prediction identification parameter indicates that the prediction method used to encode the current block is bidirectional prediction.

24. The method according to claim 23, wherein: The initial reference block includes a first reference block and a second reference block, the prediction parameter also includes a motion vector and a reference image index, and the determining the prediction parameter of the current block further includes: Determine, according to the first reference block, a first initial motion vector in a first reference image and a first reference image index for indicating the first reference image; determining, according to the second reference block, a second initial motion vector in a second reference image and a second reference image index for indicating the second reference image; The motion vector includes the first initial motion vector and the second initial motion vector, and the reference image index includes the first reference image index and the second reference image index.

25. The method according to claim 23, wherein: The initial reference block includes a first reference block and a second reference block, the prediction parameter also includes a motion vector difference and a reference image index, and the determining of the prediction parameter of the current block further includes: Determine, according to the first reference block, a first initial motion vector in a first reference image and a first reference image index for indicating the first reference image; and determine a first initial motion vector prediction value of the current block in the first reference image, and determine a first motion vector difference value according to the first initial motion vector and the first initial motion vector prediction value; Determine, according to the second reference block, a second initial motion vector in a second reference image and a second reference image index for indicating the second reference image; and determine a second initial motion vector prediction value of the current block in the second reference image, and determine a second motion vector difference value according to the second initial motion vector and the second initial motion vector prediction value; The motion vector difference includes the first motion vector difference and the second motion vector difference, and the reference image index includes the first reference image index and the second reference image index.

26. The method of claim 23, wherein: The initial reference block includes a first reference block and a second reference block, the prediction parameter also includes a motion vector index and a reference image index, and the determining the prediction parameter of the current block further includes: Constructing a motion vector candidate list of the current block; Determine, according to the first reference block, a first initial motion vector in a first reference image and a first reference image index indicating the first reference image, and determine a first motion vector index of the first initial motion vector in the motion vector candidate list; Determine, according to the second reference block, a second initial motion vector in a second reference image and a second reference image index indicating the second reference image, and determine a second motion vector index of the second initial motion vector in the motion vector candidate list; The motion vector index includes the first motion vector index and the second motion vector index, and the reference image index includes the first reference image index and the second reference image index.

27. The method according to any one of claims 24 to 26, wherein: The determining, according to the initial reference block, a macropixel co-located block corresponding to the initial reference block comprises: Determine, according to the first reference block, at least one first macropixel co-located block of the first reference block in the first reference image; At least one second macropixel co-located block of the second reference block in the second reference image is determined according to the second reference block.

28. The method according to claim 27, wherein: The determining, according to the first reference block, at least one first macropixel co-located block of the first reference block in the first reference image comprises: Determine a first current macropixel and at least one first candidate macropixel in the first reference image, wherein the first current macropixel includes at least a portion of the first reference block; Determine, according to the positional relationship of the first reference block in the first current macropixel, respective macropixel co-located blocks in the at least one first candidate macropixel to obtain the at least one first macropixel co-located block; The determining, according to the second reference block, at least one second macropixel co-located block of the second reference block in the second reference image comprises: Determine a second current macropixel and at least one second candidate macropixel in the second reference image, wherein the second current macropixel includes at least a portion of the second reference block; According to the positional relationship of the second reference block in the second current macropixel, respective macropixel co-located blocks in the at least one second candidate macropixel are determined to obtain the at least one second macropixel co-located block.

29. The method according to claim 28, wherein: The step of searching respectively according to the initial reference block and the macro pixel co-located block to determine the target motion parameter comprises: Searching the preset areas of the corresponding macropixels according to the initial reference block and the macropixel co-located block respectively to determine a plurality of groups of candidate motion parameters, wherein each group of candidate motion parameters includes a first candidate motion vector and a second candidate motion vector; Performing cost calculations on the multiple groups of candidate motion parameters respectively to obtain multiple groups of cost results; A minimum cost result is determined from the multiple groups of cost results, and a group of candidate motion parameters corresponding to the minimum cost result is determined as the target motion parameters.

30. The method of claim 29, wherein: The step of searching respectively in preset areas of the corresponding macropixels according to the initial reference block and the macropixel co-located block to determine multiple groups of candidate motion parameters includes: Taking the initial reference block as the center, searching in a first preset area of ​​macro pixels to which the initial reference block belongs to determine at least one set of first candidate motion parameters; Taking the macro pixel co-located block as the center, searching in a second preset area of ​​macro pixels to which the macro pixel co-located block belongs to determine at least one set of second candidate motion parameters; The at least one set of first candidate motion parameters and the at least one set of second candidate motion parameters are determined as the plurality of sets of candidate motion parameters.

31. The method according to claim 30, wherein: The searching in a first preset area of ​​macro pixels to which the initial reference block belongs to determine at least one set of first candidate motion parameters includes: Taking the first reference block as the center, searching in a first preset area of ​​the first current macropixel to determine at least one first candidate motion vector; and determining at least one second candidate motion vector corresponding to the second reference block according to the at least one first candidate motion vector and the first preset area of ​​the second current macropixel taking the second reference block as the center; The at least one set of first candidate motion parameters is determined based on the at least one first candidate motion vector and the at least one second candidate motion vector.

32. The method according to claim 31, wherein: A first offset between the first initial motion vector and the first candidate motion vector and a second offset between the second initial motion vector and the second candidate motion vector have a symmetrical relationship.

33. The method of claim 29, wherein: The cost calculation is performed on the multiple groups of candidate motion parameters respectively to obtain multiple groups of cost results, including: Based on each group of candidate motion parameters, determine a first candidate block corresponding to the first candidate motion vector and a second candidate block corresponding to the second candidate motion vector; The cost results of the first candidate block and the second candidate block are calculated according to a preset criterion to obtain the cost results corresponding to each group of candidate motion parameters.

34. The method of claim 29, wherein: The cost calculation is performed on the multiple groups of candidate motion parameters respectively to obtain multiple groups of cost results, including: Based on each group of candidate motion parameters, determine a first candidate block corresponding to the first candidate motion vector and a second candidate block corresponding to the second candidate motion vector, and perform weighted calculation based on the first candidate block and the second candidate block to determine a candidate prediction block; Performing weighted calculation based on the first reference block and the second reference block to determine an initial prediction block; The cost results of the candidate prediction blocks and the initial prediction blocks are calculated according to a preset criterion to obtain the cost results corresponding to each group of candidate motion parameters.

35. The method according to claim 33 or 34, wherein: The preset criteria include at least one of the following: the sum of absolute errors SAD, the sum of absolute transformed errors SATD, the mean square error MSE, the sum of squared errors SSE, and the mean absolute error MAE.

36. A method according to any one of claims 19 to 35, wherein: The target motion parameters include a first target motion vector and a second target motion vector; and determining the prediction value of the current block according to the target motion parameters includes: Determining a first prediction block of the current block according to the first target motion vector; Determining a second prediction block of the current block according to the second target motion vector; A prediction value of the current block is determined according to the first prediction block and the second prediction block.

37. The method of claim 36, wherein: The step of determining the prediction value of the current block according to the first prediction block and the second prediction block includes: A prediction value of the current block is determined by performing a weighted calculation based on the first prediction block and the second prediction block.

38. A code stream, wherein The code stream is generated by bit encoding according to the information to be encoded; wherein the information to be encoded includes at least one of the following: The prediction difference of the current block and the prediction parameters of the current block.

39. An encoder, comprising a first determining unit, a first searching unit and a first predicting unit, wherein: The first determination unit is configured to determine an initial reference block of the current block; and determine a macropixel co-located block corresponding to the initial reference block; The first search unit is configured to search according to the initial reference block and the macropixel co-located block respectively to determine the target motion parameter; The first prediction unit is configured to determine a prediction value of the current block according to the target motion parameter.

40. An encoder, comprising a first memory and a first processor, wherein: The first memory is used to store a computer program that can be run on the first processor; The first processor is configured to execute the method according to any one of claims 19 to 37 when running the computer program.

41. A decoder, comprising a second determination unit, a second search unit and a second prediction unit, wherein: The second determination unit is configured to determine an initial reference block of the current block; and determine a macropixel co-located block corresponding to the initial reference block; The second search unit is configured to search according to the initial reference block and the macropixel co-located block respectively to determine the target motion parameter; The second prediction unit is configured to determine a prediction value of the current block according to the target motion parameter.

42. A decoder comprising a second memory and a second processor, wherein: The second memory is used to store a computer program that can be run on the second processor; The second processor is configured to execute the method according to any one of claims 1 to 18 when running the computer program.

43. A computer-readable storage medium, wherein: The computer-readable storage medium stores a computer program, and when the computer program is executed by at least one processor, the method according to any one of claims 1 to 18 or the method according to any one of claims 19 to 37 is implemented.

Images