Encoding method, encoder and electronic equipment

Abstract

The invention provides a coding method, a coder and electronic equipment, and the coding method comprises the steps: carrying out the motion estimation of a current block in a current image, and determining a first motion parameter of the current block; the first motion parameter points to an initial position in a reference image of the current image; in a first search area in an initial macro pixel where the initial position is located, determining at least one first position except the initial position; a second motion parameter of the reference block is determined based on the initial position and the at least one first position. The encoding method can improve the motion estimation effect and encoding performance of the encoder.

Description

Coding method, encoder, and electronic device Technical Field

[0001] The embodiments of the present application relate to the field of coding and decoding technology, and more specifically, to a coding method, an encoder, and an electronic device. Background Art

[0002] Digital video compression technology mainly compresses huge digital image video data for easy transmission and storage.

[0003] With the surge in Internet videos and people's increasing demand for video clarity, although existing digital video compression standards can save a lot of video data, there is still a need to pursue better digital video compression technology to reduce the bandwidth and traffic pressure of digital video transmission.

[0004] Summary of the Invention

[0005] The present application provides a coding method, an encoder, and an electronic device, which can improve the motion estimation effect and coding performance of the encoder.

[0006] In a first aspect, the present application provides an encoding method, comprising:

[0007] Performing motion estimation on a current block in a current image to determine a first motion parameter of the current block, wherein the first motion parameter points to an initial position in a reference image of the current image;

[0008] Determine at least one first position other than the initial position within a first search area where the initial position is located; the first search area is smaller than or equal to an initial macropixel where the initial position is located;

[0009] Based on the initial position and the at least one first position, a second motion parameter of the reference block is determined.

[0010] In a second aspect, the present application provides an encoder, comprising:

[0011] an estimating unit, configured to perform motion estimation on a current block in a current image and determine a first motion parameter of the current block; the first motion parameter points to an initial position in a reference image of the current image;

[0012] A first determining unit is configured to determine at least one first position other than the initial position within a first search area where the initial position is located, wherein the first search area is smaller than or equal to an initial macropixel where the initial position is located;

[0013] The second determining unit is configured to determine a second motion parameter of the reference block based on the initial position and the at least one first position.

[0014] In a third aspect, the present application provides an encoder, comprising:

[0015] a processor adapted to implement computer instructions; and,

[0016] A computer-readable storage medium stores computer instructions, wherein the computer instructions are suitable for being loaded by a processor and executing the encoding method in the second aspect or its various implementations mentioned above.

[0017] In one implementation, there are one or more processors and one or more memories.

[0018] In one implementation, the computer-readable storage medium may be integrated with the processor, or the computer-readable storage medium may be provided separately from the processor.

[0019] In a fourth aspect, the present application provides a computer-readable storage medium storing computer instructions. When the computer instructions are read and executed by a processor of a computer device, the computer device executes the encoding method involved in the first aspect mentioned above.

[0020] In a fifth aspect, the present application provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the encoding method of the first aspect mentioned above.

[0021] In a sixth aspect, the present application provides a code stream, which is generated by the method described in the first aspect mentioned above.

[0022] Based on the above technical solution, for the encoding method provided in the embodiment of the present application, the position pointed to by the first motion parameter determined by motion estimation is used as the initial position, and then at least one first position other than the initial position is determined within the first search area within the initial macropixel where the initial position is located; based on the initial position and the at least one first position, the second motion parameter of the reference block is determined; this is equivalent to correcting the first motion parameter by introducing the at least one first position and obtaining the corrected second motion parameter, which can improve the accuracy of the second motion parameter, and thus improve the motion estimation effect and encoding performance of the encoder. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] FIG1 is a schematic block diagram of a coding and decoding system provided in an embodiment of the present application.

[0024] FIG2 is a schematic block diagram of an encoder provided in an embodiment of the present application.

[0025] FIG3 is a schematic structural diagram of the relationship between a coding tree unit and a coding unit provided in an embodiment of the present application.

[0026] FIG4 is a schematic block diagram of a decoder provided in an embodiment of the present application.

[0027] FIG5 is an example of a light field video transmission process provided by an embodiment of the present application.

[0028] FIG6 is an example of the initial position and the co-location position provided in an embodiment of the present application.

[0029] FIG. 7 is an example of a light field image provided by an embodiment of the present application.

[0030] FIG8 is a schematic flowchart of the encoding method provided in an embodiment of the present application.

[0031] FIG9 is an example of a search principle for at least one first position provided in an embodiment of the present application.

[0032] FIG10 is an example of the co-location provided in an embodiment of the present application.

[0033] FIG11 is an example of at least one layer provided by an embodiment of the present application.

[0034] FIG12 is another example of at least one layer provided by an embodiment of the present application.

[0035] FIG13 is an example of an embodiment of the present application providing a search spacing of a first search area that is equal to a search spacing used by a first layer.

[0036] FIG14 is an example of an embodiment of the present application providing a first search area where the search spacing is smaller than the search spacing used by the first layer.

[0037] FIG15 is a schematic block diagram of an encoder provided in an embodiment of the present application.

[0038] FIG16 is a schematic structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0039] The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings.

[0040] It should be noted that the terms used in the implementation method part of this application are only used to explain the specific embodiments of this application and are not intended to limit this application.

[0041] For example, the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The term "at least one" is only a description of the combination relationship of enumerated objects, indicating that one or more items may exist. For example, at least one of the following: A, B, C can mean the following combinations: A exists alone, B exists alone, C exists alone, A and B exist at the same time, A and C exist at the same time, B and C exist at the same time, and A, B, and C exist at the same time. The term "multiple" refers to two or more. The character " / " generally indicates that the related objects before and after are in an "or" relationship.

[0042] For another example, the term "corresponding" may indicate that there is a direct or indirect correspondence between the two, or that there is an association relationship between the two, or a relationship between indication and being indicated, configuration and being configured, etc. The term "indication" may be a direct indication, an indirect indication, or an indication of an association relationship. For example, A indicates B, which may mean that A directly indicates B, for example, B can be obtained through A; it may also mean that A indirectly indicates B, for example, A indicates C, and B can be obtained through C; it may also mean that there is an association relationship between A and B. The term "predefined" or "preconfigured" may refer to the pre-storage of corresponding codes, tables or other relevant information that can be used for indication in a device (for example, including an encoder or decoder), or it may refer to an agreement agreed upon by a protocol. "Protocol" may refer to any standard protocol in the field of encoding and decoding, and this application does not limit this. The term "when..." may be interpreted as "if" or "if" or "when..." or "in response to" and similar descriptions. Similarly, depending on the context, the phrase "if determined" or "if (stated condition or event) is detected" can be interpreted as "when determined" or "in response to determining" or "when (stated condition or event) is detected" or "in response to detecting (stated condition or event)" and similar descriptions. The terms "first", "second", "third", "fourth", "Ath", "Bth" and the like are used to distinguish different objects rather than to describe a specific order. The terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. Among them, digital video compression technology is mainly used to compress huge digital image video data for easy transmission and storage.

[0043] The solution provided by the embodiment of the present application can be applied to the field of digital compression technology.

[0044] For example, the solution provided in the embodiments of the present application can be applied in the field of video coding technology.

[0045] Among them, the field of video coding technology includes but is not limited to at least one of the following: image coding and decoding field, video coding and decoding field, hardware video coding and decoding field, dedicated circuit video coding and decoding field and real-time video coding and decoding field. In addition, the solution provided in the embodiment of the present application can be combined with the following standards: Audio Video Coding Standard (AVS), second-generation AVS standard (AVS2) or third-generation AVS standard (AVS3). For example, including but not limited to: H.264 / Audio Video Coding (AVC) standard, H.265 / High Efficiency Video Coding (HEVC) standard and H.266 / Versatile Video Coding (VVC) standard. In addition, the solution provided in the embodiment of the present application can be used for lossy compression of images, and can also be used for lossless compression of images. Among them, the lossless compression can be visually lossless compression or mathematically lossless compression.

[0046] The video coding standard may adopt a block-based hybrid coding framework.

[0047] The hybrid coding framework includes modules such as prediction, transform, quantization, entropy coding, and in-loop filtering. The prediction module includes intra-frame prediction and / or inter-frame prediction. Because adjacent pixels within a video frame are strongly correlated, intra-frame prediction is used in video coding and decoding to eliminate spatial redundancy between adjacent pixels. Intra-frame prediction only references information from the same frame to predict pixel information within the current block. Because adjacent frames in a video have strong similarities, inter-frame prediction is used in video coding and decoding to eliminate temporal redundancy between adjacent frames, thereby improving coding efficiency. Inter-frame prediction includes motion estimation and motion compensation. Inter-frame prediction references image information from different frames and uses motion estimation to search for the motion vector information that best matches the current block. Transformation converts the predicted image block to the frequency domain, redistributing the energy. Combined with quantization, it removes information that is insensitive to the human eye, eliminating visual redundancy. Entropy coding eliminates character redundancy based on the current context model and the probabilistic information of the binary bitstream.

[0048] The basic process of the video encoder is as follows:

[0049] The encoder first divides a frame of image into blocks; then predicts the current block in the current image to obtain the predicted block of the current block; then subtracts the predicted block from the original block of the current block to obtain the residual block; transforms and quantizes the residual block to obtain a quantization coefficient matrix; then entropy codes the quantization coefficient matrix to obtain the output bitstream.

[0050] The basic process of the video decoder is as follows:

[0051] The decoder predicts the current block to obtain a predicted block. It also parses the bitstream to obtain a quantization coefficient matrix, which it then dequantizes and inversely transforms to produce a residual block. The predicted and residual blocks are then combined to form a reconstructed block. The reconstructed blocks form a reconstructed image, which is then subjected to image-based or block-based loop filtering to produce the decoded image.

[0052] It is worth noting that the current block may be a current coding unit (CU) or a current prediction unit (PU), etc.

[0053] In addition, the encoder also needs to perform similar operations as the decoder to obtain a decoded image. The decoded image can be a reference frame for inter-frame prediction for subsequent frames. The block division information, prediction, transformation, quantization, entropy coding, loop filtering and other mode information or parameter information determined by the encoder need to be written into the bitstream if necessary. The decoder determines the same block division information, prediction, transformation, quantization, entropy coding, loop filtering and other mode information or parameter information as the encoder by parsing and analyzing the existing information, thereby ensuring that the decoded image obtained by the encoder and the decoder are the same. The decoded image obtained by the encoder is also usually called a reconstructed image. The codec can divide the current block into prediction units during prediction and into transformation units during transformation. The division of prediction units and transformation units can be different. The above is the basic process of the video codec under the block-based hybrid coding framework. With the development of technology, some modules or steps of the framework or process may be optimized, and this application does not make specific restrictions on this.

[0054] For ease of understanding, the video encoding and decoding system involved in the embodiment of the present application is first introduced with reference to FIG1 .

[0055] FIG1 is a schematic block diagram of a coding and decoding system involved in an embodiment of the present application.

[0056] As shown in FIG1 , the encoding and decoding system 100 includes an encoding device 110 and a decoding device 120 .

[0057] The encoding device 110 is used to encode (which can be understood as compressing) video or image data to generate a code stream, and transmit the code stream to the decoding device 120. The decoding device 120 decodes the code stream generated by the encoding device 110 to obtain decoded video or image data.

[0058] The encoding device 110 can be understood as a device capable of encoding a video or image, and the decoding device 120 can be understood as a device capable of decoding a video or image. The encoding device 110 can modulate the encoded data according to a communication standard and transmit the modulated data to the decoding device 120. The encoding device 110 or the decoding device 120 includes a wider range of devices, such as smartphones, desktop computers, mobile computing devices, notebook computers (e.g., laptops), tablet computers, set-top boxes, televisions, cameras, display devices, digital media players, video game consoles, and in-vehicle computers.

[0059] The encoding device 110 may transmit the encoded data (eg, a code stream) to the decoding device 120 via the channel 130 .

[0060] Channel 130 may include one or more media and / or devices capable of transmitting encoded data from encoding device 110 to decoding device 120. Channel 130 may include one or more communication media that enable encoding device 110 to transmit encoded data directly to decoding device 120 in real time. Communication media may include wireless communication media, such as radio frequency spectrum. Communication media may also include wired communication media, such as one or more physical transmission lines. Channel 130 may include a storage medium that can store data encoded by encoding device 110. Storage media include various locally accessible data storage media, such as optical disks, DVDs, and flash memory. Decoding device 120 may retrieve the encoded data from the storage medium. Channel 130 may also include a storage server that can store the data encoded by encoding device 110. Decoding device 120 may download the stored encoded data from the storage server. Alternatively, the storage server may store the encoded data and transmit it to decoding device 120, such as a web server (e.g., for a website) or a File Transfer Protocol (FTP) server.

[0061] The encoding device 110 includes an encoder 112 and an output interface 113 .

[0062] The output interface 113 may include a modulator / demodulator (modem) and / or a transmitter. The encoder 112 transmits the encoded data directly to the decoding device 120 via the output interface 113. The encoded data may also be stored on a storage medium or storage server for subsequent reading by the decoding device 120.

[0063] In addition to the encoder 112 and the input interface 113 , the encoding device 110 may also include a video source 111 or an image source.

[0064] Video source 111 may include at least one of a video capture device (e.g., a video camera), a video archive, a video input interface, and a computer graphics system. The video input interface is used to receive video data from a video content provider, and the computer graphics system is used to generate video data. Encoder 112 encodes the video data from video source 111 to generate a bitstream. The video data may include one or more pictures or a sequence of pictures. The bitstream contains encoding information for the picture or picture sequence in the form of a bitstream. The encoding information may include the encoded picture data and associated data. The associated data may include a sequence parameter set (SPS), a picture parameter set (PPS), and other syntax structures. An SPS may contain parameters applicable to one or more sequences. A PPS may contain parameters applicable to one or more pictures. A syntax structure is a set of zero or more syntax elements arranged in a specified order within the bitstream.

[0065] The decoding device 120 includes an input interface 121 and a decoder 122. The input interface 121 may include a receiver and / or a modem.

[0066] The decoding device 120 may include a display device 123 in addition to the input interface 121 and the decoder 122 .

[0067] The input interface 121 can receive the encoded data via the channel 130. The decoder 122 is configured to decode the encoded data to obtain decoded data and transmit the decoded data to the display device 123. The display device 123 displays the decoded data. The display device 123 can be integrated with the decoding device 120 or external to the decoding device 120. The display device 123 can include various display devices, such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or other types of display devices.

[0068] It should be understood that Figure 1 is only an example of the present application and should not be understood as a display of the present application. That is to say, the technical solution of the embodiment of the present application is not limited to the system framework shown in Figure 1. For example, the technology of the present application can also be applied to unilateral video encoding or unilateral video decoding.

[0069] The following is an introduction to the video encoding framework involved in the embodiments of the present application.

[0070] FIG2 is a schematic block diagram of a video encoder 200 according to an embodiment of the present application.

[0071] It should be understood that the video encoder 200 can be applied to image data in luminance and chrominance (YCbCr, YUV) format. For example, the YUV ratio can be 4:2:0, 4:2:2, or 4:4:4, where Y represents brightness (Luma), Cb (U) represents blue chrominance, Cr (V) represents red chrominance, and U and V represent chrominance (Chroma) used to describe color and saturation. For example, in terms of color format, 4:2:0 means that every 4 pixels have 4 luminance components and 2 chrominance components (YYYYCbCr), 4:2:2 means that every 4 pixels have 4 luminance components and 4 chrominance components (YYYYCbCrCbCr), and 4:4:4 represents full pixel display (YYYYCbCrCbCrCbCrCbCr). Of course, it can also be applied to image data in red-green-blue (RGB) format, but this application does not specifically limit this.

[0072] After the video encoder 200 reads the video stream, it can divide each frame of the video stream into several coding tree units (CTUs). In some examples, CTUs may be referred to as "tree blocks", "largest coding units" (LCUs) or "coding tree blocks" (CTBs). Each CTU can be associated with a pixel block of equal size within the image. Each pixel can correspond to one luminance (luminance or luma) sample and two chrominance (chroma) samples. Therefore, each CTU can be associated with one luminance sample block and two chrominance sample blocks. The size of a CTU can be, for example, 128×128, 64×64, 32×32, etc. Figure 3 is a schematic structural diagram of the relationship between the coding tree unit and the coding unit provided in an embodiment of the present application. As shown in Figure 3, a CTU can be further divided into several coding units (CUs) for encoding, and the CU can be a rectangular block or a square block. A CU can be further divided into prediction units (PUs) and transform units (TUs), allowing for separation of coding, prediction, and transform, and greater flexibility in processing. In one example, a CTU is divided into CUs using a tree (e.g., a quadtree), and a CU is divided into TUs and PUs using a tree (e.g., a quadtree).

[0073] The video encoder and video decoder can support various PU sizes.

[0074] Assuming that the size of a particular CU is 2N×2N, the video encoder and video decoder may support PU sizes of 2N×2N or N×N for intra prediction, and support symmetric PUs of 2N×2N, 2N×N, N×2N, N×N, or similar sizes for inter prediction. The video encoder and video decoder may also support asymmetric PUs of 2N×nU, 2N×nD, nL×2N, and nR×2N for inter prediction.

[0075] As shown in Figure 2, the video encoder 200 may include: a prediction unit 210, a residual unit 220, a transform / quantization unit 230, an inverse transform / quantization unit 240, a reconstruction unit 250, a loop filter unit 260, a decoded image buffer 270, and an entropy coding unit 280. It should be noted that the video encoder 200 may include more, fewer, or different functional components. In this application, the current block may be referred to as the current coding unit (CU) or the current prediction unit (PU), etc. The prediction block may also be referred to as a predicted image block or an image prediction block, and the reconstructed image block may also be referred to as a reconstructed block or an image reconstructed image block.

[0076] The prediction unit 210 includes an inter-frame prediction unit 211 and an intra-frame prediction unit 212. Because there is a strong correlation between adjacent pixels in a video image, intra-frame prediction is used in video coding and decoding to eliminate spatial redundancy between adjacent pixels. Because there is a strong similarity between adjacent images in a video, inter-frame prediction is used to eliminate temporal redundancy between adjacent images, thereby improving coding efficiency.

[0077] The inter-frame prediction unit 211 is capable of inter-frame prediction, which includes motion estimation and motion compensation. It can reference image information from different frames. Inter-frame prediction uses motion information to find a reference block from a reference frame and then generates a prediction block based on the reference block to eliminate temporal redundancy. The reference frame can be a P-frame and / or a B-frame. P-frames refer to forward-predicted frames, and B-frames refer to bidirectionally predicted frames. Inter-frame prediction uses motion information to find a reference block and then generates a prediction block based on the reference block. Motion information includes the frame list to which the reference frame belongs, the frame index, and the motion vector. The motion vector can be integer-pixel or fractional-pixel. If the motion vector is fractional-pixel, interpolation filtering is used in the reference frame to generate the required fractional-pixel block. The reference block is the integer-pixel or fractional-pixel block found based on the motion vector. Some technologies directly use the reference block as the prediction block, while others further process the reference block to generate a prediction block. Reprocessing a reference block to generate a prediction block can also be understood as using the reference block as the prediction block and then processing the prediction block to generate a new prediction block.

[0078] The intra-frame prediction unit 212 only refers to the information of the same frame image to predict the pixel information in the current code image block to eliminate spatial redundancy. The reference frame used for intra-frame prediction can be an I frame.

[0079] Intra-frame prediction uses multiple prediction modes. Both angular and non-angular prediction modes can be used to predict the image block to be coded, yielding a prediction block. Based on the rate-distortion information calculated from the prediction block and the image block to be coded, the optimal prediction mode for the image block to be coded is selected and written into the bitstream for transmission to the decoder. The decoder parses the prediction mode, predicts the predicted block for the target decoding block, and overlays it with the time-domain residual block obtained from the bitstream to reconstruct the block.

[0080] Taking the H series of international digital video coding standards as an example, the H.264 / AVC standard has 8 angle prediction modes and 1 non-angle prediction mode, and H.265 / HEVC is expanded to 33 angle prediction modes and 2 non-angle prediction modes. The intra-frame prediction modes used by HEVC are planar mode, direct current (DC) and 33 angle modes, a total of 35 prediction modes. The intra-frame modes used by VVC are planar, DC and 65 angle modes, a total of 67 prediction modes, which include traditional prediction modes and non-traditional prediction modes. Non-traditional prediction modes may include matrix weighted intra-frame prediction (MIP) mode. Traditional prediction modes include: planar mode with mode number 0, DC mode with mode number 1, and angle prediction modes with mode numbers 2 to 66. It should be noted that with the increase of angle modes, the prediction results of intra-frame prediction will be more accurate and more in line with the needs of the development of high-definition and ultra-high-definition digital video. The above-mentioned intra-frame prediction mode is only an example of this application and should not limit this application.

[0081] The residual unit 220 may generate a residual block for the CU based on the pixel blocks of the CU and the prediction blocks of the PUs of the CU. For example, the residual unit 220 may generate the residual block for the CU such that each sample in the residual block has a value equal to the difference between the sample in the pixel blocks of the CU and the corresponding sample in the prediction blocks of the PUs of the CU.

[0082] The transform / quantization unit 230 may quantize the transform coefficients. The transform / quantization unit 230 may quantize the transform coefficients associated with the TUs of a CU based on a quantization parameter (QP) value associated with the CU. The video encoder 200 may adjust the degree of quantization applied to the transform coefficients associated with the CU by adjusting the QP value associated with the CU.

[0083] The inverse transform / quantization unit 240 may apply inverse quantization and inverse transform, respectively, to the quantized transform coefficients to reconstruct a residual block from the quantized transform coefficients.

[0084] Reconstruction unit 250 may add samples of the reconstructed residual block to corresponding samples of one or more prediction blocks generated by prediction unit 210 to generate a reconstructed image block associated with the TU. By reconstructing the sample blocks of each TU of a CU in this manner, video encoder 200 can reconstruct the pixel blocks of the CU.

[0085] The loop filter unit 260 processes the inverse-transformed and inverse-quantized pixels to compensate for distortion and provide a better reference for subsequent pixel encoding. For example, it can perform deblocking filtering to reduce the blocking artifacts of pixel blocks associated with the CU. In some embodiments, the loop filter unit 260 includes a deblocking filter (DBF) unit and a sample adaptive offset / adaptive loop filter (SAO / ALF) unit. The DBF unit is used to remove blocking artifacts, while the SAO / ALF unit is used to remove ringing artifacts.

[0086] The decoded image buffer 270 may store reconstructed pixel blocks.

[0087] The inter-frame prediction unit 211 can use the reference image containing the reconstructed pixel block in the decoded image buffer 270 to perform inter-frame prediction on the PU of other images. In addition, the intra-frame prediction unit 212 can use the reconstructed pixel block in the decoded image buffer 270 to perform intra-frame prediction on other PUs in the same image as the CU.

[0088] The entropy coding unit 280 may receive the quantized transform coefficients from the transform / quantization unit 230. The entropy coding unit 280 may perform one or more entropy coding operations on the quantized transform coefficients to generate entropy-coded data.

[0089] FIG4 is a schematic block diagram of a video decoder according to an embodiment of the present application.

[0090] 4 , the video decoder 300 includes an entropy decoding unit 310, a prediction unit 320, an inverse quantization / transformation unit 330, a reconstruction unit 340, a loop filter unit 350, and a decoded picture buffer 360. It should be noted that the video decoder 300 may include more, fewer, or different functional components.

[0091] The video decoder 300 may receive a bitstream. The entropy decoding unit 310 may parse the bitstream to extract syntax elements from the bitstream. As part of parsing the bitstream, the entropy decoding unit 310 may parse the entropy-encoded syntax elements in the bitstream. The prediction unit 320, the inverse quantization / transform unit 330, the reconstruction unit 340, and the loop filter unit 350 may decode the video data based on the syntax elements extracted from the bitstream, thereby generating decoded video data.

[0092] The prediction unit 320 includes an intra prediction unit 322 and an inter prediction unit 321 .

[0093] The intra-prediction unit 322 may perform intra-prediction to generate a prediction block for the PU. The intra-prediction unit 322 may use an intra-prediction mode to generate a prediction block for the PU based on pixel blocks of spatially neighboring PUs. The intra-prediction unit 322 may also determine the intra-prediction mode for the PU based on one or more syntax elements parsed from the codestream.

[0094] The inter-frame prediction unit 321 may construct a first reference picture list (List 0) and a second reference picture list (List 1) based on syntax elements parsed from the codestream. In addition, if a PU is encoded using inter-frame prediction, the entropy decoding unit 310 may parse the motion information of the PU. The inter-frame prediction unit 321 may determine one or more reference blocks for the PU based on the motion information of the PU. The inter-frame prediction unit 321 may generate a prediction block for the PU based on the one or more reference blocks of the PU.

[0095] The inverse quantization / transform unit 330 may inversely quantize (i.e., dequantize) the transform coefficients associated with the TU. The inverse quantization / transform unit 330 may use the QP value associated with the CU of the TU to determine the degree of quantization. After inverse quantizing the transform coefficients, the inverse quantization / transform unit 330 may apply one or more inverse transforms to the inverse quantized transform coefficients to generate a residual block associated with the TU.

[0096] The reconstruction unit 340 uses the residual block associated with the TU of the CU and the prediction block of the PU of the CU to reconstruct the pixel block of the CU. For example, the reconstruction unit 340 can add samples of the residual block to corresponding samples of the prediction block to reconstruct the pixel block of the CU to obtain a reconstructed image block.

[0097] The loop filtering unit 350 may perform a deblocking filtering operation to reduce blocking artifacts of pixel blocks associated with a CU.

[0098] The video decoder 300 may store the reconstructed image of the CU in the decoded image buffer 360. The video decoder 300 may use the reconstructed image in the decoded image buffer 360 as a reference image for subsequent prediction, or transmit the reconstructed image to a display device for presentation.

[0099] Combining Figures 2 and 4, the basic process of video encoding and decoding is as follows:

[0100] At the encoding end, a frame of an image is divided into image blocks. For the current block, the prediction unit 210 uses intra-frame prediction or inter-frame prediction to predict the prediction block of the current block (i.e., the block to be encoded). The residual unit 220 can calculate a residual block based on the predicted block and the original block of the current block (i.e., the block to be encoded), which is the difference between the predicted block and the original block. This residual block can also be referred to as residual information. This residual block is transformed and quantized by the transform / quantization unit 230, thereby removing information that is insensitive to the human eye and eliminating visual redundancy. Optionally, the residual block before transformation and quantization by the transform / quantization unit 230 can be referred to as a time-domain residual block, and the time-domain residual block after transformation and quantization by the transform / quantization unit 230 can be referred to as a frequency residual block or a frequency-domain residual block. The entropy coding unit 280 receives the quantized change coefficients output by the change quantization unit 230 and can perform entropy coding on the quantized change coefficients to output a bitstream. For example, the entropy coding unit 280 can eliminate character redundancy based on the target context model and probability information of the binary bitstream.

[0101] At the decoding end, the entropy decoding unit 310 can parse the code stream to obtain the prediction information, quantization coefficient matrix, etc. of the current block (i.e., the block to be decoded). The prediction unit 320 uses intra-frame prediction or inter-frame prediction based on the prediction information to predict the prediction block of the current block (i.e., the block to be decoded). The inverse quantization / transformation unit 330 uses the quantization coefficient matrix obtained from the code stream to inverse quantize and inverse transform the quantization coefficient matrix to obtain a residual block. The reconstruction unit 340 adds the prediction block and the residual block to obtain a reconstructed block. The reconstructed blocks constitute a reconstructed image, and the loop filtering unit 350 performs loop filtering on the reconstructed image based on the image or block to obtain a decoded image. It is worth noting that the encoding end also needs to use operations similar to those of the decoder to obtain a decoded image. The decoded image can also be called a reconstructed image, and the reconstructed image can be a subsequent frame as a reference frame for inter-frame prediction.

[0102] In addition, the block division information determined by the encoder, as well as mode information or parameter information such as prediction, transform, quantization, entropy coding, and loop filtering, etc., are carried in the bitstream when necessary. The decoder parses the bitstream and analyzes the existing information to determine the same block division information, prediction, transform, quantization, entropy coding, loop filtering, etc. mode information or parameter information as the encoder, thereby ensuring that the decoded image obtained by the encoder and the decoder are the same.

[0103] It should be noted that, due to the need for parallel processing, images can be divided into slices, etc., and slices in the same image can be processed in parallel, that is, there is no data dependency between them. The term "frame" can be understood as an image or a slice, etc. In addition, of course, Figures 1 to 4 are only examples of the present application and should not be understood as limiting the present application. In other alternative embodiments, the decoding method and encoding method provided in the embodiments of the present application can also be applied to any other type of coding and decoding system, coding framework or decoding framework that meets its application conditions. For example, with the development of technology, some modules in the system or framework mentioned above or some steps in the above process may be optimized. In this case, the decoding method and encoding method provided in the embodiments of the present application can also be applied to the system, framework and process optimized thereon.

[0104] This application is applicable to the encoding and decoding of light field videos.

[0105] Light field video, captured by multiple cameras or camera arrays, is a research topic within the MPEG Lenslet Video Coding (LVC) Working Group. Unlike conventional camera imaging models, light field video uses a microlens array in front of the imaging plane. This allows light from the same point on the object plane to be captured simultaneously by multiple microlenses, effectively capturing the same point from multiple angles.

[0106] FIG5 is an example of a light field video transmission process provided by an embodiment of the present application.

[0107] As shown in FIG5 , a light field video can be obtained by a capture / shooting device (e.g., a light field camera), processed into a light field video in a specific data format, and input into a codec. The codec encodes and decodes the light field video and outputs the light field video in a specific data format. The display device receives the light field video in the specific data format and displays it.

[0108] Due to its special imaging model, the visual effect of light field images is very different from that of traditional pictures, which also leads to the fact that compression methods for general images or videos are not effective when processing light field images or videos. The emergence of the LVC working group is to solve this problem and study compression methods that are more suitable for light field videos. However, it is worth noting that LVC can still use conventional video codec tools (AVC, HEVC or VVC, etc.) in the encoding and / or decoding part. For example, LVC can use the system described in Figure 1 or the framework shown in Figures 2 to 3.

[0109] The following describes the technologies related to the technical solutions provided in the embodiments of this application.

[0110] (1) Improvement of motion estimation search algorithm.

[0111] To improve the compression performance of light field videos, the arrangement of macropixels in light field images can be used to improve the motion estimation search algorithm in the inter-frame prediction stage. Motion estimation refers to finding a best-matching block in the reference image that minimizes the rate distortion of the current coded block in the current image. The optimization goal of the search algorithm is to find the best-matching block more quickly and accurately and point to it with the final motion vector. A specific improvement to the motion estimation search algorithm can be as follows: in the motion estimation stage, the position indicated by the preliminary predicted motion vector is used as the initial position (also called the initial point, initial search position, or initial search point). Within a certain search range, the co-location position within each macropixel (i.e., the location of the co-location point) is searched sequentially according to the macropixel arrangement template; then, the best position is selected between the initial position and the co-location position.

[0112] Among them, the position of the co-location position in the macro-pixel to which it belongs is the same as the position of the initial position in the macro-pixel to which it belongs. The spacing of the co-location positions is the spacing of macro-pixels. For example, the position of the image block corresponding to the co-location position in the macro-pixel to which it belongs is the same as the position of the image block corresponding to the initial position in the macro-pixel to which it belongs. The image block corresponding to the co-location position can be an image block with the upper left corner (or lower left corner, upper right corner, lower right corner, center position or other specific position) as the co-location position; similarly, the image block corresponding to the initial position can be an image block with the upper left corner (or lower left corner, upper right corner, lower right corner, center position or other specific position) as the initial position.

[0113] FIG6 is an example of the initial position and the co-location position provided in an embodiment of the present application.

[0114] As shown in FIG6 , the search range of the co-location position includes a plurality of macropixels, and the plurality of macropixels includes an initial macropixel, and the initial macropixel includes an image block corresponding to the initial position. In addition, for any macropixel other than the initial macropixel among the plurality of macropixels, the arbitrary macropixel includes an image block corresponding to the co-location position. The position of the image block corresponding to any co-location position in the macropixel where it is located is the same as the position of the initial position in the initial macropixel. Specifically, as shown in (a) in FIG6 , the image block corresponding to the initial position and the image block corresponding to the co-location position are both located in the middle of the macropixel to which they belong, and as shown in (b) in FIG6 , the image block corresponding to the initial position and the image block corresponding to the co-location position are both located in the lower right corner of the macropixel to which they belong.

[0115] FIG. 7 is an example of a light field image provided by an embodiment of the present application.

[0116] As shown in Figure 7, since light field images are composed of a series of regularly arranged macropixels, there is a strong correlation between adjacent macropixels based on the imaging principle of light field cameras. Therefore, during the motion estimation search process, searching for the collocated positions of each macropixel can more fully utilize the correlation of the light field image, resulting in more efficient compression performance.

[0117] It should be understood that the term “macro pixel” may also be referred to as “micro image”, “micro lens image” or other terms with similar meanings.

[0118] However, although this improved method takes into account the macro-pixel arrangement rules of the light field image, since the best candidates for the matching blocks are not necessarily arranged strictly according to the macro-pixel spacing, searching only for the same-site positions within the macro-pixel may miss some local optimal points. In view of this, the present application provides a coding method that can fully utilize the correlation of the light field image to a greater extent, thereby further optimizing the motion estimation effect of light field video compression. Specifically, the encoder can search for at least one first position in a first search area within the initial macro-pixel where the initial position is located, and optimize the initial position based on the at least one first position; optionally, the encoder can also search for the same-site position of the initial position based on the search for at least one first position, and then optimize the initial position based on the same-site position; optionally, the encoder can also search for at least one second position in a second search area within the macro-pixel where the same-site position is located based on the search for the same-site position, and then optimize the initial position based on the at least one second position. For example, the encoder can appropriately add a local grid search on the basis of searching for the same-site position within the macropixel, that is, offset the candidate reference block position obtained based on the same-site position within the macropixel to maximize the correlation with the current block, thereby further optimizing the motion estimation effect of light field video compression.

[0119] FIG8 is a schematic flow chart of an encoding method 400 provided in an embodiment of the present application. It should be understood that the encoding method 400 can be executed by an encoder. For example, the encoding method 400 can be executed by the encoding device 110 or encoder 112 shown in FIG1 . Another example is that the encoding method 400 can be executed by the video encoder 200 shown in FIG2 . For ease of description, the following exemplary description of the encoding method 400 is provided using an encoder as the execution subject.

[0120] As shown in FIG8 , the encoding method 400 may include:

[0121] S410 , the encoder performs motion estimation on a current block in a current image, and determines a first motion parameter of the current block; the first motion parameter points to an initial position in a reference image of the current image.

[0122] Exemplarily, the current image and the reference image of the current image are light field images.

[0123] Exemplarily, the encoder may adopt an advanced motion vector prediction model (AMVP) mode or other types of prediction modes to perform motion estimation on the current block to obtain the first motion parameter.

[0124] Exemplarily, the initial position may be a position of a reference block pointed to by the first motion parameter in the reference image. Alternatively, the first motion parameter points to a reference block in the reference image, and the initial position is a position of the reference block. For example, the initial position may be the upper left corner, lower left corner, upper right corner, lower right corner, center, or other specific position of the reference block.

[0125] Exemplarily, the initial position may be the position of the pixel to which the first motion parameter points in the reference image.

[0126] It is worth noting that the term "position" can be equivalently replaced by "pixel point," "the location of the pixel point," or other descriptions with similar meanings, and this application does not specifically limit this. For example, the initial position can be equivalently replaced by the initial pixel point, the location of the initial pixel point, or other terms with similar meanings.

[0127] S420: The encoder determines at least one first position other than the initial position within a first search area where the initial position is located; the first search area is smaller than or equal to an initial macropixel where the initial position is located.

[0128] Exemplarily, the first search area includes an area centered on the initial position.

[0129] Exemplarily, the center position of the first search area is the center, upper left corner, lower left corner, upper right corner, lower right corner or other position of the initial macro pixel.

[0130] Exemplarily, the range of the first search area may be any value. For example, the range of the first search area may be any positive integer.

[0131] Exemplarily, the first search area may be a predefined search area. For example, the first search area may be implemented by pre-storing a corresponding code, table, or other method for indicating relevant information in an encoder, or the first search area may be agreed upon or defined by a standard protocol.

[0132] Exemplarily, the first search area may be equal to the initial macropixel, in which case the concept of the first search area may not be introduced. That is, the encoder determines the at least one first position within the initial macropixel.

[0133] Exemplarily, the first search area may be smaller than the initial macropixel, in which case the first search area may also be referred to as a local search area, that is, the encoder determines the at least one first position within a local area within the initial macropixel.

[0134] Exemplarily, the encoder may search for the at least one first position within the first search area using any search method. For example, the encoder may search for the at least one first position within the first search area using a grid search method (also referred to as a mesh search method). For another example, the encoder may search for the at least one first position within the first search area using a random sampling method.

[0135] FIG9 is an example of a search principle for at least one first position provided in an embodiment of the present application.

[0136] As shown in Figure 9, the encoder can search for the at least one first position within the first search area using a grid search method. Specifically, assuming that the first search area includes 7×7 pixels and the search spacing of the grid search is 1 pixel, the encoder determines the pixel points at the intersection of the 1st, 3rd, 5th, and 7th rows and the 1st, 3rd, 5th, and 7th columns of the 7×7 pixels as the searched 4×4 pixels, i.e., the at least one first position.

[0137] S430: The encoder determines a second motion parameter of the reference block based on the initial position and the at least one first position.

[0138] Exemplarily, the encoder may determine an optimal position among the initial position and the at least one first position, and determine a motion parameter pointing to the optimal position as the second motion parameter.

[0139] For example, the encoder may consider other positions based on the initial position and the at least one first position. That is, the encoder may determine an optimal position among the initial position, the at least one first position, and the other positions to be considered, and determine the motion parameters pointing to the optimal position as the second motion parameters. For example, the other positions may include one or more co-located positions.

[0140] Exemplarily, the second motion parameter may be a final motion parameter.

[0141] Exemplarily, the second motion parameter may be a motion parameter that needs to be further optimized, that is, the encoder may further process the second motion parameter to obtain the final motion parameter.

[0142] It is worth noting that the term "motion parameter" is intended to include a motion vector or other motion parameter that can represent a position of an image block in a reference image relative to a specific position. The specific position can be a position in the reference image that is the same as the position of the current block in the current image.

[0143] Exemplarily, after obtaining the second motion parameter, the encoder can use the reference block pointed to by the second motion parameter to predict the current block to obtain a predicted block of the current block, and then determine the residual block of the current block based on the original block of the current block and the predicted block of the current block; further, the encoder can quantize and entropy encode the residual block to obtain a code stream, such as a code stream of a light field image or a light field video.

[0144] In this embodiment, the position pointed to by the first motion parameter determined by motion estimation is used as the initial position, and then at least one first position other than the initial position is determined within a first search area within the initial macropixel where the initial position is located; based on the initial position and the at least one first position, the second motion parameter of the reference block is determined; this is equivalent to correcting the first motion parameter by introducing the at least one first position and obtaining the corrected second motion parameter, which can improve the accuracy of the second motion parameter, and thus improve the motion estimation effect and encoding performance of the encoder.

[0145] In some embodiments, the S430 may include:

[0146] The encoder determines the motion parameter pointing to the position with the minimum rate-distortion cost between the initial position and the at least one first position as the second motion parameter.

[0147] Exemplarily, the encoder first determines the rate-distortion cost corresponding to the initial position and the rate-distortion cost corresponding to any one of the at least one first position, and then determines the position with the smallest rate-distortion cost as the optimal position based on the rate-distortion cost corresponding to the initial position and the rate-distortion cost corresponding to any one of the first positions; then the encoder executes the motion parameters of the optimal position and determines them as the second motion parameters.

[0148] It is worth noting that the term "rate-distortion cost" is intended to be a parameter used to characterize distortion and bit rate. For example, "rate-distortion cost" may include Peak Signal to Noise Ratio (PSNR), Mean Structural Similarity Index Measure (MSSIM) or other parameters with similar functions. This application does not specifically limit the calculation method of "rate-distortion cost". Of course, in other alternative embodiments, "rate-distortion cost" can also be replaced by a parameter that is only used to characterize distortion or bit rate to reduce complexity, and even "rate-distortion cost" can be replaced by other types of indicators used to characterize encoding performance or decoding performance, and this application does not specifically limit this.

[0149] Exemplarily, when there are multiple positions between the initial position and the at least one first position that correspond to the positions with the lowest rate-distortion cost, the encoder may determine the motion parameters pointing to any one of the multiple positions as the second motion parameters. For example, the encoder may determine the motion parameters pointing to the position closest to the initial position among the multiple positions as the second motion parameters. For another example, the encoder may randomly select a position from the multiple positions and determine the motion parameters pointing to this position as the second motion parameters.

[0150] Of course, in other alternative embodiments, the encoder may determine the motion parameter that satisfies the first condition between the initial position and the at least one first position as the second motion parameter. For example, the first condition may include: the rate-distortion cost is less than or equal to a preset threshold, or the first condition may include: the rate-distortion cost is equal to a minimum value (for example, the minimum of the rate-distortion cost corresponding to the initial position and the rate-distortion cost corresponding to the at least one first position). The first condition may be a predefined condition. For example, the first condition may be implemented by pre-saving a corresponding code, table, or other method that can be used to indicate relevant information in the encoder, or the first condition may be agreed upon or defined by a standard protocol.

[0151] In some embodiments, the S430 may include:

[0152] Determine at least one collocated position of the initial position based on the position of the initial position within the initial macropixel; and determine the second motion parameter based on the initial position, the at least one first position, and the at least one collocated position.

[0153] Exemplarily, for any one of the at least one collocated position, a position of the any one collocated position within the macropixel where the any one collocated position is located is the same as a position of the initial position within the initial macropixel.

[0154] For example, the position of the image block corresponding to any one of the collocated positions within the macropixel where the collocated position is located is the same as the position of the image block corresponding to the initial position within the initial macropixel. The image block corresponding to any one of the collocated positions may be an image block whose upper left corner (or lower left corner, upper right corner, lower right corner, center position, or other specific position) is the collocated position; similarly, the image block corresponding to the initial position may be an image block whose upper left corner (or lower left corner, upper right corner, lower right corner, center position, or other specific position) is the initial position.

[0155] It should be understood that the image block corresponding to any one of the co-location positions and the image block corresponding to the initial position can be specifically referred to the example in FIG6 , and will not be described again here to avoid repetition.

[0156] In this embodiment, the co-location position is considered on the basis of the initial position and the at least one first position, that is, the first motion parameter is corrected by introducing the at least one first position and the at least one co-location position, and the corrected second motion parameter is obtained. Compared with the scheme of correcting the first motion parameter without considering the co-location position, the correlation between the reference block pointed to by the second motion parameter and the current block is increased, thereby improving the motion estimation effect and encoding performance of the encoder.

[0157] In some embodiments, at least one macropixel is determined within the search range of the at least one co-location position; and within any one of the at least one macropixel, the position that is the same as the position of the initial position within the initial macropixel is determined as the co-location position in the at least one co-location position.

[0158] Exemplarily, the search range of the at least one co-location position covers a plurality of macro pixels.

[0159] For example, when the search range of the at least one collocated position is a rectangular range, the rectangular range may be any value greater than the spacing between collocated positions. The spacing between collocated positions is equal to the spacing between macro pixels.

[0160] Exemplarily, the search range of the at least one co-location location may be a predefined search area. For example, the search range may be implemented by pre-storing a corresponding code, table, or other method for indicating relevant information in an encoder, or the search range of the at least one co-location location may be agreed upon or defined by a standard protocol.

[0161] Exemplarily, the encoder determines the at least one macropixel based on a macropixel arrangement rule, and then determines, within any macropixel within the at least one macropixel, a collocated position of the at least one collocated position within the any macropixel based on the position of the initial position within the initial macropixel. The arrangement rule includes, but is not limited to, at least one of the following: macropixel shape, macropixel spacing, and macropixel arrangement direction.

[0162] Exemplarily, within each of the at least one macropixel, the encoder determines a position identical to the position of the initial position within the initial macropixel as the collocated position in the at least one collocated position. In other words, the position of the collocated position within each macropixel is identical to the position of the initial position within the initial macropixel.

[0163] FIG10 is an example of the co-location provided in an embodiment of the present application.

[0164] As shown in Figure 10, assuming that the macropixels are hexagonal and closely arranged, the search range for the collocated location includes multiple macropixels, including the initial macropixel where the initial location is located. For any macropixel other than the initial macropixel among the multiple macropixels, the collocated location is included within the macropixel; the position of any collocated location within the macropixel is the same as the position of the initial location within the initial macropixel. Specifically, as shown in Figure 10, the initial location and any collocated location are both the center positions of the macropixels to which they belong.

[0165] In some embodiments, a motion parameter pointing to a position with the minimum rate-distortion cost among the initial position, the at least one first position, and the at least one co-located position is determined as the second motion parameter.

[0166] Exemplarily, the encoder first determines the rate-distortion cost corresponding to the initial position, the rate-distortion cost corresponding to any one of the at least one first position, and the rate-distortion cost corresponding to any one of the at least one collocated position, and then, based on the rate-distortion cost corresponding to the initial position, the rate-distortion cost corresponding to any one of the first positions, and the rate-distortion cost corresponding to any one of the collocated positions, determines the position with the minimum rate-distortion cost as the optimal position; then, the encoder executes the motion parameters of the optimal position and determines them as the second motion parameters. The term "rate-distortion cost" can be referred to in the above description and will not be repeated here to avoid repetition.

[0167] Exemplarily, when there are multiple positions with the lowest rate-distortion cost corresponding to the initial position, the at least one first position, and the at least one collocated position, the encoder may determine the motion parameters pointing to any one of the multiple positions as the second motion parameters. For example, the encoder may determine the motion parameters pointing to the position closest to the initial position among the multiple positions as the second motion parameters. For another example, the encoder may randomly select a position from the multiple positions and determine the motion parameters pointing to this position as the second motion parameters.

[0168] Of course, in other alternative embodiments, the encoder may determine the motion parameter that satisfies the second condition among the initial position, the at least one first position, and the at least one co-location position as the second motion parameter. For example, the second condition may include: the rate-distortion cost is less than or a preset threshold, or the second condition may include: the rate-distortion cost is equal to a minimum value (for example, the minimum value of the rate-distortion cost corresponding to the initial position, the rate-distortion cost corresponding to the at least one first position, and the rate-distortion cost corresponding to the at least one co-location position). The second condition may be a predefined condition. For example, the second condition may be implemented by pre-saving a corresponding code, table, or other method that can be used to indicate relevant information in the encoder, or the second condition may be agreed or defined by a standard protocol.

[0169] In some embodiments, within a second search area where a first co-location position among the at least one co-location position is located, at least one second position other than the first co-location position is determined; the second search area is smaller than or equal to the macropixel where the first co-location position is located; and the motion parameter pointing to the position with the smallest rate-distortion cost among the initial position, the at least one first position, the at least one co-location position, and the at least one second position is determined as the second motion parameter.

[0170] Exemplarily, the second search area includes an area centered around the first co-location position.

[0171] Exemplarily, the center position of the second search area is the center, upper left corner, lower left corner, upper right corner or lower right corner or other position of the macro pixel where the first co-location position is located.

[0172] Exemplarily, the range of the second search area may be any value. For example, the range of the second search area may be any positive integer.

[0173] Exemplarily, the second search area may be a predefined search area. For example, the second search area may be implemented by pre-storing a corresponding code, table, or other method for indicating relevant information in an encoder, or the second search area may be agreed upon or defined by a standard protocol.

[0174] Exemplarily, the second search area may be equal to the macropixel where the first collocated position is located. In this case, the concept of the second search area may not be introduced. That is, the encoder determines the at least one second position within the macropixel where the first collocated position is located.

[0175] Exemplarily, the second search area may be smaller than the macropixel where the first collocated position is located. In this case, the second search area may also be referred to as a local search area. That is, the encoder determines the at least one second position within a local area within the macropixel where the first collocated position is located.

[0176] Exemplarily, the encoder can search for the at least one second position in the second search area by any search method. For example, the encoder can search for the at least one second position in the second search area by a grid search method (also referred to as a grid search method). For another example, the encoder can search for the at least one second position in the second search area by a random sampling method. It is worth noting that the way the encoder searches for the at least one second position in the second search area is the same as or different from the way the encoder searches for the at least one first position in the first search area. For details, please refer to the scheme involved in Figure 9 above. To avoid repetition, it will not be described here.

[0177] Exemplarily, the encoder first determines the rate-distortion cost corresponding to the initial position, the rate-distortion cost corresponding to any one of the at least one first position, the rate-distortion cost corresponding to any one of the at least one co-location position, and the rate-distortion cost corresponding to any one of the at least one second position, and then, based on the rate-distortion cost corresponding to the initial position, the rate-distortion cost corresponding to any one of the first position, the rate-distortion cost corresponding to any one of the co-location position, and the rate-distortion cost corresponding to any one of the second position, determines the position with the minimum rate-distortion cost as the optimal position; then, the encoder executes the motion parameters of the optimal position and determines them as the second motion parameters. The term "rate-distortion cost" can be referred to the description above, and will not be repeated here to avoid repetition.

[0178] Exemplarily, when there are multiple positions corresponding to the minimum rate-distortion cost among the initial position, the at least one first position, the at least one collocated position, and the at least one second position, the encoder may determine the motion parameters pointing to any one of the multiple positions as the second motion parameters. For example, the encoder may determine the motion parameters pointing to the position closest to the initial position among the multiple positions as the second motion parameters. For another example, the encoder may randomly select a position from the multiple positions and determine the motion parameters pointing to this position as the second motion parameters.

[0179] Of course, in other alternative embodiments, the encoder may determine the motion parameter that satisfies the third condition among the initial position, the at least one first position, the at least one co-location position, and the at least one second position as the second motion parameter. For example, the third condition may include: the rate-distortion cost is less than or a preset threshold, or the third condition may include: the rate-distortion cost is equal to a minimum value (for example, the minimum value of the rate-distortion cost corresponding to the initial position, the rate-distortion cost corresponding to the at least one first position, the rate-distortion cost corresponding to the at least one co-location position, and the rate-distortion cost corresponding to the at least one second position). The third condition may be a predefined condition. For example, the third condition may be implemented by pre-saving a corresponding code, table, or other method that can be used to indicate relevant information in the encoder, or the third condition may be agreed or defined by a standard protocol.

[0180] In this embodiment, the at least one second position is considered on the basis of the initial position, the at least one first position, and the at least one co-location position. That is, the first motion parameter is corrected by introducing the at least one first position, the at least one co-location position, and the at least one second position. Compared with the scheme of correcting the first motion parameter without considering the second position, the correlation between the reference block pointed to by the second motion parameter and the current block is increased, thereby improving the motion estimation effect and encoding performance of the encoder.

[0181] In some embodiments, the range of the second search area is the same as the range of the first search area.

[0182] Of course, in other alternative embodiments, the range of the second search area and the range of the first search area may also be different, and this application does not make any specific limitation on this.

[0183] For example, the range of the second search area may be smaller than the range of the first search area.

[0184] In some embodiments, a first search distance is determined based on a first distance between the first co-located position and the initial position; and based on the first search distance, the at least one second position is searched within the second search area in a grid search manner.

[0185] Exemplarily, the encoder determines, based on a predefined first mapping relationship, the interval corresponding to the first distance in the first mapping relationship as the first search interval. The first mapping relationship may include multiple distances and intervals corresponding to each distance. The first mapping relationship may be implemented by pre-storing a corresponding code, table, or other method for indicating relevant information in the encoder, or the first mapping relationship may be agreed upon or defined by a standard protocol.

[0186] Exemplarily, the encoder determines the distance corresponding to the first interval in which the first distance is located as the first search distance. For example, the encoder may determine the distance corresponding to the first interval in the second mapping relationship as the first search distance based on a predefined second mapping relationship. The second mapping relationship may include multiple distance intervals and the distances corresponding to each distance interval. The second mapping relationship may be implemented by pre-saving a corresponding code, table, or other method that can be used to indicate relevant information in the encoder, or the second mapping relationship may be agreed upon or defined by a standard protocol.

[0187] Exemplarily, the first search interval may be any value, for example, 0 or a positive integer.

[0188] Of course, in other alternative embodiments, the first search distance may be determined based on the search distance of the first search area.

[0189] For example, the first search spacing may be the sum of the search spacing of the first search area and a first value, and the first value may be determined based on the distance between the first co-location position and the initial position. For example, the first value is positively correlated with the distance between the first co-location position and the initial position, that is, the greater the distance between the first co-location position and the initial position, the greater the first value. Since the macropixels closer to the initial position have a stronger correlation with the current block, the first value is positively correlated with the distance between the first co-location position and the initial position. A more intensive search may be performed on the search area within the macropixels close to the initial position, and a more sparse search or no search may be performed on the search area within the macropixels farther away. This can improve the coding efficiency while ensuring the motion estimation effect and coding performance of the encoder.

[0190] In this embodiment, since it is difficult to ensure strict consistency in the spacing of macropixels in the light field image, a first search spacing is determined based on a first distance between the first co-located position and the initial position. Based on the first search spacing, a grid search is performed within the second search area to search for the at least one second position, thereby improving the robustness of the search.

[0191] In some embodiments, the first distance is positively correlated with the first search distance.

[0192] Exemplarily, the larger the first distance is, the larger the first search interval is.

[0193] Since the macro pixels closer to the initial position have a stronger correlation with the current block, the first distance is positively correlated with the first search interval. A more intensive search can be performed on the search area within the macro pixels close to the initial position, and a more sparse search or no search can be performed on the search area within the macro pixels farther away. This can improve the coding efficiency while ensuring the motion estimation effect and coding performance of the encoder.

[0194] In some embodiments, the search distance of the first search area is less than or equal to the search distance corresponding to the distance between the initial position and the co-location position closest to the initial position among the at least one co-location position.

[0195] Exemplarily, assuming that the co-location position closest to the initial position among the at least one co-location position is recorded as the closest co-location position, the search spacing of the first search area is smaller than the search spacing corresponding to the distance between the closest co-location position and the initial position. This ensures that the search area in the initial macro-pixel is searched most densely, and a sparser search spacing is adopted for the search area within the macro-pixel where the closest co-location position is located. This can improve the coding efficiency while ensuring the motion estimation effect and coding performance of the encoder.

[0196] Exemplarily, assuming that the co-location position closest to the initial position among the at least one co-location position is recorded as the closest co-location position, the search spacing of the first search area is equal to the search spacing corresponding to the distance between the closest co-location position and the initial position. This can ensure that the search area in the initial macropixel and the search area in the macropixel where the closest co-location position is located are searched most intensively, thereby ensuring the motion estimation effect and encoding performance of the encoder.

[0197] In some embodiments, the at least one co-location position is divided into at least one layer; based on the spacing corresponding to the first layer where the first co-location position is located, the at least one second position is searched within the second search area in a grid search manner.

[0198] Illustratively, any one of the at least one layer comprises one or more of the at least one co-location positions.

[0199] Exemplarily, the encoder divides the at least one co-location position into at least one layer; then determines the spacing corresponding to the first layer where the first co-location position is located; then, the encoder performs a grid search in the search area where some of the co-location positions (for example, the screened co-location positions, which include the first co-location position) or all the co-location positions on the first layer are located based on the spacing corresponding to the first layer in a grid search manner.

[0200] In other words, starting from the initial position, from the inside to the outside, the encoder can divide the at least one co-location position into M layers (numbered 1 to M, the closer to the initial position, the smaller the number). Taking the macro pixel as a hexagon as an example, each layer is a hexagon connected by several co-location positions (it may also be a rhombus or other shape). The encoder performs a local grid search in the search area where the initial position and the co-location positions on each layer are located (not all of the co-location positions, several co-location positions can be selected). The search spacing used when performing the local grid search in the search area where the initial position is located is P0, and the search spacing used when performing the local grid search in the search area where the co-location positions on each layer are located is P i(0 < i ≤ M), and satisfying that when x < y, P x ≤ P y . Meanwhile, the range sizes of the local grid searches adopted everywhere are all the same.

[0201] Exemplarily, the encoder determines, based on a predefined third mapping relationship, the spacing corresponding to the first layer in the third mapping relationship as the first search spacing. Wherein, the third mapping relationship may include multiple layers and the spacings corresponding to each layer. The third mapping relationship may be implemented by pre-saving corresponding codes, tables or other means for indicating relevant information in the encoder, or the third mapping relationship may be agreed or defined by a standard protocol.

[0202] It should be noted that in other alternative embodiments, the term "layer" may also be equivalently replaced with terms having similar meanings such as "hierarchy", "group", "sub-group", "set", etc., and the present application does not make specific limitations thereto.

[0203] In some embodiments, starting from the initial position, along a plurality of predefined directions, the same-position point positions in the at least one same-position point position that have the same number of same-position point positions spaced from the initial position are divided into the same-position point positions included in one layer of the at least one layer.

[0204] Exemplarily, the plurality of directions may be implemented by pre-saving corresponding codes, tables or other means for indicating relevant information in the encoder, or the plurality of directions may be agreed or defined by a standard protocol.

[0205] Exemplarily, starting from the initial position, along a plurality of predefined directions, the encoder may only divide the same-position point positions in the at least one same-position point position that have the same number of same-position point positions spaced from the initial position into the same-position point positions included in one layer of the at least one layer.

[0206] Exemplarily, starting from the initial position, along a plurality of predefined directions, the encoder only divides the same-position point positions in the at least one same-position point position that have 0 same-position point positions spaced from the initial position into the same-position point positions included in the first layer of the at least one layer; starting from the initial position, along a plurality of predefined directions, the encoder only divides the same-position point positions in the at least one same-position point position that have 1 same-position point position spaced from the initial position into the same-position point positions included in the second layer of the at least one layer; and so on, until all the at least one same-position point position is divided, the same-position point positions included in each layer of the at least one layer are obtained.

[0207] Exemplarily, the encoder takes the initial position as the starting point, and along multiple predefined directions, only divides the at least one co-location position whose number of co-location positions separated from the initial position is 1, into the co-location positions included in the first layer of the at least one layer; the encoder takes the initial position as the starting point, and along multiple predefined directions, only divides the at least one co-location position whose number of co-location positions separated from the initial position is 2, into the co-location positions included in the second layer of the at least one layer; and so on, until the at least one co-location position is divided, the co-location positions included in each layer of the at least one layer are obtained.

[0208] Exemplarily, with the initial position as the starting point, along predefined multiple directions, the encoder can divide all the co-location positions on the connecting line formed by the co-location positions in the at least one co-location position and the same number of co-location positions spaced from the initial position into the co-location positions included in one layer of the at least one layer.

[0209] Exemplarily, the encoder takes the initial position as the starting point and, along multiple predefined directions, the encoder can divide all the co-location positions on the line formed by the co-location positions in the at least one co-location position and the number of which is 0 between the co-location positions in the initial position, into the co-location positions included in the first layer of the at least one layer; the encoder takes the initial position as the starting point and, along multiple predefined directions, the encoder can divide all the co-location positions on the line formed by the co-location positions in the at least one co-location position and the number of which is 1 between the co-location positions in the initial position, into the co-location positions included in the second layer of the at least one layer; and so on, until the at least one co-location position is divided, the co-location positions included in each layer of the at least one layer are obtained.

[0210] Exemplarily, the encoder takes the initial position as the starting point and, along multiple predefined directions, the encoder can divide all the co-location positions on the line formed by the co-location positions of the at least one co-location position and the number of which is 1 between the co-location positions of the at least one co-location position and the initial position, into the co-location positions included in the first layer of the at least one layer; the encoder takes the initial position as the starting point and, along multiple predefined directions, the encoder can divide all the co-location positions on the line formed by the co-location positions of the at least one co-location position and the number of which is 2 between the co-location positions of the at least one co-location position and the initial position, into the co-location positions included in the second layer of the at least one layer; and so on, until the at least one co-location position is divided, the co-location positions included in each layer of the at least one layer are obtained.

[0211] Of course, the encoder can divide the at least one co-location position into at least one layer in other ways, and this application does not specifically limit this. For example, in other alternative embodiments, the encoder takes the initial position as the starting point and, along multiple predefined radii, divides the co-location positions in the at least one co-location position and located on the circle corresponding to any one of the multiple predefined radii into co-location positions in the layer corresponding to any one of the predefined radii. The multiple predefined radii can be implemented by pre-saving corresponding codes, tables or other methods that can be used to indicate relevant information in the encoder, or the multiple predefined radii can be agreed or defined by a standard protocol.

[0212] In some embodiments, the multiple directions include directions perpendicular to a side of the initial macropixel and / or directions parallel to a diagonal line of the initial macropixel.

[0213] Exemplarily, the multiple directions include directions perpendicular to respective sides of the initial macropixel.

[0214] FIG11 is an example of at least one layer provided by an embodiment of the present application.

[0215] As shown in FIG. 11 , assuming that the initial macropixel is a hexagonal macropixel, the multiple directions include six directions perpendicular to the six sides of the initial macropixel. The encoder takes the initial position as the starting point, and along the six directions, divides all the co-location positions (for example, including six co-location positions) on the line formed by the co-location positions of the at least one co-location position and the number of which is 0 from the initial position into the co-location positions included in the first layer of the at least one layer; along the six directions, divides all the co-location positions (for example, including 12 co-location positions) on the line formed by the at least one co-location position and the number of which is 1 from the initial position into the co-location positions included in the second layer of the at least one layer; along the six directions, divides all the co-location positions (for example, including 18 co-location positions) on the line formed by the at least one co-location position and the number of which is 2 from the initial position into the co-location positions included in the third layer of the at least one layer; and so on, until the at least one co-location position is divided, the co-location positions included in each layer of the at least one layer are obtained.

[0216] FIG12 is an example of at least one layer provided in an embodiment of the present application.

[0217] As shown in FIG12 , assuming that the initial macropixel is a hexagonal macropixel, the multiple directions include six directions perpendicular to the six sides of the initial macropixel and two directions parallel to the horizontal diagonal of the initial macropixel, for a total of eight directions. The encoder takes the initial position as a starting point and, along these eight directions, divides all collocated positions (e.g., eight collocated positions) on a line formed by the collocated positions of the at least one collocated position and the number of collocated positions spaced from the initial position is zero into the collocated positions of the first layer of the at least one layer; along these eight directions, divides all collocated positions (e.g., sixteen collocated positions) on a line formed by the collocated positions of the at least one collocated position and the number of collocated positions spaced from the initial position is one into the collocated positions of the second layer of the at least one layer; and so on, until the at least one collocated position is completely divided, thereby obtaining the collocated positions of each layer of the at least one layer.

[0218] It is worth noting that the multiple directions, including directions perpendicular to the sides of the initial macropixels and / or directions parallel to the diagonals of the initial macropixels, are merely examples of this application and should not be construed as limiting this application. For example, in other alternative embodiments, the multiple directions may also be directions determined by the encoder based on a regular arrangement of macropixels in the image; the regular arrangement may include, but is not limited to, at least one of the following: macropixel shape, macropixel spacing, and macropixel arrangement direction; for example, the multiple directions may include the macropixel arrangement direction.

[0219] In some embodiments, a shape formed by connecting the co-located positions on a layer in the at least one layer is the same as or different from a shape of the initial macro pixel.

[0220] For example, the co-location positions on the layer in the at least one layer may be connected to form a specific shape, such as a hexagon, a diamond or other shapes.

[0221] For example, as shown in FIG11 , the shape formed by connecting the collocated positions on the layers in the at least one layer is a hexagon, which is the same as the shape of the initial macropixel. As shown in FIG12 , the shape formed by connecting the collocated positions on the layers in the at least one layer is a rhombus, which is different from the shape of the initial macropixel.

[0222] In some embodiments, the distance between the first layer and the initial position is positively correlated with the spacing corresponding to the first layer.

[0223] Exemplarily, the greater the distance between the first layer and the initial position, the greater the first search distance.

[0224] Since the closer the macropixel is to the initial position, the stronger the correlation with the current block, the distance between the first layer and the initial position is positively correlated with the spacing corresponding to the first layer. A more intensive search can be performed on the search area within the macropixel close to the initial position, and a more sparse search or no search can be performed on the search area within the macropixel farther away. This can improve the coding efficiency while ensuring the motion estimation effect and coding performance of the encoder.

[0225] In some embodiments, the search distance of the first search area is less than or equal to the search distance corresponding to the layer with the smallest distance from the initial position in the at least one layer.

[0226] Exemplarily, the search distance of the first search area is less than or equal to the search distance corresponding to the layer with the shortest minimum distance from the initial position in the at least one layer.

[0227] Exemplarily, the search distance of the first search area is less than or equal to the search distance corresponding to the layer with the smallest maximum minimum distance from the initial position in the at least one layer.

[0228] Exemplarily, starting from the initial position, from the inside to the outside, the encoder can divide the at least one co-location position into M layers (numbered 1 to M, the closer to the initial position, the smaller the number), then the search spacing of the first search area is less than or equal to the search spacing corresponding to the layer numbered 1.

[0229] In this embodiment, the search spacing of the first search area is equal to the search spacing corresponding to the layer with the smallest distance from the initial position in the at least one layer, which can ensure that the search area in the initial macropixel and the closest layer are searched most intensively, thereby ensuring the motion estimation effect and encoding performance of the encoder. The search spacing of the first search area is less than the search spacing corresponding to the layer with the smallest distance from the initial position in the at least one layer, which can ensure that the search area in the initial macropixel is searched most intensively, and a sparser search spacing is used for the closest layer, thereby improving encoding efficiency while ensuring the motion estimation effect and encoding performance of the encoder.

[0230] FIG13 is an example of an embodiment of the present application providing a search spacing of a first search area that is equal to a search spacing used by a first layer.

[0231] As shown in Figure 13, the search spacing of the first search area can be equal to the search spacing corresponding to the layer in the at least one layer that is the smallest distance from the initial position. For example, the search spacing of the first search area and the search spacing of the search area containing the co-located position on the first layer from the inside to the outside of the at least one layer are both 2, and the search spacing of the search area containing the co-located position on the second layer is 4.

[0232] FIG14 is an example of an embodiment of the present application providing a first search area where the search spacing is smaller than the search spacing used by the first layer.

[0233] As shown in Figure 14, the search spacing of the first search area can be smaller than the search spacing corresponding to the layer with the smallest distance from the initial position in the at least one layer. For example, if the search spacing of the first search area is 2, the search spacing of the search areas containing the co-located positions on the first layer from the inside out of the at least one layer is 3, and the search spacing of the search areas containing the co-located positions on the second layer is 4.

[0234] In some embodiments, the first search area includes an area centered at the initial position and having a side length or radius of a preset value.

[0235] Exemplarily, the preset value may be realized by pre-saving a corresponding code, table or other method for indicating relevant information in the encoder, or the preset value may be agreed or defined by a standard protocol.

[0236] Exemplarily, the first search area includes a circular area with the initial position as the center and a radius of the preset value.

[0237] Exemplarily, the first search area includes a rectangular area centered at the initial position and having a side length (eg, a maximum side length or a minimum side length) equal to the preset value.

[0238] Exemplarily, the first search area includes a diamond-shaped area with the initial position as the center and a diagonal side length (eg, maximum side length or minimum side length) being the preset value.

[0239] In some embodiments, the first search area is a rectangular search area, and the maximum side length of the rectangular search area is less than or equal to the minimum value of the distance between two adjacent macro pixels in the reference image in the horizontal direction and the distance in the vertical direction.

[0240] Exemplarily, the maximum side length of the rectangular search area is less than or equal to the minimum value of the distance between two adjacent macro pixels in the reference image in the horizontal direction and in the vertical direction, and circumscribes the initial macro pixel.

[0241] The preferred embodiments of the present application are described in detail above in conjunction with the accompanying drawings. However, the present application is not limited to the specific details in the embodiments mentioned above. Within the technical concept of the present application, the technical solution of the present application can be subjected to a variety of simple modifications, and these simple modifications all fall within the scope of protection of the present application. For example, the various specific technical features described in the specific embodiments mentioned above can be combined in any suitable manner unless there is any contradiction. In order to avoid unnecessary repetition, the present application will not further explain various possible combinations. For another example, the various different embodiments of the present application can also be arbitrarily combined, and as long as they do not violate the ideas of the present application, they should also be regarded as the contents disclosed in the present application.

[0242] It should also be understood that in the various method embodiments of the present application, the size of the serial numbers of the processes involved above does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0243] The following describes in detail an embodiment of the device of the present application in conjunction with FIG. 15 and FIG. 16 .

[0244] FIG15 is a schematic block diagram of an encoder 500 provided in an embodiment of the present application.

[0245] As shown in FIG15 , the encoder 500 may include:

[0246] An estimation unit 510 is configured to perform motion estimation on a current block in a current image and determine a first motion parameter of the current block; the first motion parameter points to an initial position in a reference image of the current image;

[0247] The determining unit 520 is configured to:

[0248] Determine at least one first position other than the initial position within a first search area where the initial position is located; the first search area is smaller than or equal to an initial macropixel where the initial position is located;

[0249] A determining unit is configured to determine a second motion parameter of the reference block based on the initial position and the at least one first position.

[0250] In some embodiments, the determining unit 520 is specifically configured to:

[0251] The motion parameter pointing to the position with the minimum rate-distortion cost corresponding to the initial position and the at least one first position is determined as the second motion parameter.

[0252] In some embodiments, the determining unit 520 is specifically configured to:

[0253] determining at least one co-located position of the initial position based on the position of the initial position within the initial macropixel;

[0254] The second motion parameter is determined based on the initial position, the at least one first position, and the at least one co-located position.

[0255] In some embodiments, the determining unit 520 is specifically configured to:

[0256] determining at least one macropixel within a search range of the at least one co-located position;

[0257] In any one of the at least one macropixel, a position that is the same as the initial position in the initial macropixel is determined as the collocated position in the at least one collocated position.

[0258] In some embodiments, the determining unit 520 is specifically configured to:

[0259] The motion parameter pointing to the position with the minimum rate-distortion cost among the initial position, the at least one first position and the at least one co-located position is determined as the second motion parameter.

[0260] In some embodiments, the determining unit 520 is specifically configured to:

[0261] Determine at least one second position other than the first co-location position within a second search area where a first co-location position among the at least one co-location position is located, wherein the second search area is smaller than or equal to the macropixel where the first co-location position is located;

[0262] The motion parameter pointing to the position with the minimum rate-distortion cost among the initial position, the at least one first position, the at least one co-located position and the at least one second position is determined as the second motion parameter.

[0263] In some embodiments, the determining unit 520 is specifically configured to:

[0264] determining a first search distance based on a first distance between the first co-located position and the initial position;

[0265] Based on the first search distance, the at least one second position is searched within the second search area in a grid search manner.

[0266] In some embodiments, the first distance is positively correlated with the first search distance.

[0267] In some embodiments, the search distance of the first search area is less than or equal to the search distance corresponding to the distance between the initial position and the co-location position closest to the initial position among the at least one co-location position.

[0268] In some embodiments, the determining unit 520 is specifically configured to:

[0269] dividing the at least one co-located location into at least one stratum;

[0270] Based on the spacing corresponding to the first layer where the first co-location positions are located, the at least one second position is searched within the second search area in a grid search manner.

[0271] In some embodiments, the determining unit 520 is specifically configured to:

[0272] Taking the initial position as the starting point, along predefined multiple directions, the co-location positions in the at least one co-location position and the same number of co-location positions as the initial position are divided into the co-location positions included in one layer of the at least one layer.

[0273] In some embodiments, the multiple directions include directions perpendicular to a side of the initial macropixel and / or directions parallel to a diagonal line of the initial macropixel.

[0274] In some embodiments, a shape formed by connecting the co-located positions on a layer in the at least one layer is the same as or different from a shape of the initial macro pixel.

[0275] In some embodiments, the distance between the first layer and the initial position is positively correlated with the spacing corresponding to the first layer.

[0276] In some embodiments, the search distance of the first search area is less than or equal to the search distance corresponding to the layer with the smallest distance from the initial position in the at least one layer.

[0277] In some embodiments, the range of the second search area is the same as the range of the first search area.

[0278] In some embodiments, the first search area includes an area centered at the initial position and having a side length or radius of a preset value.

[0279] In some embodiments, the first search area is a rectangular search area, and the maximum side length of the rectangular search area is less than or equal to the minimum value of the distance between two adjacent macro pixels in the reference image in the horizontal direction and the distance in the vertical direction.

[0280] It should be understood that the device embodiment of the encoder and the method embodiment of the encoding method can correspond to each other, and similar descriptions can refer to the method embodiment. To avoid repetition, they are not described here. Specifically, the encoder 500 shown in Figure 15 can correspond to the corresponding subject in the encoding method 400 of the embodiment of the present application, and the aforementioned and other operations and / or functions of the various units in the encoder 500 are respectively for implementing the corresponding processes in various methods such as the encoding method 400.

[0281] It should also be understood that the various units in the encoder 500 that the embodiment of the present application is related to are divided based on logical functions. In practical applications, the function of a unit can also be realized by multiple units, or the function of multiple units is realized by one unit, or even, these functions can also be assisted by one or more other units. For example, part or all of the encoder 500 is merged into one or several other units. For another example, a certain (some) unit in the encoder 500 can also be split into multiple smaller units in function to constitute, which can achieve the same operation without affecting the realization of the technical effect of the embodiment of the application. For another example, the encoder 500 can also include other units. In practical applications, these functions can also be assisted by other units and can be realized by the collaboration of multiple units.

[0282] According to another embodiment of the present application, the encoder 500 involved in the embodiment of the present application can be constructed by running a computer program (including program code) capable of executing each step involved in the corresponding method on a general-purpose computing device of a general-purpose computer including processing elements and storage elements such as a central processing unit (CPU), a random access memory medium (RAM), and a read-only memory medium (ROM), and realizing the encoding method of the embodiment of the present application. The computer program can be recorded on, for example, a computer-readable storage medium, and loaded into an electronic device through a computer-readable storage medium, and run therein to realize the corresponding method of the embodiment of the present application. In other words, the units involved above can be implemented in hardware form, or can be implemented by instructions in the form of software, or can be implemented in the form of a combination of hardware and software. Specifically, the steps of the method embodiment in the embodiment of the present application can be completed by the integrated logic circuit of the hardware in the processor and / or the instructions in the form of software, and the steps of the method disclosed in the embodiment of the present application can be directly embodied as the execution of the hardware decoding processor, or the execution is completed by the hardware and software combination in the decoding processor. Optionally, the software can be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, and a register. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps in the above-mentioned method embodiment in combination with its hardware.

[0283] FIG16 is a schematic structural diagram of an electronic device 600 provided in an embodiment of the present application.

[0284] As shown in Figure 16, the electronic device 600 includes at least a processor 610 and a computer-readable storage medium 620. The processor 610 and the computer-readable storage medium 620 may be connected via a bus or other means. The computer-readable storage medium 620 is used to store a computer program 621, which includes computer instructions. The processor 610 is used to execute the computer instructions stored in the computer-readable storage medium 620. The processor 610 is the computing core and control core of the electronic device 600 and is suitable for implementing one or more computer instructions, specifically loading and executing one or more computer instructions to implement the corresponding method flow or corresponding function.

[0285] For example, the processor 610 may also be referred to as a central processing unit (CPU). The processor 610 may include, but is not limited to, a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, discrete hardware components, and the like.

[0286] Exemplarily, the computer-readable storage medium 620 may be a high-speed RAM memory, or a non-volatile memory (Non-Volatile Memory), such as at least one disk memory; optionally, it may be at least one computer-readable storage medium located away from the aforementioned processor 610. Specifically, the computer-readable storage medium 620 includes, but is not limited to: volatile memory and / or non-volatile memory. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus random access memory (DR RAM).

[0287] Exemplarily, the electronic device 600 may be a decoder or decoding framework involved in an embodiment of the present application; computer instructions are stored in the computer-readable storage medium 620; the computer instructions stored in the computer-readable storage medium 620 are loaded and executed by the processor 610 to implement the corresponding steps in the decoding method provided in an embodiment of the present application; in other words, the computer instructions in the computer-readable storage medium 620 are loaded by the processor 610 and the corresponding steps are executed, which will not be repeated here to avoid repetition.

[0288] According to another aspect of the present application, the present application also provides a coding and decoding system, including the encoder and decoder mentioned above.

[0289] According to another aspect of the present application, the present application also provides a computer-readable storage medium (Memory), which is a memory device in the electronic device 600 for storing programs and data. For example, a computer-readable storage medium 620. It can be understood that the computer-readable storage medium 620 here can include both the built-in storage medium in the electronic device 600 and, of course, the extended storage medium supported by the electronic device 600. The computer-readable storage medium provides a storage space, which stores the operating system of the electronic device 600. In addition, one or more computer instructions suitable for being loaded and executed by the processor 610 are also stored in the storage space. These computer instructions can be one or more computer programs 621 (including program codes).

[0290] According to another aspect of the present application, the present application also provides a computer program product or computer program, which includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. For example, computer program 621. In this case, the data processing device 600 can be a computer, the processor 610 reads the computer instructions from the computer-readable storage medium 620, and the processor 610 executes the computer instructions so that the computer performs the encoding method provided in the various optional modes mentioned above. In other words, when implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process of the embodiment of the present application is run in whole or in part or the functions of the embodiment of the present application are implemented. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means.

[0291] According to another aspect of the present application, the present application further provides a code stream, which may be a code stream generated using the encoding method provided in an embodiment of the present application.

[0292] Those skilled in the art will appreciate that the units and process steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0293] Finally, it should be noted that the above content is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A coding method, characterized in that: include: Performing motion estimation on a current block in a current image to determine a first motion parameter of the current block; The first motion parameter points to an initial position in a reference image of the current image; Determine at least one first position other than the initial position in a first search area where the initial position is located; the first search area is smaller than or equal to an initial macropixel where the initial position is located; Based on the initial position and the at least one first position, a second motion parameter of the reference block is determined.

2. The method according to claim 1, characterized in that The determining, based on the initial position and the at least one first position, a second motion parameter of the reference block comprises: A motion parameter pointing to a position with the minimum rate-distortion cost corresponding to the initial position and the at least one first position is determined as the second motion parameter.

3. The method according to claim 1, characterized in that The determining, based on the initial position and the at least one first position, a second motion parameter of the reference block comprises: determining at least one co-location position of the initial position based on the position of the initial position within the initial macropixel; The second motion parameter is determined based on the initial position, the at least one first position and the at least one co-location position.

4. The method according to claim 3, characterized in that The determining, based on the position of the initial position within the initial macropixel, at least one co-location position of the initial position comprises: Determining at least one macropixel within a search range of the at least one co-located position; In any one of the at least one macropixel, a position that is the same as the position of the initial position in the initial macropixel is determined as the co-location position in the at least one co-location position.

5. The method according to claim 3 or 4, characterized in that: The determining the second motion parameter based on the initial position, the at least one first position and the at least one co-location position comprises: The motion parameter pointing to the position with the smallest rate-distortion cost among the initial position, the at least one first position and the at least one co-located position is determined as the second motion parameter.

6. The method according to claim 3 or 4, characterized in that: The determining the second motion parameter based on the initial position, the at least one first position and the at least one co-location position comprises: Determine at least one second position other than the first co-location position in a second search area where a first co-location position among the at least one co-location position is located; the second search area is smaller than or equal to the macropixel where the first co-location position is located; The motion parameter pointing to the position with the smallest rate-distortion cost among the initial position, the at least one first position, the at least one co-located position and the at least one second position is determined as the second motion parameter.

7. The method according to claim 6, characterized in that Determining at least one second location other than the first co-location location in a second search area where a first co-location location among the at least one co-location location is located comprises: determining a first search distance based on a first distance between the first co-location position and the initial position; Based on the first search interval, the at least one second position is searched in the second search area in a grid search manner.

8. The method according to claim 7, characterized in that The first distance is positively correlated with the first search interval.

9. The method according to claim 7, characterized in that: The search distance of the first search area is less than or equal to the search distance corresponding to the distance between the initial position and the co-location position closest to the initial position among the at least one co-location position.

10. The method according to claim 6, characterized in that Determining at least one second location other than the first co-location location in a second search area where a first co-location location among the at least one co-location location is located comprises: dividing the at least one co-located location into at least one stratum; Based on the spacing corresponding to the first layer where the first co-location positions are located, the at least one second position is searched in the second search area in a grid search manner.

11. The method according to claim 10, characterized in that The dividing the at least one co-site position into at least one layer comprises: Taking the initial position as the starting point, along predefined multiple directions, the same number of co-location positions in the at least one co-location position and the same number of co-location positions spaced from the initial position are divided into co-location positions included in one layer in the at least one layer.

12. The method according to claim 11, characterized in that The multiple directions include a direction perpendicular to a side of the initial macropixel and / or a direction parallel to a diagonal line of the initial macropixel.

13. The method according to claim 11, characterized in that A shape formed by connecting the co-location positions on a layer in the at least one layer is the same as or different from the shape of the initial macro pixel.

14. The method according to any one of claims 10 to 13, characterized in that The distance between the first layer and the initial position is positively correlated with the spacing corresponding to the first layer.

15. The method according to any one of claims 10 to 14, characterized in that The search distance of the first search area is less than or equal to the search distance corresponding to the layer with the smallest distance from the initial position in the at least one layer.

16. The method according to any one of claims 6 to 15, characterized in that The range of the second search area is the same as the range of the first search area.

17. The method according to any one of claims 1 to 16, characterized in that The first search area includes an area centered at the initial position and having a side length or a radius of a preset value.

18. The method according to claim 17, characterized in that The first search area is a rectangular search area, and the maximum side length of the rectangular search area is less than or equal to the minimum value of the distance between two adjacent macro pixels in the reference image in the horizontal direction and in the vertical direction.

19. The method according to any one of claims 1 to 18, characterized in that The current image and the reference image of the current image are light field images.

20. An encoder, characterized in that: include: an estimating unit, configured to perform motion estimation on a current block in a current image and determine a first motion parameter of the current block; The first motion parameter points to an initial position in a reference image of the current image; Identify units for: Determine at least one first position other than the initial position in a first search area where the initial position is located; the first search area is smaller than or equal to an initial macropixel where the initial position is located; A determining unit is used to determine a second motion parameter of the reference block based on the initial position and the at least one first position.

21. An electronic device, characterized in that: include: a processor adapted to execute a computer program; A computer-readable storage medium having a computer program stored therein, wherein the computer program, when executed by the processor, implements the method according to any one of claims 1 to 19.

22. A computer-readable storage medium, characterized in that: Used to store a computer program, which, when executed on a computer, causes the computer to execute the method according to any one of claims 1 to 19.

23. A computer program product comprising a computer program / instructions, characterized in that The computer program / instructions, when executed by a processor, implement the method according to any one of claims 1 to 19.

24. A code stream, characterized in that The code stream is a code stream generated by the method according to any one of claims 1 to 19.

Images