Method for determining image block prediction samples and encoding and decoding device

Abstract

The present application provides a method for determining predicted samples of an image block and a coding and decoding device, the method comprising: obtaining an index of a reference pixel corresponding to each pixel in at least one pixel in an image block; determining at least one of a reference pixel index matrix, an unmatched pixel index matrix and an unmatched pixel list according to the index of the reference pixel corresponding to each pixel in at least one pixel; and determining predicted samples of the pixel points according to at least one of the reference pixel index matrix, the unmatched pixel index matrix and the unmatched pixel list, and then determining predicted samples of the image block according to the predicted samples of the pixel points, without exporting predicted samples of YUV444, thereby reducing the memory size required for exporting the predicted samples and facilitating hardware implementation.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image processing technology, and in particular to a method for determining image block prediction samples and a coding and decoding device. Background Art

[0002] Digital video technology can be incorporated into a variety of video devices, such as digital televisions, smart phones, computers, e-readers or video players, etc. With the development of video technology, the amount of data included in video data is large. In order to facilitate the transmission of video data, video devices implement video compression technology to make video data more efficiently transmitted or stored.

[0003] At present, spatial prediction or temporal prediction is used to reduce or eliminate redundant information in video data to achieve video data compression. Motion compensation is a commonly used prediction method for video coding. Based on the redundant characteristics of video content in the time domain or spatial domain, the prediction value of the current coding block is derived from the encoded area. Prediction methods based on motion compensation include: inter-frame prediction, intra-frame block copy, intra-frame string copy, etc. The intra-frame string copy prediction method is to divide a coding block into a series of pixel strings according to a certain scanning order. The encoder encodes the type, length and prediction value information of each string of the current coding block in the bitstream. Correspondingly, the decoder derives the prediction sample of the current image block based on the type, length and prediction value information of each string carried in the bitstream, and determines the reconstruction value of the current image block based on the prediction sample of the current image block.

[0004] However, currently the decoding end occupies a large amount of memory when exporting prediction samples. Summary of the invention

[0005] The present application provides a method for determining image block prediction samples and a coding and decoding device, which reduces the memory size required for deriving the prediction samples and is conducive to hardware implementation.

[0006] In a first aspect, a method for determining a prediction sample of an image block is provided, comprising:

[0007] Obtaining an index of a reference pixel corresponding to each pixel in at least one pixel in the image block;

[0008] Determine at least one of a reference pixel index matrix, an unmatched pixel index matrix, and an unmatched pixel list according to an index of a reference pixel corresponding to each pixel in the at least one pixel;

[0009] For each pixel of the at least one pixel, determining a prediction sample of the pixel according to at least one of the reference pixel index matrix, the unmatched pixel index matrix and the unmatched pixel list;

[0010] Determining the prediction samples of the image block according to the prediction samples of the pixel points;

[0011] Among them, the reference pixel index matrix includes the index of the reference pixel corresponding to each pixel in the at least one pixel point, the unmatched pixel list includes the correspondence between the value of the unmatched pixel in the at least one pixel point and the index of the unmatched pixel, and the unmatched pixel index matrix includes the index of the unmatched pixel in the at least one pixel point in the unmatched pixel list.

[0012] In a second aspect, a device for determining a prediction sample of an image block is provided, comprising:

[0013] An acquisition unit, configured to obtain an index of a reference pixel corresponding to each pixel in at least one pixel in the image block;

[0014] A first determining unit, configured to determine at least one of a reference pixel index matrix, an unmatched pixel index matrix, and an unmatched pixel list according to an index of a reference pixel corresponding to each pixel of the at least one pixel;

[0015] A second determining unit, configured to determine, for each pixel of the at least one pixel, a prediction sample of the pixel according to at least one of the reference pixel index matrix, the unmatched pixel index matrix and the unmatched pixel list;

[0016] A third determining unit, configured to determine a prediction sample of the image block according to the prediction sample of the pixel point;

[0017] Among them, the reference pixel index matrix includes the index of the reference pixel corresponding to each pixel in the at least one pixel point, the unmatched pixel list includes the correspondence between the value of the unmatched pixel in the at least one pixel point and the index of the unmatched pixel, and the unmatched pixel index matrix includes the index of the unmatched pixel in the at least one pixel point in the unmatched pixel list.

[0018] In a third aspect, a coding device is provided, comprising a processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the method in the first aspect or its implementations.

[0019] In a fourth aspect, a decoding device is provided, comprising a processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the method in the first aspect or its implementations.

[0020] In a fifth aspect, a chip is provided for implementing the method in the first aspect or its various implementations. Specifically, the chip includes: a processor for calling and running a computer program from a memory so that a device equipped with the chip executes the method in the first aspect or its various implementations.

[0021] In a sixth aspect, a computer-readable storage medium is provided for storing a computer program, wherein the computer program enables a computer to execute the method in the above-mentioned first aspect or its various implementations.

[0022] In a seventh aspect, a computer program product is provided, comprising computer program instructions, which enable a computer to execute the method in the above-mentioned first aspect or its various implementations.

[0023] In an eighth aspect, a computer program is provided, which, when executed on a computer, enables the computer to execute the method in the first aspect or its various implementations.

[0024] Through the technical solution provided by the present application, the index of the reference pixel corresponding to each pixel in at least one pixel in the image block is obtained; according to the index of the reference pixel corresponding to each pixel in the at least one pixel, at least one of the reference pixel index matrix, the unmatched pixel index matrix and the unmatched pixel list is determined; and according to at least one of the reference pixel index matrix, the unmatched pixel index matrix and the unmatched pixel list, the predicted samples of the pixel are determined, and then the predicted samples of the image block are determined according to the predicted samples of the pixel, without exporting the predicted samples of YUV444, which reduces the memory size required for exporting the predicted samples and is conducive to hardware implementation. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 A schematic block diagram of a video encoding and decoding system involved in an embodiment of the present application;

[0027] Figure 2 is a schematic block diagram of a video encoder provided in an embodiment of the present application;

[0028] Figure 3 is a schematic block diagram of a decoding framework provided in an embodiment of the present application;

[0029] Figure 4A schematic diagram of an intra-frame block copy involved in an embodiment of the present application;

[0030] Figure 5 It is a schematic diagram of an intra-frame string replication involved in an embodiment of the present application;

[0031] Figure 6 A flowchart of a method for determining image block prediction samples provided in an embodiment of the present application;

[0032] Fig. 7A A schematic diagram of an image block in an example of the present application;

[0033] Figure 7B A schematic diagram of at least one pixel in an example of the present application;

[0034] Figure 7C A schematic diagram of a reference pixel index matrix in an example of the present application;

[0035] Fig.7D A schematic diagram of an unmatched pixel index matrix in an example of the present application;

[0036] Figure 8 A flowchart of another method for determining image block prediction samples provided in an embodiment of the present application;

[0037] Fig. 9 A flowchart of another method for determining image block prediction samples provided in an embodiment of the present application;

[0038] Fig. 10A A schematic diagram of a color sampling mode involved in an embodiment of the present application;

[0039] Fig. 10B This is a schematic diagram of another color sampling mode involved in an embodiment of the present application;

[0040] Fig. 10C This is a schematic diagram of another color sampling mode involved in an embodiment of the present application;

[0041] Fig.11A A schematic diagram of a chroma prediction sample in an example of the present application;

[0042] Fig. 11B A schematic diagram of chroma prediction samples in another example of the present application;

[0043] Fig.12 is a schematic block diagram of a device for determining image block prediction samples provided by an embodiment of the present application;

[0044] Fig.13 It is a schematic block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0046] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0047] The present application can be applied to the field of image coding and decoding, the field of video coding and decoding, the field of hardware video coding and decoding, the field of dedicated circuit video coding and decoding, the field of real-time video coding and decoding, etc. For example, the scheme of the present application can be combined with an audio and video coding standard (AVS), such as the H.264 / audio video coding (AVC) standard, the H.265 / high efficiency video coding (HEVC) standard, and the H.266 / versatile video coding (VVC) standard. Alternatively, the scheme of the present application can be combined with other proprietary or industry standards and operate, and the standards include ITU-TH.261, ISO / IECMPEG-1Visual, ITU-TH.262 or ISO / IECMPEG-2Visual, ITU-TH.263, ISO / IECMPEG-4Visual, ITU-TH.264 (also known as ISO / IECMPEG-4AVC), including scalable video coding (SVC) and multi-view video coding (MVC) extensions. It should be understood that the technology of the present application is not limited to any specific coding standard or technology.

[0048] To facilitate understanding, first combine Figure 1The video encoding and decoding system involved in the embodiments of the present application is introduced.

[0049] Figure 1 1 is a schematic block diagram of a video encoding and decoding system 100 involved in an embodiment of the present application. It should be noted that: Figure 1 This is just an example. The video encoding and decoding system of the embodiment of the present application includes but is not limited to Figure 1 As shown. Figure 1 As shown, the video encoding and decoding system 100 includes an encoding device 110 and a decoding device 120. The encoding device is used to encode (which can be understood as compressing) video data to generate a code stream, and transmit the code stream to the decoding device. The decoding device decodes the code stream generated by the encoding device to obtain decoded video data.

[0050] The encoding device 110 of the embodiment of the present application can be understood as a device with a video encoding function, and the decoding device 120 can be understood as a device with a video decoding function, that is, the embodiment of the present application includes a wider range of devices for the encoding device 110 and the decoding device 120, such as smartphones, desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, televisions, cameras, display devices, digital media players, video game consoles, vehicle-mounted computers, etc.

[0051] In some embodiments, the encoding device 110 may transmit the encoded video data (eg, a code stream) to the decoding device 120 via the channel 130. The channel 130 may include one or more media and / or devices capable of transmitting the encoded video data from the encoding device 110 to the decoding device 120.

[0052] In one example, the channel 130 includes one or more communication media that enable the encoding device 110 to transmit the encoded video data directly to the decoding device 120 in real time. In this example, the encoding device 110 can modulate the encoded video data according to the communication standard and transmit the modulated video data to the decoding device 120. The communication medium includes a wireless communication medium, such as a radio frequency spectrum, and optionally, the communication medium may also include a wired communication medium, such as one or more physical transmission lines.

[0053] In another example, the channel 130 includes a storage medium, which can store the video data encoded by the encoding device 110. The storage medium includes a variety of locally accessible data storage media, such as optical disks, DVDs, flash memories, etc. In this example, the decoding device 120 can obtain the encoded video data from the storage medium.

[0054] In another example, the channel 130 may include a storage server that can store the video data encoded by the encoding device 110. In this example, the decoding device 120 can download the stored encoded video data from the storage server. Alternatively, the storage server can store the encoded video data and transmit the encoded video data to the decoding device 120, such as a web server (e.g., for a website), a file transfer protocol (FTP) server, etc.

[0055] In some embodiments, the encoding device 110 includes a video encoder 112 and an output interface 113. The output interface 113 may include a modulator / demodulator (modem) and / or a transmitter.

[0056] In some embodiments, the encoding device 110 may include a video source 111 in addition to the video encoder 112 and the input interface 113 .

[0057] The video source 111 may include at least one of a video acquisition device (eg, a video camera), a video archive, a video input interface, and a computer graphics system, wherein the video input interface is used to receive video data from a video content provider, and the computer graphics system is used to generate video data.

[0058] The video encoder 112 encodes the video data from the video source 111 to generate a code stream. The video data may include one or more pictures or a sequence of pictures. The code stream contains the coding information of the picture or the sequence of pictures in the form of a bit stream. The coding information may include the coded picture data and associated data. The associated data may include a sequence parameter set (SPS for short), a picture parameter set (PPS for short) and other syntax structures. The SPS may contain parameters applied to one or more sequences. The PPS may contain parameters applied to one or more pictures. The syntax structure refers to a set of zero or more syntax elements arranged in a specified order in the code stream.

[0059] The video encoder 112 transmits the encoded video data directly to the decoding device 120 via the output interface 113. The encoded video data may also be stored in a storage medium or a storage server for subsequent reading by the decoding device 120.

[0060] In some embodiments, the decoding device 120 includes an input interface 121 and a video decoder 122 .

[0061] In some embodiments, the decoding device 120 may include a display device 123 in addition to the input interface 121 and the video decoder 122 .

[0062] The input interface 121 includes a receiver and / or a modem. The input interface 121 can receive the encoded video data through the channel 130 .

[0063] The video decoder 122 is used to decode the encoded video data to obtain decoded video data, and transmit the decoded video data to the display device 123 .

[0064] The display device 123 displays the decoded video data. The display device 123 may be integrated with the decoding device 120 or external to the decoding device 120. The display device 123 may include a variety of display devices, such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or other types of display devices.

[0065] also, Figure 1 This is only an example, and the technical solution of the embodiment of the present application is not limited to Figure 1 For example, the technology of the present application can also be applied to single-sided video encoding or single-sided video decoding.

[0066] The following is an introduction to the video encoding framework involved in the embodiments of the present application.

[0067] Figure 2 2 is a schematic block diagram of a video encoder 200 provided in an embodiment of the present application. It should be understood that the video encoder 200 can be used to perform lossy compression on an image, or can be used to perform lossless compression on an image. The lossless compression can be visually lossless compression or mathematically lossless compression.

[0068] The video encoder 200 may be applied to image data in luminance and chrominance (YCbCr, YUV) format.

[0069] For example, the video encoder 200 reads video data, and for each frame of the video data, divides the frame into a number of coding tree units (CTUs). In some examples, CTBs may be referred to as "tree blocks", "largest coding units" (LCUs) or "coding tree blocks" (CTBs). Each CTU may be associated with a pixel block of equal size within the image. Each pixel may correspond to a luminance (luminance or luma) sample and two chrominance (chrominance or chroma) samples. Therefore, each CTU may be associated with a luminance sample block and two chrominance sample blocks. The size of a CTU is, for example, 128×128, 64×64, 32×32, etc. A CTU may be further divided into a number of coding units (CUs) for encoding, and a CU may be a rectangular block or a square block. CU can be further divided into prediction unit (PU) and transform unit (TU), so that coding, prediction and transform are separated and more flexible in processing. In one example, CTU is divided into CU in quadtree mode, and CU is divided into TU and PU in quadtree mode.

[0070] The video encoder and the video decoder may support various PU sizes. Assuming that the size of a particular CU is 2N×2N, the video encoder and the video decoder may support a PU size of 2N×2N or N×N for intra-frame prediction, and support symmetric PUs of 2N×2N, 2N×N, N×2N, N×N or similar sizes for inter-frame prediction. The video encoder and the video decoder may also support asymmetric PUs of 2N×nU, 2N×nD, nL×2N, and nR×2N for inter-frame prediction.

[0071] In some embodiments, Figure 2 As shown, the video encoder 200 may include: a prediction unit 210, a residual unit 220, a transform / quantization unit 230, an inverse transform / quantization unit 240, a reconstruction unit 250, a loop filter unit 260, a decoded image buffer 270 and an entropy coding unit 280. It should be noted that the video encoder 200 may include more, fewer or different functional components.

[0072] Optionally, in the present application, the current block may be referred to as a current coding unit (CU) or a current prediction unit (PU), etc. A prediction block may also be referred to as a prediction image block or an image prediction block, and a reconstructed image block may also be referred to as a reconstructed block or an image reconstructed image block.

[0073] In some embodiments, the prediction unit 210 includes an inter-frame prediction unit 211 and an intra-frame prediction unit 212. Since there is a strong correlation between adjacent pixels in a frame of a video, the intra-frame prediction method is used in the video coding and decoding technology to eliminate the spatial redundancy between adjacent pixels. Since there is a strong similarity between adjacent frames in a video, the inter-frame prediction method is used in the video coding and decoding technology to eliminate the temporal redundancy between adjacent frames, thereby improving the coding efficiency.

[0074] The inter-frame prediction unit 211 can be used for inter-frame prediction. Inter-frame prediction can refer to the image information of different frames. Inter-frame prediction uses motion information to find reference blocks from reference frames and generates prediction blocks based on the reference blocks to eliminate temporal redundancy. The frames used for inter-frame prediction can be P frames and / or B frames. P frames refer to forward prediction frames, and B frames refer to bidirectional prediction frames. Motion information includes a reference frame list where the reference frame is located, a reference frame index, and a motion vector. The motion vector can be an integer pixel or a sub-pixel. If the motion vector is a sub-pixel, it is necessary to use interpolation filtering in the reference frame to make the required sub-pixel block. Here, the integer pixel or sub-pixel block in the reference frame found according to the motion vector is called a reference block. Some technologies will directly use the reference block as a prediction block, and some technologies will generate a prediction block based on the reference block. Reprocessing the prediction block based on the reference block can also be understood as using the reference block as a prediction block and then processing the prediction block to generate a new prediction block.

[0075] The most commonly used inter-frame prediction methods currently include: geometric partitioning mode (GPM) in the VVC video codec standard and angular weighted prediction (AWP) in the AVS3 video codec standard. These two intra-frame prediction modes have something in common in principle.

[0076] The intra-frame prediction unit 212 only refers to the information of the same frame image to predict the pixel information in the current code image block to eliminate spatial redundancy. The frame used for intra-frame prediction can be an I frame.

[0077] The intra-frame prediction modes used by HEVC are Planar, DC, and 33 angle modes, for a total of 35 prediction modes. The intra-frame modes used by VVC are Planar, DC, and 65 angle modes, for a total of 67 prediction modes. The intra-frame modes used by AVS3 are DC, Plane, Bilinear, and 63 angle modes, for a total of 66 prediction modes.

[0078] In some embodiments, the intra prediction unit 212 may be implemented using intra block copy technology and intra string copy technology.

[0079] The residual unit 220 may generate a residual block of the CU based on the pixel blocks of the CU and the prediction blocks of the PUs of the CU. For example, the residual unit 220 may generate a residual block of the CU so that each sample in the residual block has a value equal to the difference between the following two: a sample in the pixel blocks of the CU and a corresponding sample in the prediction blocks of the PUs of the CU.

[0080] The transform / quantization unit 230 may quantize the transform coefficients. The transform / quantization unit 230 may quantize the transform coefficients associated with the TUs of the CU based on a quantization parameter (QP) value associated with the CU. The video encoder 200 may adjust the degree of quantization applied to the transform coefficients associated with the CU by adjusting the QP value associated with the CU.

[0081] The inverse transform / quantization unit 240 may apply inverse quantization and inverse transform to the quantized transform coefficients, respectively, to reconstruct a residual block from the quantized transform coefficients.

[0082] The reconstruction unit 250 may add the samples of the reconstructed residual block to the corresponding samples of one or more prediction blocks generated by the prediction unit 210 to generate a reconstructed image block associated with the TU. By reconstructing the sample blocks of each TU of the CU in this manner, the video encoder 200 may reconstruct the pixel blocks of the CU.

[0083] Loop filtering unit 260 may perform a deblocking filtering operation to reduce blocking effects of pixel blocks associated with a CU.

[0084] In some embodiments, the loop filter unit 260 includes a deblocking filter unit and a sample adaptive offset / adaptive loop filter (SAO / ALF) unit, wherein the deblocking filter unit is used to remove blocking effects, and the SAO / ALF unit is used to remove ringing effects.

[0085] The decoded image buffer 270 may store the reconstructed pixel blocks. The inter prediction unit 211 may use the reference image containing the reconstructed pixel blocks to perform inter prediction on PUs of other images. In addition, the intra prediction unit 212 may use the reconstructed pixel blocks in the decoded image buffer 270 to perform intra prediction on other PUs in the same image as the CU.

[0086] The entropy encoding unit 280 may receive the quantized transform coefficients from the transform / quantization unit 230. The entropy encoding unit 280 may perform one or more entropy encoding operations on the quantized transform coefficients to generate entropy-encoded data.

[0087] Figure 3It is a schematic block diagram of a decoding framework 300 provided in an embodiment of the present application.

[0088] like Figure 3 As shown, the video decoder 300 includes an entropy decoding unit 310, a prediction unit 320, an inverse quantization / transformation unit 330, a reconstruction unit 340, a loop filter unit 350, and a decoded image buffer 360. It should be noted that the video decoder 300 may include more, fewer, or different functional components.

[0089] The video decoder 300 may receive a bitstream. The entropy decoding unit 310 may parse the bitstream to extract syntax elements from the bitstream. As part of parsing the bitstream, the entropy decoding unit 310 may parse the syntax elements in the bitstream that have been entropy encoded. The prediction unit 320, the inverse quantization / transformation unit 330, the reconstruction unit 340, and the loop filter unit 350 may decode the video data according to the syntax elements extracted from the bitstream, that is, generate decoded video data.

[0090] In some embodiments, the prediction unit 320 includes an intra-prediction unit 321 and an inter-prediction unit 322 .

[0091] The intra prediction unit 321 may perform intra prediction to generate a prediction block of a PU. The intra prediction unit 321 may use an intra prediction mode to generate a prediction block of a PU based on a pixel block of a spatially neighboring PU. The intra prediction unit 321 may also determine an intra prediction mode of a PU according to one or more syntax elements parsed from a code stream.

[0092] The inter prediction unit 322 may construct a first reference image list (list 0) and a second reference image list (list 1) according to the syntax elements parsed from the code stream. In addition, if the PU is encoded using inter prediction, the entropy decoding unit 310 may parse the motion information of the PU. The inter prediction unit 322 may determine one or more reference blocks of the PU according to the motion information of the PU. The inter prediction unit 322 may generate a prediction block of the PU according to the one or more reference blocks of the PU.

[0093] The inverse quantization / transform unit 330 may inversely quantize (ie, dequantize) the transform coefficients associated with the TU. The inverse quantization / transform unit 330 may use the QP value associated with the CU of the TU to determine the degree of quantization.

[0094] After inverse quantizing the transform coefficients, the inverse quantization / transform unit 330 may apply one or more inverse transforms to the inverse quantized transform coefficients in order to generate a residual block associated with the TU.

[0095] The reconstruction unit 340 uses the residual block associated with the TU of the CU and the prediction block of the PU of the CU to reconstruct the pixel block of the CU. For example, the reconstruction unit 340 may add samples of the residual block to corresponding samples of the prediction block to reconstruct the pixel block of the CU to obtain a reconstructed image block.

[0096] The loop filtering unit 350 may perform a deblocking filtering operation to reduce blocking effects of pixel blocks associated with a CU.

[0097] The video decoder 300 may store the reconstructed image of the CU in the decoded image buffer 360. The video decoder 300 may use the reconstructed image in the decoded image buffer 360 as a reference image for subsequent prediction, or transmit the reconstructed image to a display device for presentation.

[0098] The basic process of video encoding and decoding is as follows: at the encoding end, a frame of image is divided into blocks, and for the current block, the prediction unit 210 uses intra-frame prediction or inter-frame prediction to generate a prediction block of the current block. The residual unit 220 can calculate the residual block based on the original block of the prediction block and the current block, that is, the difference between the original block of the prediction block and the current block, and the residual block can also be called residual information. The residual block can remove information that is not sensitive to the human eye through the transformation and quantization process of the transformation / quantization unit 230 to eliminate visual redundancy. Optionally, the residual block before transformation and quantization by the transformation / quantization unit 230 can be called a time domain residual block, and the time domain residual block after transformation and quantization by the transformation / quantization unit 230 can be called a frequency residual block or a frequency domain residual block. The entropy coding unit 280 receives the quantized change coefficient output by the change quantization unit 230, and can entropy encode the quantized change coefficient and output a bit stream. For example, the entropy coding unit 280 can eliminate character redundancy according to the target context model and the probability information of the binary bit stream.

[0099] At the decoding end, the entropy decoding unit 310 can parse the code stream to obtain the prediction information, quantization coefficient matrix, etc. of the current block. The prediction unit 320 uses intra-frame prediction or inter-frame prediction to generate a prediction block of the current block based on the prediction information. The inverse quantization / transformation unit 330 uses the quantization coefficient matrix obtained from the code stream to inversely quantize and inversely transform the quantization coefficient matrix to obtain a residual block. The reconstruction unit 340 adds the prediction block and the residual block to obtain a reconstructed block. The reconstructed blocks constitute a reconstructed image, and the loop filtering unit 350 performs loop filtering on the reconstructed image based on the image or on the block to obtain a decoded image. The encoding end also requires similar operations as the decoding end to obtain a decoded image. The decoded image can also be called a reconstructed image, and the reconstructed image can be used as a reference frame for inter-frame prediction for subsequent frames.

[0100] It should be noted that the block division information determined by the encoder, as well as the mode information or parameter information such as prediction, transformation, quantization, entropy coding, loop filtering, etc., are carried in the bitstream when necessary. The decoder parses the bitstream and determines the same block division information, prediction, transformation, quantization, entropy coding, loop filtering, etc. mode information or parameter information as the encoder by analyzing the existing information, thereby ensuring that the decoded image obtained by the encoder is the same as the decoded image obtained by the decoder.

[0101] The above is the basic process of the video codec under the block-based hybrid coding framework. With the development of technology, some modules or steps of the framework or process may be optimized. The present application is applicable to the basic process of the video codec under the block-based hybrid coding framework, but is not limited to the framework and process.

[0102] The intra-frame block copy technology and the intra-frame string copy technology are introduced below.

[0103] Intra Block Copy (IBC) is an intra-frame coding tool adopted in the HEVC Screen Content Coding (SCC) extension, which significantly improves the coding efficiency of screen content. In AVS3 and VVC, IBC technology is also adopted to improve the performance of screen content coding. IBC uses the spatial correlation of screen content video and uses the coded image pixels on the current image to predict the pixels of the current block to be coded, which can effectively save the bits required for coding pixels. Figure 4 As shown in Figure 1, the displacement between the current coding block and its reference block in IBC is called a block vector (BV for short). VVC uses a mode similar to the AMVP mode in inter-frame prediction to predict BV and allows BVD to be encoded using 1 or 4 pixel resolution.

[0104] Intra String Copy (ISC) is also called intra prediction. It divides a coding block into a series of pixel strings or unmatched pixels according to a certain scanning order (raster scanning, round-trip scanning, Zig-Zag scanning, etc.). Similar to IBC, the current string searches for a reference string of the same shape in the encoded area of ​​the current frame image. Based on this, the prediction information of the current string is derived. By encoding the original signal and prediction information of the current string, the residual signal of the current string is obtained, and the residual signal is encoded. For example: Figure 5 is a schematic diagram of an intra-frame string replication involved in an embodiment of the present application, such as Figure 5As shown, the 28 white pixels are string 1, the 35 light gray pixels are string 2, and the 1 black pixel represents an unmatched pixel (an unmatched pixel is also called an isolated point, and the pixel value of the unmatched pixel is directly encoded instead of being derived from the predicted value of the reference string). Among them, the reference string of string 1 is on its left, and the displacement of string 1 to its corresponding reference string is represented by string vector 1. The reference string of string 2 is above it, and the displacement of string 2 to its corresponding reference string is represented by string vector 2.

[0105] The intra-frame string replication technology needs to encode the string vector (SV) corresponding to each string in the current coding block, the string length, and the flag of whether there is a matching string, etc. Among them, the string vector (SV) represents the displacement of the string to be encoded to its reference string. The string length represents the number of pixels contained in the string.

[0106] The equivalent string and unit vector string mode is a sub-mode of intra-frame string replication and was adopted into the AVS3 standard in October 2020. Similar to intra-frame string replication, this mode divides a coding / decoding block into a series of pixel strings or unmatched pixels in a certain scanning order. The type of pixel string can be an equivalent string or a unit base vector string. The characteristic of an equivalent string is that all pixels in the pixel string have the same prediction value. The unit vector string (also called unit base vector string, unit offset string, copy above string, etc.) is characterized in that its displacement vector is (0, -1), and its implementation is relatively simple. Each pixel of the string uses the pixel above as the prediction value of the current pixel. The equivalent string mode requires encoding the type, length, and prediction value information of each string of the current coding block in the bitstream. Accordingly, the decoding end derives the prediction sample from the information decoded from the bitstream.

[0107] There are several ways to encode the prediction value: 1) directly encode the prediction value; 2) construct a reference pixel candidate list L1, and encode the index of the prediction value in L1; 3) construct a first reference pixel list L0, derive the reference pixel candidate list L1 from the list according to the reuse flag reuse_flag, encode reuse_flag and the index of the encoded prediction value in L1. In the current implementation of the equivalent string, the above method 3) is used to encode the prediction value. It should be noted that the above method does not record the prediction value in the list, but records the position of the reference pixel in the image.

[0108] At present, the medium value string and unit basis vector string modes in AVS3 are implemented for video images in YUV420 format. The current prediction sample matrix is ​​recorded as pred_y, pred_cb, pred_cr. This method uses a memory with a height of LCU height, a width of image width, and a channel number of 3 to derive the prediction sample, recorded as LcuRowBuf[ch][x][y]. Then, the value of a pixel sample can be confirmed according to the channel, horizontal coordinate, and vertical coordinate of the color component.

[0109] The current process of exporting prediction samples in AVS3 is as follows:

[0110] Step 1: For a CU with equal value string and unit base vector string mode, the CU is divided into a series of equal value strings, unit base vector strings or unmatched pixels. The prediction samples of each part are derived in turn according to the following method. The coordinates of the current pixel are (x, y):

[0111] Case 1: If the type of the current string where the current pixel is located is an equal value string, execute the following steps 11 to 13:

[0112] Step 11, decoding the reference pixel value of the current pixel from the bitstream, for example, directly decoding from the bitstream or confirming by a point vector, and parsing comp_flag from the bitstream, where comp_flag is used to indicate whether the chrominance component value of the current pixel exists;

[0113] Step 12, assign the Y component LcuRowBuf[0][x][y] of the current position (x, y);

[0114] Step 13, if the chrominance component value of the current position (x, y) exists, assign the Cb component LcuRowBuf[1][x][y] and the Cr component LcuRowBuf[2][x][y] of the current position (x, y);

[0115] Case 2: If the type of the string where the current pixel is located is an unmatched pixel, the following steps 21 and 22 are executed:

[0116] Step 21, if the coordinates of the current pixel are an integer multiple of 2 (i.e., x%2==0&&y%2==0), decode the pixel values ​​of the Y, Cb, and Cr components from the bitstream to obtain LcuRowBuf[0][x][y], LcuRowBuf[1][x][y], LcuRowBuf[2][x][y];

[0117] Step 22, if the coordinates of the current pixel point are not an integer multiple of 2, decode the pixel value of the Y component from the bitstream, LcuRowBuf[0][x][y], and set the values ​​of the Cb and Cr components LcuRowBuf[1][x][y], LcuRowBuf[2][x][y] to 0.

[0118] Case 3: If the type of the string where the current pixel is located is a unit base vector string, the pixel value of the adjacent pixel above the current pixel is used as the reference pixel value of the current pixel, that is, LcuRowBuf[0][x][y] = LcuRowBuf[0][x][y-1], LcuRowBuf[1][x][y] = LcuRowBuf[1][x][y-1], LcuRowBuf[2][x][y] = LcuRowBuf[2][x][y-1];

[0119] Step 2: After the prediction samples of the entire CU are derived, the prediction samples of the chroma components are downsampled, and the chroma sample matrix of size w×h is downsampled to a chroma sample matrix of size w / 2×h / 2. Specifically, the non-zero pixels in each 2×2 sub-block in the prediction samples of the entire CU are averaged to obtain the downsampled chroma sample values.

[0120] Step 3: derive the values ​​of pred_y, pred_u and pred_v according to the position of each pixel in the CU. For the luma prediction sample, pred_y is the pixel value in LcuRowBuf[0], pred_cb is the downsampled chroma sample value in LcuRowBuf[1], and pred_cr is the downsampled chroma sample value in LcuRowBuf[2].

[0121] From the above, it can be seen that in the current AVS3, for video images in YUV4:2:0 format or YUV4:2:2, the prediction sample export needs to use the YUV4:4:4 size internal, which occupies a large memory space and is not conducive to the implementation of the decoding hardware device.

[0122] In order to solve the above technical problems, the present application determines at least one of a reference pixel index matrix, an unmatched pixel index matrix and an unmatched pixel list, and determines the prediction samples of the pixel points based on the reference pixel index matrix, the unmatched pixel index matrix and at least one of the unmatched pixel list. Then, based on the prediction samples of the pixel points, the prediction samples of the image block are determined. There is no need to export the prediction samples of YUV444 and then downsample them, thereby reducing the memory size required for exporting the prediction samples, which is beneficial to hardware implementation.

[0123] The technical solution provided in the embodiments of the present application is described in detail below in conjunction with specific embodiments.

[0124] It should be noted that the method of the present application is applicable to the encoding end and the decoding end, and the process of determining the image block prediction sample at the encoding end and the decoding end is basically similar. Here, the method for determining the image block prediction sample provided by the present application is first described by taking the decoding end as an example, and the encoding end can refer to the decoding end.

[0125] Figure 6 A flowchart of a method for determining image block prediction samples provided in an embodiment of the present application. The method in the embodiment of the present application is applied to Figure 1 or Figure 3 The decoding end shown in Figure 6 As shown, including:

[0126] S610: Obtain an index of a reference pixel corresponding to at least one pixel in the image block.

[0127] The execution subject of the embodiments of the present application includes but is not limited to the following devices: a decoder, or a device for determining image block prediction samples, such as a desktop computer, a mobile computing device, a notebook (e.g., laptop) computer, a tablet computer, a set-top box, a handheld device such as a smart phone, a television, a camera, a display device, a digital media player, a video game console, a car computer, or the like.

[0128] At the encoding end, the image block is divided into a series of pixel strings according to a certain scanning order, and each pixel string includes at least one pixel point.

[0129] The encoding end encodes the length, type and prediction value information of the pixel string in the bitstream. In this way, when decoding an image block, the decoding end can decode the type of the pixel string where the pixel points in the image block are located from the bitstream, and the pixel string includes at least one pixel point.

[0130] Optionally, the type of the pixel string includes any one of the following: an equal value string, an unmatched pixel, and a unit base vector string.

[0131] After decoding the type of the pixel string to which each pixel in at least one pixel in the image block belongs from the code stream, the value of the reference pixel of the pixel is determined according to the type of the pixel string.

[0132] For each pixel of the at least one pixel, the index of the reference pixel corresponding to each pixel is determined according to the following method, specifically as follows:

[0133] Case 1: If the type of the pixel string to which the pixel point belongs is an equal-value string, the decoded code stream obtains the index of the reference pixel corresponding to the pixel string. Since the reference image value of each pixel point in the equal-value string is the same, the index of the reference pixel corresponding to the equal-value string to which the pixel point belongs can be determined as the index of the reference pixel corresponding to the pixel point.

[0134] In this manner, the value of the reference pixel corresponding to the pixel point can also be determined. For example, according to the index of the reference pixel corresponding to the pixel point, the value of the reference pixel corresponding to the index is searched in the preset first reference pixel list ref_list, and the value of the reference pixel corresponding to the index is determined as the value of the reference pixel of the pixel point.

[0135] Exemplarily, the preset first reference pixel list ref_list is shown in Table 1:

[0136] Table 1

[0137] index The value of the reference pixel 1 A1 2 A2 3 A3 …… ……

[0138] Since all pixels in an equal value string have the same reference pixel value, if the type of the pixel string to which the pixel belongs is an equal value string, the encoder encodes the index of the reference pixel value corresponding to the pixel string in the bitstream, for example, the reference pixel value corresponding to all pixels in the pixel string is A2, and the index of the reference pixel value A2 is 2, and the encoder encodes the index 2 in the bitstream. The decoder decodes the bitstream and obtains the index of the reference pixel value corresponding to the pixel string, for example, 2, and then searches the reference pixel value A2 corresponding to the index 2 from the above Table 1, and determines the reference pixel value A2 as the reference pixel value corresponding to the pixel string.

[0139] Since the values ​​of the reference pixels corresponding to all the pixels in the pixel string are the same, namely A2, it can be determined that the value of the reference pixel of the current pixel is also A2.

[0140] Case 2: If the type of the pixel string to which the pixel belongs is an unmatched pixel, the pixel can be optionally called an unmatched pixel. In this way, the predicted value of the pixel is obtained by decoding the bitstream, and the predicted value of the pixel is placed in the unmatched pixel list. According to the position of the predicted value of the pixel in the unmatched pixel list, the index of the reference pixel corresponding to the pixel is determined.

[0141] Specifically, if the type of the pixel string to which the pixel point belongs is an unmatched pixel, it means that the pixel string includes a pixel point, such as Figure 5The unmatched pixel shown in the black area is the pixel point. When encoding, the encoder encodes the predicted value of the unmatched pixel in the bitstream. In this way, if the pixel point is an unmatched pixel point, the decoder can directly decode the predicted value of the pixel point from the bitstream, place the predicted value of the pixel point in the unmatched pixel list, and determine the index of the reference pixel corresponding to the pixel point according to the position of the predicted value of the pixel point in the unmatched pixel list.

[0142] Case 3: If the type of the pixel string to which the pixel point belongs is a unit base vector string, then the value of the decoded pixel point adjacent to the pixel point is obtained, and the index corresponding to the value of the decoded pixel point is queried in the preset first reference pixel list, and the index is determined as the index of the reference pixel corresponding to the pixel point.

[0143] Specifically, since each pixel in the unit base vector string uses the value of the pixel adjacent to it as the value of its own reference pixel, if the type of the pixel string to which the pixel belongs is a unit base vector string, the decoding end obtains the value of the decoded pixel adjacent to the pixel, searches for the index corresponding to the value of the decoded pixel in the preset first reference pixel list, and determines the index as the index of the reference pixel corresponding to the pixel.

[0144] In some embodiments, the index of the reference pixel may also be understood as the index of the reference pixel position.

[0145] S620. Determine at least one of a reference pixel index matrix, an unmatched pixel index matrix, and an unmatched pixel list according to an index of a reference pixel corresponding to each pixel in at least one pixel.

[0146] The reference pixel index matrix ref_index includes the index of the reference pixel corresponding to each pixel point in at least one pixel point.

[0147] The unmatched pixel list unmatched_pixel_list includes a correspondence between the value of the unmatched pixel and the index of the unmatched pixel in at least one pixel point.

[0148] The unmatched pixel index matrix unmatched_pixel_index includes the index of the unmatched pixels in the unmatched pixel list unmatched_pixel_list in at least one pixel point.

[0149] It should be noted that the size of the reference pixel index matrix and the size of the unmatched pixel index matrix are determined by the number of at least one pixel. For example, if at least one pixel includes all the pixels in the image block, the size of the reference pixel index matrix and the size of the unmatched pixel index matrix are consistent with the size of the image block, for example, both are 8X8. If at least one pixel includes two rows of pixels in the image block, the size of the reference pixel index matrix and the size of the unmatched pixel index matrix are consistent with the size of the two rows of pixels, for example, 2X8.

[0150] In one example, suppose an image block such as Fig. 7A As shown, at least one pixel point is Figure 7B As shown, the reference pixel index matrix is ​​as follows Figure 7C As shown, each element represents the index of the reference pixel corresponding to the pixel point at the element position, wherein c1 indicates that the pixel point at the position is an unmatched pixel. Fig.7D is the unmatched pixel index matrix, each element indicates whether the pixel at the element position is an unmatched pixel, such as Fig.7D As shown, if the index value is 0, it means that the pixel at this position is not an unmatched pixel. If the index value is not 0, for example Fig.7D When c1 is shown in , it means that the pixel at this position is an unmatched pixel.

[0151] In some embodiments, the first reference pixel list ref_list includes chromaticity values ​​and luminance values ​​of reference pixels, for example, includes the values ​​of luminance Y and chromaticity Cb, Cr of each reference pixel.

[0152] In some embodiments, the first reference pixel list ref_list includes position information of the reference pixel, so that the value of the reference pixel can be obtained according to the position information of the reference pixel.

[0153] Optionally, the position information of the reference pixel is mainly used to confirm the position of the reference pixel. If the reference pixel is in the current region, the position information of the reference pixel may be the coordinates of the reference pixel in the current region. If the reference pixel is in the current LCU row, the position information of the reference pixel may be the coordinates of the reference pixel in the current LCU row.

[0154] In some embodiments, the first reference pixel list corresponds to a luminance sublist Y_only_list, and each element in the luminance sublist is used to indicate whether the corresponding reference pixel in the first reference pixel list has a luminance value that is not zero and a chrominance value that is zero. For example, Y_only_list is used to record information on whether the corresponding element of ref_list only has Y, and the chrominance values ​​Cb and Cr are set to 0.

[0155] In some embodiments, the unmatched pixel list corresponds to an unmatched pixel brightness sub-list ump_y_only_list, and each element in the unmatched pixel brightness sub-list ump_y_only_list is used to indicate whether the corresponding unmatched pixel in the unmatched pixel list has a brightness value that is not zero and a chrominance value that is zero. For example, ump_y_only_list is used to record whether the corresponding element of unmatched_pixel_list only has Y information, and the chrominance values ​​Cb and Cr are set to 0.

[0156] S630. For each pixel of the at least one pixel, determine a prediction sample of the pixel according to at least one of a reference pixel index matrix, an unmatched pixel index matrix, and an unmatched pixel list.

[0157] S640: Determine the prediction samples of the image block according to the prediction samples of the pixel points.

[0158] Since the process of determining the prediction sample of each pixel point in at least one pixel point is the same, for the convenience of description, one pixel point is taken as an example for explanation, and other pixel points can be used as a reference.

[0159] In the present application, the prediction samples of the pixel points are determined according to at least one of the reference pixel index matrix, the unmatched pixel index matrix and the unmatched pixel list, and the prediction samples of the image block are determined according to the prediction samples of the pixel points. Compared with the prior art that uses YUV4:4:4 size memory to derive the prediction samples and then downsamples them, the present application reduces the memory usage in the process of determining the prediction samples of the image block.

[0160] The process of determining the prediction sample of the pixel point according to at least one of the reference pixel index matrix, the unmatched pixel index matrix and the unmatched pixel list in the above S630 is described in detail below with reference to a specific example.

[0161] Figure 8 A flowchart of another method for determining image block prediction samples provided in an embodiment of the present application is shown in FIG. Figure 8 As shown, the above S630 includes:

[0162] S701, determining the type of the pixel;

[0163] S702: Determine a prediction sample for the pixel according to the type of the pixel and at least one of a reference pixel index matrix, an unmatched pixel index matrix, and an unmatched pixel list.

[0164] In some embodiments, the method of determining the type of the pixel in S701 includes but is not limited to the following methods:

[0165] Method 1: According to the position information of the pixel, the index of the reference pixel corresponding to the pixel is searched in the reference pixel index matrix; according to the index of the reference pixel corresponding to the pixel, the type of the pixel is determined. For example, if the value of the index of the reference pixel corresponding to the pixel is a first value, the type of the pixel is determined to be an unmatched pixel. Exemplarily, the first value is Figure 7C c1 in , that is, when the index value of the reference pixel corresponding to the pixel is c1, the type of the pixel is determined to be an unmatched pixel. For another example, if the index of the reference pixel corresponding to the pixel is the same as the index of the reference pixel corresponding to multiple pixels adjacent to the pixel, the type of the pixel is determined to be an equal value string. For another example, if the index of the reference pixel corresponding to the pixel is different from the index of the reference pixel corresponding to multiple pixels adjacent to the pixel, the type of the pixel is determined to be a unit base vector string.

[0166] Method 2: obtain the type of the pixel string to which each pixel in at least one pixel belongs, determine the type of the vector string to which the pixel belongs as the type of the pixel, and obtain a pixel type matrix, which includes the type of each pixel in at least one pixel. In this way, the type corresponding to the pixel can be queried in the pixel type matrix based on the position information of the pixel later.

[0167] The prediction samples of the pixel points of the present application include brightness prediction samples and / or chrominance prediction samples, wherein the brightness prediction samples can be understood as the prediction samples of the pixel points under the brightness component, and the chrominance prediction samples can be understood as the prediction samples of the pixel points under the chrominance component. In some embodiments, the prediction samples under the brightness component are also called brightness prediction values, and the prediction samples under the bright chrominance component are also called chrominance prediction values.

[0168] First, the process of determining the brightness prediction samples of the pixel points is introduced.

[0169] If the prediction samples of the pixel point include brightness prediction samples, the above S702 includes the following steps:

[0170] S702-A1, if the type of the pixel point is an equal value string or a unit base vector string, determine the brightness prediction sample of the pixel point according to the position information of the pixel point, the reference pixel index matrix and the preset first reference pixel list;

[0171] S702-A2: If the type of the pixel is an unmatched pixel, determine the brightness prediction sample of the pixel according to the position information of the pixel, the unmatched pixel index matrix and the unmatched pixel list.

[0172] In a possible implementation of S702-A1 above, according to the position information of the pixel, as well as the reference pixel index matrix and the preset first reference pixel list, determining the brightness prediction sample of the pixel includes: determining the index of the reference pixel corresponding to the pixel from the reference pixel index matrix according to the position information of the pixel; searching the value of the reference pixel corresponding to the pixel in the first reference pixel list according to the index of the reference pixel corresponding to the pixel; determining the brightness value of the reference pixel corresponding to the pixel as the brightness prediction sample value of the pixel. Optionally, the brightness value of the reference pixel corresponding to the pixel can also be processed, for example, multiplied by a preset coefficient, and the processed brightness value is determined as the brightness prediction sample value of the pixel.

[0173] In a possible implementation of S702-A2 above, according to the position information of the pixel point, as well as the unmatched pixel index matrix and the unmatched pixel list, determining the brightness prediction sample of the pixel point may include: determining the index of the unmatched pixel corresponding to the pixel point from the unmatched pixel index matrix according to the position information of the pixel point; querying the value of the unmatched pixel corresponding to the pixel point in the unmatched pixel list according to the index of the unmatched pixel corresponding to the pixel point; and determining the brightness value of the unmatched pixel corresponding to the pixel point as the brightness prediction sample value of the pixel point. Optionally, the brightness value of the unmatched pixel corresponding to the pixel point may also be processed, for example, multiplied by a preset coefficient, and the processed brightness value is determined as the brightness prediction sample value of the pixel point.

[0174] The following is an introduction to the process of determining the chrominance prediction sample for the pixel.

[0175] If the prediction samples of the pixel point include chrominance prediction samples, such as Fig. 9 As shown, the above S702 includes the following steps:

[0176] S801: Determine the color sampling mode of the image block and the position information of the chroma prediction samples of the pixel points.

[0177] During encoding, the encoding end divides the current image to be encoded into multiple image blocks, and encodes each image block. Correspondingly, the decoding end also decodes each image block in the current image. Therefore, the color sampling mode of the image block described in the embodiment of the present application is the color sampling mode of the image in which the image block is located.

[0178] In some embodiments, the color format of the video image of the embodiment of the present application is YUV. YUV is mainly used to optimize the transmission of color video signals. Compared with RGB video signal transmission, its biggest advantage is that it only takes up very little bandwidth (RGB requires three independent video signals to be transmitted simultaneously). Among them, "Y" represents brightness (Luminance or Luma), that is, the grayscale value; while "U" and "V" represent chrominance (Chrominance or Chroma), which is used to describe the color and saturation of the image and is used to specify the color of the pixel. "Brightness" is established through the RGB input signal by superimposing specific parts of the RGB signal together. "Chroma" defines two aspects of color-hue and saturation, represented by Cr and Cb respectively. Among them, Cr reflects the difference between the red part of the RGB input signal and the brightness value of the RGB signal. And Cb reflects the difference between the blue part of the RGB input signal and the brightness value of the RGB signal. The importance of using the YUV color space is that its brightness signal Y and chrominance signals U and V are separated. If there is only a Y signal component but no U and V components, then the image represented in this way is a black and white grayscale image. Color television uses YUV space in order to use the brightness signal Y to solve the compatibility problem between color television and black and white television, so that black and white television can also receive color television signals.

[0179] The storage format of the YUV code stream is actually closely related to its color sampling mode. There are three mainstream color sampling modes: YUV4:4:4, YUV4:2:2 and YUV4:2:0. The numbers in the ratio N1:N2:N3 are the relative sampling rates in the horizontal direction. N1 represents the number of Y samples in odd and even rows, N2 represents the number of U and V samples in odd rows, and N3 represents the number of U and V samples in even rows.

[0180] Among them, Fig. 10A As shown, YUV4:4:4 means that the chroma channels are not downsampled, that is, each Y sample corresponds to a set of UV samples.

[0181] like Fig. 10B As shown, YUV4:2:2 means 2:1 horizontal downsampling, no vertical downsampling. That is, for each scan line, every two U or V samples correspond to four Y samples. That is, every two Y samples share a set of UV components.

[0182] like Fig. 10C As shown, YUV4:2:0 means 2:1 horizontal downsampling and 2:1 vertical downsampling, that is, every four Y samples share a set of UV samples.

[0183] It should be noted that the color sampling modes involved in the embodiments of the present application include but are not limited to Fig. 10A , Fig. 10B and Fig. 10C Optionally, a color sampling mode for downsampling the brightness component is also included, such as YUV3:2:0, YUV3:2:2, etc.

[0184] In some embodiments, the color sampling mode of the image block is a default mode, for example, the color sampling mode of the image block of the encoding end and the decoding end is YUV4:2:0 by default.

[0185] In some embodiments, the encoding end carries the color sampling mode information of the image block in the bitstream, so that the decoding end can obtain the color sampling mode of the image block by decoding the bitstream.

[0186] In this application, the size of the chroma prediction sample of a pixel is related to the color sampling mode of the image block. For example, when the color sampling mode is YUV444, the size of the chroma prediction sample of the pixel is the same as the size of the brightness prediction sample of the pixel, and the position information of the chroma prediction sample of the pixel is the position of the brightness prediction sample of the pixel. When the color sampling mode is YUV422, the chroma prediction sample of the pixel corresponds to the two brightness prediction samples of the pixel. If the position coordinates of at least one pixel are as follows: Figure 7B As shown, if the color sampling mode is YUV422, the size of the chrominance prediction sample corresponding to the at least one pixel point is as follows Fig.11A For example, when the position coordinates of a pixel are (1,2), the position coordinates of the chrominance prediction sample of the pixel are (0,2). When the color sampling mode is YUV420, the chrominance prediction sample of the pixel corresponds to the 4 brightness prediction samples of the pixel. If at least one pixel has a position coordinate of (1,2), the chrominance prediction sample of the pixel corresponds to the 4 brightness prediction samples of the pixel. Figure 7B As shown, if the color sampling mode is YUV420, the size of the chrominance prediction sample corresponding to the at least one pixel point is as follows Fig. 11B As shown, for example, when the position coordinates of a pixel point are (1, 2), the position coordinates of the chrominance prediction sample of the pixel point are (0, 2).

[0187] S802. Determine position information of M luminance prediction samples corresponding to the chrominance prediction samples of the pixel according to the color sampling mode of the image block and the position information of the chrominance prediction samples of the pixel, where M is a positive integer.

[0188] In some embodiments, if the color sampling mode is YUV4:2:0, M is 4; if the color sampling mode is YUV4:2:2, M is 2; if the color sampling mode is YUV4:4:4, M is 1.

[0189] Specifically, the M brightness prediction samples corresponding to the chroma prediction samples of the pixel point can be determined according to the color sampling mode of the image block. For example, if the color sampling mode of the image block is YUV444, the chroma prediction sample of the pixel point corresponds to one brightness prediction sample; if the color sampling mode of the image block is YUV422, the chroma prediction sample of the pixel point corresponds to two brightness prediction samples; if the color sampling mode of the image block is YUV420, the chroma prediction sample of the pixel point corresponds to 4 brightness prediction samples.

[0190] For example, if the color sampling mode of the image block is YUV422, and the position coordinates of the pixel point are (1,2), Fig.11A The position coordinates of the chrominance prediction sample of the pixel point are (0,2), and the position coordinates of the chrominance prediction sample (0,2) correspond to two brightness prediction samples, and the position coordinates of these two brightness prediction samples are (0,2) and (1,2). If the color sampling mode of the image block is YUV420, assuming that the position coordinates of the pixel point are (1,2), in Fig. 11B The position coordinates of the chrominance prediction sample of the pixel point are (0,2), and the position coordinates of the chrominance prediction sample (0,2) correspond to 4 brightness prediction samples, and the position coordinates of these 4 brightness prediction samples are (0,2), (1,2), (0,3) and (1,3).

[0191] S803. Determine the value of a reference pixel corresponding to the position information of at least one brightness prediction sample among the M brightness prediction samples according to the position information of the M brightness prediction samples and the type of the pixel points, and at least one of the reference pixel index matrix, the unmatched pixel index matrix and the unmatched pixel list.

[0192] In the present application, when the types of pixel points are different, the above process of determining the value of the reference pixel corresponding to the position information of at least one brightness prediction sample among the M brightness prediction samples based on the position information of the M brightness prediction samples, and at least one of the reference pixel index matrix, the unmatched pixel index matrix and the unmatched pixel list is different.

[0193] In some embodiments, if the type of pixel point is an equal value string or a unit basis vector string, at this time, the above S803 includes S803-A: according to the position information of M brightness prediction samples, as well as the reference pixel index matrix and the preset first reference pixel list, determine the value of the reference pixel corresponding to the position information of at least one brightness prediction sample among the M brightness prediction samples.

[0194] For example, based on the position information of M brightness prediction samples, the index of the reference pixel corresponding to the position information of at least one brightness prediction sample among the M brightness prediction samples is queried in the reference pixel index matrix; based on the index of the reference pixel corresponding to the position information of at least one brightness prediction sample among the M brightness prediction samples, the value of the reference pixel corresponding to the position information of at least one brightness prediction sample among the M brightness prediction samples is determined from the first reference pixel list.

[0195] In some embodiments, if the type of the pixel point is an unmatched pixel, at this time, the above S803 includes S803-B: determining the value of the reference pixel corresponding to the position information of at least one brightness prediction sample among the M brightness prediction samples based on the position information of the M brightness prediction samples, the unmatched pixel index matrix and the unmatched pixel list.

[0196] For example, based on the position information of M brightness prediction samples, determine the first brightness prediction sample among the M brightness prediction samples that has the same position information as the pixel point; based on the position information of the first brightness prediction sample, determine the index of the reference pixel corresponding to the position information of the first brightness prediction sample in an unmatched pixel index matrix or a reference pixel index matrix; based on the index of the reference pixel corresponding to the position information of the first brightness prediction sample, determine the value of the reference pixel corresponding to the position information of the first brightness prediction sample in an unmatched pixel list.

[0197] In this example, if M is greater than or equal to 2, the M brightness prediction samples also include a second brightness prediction sample corresponding to the position information of the pixel point. At this time, the method of the embodiment of the present application also includes: according to the position information of the second brightness prediction sample, querying the index of the reference pixel corresponding to the position information of the second brightness prediction sample in the reference pixel index matrix; according to the index of the reference pixel corresponding to the position information of the second brightness prediction sample, determining the value of the reference pixel corresponding to the position information of the second brightness prediction sample from the first reference pixel list.

[0198] S804: Determine a chrominance prediction sample value of the pixel point according to the value of a reference pixel corresponding to the position information of at least one luminance prediction sample among the M luminance prediction samples.

[0199] The methods for determining the chrominance prediction sample value of the pixel in S804 include but are not limited to the following:

[0200] Method 1: determine the value of the reference pixel corresponding to the position information of a target brightness prediction sample among M brightness prediction samples; determine the chrominance prediction sample value of the pixel point according to the chrominance value of the reference pixel corresponding to the position information of the target brightness prediction sample.

[0201] Optionally, the target brightness prediction sample may be any brightness prediction sample among the M brightness prediction samples.

[0202] Optionally, the target brightness prediction sample is a brightness prediction sample located at the upper left corner among the M brightness prediction samples.

[0203] In some embodiments, after obtaining the position information of the target predicted brightness sample, the chrominance value of the reference pixel corresponding to the position information of the target predicted brightness sample is used to determine the chrominance predicted sample value of the pixel point.

[0204] In some embodiments, after obtaining the position information of the target predicted brightness sample, it is determined whether the chrominance value of the reference pixel corresponding to the position information of the target predicted brightness sample is 0.

[0205] If it is determined that the chromaticity value of the reference pixel corresponding to the position information of the target predicted brightness sample is not 0, the chromaticity value of the reference pixel corresponding to the position information of the target predicted brightness sample is used to determine the chromaticity predicted sample value of the pixel.

[0206] If the chroma value of the reference pixel corresponding to the position information of the target luminance prediction sample is 0, the decoded chroma prediction sample value adjacent to the chroma prediction sample above the pixel is determined as the chroma prediction sample value of the pixel. For example, if the position coordinates of the chroma prediction sample of the pixel are (0,2), the value of the chroma prediction sample (0,0) is determined as the chroma prediction sample value of the pixel. Alternatively, M luminance prediction samples corresponding to the chroma prediction samples adjacent to the chroma prediction sample above the pixel are obtained, and the chroma value of the reference pixel corresponding to the position information of a luminance prediction sample in the upper left corner of the M luminance prediction samples is determined as the chroma prediction value of the pixel.

[0207] Method 2: If the above-mentioned first reference pixel list includes position information of reference pixels, a target brightness prediction sample is determined from M brightness prediction samples; based on the position information of the target brightness prediction sample, if it is determined that the coordinates of the target brightness prediction sample are located in a preset position list, the chromaticity value of the reference pixel corresponding to the coordinates of the target brightness prediction sample is determined as the chromaticity prediction sample value of the pixel point; or, if it is determined that the first pixel point corresponding to the coordinates of the target brightness prediction sample belongs to an equal value string, and the first pixel point is the first pixel point in the equal value string whose coordinate value is an integer multiple of 2, the chromaticity value of the reference pixel corresponding to the coordinates of the target brightness prediction sample is determined as the chromaticity prediction sample value of the pixel point.

[0208] The preset position list includes at least one of the following: a first reference pixel list and a second reference pixel list.

[0209] The second reference pixel list may be understood as an updated list of the first reference pixel list.

[0210] In some embodiments, if the reference pixel corresponding to the coordinates of the target brightness prediction sample is not in the preset position list, or the first pixel corresponding to the coordinates of the target brightness prediction sample does not belong to the equal value string, or the first pixel belongs to the equal value string and the first pixel is not the pixel whose first coordinate value is an integer multiple of 2 in the equal value string, the method further includes: downsampling the chromaticity value of the reference pixel corresponding to the position information of at least one brightness prediction sample among the M brightness prediction samples to obtain the chromaticity prediction sample value of the pixel. For example, the average value of the chromaticity value of the reference pixel corresponding to the position information of at least one brightness prediction sample among the M brightness prediction samples is used as the chromaticity prediction sample value of the pixel; or the average value of the non-zero chromaticity value among the chromaticity values ​​of the reference pixel corresponding to the position information of at least one brightness prediction sample among the M brightness prediction samples is used as the chromaticity prediction sample value of the pixel.

[0211] Method three, down-sample the chrominance value of the reference pixel corresponding to the position information of at least one luminance prediction sample among the M luminance prediction samples to obtain the chrominance prediction sample value of the pixel. For example, the average value of the chrominance values ​​of the reference pixels corresponding to the position information of at least one luminance prediction sample among the M luminance prediction samples is used as the chrominance prediction sample value of the pixel; or the average value of the non-zero chrominance values ​​of the chrominance values ​​of the reference pixels corresponding to the position information of at least one luminance prediction sample among the M luminance prediction samples is used as the chrominance prediction sample value of the pixel.

[0212] It should be understood that Figures 6 to 11B This is only an example of the present application and should not be construed as limiting the present application.

[0213] The preferred embodiments of the present application are described in detail above in conjunction with the accompanying drawings. However, the present application is not limited to the specific details in the above embodiments. Within the technical concept of the present application, the technical solution of the present application can be subjected to a variety of simple modifications, and these simple modifications all belong to the protection scope of the present application. For example, the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present application will not further explain various possible combinations. For another example, the various different embodiments of the present application can also be arbitrarily combined, as long as they do not violate the ideas of the present application, they should also be regarded as the contents disclosed in the present application.

[0214] It should also be understood that in the various method embodiments of the present application, the size of the sequence number of each process does not mean the order of execution, and the order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. In addition, in the embodiments of the present application, the term "and / or" is merely a description of the association relationship of associated objects, indicating that three relationships may exist. Specifically, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the objects associated before and after are in an "or" relationship.

[0215] Combination of the above Figures 6 to 11B , describes in detail the method embodiment of the present application, and the following is combined with Figures 12 to Fig.13 , describe in detail the device embodiments of the present application.

[0216] Fig.12 1 is a schematic block diagram of an apparatus for determining image block prediction samples provided in an embodiment of the present application, and the apparatus may belong to a decoding end, such as a decoding device. Optionally, the apparatus may also belong to an encoding end, such as an encoding device.

[0217] like Fig.12 As shown, the image block prediction sample determination device 10 may include:

[0218] An acquisition unit 11 is used to obtain an index of a reference pixel corresponding to each pixel in at least one pixel in the image block;

[0219] A first determining unit 12 is used to determine at least one of a reference pixel index matrix, an unmatched pixel index matrix and an unmatched pixel list according to an index of a reference pixel corresponding to each pixel in the at least one pixel;

[0220] A second determining unit 13 is used to determine, for each pixel of the at least one pixel, a prediction sample of the pixel according to at least one of the reference pixel index matrix, the unmatched pixel index matrix and the unmatched pixel list;

[0221] A third determining unit 14 is used to determine the prediction sample of the image block according to the prediction sample of the pixel point;

[0222] Among them, the reference pixel index matrix includes the index of the reference pixel corresponding to each pixel in the at least one pixel point, the unmatched pixel list includes the correspondence between the value of the unmatched pixel in the at least one pixel point and the index of the unmatched pixel, and the unmatched pixel index matrix includes the index of the unmatched pixel in the at least one pixel point in the unmatched pixel list.

[0223] In some embodiments, the type of the pixel string to which the pixel point belongs includes any one of the following: an equal value string, an unmatched pixel, and a unit base vector string.

[0224] In some embodiments, the second determination unit 13 is specifically used to determine the type of the pixel point; determine the predicted sample of the pixel point according to the type of the pixel point and at least one of the reference pixel index matrix, the unmatched pixel index matrix and the unmatched pixel list.

[0225] In some embodiments, the second determination unit 13 is specifically used to query the index of the reference pixel corresponding to the pixel point in the reference pixel index matrix according to the position information of the pixel point; and determine the type of the pixel point according to the index of the reference pixel corresponding to the pixel point.

[0226] In some embodiments, the second determination unit 13 is specifically configured to determine that the type of the pixel point is an unmatched pixel if the value of the index of the reference pixel corresponding to the pixel point is a first value.

[0227] In some embodiments, the second determination unit 13 is specifically used to obtain the type of the pixel string to which each pixel point in the at least one pixel point belongs; determine the type of the vector string to which the pixel point belongs as the type of the pixel point, and obtain a pixel type matrix, wherein the pixel type matrix includes the type of each pixel point in the at least one pixel point; and query the type corresponding to the pixel point in the pixel type matrix according to the position information of the pixel point.

[0228] In some embodiments, if the predicted samples of the pixel point include brightness prediction samples, the second determination unit 13 is specifically used to determine the brightness prediction samples of the pixel point according to the position information of the pixel point, the reference pixel index matrix and the preset first reference pixel list if the type of the pixel point is an equal value string or a unit base vector string; if the type of the pixel point is an unmatched pixel, determine the brightness prediction samples of the pixel point according to the position information of the pixel point, the unmatched pixel index matrix and the unmatched pixel list.

[0229] In some embodiments, the second determination unit 13 is specifically used to determine the index of the reference pixel corresponding to the pixel point from the reference pixel index matrix according to the position information of the pixel point; query the value of the reference pixel corresponding to the pixel point in the first reference pixel list according to the index of the reference pixel corresponding to the pixel point; and determine the brightness value of the reference pixel corresponding to the pixel point as the brightness prediction sample value of the pixel point.

[0230] In some embodiments, the second determination unit 13 is specifically used to determine the index of the unmatched pixel corresponding to the pixel point from the unmatched pixel index matrix according to the position information of the pixel point; query the value of the unmatched pixel corresponding to the pixel point in the unmatched pixel list according to the index of the unmatched pixel corresponding to the pixel point; and determine the brightness value of the unmatched pixel corresponding to the pixel point as the brightness prediction sample value of the pixel point.

[0231] In some embodiments, if the predicted samples of the pixel point include chroma prediction samples, the second determination unit 13 is specifically used to determine the color sampling mode of the image block and the position information of the chroma prediction samples of the pixel point; determine the position information of M luminance prediction samples corresponding to the chroma prediction samples of the pixel point according to the color sampling mode of the image block and the position information of the chroma prediction samples of the pixel point, where M is a positive integer; determine the value of the reference pixel corresponding to the position information of at least one luminance prediction sample among the M luminance prediction samples according to the position information of the M luminance prediction samples and the type of the pixel point, as well as at least one of the reference pixel index matrix, the unmatched pixel index matrix and the unmatched pixel list; determine the chroma prediction sample value of the pixel point according to the value of the reference pixel corresponding to the position information of at least one luminance prediction sample among the M luminance prediction samples.

[0232] In some embodiments, the second determination unit 13 is specifically used to determine the value of the reference pixel corresponding to the position information of at least one of the M brightness prediction samples according to the position information of the M brightness prediction samples, the reference pixel index matrix and the first reference pixel list if the type of the pixel point is an equal value string or a unit basis vector string; if the type of the pixel point is an unmatched pixel, determine the value of the reference pixel corresponding to the position information of at least one of the M brightness prediction samples according to the position information of the M brightness prediction samples, the unmatched pixel index matrix and the unmatched pixel list.

[0233] In some embodiments, the second determination unit 13 is specifically used to query the index of the reference pixel corresponding to the position information of at least one brightness prediction sample among the M brightness prediction samples in the reference pixel index matrix according to the position information of the M brightness prediction samples; and determine the value of the reference pixel corresponding to the position information of at least one brightness prediction sample among the M brightness prediction samples from the first reference pixel list according to the index of the reference pixel corresponding to the position information of at least one brightness prediction sample among the M brightness prediction samples.

[0234] In some embodiments, the second determination unit 13 is specifically used to determine, according to the position information of the M brightness prediction samples, a first brightness prediction sample among the M brightness prediction samples that has the same position information as the pixel point; according to the position information of the first brightness prediction sample, determine, in the unmatched pixel index matrix or the reference pixel index matrix, the index of the reference pixel corresponding to the position information of the first brightness prediction sample; according to the index of the reference pixel corresponding to the position information of the first brightness prediction sample, determine, in the unmatched pixel list, the value of the reference pixel corresponding to the position information of the first brightness prediction sample.

[0235] In some embodiments, if the M brightness prediction samples also include a second brightness prediction sample corresponding to the position information of the pixel point, the second determination unit 13 is further used to query the index of the reference pixel corresponding to the position information of the second brightness prediction sample in the reference pixel index matrix according to the position information of the second brightness prediction sample; and determine the value of the reference pixel corresponding to the position information of the second brightness prediction sample from the first reference pixel list according to the index of the reference pixel corresponding to the position information of the second brightness prediction sample.

[0236] In some embodiments, the second determination unit 13 is specifically used to determine the value of a reference pixel corresponding to the position information of a target brightness prediction sample among the M brightness prediction samples; and determine the chrominance prediction sample value of the pixel point according to the chrominance value of the reference pixel corresponding to the position information of the target brightness prediction sample.

[0237] In some embodiments, the second determination unit 13 is specifically used to determine the chromaticity prediction sample value of the pixel point using the chromaticity value of the reference pixel of the position information of the target luminance prediction sample if the chromaticity value of the reference pixel corresponding to the position information of the target luminance prediction sample is not 0.

[0238] In some embodiments, if the chrominance value of the reference pixel corresponding to the position information of the target luminance prediction sample is 0, the second determination unit 13 is also used to determine the decoded chrominance prediction sample value adjacent to the chrominance prediction sample above the pixel point as the chrominance prediction sample value of the pixel point.

[0239] In some embodiments, if the first reference pixel list includes position information of reference pixels, the second determination unit 13 is specifically used to determine a target brightness prediction sample from the M brightness prediction samples; based on the position information of the target brightness prediction sample, if it is determined that the coordinates of the target brightness prediction sample are located in a preset position list, or it is determined that the first pixel point corresponding to the coordinates of the target brightness prediction sample belongs to an equal value string, and the first pixel point is the first pixel point in the equal value string whose coordinate value is an integer multiple of 2, then the chromaticity value of the reference pixel corresponding to the coordinates of the target brightness prediction sample is determined as the chromaticity prediction sample value of the pixel point, and the predicted position list includes the position information of the pixel points where the chromaticity prediction sample is not downsampled.

[0240] In some embodiments, if the reference pixel corresponding to the coordinates of the target brightness prediction sample is not in a preset position list, or the first pixel point corresponding to the coordinates of the target brightness prediction sample does not belong to an equal value string, or the first pixel point belongs to an equal value string and the first pixel point is not the first pixel point in the equal value string whose coordinate value is an integer multiple of 2, then the second determination unit 13 is also used to downsample the chromaticity value of the reference pixel corresponding to the position information of at least one brightness prediction sample among the M brightness prediction samples to obtain the chromaticity prediction sample value of the pixel point.

[0241] Optionally, the target brightness prediction sample is a brightness prediction sample located at the upper left corner among the M brightness prediction samples.

[0242] In some embodiments, the second determining unit 13 is specifically configured to downsample the chrominance value of the reference pixel corresponding to the position information of at least one luminance prediction sample among the M luminance prediction samples to obtain the chrominance prediction sample value of the pixel.

[0243] In some embodiments, the second determination unit 13 is specifically used to use the average value of the chromaticity values ​​of the reference pixels corresponding to the position information of at least one luminance prediction sample among the M luminance prediction samples as the chromaticity prediction sample value of the pixel point; or, to use the average value of non-zero chromaticity values ​​in the chromaticity values ​​of the reference pixels corresponding to the position information of at least one luminance prediction sample among the M luminance prediction samples as the chromaticity prediction sample value of the pixel point.

[0244] In some embodiments, if the color sampling mode is YUV4:2:0, the M is 4; if the color sampling mode is YUV4:2:2, the M is 2; if the color sampling mode is YUV4:4:4, the M is 1.

[0245] In some embodiments, the first reference pixel list includes chrominance values ​​and luminance values ​​of reference pixels.

[0246] In some embodiments, the first reference pixel list corresponds to a brightness sub-list, and each element in the brightness sub-list is used to indicate whether the corresponding reference pixel in the first reference pixel list has a brightness value that is not zero and a chrominance value that is zero.

[0247] In some embodiments, the unmatched pixel list corresponds to a brightness sub-list of unmatched pixels, and each element in the brightness sub-list of unmatched pixels is used to indicate whether the corresponding unmatched pixel in the unmatched pixel list has a non-zero brightness value and a zero chrominance value.

[0248] It should be understood that the device embodiment and the method embodiment may correspond to each other, and similar descriptions may refer to the method embodiment. To avoid repetition, no further description is given here. Specifically, the device shown in FIG11 can execute the method embodiment corresponding to the above-mentioned decoding end, and the aforementioned and other operations and / or functions of each module in the device are respectively for implementing the corresponding processes of the method embodiment corresponding to the above-mentioned decoding end, and for the sake of brevity, no further description is given here.

[0249] The above describes the device of the embodiment of the present application from the perspective of the functional module in conjunction with the accompanying drawings. It should be understood that the functional module can be implemented in hardware form, can be implemented by instructions in software form, and can also be implemented by a combination of hardware and software modules. Specifically, the steps of the method embodiment in the embodiment of the present application can be completed by the hardware integrated logic circuit and / or software form instructions in the processor, and the steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to perform, or a combination of hardware and software modules in the decoding processor to perform. Optionally, the software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory, and completes the steps in the above method embodiment in conjunction with its hardware.

[0250] Fig.13 is a schematic block diagram of an electronic device 30 provided in an embodiment of the present application. The electronic device 30 may be a decoding device or an encoding device.

[0251] like Fig.13 As shown, the electronic device 30 may be the video decoder described in the embodiment of the present application, and the electronic device 30 may include:

[0252] The memory 33 and the processor 32, the memory 33 is used to store the computer program 34 and transmit the program code 34 to the processor 32. In other words, the processor 32 can call and run the computer program 34 from the memory 33 to implement the method in the embodiment of the present application.

[0253] For example, the processor 32 may be configured to execute the steps in the above method 200 according to the instructions in the computer program 34 .

[0254] In some embodiments of the present application, the processor 32 may include but is not limited to:

[0255] General-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc.

[0256] In some embodiments of the present application, the memory 33 includes but is not limited to:

[0257] Volatile memory and / or non-volatile memory. Among them, the non-volatile memory can be read-only memory (ROM), programmable ROM (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM) or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM) and direct memory bus random access memory (Direct Rambus RAM, DR RAM).

[0258] In some embodiments of the present application, the computer program 34 may be divided into one or more units, which are stored in the memory 33 and executed by the processor 32 to complete the method provided by the present application. The one or more units may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program 34 in the electronic device 30.

[0259] like Fig.13 As shown, the electronic device 30 may further include:

[0260] The transceiver 33 may be connected to the processor 32 or the memory 33 .

[0261] The processor 32 may control the transceiver 33 to communicate with other devices, specifically, to send information or data to other devices, or to receive information or data sent by other devices. The transceiver 33 may include a transmitter and a receiver. The transceiver 33 may further include an antenna, and the number of antennas may be one or more.

[0262] It should be understood that the various components in the electronic device 30 are connected via a bus system, wherein the bus system includes not only a data bus but also a power bus, a control bus and a status signal bus.

[0263] The present application also provides a computer storage medium on which a computer program is stored, and when the computer program is executed by a computer, the computer can perform the method of the above method embodiment. In other words, the present application embodiment also provides a computer program product containing instructions, and when the instructions are executed by a computer, the computer can perform the method of the above method embodiment.

[0264] When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integration. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a digital video disc (digital video disc, DVD)), or a semiconductor medium (e.g., a solid state drive (solid state disk, SSD)), etc.

[0265] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0266] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the unit is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0267] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. For example, each functional unit in each embodiment of the present application may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0268] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for determining image block prediction samples, characterized in that: include: Obtaining an index of a reference pixel corresponding to each pixel in at least one pixel in the image block; Determine at least one of a reference pixel index matrix, an unmatched pixel index matrix, and an unmatched pixel list according to an index of a reference pixel corresponding to each pixel in the at least one pixel; For each pixel of the at least one pixel, determine the type of the pixel, and determine the prediction sample of the pixel according to the type of the pixel, the reference pixel index matrix, the unmatched pixel index matrix and the unmatched pixel list, wherein the type of the pixel includes any one of the following: an equivalent value string, an unmatched pixel, and a unit base vector string; Determining the prediction samples of the image block according to the prediction samples of the pixel points; Among them, the reference pixel index matrix includes the index of the reference pixel corresponding to each pixel in the at least one pixel point, the unmatched pixel list includes the correspondence between the value of the unmatched pixel in the at least one pixel point and the index of the unmatched pixel, and the unmatched pixel index matrix includes the index of the unmatched pixel in the at least one pixel point in the unmatched pixel list.

2. The method according to claim 1, characterized in that The determining the type of the pixel point includes: According to the position information of the pixel point, query the reference pixel index matrix for the index of the reference pixel corresponding to the pixel point; The type of the pixel point is determined according to the index of the reference pixel corresponding to the pixel point.

3. The method according to claim 2, characterized in that The determining the type of the pixel point according to the index of the reference pixel corresponding to the pixel point includes: If the value of the index of the reference pixel corresponding to the pixel point is a first value, it is determined that the type of the pixel point is an unmatched pixel.

4. The method according to claim 1, characterized in that: The method further comprises: Obtaining a type of a pixel string to which each pixel in the at least one pixel belongs; Determine the type of the pixel string to which the pixel point belongs as the type of the pixel point, and obtain a pixel type matrix, wherein the pixel type matrix includes the type of each pixel point in the at least one pixel point; The determining the type of the pixel point includes: According to the position information of the pixel point, the type corresponding to the pixel point is queried in the pixel type matrix.

5. The method according to claim 1, characterized in that: If the predicted sample of the pixel point includes a brightness predicted sample, determining the predicted sample of the pixel point according to the type of the pixel point and at least one of the reference pixel index matrix, the unmatched pixel index matrix, and the unmatched pixel list includes: If the type of the pixel point is an equal value string or a unit base vector string, determining a brightness prediction sample of the pixel point according to the position information of the pixel point, the reference pixel index matrix and the first reference pixel list; If the type of the pixel is an unmatched pixel, a brightness prediction sample of the pixel is determined according to the position information of the pixel, the unmatched pixel index matrix and the unmatched pixel list.

6. The method according to claim 5, characterized in that The step of determining the brightness prediction sample of the pixel point according to the position information of the pixel point, the reference pixel index matrix and the first reference pixel list includes: Determining, according to the position information of the pixel point, the index of the reference pixel corresponding to the pixel point from the reference pixel index matrix; According to the index of the reference pixel corresponding to the pixel point, query the first reference pixel list for a value of the reference pixel corresponding to the pixel point; The brightness value of the reference pixel corresponding to the pixel point is determined as the brightness prediction sample value of the pixel point.

7. The method according to claim 5, characterized in that The step of determining the brightness prediction sample of the pixel point according to the position information of the pixel point, the unmatched pixel index matrix and the unmatched pixel list comprises: Determining, according to the position information of the pixel point, the index of the unmatched pixel corresponding to the pixel point from the unmatched pixel index matrix; According to the index of the unmatched pixel corresponding to the pixel point, query the value of the unmatched pixel corresponding to the pixel point in the unmatched pixel list; The brightness value of the unmatched pixel corresponding to the pixel point is determined as the brightness prediction sample value of the pixel point.

8. The method according to claim 1, characterized in that If the predicted sample of the pixel includes a chroma predicted sample, determining the predicted sample of the pixel according to the type of the pixel and at least one of the reference pixel index matrix, the unmatched pixel index matrix, and the unmatched pixel list includes: Determining a color sampling mode of the image block and position information of a chroma prediction sample of the pixel point; Determine, according to the color sampling mode of the image block and the position information of the chrominance prediction sample of the pixel, the position information of M luminance prediction samples corresponding to the chrominance prediction sample of the pixel, where M is a positive integer; Determine, according to the position information of the M brightness prediction samples and the type of the pixel, and at least one of the reference pixel index matrix, the unmatched pixel index matrix, and the unmatched pixel list, a value of a reference pixel corresponding to the position information of at least one brightness prediction sample in the M brightness prediction samples; Determine the chrominance prediction sample value of the pixel point according to the value of the reference pixel corresponding to the position information of at least one luminance prediction sample among the M luminance prediction samples.

9. The method according to claim 8, characterized in that The determining, according to the position information of the M brightness prediction samples and the type of the pixel point, and at least one of the reference pixel index matrix, the unmatched pixel index matrix, and the unmatched pixel list, a value of a reference pixel corresponding to the position information of at least one brightness prediction sample in the M brightness prediction samples, comprises: If the type of the pixel point is an equal value string or a unit basis vector string, determining a value of a reference pixel corresponding to the position information of at least one brightness prediction sample among the M brightness prediction samples according to the position information of the M brightness prediction samples, the reference pixel index matrix and the first reference pixel list; If the type of the pixel point is an unmatched pixel, the value of the reference pixel corresponding to the position information of at least one brightness prediction sample among the M brightness prediction samples is determined according to the position information of the M brightness prediction samples, the unmatched pixel index matrix and the unmatched pixel list.

10. The method according to claim 9, characterized in that The determining, according to the position information of the M brightness prediction samples, the reference pixel index matrix and the first reference pixel list, a value of a reference pixel corresponding to the position information of at least one brightness prediction sample among the M brightness prediction samples comprises: According to the position information of the M brightness prediction samples, querying the reference pixel index matrix for an index of a reference pixel corresponding to the position information of at least one brightness prediction sample among the M brightness prediction samples; According to the index of the reference pixel corresponding to the position information of at least one brightness prediction sample among the M brightness prediction samples, the value of the reference pixel corresponding to the position information of at least one brightness prediction sample among the M brightness prediction samples is determined from the first reference pixel list.

11. The method according to claim 9, characterized in that The determining, according to the position information of the M brightness prediction samples, the unmatched pixel index matrix and the unmatched pixel list, a value of a reference pixel corresponding to the position information of at least one brightness prediction sample in the M brightness prediction samples comprises: Determine, according to the position information of the M brightness prediction samples, a first brightness prediction sample among the M brightness prediction samples having the same position information as the pixel point; Determine, according to the position information of the first brightness prediction sample, in the unmatched pixel index matrix or the reference pixel index matrix, the index of the reference pixel corresponding to the position information of the first brightness prediction sample; According to the index of the reference pixel corresponding to the position information of the first brightness prediction sample, the value of the reference pixel corresponding to the position information of the first brightness prediction sample is determined in the unmatched pixel list.

12. The method according to claim 11, characterized in that If the M brightness prediction samples also include a second brightness prediction sample having position information corresponding to the pixel point, the method further includes: According to the position information of the second brightness prediction sample, querying the reference pixel index matrix for the index of the reference pixel corresponding to the position information of the second brightness prediction sample; According to the index of the reference pixel corresponding to the position information of the second brightness prediction sample, the value of the reference pixel corresponding to the position information of the second brightness prediction sample is determined from the first reference pixel list.

13. The method according to any one of claims 8 to 12, characterized in that: The determining the chrominance prediction sample value of the pixel point according to the value of the reference pixel corresponding to the position information of at least one luminance prediction sample among the M luminance prediction samples comprises: Determine a value of a reference pixel corresponding to position information of a target brightness prediction sample among the M brightness prediction samples; The chrominance prediction sample value of the pixel point is determined according to the chrominance value of the reference pixel corresponding to the position information of the target luminance prediction sample.

14. The method according to claim 13, characterized in that The step of determining the chrominance prediction sample value of the pixel point according to the chrominance value of the reference pixel corresponding to the position information of the target brightness prediction sample comprises: If the chromaticity value of the reference pixel corresponding to the position information of the target brightness prediction sample is not 0, the chromaticity value of the reference pixel corresponding to the position information of the target brightness prediction sample is used to determine the chromaticity prediction sample value of the pixel point.

15. The method according to claim 14, characterized in that If the chrominance value of the reference pixel corresponding to the position information of the target brightness prediction sample is 0, the method further includes: The decoded chroma prediction sample value adjacent to the chroma prediction sample above the pixel is determined as the chroma prediction sample value of the pixel.

16. The method according to any one of claims 8 to 12, characterized in that: The determining the chrominance prediction sample value of the pixel point according to the value of the reference pixel corresponding to the position information of at least one luminance prediction sample among the M luminance prediction samples comprises: Determine a target brightness prediction sample from the M brightness prediction samples; According to the position information of the target brightness prediction sample, if it is determined that the coordinates of the target brightness prediction sample are located in a preset position list, or it is determined that the first pixel point corresponding to the coordinates of the target brightness prediction sample belongs to an equal value string, and the first pixel point is the first pixel point in the equal value string whose coordinate value is an integer multiple of 2, then the chromaticity value of the reference pixel corresponding to the coordinates of the target brightness prediction sample is determined as the chromaticity prediction sample value of the pixel point, and the predicted position list includes the position information of the pixel points where the chromaticity prediction sample is not downsampled.

17. The method according to claim 16, characterized in that If the reference pixel corresponding to the coordinate of the target brightness prediction sample is not in the preset position list, or the first pixel point corresponding to the coordinate of the target brightness prediction sample does not belong to the equal value string, or the first pixel point belongs to the equal value string and the first pixel point is not the first pixel point in the equal value string whose coordinate value is an integer multiple of 2, the method further includes: Down-sample the chrominance value of the reference pixel corresponding to the position information of at least one luminance prediction sample among the M luminance prediction samples to obtain the chrominance prediction sample value of the pixel point.

18. The method according to claim 15 or 17, characterized in that The target brightness prediction sample is a brightness prediction sample located at the upper left corner among the M brightness prediction samples.

19. The method according to claim 8, characterized in that The determining the chrominance prediction sample value of the pixel point according to the value of the reference pixel corresponding to the position information of at least one luminance prediction sample among the M luminance prediction samples comprises: Down-sample the chrominance value of the reference pixel corresponding to the position information of at least one luminance prediction sample among the M luminance prediction samples to obtain the chrominance prediction sample value of the pixel point.

20. The method according to claim 17 or 19, characterized in that The step of downsampling the chrominance value of a reference pixel corresponding to the position information of at least one luminance prediction sample among the M luminance prediction samples to obtain the chrominance prediction sample value of the pixel point comprises: Using the average value of the chrominance values ​​of the reference pixels corresponding to the position information of at least one luminance prediction sample among the M luminance prediction samples as the chrominance prediction sample value of the pixel point; or An average value of non-zero chrominance values ​​in chrominance values ​​of reference pixels corresponding to position information of at least one luminance prediction sample among the M luminance prediction samples is used as the chrominance prediction sample value of the pixel point.

21. The method according to any one of claims 8 to 12, characterized in that: If the color sampling mode is YUV4:2:0, the M is 4; if the color sampling mode is YUV4:2:2, the M is 2; if the color sampling mode is YUV4:4:4, the M is 1.

22. The method according to claim 5, characterized in that The first reference pixel list includes chrominance values ​​and luminance values ​​of reference pixels; or, the first reference pixel list includes position information of reference pixels.

23. The method according to claim 5, characterized in that The first reference pixel list corresponds to a brightness sub-list, and each element in the brightness sub-list is used to indicate whether the brightness value of the corresponding reference pixel in the first reference pixel list is not zero and the chrominance value is zero.

24. The method according to any one of claims 1 to 12, characterized in that: The unmatched pixel list corresponds to a brightness sub-list of unmatched pixels, and each element in the brightness sub-list of unmatched pixels is used to indicate whether the brightness value of the corresponding unmatched pixel in the unmatched pixel list is not zero and the chrominance value is zero.

25. The method according to claim 16, characterized in that The preset position list includes at least one of the following: a first reference pixel list and a second reference pixel list, where the second reference pixel list is an updated list of the first reference pixel list.

26. A device for determining image block prediction samples, characterized in that: include: An acquisition unit, configured to obtain an index of a reference pixel corresponding to each pixel in at least one pixel in the image block; A first determining unit, configured to determine at least one of a reference pixel index matrix, an unmatched pixel index matrix, and an unmatched pixel list according to an index of a reference pixel corresponding to each pixel of the at least one pixel; a second determining unit, configured to determine, for each pixel of the at least one pixel, a type of the pixel, and determine a prediction sample of the pixel according to the type of the pixel, the reference pixel index matrix, an unmatched pixel index matrix, and an unmatched pixel list, wherein the type of the pixel includes any one of the following: an equivalent value string, an unmatched pixel, and a unit base vector string; A third determining unit, configured to determine a prediction sample of the image block according to the prediction sample of the pixel point; Among them, the reference pixel index matrix includes the index of the reference pixel corresponding to each pixel in the at least one pixel point, the unmatched pixel list includes the correspondence between the value of the unmatched pixel in the at least one pixel point and the index of the unmatched pixel, and the unmatched pixel index matrix includes the index of the unmatched pixel in the at least one pixel point in the unmatched pixel list.

27. A decoding device, characterized in that: including a processor and a memory; The memory is used to store computer programs; The processor is used to call and run the computer program stored in the memory to execute the method according to any one of claims 1 to 25.

28. A coding device, characterized in that including a processor and a memory; The memory is used to store computer programs; The processor is used to call and run the computer program stored in the memory to execute the method according to any one of claims 1 to 25.

29. A computer-readable storage medium, characterized in that: Used to store a computer program, the computer program causing a computer to execute the method according to any one of claims 1 to 25.

Images