Decoding method, encoding method, decoder, and encoder for unmatched pixels

Abstract

The application provides a decoding method and an encoder of unmatched pixels, and relates to the technical field of video or image processing. The decoding method of unmatched pixels comprises the following steps: decoding code streams of a target sequence through at least two entropy decoding modes to obtain a binary symbol string of unmatched pixels in a target image block; performing inverse binarization on the binary symbol string to be inverse binarized to obtain a component value of the unmatched pixels; and obtaining the target image block based on the component value of the unmatched pixels. The method can improve the flexibility of coding and is beneficial to balancing the coding performance and the coding overhead of unmatched pixels.

Description

Technical Field

[0001] This application relates to the field of video or image processing technology, and more specifically, to methods for decoding, encoding, decoding, and encoders of mismatched pixels. Background Technology

[0002] Digital video compression technology primarily compresses massive amounts of digital video data for easier transmission and storage. With the surge in internet video and increasing demands for video clarity, while existing digital video compression standards can decompress video, there is still a need to pursue better digital video compression technologies to improve compression performance. Entropy coding is an important video compression technique. Commonly used entropy coding methods include Content Adaptive Binary Arithmetic Coding (CABAC) and Content Adaptive Variable Length Coding (CAVLC). The CABAC process mainly involves three steps: binaryization, context modeling, and binary arithmetic coding.

[0003] After binarizing the input syntax elements, the binary data can be encoded using either a regular encoding mode or a bypass encoding mode. The bypass encoding mode does not require assigning a specific probability model to each binary bit; the input binary is directly encoded using a simple bypass encoder to speed up the entire encoding and decoding process.

[0004] Generally, different syntactic elements are not completely independent, and even identical syntactic elements possess a certain degree of memory. Therefore, according to conditional entropy theory, using other encoded syntactic elements for conditional encoding can further improve encoding performance compared to independent encoding or memoryless encoding. This encoded symbolic information used as conditions is called context. In conventional encoding, the binary bits (bins) of syntactic elements sequentially enter the context modeler. The encoder assigns an appropriate probability model to each input binary bit based on the values ​​of previously encoded syntactic elements or binary bits; this process is called context modeling. The context model corresponding to a syntactic element can be located using `ctxIdxInc` (context index increment) and `ctxIdxStart` (context index start). After the bins and assigned probability models are fed into the binary arithmetic encoder for encoding, the context model needs to be updated based on the bin values; this is the adaptive process in encoding.

[0005] However, in the AVS3 standard, for serially copied intra-predictive mode coded blocks, the Y, Cb, and Cr components of unmatched pixels are directly encoded instead of being derived from predicted values. Each component of an unmatched pixel is encoded according to its bit depth. For example, if the current image has a bit depth of 10 bits, encoding the Y, Cb, and Cr components of an unmatched pixel requires 3*10 bits of sign bit encoding. That is, each sign bit is encoded in bypass mode, resulting in excessive encoding overhead for unmatched pixels. Summary of the Invention

[0006] This application provides a decoding method, encoding method, decoder, and encoder for mismatched pixels, which can improve the flexibility of encoding and help balance the encoding performance and encoding overhead of mismatched pixels.

[0007] On the one hand, this application provides a method for decoding unmatched pixels, including:

[0008] The target sequence's bitstream is decoded using at least two entropy decoding methods to obtain a binary symbol string of unmatched pixels in the target image block;

[0009] By debinarizing the binary symbol string to be debinarized, the component value of the unmatched pixel is obtained;

[0010] The target image patch is obtained based on the component values ​​of the unmatched pixel.

[0011] On the other hand, this application provides a method for encoding unmatched pixels, including:

[0012] The target image frame in the target sequence is divided into multiple image blocks, and these multiple image blocks include the target image block;

[0013] Adjust the component values ​​of unmatched pixels in the target image patch;

[0014] By binarizing the component values ​​of the unmatched pixel, a binary symbol string of the unmatched pixel is obtained;

[0015] The target sequence's bitstream is obtained by encoding the binary symbol string of the unmatched pixel using at least two entropy coding methods.

[0016] On the other hand, embodiments of this application provide a decoder for executing the methods in the first aspect or its various implementations described above. Specifically, the decoder includes functional units for executing the methods in the first aspect or its various implementations described above.

[0017] On the other hand, embodiments of this application provide an encoder for executing the methods in the second aspect or its various implementations described above. Specifically, the encoder includes functional units for executing the methods in the second aspect or its various implementations described above.

[0018] On the other hand, embodiments of this application provide an encoding / decoding device, including:

[0019] Processor, adapted to implement computer instructions; and,

[0020] A computer-readable storage medium storing computer instructions adapted to be loaded and executed by a processor to perform the methods as described in any of the first to second aspects above or their respective implementations.

[0021] On the other hand, embodiments of this application provide a computer-readable storage medium storing computer instructions that, when read and executed by a processor of a computer device, cause the computer device to perform any of the first to second aspects or their respective implementations described above.

[0022] On the other hand, embodiments of this application provide a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods of any one of the first to second aspects or their respective implementations described above.

[0023] In this embodiment of the application, the target sequence bitstream is decoded by at least two entropy decoding methods, that is, the target sequence bitstream is decoded by multiple entropy decoding methods, which can improve the flexibility of encoding and coding and is conducive to balancing the encoding and coding performance and the encoding overhead of mismatched pixels. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a schematic block diagram of a video encoding and decoding system provided in an embodiment of this application.

[0026] Figure 2 This is a schematic block diagram of a video encoder provided in an embodiment of this application.

[0027] Figure 3 This is a schematic block diagram of the decoding framework provided in the embodiments of this application.

[0028] Figure 4 This is a schematic diagram of an intra-frame string copying method provided in an embodiment of this application.

[0029] Figure 5 This is a schematic flowchart of the encoding method for unmatched pixels provided in the embodiments of this application.

[0030] Figure 6 This is a schematic flowchart of the decoding method for unmatched pixels provided in the embodiments of this application.

[0031] Figure 7 This is a schematic block diagram of the encoder provided in the embodiments of this application.

[0032] Figure 8 This is a schematic block diagram of the decoder provided in the embodiments of this application.

[0033] Figure 9 This is a schematic block diagram of the encoding / decoding device provided in the embodiments of this application. Detailed Implementation

[0034] The technical solutions of the embodiments of this application will now be described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0035] The technology in this application is not limited to any encoding or decoding standard or technology.

[0036] The solutions provided in this application can be applied to the field of digital video coding technology, such as image encoding and decoding, video encoding and decoding, hardware video encoding and decoding, dedicated circuit video encoding and decoding, and real-time video encoding and decoding. Alternatively, the solutions provided in this application can be combined with audio video coding standards (AVS), second-generation AVS standards (AVS2), or third-generation AVS standards (AVS3). Specifically, this includes, but is not limited to, the H.264 / Audio Video Coding (AVC) standard, the H.265 / High Efficiency Video Coding (HEVC) standard, and the H.266 / Versatile Video Coding (VVC) standard. Alternatively, the solutions provided in this application can be incorporated into other proprietary or industry standards. For example, they may specifically include ITU-TH.261, ISO / IEC MPEG-1 Visual, ITU-TH.262 or ISO / IEC MPEG-2 Visual, ITU-TH.263, ISO / IEC MPEG-4 Visual, ITU-TH.264 (also known as ISO / IEC MPEG-4 AVC), and also include Scalable Video Codec (SVC) and Multi-View Video Codec (MVC) extensions.

[0037] Furthermore, the solution provided in this application can be used for lossy compression of images, or for lossless compression of images. This lossless compression can be visually lossless compression or mathematically lossless compression.

[0038] To facilitate understanding, let's first combine... Figure 1 The video encoding and decoding system involved in the embodiments of this application will be described.

[0039] Figure 1 This is a schematic block diagram of a video encoding / decoding system 100 according to an embodiment of this application. It should be noted that... Figure 1 This is merely an example; the video encoding and decoding system in this application includes, but is not limited to, [other systems]. Figure 1 As shown. Figure 1As shown, the video encoding / decoding system 100 includes an encoding device 110 and a decoding device 120. The encoding device encodes (can be understood as compressing) video data to generate a bitstream and transmits the bitstream to the decoding device. The decoding device decodes the bitstream generated by the encoding device to obtain the decoded video data.

[0040] Encoding device 110 can be understood as a device with video encoding capabilities, and decoding device 120 can be understood as a device with video decoding capabilities. That is, in this embodiment, encoding device 110 and decoding device 120 include a wider range of devices, such as smartphones, desktop computers, mobile computing devices, laptops (e.g., tablet computers), tablet computers, set-top boxes, televisions, cameras, display devices, digital media players, video game consoles, in-vehicle computers, etc. In some embodiments, encoding device 110 can transmit encoded video data (such as a bitstream) to decoding device 120 via channel 130. Channel 130 may include one or more media and / or devices capable of transmitting encoded video data from encoding device 110 to decoding device 120.

[0041] In one example, channel 130 includes one or more communication media that enable encoding device 110 to transmit encoded video data directly to decoding device 120 in real time. In this example, encoding device 110 can modulate the encoded video data according to a communication standard and transmit the modulated video data to decoding device 120. The communication media includes wireless communication media, such as radio frequency spectrum; optionally, the communication media may also include wired communication media, such as one or more physical transmission lines. In another example, channel 130 includes a storage medium that can store the video data encoded by encoding device 110. The storage medium includes various local access data storage media, such as optical discs, DVDs, flash memory, etc. In this example, decoding device 120 can retrieve the encoded video data from the storage medium. In another example, channel 130 may include a storage server that can store the video data encoded by encoding device 110. In this example, decoding device 120 can download the stored encoded video data from the storage server. Optionally, the storage server can store the encoded video data and transmit the encoded video data to the decoding device 120, such as a web server (e.g., for a website), a file transfer protocol (FTP) server, etc.

[0042] In some embodiments, the encoding device 110 includes a video encoder 112 and an output interface 113. The output interface 113 may include a modulator / demodulator (modem) and / or a transmitter.

[0043] In some embodiments, in addition to the video encoder 112 and the input interface 113, the encoding device 110 may also include a video source 111. The video source 111 may include at least one of a video acquisition device (e.g., a video camera), a video archive, a video input interface, and a computer graphics system, wherein the video input interface is used to receive video data from a video content provider, and the computer graphics system is used to generate the video data.

[0044] Video encoder 112 encodes video data from video source 111 to generate a bitstream. The video data may include one or more pictures or a sequence of pictures. The bitstream contains the encoding information of the pictures or picture sequences in the form of a bitstream. The encoding information may include encoded image data and associated data. The associated data may include a sequence parameter set (SPS), a picture parameter set (PPS), and other syntax structures. The SPS may contain parameters applied to one or more sequences. The PPS may contain parameters applied to one or more pictures. A syntax structure refers to a set of zero or more syntax elements arranged in a specified order within the bitstream.

[0045] The video encoder 112 transmits the encoded video data directly to the decoding device 120 via the output interface 113. The encoded video data can also be stored on a storage medium or a storage server for subsequent retrieval by the decoding device 120.

[0046] In some embodiments, the decoding device 120 includes an input interface 121 and a video decoder 122. In some embodiments, in addition to the input interface 121 and the video decoder 122, the decoding device 120 may also include a display device 123. The input interface 121 includes a receiver and / or a modem. The input interface 121 can receive encoded video data through channel 130. The video decoder 122 is used to decode the encoded video data to obtain decoded video data, and transmit the decoded video data to the display device 123.

[0047] Display device 123 displays the decoded video data. Display device 123 may be integrated with decoding device 120 or external to decoding device 120. Display device 123 may include various display devices, such as liquid crystal display (LCD), plasma display, organic light-emitting diode (OLED) display, or other types of display devices.

[0048] It should be noted that, Figure 1 This is merely one example of the present application, and the technical solutions of the embodiments of the present application are not limited to this. Figure 1For example, the technology of this application can also be applied to one-sided video encoding or one-sided video decoding.

[0049] The video coding framework involved in the embodiments of this application is described below.

[0050] Figure 2 This is a schematic block diagram of the video encoder 200 provided in an embodiment of this application. It should be understood that the video encoder 200 can be used for lossy compression of images or lossless compression of images. The lossless compression can be visually lossless compression or mathematically lossless compression.

[0051] The video encoder 200 can be applied to image data in luminance-chrominance (YCbCr, YUV) format.

[0052] For example, the video encoder 200 reads video data and divides each frame into several coding tree units (CTUs). In some examples, CTUs may be called "tree blocks," "largest coding unit" (LCU), or "coding treeblock" (CTB). Each CTU can be associated with a pixel block of equal size within the image. Each pixel can correspond to one luminance (luma) sample and two chrominance (chroma) samples. Therefore, each CTU can be associated with one luminance sample block and two chrominance sample blocks. The size of a CTU is, for example, 128×128, 64×64, 32×32, etc. A CTU can be further divided into several coding units (CUs) for encoding. CUs can be rectangular or square blocks. The CU can be further divided into prediction units (PU) and transform units (TU), thus separating encoding, prediction, and transformation for more flexible processing. In one example, the CTU is divided into CUs using a quadtree structure, and the CUs are further divided into TUs and PUs using a quadtree structure.

[0053] The video encoder 200 supports various PU sizes. Assuming a specific CU size of 2N×2N, the video encoder and decoder can support PU sizes of 2N×2N or N×N for intra-frame prediction, and support symmetric PUs of 2N×2N, 2N×N, N×2N, N×N, or similar sizes for inter-frame prediction. The video encoder and decoder can also support asymmetric PUs of 2N×nU, 2N×nD, nL×2N, and nR×2N for inter-frame prediction.

[0054] like Figure 2 As shown, the video encoder 200 may include:

[0055] The video encoder 200 includes a prediction unit 210, a residual unit 220, a transform / quantization unit 230, an inverse transform / quantization unit 240, a reconstruction unit 250, a loop filtering unit 260, a decoded image buffer 270, and an entropy coding unit 280. It should be noted that the video encoder 200 may contain more, fewer, or different functional components.

[0056] The prediction unit 210 includes an inter-frame prediction unit 211 and an intra-frame prediction unit 212. Because there is strong correlation between adjacent pixels in a video frame, intra-frame prediction is used in video encoding and decoding techniques to eliminate spatial redundancy between adjacent pixels. Because there is strong similarity between adjacent frames in a video, inter-frame prediction is used in video encoding and decoding techniques to eliminate temporal redundancy between adjacent frames, thereby improving coding efficiency.

[0057] Inter-frame prediction unit 211 can be used for inter-frame prediction. Inter-frame prediction can refer to image information from different frames. Inter-frame prediction uses motion information to find reference blocks from reference frames and generates prediction blocks based on the reference blocks to eliminate temporal redundancy. The frames used for inter-frame prediction can be P-frames and / or B-frames. P-frames refer to forward prediction frames, and B-frames refer to bidirectional prediction frames. Motion information includes a list of reference frames, the reference frame index, and motion vectors. Motion vectors can be integer-pixel or fractional-pixel. If the motion vector is fractional-pixel, interpolation filtering needs to be used in the reference frame to create the required fractional-pixel blocks. Here, the integer-pixel or fractional-pixel blocks in the reference frame found based on the motion vectors are called reference blocks. Some techniques directly use the reference blocks as prediction blocks, while others process the reference blocks to generate prediction blocks. Processing the reference blocks to generate prediction blocks can also be understood as using the reference blocks as prediction blocks and then processing them to generate new prediction blocks. The most commonly used inter-frame prediction methods include geometric partitioning mode (GPM) in the VVC video codec standard and angular weighted prediction (AWP) in the AVS3 video codec standard. These two intra-frame prediction modes share common principles.

[0058] The intra-prediction unit 212 refers only to information from the same frame image to predict pixel information within the current code image block, thereby eliminating spatial redundancy. The frame used for intra-prediction can be an I-frame. HEVC uses Planar, DC, and 33 angle modes for a total of 35 prediction modes. VVC uses Planar, DC, and 65 angle modes for a total of 67 prediction modes. AVS3 uses DC, Plane, Bilinear, and 63 angle modes for a total of 66 prediction modes. In some embodiments, the intra-prediction unit 212 can be implemented using intra-block copying and intra-string copying techniques.

[0059] The residual unit 220 can generate a residual block of the CU based on the pixel block of the CU and the prediction block of the PU of the CU. For example, the residual unit 220 can generate a residual block of the CU such that each sample in the residual block has a value equal to the difference between the sample in the pixel block of the CU and the corresponding sample in the prediction block of the PU of the CU.

[0060] Transform / quantization unit 230 quantizes the transform coefficients. Transform / quantization unit 230 quantizes the transform coefficients associated with the TU of the CU based on the quantization parameter (QP) value associated with the CU. Video encoder 200 can adjust the degree of quantization applied to the transform coefficients associated with the CU by adjusting the QP value associated with the CU. Inverse transform / quantization unit 240 applies inverse quantization and inverse transform to the quantized transform coefficients to reconstruct the residual block from the quantized transform coefficients.

[0061] The reconstruction unit 250 can add samples of the reconstructed residual block to corresponding samples of one or more prediction blocks generated by the prediction unit 210 to produce a reconstructed image block associated with the TU. By reconstructing the sampled blocks of each TU of the CU in this way, the video encoder 200 can reconstruct the pixel blocks of the CU.

[0062] The loop filtering unit 260 can perform deblocking filtering operations to reduce the block effect of pixel blocks associated with the CU. In some embodiments, the loop filtering unit 260 includes a deblocking filtering unit and a sample adaptive compensation / adaptive loop filtering (SAO / ALF) unit, wherein the deblocking filtering unit is used to remove the block effect and the SAO / ALF unit is used to remove the ringing effect.

[0063] The decoded image buffer 270 can store reconstructed pixel blocks. The inter-frame prediction unit 211 can use a reference image containing the reconstructed pixel blocks to perform inter-frame prediction on PUs of other images. In addition, the intra-frame prediction unit 212 can use the reconstructed pixel blocks in the decoded image buffer 270 to perform intra-frame prediction on other PUs in the same image as the CU.

[0064] Entropy coding unit 280 can receive quantized transform coefficients from transform / quantization unit 230. Entropy coding unit 280 can perform one or more entropy coding operations on the quantized transform coefficients to produce entropy-coded data.

[0065] Figure 3 This is a schematic block diagram of the decoding framework 300 provided in the embodiments of this application.

[0066] like Figure 3 As shown, the video decoder 300 includes:

[0067] The video decoder 300 includes an entropy decoding unit 310, a prediction unit 320, an inverse quantization / transformation unit 330, a reconstruction unit 340, a loop filtering unit 350, and a decoded image buffer 360. It should be noted that the video decoder 300 may contain more, fewer, or different functional components.

[0068] The video decoder 300 can receive a bitstream. The entropy decoding unit 310 can parse the bitstream to extract syntax elements from it. As part of parsing the bitstream, the entropy decoding unit 310 can parse the entropy-encoded syntax elements in the bitstream. The prediction unit 320, the dequantization / transform unit 330, the reconstruction unit 340, and the loop filtering unit 350 can decode the video data based on the syntax elements extracted from the bitstream, i.e., generate decoded video data.

[0069] The prediction unit 320 includes an intra-frame prediction unit 321 and an inter-frame prediction unit 322.

[0070] Intra-prediction unit 321 can perform intra-prediction to generate prediction blocks for the PU. Intra-prediction unit 321 can use an intra-prediction mode to generate prediction blocks for the PU based on pixel blocks of spatially adjacent PUs. Intra-prediction unit 321 can also determine the intra-prediction mode of the PU based on one or more syntax elements parsed from the bitstream.

[0071] Inter-frame prediction unit 322 can construct a first reference image list (list 0) and a second reference image list (list 1) based on the syntax elements parsed from the bitstream. Furthermore, if the PU uses inter-frame prediction coding, the entropy decoding unit 310 can parse the motion information of the PU. Inter-frame prediction unit 322 can determine one or more reference blocks of the PU based on the motion information of the PU. Inter-frame prediction unit 322 can generate prediction blocks for the PU based on one or more reference blocks of the PU.

[0072] The dequantization / transform unit 330 reversibly quantizes (i.e., dequantizes) the transform coefficients associated with the TU. The dequantization / transform unit 330 can use the QP value associated with the CU of the TU to determine the degree of quantization. After dequantizing the transform coefficients, the dequantization / transform unit 330 can apply one or more inverse transforms to the dequantized transform coefficients to produce a residual block associated with the TU.

[0073] The reconstruction unit 340 uses the residual block associated with the TU of the CU and the prediction block of the PU of the CU to reconstruct the pixel block of the CU. For example, the reconstruction unit 340 can add the sample of the residual block to the corresponding sample of the prediction block to reconstruct the pixel block of the CU, thereby obtaining the reconstructed image block.

[0074] The loop filter unit 350 can perform deblocking filtering operations to reduce the block effect of pixel blocks associated with the CU.

[0075] The video decoder 300 can store the reconstructed image of the CU in the decoded image buffer 360. The video decoder 300 can use the reconstructed image in the decoded image buffer 360 as a reference image for subsequent prediction, or transmit the reconstructed image to a display device for presentation.

[0076] The basic process of video encoding and decoding is as follows:

[0077] At the encoding end, a frame of image is divided into blocks. For the current block, prediction unit 210 uses intra-frame prediction or inter-frame prediction to generate a prediction block for the current block. Residual unit 220 can calculate a residual block based on the prediction block and the original block of the current block, i.e., the difference between the prediction block and the original block of the current block. This residual block can also be called residual information. This residual block is transformed and quantized by transform / quantization unit 230 to remove information that is not sensitive to the human eye, thereby eliminating visual redundancy. Optionally, the residual block before transformation and quantization by transform / quantization unit 230 can be called a temporal residual block, and the temporal residual block after transformation and quantization by transform / quantization unit 230 can be called a frequency residual block or a frequency domain residual block. Entropy coding unit 280 receives the quantized change coefficients output by change quantization unit 230 and can perform entropy coding on the quantized change coefficients to output a bitstream. For example, entropy coding unit 280 can eliminate character redundancy based on the target context model and the probability information of the binary bitstream. Optionally, in this application, the current block can be referred to as the current coding unit (CU) or the current prediction unit (PU), etc. The prediction block can also be referred to as the predicted image block or the image prediction block, and the reconstructed image block can also be referred to as the reconstruction block or the image reconstruction block.

[0078] At the decoding end, the entropy decoding unit 310 can parse the bitstream to obtain the prediction information and quantization coefficient matrix of the current block. The prediction unit 320 uses intra-frame prediction or inter-frame prediction to generate the prediction block of the current block based on the prediction information. The dequantization / transform unit 330 uses the quantization coefficient matrix obtained from the bitstream to perform dequantization and inverse transform on the quantization coefficient matrix to obtain the residual block. The reconstruction unit 340 adds the prediction block and the residual block to obtain the reconstructed block. The reconstructed blocks form the reconstructed image. The loop filtering unit 350 performs loop filtering on the reconstructed image based on the image or based on the blocks to obtain the decoded image. The encoding end also needs similar operations to the decoding end to obtain the decoded image. This decoded image can also be called the reconstructed image, which can be used as a reference frame for inter-frame prediction in subsequent frames.

[0079] It should be noted that the block partitioning information determined at the encoding end, as well as mode information or parameter information such as prediction, transform, quantization, entropy coding, and loop filtering, are carried in the bitstream when necessary. The decoding end determines the same block partitioning information, prediction, transform, quantization, entropy coding, and loop filtering mode information or parameter information as the encoding end by parsing the bitstream and analyzing existing information, thereby ensuring that the decoded image obtained by the encoding end is the same as the decoded image obtained by the decoding end.

[0080] It should also be understood that the above is the basic process of a video codec under a block-based hybrid coding framework. With the development of technology, some modules or steps of this framework or process may be optimized. This application is applicable to the basic process of a video codec under this block-based hybrid coding framework, but is not limited to this framework and process.

[0081] The above text combined Figures 1 to 3 The encoding / decoding systems, encoders, and decoders applicable to the embodiments of this application are described. From the perspective of signal acquisition methods, video signals can be divided into two types: those captured by a camera and those generated by a computer. Due to differences in statistical characteristics, their corresponding compression encoding methods may also differ, but they will all perform one or more of the following operations and processes on the input raw video signal:

[0082] 1) Block Partition Structure: The input image is divided into several non-overlapping processing units based on its size. Each processing unit undergoes a similar compression operation. This processing unit is called a CTU or LCU. Further subdivisions can be made below the CTU to obtain one or more basic coding units, called CUs. Each CU is the most basic element in a coding process. The following describes the various coding methods that can be used for each CU.

[0083] 2) Predictive Coding: This includes intra-frame prediction and inter-frame prediction. The original video signal is predicted using a selected reconstructed video signal to obtain the residual video signal. The encoder needs to determine the most suitable predictive coding mode from among many possible modes for the current CU and inform the decoder.

[0084] a. Intra-frame prediction: The predicted signal comes from a region within the same image that has already been encoded and reconstructed.

[0085] b. Inter-frame prediction: The predicted signal comes from other encoded images (called reference images) that are different from the current image.

[0086] 3) Transform & Quantization: The residual video signal undergoes transform operations such as DFT and DCT to convert the signal to the transform domain, resulting in transform coefficients. In the transform domain, the signal undergoes further lossy quantization, losing some information to make the quantized signal more suitable for compression. Some video coding standards may offer more than one transform option; therefore, the encoder needs to select one transform for the current encoding CU and inform the decoder. The fineness of quantization is usually determined by the quantization parameter (QP). A larger QP value means that coefficients with a wider range of values ​​will be quantized into the same output, which usually leads to greater distortion and a lower bitrate. Conversely, a smaller QP value means that coefficients with a smaller range of values ​​will be quantized into the same output, which usually leads to less distortion and a higher bitrate.

[0087] 4) Entropy Coding or Statistical Coding: The quantized transform domain signal is statistically compressed and encoded based on the frequency of each value, ultimately outputting a binary (0 or 1) compressed bitstream. Simultaneously, other information generated during encoding, such as the selected mode and motion vectors, also requires entropy coding to reduce the bit rate. Statistical coding is a lossless coding method that effectively reduces the bit rate required to represent the same signal. Common statistical coding methods include Variable Length Coding (VLC) or Content Adaptive Binary Arithmetic Coding (CABAC).

[0088] 5) Loop Filtering: After the encoded image undergoes inverse quantization, inverse transform, and prediction compensation operations (the reverse of operations 2-4 above), a reconstructed decoded image is obtained. Compared to the original image, the reconstructed image differs in some information due to the influence of quantization, resulting in distortion. Filtering the reconstructed image, such as deblocking, SAO, or ALF filters, can effectively reduce the distortion caused by quantization. Since these filtered reconstructed images will serve as a reference for subsequent encoded images to predict future signals, the above filtering operations are also called loop filtering, or filtering operations within the coding loop.

[0089] During the decoding process, for each CU, after obtaining the compressed bitstream, the decoder first performs entropy decoding to obtain various mode information and quantized transform coefficients. Each coefficient undergoes inverse quantization and inverse transform to obtain the residual signal. On the other hand, based on the known coding mode information, the prediction signal corresponding to that CU can be obtained. The two are then added together to obtain the reconstructed signal. Finally, the reconstructed value of the decoded image needs to undergo a loop filtering operation to generate the final output signal.

[0090] Mainstream video coding standards, such as HEVC, VVC, and AVS3, all employ a block-based hybrid coding framework. This framework divides the raw video data into a series of coded blocks and combines video coding methods such as prediction, transform, and entropy coding to achieve video data compression. Motion compensation is a commonly used prediction method in video coding. Based on the redundancy characteristics of video content in the temporal or spatial domains, motion compensation derives the predicted value of the current coded block from the already coded regions. These prediction methods include inter-frame prediction, block-copy intra-frame prediction, and string-copy intra-frame prediction. In specific coding implementations, these prediction methods may be used individually or in combination.

[0091] The following section introduces intra-frame prediction for string duplication.

[0092] 1. Ordinary string prediction sub-mode in string copy intra-frame prediction.

[0093] Intra String Copy (ISC) divides a coded block into a series of pixel strings or unmatched pixels according to a certain scanning order (raster scan, round-trip scan, and Zig-Zag scan, etc.). Similar to IBC, each string searches for a reference string of the same shape in the currently coded region of the image, derives the predicted value of the current string, and effectively saves bits by encoding the residual between the current string's pixel value and the predicted value instead of directly encoding the pixel value.

[0094] Figure 4 A schematic diagram of intra-frame prediction for string duplication is given.

[0095] like Figure 4 As shown, the dark gray area represents the encoded area, the 28 white pixels represent string 1, the 35 light gray pixels represent string 2, and the 1 black pixel represents an unmatched pixel. Unmatched pixels are also called isolated points. The raw values ​​of unmatched pixels are directly encoded, rather than derived from predicted values. It should be noted that intra-frame string copying technology requires encoding the string vector (SV), string length, and a flag indicating whether a matching string exists for each string in the current coding block. The string vector (SV) represents the displacement of the string to be encoded from its reference string. The string length represents the number of pixels contained in the string. For example... Figure 4As shown, the reference string of string 1 is to its left, and the displacement of string 1 to its corresponding reference string is represented by string vector 1. The reference string of string 2 is above it, and the displacement of string 2 to its corresponding reference string is represented by string vector 2.

[0096] 2. String copying of equivalent strings and unit basis vector string sub-patterns in intra-frame prediction.

[0097] Equivalent string and unit vector string modes are sub-modes of string-copy intra-prediction, adopted in the AVS3 standard in October 2020. Similar to string-copy intra-prediction, this mode divides an encoded / decoded block into a series of pixel strings or unmatched pixels according to a certain scan order. The pixel strings can be of type equivalent string or unit vector string. The characteristic of an equivalent string is that all pixels in the string have the same predicted value. The characteristic of a unit vector string (also called a unit base vector string, unit offset string, copy above string, etc.) is that its displacement vector is (0, -1), and each pixel in this string uses the pixel above it as the predicted value for the current pixel. The equivalent string mode requires encoding the type, length, and predicted value information of each string in the current encoded block in the bitstream. Similar to the ordinary string prediction sub-mode, the predicted values ​​of unmatched pixels are directly encoded, rather than derived from the predicted values.

[0098] In the AVS3 standard, for serially copied intra-predictive mode coded blocks, the Y, Cb, and Cr components of unmatched pixels are directly encoded instead of being derived from predicted values. Each component of an unmatched pixel is encoded according to its bit depth. For example, if the current image has a bit depth of 10 bits, encoding the Y, Cb, and Cr components of an unmatched pixel requires 3*10 bits of sign bit encoding. That is, each sign bit is encoded in bypass mode, resulting in excessive encoding overhead for unmatched pixels.

[0099] This application proposes an encoding method for unmatched pixels. This method adjusts the component values ​​of the unmatched pixels at the encoding end and selects an appropriate context model for encoding, thereby reducing the encoding overhead of unmatched pixels and improving encoding performance. It should be noted that the method provided in this application is applicable to codecs using string-copy intra-prediction mode or other modes that require encoding of unmatched pixels (such as palette mode), and this application does not specifically limit its application in this regard.

[0100] Figure 5This is a schematic flowchart of a method 400 for encoding unmatched pixels provided in an embodiment of this application. The executing entity of this method 400 can be, but is not limited to, devices such as encoders or devices for performing block vector encoding, such as desktop computers, mobile computing devices, laptops (e.g., laptops), tablet computers, set-top boxes, smartphones and other handheld devices, televisions, cameras, display devices, digital media players, video game consoles, in-vehicle computers, or similar devices. For example, Figure 1 The encoding device 110 shown or Figure 2 The video encoder 200 shown.

[0101] like Figure 5 As shown, the encoding method 400 for the unmatched pixel may include:

[0102] S410, the target image frame in the target sequence is divided into multiple image blocks, and the multiple image blocks include the target image block;

[0103] S420, Adjust the component values ​​of unmatched pixels in the target image block;

[0104] S430, by binarizing the component values ​​of the unmatched pixel, a binary symbol string of the unmatched pixel is obtained;

[0105] S440, the binary symbol string of the unmatched pixel is encoded using at least two entropy coding methods to obtain the bitstream of the target sequence.

[0106] In this embodiment of the application, the target sequence bitstream is decoded by at least two entropy coding methods, that is, the target sequence bitstream is encoded by multiple entropy coding methods, which can improve the coding flexibility and help balance the coding performance and coding overhead of mismatched pixels.

[0107] It should be noted that the component values ​​of the unmatched pixels involved in this application can be understood as the values ​​of the color components of the unmatched pixels. For example, for a YUV image, the component values ​​of the unmatched pixels can be the values ​​of the Y component, U component, or V component of the unmatched pixels.

[0108] In some embodiments, the length of the binary symbol string is M, where M is an integer greater than 0, and the at least two entropy encoding methods include a first entropy encoding method and a second entropy encoding method; wherein, S440 may include:

[0109] By decoding the first MN bits of the binary symbol string using the first entropy encoding method, and by decoding the last N bits of the binary symbol string using the second entropy encoding method, the binary symbol string of unmatched pixels in the target image block is obtained.

[0110] In other words, the encoder encodes the binary symbol string of the unmatched pixel using two entropy encoding methods to obtain the bitstream of the target sequence.

[0111] In some embodiments, the first entropy encoding method is a bypass mode, and the second entropy encoding method is a context-based binary arithmetic encoding (CABAC) mode.

[0112] In this embodiment, the binary symbol string of the unmatched pixel is encoded using at least two entropy encoding methods. Since these at least two entropy encoding methods are constructed such that the first MN bits of the binary symbol string are entropy encoded in a bypass mode, and the last N bits are entropy encoded in a context-based binary arithmetic encoding (CABAC) mode, on the one hand, designing the first MN bits of the binary symbol string to use a bypass mode for entropy encoding ensures encoding performance; on the other hand, using the last N bits of the binary symbol string in a context-based binary arithmetic encoding (CABAC) mode reduces the encoding overhead of the unmatched pixel. Based on this, the solution provided in this application can reduce the encoding overhead of the unmatched pixel while ensuring encoding performance.

[0113] It should be noted that the bypass mode can be understood as the CABAC mode under specific probability information. However, due to the setting of this specific probability information, the use of the bypass mode for entropy coding will result in a large coding overhead.

[0114] In some embodiments, the method 400 may further include:

[0115] Write at least one of the following identifiers into the bitstream;

[0116] First logo, second logo, third logo, or fourth logo;

[0117] Wherein, the first identifier is used to indicate whether the at least two entropy coding methods are used for entropy coding of unmatched pixels in the target sequence; the second identifier is used to indicate whether the at least two entropy coding methods are used for entropy coding of unmatched pixels in the target image frame in the target sequence, the target image frame including the target image block; the third identifier is used to indicate whether the at least two entropy coding methods are used for entropy coding of unmatched pixels in the target slice in the target image frame, the target slice including the target image block; and the fourth identifier is used to indicate whether the at least two entropy coding methods are used for entropy coding of unmatched pixels in the target image block.

[0118] It should be noted that the specific values of the first identifier, the second identifier, the third identifier, or the fourth identifier can be obtained by querying in the corresponding parameter set, or can be obtained by other means. The embodiments of the present application do not make specific limitations in this regard. For example, the values of each identifier in the parameter set can be pre-set parameters. For example, before encoding, the user can set the values of each identifier in the parameter set by setting the parameters of the encoder.

[0119] In some embodiments, the at least two entropy encoding methods are default modes for unmatched pixels; wherein, the S440 may include:

[0120] Encoding the binary symbol string of the unmatched pixels through the default mode to obtain the code stream of the target sequence.

[0121] In other words, the encoder can determine whether the target image block uses the encoding method provided by the present application for unmatched pixels through at least one of the following methods: [[ID=Eleven]]

[0122] [[ID=Twelve]]a), By default, use the encoding method provided by the present application for unmatched pixels in the target coding block, that is, no coding mark needs to be encoded in the code stream. b), Encoding a sequence-level mark in the code stream, indicating that all unmatched pixels in the target sequence use the encoding method provided by the present application for unmatched pixels. c), Encoding an image-level (frame-level) mark in the code stream, indicating that all unmatched pixels in the target image frame use the encoding method provided by the present application for unmatched pixels. d), Encoding a slice-level mark in the code stream, indicating that all unmatched pixels in the target slice use the encoding method provided by the present application for unmatched pixels. e), Encoding a CU-level (i.e., image block-level) mark in the code stream, indicating that all unmatched pixels in the target image block use the encoding method provided by the present application for unmatched pixels.

[0123] In some embodiments, the S420 may include:

[0124] Adjusting the component value of the unmatched pixel to (val>>N)<<N; where val represents the component value of the unmatched pixel before adjustment, >> represents the right shift operator, and << represents the left shift operator.

[0125] In other words, the encoding end adjusts the pixels of the unmatched pixels and selects a suitable context model for encoding the component value of the adjusted unmatched pixel; correspondingly, the decoding end selects the corresponding context model for decoding.

[0126] In some embodiments, before adjusting the component value of the unmatched pixel to (val>>N)<<N, the method 400 may further include:

[0127] Determine the value of N based on quantization parameters.

[0128] In one implementation, the encoder can determine the value of N according to the quantization parameter for the target image block. For example, the larger the quantization parameter of the target image block, the larger or smaller the value of N. For example, the encoder can determine the value of N according to the correspondence between at least one quantization parameter and at least one numerical value. Specifically, the encoder can determine the numerical value corresponding to the target image block as the value of N.

[0129] In some embodiments, the component value of the unmatched pixel is a chrominance component value or a luminance component value, and the value of N used for the chrominance component value is different from the value of N used for the luminance component value.

[0130] In one implementation, denote val_Y as the luminance component value of the unmatched pixel before adjustment. The encoding end can adjust the luminance component value of the unmatched pixel to (val_Y >> N1) << N1; again, denote val_U as the chrominance component value of the unmatched pixel before adjustment. The encoding end can adjust the chrominance component value of the unmatched pixel to (val_U >> N2) << N2. Optionally, the value of N1 and the value of N2 can be the same or different, and the present application does not make specific limitations in this regard. For example, the value of N1 is less than the value of N2 to ensure the encoding performance of the luminance component to a limited extent.

[0131] In some embodiments, the S420 may include:

[0132] Adjust the component value of the unmatched pixel to val / K * K;

[0133] Where val represents the component value of the unmatched pixel before adjustment, / represents integer division operation, and * represents multiplication operation.

[0134] It should be noted that the present application embodiments do not make specific limitations on the value of K. For example, K can be equal to N or not equal to N. Again, K can be a positive integer greater than 0.

[0135] In some embodiments, before adjusting the component value of the unmatched pixel to val / K * K, the method 400 may further include:

[0136] Determine the value of K based on quantization parameters.

[0137] In one implementation, the encoder can determine the value of K based on the quantization parameters for the target image patch. For example, the larger the quantization parameters of the target image patch, the larger or smaller the value of K. Alternatively, the encoder can determine the value of K based on the correspondence between at least one quantization parameter and at least one numerical value. Specifically, the encoder can determine the value of K as the numerical value corresponding to the target image patch.

[0138] In some embodiments, the component value of the unmatched pixel is a chroma component value or a luminance component value, wherein the value of K used for the chroma component value is different from the value of K used for the luminance component value.

[0139] In one implementation, let val_Y be the luminance component value of the unmatched pixel before adjustment. The encoder can adjust the luminance component value of the unmatched pixel to val_Y / K1*K1. Similarly, let val_U be the chrominance component value of the unmatched pixel before adjustment. The encoder can adjust the chrominance component value of the unmatched pixel to val_U / K2*K2. Optionally, the values ​​of K1 and K2 can be the same or different; this application does not impose specific limitations on this. For example, the value of K1 can be less than the value of K2 to ensure the encoding performance of the luminance component to a certain extent. As an example, the decoding method provided by this application can be applied only to the luminance component, i.e., N1 is not 0 and N2 is 0. As another example, the decoding method provided by this application can be applied only to the chrominance component, i.e., N2 is not 0 and N1 is 0.

[0140] It should be noted that in the embodiments of this application, N and K can be used in combination. That is, K can be used to adjust the component value of the unmatched pixel, and N can be used to determine the sign bit in the binary symbol string corresponding to each of the at least two entropy coding methods.

[0141] The following example uses 10-bit encoding of the input video as an illustration to explain the encoding method provided in this application.

[0142] Method 1:

[0143] a) Let val be the luminance or chrominance component value of the currently unmatched pixel. The encoder will adjust the value of val to (val>>2)<<2.

[0144] b) The encoding end uses a fixed-length code to binarize the adjusted val, deriving a binary symbol string of length 10. The first 8 bits of the binary symbol string are entropy encoded using a bypass mode, and the last 2 bits are entropy encoded using a CABAC mode.

[0145] Correspondingly, the decoding end decodes a 10-bit binary symbol string from the bitstream. The first 8 bits are entropy encoded using a bypass mode, and the last 2 bits are entropy encoded using a CABAC mode. Then, it is debinarized using a fixed-length code to obtain the value of val.

[0146] Method 2:

[0147] a) The encoder sets the value of N according to the quantization parameter. For 27, 32, 38 and 45, the value of N is set to 2, 2, 3 and 3 respectively.

[0148] b) The encoding end adjusts the value of val to (val >> N) < <N;

[0149] c) The encoder uses a fixed-length code to binarize the adjusted val, deriving a binary symbol string of length 10. The first 10-N bits of the binary symbol string are entropy encoded using a bypass mode, and the last N bits are entropy encoded using a CABAC mode.

[0150] Correspondingly, the decoding end decodes a 10-bit binary symbol string from the bitstream. The first 10-N bits are entropy encoded using a bypass mode, and the last N bits are entropy encoded using a CABAC mode. Then, it is debinarized using a fixed-length code to obtain the value of val.

[0151] Method 3:

[0152] a) The encoder sets the value of M according to the quantization parameter. For 27, 32, 38, and 45, the value of M is set to 2, 2, 3, and 3 respectively.

[0153] b) Record the value of val as the luminance or chrominance component value of the currently unmatched pixel. If val is the luminance component value, the encoder sets N=M; otherwise, if val is the chrominance component value, the encoder sets N=M+1.

[0154] c) The encoding end adjusts the value of val to (val >> N) < <N;

[0155] d) The encoding end uses a fixed-length code to binarize the adjusted val, deriving a binary symbol string of length 10. The first 10-N bits of the binary symbol string are entropy encoded using a bypass mode, and the last N bits are entropy encoded using a CABAC mode.

[0156] Correspondingly, the decoding end decodes a 10-bit binary symbol string from the bitstream. The first 10-N bits are entropy encoded using a bypass mode, and the last N bits are entropy encoded using a CABAC mode. Then, it is debinarized using a fixed-length code to obtain the value of val.

[0157] The preferred embodiments of this application have been described in detail above with reference to the accompanying drawings. However, this application is not limited to the specific details of the above embodiments. Within the scope of the technical concept of this application, various simple modifications can be made to the technical solutions of this application, and these simple modifications all fall within the protection scope of this application. For example, the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, this application will not describe the various possible combinations separately. Furthermore, various different embodiments of this application can also be arbitrarily combined, as long as they do not violate the spirit of this application, they should also be considered as the content disclosed in this application.

[0158] It should also be understood that, in the various method embodiments of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. Furthermore, in the embodiments of this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. Specifically, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0159] The above text combined Figure 5 The encoding method for unmatched pixels provided in the embodiments of this application is described in detail below. Figure 6 The decoding method for unmatched pixels provided in the embodiments of this application will be described.

[0160] Figure 6 This is a schematic flowchart of a decoding method 500 for unmatched pixels provided in an embodiment of this application. The executing entity of this method 500 can be, but is not limited to, a decoder or a device for performing block vector decoding, such as a desktop computer, mobile computing device, laptop (e.g., tablet computer), set-top box, smartphone, television, camera, display device, digital media player, video game console, in-vehicle computer, or similar devices. For example, Figure 1 The decoding device 120 shown Figure 3 The video decoder 300 shown.

[0161] like Figure 6 As shown, the decoding method 500 for the unmatched pixel may include:

[0162] S510, decode the bitstream of the target sequence using at least two entropy decoding methods to obtain a binary symbol string of unmatched pixels in the target image block;

[0163] S520, by debinarizing the binary symbol string to be debinarized, the component value of the unmatched pixel is obtained;

[0164] S530, the target image block is obtained based on the component values ​​of the unmatched pixel.

[0165] In some embodiments, the length of the binary symbol string is M, where M is an integer greater than 0, and the at least two entropy decoding methods include a first entropy decoding method and a second entropy decoding method; wherein, S510 may include:

[0166] By decoding the first MN bits of the binary symbol string using the first entropy decoding method, and by decoding the last N bits of the binary symbol string using the second entropy decoding method, the binary symbol string of unmatched pixels in the target image block is obtained.

[0167] In some embodiments, the first entropy decoding method is a bypass mode, and the second entropy decoding method is a context-based binary arithmetic encoding (CABAC) mode.

[0168] In this embodiment, the target sequence bitstream is decoded using at least two entropy decoding methods. These methods are constructed such that the first MN bits of the binary symbol string are decoded using a bypass mode, while the last N bits are decoded using context-based binary arithmetic coding (CABAC). On one hand, designing the first MN bits of the binary symbol string to use a bypass mode for entropy decoding ensures decoding performance. On the other hand, using context-based binary arithmetic coding (CABAC) for entropy decoding of the last N bits reduces the encoding overhead of unmatched pixels. Therefore, the solution provided in this application can reduce the encoding overhead of unmatched pixels while ensuring encoding performance.

[0169] In some embodiments, before decoding the bitstream of the target sequence using at least two entropy decoding methods to obtain the binary symbol string of unmatched pixels in the target image block, the method 500 may further include:

[0170] By parsing the bitstream of the target sequence, at least one of the following identifiers can be obtained:

[0171] First logo, second logo, third logo, or fourth logo;

[0172] Wherein, the first identifier is used to indicate whether entropy decoding is performed on unmatched pixels in the target sequence using the at least two entropy decoding methods; the second identifier is used to indicate whether entropy decoding is performed on unmatched pixels in a target image frame in the target sequence using the at least two entropy decoding methods, the target image frame including the target image block; the third identifier is used to indicate whether entropy decoding is performed on unmatched pixels in a target patch in the target image frame using the at least two entropy decoding methods, the target patch including the target image block; the fourth identifier is used to indicate whether entropy decoding is performed on unmatched pixels in the target image block using the at least two entropy decoding methods; wherein, S510 may include:

[0173] If the first identifier, the second identifier, the third identifier, or the third identifier is used to indicate that entropy decoding is performed using the at least two entropy decoding methods, then the bitstream of the target sequence is decoded using the at least two entropy decoding methods to obtain a binary symbol string of unmatched pixels in the target image block.

[0174] In some embodiments, the at least two entropy decoding methods are a default mode for unmatched pixels; wherein, S510 may include:

[0175] By decoding the target sequence's bitstream using this default mode, a binary symbol string of unmatched pixels in the target image block is obtained.

[0176] In other words, the decoder can determine whether the target image block uses the decoding method for unmatched pixels provided in this application by at least one of the following methods:

[0177] a) For unmatched pixels in the target decoding block, the decoding method for unmatched pixels provided in this application is used by default, i.e., no decoding marker is required in the bitstream. b) A sequence-level marker is decoded in the bitstream, indicating that the decoding method for unmatched pixels provided in this application is used for all unmatched pixels in all decoding blocks of the target sequence. c) An image-level (frame-level) marker is decoded in the bitstream, indicating that the decoding method for unmatched pixels provided in this application is used for all unmatched pixels in the target image frame. d) A slice-level marker is decoded in the bitstream, indicating that the decoding method for unmatched pixels provided in this application is used for all unmatched pixels in the target slice. e) A CU-level (i.e., image block-level) marker is decoded in the bitstream, indicating that the decoding method for unmatched pixels provided in this application is used for all unmatched pixels in the target image block.

[0178] In some embodiments, before decoding the bitstream of the target sequence using at least two entropy decoding methods to obtain the binary symbol string of unmatched pixels in the target image block, the method 500 may further include:

[0179] Determine the value of N based on quantization parameters.

[0180] In one implementation, the decoder can determine the value of N according to the quantization parameter for the target image block. For example, the larger the quantization parameter of the target image block, the larger or smaller the value of N. For example, the decoder can determine the value of N according to the correspondence between at least one quantization parameter and at least one numerical value. Specifically, the decoder can determine the numerical value corresponding to the target image block as the value of N.

[0181] In some embodiments, the component value of the unmatched pixel is a chrominance component value or a luminance component value, and the value of N used for the chrominance component value is different from the value of N used for the luminance component value.

[0182] In one implementation, denote val_Y as the luminance component value of the unmatched pixel before adjustment. The encoding end can adjust the luminance component value of the unmatched pixel to (val_Y >> N1) << N1; for another example, denote val_U as the chrominance component value of the unmatched pixel before adjustment. The encoding end can adjust the chrominance component value of the unmatched pixel to (val_U >> N2) << N2. Optionally, the value of N1 and the value of N2 can be the same or different, and this application does not make specific limitations in this regard. For example, the value of N1 is less than the value of N2 to ensure the encoding performance of the luminance component. As an example, the decoding method provided in this application can be applied only to the luminance component, that is, N1 is not 0 and N2 is 0. As another example, the decoding method provided in this application can be applied only to the chrominance component, that is, N2 is not 0 and N1 is 0.

[0183] It should be noted that the decoding process is the reverse operation of the encoding process. The decoding method 500 can refer to the relevant description of the encoding method 400. To avoid repetition, it will not be elaborated here.

[0184] As described above in combination with Figures 5 to 6 , the method embodiments of this application have been described in detail. Below in combination with Figures 7 to 9 , the device embodiments of this application will be described in detail.

[0185] Figure 7 is a schematic flowchart of the encoder 600 provided by the embodiments of this application.

[0186] As Figure 7 shown, the encoder 600 may include:

[0187] A partitioning unit 610, configured to partition a target image frame in a target sequence into multiple image blocks, where the multiple image blocks include a target image block;

[0188] An adjustment unit 620, configured to adjust the component value of an unmatched pixel in the target image block;

[0189] Binarization unit 630 is used to obtain a binary symbol string of the unmatched pixel by binarizing the component values ​​of the unmatched pixel;

[0190] The encoding unit 640 is used to encode the binary symbol string of the unmatched pixel using at least two entropy encoding methods to obtain the bitstream of the target sequence.

[0191] In this embodiment of the application, the target sequence bitstream is decoded by at least two entropy coding methods, that is, the target sequence bitstream is encoded by multiple entropy coding methods, which can improve the flexibility of encoding and coding and is conducive to balancing the encoding performance and encoding overhead of mismatched pixels.

[0192] In some embodiments, the length of the binary symbol string is M, where M is an integer greater than 0, and the at least two entropy encoding methods include a first entropy encoding method and a second entropy encoding method; wherein, the encoding unit 640 is specifically used for:

[0193] The target sequence's code stream is obtained by encoding the first MN bits of the binary symbol string using the first entropy encoding method and the last N bits of the binary symbol string using the second entropy encoding method.

[0194] In some embodiments, the first entropy encoding method is a bypass mode, and the second entropy encoding method is a context-based binary arithmetic encoding (CABAC) mode.

[0195] In some embodiments, the encoding unit 640 is further configured to:

[0196] Write at least one of the following identifiers into the bitstream;

[0197] First logo, second logo, third logo, or fourth logo;

[0198] Wherein, the first identifier is used to indicate whether the at least two entropy coding methods are used for entropy coding of unmatched pixels in the target sequence; the second identifier is used to indicate whether the at least two entropy coding methods are used for entropy coding of unmatched pixels in the target image frame in the target sequence, the target image frame including the target image block; the third identifier is used to indicate whether the at least two entropy coding methods are used for entropy coding of unmatched pixels in the target slice in the target image frame, the target slice including the target image block; and the fourth identifier is used to indicate whether the at least two entropy coding methods are used for entropy coding of unmatched pixels in the target image block.

[0199] In some embodiments, the at least two entropy encoding methods are the default mode for unmatched pixels; wherein, the encoding unit 640 is specifically used for:

[0200] Encoding the binary symbol string of the unmatched pixel through the default mode to obtain the code stream of the target sequence.

[0201] In some embodiments, the adjustment unit 620 is specifically configured to:

[0202] Adjust the component value of the unmatched pixel to (val >> N) << N;

[0203] Where val represents the component value of the unmatched pixel before adjustment, >> represents the right shift operator, and << represents the left shift operator.

[0204] In some embodiments, before adjusting the component value of the unmatched pixel to (val >> N) << N, the adjustment unit 620 is further configured to:

[0205] Determine the value of N based on the quantization parameter.

[0206] In some embodiments, the component value of the unmatched pixel is a chrominance component value or a luminance component value, and the value of N used for the chrominance component value is different from the value of N used for the luminance component value.

[0207] In some embodiments, S420 may include:

[0208] Adjust the component value of the unmatched pixel to val / K * K;

[0209] Where val represents the component value of the unmatched pixel before adjustment, / represents the integer division operation, and * represents the multiplication operation.

[0210] Figure 8 It is a schematic flowchart of the decoder 700 provided by the embodiments of the present application.

[0211] As Figure 8 shown, the decoder 700 may include:

[0212] A decoding unit 710, configured to decode the code stream of the target sequence through at least two entropy decoding methods to obtain the binary symbol string of the unmatched pixel in the target image block;

[0213] An inverse binarization unit 720, configured to perform inverse binarization on the binary symbol string to be inverse binarized to obtain the component value of the unmatched pixel;

[0214] A processing unit 730, configured to obtain the target image block based on the component value of the unmatched pixel.

[0215] In some embodiments, the length of the binary symbol string is M, M is an integer greater than 0, and the at least two entropy decoding methods include a first entropy decoding method and a second entropy decoding method; wherein, the decoding unit 710 is specifically configured to:

[0216] By decoding the first MN bits of the binary symbol string using the first entropy decoding method, and by decoding the last N bits of the binary symbol string using the second entropy decoding method, the binary symbol string of unmatched pixels in the target image block is obtained.

[0217] In some embodiments, the first entropy decoding method is a bypass mode, and the second entropy decoding method is a context-based binary arithmetic encoding (CABAC) mode.

[0218] In some embodiments, before decoding the bitstream of the target sequence using at least two entropy decoding methods to obtain the binary symbol string of unmatched pixels in the target image block, the decoding unit 710 is further configured to:

[0219] By parsing the bitstream of the target sequence, at least one of the following identifiers can be obtained:

[0220] First logo, second logo, third logo, or fourth logo;

[0221] Wherein, the first identifier is used to indicate whether entropy decoding is performed on unmatched pixels in the target sequence using the at least two entropy decoding methods; the second identifier is used to indicate whether entropy decoding is performed on unmatched pixels in a target image frame in the target sequence using the at least two entropy decoding methods, the target image frame including the target image block; the third identifier is used to indicate whether entropy decoding is performed on unmatched pixels in a target slice in the target image frame using the at least two entropy decoding methods, the target slice including the target image block; the fourth identifier is used to indicate whether entropy decoding is performed on unmatched pixels in the target image block using the at least two entropy decoding methods; wherein, the decoding unit 710 is specifically used for:

[0222] If the first identifier, the second identifier, the third identifier, or the third identifier is used to indicate that entropy decoding is performed using the at least two entropy decoding methods, then the bitstream of the target sequence is decoded using the at least two entropy decoding methods to obtain a binary symbol string of unmatched pixels in the target image block.

[0223] In some embodiments, the at least two entropy decoding methods are the default mode for unmatched pixels; wherein, the decoding unit 710 is specifically used for:

[0224] By decoding the target sequence's bitstream using this default mode, a binary symbol string of unmatched pixels in the target image block is obtained.

[0225] In some embodiments, before decoding the bitstream of the target sequence using at least two entropy decoding methods to obtain the binary symbol string of unmatched pixels in the target image block, the decoding unit 710 is further configured to:

[0226] The value of N is determined based on the quantization parameters.

[0227] In some embodiments, the component value of the unmatched pixel is a chroma component value or a luminance component value, wherein the value of N used for the chroma component value is different from the value of N used for the luminance component value.

[0228] It should be understood that the device embodiments and method embodiments can correspond to each other, and similar descriptions can be found in the method embodiments. To avoid repetition, further details are omitted here. Specifically, Figure 7 The encoder 600 shown can correspond to the corresponding subject in the method 400 of the embodiments of this application, that is, the foregoing and other operations and / or functions of each unit in the encoder 600 are respectively for implementing the corresponding processes in the various methods such as method 400. Figure 8 The decoder 700 shown can correspond to the corresponding subject in the method 500 of the embodiments of this application, and the foregoing and other operations and / or functions of each unit in the decoder 700 are respectively for implementing the corresponding processes in the various methods such as method 500.

[0229] It should also be understood that the various units in the encoder 600 or decoder 700 involved in the embodiments of this application can be individually or entirely merged into one or more other units, or some of the units can be further divided into multiple functionally smaller units. This can achieve the same operation without affecting the technical effect of the embodiments of this application. The above-mentioned units are based on logical function division. In practical applications, the function of one unit can also be implemented by multiple units, or the function of multiple units can be implemented by one unit. In other embodiments of this application, the encoder 600 or decoder 700 may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented collaboratively by multiple units. According to another embodiment of this application, the encoder 600 or decoder 700 of the embodiments of this application can be constructed by running a computer program (including program code) capable of performing the steps involved in the corresponding method on a general-purpose computing device including processing elements and storage elements such as a central processing unit (CPU), random access storage medium (RAM), and read-only storage medium (ROM), and by implementing the encoding method or decoding method for unmatched pixels of the embodiments of this application. The computer program can be recorded on, for example, a computer-readable storage medium, loaded into an electronic device through the computer-readable storage medium, and run therein to implement the corresponding methods of the embodiments of this application.

[0230] In other words, the units mentioned above can be implemented in hardware, in software instructions, or in a combination of hardware and software. Specifically, the steps of the method embodiments in this application can be completed by the integrated logic circuits in the processor's hardware and / or by software instructions. The steps of the method disclosed in the embodiments of this application can be directly manifested as being executed by a hardware decoding processor, or executed by a combination of hardware and software in the decoding processor. Optionally, the software can reside in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. This storage medium is located in memory, and the processor reads the information in the memory and completes the steps in the above method embodiments in conjunction with its hardware.

[0231] Figure 9 This is a schematic structural diagram of the encoding / decoding device 800 provided in the embodiments of this application.

[0232] like Figure 9 As shown, the encoding / decoding device 800 includes at least a processor 710 and a computer-readable storage medium 720. The processor 710 and the computer-readable storage medium 720 can be connected via a bus or other means. The computer-readable storage medium 720 stores a computer program 721, which includes computer instructions. The processor 710 executes the computer instructions stored in the computer-readable storage medium 720. The processor 710 is the computing and control core of the encoding / decoding device 800, and is suitable for implementing one or more computer instructions, specifically for loading and executing one or more computer instructions to achieve corresponding method flows or corresponding functions.

[0233] As an example, processor 710 may also be referred to as a central processing unit (CPU). Processor 710 may include, but is not limited to: general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0234] As an example, the computer-readable storage medium 720 may be a high-speed RAM memory or a non-volatile memory, such as at least one disk storage device; optionally, it may also be at least one computer-readable storage medium located remotely from the aforementioned processor 710. Specifically, the computer-readable storage medium 720 includes, but is not limited to, volatile memory and / or non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory may be random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0235] In one implementation, the encoding / decoding device 800 may be the encoding end, encoder, or encoding framework involved in the embodiments of this application; the computer-readable storage medium 720 stores first computer instructions; the processor 710 loads and executes the first computer instructions stored in the computer-readable storage medium 720 to implement the corresponding steps in the encoding method for unmatched pixels provided in the embodiments of this application; in other words, the first computer instructions in the computer-readable storage medium 720 are loaded and executed by the processor 710 to implement the corresponding steps, which will not be described again here to avoid repetition.

[0236] In one implementation, the encoding / decoding device 800 may be the decoding end, decoder, or decoding framework involved in the embodiments of this application; the computer-readable storage medium 720 stores second computer instructions; the processor 710 loads and executes the second computer instructions stored in the computer-readable storage medium 720 to implement the corresponding steps in the decoding method for unmatched pixels provided in the embodiments of this application; in other words, the second computer instructions in the computer-readable storage medium 720 are loaded and executed by the processor 710 to implement the corresponding steps, which will not be described again here to avoid repetition.

[0237] According to another aspect of this application, embodiments of this application also provide a computer-readable storage medium (Memory), which is a memory device in the encoding / decoding device 800 for storing programs and data. For example, a computer-readable storage medium 720. It is understood that the computer-readable storage medium 720 here may include the built-in storage medium in the encoding / decoding device 800, or it may include extended storage media supported by the encoding / decoding device 800. The computer-readable storage medium provides storage space that stores the operating system of the encoding / decoding device 800. Furthermore, the storage space also stores one or more computer instructions suitable for loading and execution by the processor 710, which may be one or more computer programs 721 (including program code).

[0238] According to another aspect of this application, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. For example, computer program 721. In this case, the encoding / decoding device 800 may be a computer, and the processor 710 reads the computer instructions from the computer-readable storage medium 720. The processor 710 executes the computer instructions, causing the computer to perform the encoding method or decoding method for unmatched pixels provided in the various alternative methods described above.

[0239] In other words, when implemented using software, it can be implemented entirely or partially in the form of a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes of the embodiments of this application are run or the functions of the embodiments of this application are implemented. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means.

[0240] Those skilled in the art will recognize that the units and process steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0241] Finally, it should be noted that the above content is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for decoding unmatched pixels, characterized in that, include: The target sequence's bitstream is decoded using at least two entropy decoding methods to obtain a binary symbol string of unmatched pixels in the target image block. The length of the binary symbol string is M, where M is an integer greater than 0. The component values ​​of the unmatched pixels are obtained by debinarizing the binary symbol string. The target image block is obtained based on the component values ​​of the unmatched pixels; Before decoding the bitstream of the target sequence using at least two entropy decoding methods to obtain the binary symbol string of unmatched pixels in the target image block, the method further includes: The value of N is determined based on the quantization parameters; The at least two entropy decoding methods include a first entropy decoding method and a second entropy decoding method. The first MN bits of the binary symbol string are decoded using the first entropy decoding method, and the last N bits of the binary symbol string are decoded using the second entropy decoding method.

2. The method according to claim 1, characterized in that, The first entropy decoding method is a bypass mode, and the second entropy decoding method is a context-based binary arithmetic encoding (CABAC) mode.

3. The method according to claim 1, characterized in that, Before decoding the bitstream of the target sequence using at least two entropy decoding methods to obtain the binary symbol string of unmatched pixels in the target image block, the method further includes: By parsing the bitstream of the target sequence, at least one of the following identifiers can be obtained: First logo, second logo, third logo, or fourth logo; Wherein, the first identifier is used to indicate whether entropy decoding is performed using the at least two entropy decoding methods for unmatched pixels in the target sequence; the second identifier is used to indicate whether entropy decoding is performed using the at least two entropy decoding methods for unmatched pixels in a target image frame in the target sequence, wherein the target image frame includes the target image block; the third identifier is used to indicate whether entropy decoding is performed using the at least two entropy decoding methods for unmatched pixels in a target slice in the target image frame, wherein the target slice includes the target image block; and the fourth identifier is used to indicate whether entropy decoding is performed using the at least two entropy decoding methods for unmatched pixels in the target image block. The step of decoding the bitstream of the target sequence using at least two entropy decoding methods to obtain the binary symbol string of unmatched pixels in the target image block includes: If the first identifier, the second identifier, the third identifier, or the fourth identifier is used to indicate that entropy decoding is performed using the at least two entropy decoding methods, then the bitstream of the target sequence is decoded using the at least two entropy decoding methods to obtain a binary symbol string of unmatched pixels in the target image block.

4. The method according to claim 1, characterized in that, The at least two entropy decoding methods are the default modes for unmatched pixels; The step of decoding the bitstream of the target sequence using at least two entropy decoding methods to obtain the binary symbol string of unmatched pixels in the target image block includes: The target sequence's bitstream is decoded using the default mode to obtain a binary symbol string of unmatched pixels in the target image block.

5. The method according to any one of claims 1 to 4, characterized in that, The component value of the unmatched pixel is a chrominance component value or a luminance component value, and the value of N used for the chrominance component value is different from the value of N used for the luminance component value.

6. A method for encoding unmatched pixels, characterized in that, It includes: Dividing the target image frame in the target sequence into multiple image blocks, where the multiple image blocks include the target image block; Adjusting the component value of the unmatched pixel in the target image block; By binarizing the component value of the unmatched pixel, obtaining a binary symbol string of the unmatched pixel, where the length of the binary symbol string is M, and M is an integer greater than 0; Encoding the binary symbol string of the unmatched pixel through at least two entropy encoding methods to obtain a bitstream of the target sequence; Where, before encoding the binary symbol string of the unmatched pixel through at least two entropy encoding methods to obtain a bitstream of the target sequence, the method further includes: Determining the value of N based on a quantization parameter; Where, the at least two entropy encoding methods include a first entropy encoding method and a second entropy encoding method, the first M - N bits of the binary symbol string are encoded using the first entropy encoding method, and the last N bits of the binary symbol string are encoded using the second entropy encoding method.

7. The method according to claim 6, characterized in that, The adjusting the component value of the unmatched pixel in the target image block includes: Adjusting the component value of the unmatched pixel to (val >> N) << N; Where, val represents the component value of the unmatched pixel before adjustment, >> represents the right shift operator, and << represents the left shift operator.

8. The method according to claim 7, characterized in that, Before adjusting the component value of the unmatched pixel to (val >> N) << N, the method further includes: Determining the value of N based on a quantization parameter.

9. The method according to claim 6, characterized in that, The adjusting the component value of the unmatched pixel in the target image block includes: Adjusting the component value of the unmatched pixel to val / K * K; Where, val represents the component value of the unmatched pixel before adjustment, / represents integer division operation, and * represents multiplication operation.

10. A decoder, characterized in that, It includes: A decoding unit, configured to decode the bitstream of the target sequence through at least two entropy decoding methods to obtain a binary symbol string of the unmatched pixel in the target image block, where the length of the binary symbol string is M, and M is an integer greater than 0; An inverse binarization unit, configured to obtain the component value of the unmatched pixel by inverse binarizing the binary symbol string; A processing unit, configured to obtain the target image block based on the component value of the unmatched pixel; Where, before the decoding unit decodes the bitstream of the target sequence through at least two entropy decoding methods to obtain a binary symbol string of the unmatched pixel in the target image block, it is further configured to: Determine the value of N based on a quantization parameter; Where, the at least two entropy decoding methods include a first entropy decoding method and a second entropy decoding method, the first M - N bits of the binary symbol string are decoded using the first entropy decoding method, and the last N bits of the binary symbol string are decoded using the second entropy decoding method.

11. An encoder, characterized in that, It includes: A dividing unit, configured to divide the target image frame in the target sequence into multiple image blocks, where the multiple image blocks include the target image block; An adjustment unit is used to adjust the component values ​​of unmatched pixels in the target image block; The binarization unit is used to obtain a binary symbol string of the unmatched pixel by binarizing the component values ​​of the unmatched pixel. The length of the binary symbol string is M, where M is an integer greater than 0. The encoding unit is used to encode the binary symbol string of the unmatched pixels using at least two entropy encoding methods to obtain the code stream of the target sequence; Before the adjustment unit encodes the binary symbol string of the unmatched pixels using at least two entropy coding methods to obtain the code stream of the target sequence, it is further used for: The value of N is determined based on the quantization parameters; The at least two entropy encoding methods include a first entropy encoding method and a second entropy encoding method. The first MN bits of the binary symbol string are encoded using the first entropy encoding method, and the last N bits of the binary symbol string are encoded using the second entropy encoding method.

12. A codec device, characterized in that, include: A processor, adapted to execute computer programs; A computer-readable storage medium storing a computer program, which, when executed by the processor, implements the decoding method as described in any one of claims 1 to 5, or the encoding method as described in any one of claims 6 to 9.

13. A computer-readable storage medium, characterized in that, Used to store a computer program that causes a computer to perform the decoding method as described in any one of claims 1 to 5, or the encoding method as described in any one of claims 6 to 9.

Images