Decoding method, encoding method and device

Abstract

A decoding method, encoding method and device for an image relate to the field of video encoding and decoding. The decoding method comprises: first, parsing a code stream to obtain one or more image frames, wherein an image frame comprises one or more CUs; ​​second, determining multiple QP values ​​of the image frame; wherein a CU comprises multiple QGs, and one QG corresponds to one QP value; finally, decoding the image frame according to the multiple QP values. Compared with the large image distortion caused by all residual coefficients in a CU using the same QP value, in this embodiment, a CU can be divided into multiple QGs, and one or more residual coefficients in each QG share a QP value. Then, the video decoder can make a more fine-grained QP decision for one or more CUs corresponding to the image frame, thereby reducing the decoding distortion of the image frame while ensuring a certain compression rate, and improving the authenticity and accuracy of video image decoding.

Description

Technical Field

[0001] The present application relates to the field of video coding and decoding, and in particular to an image decoding method, encoding method and device. Background Art

[0002] In video coding and decoding technology, video compression technology is particularly important. Video compression technology performs spatial (intra-image) prediction and / or temporal (inter-image) prediction to reduce or remove redundant information inherent in video sequences. The basic principle of video compression is to quantize the correlation between the spatial domain, temporal domain and codewords to remove redundancy as much as possible. Quantization refers to the process of mapping the continuous value (or a large number of discrete values) of a signal into a finite number of discrete amplitudes, realizing a many-to-one mapping of signal values.

[0003] The current practice is that for one or more coding units (CUs) included in a frame of image, the encoder obtains the quantization parameter (QP) of each CU and encodes the CU according to the QP to obtain a bitstream; correspondingly, the decoder dequantizes the bitstream to obtain the QP of the CU and decodes the CU according to the QP. CUs are divided according to image content, and the encoder and decoder use the same QP to quantize the image content corresponding to a CU, resulting in large quantization distortion in the image encoding and decoding process. Summary of the invention

[0004] The present application provides an image decoding method, an encoding method and an apparatus, which solve the problem of large quantization distortion in the image encoding and decoding process.

[0005] This application adopts the following technical solution.

[0006] In a first aspect, the present application provides a method for decoding an image, which can be applied to a video decoding system, or the method can be applied to a decoding end that can support video decoding to implement the method, for example, the decoding end includes a chip system or a video decoder, and the method includes: first, parsing the code stream to obtain one or more image frames, and an image frame includes one or more CUs; ​​second, determining multiple QP values ​​of the image frame; wherein a CU includes multiple QGs, and one QG corresponds to one QP value; finally, decoding the image frame according to the multiple QP values.

[0007] Compared with the larger image distortion caused by using the same QP value for all residual coefficients in a CU, in the decoding process of the video image provided in this embodiment, a CU can be divided into multiple QGs, and one or more residual coefficients in each QG share a QP value. Then, the video decoder can make a more fine-grained QP decision for one or more CUs corresponding to the image frame, thereby reducing the decoding distortion of the image frame while ensuring a certain compression rate, and improving the authenticity and accuracy of video image decoding.

[0008] In an optional implementation, the one CU includes multiple residual coefficients, the one QG includes a portion of the multiple residual coefficients, and the portion of the residual coefficients share the one QP value.

[0009] In an optional implementation, before determining multiple QP values ​​of the image frame, the method also includes: dividing a CU included in the image frame according to a first rule to obtain the multiple QGs; wherein the first rule includes a division domain and a division method, the division domain is a transform domain or a pixel domain, and the division method includes at least one of uniform division and non-uniform division.

[0010] In an optional implementation, the one CU includes multiple residual coefficients, and the positions of the multiple residual coefficients are marked by coordinates, and the coordinates include a horizontal coordinate and a vertical coordinate. If the partition domain is a transform domain, the one CU included in the one image frame is partitioned according to a first rule to obtain the multiple QGs, including: among the multiple residual coefficients, the residual coefficients whose coordinate sum does not reach a first coordinate threshold are partitioned into a first QG, and the residual coefficients whose coordinate sum reaches the first coordinate threshold are partitioned into a second QG; the coordinate sum is the sum of the horizontal coordinate and the vertical coordinate of the residual coefficient.

[0011] Alternatively, among the multiple residual coefficients, the residual coefficients whose coordinates and values ​​do not reach the first coordinate threshold are divided into the first QG, the residual coefficients whose coordinates and values ​​reach the first coordinate threshold but do not reach the second coordinate threshold are divided into the second QG, and the residual coefficients whose coordinates and values ​​reach the second coordinate threshold are divided into the third QG; the second coordinate threshold is greater than the first coordinate threshold.

[0012] In an optional implementation, the one CU includes multiple residual coefficients, and if the division domain is a transform domain, dividing a CU included in the one image frame according to a first rule to obtain the multiple QGs includes:

[0013] The multiple residual coefficients are sorted, and the residual coefficients that do not reach the first ratio threshold among the multiple residual coefficients are divided into the first QG, and the residual coefficients that reach the first ratio threshold are divided into the second QG; the sorting method of the multiple residual coefficients is any one of the following: Z-shaped, reverse Z-shaped.

[0014] Alternatively, the multiple residual coefficients are sorted, and the residual coefficients that do not reach the first proportion threshold among the multiple residual coefficients are divided into the first QG, the residual coefficients that reach the first proportion threshold but do not reach the second proportion threshold are divided into the second QG, and the residual coefficients that reach the second proportion threshold are divided into the third QG; the second proportion threshold is greater than the first proportion threshold.

[0015] In an optional implementation, the CU includes multiple residual coefficients. If the division domain is a pixel domain, the CU included in the image frame is divided according to a first rule to obtain the multiple QGs, including: symmetrically dividing the multiple residual coefficients in the horizontal or vertical direction to obtain two QGs containing the same number of residual coefficients.

[0016] Alternatively, the multiple residual coefficients are symmetrically divided in the horizontal or vertical direction to obtain three QGs; the number of residual coefficients contained in two non-adjacent QGs among the three QGs is consistent, and the number of residual coefficients contained in another QG is consistent with the sum of the numbers of residual coefficients contained in the two non-adjacent QGs.

[0017] Alternatively, the plurality of residual coefficients are divided horizontally or vertically to obtain two QGs containing different numbers of residual coefficients.

[0018] Alternatively, the multiple residual coefficients are divided horizontally or vertically to obtain three QGs; the residual coefficients included in the three QGs do not have a symmetrical relationship.

[0019] In an optional implementation, a QP value corresponding to the one QG includes a luminance QP value and a chrominance QP value. Determining multiple QP values ​​of the one image frame includes: respectively acquiring the luminance QP value and the chrominance QP value of the one QG.

[0020] Alternatively, a luminance QP value of the one QG is obtained; and a chrominance QP value of the one QG is determined based on the luminance QP value.

[0021] In an optional implementation, determining multiple QP values ​​of the image frame includes: parsing the code stream to obtain marking information of the image frame, the marking information is used to indicate the QP value of the QG, and / or the marking information is used to indicate the QP value of the CU.

[0022] In an optional implementation, multiple QP values ​​of the image frame are determined, including: first, parsing the code stream to obtain marking information of the image frame, wherein the marking information is used to indicate the QP offset of the QG; second, determining the QP value of the QG based on the predicted QP value of the QG and the marking information.

[0023] In an optional implementation, determining the QP value of the QG according to the predicted QP value of the QG and the mark information includes: obtaining the predicted QP value of the QG, and taking the sum of the predicted QP value of the QG and the QP offset as the QP value of the QG.

[0024] In an optional implementation, determining multiple QP values ​​of the image frame includes: obtaining a predicted QP value of the QG, and determining the QP value of the QG based on the predicted QP value of the QG and derivation information; wherein the derivation information is any one or a combination of the following: flatness information or texture information of the QG, remaining space or distortion constraint information of the code stream buffer.

[0025] In an optional implementation, if the derivation information is the distortion constraint information, and the distortion constraint information indicates a distortion threshold of any one of the multiple QGs, then the QP value of the QG is determined according to the predicted QP value of the QG and the derivation information, including: determining the predicted distortion corresponding to the predicted QP value; if the predicted distortion is less than or equal to the distortion threshold, using the predicted QP value as the QP value of the QG; if the predicted distortion is greater than the distortion threshold, using the QP value determined by the distortion threshold as the QP value of the QG.

[0026] In an optional implementation, if the derived information is the content information of the QG or the remaining space of the code stream buffer, the QP value of the QG is determined based on the predicted QP value of the QG and the derived information, including: determining the QP offset of the QG based on the derived information; and taking the sum of the predicted QP value of the QG and the QP offset as the QP value of the QG.

[0027] In an optional implementation, obtaining the predicted QP value of the QG includes: obtaining the QP value of at least one other QG adjacent to the QG in the CU; and determining the predicted QP value of the QG based on the QP value of the at least one other QG.

[0028] Alternatively, the QP value of the CU is used as the predicted QP value of the QG.

[0029] In an optional implementation, the one image frame includes at least a first partial CU and a second partial CU, the first partial CU and the second partial CU have no overlapping area, and the QP values ​​of the first partial CU and the second partial CU are obtained in different ways.

[0030] In an optional implementation, determining multiple QP values ​​of the one image frame includes: parsing the code stream to obtain the mark information of the one image frame, the mark information includes the QP offset of the first part CU; determining the QP value of the first part CU according to the mark information. And, for the second part CU, obtaining the predicted QP value of the second part CU; determining the QP value of the second part CU according to the predicted QP value and derived information of the second part CU; wherein the derived information is any one or a combination of the following: flatness information or texture information of the second part CU, remaining space or distortion constraint information of the code stream buffer.

[0031] In an optional implementation, decoding the one image frame according to the multiple QP values ​​includes: first, for each QP value in the multiple QP values, obtaining the quantization step size Qstep corresponding to the QP value; second, obtaining the horizontal value contained in the QG corresponding to the QP value; finally, according to the selected quantizer combination, dequantizing the horizontal value of the QG; the quantizer combination includes one or more quantizers. For example, the quantizer is a uniform quantizer or a non-uniform quantizer.

[0032] In an optional implementation, the quantizer combination is determined by the marking information carried by the code stream, or the distribution of residual coefficients in the QG.

[0033] In an optional implementation, the horizontal value of the QG is dequantized, including: first, determining the partition domain type of the QG. Secondly, if the partition domain type of the QG is the transform domain, a quantization matrix matching the parameter information of the QG is selected from the matrix template library of the decoding end; the matrix template library includes multiple types of quantization matrix templates, and the parameter information includes any one or more combinations of the following: the size of the QG, the size of the CU where the QG is located, luminance channel information, and flatness information. Finally, the horizontal value in the QG is dequantized using the quantization matrix of the QG to obtain the residual coefficient of the QG.

[0034] In an optional implementation, the multiple types of quantization matrix templates include flat block templates and texture block templates; the Qstep of the residual coefficients in the flat block template whose frequency is higher than a frequency threshold is greater than or equal to the Qstep of the residual coefficients in the flat block template whose frequency does not reach the frequency threshold; the Qstep of the residual coefficients in the texture block template whose frequency is higher than the frequency threshold is less than or equal to the Qstep of the residual coefficients in the texture block template whose frequency does not reach the frequency threshold.

[0035] In an optional implementation, the quantization matrix templates included in the matrix template library are obtained by any one or more of the following types of transformations: discrete cosine transform (DCT), discrete sine transform (DST), integer transform or discrete wavelet transform (DWT).

[0036] In an optional implementation, the QG includes one or more pixels of the image frame.

[0037] In an optional implementation, multiple QP values ​​of the image frame are determined, including: parsing the code stream to determine one or more QGs marked in the CU; the one or more marked QGs need to be dequantized during the decoding process, and the unmarked QGs in the CU are not dequantized; for each of the one or more marked QGs, the QP value of each QG is obtained.

[0038] Alternatively, determine a scanning order of all QGs included in the one CU; wherein the scanning order includes any one or a combination of the following: from top to bottom, from left to right, zigzag or reverse zigzag order; for each QG of all the QGs, obtain the QP value of each QG according to the scanning order.

[0039] In an optional implementation manner, QP values ​​corresponding to at least two QGs among the multiple QGs are different.

[0040] In a second aspect, the present application provides a method for decoding an image, which can be applied to a video decoding system, or the method can be applied to a decoding end that can support video decoding to implement the method, for example, the decoding end includes a chip system, and the method is executed by the decoding end, and the method includes: parsing a code stream to obtain one or more image frames, an image frame includes one or more CUs; ​​determining multiple QP values ​​of the image frame; wherein a CU includes multiple pixels, one pixel corresponds to a QP value, and the QP values ​​of at least two pixels among the multiple pixels are different; decoding the image frame according to the multiple QP values.

[0041] In an optional implementation, determining multiple QP values ​​of the image frame includes: obtaining a predicted QP value of the pixel point, and determining the QP value of the pixel point based on the predicted QP value of the pixel point and derivation information; wherein the derivation information is information of one or more reconstructed pixels around the pixel point.

[0042] In another optional implementation, the predicted QP value of the pixel point is the QP value of the CU or QG where the pixel point is located, or is derived based on the QP values ​​of one or more reconstructed pixels around the pixel point, wherein the derivation method includes calculating at least one of the mean, median or mode.

[0043] In another optional implementation, the reconstructed pixel point is a pixel point in a square area with a side length of 3 or 5 and centered on the one pixel point, or a pixel point in a diamond area with a diagonal length of 3 or 5.

[0044] In another optional implementation, the information of the reconstructed pixel point includes any one or a combination of the following: pixel value, flatness information or texture information, background brightness, and contrast.

[0045] In another optional implementation, the QP value of the pixel is determined based on the predicted QP value and the derived information of the pixel, including: determining the indication information of the pixel based on the information of one or more reconstructed pixels around the pixel; if the indication information is less than or equal to the first threshold, and the predicted QP value is greater than or equal to the QP value corresponding to the distortion just perceptible to the human eye, then the QP value corresponding to the distortion just perceptible to the human eye is used as the QP value of the pixel. The QP value corresponding to the distortion just perceptible to the human eye is a preset value (such as image-level or CU-level information preset by the encoding end or the decoding end), the QP value corresponding to the distortion just perceptible to the human eye is parsed from the bitstream (image-level or CU-level transmission), or the QP value corresponding to the distortion just perceptible to the human eye is derived based on the flatness information or texture information, background brightness, and contrast information of the surrounding reconstructed CU.

[0046] In a third aspect, the present application provides an image encoding method, which can be applied to a video decoding system, or the method can be applied to an encoding end that can support video decoding to implement the method, for example, the encoding end includes a chip system, and the method is executed by the encoding end, and the method includes: dividing an image frame into one or more CUs; ​​determining multiple QP values ​​of the image frame; wherein a CU includes multiple QGs, and one QG corresponds to one QP value; encoding the image frame according to the multiple QP values.

[0047] In a fourth aspect, the present application provides an image encoding method, which can be applied to a video decoding system, or the method can be applied to an encoding end that can support video decoding to implement the method, for example, the encoding end includes a chip system, and the method is executed by the encoding end, and the method includes: dividing an image frame into one or more CUs; ​​determining multiple QP values ​​of the image frame; wherein a CU includes multiple pixels, one pixel corresponds to a QP value, and the QP values ​​of at least two pixels among the multiple pixels are different; encoding the image frame according to the multiple QP values.

[0048] In a fifth aspect, the present application provides an image decoding device, which is applied to a decoding end, and includes various modules for implementing the method in any possible implementation of the first aspect or the second aspect, such as a code stream parsing unit, a QP decision unit, and an image decoding unit.

[0049] The beneficial effects can be found in the description of any one of the first aspect or the second aspect, which will not be repeated here. The decoding device has the function of implementing the behavior in the method example of any one of the first aspect or the second aspect. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.

[0050] In a sixth aspect, the present application provides an image encoding device, which is applied to an encoding end, and the encoding device includes various modules for implementing the method in any possible implementation of the third aspect or the fourth aspect. For example, the encoding device includes: an image segmentation unit, a QP decision unit, and an image encoding unit. The beneficial effects can be found in the description of any one of the third aspect or the fourth aspect, and will not be repeated here. The encoding device has the function of implementing the behavior in the method instance of any one of the third aspect or the fourth aspect above. The function can be implemented by hardware, or it can be implemented by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.

[0051] In a seventh aspect, the present application provides an electronic device comprising a processor and a memory, wherein the memory is used to store computer instructions, and the processor is used to call and execute the computer instructions from the memory to implement a method of any one of the implementation modes of the first to fourth aspects.

[0052] For example, the electronic device may be a video encoder, or an encoding end including a video encoder.

[0053] For another example, the electronic device may refer to a video decoder, or a decoding end including a video decoder.

[0054] In an eighth aspect, the present application provides a computer-readable storage medium, in which a computer program or instruction is stored. When the computer program or instruction is executed by a computing device or a storage system where the computing device is located, a method of implementing any one of the first to fourth aspects is implemented.

[0055] In a ninth aspect, the present application provides a computer program product, which includes instructions. When the computer program product runs on a computing device or a processor, the computing device or the processor executes the instructions to implement a method according to any one of the implementation methods of the first to fourth aspects.

[0056] In a tenth aspect, the present application provides a chip comprising a memory and a processor, wherein the memory is used to store computer instructions, and the processor is used to call and execute the computer instructions from the memory to implement a method of any one of the implementation modes in the first to fourth aspects.

[0057] In the eleventh aspect, the present application provides a video decoding system, which includes an encoding end and a decoding end, the decoding end is used to implement the method of any one of the implementation methods of the first aspect to the second aspect, and the encoding end is used to implement the method of any one of the implementation methods of the third aspect to the fourth aspect.

[0058] Based on the implementations provided in the above aspects, the present application can also be further combined to provide more implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 An exemplary block diagram of a video decoding system provided by the present application;

[0060] Figure 2 An exemplary block diagram of a video encoder provided by the present application;

[0061] Figure 3 An exemplary block diagram of a video decoder provided by the present application;

[0062] Figure 4 A schematic diagram of a video encoding / decoding process provided by this application;

[0063] Figure 5 A schematic diagram of a flow chart of an image decoding method provided by the present application;

[0064] Figure 6 A schematic diagram of a transform domain partition provided in this application;

[0065] Figure 7 A schematic diagram of pixel domain division provided in this application;

[0066] Figure 8 A schematic diagram of a process for obtaining QP by using a predictive coding method provided in this application;

[0067] Fig. 9 A schematic diagram of pixel distribution provided in this application;

[0068] Fig.10 A schematic diagram of a process of decoding an image frame provided by the present application;

[0069] Fig.11 A schematic diagram of a quantization matrix template provided for this application;

[0070] Fig.12 A schematic diagram of a flow chart of an image encoding method provided by the present application;

[0071] Fig.13 A schematic diagram of the structure of a decoding device provided by the present application;

[0072] Fig.14 A schematic diagram of the structure of an encoding device provided in this application;

[0073] Fig.15 A schematic diagram of the structure of an electronic device provided in this application. DETAILED DESCRIPTION

[0074] In order to make the description of the following embodiments clear and concise, a brief introduction to the related technology is first given.

[0075] Figure 1 An exemplary block diagram of a video decoding system provided for the present application. As used herein, the term "video decoder" generally refers to both a video encoder and a video decoder. In the present application, the term "video decoding" or "decoding" may generally refer to video encoding or video decoding. The video encoder 100 and the video decoder 200 of the video decoding system 1 are used to predict the motion information of the current decoded image block or its sub-block, such as a motion vector, according to various method examples described in any of the multiple new inter-frame prediction modes proposed in the present application, so that the predicted motion vector is as close as possible to the motion vector obtained using the motion estimation method, so that there is no need to transmit the motion vector difference during encoding, thereby further improving the encoding and decoding performance.

[0076] like Figure 1 As shown in , the video decoding system 1 includes an encoding end 10 and a decoding end 20. The encoding end 10 generates encoded video data. Therefore, the encoding end 10 can be referred to as a video encoding device. The decoding end 20 can decode the encoded video data generated by the encoding end 10. Therefore, the decoding end 20 can be referred to as a video decoding device. Various implementations of the encoding end 10, the decoding end 20, or both may include one or more processors and a memory coupled to the one or more processors. The memory may include, but is not limited to, RAM, ROM, EEPROM, flash memory, or any other media that can be used to store desired program code in the form of instructions or data structures that can be accessed by a computer, as described herein.

[0077] The encoding end 10 and the decoding end 20 may include various devices, including desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, telephone handsets such as so-called "smart" phones, televisions, cameras, display devices, digital media players, video game consoles, car computers, or the like.

[0078] The decoding end 20 may receive the encoded video data from the encoding end 10 via the link 30. The link 30 may include one or more media or devices capable of moving the encoded video data from the encoding end 10 to the decoding end 20. In one example, the link 30 may include one or more communication media that enable the encoding end 10 to transmit the encoded video data directly to the decoding end 20 in real time. In this example, the encoding end 10 may modulate the encoded video data according to a communication standard (e.g., a wireless communication protocol), and may transmit the modulated video data to the decoding end 20. The one or more communication media may include wireless and / or wired communication media, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The one or more communication media may form part of a packet-based network, such as a local area network, a wide area network, or a global network (e.g., the Internet). The one or more communication media may include routers, switches, base stations, or other devices that facilitate communication from the encoding end 10 to the decoding end 20.

[0079] In another example, the encoded data may be output from output interface 140 to storage device 40. Similarly, the encoded data may be accessed from storage device 40 via input interface 240. Storage device 40 may include any of a variety of distributed or locally accessed data storage media, such as a hard drive, a Blu-ray disc, a digital versatile disc (DVD), a compact disc read-only memory (CD-ROM), flash memory, volatile or non-volatile memory, or any other suitable digital storage media for storing encoded video data.

[0080] In another example, the storage device 40 may correspond to a file server or another intermediate storage device that can hold the encoded video generated by the encoding end 10. The decoding end 20 can access the stored video data from the storage device 40 via streaming or downloading. The file server can be any type of server capable of storing the encoded video data and transmitting the encoded video data to the decoding end 20. Example file servers include a network server (e.g., for a website), a file transfer protocol (FTP) server, a network attached storage (NAS) device, or a local disk drive. The decoding end 20 can access the encoded video data through any standard data connection (including an Internet connection). This may include a wireless channel (e.g., a wireless fidelity (Wi-Fi) connection), a wired connection (e.g., a digital subscriber line (DSL), a cable modem, etc.), or a combination of the two suitable for accessing the encoded video data stored on the file server. The transmission of the encoded video data from the storage device 40 can be streaming, downloading, or a combination of the two.

[0081] The image decoding method provided by the present application can be applied to video encoding and decoding to support a variety of multimedia applications, such as over-the-air television broadcasting, cable television transmission, satellite television transmission, streaming video transmission (e.g., via the Internet), encoding of video data stored on a data storage medium, decoding of video data stored on a data storage medium, or other applications. In some examples, the video decoding system 1 can be used to support one-way or two-way video transmission to support applications such as video streaming, video playback, video broadcasting and / or video telephony.

[0082] Figure 1 The video decoding system 1 described in is only an example, and the technology of the present application can be applied to video decoding settings (e.g., video encoding or video decoding) that do not necessarily include any data communication between the encoding device and the decoding device. In other examples, data is retrieved from local storage, streamed over a network, etc. A video encoding device may encode data and store the data to a memory, and / or a video decoding device may retrieve data from a memory and decode the data. In many examples, encoding and decoding are performed by devices that do not communicate with each other but only encode data to a memory and / or retrieve data from a memory and decode data.

[0083] exist Figure 1In an example of , the encoding end 10 includes a video source 120, a video encoder 100, and an output interface 140. In some examples, the output interface 140 may include a modulator / demodulator (modem) and / or a transmitter. The video source 120 may include a video capture device (e.g., a camera), a video archive containing previously captured video data, a video feed interface for receiving video data from a video content provider, and / or a computer graphics system for generating video data, or a combination of such sources of video data.

[0084] The video encoder 100 may encode video data from a video source 120. In some examples, the encoding end 10 transmits the encoded video data directly to the decoding end 20 via an output interface 140. In other examples, the encoded video data may also be stored on a storage device 40 for later access by the decoding end 20 for decoding and / or playback.

[0085] exist Figure 1 In the example of , the decoding end 20 includes an input interface 240, a video decoder 200, and a display device 220. In some examples, the input interface 240 includes a receiver and / or a modem. The input interface 240 can receive the encoded video data via the link 30 and / or from the storage device 40. The display device 220 can be integrated with the decoding end 20 or can be external to the decoding end 20. In general, the display device 220 displays the decoded video data. The display device 220 may include a variety of display devices, for example, a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or other types of display devices.

[0086] although Figure 1 100 and 200 may each be integrated with an audio encoder and decoder, and may include appropriate multiplexer-demultiplexer units or other hardware and software to handle the encoding of both audio and video in a common data stream or in separate data streams. In some examples, the demultiplexer (MUX-DEMUX) unit may conform to the ITU H.223 multiplexer protocol, or other protocols such as the user datagram protocol (UDP), if applicable.

[0087] The video encoder 100 and the video decoder 200 may each be implemented as any of a variety of circuits such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, hardware, or any combination thereof. If the present application is implemented in part in software, the device may store instructions for the software in a suitable non-volatile computer-readable storage medium, and may use one or more processors to execute the instructions in hardware to implement the technology of the present application. Any of the foregoing (including hardware, software, a combination of hardware and software, etc.) may be considered as one or more processors. Each of the video encoder 100 and the video decoder 200 may be included in one or more encoders or decoders, any of which may be integrated as part of a combined encoder / decoder (encoder-decoder) in a corresponding device.

[0088] The application may generally refer to the video encoder 100 as "signaling" or "transmitting" certain information to another device, such as the video decoder 200. The term "signaling" or "transmitting" may generally refer to the transmission of syntax elements and / or other data used to decode the compressed video data. This transmission may occur in real time or near real time. Alternatively, this communication may occur over a period of time, such as when the syntax elements are stored in the encoded bitstream to a computer-readable storage medium at the time of encoding, and the decoding device may then retrieve the syntax elements at any time after the syntax elements are stored to such medium.

[0089] JCT-VC developed the H.265 (HEVC) standard. HEVC standardization is based on an evolutionary model of a video decoding device called the HEVC test model (HEVCmodel, HM). The latest standard document of H.265 can be obtained from http: / / www.itu.int / rec / T-REC-H.265, and the latest version of the standard document is H.265 (12 / 16), which is incorporated herein by reference in its entirety. HM assumes that the video decoding device has several additional capabilities relative to the existing algorithm of ITU-TH.264 / AVC. For example, H.264 provides 9 intra-frame prediction coding modes, while HM can provide up to 35 intra-frame prediction coding modes.

[0090] JVET is committed to the development of the H.266 standard. The process of H.266 standardization is based on an evolutionary model of a video decoding device called the H.266 test model. The algorithm description of H.266 is available from http: / / phenix.int-evry.fr / jvet, where the latest algorithm description is contained in JVET-F1001-v2, which is incorporated herein by reference in its entirety. At the same time, the reference software of the JEM test model can be obtained from https: / / jvet.hhi.fraunhofer.de / svn / svn_HMJEMSoftware / , which is also incorporated herein by reference in its entirety.

[0091] In general, the working model of HM describes that a video frame or image can be divided into a sequence of tree blocks or largest coding units (LCUs) that include both luma and chroma samples, and LCUs are also called coding tree units (CTUs). Tree blocks have a similar purpose to macroblocks of the H.264 standard. A slice includes several consecutive tree blocks in decoding order. A video frame or image can be partitioned into one or more slices. Each tree block can be split into coding units (CUs) according to a quadtree. For example, a tree block that is the root node of a quadtree can be split into four child nodes, and each child node can be a parent node and split into another four child nodes. The final unsplittable child nodes that are leaf nodes of the quadtree include decoding nodes, such as decoded video blocks. Syntax data associated with the decoded bitstream can define the maximum number of times a tree block can be split, and can also define the minimum size of a decoding node.

[0092] The size of a CU corresponds to the size of a decoding node and must be square in shape. The size of a CU can range from 8x8 pixels up to a maximum of 64x64 pixels or larger treeblock size.

[0093] A video sequence typically includes a series of video frames or images. A group of pictures (GOP) exemplarily includes a series of one or more video images. A GOP may include syntax data in the header information of the GOP, in the header information of one or more of the images, or elsewhere, and the syntax data describes the number of images included in the GOP. Each slice of the image may include slice syntax data describing the encoding mode of the corresponding image. The video encoder 100 typically operates on video blocks within individual video slices to encode video data. A video block may correspond to a decoding node within a CU. A video block may have a fixed or variable size and may differ in size according to a specified decoding standard.

[0094] In this application, "NxN" and "N by N" may be used interchangeably to refer to the pixel size of a video block in terms of a vertical dimension and a horizontal dimension, e.g., 16x16 pixels or 16 by 16 pixels. In general, a 16x16 block will have 16 pixels in the vertical direction (y=16) and 16 pixels in the horizontal direction (x=16). Likewise, an NxN block generally has N pixels in the vertical direction and N pixels in the horizontal direction, where N represents a non-negative integer value. The pixels in a block may be arranged in rows and columns. Furthermore, a block need not necessarily have the same number of pixels in the horizontal direction as in the vertical direction. For example, a block may include NxM pixels, where M is not necessarily equal to N.

[0095] After intra / inter predictive decoding using a CU, the video encoder 100 may calculate residual data for the CU. The CU may include pixel data in a spatial domain (also referred to as a pixel domain), and the CU may include coefficients in a transform domain after a transform (e.g., a discrete cosine transform (DCT), an integer transform, a discrete wavelet transform, or a conceptually similar transform) is applied to the residual video data. The residual data may correspond to a pixel difference between a pixel of an unencoded image and a prediction value corresponding to the CU. The video encoder 100 may form a CU including the residual data and generate transform coefficients for the CU.

[0096] After any transform to produce transform coefficients, the video encoder 100 may perform quantization of the transform coefficients. Quantization illustratively refers to the process of quantizing the coefficients to possibly reduce the amount of data used to represent the coefficients, thereby providing further compression. The quantization process may reduce the bit depth associated with some or all of the coefficients. For example, an n-bit value may be truncated to an m-bit value during quantization, where n is greater than m.

[0097] In some possible embodiments, the video encoder 100 may scan the quantized transform coefficients using a predefined scanning order to generate a serialized vector that can be entropy encoded. In other possible embodiments, the video encoder 100 may perform adaptive scanning. After scanning the quantized transform coefficients to form a one-dimensional vector, the video encoder 100 may entropy decode the one-dimensional vector according to context-based adaptive variable-length code (CAVLC), context-based adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) decoding or other entropy decoding methods. The video encoder 100 may also entropy encode syntax elements associated with the encoded video data for use by the video decoder 200 to decode the video data.

[0098] To perform CABAC, the video encoder 100 may assign context within a context model to a symbol to be transmitted. The context may be related to whether the adjacent values ​​of the symbol are non-zero. To perform CAVLC, the video encoder 100 may select a variable length code for the symbol to be transmitted. The codewords in variable-length decoding (VLC) may be constructed so that relatively shorter codes correspond to symbols with greater probability, while longer codes correspond to symbols with less probability. In this way, the use of VLC can achieve the purpose of saving code rate relative to using equal length codewords for each symbol to be transmitted. The probability in CABAC can be determined based on the context assigned to the symbol.

[0099] In an embodiment of the present application, the video encoder may perform inter-frame prediction to reduce temporal redundancy between images. The present application may refer to the CU that the video decoder is currently decoding as the current CU. The present application may refer to the image that the video decoder is currently decoding as the current image.

[0100] Figure 21 is an exemplary block diagram of a video encoder provided by the present application. The video encoder 100 is used to output video to a post-processing entity 41. The post-processing entity 41 represents an example of a video entity that can process the encoded video data from the video encoder 100, such as a media aware network element (MANE) or a splicing / editing device. In some cases, the post-processing entity 41 may be an example of a network entity. In some video encoding systems, the post-processing entity 41 and the video encoder 100 may be parts of separate devices, while in other cases, the functionality described with respect to the post-processing entity 41 may be performed by the same device including the video encoder 100. In one example, the post-processing entity 41 is Figure 1 An example of a storage device 40.

[0101] exist Figure 2 In the example of , the video encoder 100 includes a prediction processing unit 108, a filter unit 106, a decoded picture buffer (DPB) 107, a summer 112, a transformer 101, a quantizer 102, and an entropy encoder 103. The prediction processing unit 108 includes an inter-frame predictor 110 and an intra-frame predictor 109. For image block reconstruction, the video encoder 100 also includes an inverse quantizer 104, an inverse transformer 105, and a summer 111. The filter unit 106 is intended to represent one or more loop filters, such as a deblocking filter, an adaptive loop filter (ALF), and a sample adaptive offset (SAO) filter. Although in Figure 2 The filter unit 106 is shown as an in-loop filter in FIG. 1 , but in other implementations, the filter unit 106 may be implemented as a post-loop filter. In one example, the video encoder 100 may further include a video data memory and a segmentation unit (not shown in the figure).

[0102] The video data memory may store video data to be encoded by the components of the video encoder 100. The video data stored in the video data memory may be obtained from the video source 120. The DPB 107 may be a reference image memory that stores reference video data for encoding the video data in the intra-frame and inter-frame decoding modes by the video encoder 100. The video data memory and the DPB 107 may be formed by any of a variety of memory devices, such as a dynamic random access memory (DRAM) including a synchronous dynamic random access memory (SDRAM), a magnetic random access memory (MRAM), a resistive random access memory (RRAM), or other types of memory devices. The video data memory and the DPB 107 may be provided by the same memory device or a separate memory device. In various examples, the video data memory may be on-chip with other components of the video encoder 100, or off-chip relative to those components.

[0103] like Figure 2 As shown, the video encoder 100 receives video data and stores the video data in a video data memory. The segmentation unit segments the video data into a number of image blocks, and these image blocks can be further segmented into smaller blocks, such as image block segmentation based on a quadtree structure or a binary tree structure. This segmentation can also include segmentation into slices, tiles or other larger units. The video encoder 100 generally describes components for encoding image blocks within a video slice to be encoded. The slice can be divided into a plurality of image blocks (and may be divided into a set of image blocks called slices). The prediction processing unit 108 can select one of a plurality of possible decoding modes for the current image block, such as one of a plurality of intra-frame decoding modes or one of a plurality of inter-frame decoding modes. The prediction processing unit 108 can provide the resulting intra-frame and inter-frame decoded blocks to the summer 112 to generate residual blocks, and to the summer 111 to reconstruct the encoded blocks used as reference images.

[0104] The intra-frame predictor 109 within the prediction processing unit 108 may perform intra-frame predictive encoding of the current image block relative to one or more neighboring blocks in the same frame or slice as the current block to be encoded to remove spatial redundancy. The inter-frame predictor 110 within the prediction processing unit 108 may perform inter-frame predictive encoding of the current image block relative to one or more prediction blocks in one or more reference images to remove temporal redundancy.

[0105] Specifically, the inter-frame predictor 110 can be used to determine the inter-frame prediction mode for encoding the current image block. For example, the inter-frame predictor 110 can use rate-distortion analysis to calculate the rate-distortion values ​​of various inter-frame prediction modes in the candidate inter-frame prediction mode set, and select the inter-frame prediction mode with the best rate-distortion characteristics. The rate-distortion analysis generally determines the amount of distortion (or error) between the encoded block and the original unencoded block encoded to generate the encoded block, as well as the bit rate (that is, the number of bits) used to generate the encoded block. For example, the inter-frame predictor 110 can determine that the inter-frame prediction mode with the lowest rate-distortion cost for encoding the current image block in the candidate inter-frame prediction mode set is the inter-frame prediction mode used for inter-frame prediction of the current image block.

[0106] The inter-frame predictor 110 is used to predict the motion information (e.g., motion vector) of one or more sub-blocks in the current image block based on the determined inter-frame prediction mode, and obtain or generate the prediction block of the current image block using the motion information (e.g., motion vector) of one or more sub-blocks in the current image block. The inter-frame predictor 110 can locate the prediction block pointed to by the motion vector in one of the reference image lists. The inter-frame predictor 110 can also generate syntax elements associated with the image block and the video slice for use by the video decoder 200 when decoding the image block of the video slice. Alternatively, in one example, the inter-frame predictor 110 performs a motion compensation process using the motion information of each sub-block to generate a prediction block for each sub-block, thereby obtaining a prediction block for the current image block; it should be understood that the inter-frame predictor 110 here performs motion estimation and motion compensation processes.

[0107] Specifically, after selecting the inter prediction mode for the current image block, the inter predictor 110 may provide information indicating the selected inter prediction mode of the current image block to the entropy encoder 103 so that the entropy encoder 103 encodes the information indicating the selected inter prediction mode.

[0108] The intra-frame predictor 109 may perform intra-frame prediction on the current image block. Specifically, the intra-frame predictor 109 may determine an intra-frame prediction mode used to encode the current block. For example, the intra-frame predictor 109 may use rate-distortion analysis to calculate rate-distortion values ​​for various intra-frame prediction modes to be tested, and select an intra-frame prediction mode with the best rate-distortion characteristics from the modes to be tested. In any case, after selecting an intra-frame prediction mode for the image block, the intra-frame predictor 109 may provide information indicating the selected intra-frame prediction mode of the current image block to the entropy encoder 103 so that the entropy encoder 103 encodes the information indicating the selected intra-frame prediction mode.

[0109] After the prediction processing unit 108 generates a prediction block for the current image block via inter-frame prediction and intra-frame prediction, the video encoder 100 forms a residual image block by subtracting the prediction block from the current image block to be encoded. The summer 112 represents one or more components that perform this subtraction operation. The residual video data in the residual block may be contained in one or more (transform unit, TU) and applied to the transformer 101. The transformer 101 transforms the residual video data into residual transform coefficients using a transform such as a discrete cosine transform (DCT) or a conceptually similar transform. The transformer 101 can convert the residual video data from a pixel value domain to a transform domain, such as a frequency domain.

[0110] The transformer 101 may send the resulting transform coefficients to the quantizer 102. The quantizer 102 quantizes the transform coefficients to further reduce the bit rate. In some examples, the quantizer 102 may then perform a scan of the matrix including the quantized transform coefficients. Alternatively, the entropy encoder 103 may perform the scan.

[0111] After quantization, the entropy encoder 103 performs entropy encoding on the quantized transform coefficients. For example, the entropy encoder 103 may perform context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), syntax-based context adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or another entropy coding method or technique. After entropy encoding by the entropy encoder 103, the encoded bitstream may be transmitted to the video decoder 200, or archived for later transmission or retrieval by the video decoder 200. The entropy encoder 103 may also perform entropy encoding on syntax elements of the current image block to be encoded.

[0112] The inverse quantizer 104 and the inverse transformer 105 apply inverse quantization and inverse transformation, respectively, to reconstruct the residual block in the pixel domain, for example, for later use as a reference block of a reference image. The summer 111 adds the reconstructed residual block to the prediction block generated by the inter-frame predictor 110 or the intra-frame predictor 109 to generate a reconstructed image block. The filter unit 106 can be applied to the reconstructed image block to reduce distortion, such as block artifacts. The reconstructed image block is then stored as a reference block in the decoded image buffer 107 and can be used by the inter-frame predictor 110 as a reference block to perform inter-frame prediction on blocks in subsequent video frames or images.

[0113] It should be understood that other structural changes of the video encoder 100 can be used to encode the video stream. For example, for some image blocks or image frames, the video encoder 100 can directly quantize the residual signal without being processed by the transformer 101, and accordingly, it does not need to be processed by the inverse transformer 105; or, for some image blocks or image frames, the video encoder 100 does not generate residual data, and accordingly, it does not need to be processed by the transformer 101, the quantizer 102, the inverse quantizer 104 and the inverse transformer 105; or, the video encoder 100 can directly store the reconstructed image block as a reference block without being processed by the filter unit 106; or, the quantizer 102 and the inverse quantizer 104 in the video encoder 100 can be combined together.

[0114] Figure 3 An exemplary block diagram of a video decoder 200 provided in the present application. Figure 3 In an example of FIG. 1 , the video decoder 200 includes an entropy decoder 203, a prediction processing unit 208, an inverse quantizer 204, an inverse transformer 205, a summer 211, a filter unit 206, and a DPB 207. The prediction processing unit 208 may include an inter-frame predictor 210 and an intra-frame predictor 209. In some examples, the video decoder 200 may perform substantially the same Figure 2 The encoding process described in video encoder 100 is the inverse of the decoding process.

[0115] During the decoding process, the video decoder 200 receives an encoded video code stream representing image blocks and associated syntax elements of an encoded video slice from the video encoder 100. The video decoder 200 can receive video data from the network entity 42, and optionally, can also store the video data in a video data memory (not shown in the figure). The video data memory can store video data to be decoded by the components of the video decoder 200, such as an encoded video code stream. The video data stored in the video data memory can be obtained, for example, from the storage device 40, from a local video source such as a camera, via a wired or wireless network communication of the video data, or by accessing a physical data storage medium. The video data memory can be used as a decoded picture buffer (CPB) for storing encoded video data from an encoded video code stream. Therefore, although in Figure 3The video data memory is not shown in the figure, but the video data memory and DPB 207 can be the same memory or separately arranged. The video data memory and DPB 207 can be formed by any of a variety of memory devices, such as dynamic random access memory (DRAM) including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. In various examples, the video data memory can be integrated on the chip with other components of the video decoder 200, or can be arranged off the chip relative to those components.

[0116] The network entity 42 may be a server, a MANE, a video editor / splicer, or other device for implementing one or more of the techniques described above. The network entity 42 may or may not include a video encoder, such as the video encoder 100. Before the network entity 42 sends the encoded video code stream to the video decoder 200, the network entity 42 may implement some of the techniques described in this application. In some video decoding systems, the network entity 42 and the video decoder 200 may be parts of separate devices, while in other cases, the functionality described with respect to the network entity 42 may be performed by the same device including the video decoder 200. In some cases, the network entity 42 may be Figure 1 An example of a storage device 40.

[0117] The entropy decoder 203 of the video decoder 200 entropy decodes the code stream to generate quantized coefficients and some syntax elements. The entropy decoder 203 forwards the syntax elements to the prediction processing unit 208. The video decoder 200 may receive syntax elements at the video slice level and / or the image block level.

[0118] When the video slice is decoded as an intra-frame decoded (I) slice, the intra-frame predictor 209 of the prediction processing unit 208 may generate a prediction block of an image block of the current video slice based on the signaled intra-frame prediction mode and data from a previously decoded block of the current frame or picture. When the video slice is decoded as an inter-frame decoded (i.e., B or P) slice, the inter-frame predictor 210 of the prediction processing unit 208 may determine an inter-frame prediction mode for decoding a current image block of the current video slice based on syntax elements received from the entropy decoder 203, and decode (e.g., perform inter-frame prediction) on the current image block based on the determined inter-frame prediction mode. Specifically, the inter-frame predictor 210 may determine whether to use a new inter-frame prediction mode to predict the current image block of the current video strip. If the syntax element indicates that the new inter-frame prediction mode is used to predict the current image block, the motion information of the current image block or the sub-block of the current image block of the current video strip is predicted based on the new inter-frame prediction mode (for example, a new inter-frame prediction mode specified by the syntax element or a default new inter-frame prediction mode), so as to obtain or generate a prediction block of the current image block or the sub-block of the current image block by using the predicted motion information of the current image block or the sub-block of the current image block through the motion compensation process. The motion information here may include reference image information and motion vectors, wherein the reference image information may include but is not limited to unidirectional / bidirectional prediction information, reference image list number and reference image index corresponding to the reference image list. For inter-frame prediction, a prediction block may be generated from one of the reference images in one of the reference image lists. The video decoder 200 may construct reference image lists, i.e., list 0 and list 1, based on the reference images stored in the DPB 207. The reference frame index of the current image may be included in one or more of the reference frame list 0 and the list 1. In some examples, the video encoder 100 may signal a specific syntax element indicating whether a new inter-frame prediction mode is used to decode a specific block, or may signal a specific syntax element indicating whether a new inter-frame prediction mode is used and indicating which new inter-frame prediction mode is used to decode a specific block. It should be understood that the inter-frame predictor 210 here performs a motion compensation process.

[0119] The inverse quantizer 204 inversely quantizes, i.e., dequantizes, the quantized transform coefficients provided in the bitstream and decoded by the entropy decoder 203. The inverse quantization process may include using the quantization parameters calculated by the video encoder 100 for each image block in the video slice to determine the degree of quantization to be applied and likewise the degree of inverse quantization to be applied. The inverse transformer 205 applies an inverse transform to the transform coefficients, such as an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process, to generate a residual block in the pixel domain.

[0120] After the inter-frame predictor 210 generates a prediction block for the current image block or a sub-block of the current image block, the video decoder 200 obtains a reconstructed block, i.e., a decoded image block, by summing the residual block from the inverse transformer 205 with the corresponding prediction block generated by the inter-frame predictor 210. The summer 211 represents a component that performs this summing operation. If necessary, a loop filter (in the decoding loop or after the decoding loop) may also be used to smooth pixel transitions or otherwise improve video quality. The filter unit 206 may represent one or more loop filters, such as a deblocking filter, an adaptive loop filter (ALF), and a sample adaptive offset (SAO) filter. Although in Figure 3 206 is shown as an in-loop filter, but in other implementations, the filter unit 206 may be implemented as a post-loop filter. In one example, the filter unit 206 is adapted to reconstruct the block to reduce block distortion, and the result is output as a decoded video stream. Furthermore, the decoded image blocks in a given frame or image may also be stored in a decoded image buffer 207, and the DPB 207 stores reference images for subsequent motion compensation. The DPB 207 may be part of a memory that may also store the decoded video for later display on a display device (e.g., Figure 1 The memory may be presented on a display device 220) or may be separate from such memory.

[0121] It should be understood that other structural changes of the video decoder 200 can be used to decode the encoded video code stream. For example, the video decoder 200 can generate an output video stream without being processed by the filter unit 206; or, for some image blocks or image frames, the entropy decoder 203 of the video decoder 200 does not decode the quantized coefficients, and accordingly does not need to be processed by the inverse quantizer 204 and the inverse transformer 205.

[0122] The techniques of the present application may be performed by any of the video encoders or video decoders described in the present application, such as Figures 1 to 3 The video encoder 100 and the video decoder 200 shown and described. That is, in one possible implementation, Figure 2 The described video encoder 100 may perform the specific techniques described below when performing inter-frame prediction during encoding of a block of video data. Figure 3 The described video decoder 200 may perform the specific techniques described below when performing inter-prediction during decoding of a block of video data. Thus, references to a general "video encoder" or "video decoder" may include the video encoder 100, the video decoder 200, or another video encoding or decoding unit.

[0123] Figure 1 to Figure 3These are merely examples provided in the present application. In some examples, the video encoder 100, the video decoder 200, and the video decoding system may include more or fewer components or units, and the present application is not limited to this.

[0124] Below Figure 1 to Figure 3 Based on the video decoding system shown, the present application provides a possible video encoding / decoding implementation, such as Figure 4 As shown, Figure 4 A schematic diagram of a video encoding / decoding process provided in the present application, wherein the video encoding / decoding implementation method includes process ① to process ⑤, and process ① to process ⑤ can be executed by any one or more of the above-mentioned encoding end 10, video decoder 100, decoding end 20 or video decoder 200.

[0125] Process ①: Divide a frame of image into one or more non-overlapping parallel coding units. The one or more parallel coding units have no dependencies and can be completely parallel / independently encoded and decoded, such as Figure 4 A parallel encoding unit 1 and a parallel encoding unit 2 are shown.

[0126] Process ②: Each parallel coding unit can be further divided into one or more non-overlapping independent coding units. The independent coding units may not depend on each other, but may share some parallel coding unit header information.

[0127] For example, the width of an independent coding unit is w_lcu and the height is h_lcu. If a parallel coding unit is divided into an independent coding unit, the size of the independent coding unit is exactly the same as the parallel coding unit; otherwise, the width of the independent coding unit should be greater than the height (unless it is an edge area).

[0128] Typically, the independent coding unit may be a fixed w_lcu×h_lcu, where w_lcu and h_lcu are both 2 to the power of N (N≥0), such as the size of the independent coding unit is 128×4, 64×4, 32×4, 16×4, 8×4, 32×2, 16×2 or 8×2.

[0129] As a possible example, the independent coding unit may be fixed to 128×4. If the size of the parallel coding unit is 256×8, the parallel coding unit may be equally divided into 4 independent coding units; if the size of the parallel coding unit is 288×10, the parallel coding unit is divided into: the first / second row is 2 128×4 + 1 32×4 independent coding units; the third row is 2 128×2 + 1 32×2 independent coding units.

[0130] It is worth noting that the independent coding unit can include three components of brightness Y, chrominance Cb, and chrominance Cr, or three components of red (red, R), green (green, G), and blue (blue, B), or only one of them. If the independent coding unit contains three components, the sizes of the three components can be exactly the same or different, which is related to the input format of the image.

[0131] Process ③: Each independent coding unit can be further divided into one or more non-overlapping coding units. The coding units within the independent coding unit can be interdependent, such as multiple coding units can perform mutual reference pre-coding and decoding.

[0132] If the coding unit and the independent coding unit have the same size (ie, the independent coding unit is divided into only one coding unit), the size thereof can be any size described in process ②.

[0133] If the independent coding unit is divided into multiple non-overlapping coding units, feasible division examples include: horizontal equal division (the height of the coding unit is the same as that of the independent coding unit, but the width is different, which can be 1 / 2, 1 / 4, 1 / 8, 1 / 16, etc.), vertical equal division (the width of the coding unit is the same as that of the independent coding unit, but the height is different, which can be 1 / 2, 1 / 4, 1 / 8, 1 / 16, etc.), horizontal and vertical equal division (quadtree division), etc., preferably horizontal equal division.

[0134] The width of the coding unit is w_cu and the height is h_cu, so its width should be greater than its height (unless it is an edge area). Usually, the coding unit can be a fixed w_cu x h_cu, where w_cu and h_cu are both 2N powers (N is greater than or equal to 0), such as 16x4, 8x4, 16x2, 8x2, 8x1, 4x1, etc.

[0135] As a possible example, the coding unit may be a fixed 16x4. If the size of the independent coding unit is 64x4, the independent coding unit may be equally divided into 4 coding units; if the size of the independent coding unit is 72x4, the coding unit may be divided into: 4 16x4 + 1 8x4.

[0136] It is worth noting that the coding unit can include three components of brightness Y, chrominance Cb, and chrominance Cr (or three components of red R, green G, and blue B), or only one of them. If it includes three components, the sizes of the components can be exactly the same or different, which is related to the image input format.

[0137] It is worth noting that process ③ is an optional step in the video coding and decoding method, and the video codec can encode / decode the residual coefficients (or residual values) of the independent coding units obtained in process ②.

[0138] Process ④: For the coding unit, it can be further divided into one or more non-overlapping prediction groups (Prediction Group, PG), PG can also be referred to as Group, each PG is encoded and decoded according to the selected prediction mode to obtain the prediction value of PG, which constitutes the prediction value of the entire coding unit, and the residual value of the coding unit is obtained based on the prediction value and the original value of the coding unit.

[0139] Process ⑤: Based on the residual values ​​of the coding units, the coding units are grouped to obtain one or more non-overlapping residual blocks (RBs). The residual coefficients of each RB are encoded and decoded according to the selected mode to form a residual coefficient stream. Specifically, it can be divided into two categories: transforming the residual coefficients and not transforming them.

[0140] Among them, the selected mode of the residual coefficient encoding and decoding method in process ⑤ may include, but is not limited to any of the following: semi-fixed length coding method, exponential Golomb coding method, Golomb-Rice coding method, truncated unary code coding method, run-length coding method, direct encoding of the original residual value, etc.

[0141] For example, a video encoder may directly encode coefficients within an RB.

[0142] For another example, the video encoder may also transform the residual block, such as DCT, DST, Hadamard transform, etc., and then encode the transformed coefficients.

[0143] As a possible example, when RB is small, the video encoder can directly perform uniform quantization on each coefficient in RB, and then perform binary encoding. If RB is large, it can be further divided into multiple coefficient groups (coefficient groups, CG), and then each CG is uniformly quantized and then binary encoded. In some embodiments of the present application, the coefficient group (CG) and the quantization group (QG) can be the same.

[0144] The following is an exemplary explanation of the residual coefficient encoding part in a semi-fixed length coding method. First, the maximum value of the absolute value of the residual in an RB block is defined as the modified maximum value (modified maximum, mm). Secondly, the number of coding bits for the residual coefficient in the RB block is determined (the number of coding bits for the residual coefficient in the same RB block is consistent). For example, if the critical limit (CL) of the current RB block is 2 and the current residual coefficient is 1, then 2 bits are required to encode the residual coefficient 1, which is expressed as 01. If the CL of the current RB block is 7, it means encoding an 8-bit residual coefficient and a 1-bit sign bit. The determination of CL is to find the minimum M value that satisfies that all residuals of the current sub-block are within the range of [-2^(M-1), 2^(M-1)]. If both -2^(M-1) and 2^(M-1) exist, M should be increased by 1, that is, M+1 bits are required to encode all residuals of the current RB block; if only one of the two boundary values ​​​​of -2^(M-1) and 2^(M-1) exists, a Trailing bit needs to be encoded to determine whether the boundary value is -2^(M-1) or 2^(M-1); if all residuals do not exist in either -2^(M-1) or 2^(M-1), there is no need to encode the Trailing bit.

[0145] In addition, for some special cases, the video encoder can also directly encode the original value of the image instead of the residual value.

[0146] The implementation of the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0147] Figure 5 A schematic diagram of a flow chart of an image decoding method provided in the present application, the image decoding method can be applied to Figure 1 In the video decoding system shown in FIG. 1 , the image decoding method can be executed by the decoding end 20. Specifically, the decoding method can be executed by the video decoder 200 included in the decoding end 20. Figure 5 , the decoding method provided in this embodiment includes the following steps.

[0148] S510: The video decoder 200 parses the acquired code stream to obtain one or more image frames.

[0149] The above-mentioned one image frame includes one or more CUs.

[0150] like Figure 5 As shown, one image frame may include 3×5=15 CUs. It is worth noting that one image frame may also include more CUs, such as 20 CUs; ​​one image frame may also include fewer CUs, such as 1 CU or 2 CUs.

[0151] S520 , the video decoder 200 determines a plurality of QP values ​​for an image frame.

[0152] Among them, a CU includes multiple QGs, and one QG corresponds to one QP value.

[0153] The aforementioned CU includes a plurality of residual coefficients, and the QG includes a portion of the residual coefficients among the aforementioned plurality of residual coefficients, and the portion of the residual coefficients shares a QP value.

[0154] It is worth noting that during the encoding process, after the video encoder quantizes the residual coefficient, the residual coefficient is also called a horizontal value. In the art, the residual coefficient and the horizontal value may also be collectively referred to as a residual coefficient. This embodiment is expressed as a residual coefficient, which should not be understood as a limitation on the present application.

[0155] Compared with the larger image distortion caused by using the same QP value for all residual coefficients in a CU, in the decoding process of the video image provided in this embodiment, a CU can be divided into multiple QGs, and one or more residual coefficients in each QG share a QP value. Then, the video decoder can make a more fine-grained QP decision for one or more CUs corresponding to the image frame, thereby reducing the decoding distortion of the image frame while ensuring a certain compression rate, and improving the authenticity and accuracy of video image decoding.

[0156] In an optional implementation, a QG may include one pixel or multiple pixels, and each pixel has a corresponding residual coefficient.

[0157] In the first possible scenario, when a QG includes a pixel, the quantization process of the QG can also be called adaptive point-by-point quantization of the image frame, and the point-by-point quantization can also be applied to CUs without QG division. The characteristic of point-by-point quantization is that it allows each pixel to use a different QP, which is equivalent to refining the granularity of the QP to the pixel level. In this way, the image frame is inversely quantized using the point-by-point quantization method, which can greatly improve the subjective quality of the image frame without changing the compression rate of the image frame.

[0158] In a second possible scenario, when a QG includes multiple pixel points, residual coefficients corresponding to the multiple pixel points included in the QG may share a QP value.

[0159] In another optional implementation, the CU may be determined in the following manner: the video decoder 200 divides a CU included in an image frame according to a first rule to obtain multiple QGs.

[0160] The first rule includes a division domain and a division method, the division domain is a transform domain or a pixel domain, and the division method includes at least one of uniform division and non-uniform division.

[0161] The positions of multiple residual coefficients contained in a CU are marked by coordinates, which may include a horizontal coordinate and a vertical coordinate. For example, the position coordinates of the residual coefficient are (i, j), i is the horizontal coordinate, and j is the vertical coordinate. Several possible examples are provided below to illustrate the first rule and the division of QG in CU.

[0162] In a first possible example, if the type of the division domain is a transform domain, the process of the video decoder 200 dividing a CU to obtain multiple QGs includes: dividing the residual coefficients whose coordinate sums do not reach the first coordinate threshold among the multiple residual coefficients into the first QG, and dividing the residual coefficients whose coordinate sums reach the first coordinate threshold into the second QG. The coordinate sum is the sum of the horizontal coordinate and the vertical coordinate of the residual coefficient.

[0163] Figure 6 A schematic diagram of transform domain division provided for the present application, wherein a CU includes areas corresponding to 16 residual coefficients, wherein the coordinates of the residual coefficient in the upper left corner are (1,1), and the coordinates of the residual coefficient in the lower right corner are (4,4).

[0164] QG (2-1) shows a QG binary method in the transform domain. For the residual coefficient at the (i, j) position, the residual coefficient that satisfies "i+j≤threshold 1" is the first QG, and the others are the second QG, such as the threshold 1 is 5.5. In addition, the residual coefficient that satisfies "i+j<threshold 1" can also be the first QG, and the others can be the second QG.

[0165] In a second possible example, if the type of the division domain is a transform domain, the process of the video decoder 200 dividing a CU to obtain multiple QGs includes: among the multiple residual coefficients, the residual coefficients whose coordinates and do not reach the first coordinate threshold are divided into the first QG, the residual coefficients whose coordinates and reach the first coordinate threshold and do not reach the second coordinate threshold are divided into the second QG, and the residual coefficients whose coordinates and reach the second coordinate threshold are divided into the third QG. The second coordinate threshold is greater than the first coordinate threshold.

[0166] like Figure 6 As shown, QG (3-1) shows a QG three-part method in the transform domain. For the residual coefficient at the (i, j) position, the residual coefficient that satisfies "i+j<threshold 1" is the first QG, the residual coefficient that satisfies "threshold 1≤i+j<threshold 2" is the second QG, and the residual coefficient that satisfies "i+j≥threshold 2" is the third QG. Threshold 2 is greater than threshold 1, such as threshold 1=5.5, threshold 2=6.5.

[0167] In addition, the residual coefficient satisfying "i+j≤threshold 1" can be the first QG, the residual coefficient satisfying "threshold 1<i+j≤threshold 2" can be the second QG, and the residual coefficient satisfying "i+j>threshold 2" can be the third QG, where threshold 2 is greater than threshold 1, such as threshold 1=5 and threshold 2=6.

[0168] In a third possible example, if the type of the partition domain is a transform domain, the process of the video decoder 200 partitioning a CU to obtain multiple QGs includes: sorting multiple residual coefficients, and sorting the residual coefficients that do not reach the first ratio threshold among the multiple residual coefficients into the first QG, and the residual coefficients that reach the first ratio threshold are sorted into the second QG. The sorting method of the multiple residual coefficients is any one of the following: zigzag, reverse zigzag.

[0169] For example, Figure 6 As shown, QG (2-2) shows a zigzag QG binary division method in the transform domain, the first 7 / 16 (43.75%) residual coefficients are divided into the first QG, and the remaining residual coefficients are divided into the second QG.

[0170] For example, Figure 6 As shown, QG (2-3) shows an inverse zigzag QG binary division method in the transform domain, the first 7 / 16 (43.75%) residual coefficients are divided into the first QG, and the remaining residual coefficients are divided into the second QG.

[0171] In a fourth possible example, if the type of the partition domain is a transform domain, the process of the video decoder 200 partitioning a CU to obtain multiple QGs includes: sorting multiple residual coefficients, and dividing the residual coefficients that do not reach the first proportional threshold among the multiple residual coefficients into the first QG, the residual coefficients that reach the first proportional threshold but do not reach the second proportional threshold into the second QG, and the residual coefficients that reach the second proportional threshold into the third QG. The second proportional threshold is greater than the first proportional threshold.

[0172] For example, Figure 6 As shown, QG (3-2) shows a Z-shaped QG three-part method in the transform domain, the first 5 / 16 (31.25%) of the residual coefficients are divided into the first QG, the last 3 / 16 (18.75%) of the residual coefficients are divided into the second QG, and the remaining residual coefficients are divided into the third QG.

[0173] For example, Figure 6 As shown, QG (3-3) shows an inverse Z-shaped QG three-part method in the transform domain, the first 5 / 16 (31.25%) of the residual coefficients are divided into the first QG, the last 3 / 16 (18.75%) of the residual coefficients are divided into the second QG, and the remaining residual coefficients are divided into the third QG.

[0174] The first to fourth possible examples are only examples given in this embodiment to illustrate the division of the transform domain in the CU, and should not be understood as limiting the present application. When the type of the division domain is the transform domain, the selection of the coordinate threshold and the ratio threshold can be determined according to the image content of the CU or the requirements of the video encoding and decoding, and the present application does not limit this. In addition, in order to improve the accuracy of QP decision-making within a CU, the transform domain can also be divided into more QGs, such as 4, 5, 10 or more, and the present application does not limit this.

[0175] When the type of the division domain is the pixel domain, several possible examples are provided below to illustrate the first rule and the division of the QG in the CU.

[0176] In a fifth possible example, the video decoder 200 divides the multiple residual coefficients symmetrically in the horizontal or vertical direction to obtain two QGs containing the same number of residual coefficients. The ratio of the residual coefficients contained in the two QGs is 1:1.

[0177] Figure 7 A schematic diagram of pixel domain division is provided for the present application, QG_pixel (pixel, P) (2-1) shows an example of symmetric binary division in the vertical direction of the CU, and QG_P (2-4) shows an example of symmetric binary division in the horizontal direction of the CU.

[0178] In a sixth possible example, the video decoder 200 divides the multiple residual coefficients symmetrically in the horizontal or vertical direction to obtain three QGs. The number of residual coefficients contained in two non-adjacent QGs of the three QGs is consistent, and the number of residual coefficients contained in another QG is consistent with the sum of the number of residual coefficients contained in the two non-adjacent QGs.

[0179] like Figure 7 As shown, QG_P(3-1) shows an example of symmetrical three-division in the vertical direction of the CU, the QGs on both sides contain the same number of residual coefficients, and the ratio of the number of residual coefficients contained in these three QGs is 1:2:1. QG_P(3-4) shows an example of symmetrical three-division in the vertical direction of the CU, and the ratio of the number of residual coefficients contained in these three QGs is 1:2:1.

[0180] In a seventh possible example, the video decoder 200 divides the multiple residual coefficients horizontally or vertically to obtain two QGs containing different numbers of residual coefficients.

[0181] For example, Figure 7 The QG_P(2-2) shown gives an example of performing a binary division in the vertical direction of the CU, and the ratio of the residual coefficients contained in the two QGs is 1:3.

[0182] For another example, QG_P(2-3) gives an example of performing binary division in the vertical direction of the CU, and the ratio of the residual coefficients contained in these two QGs is 3:1.

[0183] For another example, QG_P(2-5) gives an example of binary division in the horizontal direction of the CU, and the ratio of the residual coefficients contained in these two QGs is 1:3.

[0184] For another example, QG_P(2-6) gives an example of performing binary division in the vertical direction of the CU, and the ratio of the residual coefficients contained in these two QGs is 3:1.

[0185] In an eighth possible example, the video decoder 200 divides the multiple residual coefficients horizontally or vertically to obtain three QGs. The residual coefficients included in the three QGs do not have a symmetric relationship.

[0186] For example, Figure 7 The shown QG_P(3-2) gives an example of trisection in the vertical direction of the CU, and the ratio of the residual coefficients contained in these three QGs is 1:1:2.

[0187] For another example, QG_P(3-3) gives an example of trisection in the vertical direction of the CU, and the ratio of the residual coefficients contained in these three QGs is 2:1:1.

[0188] For another example, QG_P(3-5) gives an example of trisection in the horizontal direction of the CU, and the ratio of the residual coefficients contained in these three QGs is 1:1:2.

[0189] For another example, QG_P(3-6) gives an example of trisection in the vertical direction of the CU, and the ratio of the residual coefficients contained in these three QGs is 2:1:1.

[0190] The fifth to eighth possible examples are only examples given in this embodiment to illustrate the division of the pixel domain in the CU, and should not be understood as limiting the present application. When the type of the division domain is the pixel domain, the proportion of the residual coefficients contained in the QG can be determined according to the image content of the CU or the requirements of the video codec, and the present application does not limit this. In addition, in order to improve the accuracy of QP decision-making within a CU, the pixel domain can also be divided into more QGs, such as 4, 5, 10 or more, and the present application does not limit this.

[0191] In this way, the video decoder 200 uses different QG division methods for the residual coefficients in the pixel domain and the transform domain, and the QP quantization process is also different, thereby reducing the distortion of image decoding. For the QP quantization process, please refer to the description of the QP value acquisition process in this article, which will not be repeated here.

[0192] Please continue to see Figure 5 After the video decoder 200 obtains multiple QP values ​​of an image frame, the image decoding method provided by this embodiment further includes the following step S530.

[0193] S530 : The video decoder 200 decodes an image frame according to multiple QP values.

[0194] For example, after the video decoder 200 decodes an image frame according to multiple QP values, a decoded image is obtained. After the video decoder 200 decodes multiple image frames, a video after the code stream is decoded is obtained.

[0195] Thus, in this embodiment, the video decoder can perform QP quantization for image decoding at a granularity of QG. Since a CU can be divided into multiple QGs, and one QG corresponds to one QP value, the image distortion is greater than that caused by using the same QP value for all residual coefficients in a CU. In this embodiment, the video decoder can make a finer-grained QP decision for one or more CUs corresponding to the image frame, thereby reducing the decoding distortion of the image frame while ensuring a certain compression rate, and improving the authenticity and accuracy of video image decoding.

[0196] Based on the technology provided in this application, during the video encoding / decoding process, the video encoder / video decoder will obtain the QP value of each QG (or CU), and several possible implementation methods are provided here.

[0197] In a possible implementation, a QP value corresponding to a QG includes a luminance QP value and a chrominance QP value. The luminance QP value (QP_Y) refers to the QP value required for quantizing or dequantizing the luminance (Luminance or Luma) of the image frame, and the chrominance QP value refers to the QP value required for quantizing or dequantizing the chrominance (Chrominance or Chroma) of the image frame. For example, with respect to the above S520, the video decoder 200 determines multiple QP values ​​for an image frame, which may include the following two possible situations.

[0198] In a first possible scenario, the video decoder 200 obtains a luma QP value and a chroma QP value of a QG respectively.

[0199] In the second possible scenario, first, the video decoder 200 obtains a luma QP value of a QG; second, the video decoder 200 determines a chroma QP value of a QG based on the luma QP value. For example, the chroma QP is QP_Y plus the QP offset values ​​of the picture parameter set (PPS) layer and the slice layer.

[0200] For example, the above-mentioned luminance QP value and chrominance QP value may be obtained by parsing the bitstream by the video decoder 200 .

[0201] When performing QP decision (or QP quantization) on the QG included in the CU, the video decoder 200 may first obtain the QP value of the CU level, and then obtain the QP value of the QG level in the CU.

[0202] Two possible implementations are provided below to determine multiple QP values ​​for an image frame.

[0203] A first possible implementation is direct encoding / decoding: the video decoder 200 parses the bitstream to obtain marking information of an image frame, where the marking information is used to indicate a QP value of a QG, and / or the marking information is used to indicate a QP value of a CU.

[0204] For example, in near-lossless compression, the probability of a small QP value occurring is higher than that of a large QP value, so the video decoder 200 may use truncated unary code, truncated Rice code, or exponential Golomb code, etc., and use the QP value included in the mark information carried by the bitstream to dequantize the residual coefficient (horizontal value) included in QG.

[0205] In this way, when the video encoder 200 makes a QP decision for a CU, the tag information may carry the QP value of the CU. Without making a QP decision for the QG in the CU, the video decoder 200 may determine the QP value of the CU based on the tag information carried by the bitstream, thereby avoiding the video decoder from deriving the image frames in the bitstream to obtain the QP value of the CU, reducing the computing resource consumption of the video decoder and improving the efficiency of image decoding.

[0206] For another example, when the video encoder 200 makes a QP decision for multiple QGs in a CU, the marking information may carry the QP value of any one of the multiple QGs.

[0207] For another example, the marking information may carry the QP value of the CU and the QP value of any QG in the CU.

[0208] In this way, when the video encoder 200 makes a QP decision for a QG, the marking information can carry the QP value of the QG, avoiding the video decoder from deriving the image frames in the bitstream to obtain the QP value of the QG, reducing the computing resource consumption of the video decoder and improving the efficiency of image decoding.

[0209] A second possible implementation method is predictive coding / decoding: the video decoder 200 encodes the difference (deltaQP) between the actual (coded) QP value and the predicted QP (predict QP, predQP) value, including the following steps: first, obtain the predQP of the current block (CU or QG); second, determine the deltaQP of the current block; finally, determine the actual (coded) QP value as: QP = predQP + deltaQP.

[0210] The following is a detailed description using the example of the video decoder 200 determining the QP value of a QG in an image frame. Figure 8 As shown, Figure 8 The present application provides a flow chart of obtaining QP by using predictive coding. The process of obtaining QP can be implemented by a video decoder or a video encoder. Here, the video decoder 200 is taken as an example for explanation. The process of obtaining QP includes the following steps.

[0211] S810 , the video decoder 200 obtains a predicted QP value of QG.

[0212] In a feasible implementation, the video decoder 200 uses the QP value of the CU where the QG is located as the predicted QP value of the QG. For example, the QP value of the CU can be determined by parsing the mark information of the bitstream by the video decoder 200.

[0213] In another feasible implementation, the video decoder 200 may first obtain the QP value of at least one other QG adjacent to the QG in the CU where the QG is located, and determine the predicted QP value of the aforementioned QG according to the QP value of the at least one other QG. For example, the video decoder 200 uses the QP values ​​of other QGs adjacent to the QG as the predicted QP value of the QG.

[0214] S820 , the video decoder 200 obtains the QP offset of QG.

[0215] In this paper, the QP offset can be expressed as deltaQP.

[0216] Please continue to see Figure 8 , with respect to S820, this embodiment provides two possible implementation methods.

[0217] In a first optional implementation, the video decoder 200 may use the marking information carried by the bitstream to determine the QP offset of the QG, such as Figure 8 S820A shown.

[0218] S820A: The video decoder 200 parses the bitstream to obtain flag information indicating a QP offset of the QG.

[0219] For example, after acquiring the bitstream, the video decoder 200 parses the bitstream to obtain the marking information of the image frame where the QG is located, and the marking information is used to indicate the QP offset (deltaQP) of the QG.

[0220] In a second optional implementation, the video decoder 200 may use the derived information to determine the QP offset of QG, such as Figure 8 S820B shown.

[0221] S820B: The video decoder 200 determines the QP offset of QG according to the derivation information of QG.

[0222] The derived information may be any one or a combination of the following: flatness information or texture information of the QG, remaining space of the code stream buffer or distortion constraint information. The flatness information or texture information is used to indicate the image gradient of the QG; the distortion constraint information indicates the distortion threshold of any QG among multiple QGs included in an image frame; and the remaining space of the code stream buffer is used to indicate the available margin of the code stream buffer (such as buffer space).

[0223] In the first case, the derived information is flatness information or texture information, and the video decoder 200 can derive the QP offset of the QG according to the flatness information or texture information. For example, the video decoder 200 calculates the texture complexity of the current block (QG), and uses a large QP (such as 20) for a QG with high texture complexity (such as reaching a texture complexity threshold); and uses a small QP (such as 5) for a QG with low texture complexity (such as not reaching the texture complexity threshold).

[0224] In the second case, the derived information is the remaining space in the bitstream buffer, and the video decoder 200 calculates the average bit number BPP of all pixels in the entire image total , and the average number of bits of the remaining uncoded pixels BPP left , if BPP left >BPP total , then reduce QP, otherwise increase QP. Among them, BPP total and BPP left It can be obtained using the following formula.

[0225]

[0226]

[0227] In the third case, if the derived information is distortion constraint information, let D be the quantization distortion (the difference between the reconstructed value after inverse quantization and the residual before quantization), and record any matrix A as:

[0228]

[0229]

[0230]

[0231] (1), if the residual in the pixel domain is quantized, the necessary and sufficient condition for the maximum distortion in the pixel domain to not exceed Δ is ||D|| max ≤Δ, from which the QP value in the pixel domain can be derived.

[0232] (2) If the residual in the transform domain is quantized, let R be the residual in the pixel domain, and U and V be the horizontal transform matrix and the vertical transform matrix. Then the maximum distortion in the pixel domain and the maximum distortion in the transform domain satisfy the following formula. Therefore, the sufficient condition for the maximum distortion in the pixel domain not to exceed Δ is that the maximum distortion in the transform domain satisfies From this, the QP value of the transform domain can be derived.

[0233] ||U -1 (URV+D)V -1 -R|| max =||U -1 DV-1|| max ≤||U -1 || ∞ ||D|| max |V -1 ||1

[0234] Please continue to see Figure 8 , the process of obtaining QP provided in this embodiment also includes the following steps.

[0235] S830 , the video decoder 200 determines the QP value of QG according to the predicted QP value of QG and the QP offset.

[0236] For example, the video decoder 200 uses the sum of the predicted QP value of the QG and the QP offset as the QP value of the QG.

[0237] It is worth noting that in the third situation provided by S820B, if the video decoder 200 obtains the QP value of QG, the following situations may occur.

[0238] The video decoder 200 determines the corresponding prediction distortion based on the reference QP value of QG. If the prediction distortion is less than or equal to the distortion threshold, the reference QP value is used as the QP value of QG; if the prediction distortion is greater than the distortion threshold, the QP value determined by the distortion threshold is used as the QP value of QG.

[0239] In Example 1, if the derived information only includes distortion constraint information, the reference QP value refers to the predicted QP value of the QG determined in S810.

[0240] In Example 2, if the derived information includes distortion constraint information and texture information (or flatness information), the reference QP value may refer to a QP value obtained by adding a deltaQP determined by the texture information (or flatness information) and a predicted QP value determined in S810.

[0241] In Example 3, if the derived information includes distortion constraint information and the remaining space of the codestream buffer, the reference QP value may refer to a QP value obtained by adding the deltaQP determined by the remaining space of the codestream buffer and the predicted QP value determined in S810.

[0242] That is to say, the above-mentioned derivation information can be used to determine the deltaQP of QG, or it can be used to directly determine the actual encoding QP value of QG. The specific use process of the derivation information can be determined according to the requirements of QP quantization / dequantization in video encoding and decoding. The above-mentioned three situations and three examples should not be understood as limitations on the present application.

[0243] The above-mentioned Examples 1 to 3 are merely examples of this embodiment to illustrate that the video decoder 200 uses derived information to determine the QP value of QG, and should not be understood as a limitation on the present application. In other examples, the derived information may also include distortion constraint information, texture information (or flatness information) and remaining space in the bitstream buffer, which is not limited in this application.

[0244] It is worth noting that the above-mentioned chroma / luminance QP values, direct coding / predictive coding, and code stream carrying / deriving information analysis are only some examples provided in this application and should not be understood as limitations on this application.

[0245] In an optional implementation, an image frame includes at least a first CU and a second CU, the first CU and the second CU do not have an overlapping area, and the QP values ​​of the first CU and the second CU are obtained in different ways. For example, the QP value of the first CU is obtained by the marker information of the bitstream, and the QP value of the second CU is obtained by the video decoder 200.

[0246] Specifically, the process of the video decoder 200 determining multiple QP values ​​of an image frame may include the following process: first, the video decoder 200 parses the bitstream to obtain the marking information of an image frame, the marking information includes the QP offset of the first part CU, and the video decoder 200 determines the QP value of the first part CU according to the marking information; second, the video decoder 200 obtains the predicted QP value of the second part CU for the second part CU, and the video decoder 200 also determines the QP value of the second part CU according to the predicted QP value and the derivation information of the second part CU. For the relevant content of the derivation information, please refer to the description of the derivation information of QG, and replace QG with CU, which will not be repeated here.

[0247] For example, the video decoder 200 divides an image into several regions, and adopts different QP processing methods for CUs in different regions. The baseline QP values ​​of different regions are transmitted at the image level, and the marking information of different regions is transmitted at the CU level. The CU-level QP of different regions can be obtained by bitstream transmission or by derivation at the decoding end. For example, if the video decoder divides the image into a region of interest (ROI) and a non-ROI region, for the CU in the ROI region (the first part CU), the QP value is obtained by bitstream transmission (such as the marking information mentioned above); for the CU in the non-ROI region (the second part CU), the QP value is obtained by derivation at the decoding end (such as the derivation information mentioned above).

[0248] As an optional implementation manner, a CU of an image frame may include multiple QGs, and the multiple QGs may be partially or completely QP-quantized.

[0249] For example, in the process of S520, if all QGs included in a CU of an image frame are quantized during the encoding process, the video decoder 200 may first determine the scanning order of all QGs included in a CU of the image frame. Then, the video decoder 200 obtains the QP value of each QG in the scanning order for each QG of all QGs. The scanning order includes any one of the following: from top to bottom, from left to right, zigzag or reverse zigzag order.

[0250] Specifically, if only some QGs are quantized and other QGs are not quantized, it is necessary to identify the positions of the QGs that are quantized and then obtain the corresponding QPs for these QGs. For example, a QP offset may be encoded for each of these QGs.

[0251] For another example, in the process of S520, if part of the QGs included in a CU of an image frame are quantized during the encoding process, the video decoder 200 can parse the bitstream to determine one or more QGs marked in a CU of an image frame, and the one or more marked QGs need to be QP inverse quantized during the decoding process; then, the video decoder 200 obtains the QP value of each QG for each of the one or more marked QGs.

[0252] Specifically, if all QGs are to be quantized, the corresponding QPs need to be obtained for each QG in turn according to a certain scanning order, where the scanning order is related to the division method, and can be from top to bottom, from left to right, in a zigzag or reverse zigzag order. For example, a QP offset can be encoded for each of these QGs.

[0253] In this way, for all QGs in a CU of an image frame, the marking information carried by the bitstream can be used to distinguish the QP quantization mode (partial quantization or full quantization) of all QGs, avoiding indiscriminate QP quantization by the video decoder, reducing the computing resources and image distortion required for QP quantization by the video decoder, and improving the decoding efficiency and accuracy of the video.

[0254] In addition, when a QG includes only one pixel in an image frame, one pixel corresponds to one QP value, and the QP values ​​of at least two pixels in all the pixels included in an image frame are different, the QP value quantization process of the pixel (point prediction mode) can adopt point-by-point quantization technology, for example, the point-by-point quantization technology includes the following process: the video decoder 200 can adaptively adjust the QP value of the current pixel according to the reconstructed pixel information around the current pixel. The reconstructed pixel information includes but is not limited to pixel value, flatness information or texture information, background brightness, contrast, etc.

[0255] It is worth noting that the adaptive point-by-point quantization technology can be applied to QG as well as to CUs without QG division. The characteristic of point-by-point quantization technology is that it allows each pixel in an image frame to use a different QP value, which is equivalent to refining the granularity of QP value quantization to the pixel level.

[0256] One way to implement point-by-point quantization technology is: let QP pred is the QP value of the current CU or QG, QP JND ≥0 is the QP value corresponding to the Just Noticeable Distortion (JND) that can be detected by the human eye, and offset>0 is the QP offset value (which can be transmitted in the bitstream or pre-set). The QP value of the current pixel is adjusted to:

[0257] Current pixel

[0258] Among them, threshold 2>threshold 1.

[0259] or,

[0260] Current pixel

[0261] The above two methods for determining the QP value of the current pixel are merely examples provided in this embodiment and should not be construed as limitations on the present application.

[0262] In this way, the video decoder adopts point-by-point quantization technology, which can significantly improve the subjective quality of the image frame and reduce the distortion of the image frame without changing the compression rate of the image frame.

[0263] A possible implementation is given below to illustrate the QP quantization process of pixels in an image frame.

[0264] For example, the video decoder 200 obtains a predicted QP value of a pixel, and determines a QP value of the pixel according to the predicted QP value of the pixel and the derived information.

[0265] In one possible scenario, the predicted QP value of a pixel is the QP value of the CU or QG where the pixel is located. In another possible scenario, the predicted QP value of a pixel is derived based on the QP values ​​of one or more reconstructed pixels around the pixel, wherein the derivation method includes calculating at least one of a mean (such as an average of the QP values ​​of multiple pixels), a median (such as the median of the QP values ​​of multiple pixels), or a mode (such as the QP value with the highest frequency among the QP values ​​of multiple pixels).

[0266] The derivation information of the above pixel point may be information of one or more reconstructed pixel points around the pixel point. The information of the reconstructed pixel point includes any one or a combination of the following: pixel value, flatness information or texture information, background brightness, and contrast. It is worth noting that the aforementioned information is only an example provided in this embodiment and should not be construed as a limitation on the present application.

[0267] In addition, the meaning of the above “surrounding” can be understood as the neighboring pixels of the pixel whose QP value is to be determined. Here are several possible examples for illustration, such as Fig. 9 As shown, Fig. 9 A schematic diagram of pixel distribution provided in this application, Fig. 9 (A) in FIG. 1 shows an example of a square area division with the current pixel as the center. Two possible situations are given here: Situation 1: The reconstructed pixel refers to the pixel in the square area with the current pixel as the center and a side length of 3, such as Fig. 9The surrounding pixel points 1 shown in (A); in case 2, the reconstructed pixel points refer to the pixel points in a square area with a side length of 5 and the current pixel point as the center, such as Fig. 9 The surrounding pixel point 2 shown in (A).

[0268] Fig. 9 (B) in FIG. 1 shows an example of a diamond region division with the current pixel as the center. Two possible situations are given here: Situation 1: The reconstructed pixel refers to the pixel in the diamond region with the current pixel as the center and the diagonal length of 3, such as Fig. 9 The surrounding pixel points 1 shown in (B); Case 2, the reconstructed pixel points refer to the pixel points in the diamond area with the current pixel point as the center and the diagonal length of 5, such as Fig. 9 The surrounding pixel point 2 shown in (B).

[0269] Fig. 9 This embodiment is only used to illustrate an example of reconstructed pixels around the current pixel, and should not be understood as limiting the present application. In other possible examples, the reconstructed pixels around the current pixel may also refer to one or two pixels adjacent to the current pixel above or below, or adjacent to the current pixel on the left or right.

[0270] As an optional implementation method, the QP value of a pixel is determined based on the predicted QP value and derived information of the pixel, including: determining indication information of the pixel based on information of one or more reconstructed pixels around the pixel; if the indication information is less than or equal to a first threshold, and the predicted QP value is greater than or equal to the QP value corresponding to the distortion just perceptible to the human eye, then the QP value corresponding to the distortion just perceptible to the human eye is used as the QP value of the pixel.

[0271] The first threshold value may be preset, or may be determined according to the compression rate or distortion rate requirement of the video codec. In addition, the first threshold value may also be determined according to user input information.

[0272] It is worth noting that the QP value corresponding to the distortion that the human eye can just perceive is image-level or CU-level information.

[0273] For example, the QP value corresponding to the distortion that can be just perceived by the human eye is obtained by parsing the bitstream. For example, the bitstream carries the QP value corresponding to the distortion that can be just perceived by the human eye, such as 20.

[0274] For example, the QP value corresponding to the distortion that can be just perceived by the human eye is derived based on the flatness information or texture information, background brightness, and contrast information of the surrounding reconstructed CU. For the process of deriving the QP value, please refer to the above Fig. 9 The relevant content will not be elaborated here.

[0275] In addition, the QP value corresponding to the distortion that can be just perceived by the human eye can also be a preset value set by the video encoder or video decoder, such as 15. That is to say, the QP value corresponding to the distortion that can be just perceived by the human eye can not only be carried in the mark information of the bitstream, but can also be parsed by the video encoder or video decoder during the video encoding and decoding process, or can also be a preset QP value. In this embodiment, the QP value corresponding to the distortion that can be just perceived by the human eye is introduced into the QP value decision of the current pixel point, so that each pixel point satisfies the judgment information corresponding to the distortion that can be just perceived by the human eye, thereby reducing image distortion and improving the subjective quality of the image.

[0276] A possible specific example is given here to illustrate the process of determining the QP value of a pixel point, as shown in Table 3. Table 3 shows a schematic diagram of determining the QP value of a current pixel point by dividing the interval into two.

[0277] Table 3

[0278] Interval Binary ≤Threshold >Threshold Method 1 <![CDATA[min(QP pred ,QP JND )]]> <![CDATA[QP pred ]]> Method 2 <![CDATA[min(QP pred ,QP JND )]]> <![CDATA[QP pred +offset]]> Method 3 <![CDATA[QP pred ]]> <![CDATA[QP pred +offset]]>

[0279] Wherein, offset>0 is a QP offset value (which can be transmitted in the bitstream or pre-set).

[0280] Method 1: When the indication information is less than or equal to the threshold, QP pred and QP JND The smaller value in is used as the QP value of the current pixel; when the indication information is greater than the threshold, the predicted QP value (QP pred ) as the QP value of the current pixel.

[0281] Method 2: When the indication information is less than or equal to the threshold, the QP pred and QP JND The smaller value in is used as the QP value of the current pixel; when the indication information is greater than the threshold, the QP pred The sum of the QP value and the QP offset is used as the QP value of the current pixel.

[0282] Method 3: When the indication information is less than or equal to the threshold, QP pred As the QP value of the current pixel; when the indication information is greater than the threshold, the QP of the pixel pred The sum of the QP value and the QP offset is used as the QP value of the current pixel.

[0283] It is worth noting that the above examples and Table 3 are only possible ways of obtaining the QP value of the current pixel provided in this embodiment, and should not be understood as limiting the present application.

[0284] In this way, the QP value corresponding to the distortion just perceptible to the human eye is introduced into the QP value decision process of the current pixel point, so that each pixel point satisfies the judgment information corresponding to the distortion just perceptible to the human eye, thereby reducing image distortion and improving the subjective quality of the image.

[0285] After the video decoder 200 obtains the QP value of an image frame, the present application provides a possible implementation method for the above S530: Fig.10 A schematic diagram of a process of decoding an image frame provided in the present application, S530 may include the following steps S1010 to S1030.

[0286] S1010 , for each QP value among a plurality of QP values, obtain a quantization step size (Qstep) corresponding to the QP value.

[0287] The video decoder can obtain Qstep by at least one of formula derivation and table lookup according to the QP value. Four possible implementation methods are provided below.

[0288] Method 1: Qstep = 2 × QP + 1.

[0289] Method 2: Let octave be the level of QP, that is, every time QP increases by octave, Qstep doubles. Octave is usually selected as 6 or 8. Let offset be an integer offset value, then:

[0290] Q step =2 (QP+offset) / octave

[0291] Method 3: Let octave be the level of QP, that is, Qstep doubles for every octave increase in QP. Octave is usually selected as 6 or 8. Let offset be an integer offset value. Respectively represent rounding up and rounding down.

[0292]

[0293] The quantization step size is: Q step =2 T .

[0294] Method 4: An example of quantization and dequantization is given below.

[0295] Quantification:

[0296] Dequantization: c′=l*Q step

[0297] Where c is the residual coefficient to be quantized (transform domain or pixel domain), l is the horizontal value after quantization, c′ is the reconstructed value after inverse quantization, Qstep is the quantization step size, f∈[0,1) is the control parameter included, and [0,1-f) is the quantization dead zone (the interval where the horizontal value is 0).

[0298] Let f∈[0,1) be the parameter controlling inclusion, {c i} is all the residual coefficients to be quantized in the current QG or CU, T can be obtained using the formula in method three, remember:

[0299]

[0300] Then when T<M, Qstep is given by the following formula, otherwise the quantization coefficient and the reconstructed value are both zero.

[0301]

[0302] A possible implementation of the fourth method is: take a = f*2 T-M Or take a=2 T-M , where M can be transmitted in the bitstream or directly taken as the bit depth.

[0303] When quantizing the residual in the pixel domain, method three requires a clip operation to ensure that the quantized coefficient can be represented by "MT" bits. The quantized coefficient of method four can be naturally represented by "MT" bits without clipping.

[0304] In addition, the uniform quantization scheme in JPEG-XS is to take f=0.5 in method 4 and transmit M in the bitstream.

[0305] It is worth noting that the larger the QP value, the larger the Qstep, the coarser the quantization, the greater the quantization distortion, and the smaller the bit rate of coefficient encoding. The parameter f is related to the length of the quantization dead zone. The smaller f is, the longer the quantization dead zone is, and the closer the horizontal value after quantization is to zero. When f=0.5, the quantization and dequantization formulas provided by the above S1010 method 4 are equivalent to rounding, and the quantization distortion is minimized. When f<0.5, the smaller f is, the greater the quantization distortion is, and the smaller the bit rate of coefficient encoding is. In H.265, f=1 / 3 for I images and f=1 / 6 for B / P images.

[0306] Please continue to see Figure 8 The above-mentioned S530 may further include the following steps.

[0307] S1020, obtaining a level value included in QG corresponding to the QP value.

[0308] S1030, dequantize the horizontal value of QG according to the selected quantizer combination.

[0309] Among them, the quantizer combination includes one or more quantizers, which are uniform quantizers or non-uniform quantizers.

[0310] For example, a uniform quantizer refers to a uniform scalar quantizer, and its quantization or inverse quantization formula can refer to the quantization and inverse quantization formulas provided in Method 4 of S1010 above. For the parameter f ∈ [0, 1), there are the following ways of taking values:

[0311] Way of taking value 1: f is taken as 0.5 or other fixed values.

[0312] Way of taking value 2: f can be adaptively determined according to the QP value, prediction mode, and whether a transform is performed.

[0313] Again, a non-uniform quantizer refers to a non-uniform scalar quantizer, and the correspondence between the quantization level value, quantization interval, and reconstructed value of inverse quantization can be obtained by looking up a table. A possible implementation is as shown in Table 1 below.

[0314] Table 1

[0315]

[0316] Among them, 0 ≤ x0 < x1 < x2 < x3 < …, the reconstructed value and quantization interval are non-uniform, and the reconstructed value is the probability centroid of the quantization interval where it is located.

[0317] The quantizer combination can employ one non-uniform scalar quantizer or multiple non-uniform scalar quantizers.

[0318] For example, the quantizer combination is determined by the marker information carried in the bitstream.

[0319] Again, it is determined by the distribution of the residual coefficients in QG. For example, the video decoder adaptively selects which quantizer to use, and the selection basis can be the mode information or transform information, and the mode information or transform information is related to the distribution of the residual coefficients in QG.

[0320] Regarding the above S1030, inverse quantization of the level value of QG can include the following process.

[0321] First, determine the partition domain type of QG.

[0322] Second, if QG is of the partition domain type of the transform domain, select a quantization matrix that matches the parameter information of QG from the matrix template library at the decoding end.

[0323] Among them, the matrix template library includes various types of quantization matrix templates, and the parameter information includes any one or more combinations of the following: the size of QG, the size of the CU where QG is located, the luminance and chrominance channel information, and the flatness information.

[0324] Finally, the horizontal values ​​in QG are dequantized using the quantization matrix of QG to obtain the residual coefficients of QG.

[0325] The residual coefficients at different positions in the pixel domain are equally important, so the quantization matrix is ​​not used. The coefficients in the transform domain are divided into low-frequency coefficients and high-frequency coefficients. The quantization matrix allows the high and low frequency coefficients to use different quantization steps, which can improve the subjective quality of the image while ensuring a certain compression rate.

[0326] The element distribution of the quantization matrix has a specific template. This application allows different quantization matrix templates to be used for coding blocks of different sizes, and a large-sized quantization matrix can be obtained by upsampling one or more small-sized quantization matrices. The quantization matrix templates included in the matrix template library are obtained by any one or more of the following types of transformations: discrete cosine transform (DCT), discrete sine transform (DST), integer transform or discrete wavelet transform (DWT). Fig.11 As shown, Fig.11 This is a schematic diagram of a quantization matrix template provided in this application, and the quantization matrix template specifically includes.

[0327] (1) If both horizontal and vertical directions are transformed by DCT / DST (see Fig.11 In (1)), the coefficients on the same anti-diagonal line have the same frequency and use the same quantization step size, while the coefficients on different anti-diagonal lines can use different quantization step sizes.

[0328] (2) If both horizontal and vertical directions are wavelet transformed (see Fig.11 In (2)), a quadtree template is used, and the four sub-blocks correspond to low / medium / high frequencies respectively. Sub-blocks in the same frequency band use the same quantization step size, and sub-blocks in different frequency bands can use different quantization step sizes.

[0329] (3) If DCT / DST is used in the vertical direction and wavelet transform is used in the horizontal direction (see Fig.11 (3) in the figure, first divide the matrix into two equal parts vertically, the left sub-block is the low frequency of the wavelet transform, and the right sub-block is the high frequency of the wavelet transform. The upper part of each sub-block is the low frequency of the DCT / DST transform, and the lower part is the high frequency of the DCT / DST transform. The matrix elements A` / B` / C` / D` can be made equal to A / B / C / D plus the same offset value.

[0330] (4) If DCT / DST is used in the horizontal direction and wavelet transform is used in the vertical direction (see Fig.11(4) in the figure), then the matrix is ​​first divided into two equal parts horizontally, the upper sub-block is the low frequency of the wavelet transform, and the lower sub-block is the high frequency of the wavelet transform. The left side of each sub-block is the low frequency of the DCT / DST transform, and the right side is the high frequency of the DCT / DST transform. The matrix elements A` / B` / C` / D` can be made equal to A / B / C / D plus the same offset value.

[0331] Optionally, the multiple types of quantization matrix templates included in the matrix template library may include a flat block template and a texture block template. The Qstep of the residual coefficients whose frequency in the flat block template is higher than the frequency threshold is greater than or equal to the Qstep of the residual coefficients whose frequency in the flat block template does not reach the frequency threshold. The Qstep of the residual coefficients whose frequency in the texture block template is higher than the frequency threshold is less than or equal to the Qstep of the residual coefficients whose frequency in the texture block template does not reach the frequency threshold.

[0332] For example, a video decoder marks the current block as a flat block or a texture block according to the flatness information, and then designs the quantization matrix template according to the texture masking effect: (1) If the current block (QG) is a flat block, the Qstep of the high-frequency coefficients in the quantization matrix is ​​greater than or equal to the Qstep of the low-frequency coefficients, because the human eye is more sensitive to the low-frequency distortion of flat blocks than the high-frequency distortion, so the high-frequency coefficients are allowed to have a greater loss. (2) If the current block (QG) is a texture block, the Qstep of the high-frequency coefficients in the quantization matrix is ​​less than or equal to the Qstep of the low-frequency coefficients, because the human eye is more sensitive to the high-frequency distortion of texture blocks than the low-frequency distortion, so the high-frequency coefficients of texture blocks are protected first.

[0333] That is to say, in the image decoding method, the video decoder first obtains the QP and Qstep of QG, parses the horizontal value from the bitstream, and then adaptively selects the quantizer to dequantize the horizontal value to obtain the reconstructed value, thereby realizing the decoding of the image frame.

[0334] In summary, compared with the larger image distortion caused by using the same QP value for all residual coefficients in a CU, in the decoding process of the video image provided in this embodiment, a CU can be divided into multiple QGs, and one or more residual coefficients in each QG share a QP value. Then, the video decoder can make a more fine-grained QP decision for one or more CUs corresponding to the image frame, thereby reducing the decoding distortion of the image frame while ensuring a certain compression rate, and improving the authenticity and accuracy of video image decoding.

[0335] Correspondingly, in the image encoding method, the video encoder first obtains the QP, Qstep and residual coefficient of QG; adaptively selects a quantizer to quantize the residual coefficient; and finally adjusts the quantization coefficient to obtain a final horizontal value, thereby realizing encoding of the image frame.

[0336] exist Figure 2 Based on the video encoder 100 shown, the present application also provides an image encoding method, such as Fig.12 As shown, Fig.12 The present application provides a flowchart of an image encoding method, which can be executed by the video encoder 100 or by an encoding end (such as Figure 1 The encoding end 10) shown in the figure is executed. Here, the encoding method implemented by the video encoder 100 is taken as an example for explanation. The image encoding method includes the following steps.

[0337] S1210 , the video encoder 100 divides an image frame into one or more CUs.

[0338] S1220 , the video encoder 100 determines a plurality of QP values ​​for one image frame.

[0339] In an example, a CU includes multiple QGs, and one QG corresponds to one QP value.

[0340] In another example, a CU includes multiple pixels, one pixel corresponds to one QP value, and the QP values ​​of at least two of the multiple pixels are different.

[0341] S1230 , the video encoder 100 encodes an image frame according to multiple QP values.

[0342] For the quantization of QP values ​​in the encoding method, please refer to the above Figures 4 to 11 The corresponding process in the decoding method will not be described here.

[0343] In this way, compared with the larger image distortion caused by using the same QP value for all residual coefficients in a CU, in the encoding process of the video image provided in this embodiment, a CU can be divided into multiple QGs (or pixel points), and one or more residual coefficients in each QG share a QP value. Then, the video encoder can make a more fine-grained QP decision for one or more CUs corresponding to the image frame, thereby reducing the encoding distortion of the image frame while ensuring a certain compression rate, and improving the authenticity and accuracy of video image encoding.

[0344] It is understandable that, in order to implement the functions in the above embodiments, the video encoder / video decoder includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and method steps of each example described in the embodiments disclosed in this application, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application scenario and design constraints of the technical solution.

[0345] Fig.13 The present application provides a structural diagram of a decoding device, wherein the decoding device 1300 includes a code stream parsing unit 1310, a QP decision unit 1320, and an image decoding unit 1330. The decoding device 1300 can be used to implement the functions of the video decoder or the decoding end in the above-mentioned decoding method embodiment, and thus can also achieve the beneficial effects of the above-mentioned decoding method embodiment. In the embodiment of the present application, the decoding device 1300 can be as follows Figure 1 The decoding end 20 or video decoder 200 shown may also be as shown in Figure 3 The video decoder 200 shown may also be a module (such as a chip) applied to the decoding end 20 or the video decoder 200 .

[0346] The code stream parsing unit 1310, the QP decision unit 1320 and the image decoding unit 1330 are used to implement Figures 4 to 11 For a more detailed description of the above-mentioned code stream parsing unit 1310, QP decision unit 1320 and image decoding unit 1330, please directly refer to Figures 4 to 11 The relevant description in the method embodiment shown is directly obtained and will not be repeated here.

[0347] Fig.14 The present application provides a schematic diagram of the structure of a coding device, the coding device 1400 includes: an image segmentation unit 1410, a QP decision unit 1420 and an image coding unit 1430. The coding device 1400 can be used to implement the functions of the video encoder or the coding end in the above decoding method embodiment, and thus can also achieve the beneficial effects of the above decoding method embodiment. In the embodiment of the present application, the coding device 1400 can be as follows Figure 1 The encoding end 10 or video encoder 100 shown may also be as shown in Figure 2 The video encoder 100 shown may also be a module (such as a chip) applied to the encoding end 10 or the video encoder 100 .

[0348] The image segmentation unit 1410, the QP decision unit 1420 and the image encoding unit 1430 are used to implement Fig.12 For a more detailed description of the above-mentioned image segmentation unit 1410, QP decision unit 1420 and image encoding unit 1430, please refer to Figures 4 to 12 The relevant description in the method embodiment shown is directly obtained and will not be repeated here.

[0349] The present application also provides an electronic device, such as Fig.15 As shown, Fig.15The electronic device 1500 is a schematic diagram of the structure of an electronic device provided by the present application, wherein the electronic device 1500 includes a processor 1510 and an interface circuit 1520. The processor 1510 and the interface circuit 1520 are coupled to each other. It is understood that the interface circuit 1520 may be a transceiver or an input / output interface. Optionally, the electronic device 1500 may further include a memory 1530 for storing instructions executed by the processor 1510 or storing input data required by the processor 1510 to execute instructions or storing data generated after the processor 1510 executes instructions.

[0350] The electronic device 1500 includes a processor 1510 and a communication interface 1520. The processor 1510 and the communication interface 1520 are coupled to each other. It is understood that the communication interface 1520 can be a transceiver or an input-output interface. Optionally, the electronic device 1500 may also include a memory 1530 for storing instructions executed by the processor 1510 or storing input data required by the processor 1510 to execute instructions, or storing data generated after the processor 1510 executes instructions.

[0351] When the electronic device 1500 is used to implement Figures 4 to 11 In the method shown, the processor 1510 and the interface circuit 1520 are used to execute the functions of the above-mentioned code stream parsing unit 1310, QP decision unit 1320 and image decoding unit 1350.

[0352] When the electronic device 1500 is used to implement Fig.12 In the method shown, the processor 1510 and the interface circuit 1520 are used to execute the functions of the above-mentioned image segmentation unit 1410, the QP decision unit 1420 and the image encoding unit 1430.

[0353] The specific connection medium between the communication interface 1520, the processor 1510 and the memory 1530 is not limited in the embodiment of the present application. Fig.15 The communication interface 1520, the processor 1510 and the memory 1530 are connected via a bus 1540. Fig.15 The connections between the other components are shown in bold lines, which are only for illustration and are not intended to be limiting. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.15 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0354] The memory 1530 can be used to store software programs and modules, such as program instructions / modules corresponding to the decoding method or encoding method provided in the embodiment of the present application. The processor 1510 executes the software programs and modules stored in the memory 1530 to perform various functional applications and data processing. The communication interface 1520 can be used to communicate signaling or data with other devices. In the present application, the electronic device 1500 can have multiple communication interfaces 1520.

[0355] It is understandable that the processor in the embodiments of the present application may be a central processing unit (CPU), a neural processing unit (NPU) or a graphic processing unit (GPU), or may be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. A general-purpose processor may be a microprocessor or any conventional processor.

[0356] The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable ROM (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC. In addition, the ASIC can be located in a network device or a terminal device. Of course, the processor and the storage medium can also be present in a network device or a terminal device as discrete components.

[0357] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instruction is loaded and executed on a computer, the process or function described in the embodiment of the present application is executed in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device or other programmable device. The computer program or instruction may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer program or instruction may be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired or wireless means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium, for example, a floppy disk, a hard disk, a tape; it may also be an optical medium, for example, a digital video disc (DVD); it may also be a semiconductor medium, for example, a solid state drive (SSD).

[0358] In the various embodiments of the present application, if there is no special explanation and logical conflict, the terms and / or descriptions between different embodiments are consistent and can be referenced to each other, and the technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationships. In the present application, "at least one" means one or more, and "more" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. In the text description of the present application, the character " / " generally indicates that the previous and next associated objects are in an "or" relationship; in the formula of the present application, the character " / " indicates that the previous and next associated objects are in a "division" relationship.

[0359] It is understood that the various numbers involved in the embodiments of the present application are only for the convenience of description and are not used to limit the scope of the embodiments of the present application. The size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic.

Claims

1. A method for decoding an image, characterized in that: The method is performed by a decoding end, and the method includes: Parse the code stream to obtain one or more image frames, where one image frame includes one or more coding units CU; Determine multiple quantization parameter QP values ​​of the image frame; wherein a CU includes multiple quantization groups QG, and one QG corresponds to one QP value; Decoding the one image frame according to the multiple QP values; The CU includes multiple residual coefficients, the QG includes a portion of the multiple residual coefficients, and the portion of the residual coefficients share the QP value.

2. The method according to claim 1, characterized in that Before determining the multiple QP values ​​of the one image frame, the method further includes: Divide a CU included in the image frame according to a first rule to obtain the multiple QGs; wherein, The first rule includes a division domain and a division method, the division domain is a pixel domain, and the division method includes uniform division.

3. The method according to claim 1, characterized in that A QP value corresponding to a QG includes a luminance QP value and a chrominance QP value; Determining a plurality of QP values ​​of the one image frame includes: Obtaining a luminance QP value and a chrominance QP value of the one QG respectively; or, Obtaining a brightness QP value of the QG; A chroma QP value for the one QG is determined based on the luma QP value.

4. The method according to claim 1, characterized in that: Decoding the one image frame according to the multiple QP values ​​includes: For each QP value among the multiple QP values, obtain a quantization step size Qstep corresponding to the QP value; Obtain the level value contained in the QG corresponding to the QP value; The level value of the QG is inversely quantized according to the selected quantizer combination; the quantizer combination includes one or more quantizers.

5. The method according to claim 4, characterized in that The quantizer combination is determined by the marking information carried by the code stream, or the quantizer combination is determined by the distribution of residual coefficients in the QG.

6. The method according to claim 4 or 5, characterized in that: The quantizer is a uniform quantizer or a non-uniform quantizer.

7. The method according to claim 1, characterized in that The QG includes one or more pixels of the image frame.

8. The method according to claim 1, characterized in that The QP values ​​corresponding to at least two QGs among the multiple QGs are different.

9. A method for encoding an image, characterized in that: The method is performed by an encoding end, and the method includes: Divide an image frame into one or more coding units CU; Determine multiple quantization parameter QP values ​​of the image frame; wherein a CU includes multiple quantization groups QG, and one QG corresponds to one QP value; Encoding the one image frame according to the multiple QP values; The one CU includes multiple residual coefficients, the one QG includes a portion of the multiple residual coefficients, and the portion of the residual coefficients share the one QP value.

10. An image decoding device, characterized in that: The decoding device comprises: a code stream parsing unit, a QP decision unit and an image decoding unit; The code stream parsing unit is used to parse the code stream to obtain one or more image frames, where one image frame includes one or more coding units CU; The QP decision unit is used to determine multiple quantization parameter QP values ​​of the image frame; wherein a CU includes multiple quantization groups QG, and one QG corresponds to one QP value; The image decoding unit is used to decode the one image frame according to the multiple QP values; The one CU includes multiple residual coefficients, the one QG includes a portion of the multiple residual coefficients, and the portion of the residual coefficients share the one QP value.

11. An image encoding device, characterized in that: The encoding device comprises: an image segmentation unit, a QP decision unit and an image encoding unit; The image segmentation unit is used to divide an image frame into one or more coding units CU; The QP decision unit is used to determine multiple quantization parameter QP values ​​of the image frame; wherein a CU includes multiple quantization groups QG, and one QG corresponds to one QP value; The image encoding unit is used to encode the one image frame according to the multiple QP values; The one CU includes multiple residual coefficients, the one QG includes a portion of the multiple residual coefficients, and the portion of the residual coefficients share the one QP value.

12. A video decoding system, characterized in that: The invention comprises an encoding end and a decoding end, wherein the encoding end is communicatively connected with the decoding end, the decoding end is used to implement the method according to any one of claims 1 to 8, and the encoding end is used to implement the method according to claim 9.

13. An electronic device, characterized in that: The invention comprises a processor and a memory, wherein the memory is used to store computer instructions, and the processor is used to call and execute the computer instructions from the memory to implement the method according to any one of claims 1 to 9.

14. A computer-readable storage medium, characterized in that: The storage medium stores a computer program or instruction, and when the computer program or instruction is executed by an electronic device, the method according to any one of claims 1 to 9 is implemented.

Images