Decoding method, encoding method and device

Abstract

An image decoding method, encoding method and apparatus, which are related to the field of video encoding and decoding, includes: first, analyzing a bitstream to obtain one or more image frames, where an image frame includes one or more CUs; ​​then, determining multiple QP values ​​for the image frame, where a CU includes multiple QGs and a QG corresponds to a QP value; and finally, decoding the image frame based on the multiple QP values.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to a Chinese patent application having application number 202111334223.8, filed on November 11, 2021, which is incorporated herein by reference.

[0002] The present invention relates to the field of video encoding and decoding, and in particular to an image decoding method, encoding method and apparatus. [Background technology]

[0003] In video encoding and decoding technology, video compression technology is particularly important. Video compression technology performs spatial (intra-picture) prediction and / or temporal (inter-picture) prediction to reduce or remove redundant information inherent in video sequences. The basic principle of video compression is to quantize the correlation between the spatial domain, the temporal domain and the codeword, and remove the redundancy as much as possible. Quantization is the process of mapping the continuous values ​​(or a large number of discrete values) of a signal to a finite number of discrete amplitudes, achieving a many-to-one mapping of signal values.

[0004] In the current practice, for one or more coding units (CUs) included in a frame of image, the encoding side obtains a quantization parameter (QP) of each CU, encodes the CU according to the QP to obtain a bitstream, and accordingly, the decoding side dequantizes the bitstream to obtain the QP of the CU, and decodes the CU according to the QP. The CU is divided according to image content, and the encoding side and the decoding side use the same QP to quantize the image content corresponding to one CU, so the quantization distortion of the image encoding and decoding process is large. Summary of the Invention [Problem to be solved by the invention]

[0005] The present invention provides an image decoding method, an image encoding method and an apparatus that solve the problem of large quantization distortion in image encoding and decoding processes. [Means for solving the problem]

[0006] The present invention adopts the following technical solutions.

[0007] According to a first aspect, the present invention provides an image decoding method, which may be applied to a video coding system, or may be applied to a decoding side capable of supporting video coding to realize the method, for example, the decoding side includes a video decoder. The method includes: first, analyzing a bitstream to obtain one or more image frames, where one image frame includes one or more CUs; ​​then, determining multiple QP values ​​for the one image frame, where one CU includes multiple QGs, and one QG corresponds to one QP value; and finally, decoding the one image frame based on the multiple QP values.

[0008] Compared with the large image distortion caused by all residual coefficients in one CU using the same QP value, in the video image decoding process provided in this embodiment, one CU may be divided into multiple QGs, and one or more residual coefficients in each QG share one QP value, so that the video decoder can make finer granularity QP decisions for one or more CUs corresponding to an image frame, thereby reducing the decoding distortion of the image frame and improving the authenticity and accuracy of video image decoding while ensuring a certain compression rate.

[0009] In one optional embodiment, the one CU includes a plurality of residual coefficients, the one QG includes a portion of the plurality of residual coefficients, and the portion of the residual coefficients share the one QP value.

[0010] In one optional embodiment, before determining multiple QP values ​​for the one image frame, the method further includes a step of dividing a CU included in the one image frame according to a first rule to obtain the multiple QGs, the first rule including a division region and a division scheme, the division region being a transformation region or a pixel region, and the division scheme including at least one of uniform division and non-uniform division.

[0011] In one optional embodiment, the one CU includes a plurality of residual coefficients, and the positions of the plurality of residual coefficients are marked with coordinates, the coordinates including an abscissa and an ordinate. When the division region is a transform region, the step of dividing the one CU included in the one image frame according to a first rule and obtaining the plurality of QGs ​​includes the step of dividing the plurality of residual coefficients, whose coordinate sum does not reach a first coordinate threshold, into a first QG, and dividing the residual coefficients, whose coordinate sum reaches the first coordinate threshold, into a second QG, and the coordinate sum is the sum of the ordinate and the ordinate of the residual coefficient.

[0012] Alternatively, among the plurality of residual coefficients, residual coefficients whose coordinate sum does not reach a first coordinate threshold are divided into a first QG, residual coefficients whose coordinate sum reaches the first coordinate threshold but does not reach a second coordinate threshold are divided into a second QG, and residual coefficients whose coordinate sum reaches the second coordinate threshold are divided into a third QG, and the second coordinate threshold is greater than the first coordinate threshold.

[0013] In one optional embodiment, when the one CU includes a plurality of residual coefficients, and the division domain is a transform domain, the step of dividing the one CU included in the one image frame according to a first rule and obtaining the plurality of QGs ​​includes: The method includes a step of sorting the plurality of residual coefficients, dividing the residual coefficients among the plurality of residual coefficients that have not reached a first proportional threshold into a first QG, and dividing the residual coefficients that have reached the first proportional threshold into a second QG, and a sorting method of the plurality of residual coefficients is either zig-zag or reverse zig-zag.

[0014] Alternatively, the plurality of residual coefficients are sorted, and among the plurality of residual coefficients, those residual coefficients that have not reached a first proportional threshold are divided into a first QG, those residual coefficients that have reached the first proportional threshold but have not reached a second proportional threshold are divided into a second QG, and those residual coefficients that have reached the second proportional threshold are divided into a third QG, and the second proportional threshold is greater than the first proportional threshold.

[0015] In one optional embodiment, when the one CU includes multiple residual coefficients and the division region is a pixel region, the step of dividing the one CU included in the one image frame according to a first rule and obtaining the multiple QGs includes a step of performing a horizontal or vertical symmetric division on the multiple residual coefficients and obtaining two QGs including the same number of residual coefficients.

[0016] Alternatively, the multiple residual coefficients are symmetrically divided horizontally or vertically to obtain three QGs, and the number of residual coefficients contained in two non-adjacent QGs among the three QGs is the same, and the number of residual coefficients contained in the remaining QG is the same as the sum of the numbers of residual coefficients contained in the two non-adjacent QGs.

[0017] Alternatively, a horizontal or vertical division is performed on the plurality of residual coefficients to obtain two QGs containing dissimilar numbers of residual coefficients.

[0018] Or, the plurality of residual coefficients are divided horizontally or vertically to obtain three QGs, and the residual coefficients included in the three QGs do not have a symmetrical relationship.

[0019] In an optional embodiment, the one QP value corresponding to the one QG includes a luma QP value and a chroma QP value, and determining the multiple QP values ​​for the one image frame includes respectively obtaining a luma QP value and a chroma QP value for the one QG.

[0020] Alternatively, a luminance QP value of the one QG is obtained, and a chromaticity QP value of the one QG is obtained based on the luminance QP value.

[0021] In one optional embodiment, the step of determining multiple QP values ​​for the one image frame includes a step of parsing the bitstream to obtain mark information for the one image frame, the mark information being used to indicate a QP value of the one QG and / or the mark information being used to indicate a QP value of the one CU.

[0022] In one optional embodiment, the step of determining multiple QP values ​​for the one image frame includes, first, parsing the bitstream to obtain mark information for the one image frame, the mark information being used to indicate a QP offset for the one QG, and, second, determining a QP value for the one QG based on a predicted QP value for the one QG and the mark information.

[0023] In one optional embodiment, the step of determining a QP value of the one QG based on the predicted QP value of the one QG and the mark information includes a step of obtaining a predicted QP value of the one QG and determining the sum of the predicted QP value of the one QG and the QP offset as the QP value of the one QG.

[0024] In one optional embodiment, the step of determining multiple QP values ​​for the one image frame includes the steps of obtaining a predicted QP value for the one QG, and determining a QP value for the one QG based on the predicted QP value for the one QG and derivation information, wherein the derivation information is any one or more combinations of flatness information or texture information, remaining space in a bitstream buffer, or distortion constraint information for the one QG.

[0025] In one optional embodiment, when the derivation information is the distortion constraint information and the distortion constraint information indicates a distortion threshold for any one of the multiple QGs, determining a QP value for the one QG based on the predicted QP value of the one QG and the derivation information includes determining a predicted distortion corresponding to the predicted QP value, determining the predicted QP value as the QP value of the QG if the predicted distortion is less than or equal to the distortion threshold, and determining a QP value determined by the distortion threshold as the QP value of the QG if the predicted distortion is greater than the distortion threshold.

[0026] In one optional embodiment, when the derivation information is content information of the one QG or remaining space in the bitstream buffer, the step of determining a QP value of the one QG based on the predicted QP value of the one QG and the derivation information includes the steps of determining a QP offset of the one QG based on the derivation information, and determining the sum of the predicted QP value of the one QG and the QP offset as the QP value of the one QG.

[0027] In one optional embodiment, the step of obtaining a predicted QP value of the one QG includes a step of obtaining a QP value of at least one other QG in the one CU that is adjacent to the one QG, and determining a predicted QP value of the one QG based on the QP value of the at least one other QG.

[0028] Alternatively, the QP value of the one CU is determined as the predicted QP value of the one QG.

[0029] In one optional embodiment, the one image frame includes at least a CU of a first part and a CU of a second part, the CU of the first part and the CU of the second part have no overlapping area, and the QP values ​​of the CU of the first part and the CU of the second part are obtained in different methods.

[0030] In one optional embodiment, the step of determining multiple QP values ​​for the one image frame includes the steps of: analyzing the bitstream to obtain mark information for the one image frame, where the mark information includes QP offsets for CUs in the first portion; determining QP values ​​for CUs in the first portion based on the mark information; for CUs in the second portion, obtaining predicted QP values ​​for CUs in the second portion; and determining QP values ​​for CUs in the second portion based on the predicted QP values ​​for CUs in the second portion and derivation information, where the derivation information is any one or more of flatness information or texture information for CUs in the second portion, remaining space in a bitstream buffer, or distortion constraint information.

[0031] In one optional embodiment, the step of decoding the one image frame based on the multiple QP values ​​includes: first, for each QP value of the multiple QP values, obtaining a quantization step Qstep corresponding to the QP value, then obtaining a level value included in a QG corresponding to the QP value, and finally, dequantizing the level value of the QG based on a selected quantizer combination, the quantizer combination including one or more quantizers. Illustratively, the quantizer is a uniform quantizer or a non-uniform quantizer.

[0032] In one optional embodiment, the quantizer combination is determined by mark information added to the bitstream or by the distribution of residual coefficients within the QG.

[0033] In one optional embodiment, the step of inverse quantizing the level values ​​of the QG includes: first, determining a division area type of the QG; then, if the division area type of the QG is a transform area, selecting a quantization matrix matching the parameter information of the QG from a matrix template library on the decoding side, where the matrix template library includes multiple types of quantization matrix templates, and the parameter information includes any one or more combinations of the size of the QG, the size of the CU in which the QG is located, luminance channel information, and flatness information; and finally, inverse quantizing the level values ​​in the QG using the quantization matrix of the QG to obtain residual coefficients of the QG.

[0034] In one optional embodiment, the multiple types of quantization matrix templates include a flat block template and a texture block template, wherein the Qstep of residual coefficients in the flat block template whose frequencies are above a frequency threshold is greater than or equal to the Qstep of residual coefficients in the flat block template whose frequencies are below the frequency threshold, and the Qstep of residual coefficients in the texture block template whose frequencies are above the frequency threshold is less than or equal to the Qstep of residual coefficients in the texture block template whose frequencies are below the frequency threshold.

[0035] In one optional embodiment, the quantization matrix templates included in the matrix template library are obtained by one or more of the following transformation methods: discrete cosine transform (DCT), discrete sine transform (DST), integer transform, or discrete wavelet transform (DWT).

[0036] In one optional embodiment, the one QG includes one or more pixel points of the one image frame.

[0037] In one optional embodiment, the step of determining multiple QP values ​​for the one image frame includes a step of analyzing the bitstream to determine one or more marked QGs in the one CU, where the one or more marked QGs need to be dequantized during a decoding process and unmarked QGs in the one CU are not dequantized, and a step of obtaining a QP value for each QG of the one or more marked QGs.

[0038] Or, a scanning order of all QGs included in the one CU is determined, the scanning order including any one or more combinations of top-to-bottom, left-to-right, zigzag or reverse zigzag orders, and for each QG of the all QGs, a QP value of each QG is obtained according to the scanning order.

[0039] In one optional embodiment, the QP values ​​corresponding to at least two QGs of the plurality of QGs ​​are different.

[0040] According to a second aspect, the present invention provides an image decoding method, which may be applied to a video coding system, or the method may be applied to a decoding side that can support video coding to realize the method, and the method is executed by the decoding side, and the method includes the steps of: analyzing a bitstream to obtain one or more image frames, where one image frame includes one or more CUs; ​​determining multiple QP values ​​for the one image frame, where one CU includes multiple pixel points, where one pixel point corresponds to one QP value, and where the QP values ​​of at least two pixel points among the multiple pixel points are different; and decoding the one image frame based on the multiple QP values.

[0041] In one optional embodiment, the step of determining multiple QP values ​​for the one image frame includes the steps of obtaining a predicted QP value for the one pixel point, and determining a QP value for the one pixel point based on the predicted QP value for the one pixel point and derived information, where the derived information is information of one or more reconstructed pixel points surrounding the one pixel point.

[0042] In another optional embodiment, the predicted QP value of the one pixel point is the QP value of a CU or QG in which the one pixel point is located, or is derived based on the QP values ​​of one or more reconstructed pixel points surrounding the one pixel point, and the derivation method includes calculating at least one of the average, median, or mode.

[0043] In another optional embodiment, the reconstructed pixel points are pixel points within a square region with sides of length 3 or 5, or within a diamond region with a diagonal length of 3 or 5, centered on the one pixel point.

[0044] In another optional embodiment, the information of the reconstructed pixel point includes any one or more combinations of pixel value, flatness or texture information, background brightness, contrast.

[0045] In another optional embodiment, the step of determining the QP value of the pixel point based on the predicted QP value of the pixel point and the derived information includes the steps of: determining indication information of the pixel point based on information of one or more reconstructed pixel points surrounding the pixel point; and, when the indication information is equal to or less than a first threshold and the predicted QP value is equal to or greater than a QP value corresponding to the distortion that is just perceptible by the human eye, determining the QP value corresponding to the distortion that is just perceptible by the human eye as the QP value of the pixel point, where the QP value corresponding to the distortion that is just perceptible by the human eye is a preset value (e.g., picture level or CU level information preset at the encoding side or decoding side), or the QP value corresponding to the distortion that is just perceptible by the human eye is obtained by analyzing the bitstream (picture level or CU level transmission), or the QP value corresponding to the distortion that is just perceptible by the human eye is derived based on flatness information or texture information, background luminance, and contrast information of surrounding reconstructed CUs.

[0046] According to a third aspect, the present invention provides an image encoding method, which may be applied to a video coding system, or the method may be applied to an encoding side that can support video coding to realize the method, and the method is executed by an encoding side, and the method includes the steps of: dividing an image frame into one or more CUs; ​​determining multiple QP values ​​for the image frame, where one CU includes multiple QGs and one QG corresponds to one QP value; and encoding the image frame based on the multiple QP values.

[0047] According to a fourth aspect, the present invention provides an image encoding method, which may be applied to a video coding system, or the method may be applied to an encoding side that can support video coding to realize the method, and the method is executed by an encoding side, the method including: dividing an image frame into one or more CUs; ​​determining multiple QP values ​​for the image frame, where a CU includes multiple pixel points, where a pixel point corresponds to a QP value, and where the QP values ​​of at least two pixel points among the multiple pixel points are different; and encoding the image frame based on the multiple QP values.

[0048] According to a fifth aspect, the present invention provides an image decoding device, the decoding device is applied to a decoding side, and the decoding device includes modules for implementing the method in any of the possible embodiments of the first aspect or the second aspect, such as a bitstream analysis unit, a QP determination unit and an image decoding unit.

[0049] The beneficial effects are described in the description of either the first or second aspect, and will not be repeated here. The decoding device has a function for implementing the operations in the example of the method of either the first or second aspect. The function may be implemented by hardware, or may be implemented by the hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.

[0050] According to a sixth aspect, the present invention provides an image encoding device, the encoding device being applied to an encoding side, and the encoding device includes each module for implementing the method in any of the possible embodiments of the third aspect or the fourth aspect. For example, the encoding device includes an image division unit, a QP determination unit and an image encoding unit. The beneficial effects are described in the description of either the third aspect or the fourth aspect, so they will not be repeated here. The encoding device has a function for implementing the operation in any of the method examples of the third aspect or the fourth aspect. The function may be implemented by hardware, or may be implemented by the hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.

[0051] According to a seventh aspect, the present invention provides an electronic device including a processor and a memory, the memory being adapted to store computer instructions, and the processor being adapted to retrieve and execute the computer instructions from the memory and to implement a method in any of the embodiments of the first to fourth aspects.

[0052] For example, the electronic device may refer to a video encoder or an encoding side that includes a video encoder.

[0053] Also, for example, the electronic device may refer to a video decoder or a decoding side including a video decoder.

[0054] According to an eighth aspect, the present invention provides a computer readable storage medium having stored thereon a computer program or instructions which, when executed by a computing device, or a storage system in which the computing device is located, performs a method in accordance with any of the embodiments of the first to fourth aspects.

[0055] According to a ninth aspect, the present invention provides a computer program product comprising instructions which, when executed on a computing device or processor, cause the computing device or processor to execute the instructions to perform a method in accordance with any of the embodiments of the first to fourth aspects.

[0056] According to a tenth aspect, the present invention provides a video coding system, the video coding system including an encoding side and a decoding side, the decoding side being used for implementing a method in any of the embodiments of the first to second aspects, and the encoding side being used for implementing a method in any of the embodiments of the third to fourth aspects.

[0057] The present invention can provide many more embodiments by further combining the embodiments provided in each of the above aspects. [Brief description of the drawings]

[0058] [Figure 1] 1 is an exemplary block diagram of a video coding system in accordance with the present invention; [Diagram 2] 1 is an exemplary block diagram of a video encoder according to the present invention; [Diagram 3] FIG. 2 is an exemplary block diagram of a video decoder according to the present invention; [Figure 4] 1 is a schematic flow chart of video encoding / decoding according to the present invention; [Diagram 5] 1 is a schematic flowchart of an image decoding method according to the present invention. [Figure 6] FIG. 2 is a schematic diagram of a transform domain decomposition according to the present invention; [Figure 7] FIG. 2 is a schematic diagram of pixel area division according to the present invention; [Figure 8] 4 is a schematic flowchart of obtaining QP in a predictive coding method according to the present invention; [Figure 9] FIG. 2 is a schematic diagram of a distribution of pixel points according to the present invention; [Figure 10]3 is a schematic flow chart of image frame decoding according to the present invention; [Figure 11] FIG. 2 is a schematic diagram of a quantization matrix template according to the present invention; [Figure 12] 1 is a schematic flow chart of an image encoding method according to the present invention; [Figure 13] FIG. 2 is a schematic structural diagram of a decoding device according to the present invention; [Figure 14] 1 is a schematic structural diagram of an encoding device according to the present invention; [Figure 15] 1 is a schematic structural diagram of an electronic device according to the present invention; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0059] In order to clarify and concisely describe the following embodiments, a brief description of the related art will first be provided.

[0060] 1 is an exemplary block diagram of a video coding system according to the present invention, and as used herein, the term "video coder" generally refers to both a video encoder and a video decoder. In the present invention, the term "video coding" or "coding" may generally refer to video encoding or video decoding. The video encoder 100 and the video decoder 200 of the video coding system 1 are used to predict motion information, e.g., motion vectors, of a currently coded image block or its sub-block according to various method examples described in any one of a number of new inter-frame prediction modes provided in the present invention, so that the predicted motion vector is as close as possible to the motion vector obtained using a motion estimation method, thereby eliminating the need to transmit differential motion vectors during encoding and further improving encoding and decoding performance.

[0061] As shown in FIG. 1, the video coding system 1 includes an encoding side 10 and a decoding side 20. The encoding side 10 generates encoded video data. The encoding side 10 may therefore be referred to as a video encoding device. The decoding side 20 can decode the encoded video data generated by the encoding side 10. The decoding side 20 may therefore be referred to as a video decoding device. Each embodiment of the encoding side 10, the decoding side 20, or both may include one or more processors and a memory coupled to the one or more processors. As described herein, the memory may include, but is not limited to, a RAM, a ROM, an EEPROM, a flash memory, or any other medium that may be used to store desired program code in the form of instructions or data structures accessible by a computer.

[0062] The encoding side 10 and the decoding side 20 may include a variety of devices, including desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, mobile phones such as so-called "smart" phones, televisions, cameras, display devices, digital media players, video game consoles, in-vehicle computers, or the like.

[0063] The decoding side 20 can receive the encoded video data from the encoding side 10 via a link 30. The link 30 may include one or more media or devices that can move the encoded video data from the encoding side 10 to the decoding side 20. In one example, the link 30 may include one or more communication media that allow the encoding side 10 to transmit the encoded video data directly to the decoding side 20 in real time. In this example, the encoding side 10 may modulate the encoded video data according to a communication standard (e.g., a wireless communication protocol) and transmit the modulated video data to the decoding side 20. The one or more communication media may include wireless and / or wired communication media, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The one or more communication media may form part of a packet-based network, such as a local area network, a wide area network, or a global network (e.g., the Internet). The one or more communication media may include a router, a switch, a base station, or another device that facilitates communication from the encoding side 10 to the decoding side 20.

[0064] In another example, the encoded data may be output from output interface 140 to storage device 40. Similarly, the encoded data may be accessed from storage device 40 via input interface 240. Storage device 40 may include any one of a number of distributed or locally accessible data storage media, such as hard disk drives, Blu-ray discs, digital video discs (DVDs), compact disc read-only memory (CD-ROMs), flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data.

[0065] In another example, the storage device 40 may correspond to a file server or another intermediate storage device capable of holding the encoded video generated by the encoding side 10. The decoding side 20 may access the stored video data from the storage device 40 by streaming transmission or download. The file server may be any type of server capable of storing the encoded video data and transmitting the encoded video data to the decoding side 20. In one example, the file server includes a network server (e.g., for a website), a file transfer protocol (FTP) server, a network attached storage (NAS) device, or a local disk drive. The decoding side 20 may access the encoded video data through any standard data connection (including an Internet connection). This may include a wireless channel (e.g., a wireless-fidelity (Wi-Fi) connection), a wired connection (e.g., a digital subscriber line (DSL), a cable modem, etc.), or a combination of both suitable for accessing the encoded video data stored in the file server. The transmission of the encoded video data from the storage device 40 may be a streaming transmission, a download transmission, or a combination of both.

[0066] The image decoding method according to the invention may be applied, for example, to terrestrial television broadcast, cable television transmission, satellite television transmission, streaming video transmission (e.g., via the Internet), encoding video data for storage on a data storage medium, decoding video data for storage on a data storage medium, or video encoding and decoding to support other applications. In some examples, the video coding system 1 may be used to support one-way or two-way video transmission, for example, to support applications such as video streaming transmission, video playback, video broadcasting and / or video telephony.

[0067] The video coding system 1 shown in FIG. 1 is only one example, and the techniques of the present invention may be applied to video coding settings (e.g., video encoding or video decoding) that do not necessarily include any data communication between an encoder and a decoder. In other examples, data may be retrieved from local memory, streamed over a network, etc. A video encoder may encode data and store it in memory, and / or a video decoder may retrieve data from memory and decode it. In many examples, encoding and decoding are performed by devices that simply encode data into memory and / or retrieve data from memory and decode it, without communicating with each other.

[0068] 1, encoding side 10 includes video source 120, video encoder 100, and output interface 140. In some examples, output interface 140 may include a modulator / demodulator (modem) and / or a transmitter. Video source 120 may include a video capture device (e.g., a camera), a video archive containing previously captured video data, a video feed interface for receiving video data from a video content provider, and / or a computer graphics system for generating video data, or a combination of these sources of video data.

[0069] Video encoder 100 may encode video data from a video source 120. In some embodiments, encoding side 10 transmits the encoded video data directly to decoding side 20 via output interface 140. In other examples, the encoded video data may be stored in storage device 40 for access by decoding side 20 for decoding and / or playback.

[0070] In the example of Fig. 1, the decoding side 20 includes an input interface 240, a video decoder 200, and a display device 220. In some examples, the input interface 240 includes a receiver and / or a modem. The input interface 240 may receive the encoded video data via the link 30 and / or from the storage device 40. The display device 220 may be integrated with the decoding side 20 or may be external to the decoding side 20. In general, the display device 220 displays the decoded video data. The display device 220 may include various display devices, such as a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or other types of display devices.

[0071] 1, in some aspects, the video encoder 100 and the video decoder 200 may be integrated with an audio encoder and decoder, respectively, and may include appropriate multiplexer-demultiplexer units or other hardware and software to accommodate encoding of both audio and video in a common data stream or separate data streams. In some examples, where applicable, the demultiplexer (MUX-DEMUX) units may conform to the ITU H.223 multiplexer protocol or other protocols, such as the user datagram protocol (UDP).

[0072] Each of the video encoder 100 and the video decoder 200 may be implemented, for example, as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, hardware, or any combination thereof. If the present invention is implemented in part in software, a device may implement the techniques of the present invention by storing instructions for the software in a suitable non-volatile computer-readable storage medium and executing the instructions in hardware using one or more processors. Any of the above (including hardware, software, a combination of hardware and software, etc.) may be considered as one or more processors. Each of the video encoder 100 and the video decoder 200 may be included in one or more encoders or decoders, and any of the encoders or decoders may be integrated as part of a composite encoder / decoder (codec) in the corresponding device.

[0073] In the present invention, the video encoder 100 may be generally referred to as another device that "signals" or "sends" certain information, for example to the video decoder 200. The terms "signaling" or "sending" may generally refer to the transmission of syntax elements and / or other data for decoding the compressed video data. This transmission may occur in real time or near real time. Alternatively, this communication may occur over a period of time, for example during encoding when storing syntax elements in the encoded bitstream to a computer-readable storage medium, and the decoding device may retrieve the syntax elements at any time after they are stored to the medium.

[0074] JCT-VC developed the H.265 (HEVC) standard. HEVC standardization is based on an evolution model of a video decoder called the HEVC Test Model (HEVC model, HM). The latest standard document of H.265 can be obtained from http: / / www.itu.int / rec / T-REC-H.265, and the latest version of the standard document is H.265(12 / 16), which is incorporated herein by full reference. HM assumes that the video decoder has some additional capabilities with respect to the existing algorithm of ITU-TH.264 / AVC. For example, H.264 provides nine intraframe predictive coding modes, and HM can provide up to 35 intraframe predictive coding modes.

[0075] JVET is working on the development of the H.266 standard. The H.266 standardization process is based on an evolution model of a video decoding device called the H.266 test model. The algorithm description of H.266 can be obtained from http: / phenix.int-evry.fr / jvet, and the latest algorithm description is included in JVET-F1001-v2, which is incorporated herein by reference in its entirety. At the same time, the reference software of the JEM test model can be obtained from https: / / jvet.hhi.fraunhofer.de / svn / svn_HMJEMSoftware / , which is also incorporated herein by reference in its entirety.

[0076] In general, the behavioral model description of the HM may divide a video frame or image into a sequence of treeblocks or largest coding units (LCUs), which contain both luma and chroma samples, and LCUs are also called coding tree units (CTUs). A treeblock has a similar purpose to a macroblock in the H.264 standard. A slice includes several treeblocks that are consecutive in a decoding order sequence. A video frame or image may be divided into one or more stripes. Each treeblock may be divided into coding units (CUs) based on a quadtree. For example, a treeblock, which is the root node of the quadtree, may be divided into four child nodes, and each child node may be further divided into other four child nodes as a parent node. The ultimately undividable child nodes, which are leaf nodes of the quadtree, include decoding nodes, such as decoded video blocks. Syntax data associated with the decoded bitstream may define the maximum number of times a treeblock may be divided, and may also define the minimum size of a decoding node.

[0077] The size of a CU corresponds to the size of a decoding node and must be square in shape. The size of a CU can range from 8x8 pixels up to 64x64 pixels or even larger, the size of a treeblock.

[0078] A video sequence typically includes a series of video frames or pictures. A group of pictures (GOP) illustratively includes a series of one or more video pictures. The GOP may include syntax data in the header information of the GOP, in the header information of one or more pictures, or elsewhere, that describes the number of pictures included in the GOP. Each slice of a picture may include slice syntax data that describes the coding mode of the corresponding picture. Video encoder 100 typically operates on video blocks within individual video slices to encode video data. The video blocks may correspond to decoding nodes within a CU. The video blocks may have a fixed or variable size, and may vary in size depending on a specified decoding standard.

[0079] In the present invention, "NxN" and "N by N" can be used interchangeably and refer to the pixel size of a video block according to the vertical and horizontal dimensions, for example, 16x16 pixels or 16 by 16 pixels. In general, a 16x16 block has 16 pixels (y=16) in the vertical direction and 16 pixels (x=16) in the horizontal direction. Similarly, an NxN block has N pixels in the vertical direction and N pixels in the horizontal direction, where N represents a non-negative integer value. The pixels in a block may be arranged in rows and columns. Also, a block does not necessarily have to have the same number of pixels in the horizontal and vertical directions. For example, a block may include NxM pixels, where M is not necessarily equal to N.

[0080] After intraframe / interframe predictive decoding of a CU is performed, video encoder 100 may calculate residual data of the CU. The CU may include pixel data in the spatial domain (also called pixel domain), and the CU may include coefficients in a transform domain after applying a transform (e.g., a discrete cosine transform (DCT), an integer transform, a discrete wavelet transform, or a conceptually similar transform) to the residual video data. The residual data may correspond to pixel differences between pixels of the uncoded image and predicted values ​​corresponding to the CU. Video encoder 100 may form a CU including the residual data and generate transform coefficients of the CU.

[0081] After generating the transform coefficients via any transformation, video encoder 100 may perform quantization of the transform coefficients. Quantization is illustratively a process in which coefficients are quantized to further compress them by reducing the amount of data to represent the coefficients. The quantization process may reduce the bit depth associated with some or all of the coefficients. For example, during quantization, an n-bit value may be truncated to an m-bit value, where n is greater than m.

[0082] In some possible embodiments, the video encoder 100 may scan the quantized transform coefficients in a predefined scan order to generate a serialized vector that may be entropy coded. In other possible embodiments, the video encoder 100 may perform an adaptive scan. After scanning the quantized transform coefficients to form a one-dimensional vector, the video encoder 100 may entropy code the one-dimensional vector based on a context-based adaptive variable-length code (CAVLC), a context-based adaptive binary arithmetic coding (CABAC), a syntax-based adaptive binary arithmetic coding (SBAC), a probability interval partitioning entropy (PIPE) coding, or other entropy coding method. The video encoder 100 may entropy code syntax elements associated with the coded video data for use by the video decoder 200 to decode the video data.

[0083] To perform CABAC, the video encoder 100 may assign a context in a context model to a transmitted symbol. The context may relate to whether the adjacent values ​​of the symbol are non-zero or not. To perform CAVLC, the video encoder 100 may select a variable-length code for the transmitted symbol. Codewords in variable-length code (VLC) may be configured such that shorter codes correspond to more likely symbols and longer codes correspond to less likely symbols. In this way, the use of VLC can save bitrate compared to using codewords of equal length for each transmitted symbol. Probabilities in CABAC can be determined based on the contexts assigned to the symbols.

[0084] In an embodiment of the present invention, the video encoder may perform inter-frame prediction to reduce temporal redundancy between images. In the present invention, the CU currently being decoded by the video decoder may be referred to as the current CU. In the present invention, the image currently being decoded by the video decoder may be referred to as the current image.

[0085] 2 is an exemplary block diagram of a video encoder according to the present invention. Video encoder 100 is used to output video to post-processing entity 41. Post-processing entity 41 represents an example of a video entity that can process the encoded video data from video encoder 100, such as a media aware network element (MANE) or a stitching / editing device. In some examples, post-processing entity 41 may be an example of a network entity. In some video encoding systems, post-processing entity 41 and video encoder 100 may be parts of separate devices, and in other cases, the functions described with respect to post-processing entity 41 may be performed by the same device that includes video encoder 100. In one example, post-processing entity 41 is an example of storage device 40 of FIG. 1.

[0086] In the example of FIG. 2, the video encoder 100 includes a prediction processing unit 108, a filter unit 106, a decoded picture buffer (DPB) 107, an adder 112, a transformer 101, a quantizer 102, and an entropy encoder 103. The prediction processing unit 108 includes an inter-frame predictor 110 and an intra-frame predictor 109. For image block reconstruction, the video encoder 100 further includes an inverse quantizer 104, an inverse transformer 105, and an adder 111. The filter unit 106 is intended to represent one or more loop filters, such as a deblocking filter, an adaptive loop filter (ALF), and a sample adaptive offset (SAO) filter. Although FIG. 2 illustrates the filter unit 106 as an in-loop filter, in other embodiments, the filter unit 106 may be implemented as an after-loop filter. In one example, the video encoder 100 may further include a video data memory, a splitting unit (not shown).

[0087] The video data memory may store video data to be encoded by the components of the video encoder 100. The video data stored in the video data memory may be obtained from the video source 120. The DPB 107 may be a reference image memory, storing reference video data for the video encoder 100 to encode the video data in intraframe and interframe coding modes. The video data memory and the DPB 107 may be formed by any one of a number of memory devices, for example, dynamic random access memory (DRAM), including synchronous dynamic random access memory (SDRAM), magnetic random access memory (MRAM), resistive random access memory (RRAM), or other types of memory devices. The video data memory and the DPB 107 may be provided by the same memory device or by separate memory devices.

[0088] As shown in FIG. 2, video encoder 100 receives video data and stores the video data in a video data memory. A partitioning unit partitions the video data into several image blocks, which may be further partitioned into smaller blocks, such as image block partitioning based on a quadtree structure or a binary tree structure. The partitioning may include partitioning into slices, tiles, or other larger units. Video encoder 100 generally refers to a component that encodes image blocks in a video slice to be encoded. The slice may be partitioned into a number of image blocks (or into sets of image blocks called slices). Prediction processing unit 108 may select one of a number of possible coding modes to be used for the current image block, for example, one of a number of intraframe coding modes or one of a number of interframe coding modes. Prediction processing unit 108 may provide the intraframe, interframe coded block to adder 112 to generate a residual block, and to adder 111 to reconstruct the coded block to be used as a reference image.

[0089] An intraframe predictor 109 in prediction processing unit 108 may perform intraframe predictive coding of the current image block relative to one or more neighboring blocks in the same frame or slice as the current block being coded to remove spatial redundancy. An interframe predictor 110 in prediction processing unit 108 may perform interframe predictive coding of the current image block relative to one or more predictive blocks in one or more reference images to remove temporal redundancy.

[0090] Specifically, the inter-frame predictor 110 may be used to determine an inter-frame prediction mode to be used for encoding a current image block. For example, the inter-frame predictor 110 may calculate a bitrate-distortion value for each inter-frame prediction mode in a set of candidate inter-frame prediction modes through a bitrate-distortion analysis, and select an inter-frame prediction mode having optimal bitrate-distortion characteristics therefrom. The bitrate-distortion analysis typically determines the amount of distortion (or error) between an encoded block and an original uncoded block that is encoded to generate the encoded block, and the bit code rate (i.e., the number of bits) for generating the encoded block. For example, the inter-frame predictor 110 may determine an inter-frame prediction mode in the set of candidate inter-frame prediction modes that has the smallest bitrate-distortion cost for encoding the current image block as the inter-frame prediction mode for performing inter-frame prediction on the current image block.

[0091] The inter-frame predictor 110 is used to predict the motion information (e.g., motion vector) of one or more sub-blocks in the current image block based on the determined inter-frame prediction mode, and obtain or generate a prediction block of the current image block using the motion information (e.g., motion vector) of one or more sub-blocks in the current image block. The inter-frame predictor 110 may locate the prediction block pointed to by the motion vector in one of the reference image lists. The inter-frame predictor 110 may further generate syntax elements related to the image block and the video slice for use when the video decoder 200 decodes the image block of the video slice. Or, in one example, the inter-frame predictor 110 obtains the prediction block of the current image block by performing a motion compensation process using the motion information of each sub-block to generate a prediction block of each sub-block. Note that here, the inter-frame predictor 110 performs a motion estimation and motion compensation process.

[0092] Specifically, after selecting an inter prediction mode for the current image block, the inter predictor 110 may provide information indicating the selected inter prediction mode for the current image block to the entropy encoder 103, such that the entropy encoder 103 encodes the information indicating the selected inter prediction mode.

[0093] The intra predictor 109 may perform intra prediction for the current image block. Specifically, the intra predictor 109 may determine an intra prediction mode for encoding the current block. For example, the intra predictor 109 may use a bit rate-distortion analysis to calculate a bit rate-distortion value for each tested intra prediction mode, and select an intra prediction mode having optimal bit rate-distortion characteristics from the tested modes. In any case, after selecting an intra prediction mode for the image block, the intra predictor 109 may provide information indicating the selected intra prediction mode for the current image block to the entropy encoder 103, such that the entropy encoder 103 encodes the information indicating the selected intra prediction mode.

[0094] After prediction processing unit 108 generates a prediction block for a current image block by inter-frame prediction, intra-frame prediction, video encoder 100 subtracts the prediction block from the current image block to be coded to form a residual image block. Adder 112 represents one or more components that perform this subtraction. The residual video data in the residual block may be included in one or more transform units (TUs) and applied to a transformer 101. The transformer 101 converts the residual video data into residual transform coefficients using a transform, such as a discrete cosine transform (DCT) or a conceptually similar transform. The transformer 101 may convert the residual video data from a pixel value domain to a transform domain, such as a frequency domain.

[0095] The transformer 101 may send the resulting transform coefficients to the quantizer 102, which quantizes the transform coefficients to further reduce the bit-code rate. In some examples, the quantizer 102 may then perform a scan on a matrix containing the quantized transform coefficients. Alternatively, the entropy encoder 103 may perform the scan.

[0096] After quantization, the entropy encoder 103 entropy codes the quantized transform coefficients. For example, the entropy encoder 103 may perform context-adaptive variable length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), syntax-based adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or other entropy coding methods or techniques. After being entropy coded by the entropy encoder 103, the coded bitstream may be transmitted to the video decoder 200 or archived for later transmission or retrieval by the video decoder 200. The entropy encoder 103 may entropy code syntax elements of the current image block being coded.

[0097] The inverse quantizer 104 and the inverse transformer 105 apply inverse quantization and inverse transformation, respectively, to reconstruct the residual block in the pixel domain, e.g., to provide a reference block for later use as a reference image. The adder 111 adds the reconstructed residual block to a prediction block generated by the inter-frame predictor 110 or the intra-frame predictor 109 to generate a reconstructed image block. A filter unit 106 may be applied to the reconstructed image block to reduce artifacts, such as block artifacts. The reconstructed image block is then stored in the decoded image buffer 107 as a reference block, and may be used as a reference block by the inter-frame predictor 110 to perform inter-frame prediction on blocks in subsequent video frames or images.

[0098] It should be noted that other structural changes of the video encoder 100 may be used to encode the video stream. For example, for some image blocks or image frames, the video encoder 100 may directly quantize the residual signal without needing to be processed by the transformer 101, and accordingly without needing to be processed by the inverse transformer 105; for some image blocks or image frames, the video encoder 100 may not generate residual data, and accordingly without needing to be processed by the transformer 101, the quantizer 102, the inverse quantizer 104, and the inverse transformer 105; or the video encoder 100 may directly store the reconstructed image block as a reference block without needing to be processed by the filter unit 106; or the quantizer 102 and the inverse quantizer 104 in the video encoder 100 may be combined.

[0099] 3 is an exemplary block diagram of a video decoder 200 according to the present invention. In the example of FIG. 3, the video decoder 200 includes an entropy decoder 203, a prediction processing unit 208, an inverse quantizer 204, an inverse transformer 205, an adder 211, a filter unit 206, and a DPB 207. The prediction processing unit 208 may include an inter-frame predictor 210 and an intra-frame predictor 209. In some examples, the video decoder 200 may perform a decoding process that is approximately the reverse of the encoding process described for the video encoder 100 of FIG. 2.

[0100] In the decoding process, the video decoder 200 receives from the video encoder 100 an encoded video bitstream representing image blocks and associated syntax elements of encoded video slices. The video decoder 200 may receive video data from a network entity 42 and may optionally store the video data in a video data memory (not shown). The video data memory may store video data to be decoded by components of the video decoder 200, e.g., an encoded video bitstream. The video data stored in the video data memory may be obtained, for example, by accessing the storage device 40, a local video source such as a camera, a wired or wireless network communication via video data, or a physical data storage medium. The video data memory may function as a decoded picture buffer (CPB) for storing encoded video data from the encoded video bitstream. Thus, although the video data memory is not shown in FIG. 3, the video data memory and the DPB 207 may be the same memory or may be separately provided memories. The video data memory and DPB 207 may be formed, for example, by any one of a number of memory devices, such as dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magnetic RAM (MRAM), resistive RAM (RRAM), or other types of memory devices.

[0101] Network entity 42 may be a server, a MANE, a video editor / cutter, or other device for implementing one or more of the techniques described above. Network entity 42 may or may not include a video encoder, such as video encoder 100. Before network entity 42 transmits the encoded video bitstream to video decoder 200, network entity 42 may implement some of the techniques described in this invention. In some video decoding systems, network entity 42 and video decoder 200 may be parts of separate devices, and in other cases, the functions described with respect to network entity 42 may be performed by the same device that includes video decoder 200. In some cases, network entity 42 may be an example of storage device 40 of FIG. 1.

[0102] The entropy decoder 203 of the video decoder 200 entropy decodes the bitstream to generate quantized coefficients and some syntax elements. The entropy decoder 203 forwards the syntax elements to the prediction processing unit 208. The video decoder 200 may receive the syntax elements at the video slice level and / or the image block level.

[0103] If the video slice is decoded as an intra-frame decoded (I) slice, the intra-frame predictor 209 of the prediction processing unit 208 may generate a prediction block of the image block of the current video slice based on the intra-frame prediction mode signaled by the signaling and data from a previously decoded block of the current frame or image. If the video slice is decoded as an inter-frame decoded (i.e., B or P) slice, the inter-frame predictor 210 of the prediction processing unit 208 may determine an inter-frame prediction mode for decoding the current image block of the current video slice based on the syntax element received from the entropy decoder 203, and decode the current image block (e.g., perform inter-frame prediction) based on the determined inter-frame prediction mode. Specifically, the inter predictor 210 may determine whether to use a new inter prediction mode to predict a current image block of a current video slice, and if the syntax element indicates to use a new inter prediction mode to predict the current image block, may predict motion information of a current image block or a sub-block of the current image block of a current video slice based on the new inter prediction mode (e.g., a new inter prediction mode specified by the syntax element or a default new inter prediction mode), thereby obtaining or generating a prediction block of the current image block or the sub-block of the current image block using the motion information of the current image block or the sub-block of the current image block predicted by the motion compensation process. Here, the motion information may include reference image information and a motion vector, and the reference image information may include, but is not limited to, unidirectional / bidirectional prediction information, a reference image list number, and a reference image index corresponding to the reference image list. For inter prediction, a prediction block may be generated from one of the reference images in one of the reference image lists. The video decoder 200 may construct the reference image lists, i.e., list 0 and list 1, based on the reference images stored in the DPB 207. The reference frame index of the current image may be included in one or more of reference frame list 0 and list 1.In some examples, video encoder 100 may signal a notification indicating whether or not to use a new inter prediction mode to decode a particular syntax element of a particular block, or may signal a notification indicating whether or not to use a new inter prediction mode and which new inter prediction mode specifically to use to decode a particular syntax element of a particular block, where inter predictor 210 performs a motion compensation process.

[0104] The inverse quantizer 204 inverse quantizes, or dequantizes, the quantized transform coefficients provided to the bitstream and decoded by the entropy decoder 203. The inverse quantization process may include using a quantization parameter calculated by the video encoder 100 for each image block in a video slice to determine the degree of quantization to be applied, and similarly determining the degree of inverse quantization to be applied. The inverse transformer 205 applies an inverse transform, such as an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process, to the transform coefficients to produce residual blocks in the pixel domain.

[0105] After the inter-frame predictor 210 generates a prediction block for the current image block or a sub-block of the current image block, the video decoder 200 adds the residual block from the inverse transformer 205 and the corresponding prediction block generated by the inter-frame predictor 210 to obtain a reconstructed block, i.e., a decoded image block. The adder 211 represents a component that performs this addition. If necessary, a loop filter (in the decoding loop or after the decoding loop) may be used to smooth the pixel transformation or otherwise improve the video quality. The filter unit 206 may represent one or more loop filters, such as a deblocking filter, an adaptive loop filter (ALF), and a sample adaptive offset (SAO) filter. Although the filter unit 206 is illustrated in FIG. 3 as an in-loop filter, in other embodiments, the filter unit 206 may be implemented as an after-loop filter. In one example, the filter unit 206 is applied to the block reconstruction to reduce blockiness, and the result is output as a decoded video stream. Decoded image blocks within a given frame or image may also be stored in a decoded image buffer 207, and reference images for subsequent motion compensation may be stored via DPB 207. DPB 207 may be part of or separate from memory that may store decoded video for subsequent display on a display device (e.g., display device 220 of FIG. 1).

[0106] It should be noted that other structural changes of the video decoder 200 may be used to decode the encoded video bitstream, for example, the video decoder 200 may generate an output video stream without being processed by the filter unit 206, or for some image blocks or image frames, the entropy decoder 203 of the video decoder 200 may not decode quantized coefficients, which do not need to be processed by the inverse quantizer 204 and the inverse transformer 205 accordingly.

[0107] The techniques of the present invention may be performed by any one of the video encoders or video decoders described in the present invention, such as the video encoder 100 and video decoder 200 shown and described in Figures 1-3. That is, in one possible embodiment, the video encoder 100 described in Figure 2 may perform certain techniques described below when performing inter-frame prediction during encoding of a block of video data. In another possible embodiment, the video decoder 200 described in Figure 3 may perform certain techniques described below when performing inter-frame prediction during decoding of a block of video data. Thus, references to a generic "video encoder" or "video decoder" may include the video encoder 100, the video decoder 200, or another video encoding or decoding unit.

[0108] 1-3 are only examples provided in the present invention, and in some examples, the video encoder 100, the video decoder 200, and the video coding system may include more or fewer components or units.

[0109] Hereinafter, based on the video coding system shown in Figures 1 to 3, the present invention provides a possible embodiment of video encoding / decoding, as shown in Figure 4, Figure 4 is a schematic flowchart of video encoding / decoding according to the present invention, the embodiment of the video encoding / decoding includes processes (1) to (5), and the processes (1) to (5) may be performed by any one or more of the above-mentioned encoding side 10, video encoder 100, decoding side 20, or video decoder 200.

[0110] Process (1): Divide one frame of an image into one or more non-overlapping parallel encoding units, which are not dependent on each other and can be encoded and decoded completely in parallel / independently, such as parallel encoding unit 1 and parallel encoding unit 2 shown in Figure 4.

[0111] Process (2): For each parallel coding unit, it may be further divided into one or more non-overlapping independent coding units, each of which may be independent of each other but may share some parallel coding unit header information.

[0112] For example, the width of an independent coding unit is w_lcu and the height is h_lcu. If a parallel coding unit is split into one independent coding unit, the size of the independent coding unit is exactly the same as the parallel coding unit, otherwise the width of the independent coding unit must be greater than the height (except for edge regions).

[0113] Typically, the independent coding unit may be a fixed w_lcu×h_lcu, where w_lcu and h_lcu are both 2 to the power of N (N≧0), for example, the size of the independent coding unit is 128×4, 64×4, 32×4, 16×4, 8×4, 32×2, 16×2, or 8×2, etc.

[0114] As one possible example, the independent coding units may be fixed at 128×4. If the size of the parallel coding units is 256×8, the parallel coding units may be divided evenly into four independent coding units, and if the size of the parallel coding units is 288×10, the parallel coding units are divided into two 128×4+one 32×4 independent coding units in the first / second row and two 128×2+one 32×2 independent coding units in the third row.

[0115] It should be noted that the independent coding unit may include three components of luminance Y, chrominance Cb, and chrominance Cr, or three components of red (R), green (G), and blue (B), or may include only one of these components. When the independent coding unit includes three components, the sizes of these three components may be completely the same or different, which is specifically related to the input format of the image.

[0116] Process (3): For each independent coding unit, it may be further divided into one or more non-overlapping coding units, and each coding unit within the independent coding unit may be dependent on each other, for example, multiple coding units may refer to each other to perform pre-encoding and pre-decoding.

[0117] If the size of the coding unit and the independent coding unit is the same (i.e., the independent coding unit is divided into only one coding unit), the size may be any size described in process (2).

[0118] When an independent coding unit is divided into multiple non-overlapping coding units, possible examples of division include equal horizontal division (where the height of the coding unit is the same as the independent coding unit but the width is different, which may be 1 / 2, 1 / 4, 1 / 8, 1 / 16, etc.), equal vertical division (where the width of the coding unit is the same as the independent coding unit but the height is different, which may be 1 / 2, 1 / 4, 1 / 8, 1 / 16, etc.), equal horizontal and vertical division (quadtree division), etc., with equal horizontal division being preferred.

[0119] If the coding unit has a width w_cu and a height h_cu, the width must be greater than the height (except for edge regions). In general, the coding unit may be a fixed w_cu×h_cu, where w_cu and h_cu are both 2 to the power of N (N is 0 or greater), such as 16×4, 8×4, 16×2, 8×2, 8×1, 4×1, etc.

[0120] As one possible example, the coding unit may be a fixed 16x4. If the size of the independent coding unit is 64x4, the independent coding unit may be divided evenly into four coding units, and if the size of the independent coding unit is 72x4, it is divided into four 16x4 + one 8x4 coding units.

[0121] Note that the coding unit may include three components of luminance Y, chrominance Cb, and chrominance Cr (or three components of red R, green G, and blue B), or may include only one of these components. When including three components, the sizes of these components may be completely the same or different, and are specifically related to the input format of the image.

[0122] Note that process (3) is an optional step in the video encoding / decoding method, and the video encoder / decoder may perform encoding / decoding on the residual coefficients (or residual values) of the independent coding units obtained in process (2).

[0123] Process (4): For the coding unit, it may be further divided into one or more non-overlapping prediction groups (PG), where PG may be abbreviated as Group, and each PG is encoded and decoded according to a selected prediction mode, and a predicted value of the PG is obtained to form a predicted value of the entire coding unit, and a residual value of the coding unit is obtained based on the predicted value and the original value of the coding unit.

[0124] Process (5): According to the residual value of the coding unit, the coding unit is grouped, one or more non-overlapping residual blocks (RBs) are obtained, and the residual coefficients of each RB are coded and decoded according to the selected mode to form a residual coefficient stream. Specifically, the residual coefficients can be divided into those that perform transformation and those that do not perform transformation.

[0125] Here, the selected mode of encoding and decoding the residual coefficients in process (5) may include, but is not limited to, any of a semi-fixed-length coding mode, an exponential Golomb coding method, a Golomb-Rice coding method, a truncated unary coding method, a run-length coding method, a method of directly coding the original residual values, etc.

[0126] For example, the video encoder may directly encode the coefficients in the RBs.

[0127] Also, for example, the video encoder may perform a transform such as DCT, DST, or Hadamard transform on the residual block, and then encode the transformed coefficients.

[0128] As one possible example, when the RB is relatively small, the video encoder may directly perform uniform quantization on each coefficient in the RB and then perform binary encoding. When the RB is relatively large, the video encoder may further divide the RB into multiple coefficient groups (CGs), and perform uniform quantization on each CG and then perform binary encoding. In some embodiments of the present invention, the coefficient groups (CGs) and the quantization groups (QGs) may be the same.

[0129] Hereinafter, the coding of residual coefficients in the semi-fixed length coding method will be described as an example. First, the maximum value of the residual absolute value in one RB block is defined as a modified maximum (mm). Next, the number of coding bits of the residual coefficients in the RB block is determined (the number of coding bits of the residual coefficients in the same RB block is the same). For example, if the critical limit (CL) of the current RB block is 2 and the current residual coefficient is 1, 2 bits are required to code the residual coefficient 1, which is represented as 01. If the CL of the current RB block is 7, this represents coding an 8-bit residual coefficient and a 1-bit code bit. The determination of CL is to find the minimum M value that satisfies that all the residuals of the current sub-block are within the range of [-2^(M-1), 2^(M-1)]. If the two boundary values, -2^(M-1) and 2^(M-1), exist simultaneously, M needs to be increased by 1, i.e., M+1 bits are required to code all residuals of the current RB block; if only one of the two boundary values, -2^(M-1) and 2^(M-1), exists, one Trailing bit needs to be coded to determine whether the boundary value is -2^(M-1) or 2^(M-1); if neither -2^(M-1) nor 2^(M-1) exists in all residuals, there is no need to code the Trailing bit.

[0130] Also, for some special cases, the video encoder may directly encode the original values ​​of the image instead of the residual values.

[0131] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0132] FIG. 5 is a schematic flowchart of an image decoding method according to the present invention, which may be applied to the video coding system shown in FIG. 1, and the image decoding method may be performed by the decoding side 20. Specifically, the decoding method may be performed by the video decoder 200 included in the decoding side 20. Referring to FIG. 5, the decoding method according to this embodiment includes the following steps:

[0133] At S510, the video decoder 200 parses the obtained bitstream to obtain one or more image frames.

[0134] Here, the one image frame includes one or more CUs.

[0135] As shown in Fig. 5, one image frame may include 3 x 5 = 15 CUs. Note that one image frame may include more CUs, such as 20 CUs, and one image frame may include fewer CUs, such as 1 CU or 2 CUs.

[0136] At S520, the video decoder 200 determines multiple QP values ​​for an image frame.

[0137] Here, one CU includes multiple QGs, and one QG corresponds to one QP value.

[0138] The one CU includes a plurality of residual coefficients, the one QG includes some of the residual coefficients, and the some residual coefficients share one QP value.

[0139] It should be noted that in the encoding process, after the video encoder quantizes the residual coefficient, the residual coefficient is also called a level value. In this field, the residual coefficient and the level value may be collectively called the residual coefficient, and in this embodiment, both are represented by the residual coefficient, which should not be understood as a limitation on the present invention.

[0140] Compared with the large image distortion caused by all residual coefficients in one CU using the same QP value, in the video image decoding process provided in this embodiment, one CU may be divided into multiple QGs, and one or more residual coefficients in each QG share one QP value, so that the video decoder can make finer granularity QP decisions for one or more CUs corresponding to an image frame, thereby reducing the decoding distortion of the image frame and improving the authenticity and accuracy of video image decoding while ensuring a certain compression rate.

[0141] In one optional embodiment, a QG may include one pixel point or multiple pixel points, each of which has a corresponding residual coefficient.

[0142] In the first possible case, when one QG contains one pixel point, the quantization process of the QG may also be called adaptive point-by-point quantization of the image frame, and the point-by-point quantization may be applied to a CU that does not perform QG division. The feature of point-by-point quantization is that each pixel point is allowed to use a different QP, which is equivalent to refining the granularity of the QP to the pixel level. In this way, by inverse quantizing the image frame using the method of point-by-point quantization, the subjective quality of the image frame can be significantly improved without changing the compression rate of the image frame.

[0143] In the second possible case, when one QG contains multiple pixel points, the residual coefficients corresponding to multiple pixel points contained in one QG may share one QP value.

[0144] In another optional embodiment, the CU may be determined by the video decoder 200 dividing one CU included in one image frame according to the first rule to obtain multiple QGs.

[0145] The first rule includes a division region and a division scheme, where the division region is a transformation domain or a pixel domain, and the division scheme includes at least one of uniform division and non-uniform division.

[0146] The positions of the residual coefficients included in one CU are marked with coordinates, and the mark may include an abscissa and an ordinate. For example, the position coordinates of the residual coefficient are (i,j), where i is the abscissa and j is the ordinate. Below, some possible examples are provided to explain the first rule and the division of the QG in the CU.

[0147] In a first possible example, when the type of the division region is a transform region, the process of the video decoder 200 dividing one CU to obtain multiple QGs includes: dividing the residual coefficients whose coordinate sum does not reach a first coordinate threshold among the multiple residual coefficients into a first QG, and dividing the residual coefficients whose coordinate sum reaches the first coordinate threshold into a second QG, where the coordinate sum is the sum of the horizontal coordinate and the vertical coordinate of the residual coefficient.

[0148] FIG. 6 is a schematic diagram of the transform domain division according to the present invention, in which one CU includes an area corresponding to 16 residual coefficients, the coordinates of the residual coefficient in the upper left corner are (1, 1) and the coordinates of the residual coefficient in the lower right corner are (4, 4).

[0149] QG(2-1) indicates a QG bisection method of the transform domain, where the residual coefficient at the (i,j) position is the first QG if it satisfies "i+j≦threshold 1" and the rest is the second QG, for example, the threshold 1 is 5.5. Alternatively, the residual coefficient that satisfies "i+j<threshold 1" may be the first QG, and the rest may be the second QG.

[0150] In a second possible example, when the type of the split region is a transform region, the process by which video decoder 200 splits one CU to obtain a plurality of QGs includes dividing, among a plurality of residual coefficients, the residual coefficients whose coordinate sum has not reached a first coordinate threshold into a first QG, dividing the residual coefficients whose coordinate sum has reached the first coordinate threshold but has not reached a second coordinate threshold into a second QG, and dividing the residual coefficients whose coordinate sum has reached the second coordinate threshold into a third QG. The second coordinate threshold is greater than the first coordinate threshold.

[0151] As shown in FIG. 6, QG(3-1) shows a three-way split method of QG in the transform region. For the residual coefficient at position (i,j), the residual coefficients satisfying "i + j < threshold 1" are the first QG, the residual coefficients satisfying "threshold 1 ≤ i + j < threshold 2" are the second QG, and the residual coefficients satisfying "i + j ≥ threshold 2" are the third QG. Threshold 2 is greater than threshold 1. For example, threshold 1 = 5.5 and threshold 2 = 6.5.

[0152] Alternatively, the residual coefficients satisfying "i + j ≤ threshold 1" may be the first QG, the residual coefficients satisfying "threshold 1 < i + j ≤ threshold 2" may be the second QG, and the residual coefficients satisfying "i + j > threshold 2" may be the third QG, where threshold 2 is greater than threshold 1. For example, threshold 1 = 5 and threshold 2 = 6.

[0153] In a third possible example, when the type of the split region is a transform region, the process by which video decoder 200 splits one CU to obtain a plurality of QGs includes sorting a plurality of residual coefficients and dividing, among the plurality of residual coefficients, the residual coefficients that have not reached a first proportional threshold into a first QG and dividing the residual coefficients that have reached the first proportional threshold into a second QG. The sorting method of the plurality of residual coefficients is either zigzag or reverse zigzag.

[0154] For example, as shown in FIG. 6, QG(2-2) shows a zigzag two-way split method of QG in the transform region. The first 7 / 16 (43.75%) of the residual coefficients are divided into the first QG, and the remaining residual coefficients are divided into the second QG.

[0155] Also, for example, as shown in FIG. 6, QG(2-3) indicates a QG inverse zigzag bipartitioning scheme of the transform domain, in which the first 7 / 16 (43.75%) of the residual coefficients are divided into the first QG, and the remaining residual coefficients are divided into the second QG.

[0156] In a fourth possible example, when the type of the division region is a transform region, the process of the video decoder 200 dividing one CU to obtain multiple QGs includes: sorting the multiple residual coefficients, dividing the multiple residual coefficients that do not reach a first proportional threshold into a first QG, dividing the multiple residual coefficients that reach the first proportional threshold but do not reach a second proportional threshold into a second QG, and dividing the multiple residual coefficients that reach the second proportional threshold into a third QG, where the second proportional threshold is greater than the first proportional threshold.

[0157] For example, as shown in FIG. 6, QG(3-2) indicates a QG zigzag three-way division scheme of the transform domain, in which the first 5 / 16 (31.25%) of the residual coefficients are divided into the first QG, the last 3 / 16 (18.75%) of the residual coefficients are divided into the second QG, and the remaining residual coefficients are divided into the third QG.

[0158] Also, for example, as shown in Figure 6, QG(3-3) indicates a QG inverse zigzag three-way division scheme of the transform domain, in which the first 5 / 16 (31.25%) of the residual coefficients are divided into the first QG, the last 3 / 16 (18.75%) of the residual coefficients are divided into the second QG, and the remaining residual coefficients are divided into the third QG.

[0159] The above first to fourth possible examples are merely examples provided by this embodiment to explain the division of the transform region in the CU, and should not be understood as limitations on the present invention. When the type of the divided region is the transform region, the selection of the coordinate threshold value and the proportional threshold value may be determined according to the image content of the CU or the needs of video encoding and decoding, and the present invention is not limited thereto. In addition, in order to improve the QP determination accuracy in one CU, the transform region may be divided into more QGs, such as 4, 5, 10, or more, and the present invention is not limited thereto.

[0160] When the type of the division region is a pixel region, some possible examples are provided below to explain the first rule and the division of the QG within a CU.

[0161] In a fifth possible example, the video decoder 200 performs a horizontal or vertical symmetric division on the residual coefficients to obtain two QGs containing the same number of residual coefficients, and the ratio of the residual coefficients contained in these two QGs is 1:1.

[0162] FIG. 7 is a schematic diagram of pixel region division according to the present invention, where QG_pixel (pixel, P) (2-1) shows an example of symmetrical bisection of a CU in the vertical direction, and QG_P (2-4) shows an example of symmetrical bisection of a CU in the horizontal direction.

[0163] In a sixth possible example, the video decoder 200 performs a horizontal or vertical symmetric division on the residual coefficients to obtain three QGs, in which the number of residual coefficients included in two non-adjacent QGs is the same, and the number of residual coefficients included in another QG is the same as the sum of the number of residual coefficients included in the two non-adjacent QGs.

[0164] As shown in Fig. 7, QG_P(3-1) is an example of a symmetrical division of a CU into three parts in the vertical direction, where the number of residual coefficients included in the QGs on both sides is the same, and the ratio of the number of residual coefficients included in these three QGs is 1:2:1. QG_P(3-4) is an example of a symmetrical division of a CU into three parts in the horizontal direction, where the ratio of the number of residual coefficients included in these three QGs is 1:2:1.

[0165] In a seventh possible example, the video decoder 200 performs a horizontal or vertical split on the residual coefficients to obtain two QGs that contain disparate numbers of residual coefficients.

[0166] For example, QG_P(2-2) shown in FIG. 7 indicates an example in which a CU is divided into two in the vertical direction, and the ratio of the residual coefficients included in these two QGs is 1:3.

[0167] Also, for example, QG_P(2-3) indicates an example in which a CU is divided into two in the vertical direction, and the ratio of the residual coefficients included in these two QGs is 3:1.

[0168] Also, for example, QG_P(2-5) indicates an example in which a CU is divided into two in the horizontal direction, and the ratio of the residual coefficients included in these two QGs is 1:3.

[0169] Also, for example, QG_P(2-6) indicates an example in which a CU is divided into two in the horizontal direction, and the ratio of the residual coefficients included in these two QGs is 3:1.

[0170] In an eighth possible example, the video decoder 200 performs a horizontal or vertical division on the multiple residual coefficients to obtain three QGs, and the residual coefficients included in the three QGs do not have a symmetrical relationship.

[0171] For example, QG_P(3-2) shown in FIG. 7 indicates an example in which a CU is divided into three in the vertical direction, and the ratio of residual coefficients included in these three QGs is 1:1:2.

[0172] Also, for example, QG_P(3-3) indicates an example in which a CU is divided into three in the vertical direction, and the ratio of residual coefficients included in these three QGs is 2:1:1.

[0173] Also, for example, QG_P(3-5) indicates an example in which a CU is divided into three in the horizontal direction, and the ratio of residual coefficients included in these three QGs is 1:1:2.

[0174] Also, for example, QG_P(3-6) indicates an example in which a CU is divided into three in the horizontal direction, and the ratio of residual coefficients included in these three QGs is 2:1:1.

[0175] The above fifth to eighth possible examples are merely examples provided by the present embodiment to explain the division of pixel regions in a CU, and should not be understood as limitations on the present invention. When the type of the division region is a pixel region, the ratio of residual coefficients included in the QG may be determined according to the image content or video encoding and decoding needs of the CU, and the present invention is not limited thereto. In addition, in order to improve the QP determination accuracy in one CU, the pixel region may be divided into more QGs, such as 4, 5, 10, or more, and the present invention is not limited thereto.

[0176] In this way, the video decoder 200 adopts different QG division methods for the residual coefficients of the pixel domain and the transform domain, and the QP quantization process is also different, so that the distortion of the image decoding is reduced. The QP quantization process is described in the description of the QP value acquisition process in this specification, so it will not be repeated here.

[0177] Continuing to refer to FIG. 5, after the video decoder 200 obtains the multiple QP values ​​of an image frame, the image decoding method according to this embodiment further includes the following step S530.

[0178] At S530, the video decoder 200 decodes an image frame based on the multiple QP values.

[0179] For example, the video decoder 200 obtains a decoded image after decoding one image frame based on multiple QP values, or obtains a video after decoding a bitstream after the video decoder 200 decodes multiple image frames.

[0180] In this way, in this embodiment, the video decoder can perform QP quantization of image decoding with QG as the granularity, and one CU may be divided into multiple QGs, and one QG corresponds to one QP value, so compared with the large image distortion caused by all residual coefficients in one CU using the same QP value, in this embodiment, the video decoder can make a finer granularity QP decision for one or more CUs corresponding to an image frame, thereby reducing the decoding distortion of the image frame while ensuring a certain compression rate, and improving the authenticity and accuracy of video image decoding.

[0181] Based on the technology provided in the present invention, in the video encoding / decoding process, the video encoder / video decoder obtains a QP value for each QG (or CU), and here we provide several possible embodiments.

[0182] In one possible embodiment, one QP value corresponding to one QG includes a luma QP value and a chroma QP value. The luma QP value (QP_Y) is a QP value required to quantize or dequantize the luma (Luminance or Luma) of an image frame, and the chroma QP value is a QP value required to quantize or dequantize the chroma (Chrominance or Chroma) of an image frame. Exemplarily, for the above S520, the video decoder 200 determining multiple QP values ​​of one image frame may include the following two possible cases:

[0183] In the first possible case, the video decoder 200 obtains the luma QP value and the chroma QP value of one QG, respectively.

[0184] In a second possible case, first, the video decoder 200 obtains the luma QP value of one QG, and then the video decoder 200 determines the chroma QP value of one QG based on the luma QP value. For example, the chroma QP value is QP_Y plus the QP offset value of the picture parameter set (PPS) layer and the Slice layer.

[0185] Exemplarily, the above luma QP value and chroma QP value may be obtained by the video decoder 200 analyzing the bitstream.

[0186] When making a QP decision (also called QP quantization) for a QG contained in a CU, the video decoder 200 may first obtain a QP value at the CU level, and then obtain a QP value at the QG level within the CU.

[0187] Below, two possible embodiments for determining multiple QP values ​​for one image frame are provided.

[0188] First possible embodiment, direct encoding / decoding: the video decoder 200 parses the bitstream to obtain mark information of one image frame, and the mark information is used to indicate a QP value of one QG and / or the mark information is used to indicate a QP value of one CU.

[0189] For example, in near-lossless compression, since the probability of a small QP value appearing is higher than that of a large QP value, the video decoder 200 may inverse quantize the residual coefficient (level value) included in the QG using a QP value included in the mark information added to the bitstream in a method such as a truncated unary code, a truncated Rice code, or an exponential Golomb code.

[0190] In this way, when the video encoder 200 makes a QP decision for one CU, the QP value of the CU may be added to the mark information. If there is no need to make a QP decision for a QG in a CU, the video decoder 200 may determine the QP value of the CU based on the mark information added to the bitstream, which avoids the video decoder from inferring image frames in the bitstream to obtain the QP value of the CU, reduces the consumption of the computational resources of the video decoder, and improves the efficiency of image decoding.

[0191] Also, for example, when the video encoder 200 determines the QP for a plurality of QGs ​​in one CU, the QP value of any one of the plurality of QGs ​​may be added to the mark information.

[0192] Furthermore, for example, the QP value of the CU and the QP value of any QG within the CU may be added to the mark information.

[0193] In this way, when the video encoder 200 makes a QP decision for a QG, the mark information may be appended with the QP value of the QG, which avoids the video decoder from inferring image frames in the bitstream to obtain the QG value of the CU, reduces the consumption of computational resources of the video decoder, and improves the efficiency of image decoding.

[0194] Second possible embodiment, predictive encoding / decoding: the video decoder 200 encodes the difference (deltaQP) between the actual (encoded) QP value and the predicted QP (predict QP, predQP) value, which includes first obtaining the predQP of the current block (CU or QG), then determining the deltaQP of the current block, and finally determining that the actual (encoded) QP value is QP=predQP+deltaQP.

[0195] The following takes the video decoder 200 determining a QP value of one QG in one image frame as an example to describe in detail. As shown in FIG. 8, FIG. 8 is a schematic flowchart of obtaining a QP in the predictive coding method according to the present invention, and the QP obtaining process may be realized by a video decoder or a video encoder. Here, the video decoder 200 is taken as an example to describe, and the QP obtaining process includes the following steps:

[0196] At S810, the video decoder 200 obtains a predicted QP value of QG.

[0197] In one possible embodiment, the video decoder 200 determines the QP value of the CU where the QG is located as the predicted QP value of the QG. Exemplarily, the QP value of the CU may be determined by the video decoder 200 analyzing mark information of the bitstream.

[0198] In another possible embodiment, the video decoder 200 may first obtain the QP value of at least one other QG adjacent to the QG in the CU where the QG is located, and determine the predicted QP value of the QG according to the QP value of the at least one other QG. For example, the video decoder 200 determines the QP value of the other QG adjacent to the QG as the predicted QP value of the QG.

[0199] At S820, the video decoder 200 obtains the QP offset of QG.

[0200] In this specification, the QP offset may be expressed as deltaQP.

[0201] Continuing to refer to FIG. 8, for S820, this embodiment provides two possible embodiments.

[0202] In a first optional embodiment, as shown in FIG. 8 at S820A, the video decoder 200 may determine the QP offset of the QG using mark information added to the bitstream.

[0203] S820A, the video decoder 200 parses the bitstream to obtain mark information indicating the QP offset of the QG.

[0204] For example, after obtaining a bitstream, the video decoder 200 parses the bitstream to obtain mark information of the image frame in which the QG is located, and the mark information is used to indicate the QP offset (deltaQP) of the QG.

[0205] In a second optional embodiment, as shown in FIG. 8 at S820B, the video decoder 200 may use the derivation information to determine a QP offset for the QG.

[0206] S820B, the video decoder 200 determines a QP offset for the QG based on the derivation information for the QG.

[0207] The derived information may be any one or more combinations of flatness information or texture information of the QG, remaining space of a bitstream buffer, or distortion constraint information, where the flatness information or texture information is used to indicate an image gradient of the QG, the distortion constraint information is used to indicate a distortion threshold of any one of multiple QGs included in one image frame, and the remaining space of the bitstream buffer is used to indicate the available remaining space of a bitstream buffer (e.g., buffer space).

[0208] In the first case, the derivation information is flatness information or texture information, and the video decoder 200 may derive a QP offset for a QG based on the flatness information or texture information. For example, the video decoder 200 may calculate the texture complexity of a current block (QG), and use a large QP (e.g., 20) for a QG with high texture complexity (e.g., reaching the texture complexity threshold), and use a small QP (e.g., 5) for a QG with low texture complexity (e.g., not reaching the texture complexity threshold).

[0209] In the second case, the derived information is the remaining space of the bitstream buffer, and the video decoder 200 calculates the average number of bits BPPtotal for all pixels in the entire image and the average number of bits BPPleft for the remaining uncoded pixels, and decreases the QP if BPPleft>BPPtotal, otherwise increases the QP, where BPPtotal and BPPleft may be obtained by the following formula:

number

[0210] In the third case, when the derived information is distortion constraint information, let D denote the quantization distortion (the difference between the reconstructed value after inverse quantization and the residual before quantization), and for any matrix A,

number

[0211] (1) When quantizing the residual in the pixel domain, the necessary and sufficient condition for the maximum distortion in the pixel domain to not exceed △ is

number

[0212] (2) When quantizing the residual in the transform domain, if R represents the residual in the pixel domain, and U and V represent the horizontal and vertical transform matrices, the following formula is satisfied between the maximum distortion in the pixel domain and the maximum distortion in the transform domain. Therefore, a sufficient condition for the maximum distortion in the pixel domain not to exceed △ is that the maximum distortion in the transform domain is

number

number

[0213] Continuing to refer to FIG. 8, the QP acquisition process provided in this embodiment further includes the following steps:

[0214] At S830, the video decoder 200 determines a QP value for QG based on the predicted QP value for QG and a QP offset.

[0215] For example, the video decoder 200 determines the sum of the predicted QP value of a QG and the QP offset as the QP value of the QG.

[0216] Note that in the third case provided in S820B, when the video decoder 200 obtains the QP value of QG, the following cases may exist.

[0217] The video decoder 200 determines a corresponding predictive distortion based on a reference QP value of the QG, and if the predictive distortion is less than or equal to a distortion threshold, determines the reference QP value as the QP value of the QG; if the predictive distortion is greater than the distortion threshold, determines the QP value determined by the distortion threshold as the QP value of the QG.

[0218] In Example 1, when the derivation information includes only distortion constraint information, the reference QP value is the predicted QP value of QG determined in S810.

[0219] In Example 2, when the derivation information includes distortion constraint information and texture information (or flatness information), the reference QP value may be a QP value obtained by adding a deltaQP determined by the texture information (or flatness information) and the predicted QP value determined in S810.

[0220] In Example 3, when the derivation information includes distortion constraint information and remaining space in the bitstream buffer, the reference QP value may be a QP value obtained by adding a deltaQP determined by the remaining space in the bitstream buffer and the predicted QP value determined in S810.

[0221] That is, the above derivation information may be a deltaQP for determining QG, or may be an actual encoded QP value for directly determining QG, and the specific use process of the derivation information may be determined according to the needs of QP quantization / dequantization in video encoding and decoding, and the above three cases and three examples should not be understood as limitations on the present invention.

[0222] The above examples 1 to 3 are merely examples to illustrate that this embodiment uses the derivation information to determine the QP value of the QG by the video decoder 200, and should not be understood as limitations on the present invention. In other examples, the derivation information may simultaneously include distortion constraint information, texture information (or flatness information), and remaining space in the bitstream buffer, and the present invention is not limited in this regard.

[0223] It should be noted that the above chrominance / luminance QP values, direct coding / predictive coding, and bitstream additional / derived information analysis are merely some examples provided in the present invention, and should not be understood as limitations on the present invention.

[0224] In one optional embodiment, an image frame includes at least a CU in a first part and a CU in a second part, the CU in the first part and the CU in the second part have no overlapping area, and the QP value of the CU in the first part is obtained in a different manner from that of the CU in the second part, for example, the QP value of the CU in the first part is obtained by adding it to mark information of a bitstream, and the QP value of the CU in the second part is obtained by deriving it by the video decoder 200.

[0225] Specifically, the process of the video decoder 200 determining multiple QP values ​​of one image frame may include: first, the video decoder 200 parses a bitstream to obtain mark information of one image frame, the mark information including a QP offset of a CU of a first portion; and the video decoder 200 determines a QP value of the CU of the first portion based on the mark information; second, the video decoder 200 obtains a predicted QP value of the CU of the second portion for the CU of the second portion; and the video decoder 200 determines a QP value of the CU of the second portion based on the predicted QP value of the CU of the second portion and the derivation information. The relevant content of the derivation information is described in the above description of the derivation information of QG, and it is sufficient to replace QG with CU, and the description will not be repeated here.

[0226] For example, the video decoder 200 divides an image into a plurality of regions, and uses different QP processing methods for CUs in different regions. It transmits reference QP values ​​for different regions at the image level, and transmits mark information for different regions at the CU level. The CU level QPs for different regions may be obtained by a bitstream transmission method, or by a decoding side derivation method. For example, the video decoder divides an image into a region of interest (ROI) and a non-ROI region, and obtains QP values ​​for CUs (CUs of the first part) in the ROI region by a bitstream transmission method (e.g., the above-mentioned mark information), and obtains QP values ​​for CUs (CUs of the second part) in the non-ROI region by a decoding side derivation method (e.g., the above-mentioned derivation information).

[0227] As an optional embodiment, one CU of one image frame may include multiple QGs, and these multiple QGs may be partially QP quantized or may be entirely QP quantized.

[0228] For example, in the process of S520, when all QGs included in one CU of one image frame are quantized in the encoding process, the video decoder 200 may first determine the scanning order of all QGs included in one CU of the one image frame. Then, for each QG of all QGs, the video decoder 200 obtains the QP value of each QG according to the scanning order. Here, the scanning order includes any one of the following orders: top to bottom, left to right, zigzag, or reverse zigzag.

[0229] Specifically, if only some QGs are quantized and others are not, we need to mark the positions of the quantized QGs and obtain the QPs corresponding to these QGs. For example, we may code one QP offset for each of these QGs.

[0230] Also, for example, in the process of S520, if some QGs included in one CU of an image frame are quantized in the encoding process, the video decoder 200 may analyze the bitstream to determine one or more marked QGs in one CU of an image frame, and in the decoding process, the one or more marked QGs need to be QP dequantized, and further, for each QG of the one or more marked QGs, the video decoder 200 obtains a QP value for each QG.

[0231] Specifically, when quantizing all the QGs, the QPs corresponding to each QG need to be obtained sequentially according to a scanning order, which is related to the partitioning scheme and may be in the order of top to bottom, left to right, zigzag or reverse zigzag. For example, one QP offset may be coded for each of these QGs.

[0232] In this way, for all QGs in one CU of one image frame, the QP quantization method (partial quantization or full quantization) of all the QGs can be distinguished using mark information added to the bitstream, thereby preventing the video decoder from performing QP quantization indiscriminately, reducing the computational resources and image distortion required by the video decoder to perform QP quantization, and improving the efficiency and accuracy of video decoding.

[0233] Also, when one QG includes only one pixel point in one image frame, one pixel point corresponds to one QP value, and when the QP values ​​of at least two pixel points among all pixel points included in one image frame are different, the QP value quantization process (point prediction mode) of the pixel point may use a point-by-point quantization technique, for example, the point-by-point quantization technique includes a process in which the video decoder 200 adaptively adjusts the QP value of the current pixel point based on the reconstructed pixel information around the current pixel point. Here, the reconstructed pixel information includes, but is not limited to, pixel value, flatness information or texture information, background luminance, contrast, etc.

[0234] The adaptive point-by-point quantization technique may be applied to the QG or may be applied to a CU that does not perform QG division. The feature of the point-by-point quantization technique is that each pixel point in one image frame is allowed to use a different QP value, which corresponds to refining the granularity of the QP value quantization to the pixel level.

[0235] One embodiment of the point-by-point quantization technique is pred Let denote the QP value of the current CU or QG, and QP JND If ≥ 0 represents a QP value corresponding to the just noticeable distortion (JND) that the human eye can perceive, and offset > 0 represents a QP offset value (which may be transmitted in the bitstream or may be preset), the QP value of the current pixel is adjusted as follows:

number

number

[0236] The above two methods of determining the QP value of the current pixel are merely examples provided in this embodiment, and should not be construed as limitations on the present invention.

[0237] In this way, by using the point-by-point quantization technique, a video decoder can significantly improve the subjective quality of an image frame and reduce distortion of the image frame without changing the compression rate of the image frame.

[0238] The following provides one possible embodiment to describe the QP quantization process of pixel points in an image frame.

[0239] Exemplarily, the video decoder 200 obtains a predicted QP value of a pixel point, and determines a QP value of the pixel point based on the predicted QP value of the pixel point and the derivation information.

[0240] In one possible case, the predicted QP value of a pixel point is the QP value of the CU or QG in which the pixel point is located. In another possible case, the predicted QP value of a pixel point is derived based on the QP values ​​of one or more reconstructed pixel points around the pixel point, and the derivation method includes calculating at least one of the average value (e.g., the average value of the QP values ​​of the multiple pixel points), the median value (e.g., the median value of the QP values ​​of the multiple pixel points), or the mode value (e.g., the QP value that occurs most frequently among the QP values ​​of the multiple pixel points).

[0241] The above-mentioned derived information of a pixel point may be information of one or more reconstructed pixel points around the pixel point, where the information of the reconstructed pixel point includes any one or more combinations of pixel value, flatness information or texture information, background brightness, and contrast. Note that the above-mentioned information is merely an example provided in this embodiment, and should not be understood as a limitation on the present invention.

[0242] Moreover, the above meaning of "surrounding" may be understood as the neighboring pixel points of the pixel point for which the QP value is determined, and some possible examples are provided here for explanation. As shown in Fig. 9, Fig. 9 is a schematic diagram of the distribution of pixel points according to the present invention, and Fig. 9 (A) shows an example of dividing a square area with a current pixel point as the center, and two possible cases are provided here, in case 1, the reconstructed pixel point is a pixel point in a square area with a side length of 3, such as the surrounding pixel point 1 shown in Fig. 9 (A), and in case 2, the reconstructed pixel point is a pixel point in a square area with a side length of 5, such as the surrounding pixel point 2 shown in Fig. 9 (A), with the current pixel point as the center.

[0243] FIG. 9B shows an example of dividing a diamond region centered on the current pixel point, where two possible cases are provided: Case 1, the reconstructed pixel point is a pixel point within a diamond region with a diagonal length of 3 centered on the current pixel point, such as surrounding pixel point 1 shown in FIG. 9B; and Case 2, the reconstructed pixel point is a pixel point within a diamond region with a diagonal length of 5 centered on the current pixel point, such as surrounding pixel point 2 shown in FIG. 9B.

[0244] 9 is merely an example for describing the reconstructed pixel points around the current pixel point in this embodiment, and should not be understood as a limitation to the present invention. In some other possible examples, the reconstructed pixel points around the current pixel point may be one or two pixel points adjacent to the current pixel point above and below, or adjacent to the current pixel point to the left and right.

[0245] As one optional embodiment, the step of determining a QP value for a pixel point based on a predicted QP value of the pixel point and derived information includes a step of determining indication information for the pixel point based on information of one or more reconstructed pixel points surrounding the pixel point, and if the indication information is less than or equal to a first threshold and the predicted QP value is greater than or equal to a QP value corresponding to a distortion that is just perceptible to the human eye, determining the QP value corresponding to the distortion that is just perceptible to the human eye as the QP value for the pixel point.

[0246] The first threshold may be preset, or may be determined based on a compression rate or distortion rate requirement of video encoding and decoding, or may be determined based on user input information.

[0247] Note that the QP value corresponding to the distortion that the human eye can just perceive is picture-level or CU-level information.

[0248] For example, a QP value corresponding to a distortion that is just perceptible to the human eye is obtained by analyzing the bitstream, and a QP value corresponding to a distortion that is just perceptible to the human eye, for example 20, is added to the bitstream.

[0249] Also, for example, the QP value corresponding to the distortion that the human eye can just perceive is derived based on the flatness information or texture information of the surrounding reconstructed CU, background luminance, and contrast information. The process of deriving the QP value is described in the relevant content of FIG. 9, so it will not be repeated here.

[0250] In addition, the QP value corresponding to the distortion that the human eye can just perceive may be a value preset by the video encoder or video decoder, for example, 15. That is, the QP value corresponding to the distortion that the human eye can just perceive may not only be added to the mark information of the bitstream, but may also be analyzed by the video encoder or video decoder during video encoding / decoding, or may be a preset QP value. In this embodiment, the QP value corresponding to the distortion that the human eye just perceives is introduced into the QP value determination of the current pixel point, so that each pixel point satisfies the judgment information corresponding to the distortion that the human eye just perceives, reduces image distortion, and improves the subjective quality of the image.

[0251] Here, one possible concrete example is provided to describe the QP value determination process of a pixel point, as shown in Table 3, which outlines the QP value determination of the interval bisection of the current pixel point. [Table 1]

[0252] Here, offset>0 is the QP offset value (which may be transmitted in the bitstream or may be set in advance).

[0253] Method 1: If the instruction information is below the threshold, QP pred and QP JND If the instruction information is greater than the threshold, the predicted QP value (QP pred ) is determined as the QP value of the current pixel point.

[0254] Method 2: If the instruction information is below the threshold, QP pred and QP JND The smaller of these two is determined as the QP value of the current pixel point. If the indication information is greater than the threshold, the QP pred The sum of the QP offset and the QP value of the current pixel point is determined.

[0255] Method 3: When the indication information is below the threshold, QPpred is determined as the QP value of the current pixel point, and if the instruction information is greater than the threshold, the QP pred The sum of the QP offset and the QP value of the current pixel point is determined.

[0256] It should be noted that the above examples and Table 3 are merely possible ways of obtaining the QP value of the current pixel point provided in this embodiment, and should not be understood as limitations on the present invention.

[0257] In this way, by introducing the QP value corresponding to the distortion that the human eye can just perceive into the QP value determination process of the current pixel point, each pixel point satisfies the judgment information corresponding to the distortion that the human eye just perceives, thereby reducing image distortion and improving the subjective quality of the image.

[0258] After the video decoder 200 obtains the QP value of an image frame, for the above S530, the present invention provides one possible embodiment, and FIG. 10 is a schematic flowchart of image frame decoding according to the present invention, and S530 may include the following steps S1010 to S1030.

[0259] S1010, for each QP value of the plurality of QP values, obtain a quantization step (Qstep) corresponding to the QP value.

[0260] The video decoder may obtain the Qstep through at least one of a formula derivation and a table lookup based on the QP value, and the following provides four possible implementation methods.

[0261] Method 1: Qstep=2×QP+1.

[0262] Method 2: Let octave represent the bit order of QP, i.e., every time QP increases by one octave, Qstep doubles, and usually octave is chosen to be 6 or 8. Let offset represent an integer offset value,

number

[0263] Method 3: Represent the bit order of QP in octave, that is, every time 1 octave of QP increases, Qstep doubles. Usually, octave is selected as 6 or 8. Represent one integer offset value with offset,

Number

Number

Number

[0264] Method 4: The following provides examples of quantization and inverse quantization.

Number

[0265] The parameter f ∈ [0, 1) controls rounding, and {c i} are all the residual coefficients to be quantized within the current QG or CU. T can be obtained by the formula of Method 3,

Number

[0266] When T < M, Qstep is given by the following formula. Otherwise, both the quantization coefficient and the reconstructed value are zero.

Number

[0267] One possible embodiment of the method 4 is a=f*2 T-M Or a=2 T-M where M may be transmitted in the bitstream or may be obtained directly as the bit depth.

[0268] When quantizing the residual in the pixel domain, method 3 requires a clip operation to make the quantized coefficients representable in "MT" bits, whereas method 4's quantized coefficients can be naturally represented in "MT" bits without clipping.

[0269] In addition, the uniform quantization scheme in JPEG-XS is f=0.5 in Method 4, and M is transmitted in the bitstream.

[0270] Note that the larger the QP value, the larger the Qstep, the coarser the quantization, the larger the quantization distortion, and the smaller the bit rate of coefficient coding. The parameter f is related to the length of the quantization dead zone, and the smaller the f, the longer the quantization dead zone, and the closer the level value after quantization is to the zero point. When f=0.5, the formula of quantization and inverse quantization provided in the above method 4 corresponds to rounding off, and the quantization distortion is the smallest. When f<0.5, the smaller the f, the larger the quantization distortion and the smaller the bit rate of coefficient coding. In H.265, I image f=1 / 3, and B / P image f=1 / 6.

[0271] Continuing to refer to FIG. 8, the above S530 may further include the following steps:

[0272] S1020, a level value included in the QG corresponding to the QP value is obtained.

[0273] S1030, inverse quantizing the level value of the QG based on the selected combination of quantizers.

[0274] The combination of quantizers includes one or more quantizers, and the quantizer is a uniform quantizer or a non-uniform quantizer.

[0275] For example, the uniform quantizer refers to a uniform scalar quantizer, and the quantization or inverse quantization formula can refer to the quantization and inverse quantization formulas provided in Method 4 of S1010 above. The parameter f ∈ [0, 1) is set as follows.

[0276] Method 1: Set f to 0.5 or other fixed value.

[0277] Method 2: f can be adaptively determined based on the QP value, prediction mode, and whether to perform transformation.

[0278] Also, for example, the non-uniform quantizer refers to a non-uniform scalar quantizer, and the correspondence between the quantization level value, quantization interval, and reconstructed value of inverse quantization may be obtained by table lookup. One possible embodiment is shown in Table 1 below.

Table 2

[0279] Here, 0 ≦ x0 < x1 < x2 < x3 <..., the reconstructed value and quantization interval are non-uniform, and the reconstructed value is the probability centroid of the quantization interval where it is located.

[0280] The combination of quantizers may use one non-uniform scalar quantizer or multiple non-uniform scalar quantizers.

[0281] For example, the combination of quantizers is determined by the mark information added to the bitstream.

[0282] Also, for example, the combination of quantizers is determined by the distribution of residual coefficients in the QG. For example, the video decoder adaptively selects which quantizer to use, and the basis for the selection may be mode information or transformation information, which is related to the distribution of residual coefficients in the QG.

[0283] Regarding S1030 above, dequantizing the level value of QG may include the following process.

[0284] First, the division region type of the QG is determined.

[0285] Next, if the division region type of the QG is a transform region, a quantization matrix that matches the parameter information of the QG is selected from the matrix template library on the decoding side.

[0286] The matrix template library includes multiple types of quantization matrix templates, and the parameter information includes any one or more combinations of the size of the QG, the size of the CU in which the QG is located, luminance channel information, and flatness information.

[0287] Finally, the level values ​​in QG are inverse quantized using the quantization matrix of QG to obtain the residual coefficients of QG.

[0288] Since residual coefficients at different positions in the pixel domain are equally important, no quantization matrix is ​​used. The coefficients in the transform domain are separated into low-frequency and high-frequency coefficients, and the quantization matrix ensures that the high and low frequency coefficients use different quantization steps, improving the subjective quality of the image while ensuring a constant compression rate.

[0289] The element distribution of the quantization matrix has a certain template, and in the present invention, coding blocks of different sizes can use different quantization matrix templates, and a large-sized quantization matrix can be obtained by upsampling one or more small-sized quantization matrices. The quantization matrix templates included in the matrix template library can be obtained by any one or more transform methods of discrete cosine transform (DCT), discrete sine transform (DST), integer transform, or discrete wavelet transform (DWT). As shown in Figure 11, Figure 11 is a schematic diagram of a quantization matrix template according to the present invention, and the quantization matrix template specifically includes:

[0290] (1) When both the horizontal and vertical directions are DCT / DST transformed (see (1) in FIG. 11), coefficients on the same anti-diagonal have the same frequency and may employ the same quantization step, while coefficients on different anti-diagonals may employ different quantization steps.

[0291] (2) When both the horizontal and vertical directions are wavelet transformed (see (2) in Figure 11), a quadtree template is adopted, and the four sub-blocks correspond to low / medium / high frequencies, respectively. The sub-blocks in the same frequency band may adopt the same quantization step, and the sub-blocks in different frequency bands may adopt different quantization steps.

[0292] (3) When the vertical direction is DCT / DST transformation and the horizontal direction is wavelet transformation (see (3) in Figure 11), first divide it evenly vertically into two, and the left sub-block is the low frequency of the wavelet transformation, and the right sub-block is the high frequency of the wavelet transformation. The upper part of each sub-block is the low frequency of the DCT / DST transformation, and the lower part is the high frequency of the DCT / DST transformation. The matrix elements A' / B' / C' / D' may be A / B / C / D with the same offset value added, respectively.

[0293] (4) When the horizontal direction is DCT / DST transformation and the vertical direction is wavelet transformation (see (4) in Figure 11), first divide the horizontal direction equally into two, and the upper sub-block is the low frequency of the wavelet transformation, and the lower sub-block is the high frequency of the wavelet transformation. The left side of each sub-block is the low frequency of the DCT / DST transformation, and the right side is the high frequency of the DCT / DST transformation. The matrix elements A' / B' / C' / D' may be A / B / C / D with the same offset value added, respectively.

[0294] Optionally, the multiple types of quantization matrix templates included in the matrix template library may include flat block templates and texture block templates, where the Qstep of residual coefficients in the flat block templates whose frequency is above a frequency threshold is equal to or greater than the Qstep of residual coefficients in the flat block templates whose frequency is below the frequency threshold, and the Qstep of residual coefficients in the texture block templates whose frequency is above the frequency threshold is equal to or less than the Qstep of residual coefficients in the texture block templates whose frequency is below the frequency threshold.

[0295] For example, the video decoder marks the current block as a flat block or a texture block based on the flatness information, and designs the quantization matrix template according to the texture masking effect. (1) If the current block (QG) is a flat block, the Qstep of the high-frequency coefficients in the quantization matrix is ​​equal to or greater than the Qstep of the low-frequency coefficients, and the human eye is more sensitive to the low-frequency distortion of flat blocks than the high-frequency distortion, so the high-frequency coefficients tolerate more loss. (2) If the current block (QG) is a texture block, the Qstep of the high-frequency coefficients in the quantization matrix is ​​equal to or less than the Qstep of the low-frequency coefficients, and the human eye is more sensitive to the high-frequency distortion of texture blocks than the low-frequency distortion, so the high-frequency coefficients of the texture blocks are preferentially protected.

[0296] That is, in the image decoding method, the video decoder first obtains the QP and Qstep of the QG, analyzes the level value from the bitstream, and then adaptively selects a quantizer, dequantizes the level value, and obtains the reconstructed value, thereby realizing the decoding of the image frame.

[0297] In conclusion, compared with the large image distortion caused by all residual coefficients in one CU using the same QP value, in the video image decoding process provided in this embodiment, one CU may be divided into multiple QGs, and one or more residual coefficients in each QG share one QP value, so that the video decoder can make finer granularity QP decisions for one or more CUs corresponding to an image frame, thereby reducing the decoding distortion of the image frame and improving the authenticity and accuracy of video image decoding while ensuring a certain compression rate.

[0298] Correspondingly, in the image encoding method, the video encoder first obtains the QP, Qstep and residual coefficients of the QG, adaptively selects a quantizer, quantizes the residual coefficients, and finally adjusts the quantization coefficients to obtain a final level value, thereby realizing the encoding of the image frame.

[0299] Based on the video encoder 100 shown in FIG. 2, the present invention further provides an image encoding method, as shown in FIG. 12, which is a schematic flowchart of the image encoding method according to the present invention, which may be performed by the video encoder 100, or by an encoding side that supports the functions of the video encoder 100 (e.g., the encoding side 10 shown in FIG. 1). Here, the encoding method is described as an example in which the video encoder 100 realizes the encoding method, and the image encoding method includes the following steps:

[0300] At S1210, the video encoder 100 divides one image frame into one or more CUs.

[0301] At S1220, video encoder 100 determines multiple QP values ​​for an image frame.

[0302] In one example, one CU includes multiple QGs, and one QG corresponds to one QP value.

[0303] In another example, one CU includes multiple pixel points, where one pixel point corresponds to one QP value, and the QP values ​​of at least two pixel points among the multiple pixel points are different.

[0304] At S1230, the video encoder 100 encodes an image frame based on multiple QP values.

[0305] The quantization of the QP values ​​in the encoding method is described in the corresponding process in the decoding method of FIGS. 4 to 11 above, and will not be repeated here.

[0306] In this way, compared with the large image distortion caused by all residual coefficients in one CU using the same QP value, in the video image encoding process provided in this embodiment, one CU may be divided into multiple QGs (or pixel points), and one or more residual coefficients in each QG share one QP value, so that the video encoder can make finer-grained QP decisions for one or more CUs corresponding to an image frame, thereby reducing the encoding distortion of the image frame while ensuring a certain compression rate, and improving the authenticity and accuracy of video image encoding.

[0307] It should be noted that, in order to realize the functions in the above embodiments, the video encoder / video decoder includes corresponding hardware structures and / or software modules for performing each function. As can be easily understood by those skilled in the art, with reference to the units and method steps of each example described in the embodiments disclosed in the present invention, the present invention can be realized in the form of hardware, or a combination of hardware and computer software. Whether a function is performed by hardware or in a manner in which computer software drives hardware depends on a specific application scenario and design constraints of the technical solution.

[0308] FIG. 13 is a schematic structural diagram of a decoding device according to the present invention, and the decoding device 1300 includes a bitstream analysis unit 1310, a QP determination unit 1320, and an image decoding unit 1330. The decoding device 1300 can be used to implement the function of the video decoder or the decoding side in the above embodiment of the decoding method, and thus can realize the beneficial effects of the above embodiment of the decoding method. In the embodiment of the present invention, the decoding device 1300 can be the decoding side 20 or the video decoder 200 shown in FIG. 1, can be the video decoder 200 shown in FIG. 3, or can be a module applied to the decoding side 20 or the video decoder 200.

[0309] The bitstream parsing unit 1310, the QP determination unit 1320, and the image decoding unit 1330 are used to implement the decoding method provided in any of the embodiments in Figures 4 to 11. A more detailed description of the above bitstream parsing unit 1310, the QP determination unit 1320, and the image decoding unit 1330 can be obtained directly by directly referring to the relevant description in the embodiment of the method shown in Figures 4 to 11, and therefore will not be repeated here.

[0310] FIG. 14 is a schematic structural diagram of an encoding device according to the present invention, the encoding device 1400 includes an image division unit 1410, a QP determination unit 1420, and an image encoding unit 1430. The encoding device 1400 can be used to implement the functions of the video encoder or encoding side in the above embodiment of the decoding method, and thus can also realize the beneficial effects of the above embodiment of the encoding method. In the embodiment of the present invention, the encoding device 1400 can be the encoding side 10 or the video encoder 100 shown in FIG. 1, can be the video encoder 100 shown in FIG. 2, or can be a module applied to the encoding side 10 or the video encoder 100.

[0311] The image splitting unit 1410, the QP determination unit 1420, and the image encoding unit 1430 are used to implement the encoding method provided in Figure 12. A more detailed description of the above image splitting unit 1410, the QP determination unit 1420, and the image encoding unit 1430 can be obtained directly by directly referring to the relevant description in the embodiment of the method shown in Figures 4 to 12, and therefore will not be repeated here.

[0312] The present invention further provides an electronic device, as shown in FIG. 15, which is a schematic structural diagram of an electronic device according to the present invention, in which the electronic device 1500 includes a processor 1510 and an interface circuit 1520. The processor 1510 and the interface circuit 1520 are coupled to each other. It should be noted that the interface circuit 1520 may be a transceiver or an input / output interface. Optionally, the electronic device 1500 may further include a memory 1530 for storing instructions to be executed by the processor 1510, or for storing input data required for the processor 1510 to execute the instructions, or for storing data generated after the processor 1510 executes the instructions.

[0313] The electronic device 1500 includes a processor 1510 and a communication interface 1520. The processor 1510 and the communication interface 1520 are coupled to each other. Note that the communication interface 1520 may be a transceiver or an input / output interface. Optionally, the electronic device 1500 may further include a memory 1530 for storing instructions to be executed by the processor 1510, or for storing input data required for the processor 1510 to execute the instructions, or for storing data generated after the processor 1510 executes the instructions.

[0314] When the electronic device 1500 is used to implement the methods shown in Figures 4 to 11, the processor 1510 and the interface circuit 1520 are used to perform the functions of the bitstream parsing unit 1310, the QP determination unit 1320, and the image decoding unit 1350 described above.

[0315] When the electronic device 1500 is used to implement the method shown in FIG. 12, the processor 1510 and the interface circuit 1520 are used to perform the functions of the image division unit 1410, the QP determination unit 1420, and the image encoding unit 1430 described above.

[0316] In the embodiment of the present invention, the specific connection medium between the communication interface 1520, the processor 1510, and the memory 1530 is not limited. In the embodiment of the present invention, in FIG. 15, the communication interface 1520, the processor 1510, and the memory 1530 are connected via a bus 1540, and the bus is shown by a thick line in FIG. 15, and the connection manner between other components is merely illustrative and is not particularly limited thereto. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of expression, only one thick line is shown in FIG. 15, but this does not mean that there is only one bus or only one type of bus.

[0317] The memory 1530 may be used to store software programs and modules, such as program instructions / modules corresponding to the decoding or encoding methods provided in the embodiments of the present invention, and the processor 1510 executes various functional applications and data processing by executing the software programs and modules stored in the memory 1530. The communication interface 1520 may be used for signaling and data communication with other devices. In the present invention, the electronic device 1500 may have multiple communication interfaces 1520.

[0318] It should be noted that the processor in the embodiments of the present invention may be a central processing unit (CPU), neural processing unit (NPU), or graphic processing unit (GPU), or may be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0319] The steps of the method in the embodiment of the present invention may be implemented by hardware or by a processor executing software instructions. The software instructions may be composed of corresponding software modules, which may be stored in a random access memory (RAM), a flash memory, a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a register, a hard disk, a removable hard disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium may be a component of the processor. The processor and the storage medium may be located in an ASIC. The ASIC may also be located in a network device or a terminal device. Of course, the processor and the storage medium may exist in a network device or a terminal device as separate components.

[0320] In the above embodiments, all or part of the invention may be realized by software, hardware, firmware, or any combination thereof. When the invention is realized using software, all or part of the invention may be realized in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer programs or instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are executed. The computer may be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user equipment, or other programmable device. The computer programs or instructions may be stored in a computer-readable storage medium, or may be transmitted from one computer-readable storage medium to another, for example, the computer programs or instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire or wirelessly. The computer-readable storage medium may be any available medium accessible by a computer, or a data storage device such as a server or data center that accumulates one or more available media. The available medium may be a magnetic medium, such as a floppy disk, hard disk, or magnetic tape, or an optical medium, such as a digital video disc (DVD), or a semiconductor medium, such as a solid state drive (SSD).

[0321] In each embodiment of the present invention, unless there is a special explanation and logical contradiction, the terms and / or explanations between different embodiments are consistent and can be cited from each other, and the technical features in different embodiments can be combined according to their internal logical relationships to form a new embodiment. In the present invention, "at least one" refers to one or more, and "multiple" refers to two or more. "And / or" describes the related relationship of related objects and indicates that there may be three types of relationships, for example, A and / or B can represent A alone, A and B simultaneously, and B alone, and A and B may be singular or plural. In the textual description of the present invention, the character " / " generally represents that the related objects before and after are in an "or" relationship, and in the formula of the present invention, the character " / " represents that the related objects before and after are in a "division" relationship.

[0322] In addition, various numbers related to the embodiments of the present invention are given only for convenience of explanation, and are not intended to limit the scope of the embodiments of the present invention. The magnitude of the numbers of each process above does not mean the execution order, and the execution order of each process should be determined by its function and internal logic. [Explanation of symbols]

[0323] 1300 Decoding device 1310 Bitstream Analysis Unit 1320 QP Decision Unit 1330 Image Decoding Unit 1400 encoding device 1410 Image Segmentation Unit 1420 QP Decision Unit 1430 Image Encoding Unit 1500 Electronic Devices 1510 Processor 1520 Communication Interface 1530 Memory

Claims

1. An image decoding method executed by a decoding side, comprising: Parsing the bitstream to obtain one or more image frames, where an image frame includes one or more coding units CU; determining multiple quantization parameter QP values ​​for the one image frame, where one CU includes multiple quantization groups QGs, and one QG corresponds to one QP value; and decoding the one image frame based on the plurality of QP values; The one CU includes a plurality of residual coefficients, the one QG includes some of the residual coefficients, and the some residual coefficients share the one QP value.

2. An image decoding method comprising:

2. prior to determining a plurality of QP values ​​for the one image frame, The method further includes a step of dividing one CU included in the one image frame according to a first rule to obtain the multiple QGs; The first rule includes a division region and a division method, the division region is a pixel region, and the division method includes uniform division; 2. The method of claim 1 .

3. One QP value corresponding to one QG includes a luminance QP value and a chromaticity QP value, The step of determining a plurality of QP values ​​for an image frame includes: Obtaining a luminance QP value and a chromaticity QP value of the one QG respectively; Or, Obtain a luminance QP value of the one QG; obtaining a chromaticity QP value of the one QG based on the luminance QP value; 2. The method of claim 1 .

4. The step of decoding the one image frame based on the plurality of QP values ​​includes: For each QP value of the plurality of QP values, obtaining a quantization step Qstep corresponding to the QP value; obtaining a level value included in a QG corresponding to the QP value; and dequantizing the level value of the QG based on the selected combination of quantizers; the combination of quantizers includes one or more quantizers; 2. The method of claim 1 .

5. The quantizer combination is determined by mark information added to the bitstream, or The combination of quantizers is determined by the distribution of residual coefficients in the QG.

5. The method of claim 4.

6. The quantizer is a uniform quantizer or a non-uniform quantizer.

5. The method of claim 4.

7. The one QG includes one or more pixel points of the one image frame.

2. The method of claim 1 .

8. QP values ​​corresponding to at least two QGs among the plurality of QGs ​​are different; 2. The method of claim 1 .

9. An image coding method executed by an encoding side, comprising: Dividing an image frame into one or more coding units (CU); determining multiple quantization parameter QP values ​​for the one image frame, where one CU includes multiple quantization groups QGs, and one QG corresponds to one QP value; and encoding the one image frame based on the plurality of QP values; The one CU includes a plurality of residual coefficients, the one QG includes some of the residual coefficients, and the some residual coefficients share the one QP value.

13. An image coding method comprising:

10. An image decoding device including a bitstream analysis unit, a QP determination unit, and an image decoding unit, The bitstream parsing unit is used for parsing the bitstream to obtain one or more image frames, where one image frame includes one or more coding units CU; The QP determination unit is used to determine multiple quantization parameter QP values ​​of the one image frame, and one CU includes multiple quantization groups QGs, and one QG corresponds to one QP value; the image decoding unit is adapted to decode the one image frame based on the plurality of QP values; The one CU includes a plurality of residual coefficients, the one QG includes some of the residual coefficients, and the some residual coefficients share the one QP value.

2. An image decoding device comprising:

11. An image encoding device including an image division unit, a QP determination unit, and an image encoding unit, The image division unit is used to divide an image frame into one or more coding units (CUs); The QP determination unit is used to determine multiple quantization parameter QP values ​​of the one image frame, and one CU includes multiple quantization groups QGs, and one QG corresponds to one QP value; the image coding unit is adapted to code the one image frame based on the plurality of QP values; The one CU includes a plurality of residual coefficients, the one QG includes some of the residual coefficients, and the some residual coefficients share the one QP value.

1. An image encoding device comprising:

12. A video coding system including an encoding side and a decoding side, The encoding side and the decoding side are communicatively connected, the decoding side is used to implement the method according to any one of claims 1 to 8, and the encoding side is used to implement the method according to claim 9. A video coding system comprising:

13. An electronic device including a processor and a memory, The memory is adapted to store computer instructions, and the processor is adapted to retrieve and execute the computer instructions from the memory to implement the method according to any one of claims 1 to 9.

1. An electronic device comprising:

14. A computer readable storage medium having stored thereon a computer program or instructions, which, when executed by an electronic device, performs the method according to any one of claims 1 to 9. A computer-readable storage medium comprising:

Images