Decoding method, encoding method, and apparatus

Abstract

An image decoding method, encoding method and apparatus, which are related to the field of video encoding and decoding, includes: first, analyzing a bitstream to obtain one or more image frames, where an image frame includes one or more CUs; ​​then, determining multiple QP values ​​for the image frame, where a CU includes multiple QGs and a QG corresponds to a QP value; and finally, decoding the image frame based on the multiple QP values.

Description

[Technical Field]

[0001] Cross-reference of related applications This application claims priority to the Chinese patent application filed on November 11, 2021, application number 202111334223.8, which is incorporated herein by reference.

[0002] The present invention relates to the field of video coding and decoding, and more particularly to image decoding methods, coding methods, and apparatus. [Background technology]

[0003] In video encoding and decoding techniques, video compression techniques are particularly important. Video compression techniques perform spatial (in-image) prediction and / or temporal (inter-image) prediction to reduce or remove inherent redundant information in a video sequence. The basic principle of video compression is to quantize the correlation between the spatial domain, the temporal domain and the codeword, and to remove as much redundancy as possible. Quantization is the process of mapping continuous values ​​(or a large number of discrete values) of a signal to a finite number of discrete amplitudes, thereby achieving a many-to-one mapping of signal values.

[0004] In the current method, for one or more coding units (CUs) contained in a single frame of an image, the encoding side obtains the quantization parameter (QP) for each CU, encodes the CU based on the QP to obtain a bitstream, and the decoding side dequantizes the bitstream to obtain the QP for the CU and decodes the CU based on the QP. Since CUs are divided based on image content, and both the encoding and decoding sides use the same QP to quantize the image content corresponding to a single CU, the quantization distortion of the image coding and decoding processes becomes large. [Overview of the Initiative] [Problems that the invention aims to solve]

[0005] The present invention provides an image decoding method, encoding method, and apparatus that solve the problem of large quantization distortion in image coding and decoding processes. [Means for solving the problem]

[0006] This invention adopts the following technical approach.

[0007] According to a first aspect, the present invention provides an image decoding method which may be applied to a video coding system, or which may be applied to a decoding side capable of supporting video coding to implement the method, for example, the decoding side includes a video decoder. The method includes the steps of: first, analyzing a bitstream to obtain one or more image frames, one image frame comprising one or more CUs; ​​second, determining a plurality of QP values ​​for the one image frame, one CU comprising a plurality of QGs, one QG corresponding to one QP value; and finally, decoding the one image frame based on the plurality of QP values.

[0008] Compared to the large image distortion caused by using the same QP value for all residual coefficients within a single CU, the video image decoding process provided in this embodiment allows a single CU to be divided into multiple QGs, and one or more residual coefficients within each QG share a single QP value. This enables the video decoder to perform finer-grained QP determination for one or more CUs corresponding to image frames, reducing decoding distortion of image frames and improving the truthfulness and accuracy of video image decoding while ensuring a constant compression ratio.

[0009] In one optional embodiment, the one CU includes a plurality of residual coefficients, the one QG includes some of the residual coefficients among the plurality of residual coefficients, and the some residual coefficients share the one QP value.

[0010] In one optional embodiment, before determining the multiple QP values ​​of the single image frame, the method further includes the step of dividing a single CU contained in the single image frame according to a first rule to obtain the multiple QGs, wherein the first rule includes a division region and a division scheme, and the division region is a picture It is an elementary region, and the division method is uniform division division include.

[0011] In one optional embodiment, the one CU includes a plurality of residual coefficients, the locations of the plurality of residual coefficients are marked by coordinates, the coordinates include a horizontal coordinate and a vertical coordinate. If the division region is a transformation region, the step of dividing the one CU contained in the one image frame according to a first rule to obtain the plurality of QGs ​​includes dividing the plurality of residual coefficients into a first QG for residual coefficients whose coordinate sum does not reach a first coordinate threshold, and dividing the residual coefficients whose coordinate sum does reach the first coordinate threshold into a second QG, the coordinate sum being the sum of the horizontal and vertical coordinates of the residual coefficients.

[0012] Alternatively, the residual coefficients whose coordinate sum does not reach the first coordinate threshold are divided into a first QG, the residual coefficients whose coordinate sum reaches the first coordinate threshold but does not reach the second coordinate threshold are divided into a second QG, and the residual coefficients whose coordinate sum reaches the second coordinate threshold are divided into a third QG, wherein the second coordinate threshold is greater than the first coordinate threshold.

[0013] In one optional embodiment, if the one CU includes a plurality of residual coefficients and the division region is a transformation region, the step of dividing the one CU contained in the one image frame according to a first rule and obtaining the plurality of QGs ​​is: The method includes sorting the plurality of residual coefficients, dividing the residual coefficients that have not reached a first proportional threshold into a first QG, and dividing the residual coefficients that have reached the first proportional threshold into a second QG, wherein the sorting method for the plurality of residual coefficients is either zig-zag or reverse zig-zag.

[0014] Alternatively, the plurality of residual coefficients are sorted, and the residual coefficients that have not reached the first proportional threshold are divided into a first QG, the residual coefficients that have reached the first proportional threshold but have not reached the second proportional threshold are divided into a second QG, and the residual coefficients that have reached the second proportional threshold are divided into a third QG, wherein the second proportional threshold is greater than the first proportional threshold.

[0015] In one optional embodiment, if the one CU includes a plurality of residual coefficients and the division region is a pixel region, the step of dividing the one CU contained in the one image frame according to a first rule to obtain the plurality of QGs ​​includes the step of performing a horizontal or vertical symmetric division with respect to the plurality of residual coefficients to obtain two QGs containing the same number of residual coefficients.

[0016] Alternatively, a symmetrical partitioning in the horizontal or vertical direction is performed on the plurality of residual coefficients to obtain three QGs, and of the three QGs, the number of residual coefficients in two non-adjacent QGs is the same, and the number of residual coefficients in the remaining QG is the same as the sum of the number of residual coefficients in the two non-adjacent QGs.

[0017] Alternatively, the plurality of residual coefficients may be divided horizontally or vertically to obtain two QGs containing a number of residual coefficients that do not match.

[0018] Alternatively, the plurality of residual coefficients may be divided horizontally or vertically to obtain three QGs, and the residual coefficients included in the three QGs may not have a symmetric relationship.

[0019] In one optional embodiment, one QP value corresponding to the one QG includes a luminance QP value and a chrominance QP value. The step of determining a plurality of QP values of the one image frame includes the step of respectively obtaining the luminance QP value and the chrominance QP value of the one QG.

[0020] Alternatively, obtain the luminance QP value of the one QG, and obtain the chrominance QP value of the one QG based on the luminance QP value.

[0021] In one optional embodiment, the step of determining a plurality of QP values of the one image frame is a step of analyzing the bitstream to obtain mark information of the one image frame, where the mark information is used to indicate the QP value of the one QG and / or the mark information is used to indicate the QP value of the one CU.

[0022] In one optional embodiment, the step of determining a plurality of QP values of the one image frame first includes the step of analyzing the bitstream to obtain mark information of the one image frame, where the mark information is used to indicate the QP offset of the one QG, and second, determining the QP value of the one QG based on the predicted QP value of the one QG and the mark information.

[0023] In one optional embodiment, the step of determining the QP value of the one QG based on the predicted QP value of the one QG and the mark information includes the step of obtaining the predicted QP value of the one QG, and determining the sum of the predicted QP value of the one QG and the QP offset as the QP value of the one QG.

[0024] In one optional embodiment, the step of determining a plurality of QP values ​​for a single image frame includes the steps of obtaining a predicted QP value for a single QG, and determining a QP value for a single QG based on the predicted QP value and derived information, wherein the derived information is one or more combinations of flatness information or texture information, remaining space in a bitstream buffer, or distortion constraint information for the single QG.

[0025] In one optional embodiment, if the derived information is the strain constraint information and the strain constraint information indicates a strain threshold for any one of the plurality of QGs, the step of determining the QP value of the one QG based on the predicted QP value of the one QG and the derived information includes the steps of: determining the predicted strain corresponding to the predicted QP value; determining the predicted QP value as the QP value of the QG if the predicted strain is less than or equal to the strain threshold; and determining the QP value determined by the strain threshold as the QP value of the QG if the predicted strain is greater than the strain threshold.

[0026] In one optional embodiment, if the derived information is content information of the one QG or remaining space in the bitstream buffer, the step of determining the QP value of the one QG based on the predicted QP value of the one QG and the derived information includes the steps of determining the QP offset of the one QG based on the derived information, and determining the sum of the predicted QP value of the one QG and the QP offset as the QP value of the one QG.

[0027] In one optional embodiment, the step of obtaining the predicted QP value of the one QG includes obtaining the QP values ​​of at least one other QG in the one CU adjacent to the one QG, and determining the predicted QP value of the one QG based on the QP values ​​of the at least one other QG.

[0028] Alternatively, the QP value of one CU is determined as the predicted QP value of one QG.

[0029] In one optional embodiment, the single image frame includes at least a first portion CU and a second portion CU, the first portion CU and the second portion CU do not have overlapping regions, and the methods for obtaining the QP values ​​of the first portion CU and the second portion CU are different.

[0030] In one optional embodiment, the step of determining a plurality of QP values ​​for a single image frame includes: analyzing the bitstream to obtain mark information for the single image frame, the mark information including the QP offset of the first portion of the CU; determining the QP value of the first portion of the CU based on the mark information; for the second portion of the CU, obtaining a predicted QP value for the second portion of the CU; and determining the QP value of the second portion of the CU based on the predicted QP value and derived information, wherein the derived information is one or more combinations of flatness information or texture information, remaining space in the bitstream buffer, or distortion constraint information for the second portion of the CU.

[0031] In one optional embodiment, the step of decoding the image frame based on the plurality of QP values ​​includes, first, for each of the plurality of QP values, the step of obtaining a quantization step Qstep corresponding to the QP value; second, the step of obtaining a level value contained in the QG corresponding to the QP value; and finally, the step of dequantizing the level value of the QG based on a selected combination of quantizers, the combination of quantizers includes one or more quantizers. Exemplarily, the quantizers are homogeneous or non-homogeneous quantizers.

[0032] In one optional embodiment, the combination of quantizers is determined by mark information appended to the bitstream or by the distribution of residual coefficients in the QG.

[0033] In one optional embodiment, the step of inverse quantizing the level value of the QG includes: first determining the partitioned region type of the QG; second, if the partitioned region type of the QG is a transformation region, selecting a quantization matrix from the decoding matrix template library that matches the parameter information of the QG, wherein the matrix template library includes a plurality of types of quantization matrix templates, and the parameter information includes one or more combinations of the size of the QG, the size of the CU on which the QG is located, brightness channel information, and flatness information; and finally, obtaining the residual coefficient of the QG by inverse quantizing the level value in the QG using the quantization matrix of the QG.

[0034] In one optional embodiment, the plurality of types of quantization matrix templates include a flat block template and a texture block template, wherein in the flat block template, the Qstep of residual coefficients whose frequency is above the frequency threshold is greater than or equal to the Qstep of residual coefficients whose frequency is not above the frequency threshold in the flat block template, and in the texture block template, the Qstep of residual coefficients whose frequency is above the frequency threshold is less than or equal to the Qstep of residual coefficients whose frequency is not above the frequency threshold in the texture block template.

[0035] In one optional embodiment, the quantization matrix templates included in the matrix template library are obtained by one or more of the following transformation schemes: discrete cosine transform (DCT), discrete sine transform (DST), integer transform, or discrete wave transform (DWT).

[0036] In one optional embodiment, the one QG includes one or more pixel points of the one image frame.

[0037] In one optional embodiment, the step of determining a plurality of QP values ​​for a single image frame includes: analyzing the bitstream to determine one or more marked QGs in a single CU, wherein the one or more marked QGs are to be dequantized in the decoding process, and the unmarked QGs in the single CU are not dequantized; and for each of the one or more marked QGs, the step of obtaining the QP value of each QG.

[0038] Alternatively, the scanning order of all QGs included in one CU is determined, the scanning order including one or more combinations of top-to-bottom, left-to-right, zigzag, or reverse zigzag, and for each QG of all QGs, the QP value of each QG is obtained according to the scanning order.

[0039] In one optional embodiment, the QP values ​​corresponding to at least two of the plurality of QGs ​​are different.

[0040] According to a second aspect, the present invention provides an image decoding method, the method may be applied to a video coding system, or the method may be applied to a decoding side capable of supporting video coding to implement the method, the method being performed by the decoding side, the method comprising: a step of analyzing a bitstream to obtain one or more image frames, the image frame comprising one or more CUs; ​​and a step of determining a plurality of QP values ​​of the image frame, the CU comprising a plurality of pixel points, the pixel point corresponding to a single QP value, and the QP values ​​of at least two of the plurality of pixel points being different. the law of nature , The QP value of a single pixel point is determined by adaptively adjusting it based on the reconstructed pixel information surrounding the current pixel point. The process includes the steps of decoding the single image frame based on the plurality of QP values.

[0041] In one optional embodiment, the step of determining a plurality of QP values ​​for a single image frame includes the steps of obtaining a predicted QP value for a single pixel point and determining a QP value for the single pixel point based on the predicted QP value and derived information, wherein the derived information is information about one or more reconstructed pixel points around the single pixel point.

[0042] In another optional embodiment, the predicted QP value of the one pixel point is the QP value of the CU or QG on which the one pixel point is located, or is derived based on the QP values ​​of one or more reconstructed pixel points around the one pixel point, the derivation method comprising calculating at least one of the mean, median, or mode. In one embodiment, QG is a subblock of CB (coding block).

[0043] In another optional embodiment, the reconstructed pixel point is centered on the one pixel point The correct Pixel points within a rectangular region, or is a diamond These are pixel points within the shape region.

[0044] In another optional embodiment, the information of the reconstructed pixel point includes one or more combinations of pixel value, flatness information or texture information, background brightness, and contrast.

[0045] In another optional embodiment, the step of determining the QP value of a pixel point based on the predicted QP value and derived information of the pixel point is to determine the indication information of the pixel point based on information of one or more reconstructed pixel points surrounding the pixel point, and the indication information and, QP values ​​that correspond to the distortion that the human eye can perceive. Based on this, the predicted QP value was adjusted.The process includes the step of determining the QP value as the QP value of the pixel point. Here, the QP value corresponding to distortion that the human eye can perceive is a preset value (for example, preset image level or CU level information on the encoding or decoding side), or the QP value corresponding to distortion that the human eye can perceive is obtained by analyzing the bitstream (image level or CU level transmission), or the QP value corresponding to distortion that the human eye can perceive is derived based on flatness information or texture information of the surrounding reconstructed CUs, background brightness, contrast information.

[0046] According to a third aspect, the present invention provides an image coding method which may be applied to a video coding system, or which may be applied to an encoding side capable of supporting video coding to implement the method, the method being performed by the encoding side, the method comprising: dividing a single image frame into one or more CUs; ​​determining a plurality of QP values ​​for the single image frame, wherein one CU includes a plurality of QGs, and one QG corresponds to one QP value; and coding the single image frame based on the plurality of QP values.

[0047] According to a fourth aspect, the present invention provides an image coding method, which may be applied to a video coding system, or which may be applied to an encoding side capable of supporting video coding to implement the method, the method being performed by the encoding side, the method comprising the steps of dividing one image frame into one or more CUs, and determining a plurality of QP values ​​for the one image frame, wherein one CU includes a plurality of pixel points, one pixel point corresponds to one QP value, and at least two of the plurality of pixel points have different QP values. the law of nature , The QP value of a single pixel point is determined by adaptively adjusting it based on the reconstructed pixel information surrounding the current pixel point. The step includes: a step of encoding the one image frame based on the plurality of QP values.

[0048] According to a fifth aspect, the present invention provides an image decoding device, which is applied to the decoding side, and the decoding device includes modules for carrying out the method in any of the possible embodiments of the first or second aspect. For example, a bitstream analysis unit, a QP determination unit, and an image decoding unit.

[0049] The beneficial effects are described in the description of either the first or second embodiment and will not be repeated here. The decoding device has a function to implement the operation described in the example of either the first or second embodiment. The function may be implemented by hardware, or by the hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above function.

[0050] According to a sixth aspect, the present invention provides an image encoding apparatus, which is applied to the encoding side, and the encoding apparatus includes modules for carrying out the method in any of the possible embodiments of the third or fourth aspect. For example, the encoding apparatus includes an image segmentation unit, a QP determination unit, and an image encoding unit. Beneficial effects are described in the description of either the third or fourth aspect and will not be repeated here. The encoding apparatus has functions to implement the operations in the example of the method in either the third or fourth aspect. These functions may be implemented by hardware, or by the hardware running corresponding software. The hardware or software includes one or more modules corresponding to the above functions.

[0051] According to a seventh aspect, the present invention provides an electronic device including a processor and a memory, the memory being used to store computer instructions, and the processor being used to call and execute the computer instructions from the memory and to carry out the methods in any of the embodiments of the first to fourth aspects.

[0052] For example, the electronic device may refer to a video encoder, or an encoding side including a video encoder.

[0053] Furthermore, for example, the electronic device may refer to a video decoder, or a decoding side including a video decoder.

[0054] According to the eighth aspect, the present invention provides a computer-readable storage medium in which a computer program or instruction is stored, and when the computer program or instruction is executed by a computing device or a storage system in which the computing device is located, the method in any embodiment of the first to fourth aspects is carried out.

[0055] According to the ninth aspect, the present invention provides a computer program product including instructions, and when the computer program product is executed on a computing device or processor, the computing device or processor is made to execute the instructions, and the method in any embodiment of the first to fourth aspects is carried out.

[0056] According to a tenth aspect, the present invention provides a video coding system comprising an encoding side and a decoding side, wherein the decoding side is used to carry out a method in any embodiment of the first to second aspects, and the encoding side is used to carry out a method in any embodiment of the third to fourth aspects.

[0057] The present invention can also be further combined to provide more embodiments based on the embodiments provided in each of the above embodiments. [Brief explanation of the drawing]

[0058] [Figure 1] This is an illustrative block diagram of a video coding system according to the present invention. [Figure 2] This is an illustrative block diagram of a video encoder according to the present invention. [Figure 3] This is an exemplary block diagram of a video decoder according to the present invention. [Figure 4] This is a schematic flowchart of the video encoding / decoding process according to the present invention. [Figure 5] This is a schematic flowchart of the image decoding method according to the present invention. [Figure 6] This is a schematic diagram of the transformation region division according to the present invention. [Figure 7] This is a schematic diagram of the pixel region division according to the present invention. [Figure 8] This is a schematic flowchart for obtaining QP using the predictive coding scheme according to the present invention. [Figure 9] This is a schematic diagram of the pixel point distribution according to the present invention. [Figure 10] This is a schematic flowchart of image frame decoding according to the present invention. [Figure 11] This is a schematic diagram of the quantization matrix template according to the present invention. [Figure 12] This is a schematic flowchart of the image coding method according to the present invention. [Figure 13] This is a schematic diagram of the decoding device according to the present invention. [Figure 14] This is a schematic diagram of the encoding device according to the present invention. [Figure 15] This is a schematic diagram of the electronic device according to the present invention. [Modes for carrying out the invention]

[0059] To make the following descriptions of each embodiment clear and concise, we will first briefly explain the related technologies.

[0060] Figure 1 is an exemplary block diagram of a video coding system according to the present invention, and as used herein, the term “video coder” generally refers to both a video encoder and a video decoder. In the present invention, the terms “video coding” or “coding” may generally refer to video encoding or video decoding. The video encoder 100 and video decoder 200 of the video coding system 1 are used to predict motion information, such as motion vectors, of an image block or subblock currently being coded, according to examples of various methods described in any one of the multiple novel interframe prediction modes provided in the present invention, so that the predicted motion vectors are as close as possible to motion vectors obtained using a motion estimation method, thereby eliminating the need to transmit differential motion vectors during encoding and further improving encoding and decoding performance.

[0061] As shown in Figure 1, the video coding system 1 includes an encoding side 10 and a decoding side 20. The encoding side 10 generates encoded video data. Therefore, the encoding side 10 may be called a video encoding device. The decoding side 20 can decode the encoded video data generated by the encoding side 10. Therefore, the decoding side 20 may be called a video decoding device. Each embodiment of the encoding side 10, the decoding side 20, or both may include one or more processors and memory coupled to the one or more processors. As described herein, the memory may include, but is not limited to, RAM, ROM, EEPROM, flash memory, or any other medium that can be used to store desired program code in the form of computer-accessible instructions or data structures.

[0062] The encoding side 10 and the decoding side 20 may include a variety of devices, including desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, mobile phones such as so-called "smartphones," televisions, cameras, display devices, digital media players, video game consoles, in-car computers, or similar devices.

[0063] The decoding side 20 can receive encoded video data from the encoding side 10 via link 30. Link 30 may include one or more media or devices that can move the encoded video data from the encoding side 10 to the decoding side 20. In one example, link 30 may include one or more communication media that enable the encoding side 10 to directly transmit the encoded video data to the decoding side 20 in real time. In this example, the encoding side 10 may modulate the encoded video data according to a communication standard (e.g., a wireless communication protocol) and transmit the modulated video data to the decoding side 20. The one or more communication media may include wireless and / or wired communication media such as a radio frequency (RF) spectrum or one or more physical transmission lines. The one or more communication media may form part of a packet-based network such as a local area network, a wide area network, or a global network (e.g., the Internet). The one or more communication media may include routers, switches, base stations, or other devices that facilitate communication from the encoding side 10 to the decoding side 20.

[0064] In another example, the encoded data may be output to the storage device 40 via the output interface 140. Similarly, the encoded data may be accessed from the storage device 40 via the input interface 240. The storage device 40 may include one of several distributed or locally accessible data storage media, such as a hard disk drive, a Blu-ray disc, a digital video disc (DVD), a compact disc read-only memory (CD-ROM), flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data.

[0065] In another example, the storage device 40 may correspond to a file server or another intermediate storage device capable of holding encoded video generated by the encoding side 10. The decoder 20 may access the video data stored from the storage device 40 by streaming transmission or download. The file server may be any type of server capable of storing encoded video data and transmitting the encoded video data to the decoder 20. In one example, the file server may include a network server (e.g., for a website), a file transfer protocol (FTP) server, a network attached storage (NAS) device, or a local disk drive. The decoder 20 may access the encoded video data via any standard data connection (including an internet connection). This may include wireless channels (e.g., wireless fidelity (Wi-Fi) connections), wired connections (e.g., digital subscriber lines (DSL), cable modems, etc.), or a combination of both suitable for accessing encoded video data stored on the file server. Transmission of the encoded video data from the storage device 40 may be by streaming transmission, download transmission, or a combination of both.

[0066] The image decoding method according to the present invention may be applied, for example, to terrestrial television broadcasting, cable television transmission, satellite television transmission, streaming video transmission (e.g., via the Internet), encoding video data for storage on a data storage medium, decoding video data for storage on a data storage medium, or video encoding and decoding to support other applications. In some examples, the video coding system 1 may be used to support one-way or two-way video transmission to support applications such as video streaming transmission, video playback, video broadcasting and / or video phone.

[0067] The video coding system 1 shown in Figure 1 is merely an example, and the technology of the present invention can be applied to video coding configurations (e.g., video coding or video decoding) that do not necessarily involve any data communication between the coding device and the decoding device. In other examples, data may be retrieved from local memory and streamed over a network. The video coding device may encode the data and store it in memory, and / or the video decoding device may retrieve the data from memory and decode it. In many examples, coding and decoding are performed by devices that encode the data into memory and / or retrieve the data from memory and decode it, without any communication between them.

[0068] In the example in Figure 1, the encoding side 10 includes a video source 120, a video encoder 100, and an output interface 140. In some examples, the output interface 140 may include a modulator / demodulator (modem) and / or transmitter. The video source 120 may include a video capture device (e.g., a camera), a video archive containing previously captured video data, a video feed interface for receiving video data from a video content provider, and / or a computer graphics system for generating video data, or a combination of these sources of video data.

[0069] The video encoder 100 can encode video data from the video source 120. In some embodiments, the encoding side 10 transmits the encoded video data directly to the decoding side 20 via the output interface 140. In other embodiments, the encoded video data may be stored in a storage device 40 for the decoding side 20 to access for decoding and / or playback.

[0070] In the example in Figure 1, the decoding side 20 includes an input interface 240, a video decoder 200, and a display device 220. In some examples, the input interface 240 includes a receiver and / or modem. The input interface 240 may receive encoded video data via link 30 and / or from storage device 40. The display device 220 may be integrated with the decoding side 20 or provided outside of the decoding side 20. Generally, the display device 220 displays the decoded video data. The display device 220 may include various display devices, such as liquid crystal displays (LCDs), plasma displays, organic light-emitting diode (OLED) displays, or other types of display devices.

[0071] Although not shown in Figure 1, in some embodiments, the video encoder 100 and video decoder 200 may be integrated with an audio encoder and decoder, respectively, and may include a suitable multiplexer-demultiplexer unit or other hardware and software to accommodate the encoding of both audio and video in a common data stream or separate data streams. In some examples, where applicable, the demultiplexer (MUX-DEMUX) unit may conform to the ITU H.223 multiplexer protocol or other protocols such as the user datagram protocol (UDP).

[0072] Each of the video encoder 100 and video decoder 200 may be implemented, for example, as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, hardware, or any combination thereof. If the present invention is partially implemented in software, the device may implement the technology of the present invention by storing instructions for the software in a suitable non-volatile computer-readable storage medium and executing the instructions in hardware using one or more processors. Any of the above (including hardware, software, and combinations of hardware and software) may be considered as one or more processors. Each of the video encoder 100 and video decoder 200 may be included in one or more encoders or decoders, and any of the encoders or decoders may be integrated as part of a composite encoder / decoder (codec) in the corresponding device.

[0073] In this invention, the video encoder 100 may generally be referred to as another device that “signales” or “transmits” specific information to, for example, the video decoder 200. The terms “signaling” or “transmitting” may generally refer to the transmission of syntax elements and / or other data for decoding compressed video data. This transmission may occur in real time or near real time. Alternatively, this communication may occur over a period of time, for example, during encoding when syntax elements in the encoded bitstream are stored in a computer-readable storage medium, and the decoder may retrieve the syntax elements at any time after they have been stored in the medium.

[0074] JCT-VC developed the H.265 (HEVC) standard. HEVC standardization is based on an evolutionary model of video decoders called the HEVC Test Model (HEVC model, HM). The latest H.265 standard document can be obtained from http: / / www.itu.int / rec / T-REC-H.265, and the latest version of the standard document is H.265(12 / 16), which is incorporated herein by full text citation. The HM assumes that the video decoder has several additional capabilities compared to the existing algorithms of ITU-TH.264 / AVC. For example, while H.264 provides nine intra-frame predictive coding modes, the HM can provide up to 35 intra-frame predictive coding modes.

[0075] JVET is working on the development of the H.266 standard. The H.266 standardization process is based on an evolutionary model of a video decoder called the H.266 test model. The H.266 algorithm description can be obtained from http: / phenix.int-evry.fr / jvet, and the latest algorithm description is included in JVET-F1001-v2, which is incorporated herein by full citation. At the same time, reference software for the JEM test model can be obtained from https: / / jvet.hhi.fraunhofer.de / svn / svn_HMJEMSoftware / , which is also incorporated herein by full citation.

[0076] Generally, the HM operating model description allows a video frame or image to be divided into a sequence of tree blocks or largest coding units (LCUs) containing both luminance and chromaticity samples, also known as coding tree units (CTUs). Tree blocks serve a similar purpose to macroblocks in the H.264 standard. A slice contains several tree blocks that are consecutive in decoding order. A video frame or image may be divided into one or more stripes. Each tree block may be divided into coding units (CUs) based on a quadtree. For example, a tree block that is the root node of a quadtree may be divided into four child nodes, and each child node may be further divided into four other child nodes with the parent node. Child nodes that cannot be ultimately divided, as leaf nodes of a quadtree, include decoding nodes such as decoded video blocks. Syntax data associated with a decoded bitstream can define the maximum number of times a tree block can be divided and can also define the minimum size of a decoding node.

[0077] The size of the CU corresponds to the size of the decoding node and must be square in shape. The size of the CU can range from 8x8 pixels to a maximum of 64x64 pixels, or even larger tree block sizes.

[0078] A video sequence typically includes a series of video frames or images. A group of pictures (GOP) exemplifies a series of one or more video images. A GOP may include syntax data describing the number of images included in the GOP, either in the GOP header information, the header information of one or more images, or elsewhere. Each slice of an image may include slice syntax data describing the encoding mode of the corresponding image. The video encoder 100 typically operates on video blocks within individual video slices to encode video data. Video blocks may correspond to decoding nodes in the CU. Video blocks may have a fixed or variable size, and their size may vary depending on a specified decoding standard.

[0079] In this invention, "N×N" and "N by N" can be used interchangeably and refer to the pixel size of a video block according to the vertical and horizontal dimensions, for example, 16×16 pixels or 16 by 16 pixels. Generally, a 16×16 block has 16 pixels vertically (y=16) and 16 pixels horizontally (x=16). Similarly, an N×N block has N pixels vertically and N pixels horizontally, where N represents a non-negative integer value. Pixels within a block may be arranged in rows and columns. Furthermore, a block does not necessarily have to have the same number of pixels horizontally and vertically. For example, a block may contain N×M pixels, where M is not necessarily equal to N.

[0080] After intra-frame / inter-frame prediction decoding of the CU, the video encoder 100 may calculate residual data for the CU. The CU may include pixel data in a spatial domain (also called the pixel domain), and the CU may include coefficients in a transformation domain after applying a transformation (e.g., discrete cosine transform (DCT), integer transformation, discrete wavelet transform, or a conceptually similar transformation) to the residual video data. The residual data may correspond to the pixel difference between the pixels of the unencoded image and the predicted values ​​corresponding to the CU. The video encoder 100 may form a CU containing the residual data and generate transformation coefficients for the CU.

[0081] After generating conversion coefficients by an arbitrary transformation, the video encoder 100 may perform quantization of the conversion coefficients. Quantization is, for example, a process of further compression by quantizing the coefficients to reduce the amount of data required to represent them. The quantization process can reduce the bit depth associated with some or all of the coefficients. For example, an n-bit value may be truncated to an m-bit value during quantization, where n is greater than m.

[0082] In some possible embodiments, the video encoder 100 may scan the quantized transformation coefficients in a predefined scan order to generate a serialized vector that can be entropy coded. In other possible embodiments, the video encoder 100 may perform an adaptive scan. After scanning the quantized transformation coefficients to form a one-dimensional vector, the video encoder 100 may entropy coded the one-dimensional vector based on context-based adaptive variable-length code (CAVLC), context-based adaptive binary arithmetic coding (CABAC), syntax-based adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or other entropy coding methods. The video encoder 100 may also entropy coded syntax elements related to the coded video data so that the video decoder 200 can use them to decode the video data.

[0083] To perform CABAC, the video encoder 100 may assign a context in a context model to the transmitted symbols. The context may relate to whether the adjacent values ​​of the symbols are non-zero or not. To perform CAVLC, the video encoder 100 may select a variable-length code for the transmitted symbols. In variable-length coding (VLC), the codewords may be configured such that relatively short codes correspond to symbols with a high probability and long codes correspond to symbols with a low probability. Thus, using VLC can save bitrate compared to using a codeword of equal length for each transmitted symbol. Based on the context assigned to the symbols, the probability in CABAC can be determined.

[0084] In embodiments of the present invention, the video encoder may perform interframe prediction to reduce temporal redundancy between images. In the present invention, the CU currently being decoded by the video decoder may be called the current CU. In the present invention, the image currently being decoded by the video decoder may be called the current image.

[0085] Figure 2 is an illustrative block diagram of a video encoder according to the present invention. The video encoder 100 is used to output video to a post-processing entity 41. The post-processing entity 41 represents an example of a video entity that can process the encoded video data from the video encoder 100, such as a media-aware network element (MANE) or a stitching / editing device. In some examples, the post-processing entity 41 may be an example of a network entity. In some video encoding systems, the post-processing entity 41 and the video encoder 100 may be parts of separate devices, and in other cases, the functions described with respect to the post-processing entity 41 may be performed by the same device including the video encoder 100. In one example, the post-processing entity 41 is an example of the storage device 40 in Figure 1.

[0086] In the example in Figure 2, the video encoder 100 includes a prediction processing unit 108, a filter unit 106, a decoded picture buffer (DPB) 107, an adder 112, a transducer 101, a quantizer 102, and an entropy encoder 103. The prediction processing unit 108 includes an inter-frame predictor 110 and an intra-frame predictor 109. For image block reconstruction, the video encoder 100 further includes an inverse quantizer 104, an inverse transducer 105, and an adder 111. The filter unit 106 is intended to represent one or more loop filters, such as a deblocking filter, an adaptive loop filter (ALF), and a sample adaptive offset (SAO) filter. In Figure 2, the filter unit 106 is shown as an in-loop filter, but in other embodiments, the filter unit 106 may be implemented as an after-loop filter. In one example, the video encoder 100 may further include a video data memory and a splitting unit (not shown).

[0087] The video data memory may store video data to be encoded by the components of the video encoder 100. The video data stored in the video data memory can be retrieved from the video source 120. The DPB 107 may be a reference image memory that stores reference video data for the video encoder 100 to encode video data in intra-frame and inter-frame coding modes. The video data memory and DPB 107 may be formed by any one of a plurality of memory devices, for example, dynamic random access memory (DRAM) including synchronous dynamic random access memory (SDRAM), magnetic random access memory (MRAM), resistive random access memory (RRAM), or other types of memory devices. The video data memory and DPB 107 may be provided by the same memory device or by separate memory devices.

[0088] As shown in Figure 2, the video encoder 100 receives video data and stores the video data in video data memory. The splitting unit divides the video data into several image blocks, which may be further divided into smaller blocks, such as image block splitting based on a quadtree or binary tree structure. This splitting may include splitting into slices, tiles or other larger units. The video encoder 100 typically indicates the components that encode the image blocks in the video slice to be encoded. The slice may be divided into multiple image blocks (or into sets of image blocks called slices). The prediction processing unit 108 may select one of several possible coding modes to be used for the current image block, for example, one of several intra-frame coding modes or one of several inter-frame coding modes. The prediction processing unit 108 may provide the intra-frame and inter-frame coded blocks to the adder 112 to generate residual blocks and to the adder 111 to reconstruct the encoded blocks to be used as reference images.

[0089] The in-frame predictor 109 within the prediction processing unit 108 may perform in-frame predictive coding of the current image block for one or more adjacent blocks in the same frame or slice as the current block being coded, in order to eliminate spatial redundancy. The inter-frame predictor 110 within the prediction processing unit 108 may perform inter-frame predictive coding of the current image block for one or more predicted blocks in one or more reference images, in order to eliminate temporal redundancy.

[0090] Specifically, the interframe predictor 110 may be used to determine the interframe prediction mode to be used to encode the current image block. For example, the interframe predictor 110 may calculate the bitrate-distortion value of each interframe prediction mode in a candidate set of interframe prediction modes by bitrate-distortion analysis and select the interframe prediction mode with the optimal bitrate-distortion characteristics from among them. Bitrate-distortion analysis typically determines the amount of distortion (or error) between the encoded block and the original unencoded block encoded to produce the encoded block, and the bitcode rate (i.e., number of bits) for producing the encoded block. For example, the interframe predictor 110 may determine the interframe prediction mode for performing interframe prediction on the current image block as the interframe prediction mode with the smallest bitrate-distortion cost for encoding the current image block in the candidate set of interframe prediction modes.

[0091] The interframe predictor 110 predicts motion information (e.g., motion vectors) for one or more subblocks within the current image block based on the determined interframe prediction mode, and is used to obtain or generate predicted blocks for the current image block using the motion information (e.g., motion vectors) for one or more subblocks within the current image block. The interframe predictor 110 may position the predicted block pointed to by the motion vector in one of the reference image lists. The interframe predictor 110 may further generate syntax elements related to the image block and video slice for use by the video decoder 200 when decoding the image block of the video slice. Alternatively, in one example, the interframe predictor 110 obtains predicted blocks for the current image block by performing a motion compensation process using the motion information of each subblock to generate predicted blocks for each subblock. Here, the interframe predictor 110 performs motion estimation and motion compensation processes.

[0092] Specifically, after selecting an inter-frame prediction mode for the current image block, the inter-frame predictor 110 may provide the entropy encoder 103 with information indicating the selected inter-frame prediction mode for the current image block, so that the entropy encoder 103 encodes information indicating the selected inter-frame prediction mode.

[0093] The in-frame predictor 109 may perform in-frame prediction on the current image block. Specifically, the in-frame predictor 109 may determine an in-frame prediction mode for encoding the current block. For example, the in-frame predictor 109 may use bitrate-distortion analysis to calculate the bitrate-distortion value for each in-frame prediction mode under test and select an in-frame prediction mode with the best bitrate-distortion characteristics from the modes under test. In any case, after selecting an in-frame prediction mode for the image block, the in-frame predictor 109 may provide the entropy encoder 103 with information indicating the selected in-frame prediction mode for the current image block, so that the entropy encoder 103 encodes information indicating the selected in-frame prediction mode.

[0094] After the prediction processing unit 108 generates a prediction block of the current image block by inter-frame prediction and intra-frame prediction, the video encoder 100 subtracts the prediction block from the current image block to be encoded to form a residual image block. The adder 112 represents one or more components that perform this subtraction. The residual video data in the residual block may be contained in one or more transform units (TUs) and applied to the converter 101. The converter 101 converts the residual video data into residual transformation coefficients using a transformation such as a discrete cosine transform (DCT) or a conceptually similar transformation. The converter 101 can convert the residual video data from the pixel value domain to the transformation domain, for example, the frequency domain.

[0095] The converter 101 may transmit the obtained conversion coefficients to the quantizer 102. The quantizer 102 quantizes the conversion coefficients to further reduce the bit code rate. In some examples, the quantizer 102 may then perform a scan of the matrix containing the quantized conversion coefficients. Alternatively, the entropy encoder 103 may perform the scan.

[0096] After quantization, the entropy encoder 103 entropy encodes the quantized transformation coefficients. For example, the entropy encoder 103 may perform context-adaptive variable-length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), syntax-based adaptive binary arithmetic coding (SBAC), stochastic interval-partitioned entropy (PIPE) coding, or other entropy encoding methods or techniques. After being entropy encoded by the entropy encoder 103, the encoded bitstream may be transmitted to the video decoder 200 or archived for later transmission or retrieval by the video decoder 200. The entropy encoder 103 may entropy encode the syntax elements of the current image block being encoded.

[0097] The inverse quantizer 104 and inverse transformer 105 apply inverse quantization and inverse transform, respectively, to reconstruct the residual block in the pixel region to provide a reference block to be used later as a reference image. The adder 111 adds the reconstructed residual block to a predicted block generated by the inter-frame predictor 110 or the intra-frame predictor 109 to generate a reconstructed image block. The filter unit 106 may be applied to the reconstructed image block to reduce distortions such as block artifacts. The reconstructed image block is then stored as a reference block in the decoded image buffer 107 and may be used as a reference block by the inter-frame predictor 110 to perform inter-frame predictions for blocks in subsequent video frames or images.

[0098] Other structural variations of the video encoder 100 may be used to encode the video stream. For example, for some image blocks or frames, processing by the converter 101 may not be necessary, and the video encoder 100 may directly quantize the residual signal, and accordingly, processing by the inverse converter 105 may not be necessary; or for some image blocks or frames, the video encoder 100 may not generate residual data, and accordingly, processing by the converter 101, quantizer 102, inverse quantizer 104, and inverse converter 105 may not be necessary; or the video encoder 100 may not be processed by the filter unit 106, and the reconstructed image blocks may be directly stored as reference blocks; or the quantizer 102 and inverse quantizer 104 within the video encoder 100 may be combined.

[0099] Figure 3 is an exemplary block diagram of a video decoder 200 according to the present invention. In the example of Figure 3, the video decoder 200 includes an entropy decoder 203, a prediction processing unit 208, an inverse quantizer 204, an inverse converter 205, an adder 211, a filter unit 206, and a DPB 207. The prediction processing unit 208 may also include an inter-frame predictor 210 and an intra-frame predictor 209. In some examples, the video decoder 200 may perform a decoding process that is substantially the reverse of the encoding process described for the video encoder 100 in Figure 2.

[0100] In the decoding process, the video decoder 200 receives an encoded video bitstream from the video encoder 100, representing image blocks of the encoded video slice and associated syntax elements. The video decoder 200 may also receive video data from the network entity 42 and optionally store the video data in a video data memory (not shown). The video data memory may store video data decoded by the components of the video decoder 200, such as the encoded video bitstream. The video data stored in the video data memory can be retrieved, for example, by accessing a local video source such as a storage device 40 or a camera, wired or wireless network communication via video data, or a physical data storage medium. The video data memory can function as a decoded image buffer (CPB) for storing the encoded video data from the encoded video bitstream. Thus, although not shown in Figure 3, the video data memory and the DPB 207 may be the same memory or they may be separate memories. The video data memory and DPB 207 may be formed by, for example, any one of a plurality of memory devices, such as dynamic random access memory (DRAM) including synchronous DRAM (SDRAM), magnetic RAM (MRAM), resistive RAM (RRAM), or other types of memory devices.

[0101] The network entity 42 may be a server, a MANE, a video editor / cutter, or other device for implementing one or more of the techniques described above. The network entity 42 may or may not include a video encoder, for example, video encoder 100. Before the network entity 42 transmits the encoded video bitstream to the video decoder 200, the network entity 42 may implement some of the techniques described in this invention. In some video decoding systems, the network entity 42 and the video decoder 200 may be parts of separate devices, and in other cases, the functions described with respect to the network entity 42 may be performed by the same device including the video decoder 200. In some cases, the network entity 42 may be an example of the storage device 40 in Figure 1.

[0102] The entropy decoder 203 of the video decoder 200 entropy-decodes the bitstream to generate quantized coefficients and several syntax elements. The entropy decoder 203 transfers the syntax elements to the prediction processing unit 208. The video decoder 200 may receive the syntax elements at the video slice level and / or image block level.

[0103] If a video slice is decoded as an in-frame decoded (I) slice, the in-frame predictor 209 of the prediction processing unit 208 may generate a predicted block for the image block of the current video slice based on the in-frame prediction mode notified by signaling and data from a previous decoded block of the current frame or image. If a video slice is decoded as an inter-frame decoded (i.e., B or P) slice, the inter-frame predictor 210 of the prediction processing unit 208 may determine an inter-frame prediction mode for decoding the current image block of the current video slice based on the syntax elements received from the entropy decoder 203, and decode the current image block based on the determined inter-frame prediction mode (e.g., perform an inter-frame prediction). Specifically, the interframe predictor 210 may decide whether or not to use a new interframe prediction mode to predict the current image block of the current video slice, and if a syntax element indicates to use a new interframe prediction mode to predict the current image block, it may predict the motion information of the current image block or a subblock of the current image block of the current video slice based on the new interframe prediction mode (e.g., a new interframe prediction mode specified by a syntax element or a default new interframe prediction mode), thereby obtaining or generating a predicted block of the current image block or a subblock of the current image block using the motion information of the current image block or a subblock of the current image block predicted by the motion compensation process. Here, the motion information may include reference image information and motion vectors, and the reference image information may include, but is not limited to, one-way / two-way prediction information, a reference image list number, and a reference image index corresponding to the reference image list. For interframe prediction, a predicted block may be generated from one of the reference images in one of the reference image lists. The video decoder 200 may construct reference image lists, i.e., list 0 and list 1, based on the reference images stored in the DPB 207. The current image's reference frame index may be included in one or more of reference frame lists 0 and 1.In some examples, the video encoder 100 may signal a notification indicating whether or not to use a new interframe prediction mode to decode a particular syntax element of a particular block, or it may signal a notification indicating whether or not to use a new interframe prediction mode, and specifically which new interframe prediction mode to use to decode a particular syntax element of a particular block. Here, the interframe predictor 210 performs a motion compensation process.

[0104] The inverse quantizer 204 inversely quantizes, i.e., dequantizes, the quantized transformation coefficients provided to the bitstream and decoded by the entropy decoder 203. The inverse quantization process may include determining the degree of quantization to be added and similarly determining the degree of inverse quantization to be added, using quantization parameters calculated by the video encoder 100 for each image block in the video slice. The inverse converter 205 applies an inverse transform, such as an inverse DCT, inverse integer transform, or a conceptually similar inverse transform process, to the transformation coefficients to generate residual blocks in the pixel region.

[0105] After the interframe predictor 210 generates a predicted block for the current image block or a subblock of the current image block, the video decoder 200 adds the residual block from the inverse converter 205 with the corresponding predicted block generated by the interframe predictor 210 to obtain a reconstructed block, i.e., a decoded image block. The adder 211 represents the component that performs this addition. Optionally, a loop filter (in or after the decoding loop) may be used to smooth the pixel transformation, or the video quality may be improved in other ways. The filter unit 206 may represent one or more loop filters, such as a deblocking filter, an adaptive loop filter (ALF), and a sample-adaptive offset (SAO) filter. In Figure 3, the filter unit 206 is shown as an in-loop filter, but in other embodiments, the filter unit 206 may be implemented as an after-loop filter. In one example, the filter unit 206 is applied to block reconstruction to reduce block distortion, and the result is output as a decoded video stream. Alternatively, decoded image blocks within a predetermined frame or image may be stored in a decoded image buffer 207, and a reference image for subsequent motion compensation may be stored via the DPB 207. The DPB 207 may be part of a memory that stores decoded video for later display on a display device (e.g., the display device 220 in Figure 1), or it may be separate from such memory.

[0106] Other structural modifications of the video decoder 200 may be used to decode the encoded video bitstream. For example, the video decoder 200 may generate an output video stream without processing by the filter unit 206, or if the entropy decoder 203 of the video decoder 200 does not decode the quantized coefficients for some image blocks or image frames, it is not necessary for them to be processed by the inverse quantizer 204 and the inverse converter 205 accordingly.

[0107] The techniques of the present invention may be performed by either a video encoder or a video decoder described herein, such as the video encoder 100 and video decoder 200 shown and described in Figures 1 to 3. That is, in one possible embodiment, the video encoder 100 shown in Figure 2 may perform certain techniques described below when performing interframe prediction during encoding of blocks of video data. In another possible embodiment, the video decoder 200 shown in Figure 3 may perform certain techniques described below when performing interframe prediction during decoding of blocks of video data. Thus, a general reference to “video encoder” or “video decoder” may include the video encoder 100, the video decoder 200, or another video encoding or decoding unit.

[0108] Figures 1 to 3 are merely examples provided in the present invention, and in some examples, the video encoder 100, video decoder 200, and video coding system may include more or fewer components or units.

[0109] Based on the video coding system shown in Figures 1 to 3, the present invention provides a possible embodiment of video coding / decoding, as shown in Figure 4, which is a schematic flowchart of video coding / decoding according to the present invention, and the embodiment of video coding / decoding includes processes (1) to (5), which may be performed by one or more of the coding side 10, video encoder 100, decoding side 20, or video decoder 200.

[0110] Process (1): Divide a single frame of image into one or more non-overlapping parallel coding units. These one or more parallel coding units can encode and decode completely in parallel / independently, without depending on each other, as shown in parallel coding unit 1 and parallel coding unit 2 in Figure 4.

[0111] Process (2): Each parallel coding unit may be further divided into one or more independent coding units that do not overlap with each other, and each independent coding unit does not have to be dependent on one another, but may share some parallel coding unit header information.

[0112] For example, the width of an independent coding unit is w_lcu and its height is h_lcu. If a parallel coding unit is divided into a single independent coding unit, the size of the independent coding unit must be exactly the same as the parallel coding unit; otherwise, the width of the independent coding unit must be greater than its height (except in edge regions).

[0113] Typically, an independent coding unit may be a fixed w_lcu × h_lcu, where both w_lcu and h_lcu are 2 to the power of N (N≧0). For example, the sizes of independent coding units can be 128×4, 64×4, 32×4, 16×4, 8×4, 32×2, 16×2, or 8×2.

[0114] As one possible example, an independent coding unit may be a fixed 128×4. If the size of a parallel coding unit is 256×8, the parallel coding unit may be divided equally into four independent coding units, and if the size of a parallel coding unit is 288×10, the parallel coding unit is divided into two 128×4 + one 32×4 independent coding units in the first / second row, and two 128×2 + one 32×2 independent coding unit in the third row.

[0115] The independent coding unit may contain three components: luminance Y, chromaticity Cb, and chromaticity Cr, or three components: red (R), green (G), and blue (B), or it may contain only one of these components. If the independent coding unit contains three components, the sizes of these three components may be exactly the same or different, specifically related to the image input format.

[0116] Process (3): Each independent coding unit may be further divided into one or more non-overlapping coding units, and each coding unit within an independent coding unit may be dependent on one another, for example, multiple coding units may cross-reference to perform pre-coding and pre-decoding.

[0117] If the size of the coding unit and the independent coding unit are the same (i.e., if the independent coding unit is divided into only one coding unit), then the size may be any of the sizes described in process (2).

[0118] When an independent coding unit is divided into multiple coding units that do not overlap, possible examples of such division include horizontal equal division (where the height of the coding units is the same as the independent coding unit, but the width is different, and may be 1 / 2, 1 / 4, 1 / 8, 1 / 16, etc.), vertical equal division (where the width of the coding units is the same as the independent coding unit, but the height is different, and may be 1 / 2, 1 / 4, 1 / 8, 1 / 16, etc.), and horizontal and vertical equal division (quadtree division), with horizontal equal division being preferred.

[0119] If the width of an encoded unit is w_cu and its height is h_cu, then the width must be greater than the height (except in the case of edge regions). Typically, an encoded unit may be a fixed w_cu × h_cu, where both w_cu and h_cu are powers of 2 (where N is greater than or equal to 0), such as 16 × 4, 8 × 4, 16 × 2, 8 × 2, 8 × 1, 4 × 1, etc.

[0120] As one possible example, the coding unit may be a fixed 16x4. If the size of the independent coding unit is 64x4, it may be divided equally into four coding units, and if the size of the independent coding unit is 72x4, it may be divided into four 16x4 + one 8x4 coding unit.

[0121] The encoding unit may include three components: luminance Y, chromaticity Cb, and chromaticity Cr (or three components: red R, green G, and blue B), or it may include only one of these components. If it includes three components, the sizes of these components may be exactly the same or different, specifically related to the image input format.

[0122] Process (3) is an optional step in the video encoding / decoding method, and the video encoder / decoder may perform encoding / decoding on the residual coefficients (or residual values) of the independent encoding units obtained in process (2).

[0123] Process (4): The coding unit may be further divided into one or more non-overlapping prediction groups (PGs), and a PG may be abbreviated as a Group. Each PG is coded and decoded according to the selected prediction mode, the predicted values ​​of the PGs are obtained to constitute the predicted values ​​of the entire coding unit, and the residual values ​​of the coding unit are obtained based on the predicted values ​​and the original values ​​of the coding unit.

[0124] Process (5): Based on the residual values ​​of the encoded units, the encoded units are grouped, one or more non-overlapping residual blocks (RBs) are obtained, and the residual coefficients of each RB are encoded and decoded according to the selected mode to form a residual coefficient stream. Specifically, the residual coefficients can be divided into those that undergo transformation and those that do not.

[0125] Here, the selected mode for encoding and decoding the residual coefficients in process (5) may include, but is not limited to, any of the following: semi-fixed-length coding mode, exponential Golomb coding mode, Golomb-Rice coding mode, truncated unary coding mode, run-length coding mode, or a method for directly encoding the original residual values.

[0126] For example, a video encoder may directly encode the coefficients within the RB.

[0127] Alternatively, for example, a video encoder may perform transformations such as DCT, DST, or Hadamard transform on the residual block before encoding the transformed coefficients.

[0128] As one possible example, if the RB is relatively small, the video encoder may directly perform a unified quantization on each coefficient in the RB before performing binary encoding. If the RB is relatively large, it may be further divided into multiple coefficient groups (CGs), and a unified quantization may be performed on each CG before performing binary encoding. In some embodiments of the present invention, the coefficient groups (CGs) and quantization groups (QGs) may be the same.

[0129] The following provides an illustrative explanation of the encoding of residual coefficients using a semi-fixed-length coding scheme. First, the maximum absolute value of residuals within a single RB block is defined as the modified maximum (mm). Next, the number of bits required to encode the residual coefficients within that RB block is determined (the number of bits required to encode residual coefficients within the same RB block is consistent). For example, if the critical limit (CL) of the current RB block is 2 and the current residual coefficient is 1, then 2 bits are needed to encode the residual coefficient of 1, which is represented as 01. If the CL of the current RB block is 7, this means encoding an 8-bit residual coefficient and a 1-bit sign bit. Determining the CL involves finding the smallest M value that satisfies the condition that all residuals in the current subblock are within the range [-2^(M-1), 2^(M-1)]. If both boundary values ​​-2^(M-1) and 2^(M-1) exist simultaneously, M must be incremented by 1, meaning M+1 bits are needed to encode all residuals in the current RB block; if only one of the two boundary values ​​-2^(M-1) and 2^(M-1) exists, one trailing bit must be encoded to determine whether that boundary value is -2^(M-1) or 2^(M-1); and if neither -2^(M-1) nor 2^(M-1) exists in any of the residuals, the trailing bit does not need to be encoded.

[0130] Furthermore, in some special cases, the video encoder may encode the original image values ​​directly, rather than the residual values.

[0131] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0132] Figure 5 is a schematic flowchart of the image decoding method according to the present invention, which may be applied to the video coding system shown in Figure 1, which may be performed by the decoding side 20, which specifically may be performed by the video decoder 200 included in the decoding side 20, and referring to Figure 5, the decoding method according to this embodiment includes the following steps.

[0133] S510, the video decoder 200, analyzes the acquired bitstream to obtain one or more image frames.

[0134] Here, the above image frame contains one or more CUs.

[0135] As shown in Figure 5, one image frame may contain 3 × 5 = 15 CUs. Note that one image frame may contain more CUs, such as 20 CUs, or fewer CUs, such as 1 or 2 CUs.

[0136] S520, the video decoder 200 determines multiple QP values ​​for a single image frame.

[0137] Here, one CU contains multiple QGs, and one QG corresponds to one QP value.

[0138] The above single CU includes multiple residual coefficients, and the single QG includes some of the aforementioned multiple residual coefficients, and these some residual coefficients share one QP value.

[0139] In the encoding process, after the video encoder quantizes the residual coefficient, the residual coefficient is also called the level value. In this art, the residual coefficient and the level value may be collectively referred to as the residual coefficient, and in this embodiment, both are represented by the residual coefficient and should not be understood as a limitation to the present invention.

[0140] Compared to the large image distortion caused by using the same QP value for all residual coefficients within a single CU, in the video image decoding process provided in this embodiment, a single CU may be divided into multiple QGs, and one or more residual coefficients within each QG share a single QP value. Therefore, the video decoder can perform finer-grained QP determination for one or more CUs corresponding to image frames, reducing decoding distortion of image frames and improving the truthfulness and accuracy of video image decoding while ensuring a constant compression ratio.

[0141] In one optional embodiment, a QG may include one or more pixel points, each having a corresponding residual coefficient.

[0142] In the first possible case, if one QG contains one pixel point, the QG quantization process may also be called adaptive point-by-point quantization of the image frame, and this point-by-point quantization may be applied to CUs that do not perform QG partitioning. A feature of point-by-point quantization is that it allows each pixel point to use a different QP, which corresponds to refining the granularity of the QP to the pixel level. In this way, by dequantizing the image frame using the point-by-point quantization method, the subjective quality of the image frame can be greatly improved without changing the compression ratio of the image frame.

[0143] In the second possible case, if a single QG contains multiple pixel points, the residual coefficients corresponding to the multiple pixel points contained in a single QG may share a single QP value.

[0144] In another optional embodiment, the CU may be determined by the video decoder 200 dividing one CU contained in one image frame according to a first rule and obtaining multiple QGs.

[0145] The first rule includes a division region and a division method, wherein the division region is a transformation region or a pixel region, and the division method includes at least one of uniform division and non-uniform division.

[0146] The locations of multiple residual coefficients contained within a single CU are marked with coordinates, which may include both horizontal and vertical coordinates. For example, the location coordinates of a residual coefficient are (i,j), where i is the horizontal coordinate and j is the vertical coordinate. Several possible examples are provided below to illustrate the first rule and the partitioning of QGs ​​within a CU.

[0147] In the first possible example, if the type of the division region is a transformation region, the process by which the video decoder 200 divides one CU to obtain multiple QGs includes dividing the residual coefficients whose coordinate sum does not reach a first coordinate threshold into a first QG, and dividing the residual coefficients whose coordinate sum does reach a first coordinate threshold into a second QG. The coordinate sum is the sum of the horizontal and vertical coordinates of the residual coefficients.

[0148] Figure 6 is a schematic diagram of the transformation region division according to the present invention, where one CU contains regions corresponding to 16 residual coefficients, with the coordinates of the residual coefficient in the upper left corner being (1,1) and the coordinates of the residual coefficient in the lower right corner being (4,4).

[0149] QG(2-1) represents a two-part QG division scheme for the transformation region. For the residual coefficient at position (i,j), the residual coefficient satisfying "i+j ≤ threshold 1" is the first QG, and the others are the second QG. For example, threshold 1 is 5.5. Alternatively, the residual coefficient satisfying "i+j < threshold 1" may be the first QG, and the others may be the second QG.

[0150] In a second possible example, when the type of the division region is a conversion region, the process by which the video decoder 200 divides one CU to obtain a plurality of QGs includes dividing, among a plurality of residual coefficients, the residual coefficients whose coordinate sum has not reached a first coordinate threshold into a first QG, dividing the residual coefficients whose coordinate sum has reached the first coordinate threshold but has not reached a second coordinate threshold into a second QG, and dividing the residual coefficients whose coordinate sum has reached the second coordinate threshold into a third QG. The second coordinate threshold is greater than the first coordinate threshold.

[0151] As shown in FIG. 6, QG(3-1) shows a three-way QG division method for the conversion region. For the residual coefficient at the (i, j) position, the residual coefficients satisfying "i + j < threshold 1" are the first QG, the residual coefficients satisfying "threshold 1 ≤ i + j < threshold 2" are the second QG, and the residual coefficients satisfying "i + j ≥ threshold 2" are the third QG. Threshold 2 is greater than threshold 1. For example, threshold 1 = 5.5 and threshold 2 = 6.5.

[0152] Also, the residual coefficients satisfying "i + j ≤ threshold 1" may be the first QG, the residual coefficients satisfying "threshold 1 < i + j ≤ threshold 2" may be the second QG, and the residual coefficients satisfying "i + j > threshold 2" may be the third QG, where threshold 2 is greater than threshold 1. For example, threshold 1 = 5 and threshold 2 = 6.

[0153] In a third possible example, when the type of the division region is a conversion region, the process by which the video decoder 200 divides one CU to obtain a plurality of QGs includes sorting the plurality of residual coefficients and dividing, among the plurality of residual coefficients, the residual coefficients that have not reached a first proportional threshold into a first QG and dividing the residual coefficients that have reached the first proportional threshold into a second QG. The sorting method of the plurality of residual coefficients is either zigzag or reverse zigzag.

[0154] For example, as shown in FIG. 6, QG(2-2) shows a zigzag two-way QG division method for the conversion region. The first 7 / 16 (43.75%) of the residual coefficients are divided into the first QG, and the remaining residual coefficients are divided into the second QG.

[0155] Furthermore, as shown in Figure 6, for example, QG(2-3) represents the inverse zigzag bipartite method of the QG of the transformation region, where the first 7 / 16 (43.75%) of the residual coefficients are divided into the first QG, and the remaining residual coefficients are divided into the second QG.

[0156] In a fourth possible example, if the type of the division region is a transformation region, the process by which the video decoder 200 divides one CU to obtain multiple QGs includes sorting the multiple residual coefficients, dividing the residual coefficients that have not reached a first proportional threshold into a first QG, dividing the residual coefficients that have reached a first proportional threshold but have not reached a second proportional threshold into a second QG, and dividing the residual coefficients that have reached a second proportional threshold into a third QG, wherein the second proportional threshold is greater than the first proportional threshold.

[0157] For example, as shown in Figure 6, QG(3-2) represents a zigzag three-part division of the QG of the transformation region, where the first 5 / 16 (31.25%) of the residual coefficients are divided into the first QG, the last 3 / 16 (18.75%) of the residual coefficients are divided into the second QG, and the remaining residual coefficients are divided into the third QG.

[0158] Furthermore, as shown in Figure 6, for example, QG(3-3) represents the QG inverse zigzag tripartitioning method of the transformation region, where the first 5 / 16 (31.25%) of the residual coefficients are divided into the first QG, the last 3 / 16 (18.75%) of the residual coefficients are divided into the second QG, and the remaining residual coefficients are divided into the third QG.

[0159] The first to fourth possible examples described above are merely examples provided by this embodiment to illustrate the division of a transformation region within a CU and should not be understood as limitations to the present invention. When the type of division region is a transformation region, the choice of coordinate thresholds and proportional thresholds may be determined according to the image content or video encoding and decoding needs of the CU, and the present invention is not limited thereto. Furthermore, to improve the accuracy of QP determination within a single CU, the transformation region may be divided into more QGs, such as 4, 5, 10, or more, and the present invention is not limited thereto.

[0160] When the type of partitioned region is a pixel region, the following provides several possible examples to illustrate the first rule and the partitioning of QG within CU.

[0161] In a fifth possible example, the video decoder 200 performs a horizontal or vertical symmetric partition on a number of residual coefficients to obtain two QGs containing the same number of residual coefficients. The ratio of the residual coefficients in these two QGs is 1:1.

[0162] Figure 7 is a schematic diagram of the pixel region division according to the present invention, where QG_pixel(pixel, P)(2-1) shows an example of symmetrical bisection in the vertical direction of the CU, and QG_P(2-4) shows an example of symmetrical bisection in the horizontal direction of the CU.

[0163] In the sixth possible example, the video decoder 200 performs a horizontal or vertical symmetric partition on multiple residual coefficients to obtain three QGs. Of the three QGs, the number of residual coefficients in two non-adjacent QGs is the same, and the number of residual coefficients in the third QG is the same as the sum of the number of residual coefficients in the two non-adjacent QGs.

[0164] As shown in Figure 7, QG_P(3-1) shows an example of symmetrical trisection of CU in the vertical direction, where the number of residual coefficients in the QGs on both sides is the same, and the ratio of the number of residual coefficients in these three QGs is 1:2:1. QG_P(3-4) shows an example of symmetrical trisection of CU in the horizontal direction, where the ratio of the number of residual coefficients in these three QGs is 1:2:1.

[0165] In a seventh possible example, the video decoder 200 performs a horizontal or vertical split on multiple residual coefficients to obtain two QGs containing a mismatch in the number of residual coefficients.

[0166] For example, QG_P(2-2) shown in Figure 7 illustrates an example of dividing the CU vertically into two parts, and the ratio of the residual coefficients included in these two QGs is 1:3.

[0167] Furthermore, for example, QG_P(2-3) shows an example of dividing CU vertically into two parts, and the ratio of the residual coefficients included in these two QGs is 3:1.

[0168] Furthermore, for example, QG_P(2-5) shows an example of dividing CU horizontally into two parts, and the ratio of the residual coefficients included in these two QGs is 1:3.

[0169] Furthermore, for example, QG_P(2-6) shows an example of dividing CU horizontally into two parts, and the ratio of the residual coefficients included in these two QGs is 3:1.

[0170] In the eighth possible example, the video decoder 200 performs a horizontal or vertical division of multiple residual coefficients to obtain three QGs. The residual coefficients included in these three QGs do not have a symmetric relationship.

[0171] For example, QG_P(3-2) shown in Figure 7 illustrates an example of dividing the CU into three vertical sections, where the ratio of residual coefficients included in these three QGs is 1:1:2.

[0172] Furthermore, for example, QG_P(3-3) shows an example of dividing CU into three vertical sections, and the ratio of the residual coefficients included in these three QGs is 2:1:1.

[0173] Furthermore, for example, QG_P(3-5) shows an example of dividing CU into three horizontal sections, and the ratio of the residual coefficients included in these three QGs is 1:1:2.

[0174] Furthermore, for example, QG_P(3-6) shows an example of dividing CU into three horizontal sections, and the ratio of the residual coefficients included in these three QGs is 2:1:1.

[0175] The possible examples 5 through 8 described above are merely examples provided by this embodiment to illustrate the partitioning of pixel regions within a CU and should not be understood as limitations to the present invention. When the type of partitioned region is a pixel region, the ratio of residual coefficients included in the QG may be determined according to the image content or video encoding and decoding needs of the CU, and the present invention is not limited thereto. Furthermore, to improve the accuracy of QP determination within a single CU, a pixel region may be partitioned into more QGs, such as 4, 5, 10, or more, and the present invention is not limited thereto.

[0176] Thus, the video decoder 200 employs different QG partitioning methods for the residual coefficients of the pixel region and the conversion region, and also employs different QP quantization processes, thereby reducing distortion in image decoding. The QP quantization process is described in the description of the QP value acquisition process in this specification, so it will not be repeated here.

[0177] Continuing to refer to Figure 5, after the video decoder 200 has obtained multiple QP values ​​for a single image frame, the image decoding method according to this embodiment further includes the following step S530.

[0178] S530, the video decoder 200, decodes a single image frame based on multiple QP values.

[0179] For example, the video decoder 200 decodes one image frame based on multiple QP values ​​and then obtains the decoded image. After the video decoder 200 decodes multiple image frames, it obtains the video after decoding the bitstream.

[0180] Thus, in this embodiment, the video decoder can perform QP quantization of image decoding using QG as the granularity, and one CU may be divided into multiple QGs. Since one QG corresponds to one QP value, compared to the large image distortion caused by using the same QP value for all residual coefficients within one CU, in this embodiment, the video decoder can perform finer-grained QP determination for one or more CUs corresponding to image frames, reducing decoding distortion of image frames and improving the truthfulness and accuracy of video image decoding while ensuring a constant compression ratio.

[0181] Based on the technology provided in the present invention, in a video encoding / decoding process, the video encoder / video decoder obtains the QP value of each QG (or CU), and several possible embodiments are provided here.

[0182] In one possible embodiment, one QP value corresponding to one QG includes a luminance QP value and a chromaticity QP value. The luminance QP value (QP_Y) is the QP value required to quantize or dequantize the luminance (Luminance or Luma) of the image frame, and the chromaticity QP value is the QP value required to quantize or dequantize the chromaticity (Chrominance or Chroma) of the image frame. Exemplarily, for S520 above, the determination of multiple QP values ​​for a single image frame by the video decoder 200 may include the following two possible cases.

[0183] In the first possible case, the video decoder 200 obtains the luminance QP value and chromaticity QP value of one QG, respectively.

[0184] In the second possible case, the video decoder 200 first obtains the luminance QP value of one QG, and then the video decoder 200 determines the chromaticity QP value of one QG based on the luminance QP value. For example, the chromaticity QP value is the QP_Y value plus the QP offset values ​​of the picture parameter set (PPS) layer and the Slice layer.

[0185] For example, the above luminance QP values ​​and chromaticity QP values ​​may be obtained by the video decoder 200 analyzing the bitstream.

[0186] When performing QP determination (or QP quantization) on a QG contained in a CU, the video decoder 200 may first obtain the QP value at the CU level, and then obtain the QP value at the QG level within the CU.

[0187] Below, we provide two possible embodiments for determining multiple QP values ​​for a single image frame.

[0188] First possible embodiment, direct encoding / decoding: The video decoder 200 analyzes the bitstream to obtain mark information for one image frame, which is used to indicate the QP value of one QG, and / or the mark information is used to indicate the QP value of one CU.

[0189] For example, in near-lossless compression, the probability of a small QP value appearing is higher than that of a large QP value. Therefore, the video decoder 200 may dequantize the residual coefficients (level values) contained in the QG using the QP values ​​contained in the mark information added to the bitstream, using methods such as truncated unali coding, truncated Rice coding, or exponential Golomb coding.

[0190] Thus, when the video encoder 200 performs a QP determination for a single CU, the mark information may include the QP value of the CU. If it is not necessary to perform a QP determination for the QG within the CU, the video decoder 200 may determine the QP value of the CU based on the mark information attached to the bitstream. This avoids the video decoder having to infer image frames in the bitstream to obtain the QP value of the CU, thereby reducing the consumption of the video decoder's computational resources and improving the efficiency of image decoding.

[0191] Furthermore, for example, when the video encoder 200 performs a QP determination for multiple QGs within a single CU, the mark information may include the QP value of any one of the multiple QGs.

[0192] Furthermore, for example, the mark information may include the QP value of the CU and the QP value of any QG within the CU.

[0193] Thus, when the video encoder 200 performs a QP determination for a single QG, the mark information may include the QP value of the QG, thereby avoiding the video decoder having to infer image frames in the bitstream to obtain the QG value of the CU, reducing the consumption of the video decoder's computational resources, and improving the efficiency of image decoding.

[0194] A second possible embodiment, predictive coding / decoding: The video decoder 200 codes the difference (deltaQP) between the actual (encoded) QP value and the predicted QP (predict QP, predQP) value, which includes the steps of first obtaining the predQP of the current block (CU or QG), then determining the deltaQP of the current block, and finally determining that the actual (encoded) QP value is QP = predQP + deltaQP.

[0195] The following describes in detail, using the example that the video decoder 200 determines the QP value of one QG within one image frame. As shown in Figure 8, Figure 8 is a schematic flowchart for obtaining QP using the predictive coding scheme according to the present invention. The QP acquisition process may be implemented by a video decoder or a video encoder. Here, the video decoder 200 is described as an example, and the QP acquisition process includes the following steps.

[0196] S810, video decoder 200, obtains the predicted QP value of QG.

[0197] In one possible embodiment, the video decoder 200 determines the QP value of the CU on which the QG is located as the predicted QP value of the QG. Exemplarily, the QP value of the CU may be determined by the video decoder 200 analyzing the mark information of the bitstream.

[0198] In another possible embodiment, the video decoder 200 may first obtain the QP values ​​of at least one other QG adjacent to the QG in the CU where the QG is located, and then determine the predicted QP value of the QG based on the QP values ​​of the at least one other QG. For example, the video decoder 200 may determine the QP values ​​of the other QGs adjacent to the QG as the predicted QP value of the QG.

[0199] S820, video decoder 200, obtains the QP offset of QG.

[0200] In this specification, the QP offset may be represented as deltaQP.

[0201] Continuing with reference to Figure 8, this embodiment provides two possible embodiments for S820.

[0202] In a first optional embodiment, as shown in Figure 8, the video decoder 200 may determine the QP offset of the QG using mark information added to the bitstream, as in S820A.

[0203] The S820A, video decoder 200, analyzes the bitstream to obtain mark information that indicates the QP offset of the QG.

[0204] For example, after acquiring a bitstream, the video decoder 200 analyzes the bitstream to obtain mark information for the image frame in which the QG is located, and this mark information is used to indicate the QP offset (deltaQP) of the QG.

[0205] In a second optional embodiment, as shown in Figure 8, S820B, the video decoder 200 may determine the QP offset of the QG using derived information.

[0206] S820B, video decoder 200 determines the QP offset of QG based on the QG derivation information.

[0207] The derived information may be one or a combination of the following: flatness information or texture information of the QG, the remaining space in the bitstream buffer, or distortion constraint information. Here, the flatness information or texture information is used to indicate the image gradient of the QG, the distortion constraint information indicates the distortion threshold of any one of the multiple QGs contained in a single image frame, and the remaining space in the bitstream buffer is used to indicate the remaining available space in the bitstream buffer (e.g., buffer space).

[0208] In the first case, the derived information is flatness information or texture information, and the video decoder 200 may derive the QP offset of the QG based on the flatness information or texture information. For example, the video decoder 200 calculates the texture complexity of the current block (QG), uses a large QP (e.g., 20) for QGs with high texture complexity (e.g., reaching the texture complexity threshold), and uses a small QP (e.g., 5) for QGs with low texture complexity (e.g., not reaching the texture complexity threshold).

[0209] In the second case, the derived information is the remaining space in the bitstream buffer, and the video decoder 200 calculates the average number of bits BPPtotal for all pixels in the entire image and the average number of bits BPPleft for the remaining unencoded pixels, and if BPPleft > BPPtotal, it decreases QP, otherwise it increases QP. Here, BPPtotal and BPPleft may be obtained by the following formulas.

number

[0210] In the third case, if the derivation information is distortion constraint information, D represents the quantization distortion (the difference between the reconstructed value after inverse quantization and the residual before quantization), and for any matrix A,

number

[0211] (1) When quantizing the residuals of a pixel region, the necessary and sufficient condition that the maximum distortion of the pixel region does not exceed △ is:

number

[0212] (2) When the residuals of the transformation region are quantized, if R represents the residuals of the pixel region and U and V represent the horizontal and vertical transformation matrices, the following equation is satisfied between the maximum distortion of the pixel region and the maximum distortion of the transformation region. Therefore, a sufficient condition for the maximum distortion of the pixel region not to exceed △ is that the maximum distortion of the transformation region is

number

number

[0213] Continuing with Figure 8, the QP acquisition process provided in this embodiment further includes the following steps.

[0214] S830, video decoder 200 determines the QP value of QG based on the predicted QP value and QP offset of QG.

[0215] For example, the video decoder 200 determines the QP value of the QG as the sum of the predicted QP value and the QP offset.

[0216] Furthermore, in the third case provided by S820B, when the video decoder 200 obtains the QP value of QG, the following cases may exist.

[0217] The video decoder 200 determines the corresponding predicted distortion based on the reference QP value of the QG. If the predicted distortion is less than or equal to the distortion threshold, it determines the reference QP value as the QP value of the QG. If the predicted distortion is greater than the distortion threshold, it determines the QP value determined by the distortion threshold as the QP value of the QG.

[0218] In Example 1, if the derived information includes only strain constraint information, the reference QP value is the predicted QP value of QG determined in S810.

[0219] In Example 2, if the derived information includes distortion constraint information and texture information (or flatness information), the reference QP value may be a QP value obtained by adding the deltaQP determined by the texture information (or flatness information) and the predicted QP value determined in S810.

[0220] In Example 3, if the derived information includes distortion constraint information and the remaining space in the bitstream buffer, the reference QP value may be a QP value obtained by adding the deltaQP determined by the remaining space in the bitstream buffer and the predicted QP value determined in S810.

[0221] That is, the derived information may be a deltaQP for determining the QG, or an actual encoded QP value for directly determining the QG, and the specific process for using the derived information may be determined according to the needs of QP quantization / dequantization in video encoding and decoding, and the above three cases and three examples should not be understood as limitations to the present invention.

[0222] Examples 1 to 3 described above are merely examples to illustrate how this embodiment uses derived information to determine the QP value of the QG, and should not be understood as limitations to the present invention. In other examples, the derived information may simultaneously include distortion constraint information, texture information (or flatness information), and the remaining space in the bitstream buffer, and the present invention is not limited thereto.

[0223] The above-mentioned chromaticity / luminance QP values, direct coding / predictive coding, and bitstream addition / derivation information analysis are merely some examples provided in the present invention and should not be understood as limitations to the present invention.

[0224] In one optional embodiment, an image frame includes at least a first portion of the CU and a second portion of the CU, the first portion of the CU and the second portion of the CU have no overlapping region, and the methods for obtaining the QP values ​​of the first portion of the CU and the second portion of the CU are different. For example, the method for obtaining the QP value of the first portion of the CU is to append it to the mark information of the bitstream, and the method for obtaining the QP value of the second portion of the CU is to derive it from the video decoder 200.

[0225] Specifically, the process by which the video decoder 200 determines multiple QP values ​​for a single image frame may include, firstly, a process in which the video decoder 200 analyzes a bitstream to obtain mark information for a single image frame, the mark information including the QP offset of the first part of the CU; a process in which the video decoder 200 determines the QP value of the first part of the CU based on the mark information; secondly, a process in which the video decoder 200 obtains a predicted QP value for the second part of the CU; and a process in which the video decoder 200 determines the QP value of the second part of the CU based on the predicted QP value and the derived information. The relevant content of the derived information is described in the explanation of the derived information for QG mentioned above, so you just need to replace QG with CU therein, and it will not be repeated here.

[0226] For example, the video decoder 200 divides a single image into multiple regions and uses different QP processing methods for CUs in different regions. Reference QP values ​​for different regions are transmitted at the image level, and mark information for different regions is transmitted at the CU level. The CU-level QP for different regions may be obtained by a bitstream transmission method or by a decoding-side derivation method. For example, the video decoder divides an image into a region of interest (ROI) and a non-ROI region, obtains QP values ​​for the CUs in the ROI region (the first part of the CU) by a bitstream transmission method (e.g., the mark information described above), and obtains QP values ​​for the CUs in the non-ROI region (the second part of the CU) by a decoding-side derivation method (e.g., the derivation information described above).

[0227] As one optional embodiment, one CU of one image frame may include multiple QGs, and these multiple QGs may be partially QP quantized or fully QP quantized.

[0228] For example, in process S520, if all QGs contained in one CU of one image frame are quantized in the encoding process, the video decoder 200 may first determine the scan order of all QGs contained in one CU of the image frame. Then, for each QG, the video decoder 200 obtains the QP value of each QG according to the scan order. Here, the scan order includes any one of the following: top to bottom, left to right, zigzag, or reverse zigzag.

[0229] Specifically, if we quantize only some QGs and leave others quantized, we need to mark the positions of the quantized QGs and obtain the corresponding QPs for those QGs. For example, we could encode one QP offset for each of these QGs.

[0230] Furthermore, for example, in the process of S520, if some QGs contained in one CU of one image frame are quantized in the encoding process, the video decoder 200 may analyze the bitstream to determine one or more marked QGs in one CU of one image frame, and in the decoding process, these one or more marked QGs need to be dequantized by QP. In addition, for each of the one or more marked QGs, the video decoder 200 obtains the QP value of each QG.

[0231] Specifically, when quantizing all QGs, it is necessary to sequentially acquire the QP corresponding to each QG according to a certain scanning order, and this scanning order is related to the partitioning scheme and may be from top to bottom, left to right, zigzag, or inverse zigzag. For example, one QP offset may be encoded for each of these QGs.

[0232] In this way, for all QGs within a single CU of a single image frame, the QP quantization method (partial quantization or full quantization) of all QGs can be distinguished using mark information added to the bitstream. This avoids the video decoder performing QP quantization indiscriminately, reduces the computational resources and image distortion required for the video decoder to perform QP quantization, and improves the decoding efficiency and accuracy of the video.

[0233] Furthermore, if a single QG contains only one pixel point within an image frame, one pixel point corresponds to one QP value. When at least two of the pixel points in an image frame have different QP values, the QP value quantization process for that pixel point (point prediction mode) may use point-by-point quantization techniques. For example, such point-by-point quantization techniques include a process in which the video decoder 200 adaptively adjusts the QP value of the current pixel point based on reconstructed pixel information surrounding the current pixel point. Here, the reconstructed pixel information includes, but is not limited to, pixel values, flatness information or texture information, background brightness, contrast, etc.

[0234] The adaptive point-by-point quantization technique may be applied to QG or to CU without QG partitioning. The characteristic of point-by-point quantization is that it allows each pixel point within a single image frame to use a different QP value, which is equivalent to refining the granularity of QP value quantization to the pixel level.

[0235] One embodiment of point-by-point quantization technology is QP pred This represents the current QP value of CU or QG, and QP JND If we consider a QP value where ≥0 corresponds to Just Noticeable Distortion (JND), and offset>0 represents the QP offset value (which may be transmitted via the bitstream or pre-set), then the current pixel's QP value is adjusted as follows:

number

number

[0236] The two current methods for determining the QP values ​​of pixels described above are merely examples provided in this embodiment and should not be understood as limitations of the present invention.

[0237] In this way, by using point-by-point quantization technology, the video decoder can significantly improve the subjective quality of image frames and reduce image frame distortion without changing the compression ratio of the image frames.

[0238] The following provides one possible embodiment for describing the QP quantization process of pixel points within an image frame.

[0239] For example, the video decoder 200 obtains a predicted QP value for one pixel point and determines the QP value for one pixel point based on the predicted QP value and derived information for that pixel point.

[0240] In one possible case, the predicted QP value of a pixel point is the QP value of the CU or QG on which the pixel point is located. In another possible case, the predicted QP value of a pixel point is derived based on the QP values ​​of one or more reconstructed pixel points surrounding the pixel point, and the derivation method includes calculating at least one of the following: mean (e.g., the mean of the QP values ​​of multiple pixel points), median (e.g., the median of the QP values ​​of multiple pixel points), or mode (e.g., the QP value that occurs most frequently among the QP values ​​of multiple pixel points).

[0241] The pixel point derivation information described above may also be information about one or more reconstructed pixel points surrounding the pixel point. Here, the information about the reconstructed pixel point includes one or more combinations of pixel value, flatness information or texture information, background brightness, and contrast. Note that some of the information described above is merely an example provided in this embodiment and should not be understood as a limitation of the present invention.

[0242] Furthermore, the meaning of "surroundings" above can be understood as the neighboring pixel points of the pixel point whose QP value is determined, and here we will provide and explain some possible examples. As shown in Figure 9, Figure 9 is a schematic diagram of the distribution of pixel points according to the present invention, and (A) in Figure 9 shows an example of dividing a square region centered on the current pixel point, and here we provide two possible cases, in case 1, the reconstructed pixel point is a pixel point in a square region with side length 3 centered on the current pixel point, such as surrounding pixel point 1 shown in Figure 9 (A), and in case 2, the reconstructed pixel point is a pixel point in a square region with side length 5 centered on the current pixel point, such as surrounding pixel point 2 shown in Figure 9 (A).

[0243] Figure 9(B) shows an example of dividing a rhombus region centered on the current pixel point, providing two possible cases: Case 1, the reconstructed pixel point is a pixel point in a rhombus region with a diagonal length of 3, centered on the current pixel point, such as surrounding pixel point 1 shown in Figure 9(B); Case 2, the reconstructed pixel point is a pixel point in a rhombus region with a diagonal length of 5, centered on the current pixel point, such as surrounding pixel point 2 shown in Figure 9(B).

[0244] Figure 9 is merely an example illustrating how this embodiment illustrates reconstructed pixel points around the current pixel point and should not be understood as a limitation to the invention. In some other possible examples, the reconstructed pixel points around the current pixel point may be one or two pixel points adjacent to the current pixel point either vertically or horizontally.

[0245] As one optional embodiment, the step of determining the QP value of a pixel point based on the predicted QP value and derived information of the pixel point includes the step of determining indication information of the pixel point based on information of one or more reconstructed pixel points surrounding the pixel point, and if the indication information is less than or equal to a first threshold and the predicted QP value is greater than or equal to the QP value corresponding to the distortion that the human eye can just perceive, then determining the QP value corresponding to the distortion that the human eye can just perceive as the QP value of the pixel point.

[0246] The first threshold may be pre-set, or it may be determined based on compression or distortion requirements for video encoding and decoding. Alternatively, the first threshold may be determined based on user input information.

[0247] Furthermore, the QP value corresponding to the distortion that the human eye can perceive is image-level or CU-level information.

[0248] For example, a QP value corresponding to the distortion that the human eye can perceive is obtained by analyzing the bitstream, and for example, a QP value corresponding to the distortion that the human eye can perceive, for example 20, is added to the bitstream.

[0249] Furthermore, for example, the QP value corresponding to the distortion that the human eye can perceive is derived based on the flatness or texture information, background brightness, and contrast information of the surrounding reconstructed CU. The process for deriving the QP value is described in the relevant section of Figure 9, so it will not be repeated here.

[0250] Furthermore, the QP value corresponding to distortion that the human eye can perceive may be a value preset by the video encoder or video decoder, for example, 15. That is, the QP value corresponding to distortion that the human eye can perceive may not only be added to the mark information of the bitstream, but may also be analyzed by the video encoder or video decoder during video encoding / decoding, or it may be a preset QP value. In this embodiment, by introducing the QP value corresponding to distortion that the human eye can perceive into the determination of the current pixel point's QP value, each pixel point satisfies the judgment information corresponding to distortion that the human eye can perceive, reducing image distortion and improving the subjective quality of the image.

[0251] Here, we provide one possible example to illustrate the process of determining the QP value of a pixel point, as shown in Table 3, which outlines the QP value determination for the current pixel point's interval bisection. [Table 1]

[0252] Here, offset > 0 is the QP offset value (which may be transmitted in the bitstream or pre-set).

[0253] Method 1: If the indicator information is below the threshold, QP pred and QP JND The smaller of the two is determined as the current QP value of the pixel point, and if the indicated information is greater than the threshold, the predicted QP value of the pixel point (QP pred This value is determined as the QP value of the current pixel point.

[0254] Method 2: If the indicator information is below the threshold, QP pred and QP JND The smaller of the two is determined as the QP value of the current pixel point, and if the indication information is greater than the threshold, QP pred The sum of this value and the QP offset is determined as the QP value of the current pixel point.

[0255] Method 3: When the indicator information is below the threshold, QPpred Determine it as the QP value of the current pixel point. When the indication information is greater than the threshold value, the sum of the QP pred and the QP offset is determined as the QP value of the current pixel point.

[0256] Note that the above examples and Table 3 are only possible acquisition methods for the QP value of the current pixel point provided in this embodiment, and should not be understood as limitations to the present invention.

[0257] In this way, by introducing the QP value corresponding to the distortion that the human eye can just perceive into the QP value determination process of the current pixel point, each pixel point satisfies the determination information corresponding to the distortion that the human eye can just perceive, reduces the image distortion, and improves the subjective quality of the image.

[0258] After the video decoder 200 obtains the QP value of one image frame, for the above S530, the present invention provides a possible embodiment. FIG. 10 is a schematic flowchart of image frame decoding according to the present invention, and S530 may include the following steps S1010 to S1030.

[0259] S1010. For each QP value of a plurality of QP values, obtain the quantization step (Qstep) corresponding to the QP value.

[0260] The video decoder may obtain Qstep by at least one of mathematical derivation and table lookup based on the QP value. Hereinafter, four possible implementation methods are provided.

[0261] Method 1: Qstep = 2 × QP + 1.

[0262] Method 2: Represent the bit order of QP in octave. That is, every time the QP increases by 1 octave, Qstep doubles. Usually, octave is selected as 6 or 8. When offset represents one integer offset value,

Number

[0263] Method 3: Represent the bit order of QP in octave. That is, every time 1 octave of QP increases, Qstep doubles. Usually, octave is selected as 6 or 8. Represent one integer offset value with offset,

Number

Number

Number

[0264] Method 4: Provide examples of quantization and inverse quantization below.

Number

[0265] f ∈ [0, 1) represents a parameter for controlling rounding, {c i} is all the residual coefficients to be quantized within the current QG or CU, T can be obtained by the formula of Method 3,

Number

[0266] When T < M, Qstep is given by the following formula. Otherwise, both the quantization coefficient and the reconstructed value are zero.

Number

[0267] One possible embodiment of Method 4 is a = f * 2 T-M or a=2 T-M Therefore, M may be transmitted as a bitstream, or it may be directly obtained as bit depth.

[0268] When quantizing the residuals of a pixel region, Method 3 requires a clip operation to enable the quantization coefficient to be represented by "MT" bits, whereas the quantization coefficient in Method 4 can be naturally represented by "MT" bits without clipping.

[0269] Furthermore, in JPEG-XS, the uniform quantization method is set to f=0.5 in Method 4, and M is transmitted as a bitstream.

[0270] Note that a larger QP value results in a larger Qstep, coarser quantization, greater quantization distortion, and a smaller bitrate for coefficient coding. The parameter f is related to the length of the quantization dead zone; a smaller f results in a longer quantization dead zone and the quantized level value approaches zero. When f=0.5, the quantization and dequantization formulas provided by method 4 above correspond to rounding, and the quantization distortion is minimized. When f<0.5, a smaller f results in greater quantization distortion and a smaller bitrate for coefficient coding. In H.265, the I image is set to f=1 / 3 and the B / P image to f=1 / 6.

[0271] Referring again to Figure 8, the above S530 may further include the following steps.

[0272] S1020 retrieves the level value contained in QG corresponding to the QP value.

[0273] S1030, based on the selected combination of quantizers, the level value of the QG is dequantized.

[0274] The combination of quantizers includes one or more quantizers, which can be uniform quantizers or non-uniform quantizers.

[0275] For example, a uniform quantizer refers to a uniform scalar quantizer. The quantization or inverse quantization formula can refer to the quantization and inverse quantization formulas provided in Method 4 of S1010 above. The parameter f ∈ [0, 1) is set as follows.

[0276] Method 1: Set f to 0.5 or other fixed value.

[0277] Method 2: f can be adaptively determined based on the QP value, prediction mode, and whether to perform transformation.

[0278] Also, for example, a non-uniform quantizer refers to a non-uniform scalar quantizer. The correspondence between the quantization level value, quantization interval, and reconstructed value of inverse quantization may be obtained by table lookup. One possible embodiment is shown in Table 1 below.

Table 2

[0279] Here, 0 ≤ x0 < x1 < x2 < x3 <..., the reconstructed value and quantization interval are non-uniform, and the reconstructed value is the probability centroid of the quantization interval where it is located.

[0280] The combination of quantizers may use one non-uniform scalar quantizer or multiple non-uniform scalar quantizers.

[0281] For example, the combination of quantizers is determined by the mark information added to the bitstream.

[0282] Furthermore, for example, the combination of quantizers is determined by the distribution of residual coefficients within the QG. For instance, a video decoder adaptively selects which quantizer to use, and the basis for this selection may be mode information or transformation information, which is related to the distribution of residual coefficients within the QG.

[0283] Regarding S1030 above, the inverse quantization of the QG level value may include the following process.

[0284] First, determine the partitioning region type of the QG.

[0285] Next, if the partitioned region type of the QG is a transformation region, a quantization matrix that matches the parameter information of the QG is selected from the matrix template library on the decoding side.

[0286] The matrix template library includes multiple types of quantization matrix templates, the parameter information of which includes one or a combination of the following: the size of the QG, the size of the CU in which the QG is located, brightness channel information, and flatness information.

[0287] Finally, the residual coefficients of QG are obtained by inverse quantizing the level values ​​in QG using the quantization matrix of QG.

[0288] Since residual coefficients at different positions within the pixel region are equally important, a quantization matrix is ​​not used. The coefficients in the transformation region are divided into low-frequency and high-frequency coefficients, and a quantization matrix causes the high and low-frequency coefficients to use different quantization steps, thereby improving the subjective quality of the image while ensuring a constant compression ratio.

[0289] The element distribution of the quantization matrix has a specific template, and in the present invention, coding blocks of different sizes can use different quantization matrix templates, and larger quantization matrices may be obtained by upsampling one or more smaller quantization matrices. The quantization matrix templates included in the matrix template library may be obtained by one or more transformation schemes among discrete cosine transform (DCT), discrete sine transform (DST), integer transform, or discrete wavelet transform (DWT). As shown in Figure 11, Figure 11 is a schematic diagram of a quantization matrix template according to the present invention, and the quantization matrix template specifically includes the following.

[0290] (1) When both the horizontal and vertical directions are DCT / DST transformations (see (1) in Figure 11), coefficients on the same anti-diagonal have the same frequency and employ the same quantization step, while coefficients on different anti-diagonal may employ different quantization steps.

[0291] (2) When both the horizontal and vertical directions are wavelet transforms (see (2) in Figure 11), a quadtree template may be adopted, with the four subblocks corresponding to low / medium / high frequencies, respectively, and the same frequency band subblocks employing the same quantization step, while subblocks in different frequency bands employing different quantization steps.

[0292] (3) When the vertical direction is DCT / DST transform and the horizontal direction is wavelet transform (see (3) in Figure 11), first divide the vertical direction equally into two parts, with the left subblock being the low frequency of the wavelet transform and the right subblock being the high frequency of the wavelet transform. The upper part of each subblock is the low frequency of the DCT / DST transform, and the lower part is the high frequency of the DCT / DST transform. The matrix elements A' / B' / C' / D' may also be A / B / C / D with the same offset value added to them, respectively.

[0293] (4) When the horizontal direction is DCT / DST transform and the vertical direction is wavelet transform (see (4) in Figure 11), first divide the horizontal direction equally into two parts, with the upper subblock being the low frequency of the wavelet transform and the lower subblock being the high frequency of the wavelet transform. Inside each subblock, the left side is the low frequency of the DCT / DST transform and the right side is the high frequency of the DCT / DST transform. The matrix elements A' / B' / C' / D' may also be A / B / C / D with the same offset value added to them, respectively.

[0294] Optionally, the matrix template library may include multiple types of quantized matrix templates, such as flat block templates and texture block templates. In flat block templates, the Qstep of residual coefficients with frequencies above the frequency threshold is greater than or equal to the Qstep of residual coefficients with frequencies below the frequency threshold. In texture block templates, the Qstep of residual coefficients with frequencies above the frequency threshold is less than or equal to the Qstep of residual coefficients with frequencies below the frequency threshold.

[0295] For example, a video decoder marks the current block as either a flat block or a textured block based on flatness information and designs a quantization matrix template according to the texture masking effect. (1) If the current block (QG) is a flat block, the Qstep of the high-frequency coefficients in the quantization matrix is ​​greater than or equal to the Qstep of the low-frequency coefficients, and the high-frequency coefficients tolerate greater loss because the human eye is more sensitive to the low-frequency distortion of a flat block than to the high-frequency distortion. (2) If the current block (QG) is a textured block, the Qstep of the high-frequency coefficients in the quantization matrix is ​​less than or equal to the Qstep of the low-frequency coefficients, and the high-frequency coefficients of a textured block are preferentially protected because the human eye is more sensitive to the high-frequency distortion of a textured block than to the low-frequency distortion.

[0296] In other words, in the image decoding method, the video decoder first obtains the QP and Qstep of the QG, analyzes the level values ​​from the bitstream, then adaptively selects a quantizer, dequantizes the level values, and obtains a reconstructed value, thereby decoding the image frame.

[0297] In conclusion, compared to the large image distortion caused by using the same QP value for all residual coefficients within a single CU, the video image decoding process provided in this embodiment allows a single CU to be divided into multiple QGs, and one or more residual coefficients within each QG share a single QP value. Therefore, the video decoder can perform finer-grained QP determination for one or more CUs corresponding to image frames, reducing decoding distortion of image frames and improving the truthfulness and accuracy of video image decoding while ensuring a constant compression ratio.

[0298] In response to this, in the image encoding method, the video encoder first obtains the QP, Qstep, and residual coefficients of the QG, adaptively selects a quantizer, quantizes the residual coefficients, and finally adjusts the quantization coefficients to obtain the final level value, thereby achieving the encoding of the image frame.

[0299] Based on the video encoder 100 shown in Figure 2, the present invention further provides an image encoding method, as shown in Figure 12, which is a schematic flowchart of the image encoding method according to the present invention, and the image encoding method may be performed by the video encoder 100 or by an encoding side (for example, the encoding side 10 shown in Figure 1) that supports the functions of the video encoder 100, and here, as an example, the video encoder 100 implements the encoding method, and the image encoding method includes the following steps.

[0300] S1210, the video encoder 100 divides one image frame into one or more CUs.

[0301] S1220, the video encoder 100 determines multiple QP values ​​for a single image frame.

[0302] In one example, one CU contains multiple QGs, and one QG corresponds to one QP value.

[0303] In another example, one CU contains multiple pixel points, each pixel point corresponds to one QP value, and at least two of the multiple pixel points have different QP values.

[0304] S1230, the video encoder 100 encodes a single image frame based on multiple QP values.

[0305] The quantization of the QP value in the encoding method is described in the corresponding process in the decoding method shown in Figures 4 to 11 above, so it will not be repeated here.

[0306] Thus, compared to the large image distortion caused by using the same QP value for all residual coefficients within a single CU, in the video image encoding process provided in this embodiment, a single CU may be divided into multiple QGs (or pixel points), and one or more residual coefficients within each QG share a single QP value. Therefore, the video encoder can perform finer-grained QP determination for one or more CUs corresponding to an image frame, reducing encoding distortion of the image frame and improving the truthfulness and accuracy of the video image encoding while ensuring a constant compression ratio.

[0307] Furthermore, in order to realize the functions in the above embodiments, the video encoder / video decoder includes corresponding hardware structures and / or software modules for performing each function. As will be readily apparent to those skilled in the art, the present invention can be implemented in hardware or as a combination of hardware and computer software, by referring to the units and method steps of each example described in the embodiments disclosed herein. Whether a function is performed in hardware or in a manner in which computer software drives the hardware depends on the specific application scenario and design constraints of the technical proposal.

[0308] Figure 13 is a schematic diagram of a decoding device according to the present invention, the decoding device 1300 including a bitstream analysis unit 1310, a QP determination unit 1320, and an image decoding unit 1330. The decoding device 1300 may be used to perform the functions of the video decoder or decoding side in the embodiment of the above decoding method, and thus can also realize the beneficial effects of the embodiment of the above decoding method. In the embodiment of the present invention, the decoding device 1300 may be the decoding side 20 or video decoder 200 shown in Figure 1, the video decoder 200 shown in Figure 3, or a module applied to the decoding side 20 or video decoder 200.

[0309] The bitstream analysis unit 1310, the QP determination unit 1320, and the image decoding unit 1330 are used to carry out the decoding method provided in any of the embodiments shown in Figures 4 to 11. A more detailed description of the bitstream analysis unit 1310, the QP determination unit 1320, and the image decoding unit 1330 can be directly obtained by referring directly to the relevant descriptions in the embodiments of the method shown in Figures 4 to 11, and is therefore not repeated here.

[0310] Figure 14 is a schematic diagram of an encoding device according to the present invention, the encoding device 1400 including an image division unit 1410, a QP determination unit 1420, and an image encoding unit 1430. The encoding device 1400 may be used to perform the functions of a video encoder or encoding side in the above embodiment of the decoding method, and thus can also realize the beneficial effects of the above embodiment of the encoding method. In the embodiment of the present invention, the encoding device 1400 may be the encoding side 10 or video encoder 100 shown in Figure 1, the video encoder 100 shown in Figure 2, or a module applied to the encoding side 10 or video encoder 100.

[0311] The image segmentation unit 1410, the QP determination unit 1420, and the image coding unit 1430 are used to implement the coding method provided in Figure 12. A more detailed explanation of the image segmentation unit 1410, the QP determination unit 1420, and the image coding unit 1430 can be directly obtained by referring directly to the relevant explanations in the embodiments of the method shown in Figures 4 to 12, and is therefore not repeated here.

[0312] The present invention further provides an electronic device, as shown in Figure 15, which is a schematic diagram of the structure of an electronic device according to the present invention, the electronic device 1500 including a processor 1510 and an interface circuit 1520. The processor 1510 and the interface circuit 1520 are coupled to each other. The interface circuit 1520 may be a transceiver or an input / output interface. Optionally, the electronic device 1500 may further include a memory 1530 for storing instructions executed by the processor 1510, or for storing input data necessary for the processor 1510 to execute instructions, or for storing data generated after the processor 1510 has executed instructions.

[0313] The electronic device 1500 includes a processor 1510 and a communication interface 1520. The processor 1510 and the communication interface 1520 are coupled to each other. The communication interface 1520 may be a transceiver or an input / output interface. Optionally, the electronic device 1500 may further include a memory 1530 for storing instructions executed by the processor 1510, or for storing input data necessary for the processor 1510 to execute instructions, or for storing data generated after the processor 1510 has executed instructions.

[0314] When the electronic device 1500 is used to carry out the methods shown in Figures 4 to 11, the processor 1510 and the interface circuit 1520 are used to perform the functions of the bitstream analysis unit 1310, the QP decision unit 1320, and the image decoding unit 1350 described above.

[0315] When the electronic device 1500 is used to carry out the method shown in Figure 12, the processor 1510 and the interface circuit 1520 are used to perform the functions of the image segmentation unit 1410, the QP determination unit 1420, and the image encoding unit 1430 described above.

[0316] In embodiments of the present invention, the specific connection medium between the communication interface 1520, processor 1510, and memory 1530 is not limited. In embodiments of the present invention, in Figure 15, the communication interface 1520, processor 1510, and memory 1530 are connected via a bus 1540, which is shown as a thick line in Figure 15. The connection methods between other components are illustrative and not particularly limited. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is shown in Figure 15, but this does not mean that there is only one bus or only one type of bus.

[0317] The memory 1530 may be used to store software programs and modules, such as program instructions / modules corresponding to the decoding or encoding methods provided in embodiments of the present invention, and the processor 1510 performs various functional applications and data processing by executing the software programs and modules stored in the memory 1530. The communication interface 1520 may be used for signaling and data communication with other devices. In the present invention, the electronic device 1500 may have a plurality of communication interfaces 1520.

[0318] In the embodiments of the present invention, the processor may be a central processing unit (CPU), a neural processing unit (NPU), or a graphics processing unit (GPU), or it may be another general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or another programmable logic device, a transistor logic device, a hardware component, or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0319] The steps of the method in embodiments of the present invention may be implemented by hardware, or by a processor executing software instructions. The software instructions may consist of corresponding software modules, which may be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, removable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from and write information to the storage medium. Of course, the storage medium may also be a component of the processor. The processor and storage medium may be located within an ASIC, which may be located within a network device or terminal device. Of course, the processor and storage medium may also exist as independent components within the network device or terminal device.

[0320] In the embodiments described above, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments may be implemented in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer programs or instructions are loaded on a computer and executed, all or part of the processes or functions described in the embodiments of the present invention are executed. The computer may be a general-purpose computer, a dedicated computer, a computer network, a network device, user equipment, or other programmable device. The computer programs or instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer programs or instructions may be transmitted by wire or wirelessly from one website, computer, server, or data center to another website, computer, server, or data center. The computer-readable storage medium may be any available medium accessible by a computer, or a data storage device such as a server or data center that integrates one or more available media. The aforementioned usable media may be magnetic media such as floppy disks, hard disks, or magnetic tapes, or optical media such as digital video discs (DVDs), or semiconductor media such as solid state drives (SSDs).

[0321] In each embodiment of the present invention, unless otherwise specified and logically contradictory, terminology and / or descriptions between different embodiments are consistent and can be referenced to one another, and technical features in different embodiments can be combined according to their internal logical relationships to form new embodiments. In the present invention, "at least one" means one or more, and "multiple" means two or more. "And / or" describes the relationship between related objects and indicates that there may be three types of relationships, for example, A and / or B can mean that A exists alone, A and B exist simultaneously, or B exists alone, and A and B may be singular or plural. In the literal description of the present invention, the letter " / " generally indicates that the preceding and succeeding related objects are in an "or" relationship, and in the formulas of the present invention, the letter " / " indicates that the preceding and succeeding related objects are in a "division" relationship.

[0322] The various numerical values ​​related to the embodiments of the present invention are merely assigned for the sake of explanation and are not intended to limit the scope of the embodiments of the present invention. The magnitude of the numbers for each process described above does not indicate the execution order; the execution order of each process should be determined by its function and internal logic. [Explanation of Symbols]

[0323] 1300 Decoder 1310 Bitstream Analysis Unit 1320 QP Decision Unit 1330 Image Decoding Unit 1400 encoding device 1410 Image Segmentation Unit 1420 QP Decision Unit 1430 Image Encoding Unit 1500 Electronic Devices 1510 Processor 1520 Communication Interface 1530 memory

Claims

1. An image decoding method performed by the decoding side, A step of analyzing a bitstream to obtain one or more image frames, wherein one image frame includes one or more encoding units CU, A step of determining multiple quantization parameter QP values ​​of one image frame, wherein one CU includes multiple quantization groups QG, one QG corresponds to one QP value, and at least two of the QGs have different QP values. The step of decoding one image frame based on the plurality of QP values ​​is included, The aforementioned one CU includes a plurality of residual coefficients, the aforementioned one QG includes some of the residual coefficients among the plurality of residual coefficients, and the aforementioned some residual coefficients share the aforementioned one QP value. Before determining the multiple QP values ​​of the aforementioned single image frame, The process further includes the step of dividing one CU contained in one image frame according to a first rule and obtaining the plurality of QGs, The first rule includes a division region and a division method, wherein the division region is a pixel region, and the division method includes uniform division. An image decoding method characterized by the following:

2. The QP value corresponding to the aforementioned QG includes a luminance QP value and a chromaticity QP value. The step of determining multiple QP values ​​for the aforementioned single image frame is: A step of obtaining the luminance QP value and chromaticity QP value of the aforementioned QG, Or, The luminance QP value of one of the aforementioned QGs is obtained, The step includes obtaining the chromaticity QP value of one QG based on the luminance QP value, The method according to feature 1.

3. The step of decoding one image frame based on the plurality of QP values ​​is: For each of the plurality of QP values, the step of obtaining a quantization step Qstep corresponding to the QP value, The steps include obtaining the level value included in QG corresponding to the aforementioned QP value, The step of dequantizing the level value of the QG based on the selected combination of quantizers, The combination of quantizers includes one or more quantizers. The method according to feature 1.

4. The combination of quantizers is determined by the mark information added to the bitstream, or The combination of quantizers is determined by the distribution of residual coefficients within the QG. The method according to feature 3.

5. The quantizer is either a homogeneous quantizer or a non-homogeneous quantizer. The method according to feature 3.

6. The aforementioned QG includes one or more pixel points of the aforementioned image frame. The method according to feature 1.

7. An image encoding method performed by the encoding side, The steps include dividing one image frame into one or more encoding units CU, A step of determining multiple quantization parameter QP values ​​of one image frame, wherein one CU includes multiple quantization groups QG, one QG corresponds to one QP value, and at least two of the QGs have different QP values. The step of encoding one image frame based on the plurality of QP values, The aforementioned one CU includes a plurality of residual coefficients, the aforementioned one QG includes some of the residual coefficients among the plurality of residual coefficients, and the aforementioned some residual coefficients share the aforementioned one QP value. Before determining the multiple QP values ​​of the aforementioned single image frame, The process further includes the step of dividing one CU contained in one image frame according to a first rule and obtaining the plurality of QGs, The first rule includes a division region and a division method, wherein the division region is a pixel region, and the division method includes uniform division. An image encoding method characterized by the following.

8. An image decoding apparatus comprising a bitstream analysis unit, a QP determination unit, and an image decoding unit, The bitstream analysis unit is used to analyze the bitstream and obtain one or more image frames, and one image frame includes one or more encoding units CU. The QP determination unit is used to determine multiple quantization parameter QP values ​​of one image frame, where one CU includes multiple quantization groups QG, one QG corresponds to one QP value, and at least two of the multiple QGs have different QP values. The image decoding unit is used to decode one image frame based on the plurality of QP values. The aforementioned one CU includes a plurality of residual coefficients, the aforementioned one QG includes some of the residual coefficients among the plurality of residual coefficients, and the aforementioned some residual coefficients share the aforementioned one QP value. The aforementioned QP determination unit further, A single CU contained in the single image frame is divided according to the first rule and used to obtain the multiple QGs, The first rule includes a division region and a division method, wherein the division region is a pixel region, and the division method includes uniform division. An image decoding device characterized by the following features.

9. An image encoding apparatus comprising an image segmentation unit, a QP determination unit, and an image encoding unit, The image splitting unit is used to split one image frame into one or more encoding units CU. The QP determination unit is used to determine multiple quantization parameter QP values ​​of one image frame, where one CU includes multiple quantization groups QG, one QG corresponds to one QP value, and at least two of the multiple QGs have different QP values. The image encoding unit is used to encode one image frame based on the plurality of QP values. The aforementioned one CU includes a plurality of residual coefficients, the aforementioned one QG includes some of the residual coefficients among the plurality of residual coefficients, and the aforementioned some residual coefficients share the aforementioned one QP value. The aforementioned QP determination unit further, A single CU contained in the single image frame is divided according to the first rule and used to obtain the multiple QGs, The first rule includes a division region and a division method, wherein the division region is a pixel region, and the division method includes uniform division. An image coding device characterized by the following:

10. A video coding system including an encoding side and a decoding side, The encoding side and the decoding side are connected in a communicative manner, the decoding side is used to carry out the method described in any one of claims 1 to 6, and the encoding side is used to carry out the method described in claim 7. A video coding system characterized by the following features.

11. An electronic device including a processor and memory, The memory is used to store computer instructions, and the processor is used to call and execute the computer instructions from the memory and to carry out the method according to any one of claims 1 to 7. An electronic device characterized by the following features.

12. A computer-readable storage medium in which a computer program or instruction is stored, wherein when the computer program or instruction is executed by an electronic device, the method according to any one of claims 1 to 7 is performed. A computer-readable storage medium characterized by the following features.

Images