Image decoding method, image encoding method and device

Abstract

The present invention discloses an image decoding method, an encoding method, and an apparatus, which include the steps of: determining a prediction residual value of a first pixel point based on a prediction scheme of the first pixel point, the prediction residual value being used to reflect a gradient of the first pixel point, the prediction scheme being used to indicate a position of a reconstructed pixel point referenced in predicting the pixel point, the first pixel point being any one pixel point in a current coding block; determining a target quantization parameter (QP) value of the first pixel point based on the prediction residual value of the first pixel point; and inverse quantizing the first pixel point based on the target QP value of the first pixel point.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This patent application claims priority to Chinese patent application No. 202210886294.7, filed on July 26, 2022, and Chinese patent applications Nos. 202210976078.1, 202310454346.8, and 202310454020.5, filed on August 15, 2022. Of these, patent applications Nos. 202310454346.8 and 202310454020.5 are divisions of Chinese patent application No. 202210976078.1, which claims priority to Chinese patent application No. 202210886294.7. The entire contents of these related applications are incorporated herein by reference.

[0002] The present invention relates to the field of computer video processing technology, and more particularly to an image decoding method, an image encoding method and an apparatus. [Background technology]

[0003] In the field of video encoding and decoding, video compression (i.e., video encoding and decoding) techniques can be used to compress the amount of video data, thereby realizing efficient transmission or storage of the video.

[0004] Video encoding and decoding refers to encoding and decoding the image of each frame of video. Taking a frame of image as an example, at the encoding side, an image encoder encodes the image to obtain a bitstream corresponding to the image and transmits it to the decoding side. At the decoding side, an image decoder analyzes the bitstream to obtain the image. Currently, an image is divided into one or more coding units (CUs). An image encoder predicts each CU, determines a residual value between the predicted value of the CU and the true value of the CU, and sequentially transforms, quantizes, and encodes the residual value to obtain a bitstream. Accordingly, an image decoder predicts each CU, sequentially dequantizes and inversely transforms the decoding result of the bitstream corresponding to the CU to obtain a residual value corresponding to the CU, and calculates the sum of the predicted value and the residual value of the CU to obtain a reconstructed value of the CU.

[0005] In the process of image encoding and decoding, quantization can realize a many-to-one mapping of signal values, effectively reducing the signal value space and achieving better compression. As can be understood, the encoding side and decoding side perform quantization and dequantization processes based on a quantization parameter (QP). Currently, one QP is set for one CU, and the encoding side obtains the QP for each CU and quantizes the residual values and transform coefficients of the CU based on the QP. Correspondingly, the decoding side obtains the QP for the CU and dequantizes the quantized coefficients obtained by analyzing the bitstream based on the QP. However, if the same QP is used to quantize all pixel points in one CU, i.e., if the same degree of quantization is performed on all pixel points in the CU, quantization distortion (image distortion due to quantization) will be large in the image encoding and decoding process. Summary of the Invention

[0006] The present invention provides an image decoding method, an encoding method and an apparatus that can reduce decoding distortion of image frames while ensuring a constant compression rate, and can improve the authenticity and accuracy of image decoding.

[0007] To achieve the above object, the embodiment of the present invention adopts the following technical solution.

[0008] According to a first aspect, an embodiment of the present invention provides an image decoding method performed by a decoding side, the method including: determining a prediction residual value of a first pixel point based on a prediction scheme of the first pixel point, the prediction residual value being used to reflect a gradient of the first pixel point, the prediction scheme being used to indicate a position of a reconstructed pixel point referenced in predicting the pixel point, the first pixel point being any one pixel point in a current coding block; determining a target quantization parameter (QP) value of the first pixel point based on the prediction residual value of the first pixel point; and inverse quantizing the first pixel point based on the target QP value of the first pixel point.

[0009] According to the image decoding method provided in the embodiment of the present invention, a video decoder determines a prediction residual value (reflecting the gradient of a first pixel point) of each pixel point in a coding block according to a prediction scheme of each pixel point (the prediction scheme is used to indicate the position of a reconstructed pixel point referenced when predicting the pixel point), and determines a QP of a granularity of each pixel point for the pixel points in the coding block, thereby dequantizing each pixel point according to the QP value of each pixel point, i.e., performing dequantization on a pixel-by-pixel basis, thereby ensuring a certain compression rate while reducing the decoding distortion of the image frame and improving the authenticity and accuracy of image decoding.

[0010] Here, the gradient of a pixel point is used to indicate the difference between the pixel point and its surrounding pixel points, and can reflect the complexity of the pixel point (for example, texture features, etc.).

[0011] In one possible embodiment, the prediction residual value of the first pixel point comprises a target value or an average of target values, wherein the target value is the gradient of the reconstructed pixel points around the first pixel point, or the target value is the absolute value of the gradient of the reconstructed pixel points around the first pixel point, or the target value is the residual value after inverse quantization of the reconstructed pixel points around the first pixel point, or the target value is the absolute value of the residual value after inverse quantization of the reconstructed pixel points around the first pixel point.

[0012] In one possible embodiment, the prediction scheme comprises making a prediction based on reconstructed pixel points on both the left and right sides of the pixel point, and / or making a prediction based on reconstructed pixel points on both the top and bottom sides of the pixel point, and / or making a prediction based on a reconstructed pixel point on the left side of the pixel point, and / or making a prediction based on a reconstructed pixel point above the pixel point.

[0013] In another possible embodiment, the prediction method of the first pixel point is to perform prediction based on reconstructed pixel points on both the left and right sides of the pixel point, and the step of determining the prediction residual value of the first pixel point based on the prediction method of the first pixel point is to calculate a difference between pixel values of the second pixel point and a third pixel point as a first difference value, or to use a residual value after inverse quantization of the second pixel point as the first difference value, where the second pixel point is the first reconstructed pixel point on the left side of the first pixel point, and the third pixel point is the second pixel point. the fourth pixel point is the first reconstructed pixel point on the right side of the first pixel point, and the fifth pixel point is the first reconstructed pixel point on the upper side; calculating a difference between the pixel values of the fourth pixel point and the fifth pixel point as a second difference value, or using a residual value after inverse quantization of the fourth pixel point as the second difference value, where the fourth pixel point is the first reconstructed pixel point to the right of the first pixel point, and the fifth pixel point is the first reconstructed pixel point on the upper side of the fourth pixel point; and using an average value of the absolute value of the first difference value and the absolute value of the second difference value as the predicted residual value of the first pixel point.

[0014] In another possible embodiment, the prediction method for the first pixel point is to perform prediction based on reconstructed pixel points above and below the pixel point, and the step of determining the prediction residual value of the first pixel point based on the prediction method for the first pixel point includes the steps of: calculating a difference between pixel values of a sixth pixel point and a seventh pixel point as a third difference value, where the sixth pixel point is the first reconstructed pixel point above the first pixel point and the seventh pixel point is the first reconstructed pixel point to the left of the sixth pixel point; calculating a difference between pixel values of an eighth pixel point and a ninth pixel point as a fourth difference value, or setting the residual value after inverse quantization of the eighth pixel point as the fourth difference value, where the eighth pixel point is the first reconstructed pixel point below the first pixel point and the ninth pixel point is the first reconstructed pixel point to the left of the eighth pixel point; and setting the average value of the absolute value of the third difference value and the absolute value of the fourth difference value as the prediction residual value of the first pixel point.

[0015] In another possible embodiment, the prediction method for the first pixel point is to perform prediction based on a reconstructed pixel point to the left of the pixel point, and the step of determining the prediction residual value of the first pixel point based on the prediction method for the first pixel point includes the step of: setting the absolute value of a difference between pixel values of a tenth pixel point and an eleventh pixel point as the prediction residual value of the first pixel point, where the tenth pixel point is the first reconstructed pixel point to the left of the first pixel point and the eleventh pixel point is the first reconstructed pixel point to the left of the tenth pixel point; or setting the absolute value of a difference between pixel values of the tenth pixel point and a twelfth pixel point as the prediction residual value of the first pixel point, where the twelfth pixel point is the first reconstructed pixel point above the tenth pixel point; or setting the absolute value of the residual value after inverse quantization of the tenth pixel point as the prediction residual value of the first pixel point.

[0016] In another possible embodiment, the prediction method for the first pixel point is to perform prediction based on a reconstructed pixel point above the pixel point, and the step of determining the prediction residual value of the first pixel point based on the prediction method for the first pixel point includes the step of setting the absolute value of the difference between the pixel values of the 13th pixel point and the 14th pixel point as the prediction residual value of the first pixel point, wherein the 13th pixel point is the first reconstructed pixel point above the first pixel point and the 14th pixel point is the first reconstructed pixel point above the 13th pixel point, or the step of setting the absolute value of the residual value after inverse quantization of the 13th pixel point as the prediction residual value of the first pixel point.

[0017] In another possible embodiment, the method further includes a step of determining a predicted QP value of the first pixel point, where the predicted QP value of the first pixel point is a QP value of a current coding block, and the step of determining a target QP value of the first pixel point based on the predicted residual value of the first pixel point includes a step of adjusting the predicted QP value of the first pixel point based on the predicted residual value to obtain the target QP value of the first pixel point, if the first pixel point is a target pixel point in the current coding block, where the target pixel point is a pixel point that is preset for adjusting the QP value, or a step of setting the predicted QP value of the first pixel point as the target QP value, if the first pixel point is a pixel point other than the target pixel point in the current coding block.

[0018] In another possible embodiment, the step of adjusting the predicted QP value of the first pixel point based on the prediction residual value to obtain a target QP value of the first pixel point includes the step of adjusting the predicted QP value of the first pixel point to obtain the target QP value if the predicted QP value of the first pixel point is greater than or equal to a first threshold and less than or equal to a second threshold, and the prediction residual value of the first pixel point is less than or equal to a third threshold; otherwise, setting the predicted QP value of the first pixel point as the target QP value of the first pixel point, and the target QP value is smaller than the predicted QP value.

[0019] In another possible embodiment, the first threshold is a QP value that corresponds to just noticeable distortion, and the second threshold is an adjustable maximum QP value.

[0020] In another possible embodiment, adjusting the predicted QP value of the first pixel point to obtain a target QP value for the first pixel point includes setting the target QP value to the larger of a QP value obtained by subtracting a predetermined offset from the predicted QP value and a QP value that corresponds to just noticeable distortion.

[0021] In another possible embodiment, the QP value corresponding to the just noticeable distortion is determined based on the bit depth of the first pixel point in the target channel or the channel index of the first pixel point.

[0022] In another possible embodiment, the adjustable maximum QP value is determined based on one or more of the bit depth of the first pixel point in the target channel, the channel index of the first pixel point, and the fullness of the bitstream buffer.

[0023] In another possible embodiment, the third threshold is determined based on one or more of the bit depth of the first pixel point in the target channel, the channel index of the first pixel point, and the texture complexity of the current block.

[0024] In another possible embodiment, the preset offset is determined based on one or more of the bit depth of the first pixel point in the target channel, the channel index of the first pixel point, the fullness of the bitstream buffer, and the texture complexity of the current coding block.

[0025] In another possible embodiment, the current coding block includes at least pixel points of a first portion and / or pixel points of a second portion, the pixel points of the first portion are set as pixel points whose QP values do not need to be adjusted, and the pixel points of the second portion are set as pixel points whose QP values should be adjusted, the pixel points of the first portion and the pixel points of the second portion are determined based on a pixel-wise prediction mode of the current coding block, and the target pixel points are any one or more of the pixel points of the second portion.

[0026] In another possible embodiment, the pixel-by-pixel prediction mode of the current coding block includes mode 1 and mode 2, where mode 1 is a vertical prediction mode for processing a vertical-type texture and mode 2 is a horizontal prediction mode for processing a horizontal-type texture. If the pixel-by-pixel prediction mode of the current coding block is mode 1, the pixel points of the second portion include at least one of a first-type pixel point and a fourth-type pixel point that is not in the first row, where the first-type pixel point is a pixel point predicted based on reconstructed pixel points on both the left and right sides of the pixel point, and the fourth-type pixel point is a pixel point predicted based on a reconstructed pixel point above the pixel point. If the pixel-by-pixel prediction mode of the current coding block is mode 2, the pixel points of the second portion include at least one of a second-type pixel point that is not in the first column, a third-type pixel point that is not in the first column, and a fourth-type pixel point that is not in the first row, where the second-type pixel point is a pixel point predicted based on reconstructed pixel points above and below the pixel point, and the third-type pixel point is a pixel point predicted based on a reconstructed pixel point to the left of the pixel point.

[0027] According to a second aspect, an embodiment of the present invention provides an image encoding method performed by an encoding side, the method including: determining a prediction residual value of a first pixel point based on a prediction scheme of the first pixel point, the prediction residual value being used to reflect a gradient of the first pixel point, the prediction scheme being used to indicate a position of a reconstructed pixel point referenced in predicting the pixel point, the first pixel point being any one pixel point in a current encoding block; determining a target quantization parameter (QP) value of the first pixel point based on the prediction residual value of the first pixel point; and quantizing the first pixel point based on the target QP value of the first pixel point.

[0028] According to the image encoding method provided in the embodiment of the present invention, a video encoder determines a prediction residual value (reflecting the gradient of a first pixel point) for each pixel point in a coding block based on a prediction scheme for each pixel point (the prediction scheme is used to indicate the position of a reconstructed pixel point referenced when predicting the pixel point), and determines a QP of a granularity for each pixel point for the pixel points in the coding block, thereby quantizing each pixel point based on the QP value of each pixel point, i.e., performing quantization on a pixel-by-pixel basis, thereby ensuring a certain compression rate while reducing the decoding distortion of the image frame and improving the authenticity and accuracy of image decoding.

[0029] As will be understood, each step in the image encoding method is an inverse operation in the image decoding method, and therefore, various possible implementation methods in the image encoding method can be inversely operated by referring to the description of various possible implementation methods in the image decoding method, and the description will be omitted here.

[0030] According to a third aspect, an embodiment of the present invention provides an image decoding device arranged at a decoding side, the decoding device comprising respective modules for implementing the method according to the first aspect and one of its possible realization manners, such as a decision unit and an inverse quantization unit.

[0031] For technical solutions and beneficial effects of the third aspect, please refer to the description of the first aspect and any one of its possible implementations. The image decoding device has a function for implementing operations in the example method of the first aspect and any one of the possible implementations of the first aspect. The function may be implemented by hardware, or may be implemented by the hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above function.

[0032] According to a fourth aspect, an embodiment of the present invention provides an image coding device arranged at the coding side, the image coding device comprising respective modules for implementing the method according to the second aspect and one of its possible realizations, such as a decision unit and a quantization unit.

[0033] For technical solutions and beneficial effects of the fourth aspect, please refer to the description of the second aspect and any one of its possible implementations. The image encoding device has functions for implementing operations in the example method of the second aspect and any one of the possible implementations of the second aspect. The functions may be implemented by hardware, or may be implemented by the hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the functions.

[0034] According to a fifth aspect, an embodiment of the present invention provides an electronic device including a processor and a memory, wherein the memory is configured to store computer instructions, and the processor is configured to retrieve and execute the computer instructions from the memory to perform a method according to any one of the first aspect, the second aspect and possible implementations thereof.

[0035] For example, the electronic device may be a video encoder or an encoding side including a video encoder.

[0036] As another example, the electronic device may be a video decoder or a decoding side that includes a video decoder.

[0037] According to a sixth aspect, embodiments of the present invention provide a computer-readable storage medium having stored thereon a computer program or instructions, which, when executed by a computing device or a storage system in which the computing device is arranged, performs a method according to the first aspect, the second aspect or any one of its possible implementations.

[0038] According to a seventh aspect, embodiments of the present invention provide a computer program product comprising instructions which, when executed on a computing device or processor, cause the computing device or processor to execute the instructions to perform the method according to the first aspect, the second aspect and any one of their possible implementations.

[0039] According to an eighth aspect, an embodiment of the present invention provides an apparatus comprising a memory and a processor, wherein the memory is configured to store computer instructions, and the processor is configured to retrieve and execute the computer instructions from the memory to perform a method according to any one of the first aspect, the second aspect and possible implementations thereof.

[0040] According to a ninth aspect, an embodiment of the present invention provides a video coding system comprising an encoding side and a decoding side, wherein the decoding side is configured to implement the method according to the first aspect and any one of its possible implementations, and the encoding side is configured to implement the method according to the second aspect and any one of its possible implementations.

[0041] The embodiments of the present invention may be further combined to provide more implementation methods based on the implementation methods provided in each of the above aspects. [Brief explanation of the drawings]

[0042] [Figure 1] 1 is an exemplary block diagram of a video coding system according to one embodiment of the present invention; [Figure 2] 1 is an illustrative block diagram of a video encoder according to an embodiment of the present invention; [Figure 3] 2 is an illustrative block diagram of a video decoder according to one embodiment of the present invention; [Figure 4] 1 is a flowchart of video encoding / decoding according to an embodiment of the present invention. [Figure 5]1 is a first flowchart of an image decoding method according to one embodiment of the present invention. [Figure 6] 10 is a second flowchart of an image decoding method according to one embodiment of the present invention. [Figure 7] FIG. 2 is a schematic diagram of a distribution of pixel points according to an embodiment of the present invention. [Figure 8] FIG. 1 is a first schematic diagram of pixel point division in a pixel-by-pixel prediction mode according to an embodiment of the present invention. [Figure 9] FIG. 2 is a second schematic diagram of pixel point division in pixel-based prediction mode according to an embodiment of the present invention. [Figure 10] FIG. 3 is a third schematic diagram of pixel point division in pixel-based prediction mode according to an embodiment of the present invention. [Figure 11] FIG. 4 is a fourth schematic diagram of pixel point division in pixel-based prediction mode according to an embodiment of the present invention. [Figure 12] FIG. 5 is a fifth schematic diagram of pixel point division in pixel-based prediction mode according to one embodiment of the present invention. [Figure 13] 10 is a third flowchart of an image decoding method according to one embodiment of the present invention. [Figure 14] 4 is a fourth flowchart of the image decoding method according to one embodiment of the present invention. [Figure 15] FIG. 6 is a sixth schematic diagram of pixel point division in pixel-based prediction mode according to one embodiment of the present invention. [Figure 16] 1 is a flowchart of an image encoding method according to an embodiment of the present invention. [Figure 17] 1 is a schematic structural diagram of an image decoding device according to the present invention; [Figure 18] 1 is a schematic structural diagram of an image encoding device according to the present invention; [Figure 19] 1 is a schematic structural diagram of an electronic device according to the present invention; DETAILED DESCRIPTION OF THE INVENTION

[0043] The term "and / or" in this specification is merely a description of the relationship between related objects and indicates that three types of relationships may exist. For example, A and / or B can indicate three cases: A exists alone, A and B exist together, and B exists alone.

[0044] Terms such as "first" and "second" used in the specification and drawings of the present invention are intended to distinguish between different objects or between different processes for the same object, and are not intended to describe a specific order of the objects. For example, terms such as "first preset value" and "second preset value" are intended to distinguish between different preset values, and are not intended to describe a specific order of the preset values.

[0045] Additionally, the terms "comprises" and "having" and any variations thereof referred to in the description of the present invention are intended to cover a non-exclusive inclusion. For example, a process, method, system, product, or device comprising a series of steps or units is not limited to the listed steps or units, but may further include other steps or units not listed, or may further include other steps or units inherent to those processes, methods, products, or devices.

[0046] It should be noted that in the embodiments of the present invention, terms such as "exemplary" or "for example" are used to represent an example, illustration, or explanation. Any embodiment or design solution described as "exemplary" or "for example" in the embodiments of the present invention should not be construed as preferred or advantageous over other embodiments or design solutions. Specifically, the use of terms such as "exemplary" or "for example" is intended to specifically present the related concept.

[0047] In describing the present invention, unless otherwise specified, "plurality" refers to two or more.

[0048] The image decoding and encoding methods provided in the embodiments of the present invention can also be applied to video decoding and video encoding. It should be understood that a video includes a series of pictures, and decoding and encoding a video essentially means decoding and encoding all the pictures included in the video.

[0049] As can be understood, quantization in the image coding process is a process of mapping continuous signal values (or a large number of discrete values) to a finite number of discrete values, and quantization can achieve a many-to-one mapping of signal values. In video coding, after a residual signal is transformed, the transform coefficients generally have a large dynamic range. Therefore, quantizing the transform coefficients can effectively reduce the signal value space and achieve better compression. However, due to the many-to-one mapping mechanism, distortion is inevitably introduced during the quantization process, which is the fundamental cause of distortion in video coding.

[0050] Inverse quantization is the reverse process of quantization: it maps the quantized coefficients to a reconstructed signal in the input signal space, where the reconstructed signal is an approximation of the input signal.

[0051] Quantization includes scalar quantization (SQ) and vector quantization. Scalar quantization is the most basic quantization method, and the input of scalar quantization is a one-dimensional scalar signal. The scalar quantization process involves first dividing the input signal space into a series of non-intersecting intervals, selecting one representative signal from each interval, and then scalar quantizing each input signal into the representative signal of the interval in which the input signal is located. Here, the interval length is called the quantization step (Qstep), the interval index is the level value (Level), i.e., the value after quantization, and the parameter representing the quantization step is the quantization parameter (QP).

[0052] The simplest scalar quantization method is uniform scalar quantization, which divides the input signal space into equally spaced intervals, and the representative signal of each interval is the midpoint of the interval.

[0053] The optimal scalar quantizer is the Lloyd-Max quantizer, which takes into account the distribution of the input signal, and the interval division is uneven. The representative signal of each interval is the probability center of gravity of that interval, and the boundary point between two adjacent intervals is the midpoint of the representative signals of these two intervals.

[0054] The following describes a system architecture applied in an embodiment of the present invention, and Figure 1 is an exemplary block diagram of a video coding system according to the present invention. In this specification, the term "video coder" generally refers to both a video encoder and a video decoder. In the present invention, the terms "video coding / decoding" or "coding / decoding" generally refer to video encoding or video decoding. A video encoder 100 and a video decoder 200 in the video coding system 1 are configured to predict motion information, such as a motion vector of a currently coded / decoded image block or its sub-block, according to various exemplary methods described in any one of multiple new inter-prediction modes provided in the present invention. This allows the predicted motion vector to be as close as possible to a motion vector obtained using a motion estimation method. This eliminates the need to transmit motion vector difference values during coding, thereby further improving coding and decoding performance.

[0055] As shown in FIG. 1, video coding system 1 includes an encoding side 10 and a decoding side 20. Encoding side 10 generates coded video data. Accordingly, encoding side 10 may be referred to as a video encoding device. Decoding side 20 may decode the coded video data generated by encoding side 10. Accordingly, decoding side 20 may be referred to as a video decoding device. Various embodiments of encoding side 10, decoding side 20, or both may include one or more processors and memory coupled to the one or more processors. The memory, as described herein, may include, but is not limited to, RAM, ROM, EEPROM, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures accessible by a computer.

[0056] The encoding side 10 and the decoding side 20 may comprise a variety of devices, including desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, telephone handsets such as so-called "smart" phones, televisions, cameras, display devices, digital media players, video game consoles, in-vehicle computers, or the like.

[0057] The decoding side 20 can receive the coded video data from the coding side 10 via a link 30. The link 30 can include one or more media or devices that can transmit the coded video data from the coding side 10 to the decoding side 20. In one example, the link 30 can include one or more communication media that enable the coding side 10 to transmit the coded video data directly to the decoding side 20 in real time. In this example, the coding side 10 can modulate the coded video data according to a communication standard (e.g., a wireless communication protocol) and transmit the modulated video data to the decoding side 20. The one or more communication media can include wireless and / or wired communication media, such as a radio frequency (RF) spectrum or one or more physical transmission paths. The one or more communication media can form part of a packet-based network, such as a local area network (LAN), a wide area network (WAN), or a global network (e.g., the Internet). The one or more communication media can include routers, switches, base stations, or other devices that facilitate communication from the coding side 10 to the decoding side 20.

[0058] In another example, the encoded data may be output from output interface 140 to storage device 40. Similarly, the encoded data may be accessed from storage device 40 via input interface 240. Storage device 40 may include any one of a data storage medium for distributed or local access, such as a hard disk drive, a Blu-ray disc, a digital versatile disc (DVD), a compact disc read-only memory (CD-ROM), flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data.

[0059] In another example, the storage device 40 corresponds to a file server or another intermediate storage device that may hold the encoded video generated by the encoding side 10. The decoding side 20 can access the stored video data from the storage device 40 by streaming or downloading. The file server may be any type of server that can store and transmit encoded video data to the decoding side 20. Exemplary file servers include a network server (e.g., for a website), a file transfer protocol (FTP) server, a network attached storage (NAS) device, or a local disk drive. The decoding side 20 can access the encoded video data via any standard data connection (including an Internet connection). This may include a wireless channel (e.g., a wireless fidelity (Wi-Fi) connection), a wired connection (e.g., a digital subscriber line (DSL), a cable modem, etc.), or a combination thereof suitable for accessing encoded video data stored on a file server. The transmission of the encoded video data from the storage device 40 may be a streaming transmission, a download transmission, or a combination thereof.

[0060] The image decoding method provided by the present invention may be applied to video encoding and decoding to support various multimedia applications, such as over-the-air television broadcasting, cable television transmission, satellite television transmission, streaming video transmission (e.g., via the Internet), encoding video data stored on a data storage medium, decoding video data stored on a data storage medium, or other applications. In some embodiments, the video coding system 1 may be configured to support unidirectional or bidirectional video transmission to support applications such as video streaming, video playback, video broadcasting, and / or video telephony.

[0061] The video coding system 1 shown in FIG. 1 is merely exemplary, and the techniques of the present invention are applicable to video encoding / decoding settings (e.g., video encoding and video decoding) that do not necessarily involve data communication between an encoding device and a decoding device. In other examples, data is retrieved from local memory and streamed over a network. A video encoding device may encode data and store it in memory, and / or a video decoding device may read data from memory and decode it. In many examples, encoding and decoding are performed by devices that do not communicate with each other but encode data to memory and / or retrieve data from memory and decode it.

[0062] 1, encoding side 10 includes video source 120, video encoder 100, and output interface 140. In some examples, output interface 140 may include a modulator / demodulator (modem) and / or a transmitter. Video source 120 may include a video capture device (e.g., a camera), a video archive containing previously captured video data, a video feed-in interface for receiving video data from a video content provider, and / or a computer graphics system for generating video data, or a combination of these sources of video data.

[0063] Video encoder 100 can encode video data from video source 120. In some embodiments, encoding side 10 transmits the encoded video data directly to decoding side 20 via output interface 140. In other examples, the encoded video data may be stored in storage device 40 for access by decoding side 20 for decoding and / or playback.

[0064] 1, the decoding side 20 includes an input interface 240, a video decoder 200, and a display device 220. In some examples, the input interface 240 includes a receiver and / or a modem. The input interface 240 can receive encoded video data via the link 30 and / or from the storage device 40. The display device 220 may be integrated with the decoding side 20 or may be external to the decoding side 20. Generally, the display device 220 displays decoded video data. The display device 220 may include various display devices, such as a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or other types of display devices.

[0065] 1, in some aspects, video encoder 100 and video decoder 200 may be integrated with an audio encoder and decoder, respectively, and may include appropriate multiplexer-demultiplexer units or other hardware and software to handle the encoding of both audio and video in a common data stream or separate data streams. In some examples, where applicable, the demultiplexer (MUX-DEMUX) units may conform to the ITU H.223 multiplexer protocol or other protocols, such as the User Datagram Protocol (UDP).

[0066] Each of the video encoder 100 and the video decoder 200 may be implemented as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, hardware, or any combination thereof. If the present invention is implemented in part in software, a device may store instructions for the software on a suitable non-volatile computer-readable storage medium and execute the instructions in hardware using one or more processors to implement the techniques of the present invention. Any of the above (including hardware, software, a combination of hardware and software, etc.) may be considered to be one or more processors. The video encoder 100 and the video decoder 200 may be included in one or more encoders or decoders, or any of the encoders or decoders may be integrated as part of a combined encoder / decoder within the corresponding device.

[0067] In the present invention, video encoder 100 may generally be referred to as "signaling" or "sending" information to, for example, video decoder 200. The terms "signaling" or "sending" may generally refer to the transmission of syntax elements and / or other data for decoding compressed video data. This transmission may occur in real time or near real time. Alternatively, this communication may occur after a period of time has elapsed, such as when storing syntax elements in an encoded bitstream to a computer-readable storage medium during encoding, and a decoding device may subsequently retrieve the syntax elements any time after they are stored to the medium.

[0068] The JCT-VC developed the H.265 (HEVC) standard. The HEVC standardization is based on a video decoder evolution model called the HEVC Test Model (HEVC model, HM). The latest version of the H.265 standard document can be obtained from http: / / www.itu.int / rec / T-REC-H.265. The latest version of the standard document is H.265(12 / 16), which is incorporated herein by reference. The HM assumes that a video decoder has some additional capabilities compared to the existing ITU-TH.264 / AVC algorithm. For example, H.264 provides nine intra-prediction coding modes, while the HM can provide up to 35 intra-prediction coding modes.

[0069] JVET is working on the development of the H.266 standard. The H.266 standardization process is based on an evolutionary model of a video decoder called the H.266 Test Model. The algorithm description for H.266 can be obtained from http: / / phenix.int-evry.fr / jvet, and the latest algorithm description is included in JVET-F1001-v2, which is incorporated herein by reference. Reference software for the JEM Test Model can also be obtained from https: / / jvet.hhi.fraunhofer.de / svn / svn_HMJEMSoftware / , which is also incorporated herein by reference.

[0070] In general, the HM operational model describes that a video frame or image can be divided into a sequence of treeblocks or largest coding units (LCUs), each containing both luma and chroma samples. LCUs are also called coding tree units (CTUs). Treeblocks have a similar purpose to macroblocks in the H.264 standard. A slice contains multiple treeblocks in succession in decoding order. A video frame or image can be divided into one or more slices. Each treeblock can be divided into coding units (CUs) based on a quadtree. For example, a treeblock, which is the root node of the quadtree, can be divided into four child nodes, and each child node can be divided into four other child nodes as a parent node. The ultimately indivisible child nodes, which are leaf nodes of the quadtree, contain decoding nodes, such as decoded video blocks. Syntax data associated with the decoded bitstream may define the maximum number of times a treeblock can be divided and the minimum size of a decoding node.

[0071] The size of a CU corresponds to the size of a decoding node and must be square in shape. The size of a CU can range from 8x8 pixels to 64x64 pixels, or even the size of a treeblock.

[0072] A video sequence typically includes a series of video frames or images. A group of pictures (GOP), for example, includes a series of one or more video images. Syntax data describing the number of images included in a GOP may be included in the header information of the GOP, in the header information of one or more images, or elsewhere. Each slice of an image may include slice syntax data describing the coding mode of the corresponding image. Video encoder 100 typically operates on video blocks within individual video slices to encode video data. Video blocks may correspond to decoding nodes within a CU. Video blocks may have fixed or variable sizes and may vary in size according to specified decoding criteria.

[0073] In the present invention, "NxN" and "N by N" may be used interchangeably to refer to the pixel size of a video block along the vertical and horizontal dimensions, such as 16x16 pixels or 16 by 16 pixels. Generally, a 16x16 block has 16 pixel points vertically (y=16) and 16 pixel points horizontally (x=16). Similarly, an NxN block typically has N pixel points vertically and N pixel points horizontally, where N represents a non-negative integer value. Pixels within a block may be arranged in rows and columns. Also, a block does not necessarily have to have the same number of pixel points horizontally and vertically. For example, a block may include NxM pixel points, where M does not necessarily equal N.

[0074] After intra / inter predictive decoding of a CU is used, video encoder 100 may calculate residual data of the CU. The CU may include pixel data in the spatial domain (also called the pixel domain) or may include coefficients in the transform domain after applying a transform (e.g., a discrete cosine transform (DCT), an integer transform, a discrete wavelet transform, or a conceptually similar transform) to the residual video data. The residual data may correspond to pixel differences between pixels of the uncoded image and predicted values corresponding to the CU. Video encoder 100 may form a CU including the residual data and generate transform coefficients for the CU.

[0075] After generating transform coefficients through any transform, video encoder 100 may perform quantization of the transform coefficients. Quantization is a process that potentially reduces the amount of data representing the coefficients, thereby providing further compression, for example, by quantizing the coefficients. The quantization process may reduce the bit depth for some or all of the coefficients. For example, during quantization, n-bit values may be truncated to m-bit values, where n is greater than m.

[0076] In some possible implementations, video encoder 100 may scan the quantized transform coefficients in a predefined scan order to generate a serialized vector that can be entropy coded. In other possible implementations, video encoder 100 may perform adaptive scanning. After scanning the quantized transform coefficients to form a one-dimensional vector, video encoder 100 may entropy code and decode the one-dimensional vector according to context-based adaptive variable-length coding (CAVLC), context-based adaptive binary arithmetic coding (CABAC), syntax-based adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or other entropy coding and decoding methods. Video encoder 100 may also entropy code syntax elements associated with the coded video data so that video decoder 200 can decode the video data.

[0077] To perform CABAC, video encoder 100 may assign a context within a context model to a transmitted symbol. The context may relate to whether neighboring values of the symbol are non-zero. To perform CAVLC, video encoder 100 may select a variable-length code for the transmitted symbol. Codewords in variable-length coding (VLC) may be configured such that relatively short codes correspond to more likely symbols and relatively long codes correspond to less likely symbols. In this way, the use of VLC can achieve the goal of saving bitrate, as opposed to using codewords of the same length for each transmitted symbol. Probabilities in CABAC may be determined based on the context assigned to the symbol.

[0078] In an embodiment of the present invention, a video encoder may perform inter-prediction to reduce temporal redundancy between images. In the present invention, a CU currently being decoded by a video decoder may be referred to as a current CU. In the present invention, a picture currently being decoded by a video decoder may be referred to as a current picture.

[0079] 2 is an exemplary block diagram of a video encoder according to the present invention. Video encoder 100 is configured to output video to post-processing entity 41. Post-processing entity 41 represents an example of a video entity that can process encoded video data from video encoder 100, such as a media aware network element (MANE) or a splicing / editing device. In some cases, post-processing entity 41 may be an example of a network entity. In some video encoding systems, post-processing entity 41 and video encoder 100 may be parts of separate devices, and in other embodiments, the functionality described with respect to post-processing entity 41 may be performed by the same device that includes video encoder 100. In one example, post-processing entity 41 is an example of storage device 40 of FIG. 1.

[0080] In the example of FIG. 2, video encoder 100 includes prediction processing unit 108, filter unit 106, decoded picture buffer (DPB) 107, summer 112, transformer 101, quantizer 102, and entropy coder 103. Prediction processing unit 108 includes inter predictor 110 and intra predictor 109. For image block reconstruction, video encoder 100 further includes inverse quantizer 104, inverse transformer 105, and summer 111. Filter unit 106 represents one or more loop filters, such as a deblocking filter, an adaptive loop filter (ALF), and a sample adaptive offset (SAO) filter. While FIG. 2 illustrates filter unit 106 as an in-loop filter, in other embodiments, filter unit 106 may be implemented as a post-loop filter. In one example, video encoder 100 may further include a video data memory and a partitioning unit (not shown).

[0081] The video data memory can store video data to be encoded by the components of the video encoder 100. The video data stored in the video data memory may be obtained from the video source 120. The DPB 107 may be a reference picture memory that stores reference video data for the video encoder 100 to encode video data in intra- and inter-encoding / decoding modes. The video data memory and the DPB 107 may be formed by any of a variety of memory devices, such as dynamic random access memory (DRAM), including synchronous dynamic random access memory (SDRAM), magnetic random access memory (MRAM), resistive random access memory (RRAM), or another type of memory device. The video data memory and the DPB 107 may be provided by the same memory device or by separate memory devices. In various embodiments, the video data memory may be on-device with other components of the video encoder 100 or off-device relative to those components.

[0082] As shown in FIG. 2, video encoder 100 receives video data and stores the video data in a video data memory. A division unit divides the video data into several image blocks, which may be further divided into smaller blocks, such as image block division based on a quadtree structure or a binary tree structure. This division may include dividing into slices, tiles, or other larger units. Video encoder 100 generally represents a component for encoding image blocks within a video slice to be encoded. A slice may be divided into multiple image blocks (or into sets of image blocks called tiles). Prediction processing unit 108 may select one of multiple possible codec modes to be used for the current image block, for example, one of multiple intra-codec modes or one of multiple inter-codec modes. Prediction processing unit 108 may supply the intra- and inter-codec blocks to adder 112 to generate a residual block, and to adder 111 to reconstruct a coding block to be used as a reference image.

[0083] An intra predictor 109 in prediction processing unit 108 may perform intra predictive coding of the current image block relative to one or more neighboring blocks in the same frame or slice as the current block to be coded to remove spatial redundancy. An inter predictor 110 in prediction processing unit 108 may perform inter predictive coding of the current image block relative to one or more predictive blocks in one or more reference images to remove temporal redundancy.

[0084] Specifically, the inter predictor 110 may be configured to determine an inter prediction mode for encoding a current image block. For example, the inter predictor 110 may use a bit rate-distortion analysis to calculate bit rate-distortion values for various inter prediction modes in a set of candidate inter prediction modes and select an inter prediction mode with optimal bit rate-distortion characteristics from among them. The bit rate-distortion analysis typically determines the amount of distortion (or error) between a coding block and its original uncoded block, and the bit rate (i.e., the number of bits) for generating the coding block. For example, the inter predictor 110 may determine an inter prediction mode in the set of candidate inter prediction modes that encodes the current image block with the lowest bit rate-distortion cost as the inter prediction mode for performing inter prediction on the current image block.

[0085] The inter predictor 110 is configured to predict motion information (e.g., motion vectors) of one or more sub-blocks in the current image block based on the determined inter prediction mode, and obtain or generate a predictive block for the current image block using the motion information (e.g., motion vectors) of one or more sub-blocks in the current image block. The inter predictor 110 may locate the predictive block pointed to by the motion vector in one of the reference image lists. The inter predictor 110 may also generate syntax elements related to the image block and the video slice for use when the video decoder 200 decodes the image block of the video slice. Alternatively, in one example, the inter predictor 110 performs a motion compensation process using the motion information of each sub-block to generate a predictive block for each sub-block and obtain a predictive block for the current image block. It should be understood that the inter predictor 110 here performs motion estimation and motion compensation processes.

[0086] Specifically, after selecting an inter prediction mode for the current image block, the inter predictor 110 may provide information indicating the selected inter prediction mode of the current image block to the entropy encoder 103, so that the entropy encoder 103 encodes the information indicating the selected inter prediction mode.

[0087] The intra predictor 109 may perform intra prediction on the current image block. Specifically, the intra predictor 109 may determine an intra prediction mode for encoding the current block. For example, the intra predictor 109 may use a bit rate-distortion analysis to calculate bit rate-distortion values for various test intra prediction modes and select an intra prediction mode with optimal bit rate-distortion characteristics from among the test modes. In either case, after selecting an intra prediction mode for the image block, the intra predictor 109 may provide information indicating the selected intra prediction mode of the current image block to the entropy encoder 103, such that the entropy encoder 103 encodes the information indicating the selected intra prediction mode.

[0088] After prediction processing unit 108 generates a prediction block for a current image block by inter-prediction or intra-prediction, video encoder 100 subtracts the prediction block from the current image block to be encoded to form a residual image block. Adder 112 represents one or more components that perform this subtraction operation. The residual video data in the residual block may be included in one or more transform units (TUs) and applied to transformer 101. Transformer 101 converts the residual video data into residual transform coefficients, such as using a discrete cosine transform (DCT) or a conceptually similar transform. Transformer 101 may convert the residual video data from the pixel value domain to a transform domain, such as the frequency domain.

[0089] The transformer 101 may send the resulting transform coefficients to the quantizer 102, which quantizes the transform coefficients to further reduce the bit rate. In some examples, the quantizer 102 may then perform a scan on a matrix containing the quantized transform coefficients. Alternatively, the entropy coder 103 may perform the scan.

[0090] After quantization, entropy encoder 103 entropy codes the quantized transform coefficients. For example, entropy encoder 103 may perform context-adaptive variable-length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), syntax-based adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or another entropy coding method or technique. After being entropy coded by entropy encoder 103, the coded bitstream may be transmitted to video decoder 200 or archived for later transmission or retrieval by video decoder 200. Entropy encoder 103 may also entropy code syntax elements of the current image block to be coded.

[0091] The inverse quantizer 104 and the inverse transformer 105 apply inverse quantization and inverse transformation, respectively, to reconstruct the residual block in the pixel domain, e.g., for subsequent use as a reference block in a reference image. The adder 111 adds the reconstructed residual block to the prediction block generated by the inter predictor 110 or the intra predictor 109 to generate a reconstructed image block. A filter unit 106 may be applied to the reconstructed image block to reduce distortions, such as block artifacts. The reconstructed image block is then stored in the decoded picture buffer 107 as a reference block and may be used by the inter predictor 110 as a reference block for inter prediction of blocks in subsequent video frames or images.

[0092] It should be understood that other structural variations of the video encoder 100 may be used to encode the video bitstream. For example, for some image blocks or image frames, the video encoder 100 may directly quantize the residual signal without needing to be processed by the transformer 101 and, accordingly, by the inverse transformer 105. Alternatively, for some image blocks or image frames, the video encoder 100 may not generate residual data and, accordingly, may not need to be processed by the transformer 101, the quantizer 102, the inverse quantizer 104, and the inverse transformer 105. Alternatively, the video encoder 100 may directly store the reconstructed image block as a reference block without needing to be processed by the filter unit 106. Alternatively, the quantizer 102 and the inverse quantizer 104 in the video encoder 100 may be integrated together.

[0093] 3 is an exemplary block diagram of a video decoder 200 consistent with the present invention. In the example of FIG. 3, the video decoder 200 includes an entropy decoder 203, a prediction processing unit 208, an inverse quantizer 204, an inverse transformer 205, an adder 211, a filter unit 206, and a DPB 207. The prediction processing unit 208 may include an inter predictor 210 and an intra predictor 209. In some examples, the video decoder 200 may perform a decoding process that is substantially the inverse of the encoding process described for the video encoder 100 of FIG. 2.

[0094] During the decoding process, video decoder 200 receives from video encoder 100 an encoded video bitstream representing image blocks and associated syntax elements of an encoded video slice. Video decoder 200 may receive video data from network entity 42 and, optionally, store the video data in a video data memory (not shown). The video data memory may store video data to be decoded by components of video decoder 200, such as an encoded video bitstream. The video data stored in the video data memory may be obtained, for example, from storage device 40, from a local video source such as a camera, via wired or wireless network communication of video data, or by accessing a physical data storage medium. The video data memory may function as a decoded picture buffer (DPB) for storing encoded video data from the encoded video bitstream. Thus, although the video data memory is not shown in FIG. 3, the video data memory and DPB 207 may be the same memory or may be separate memories. Video data memory and DPB 207 may be formed by any of a variety of memory devices, such as DRAM, including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. In various embodiments, the video data memory may be integrated on-device with other components of video decoder 200 or may be external to those components.

[0095] Network entity 42 may be a server, a MANE, a video editor / splicing device, or other device for implementing one or more of the techniques described above. Network entity 42 may or may not include a video encoder, such as video encoder 100. Network entity 42 may implement some of the techniques described in this invention before transmitting the encoded video bitstream to video decoder 200. In some video decoding systems, network entity 42 and video decoder 200 may be parts of separate devices, while in other cases, the functionality described with respect to network entity 42 may be performed by the same device that includes video decoder 200. In some cases, network entity 42 may be an example of storage device 40 of FIG. 1.

[0096] The entropy decoder 203 of the video decoder 200 entropy decodes the bitstream to generate quantized coefficients and some syntax elements. The entropy decoder 203 forwards the syntax elements to the prediction processing unit 208. The video decoder 200 can receive syntax elements at the video slice level and / or the image block level.

[0097] If the video slice is decoded as an intra-decoded (I) slice, intra predictor 209 of prediction processing unit 208 may generate a prediction block for the image block of the current video slice based on the intra-prediction mode signaled by the signaling and data from previously decoded blocks of the current frame or image. If the video slice is decoded as an inter-decoded (i.e., B or P) slice, inter predictor 210 of prediction processing unit 208 may determine an inter-prediction mode for decoding the current image block of the current video slice based on the syntax element received from entropy decoder 203 and decode (e.g., perform inter prediction) the current image block based on the determined inter-prediction mode. Specifically, inter predictor 210 may determine whether to predict the current image block of the current video slice using a new inter-prediction mode, and if the syntax element indicates that the current image block should be predicted using the new inter-prediction mode, predict motion information of the current image block or a sub-block of the current image block of the current video slice based on the new inter-prediction mode (e.g., the new inter-prediction mode specified by the syntax element or the default new inter-prediction mode). Thus, a prediction block for the current image block or a sub-block of the current image block may be obtained or generated using motion information of the predicted current image block or a sub-block of the current image block through a motion compensation process. Here, the motion information may include reference image information and a motion vector, and the reference image information may include, but is not limited to, unidirectional / bidirectional prediction information, a reference image list number, and a reference image index corresponding to the reference image list. In the case of inter prediction, the prediction block may be generated from one reference image in the reference image list. The video decoder 200 may construct reference image lists, i.e., List 0 and List 1, based on the reference images stored in the DPB 207. The reference frame index of the current image may be included in one or more of Reference Frame List 0 and List 1.In some examples, video encoder 100 may transmit a signal indicating whether to use a new inter-prediction mode to decode a particular syntax element of a particular block, or may transmit a signal indicating whether and which new inter-prediction mode to use to decode a particular syntax element of a particular block. Note that inter predictor 210 here performs motion compensation processing.

[0098] The inverse quantizer 204 inverse quantizes the quantized transform coefficients provided to the bitstream and decoded by the entropy decoder 203. The inverse quantization process may include using quantization parameters calculated by the video encoder 100 for each image block in a video slice to determine the degree of quantization to apply and determining the degree of inverse quantization to apply. The inverse transformer 205 applies an inverse transform to the transform coefficients, such as an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process, to produce residual blocks in the pixel domain.

[0099] After the inter predictor 210 generates a prediction block for the current image block or a sub-block of the current image block, the video decoder 200 obtains a reconstructed block, i.e., a decoded image block, by adding the residual block from the inverse transformer 205 and the corresponding prediction block generated by the inter predictor 210. The adder 211 represents a component that performs this addition operation. If necessary, a loop filter (in the decoding loop or after the decoding loop) may be used to smooth pixel transitions or otherwise improve video quality. The filter unit 206 may represent one or more loop filters, such as a deblocking filter, an adaptive loop filter (ALF), and a sample adaptive offset (SAO) filter. While FIG. 3 illustrates the filter unit 206 as an in-loop filter, in other embodiments, the filter unit 206 may be implemented as a post-loop filter. In one example, the filter unit 206 is configured to reconstruct blocks to reduce blockiness and output the result as a decoded video bitstream. Decoded image blocks in a given frame or picture may also be stored in a decoded picture buffer 207, and reference pictures for subsequent motion compensation may be stored via DPB 207. DPB 207 may be part of a memory that stores decoded video for subsequent display on a display device (e.g., display device 220 of FIG. 1), or may be separate from such memory.

[0100] It should be understood that other structural variations of the video decoder 200 may be used to decode the encoded video bitstream. For example, the video decoder 200 may generate an output video bitstream without being processed by the filter unit 206, or for some image blocks or frames, the entropy decoder 203 of the video decoder 200 does not decode quantized coefficients, which accordingly do not need to be processed by the inverse quantizer 204 and the inverse transformer 205.

[0101] The techniques of the present invention may be performed by any of the video encoders or video decoders described herein, such as the video encoder 100 and video decoder 200 shown and described in Figures 1-3. That is, in one possible embodiment, the video encoder 100 illustrated in Figure 2 may perform certain techniques described below when performing inter prediction during encoding of blocks of video data. In another possible embodiment, the video decoder 200 illustrated in Figure 3 may perform certain techniques described below when performing inter prediction during decoding of blocks of video data. Accordingly, references to a generic "video encoder" or "video decoder" may include the video encoder 100, the video decoder 200, or another video encoding or decoding unit.

[0102] 1-3 are merely examples according to embodiments of the present invention, and in some examples, video encoder 100, video decoder 200, and video coding system may include more or fewer components or units without limiting the present invention.

[0103] In the following, based on the video coding systems shown in Figures 1 to 3, an embodiment of the present invention provides a possible video encoding / decoding embodiment. Figure 4 is a flowchart of a video encoding / decoding process according to the present invention. As shown in Figure 4, the video encoding / decoding embodiment includes processes (1) to (5), which can be performed by any one or more of the encoding side 10, the video encoder 100, the decoding side 20, or the video decoder 200.

[0104] Process (1): Divide one frame of an image into one or more parallel coding units that do not overlap each other. There is no dependency between the one or more parallel coding units, and they can be coded and decoded completely in parallel / independently, as shown in Figure 4, for parallel coding unit 1 and parallel coding unit 2.

[0105] Process (2): For each parallel coding unit, it may be further divided into one or more independent coding units that do not overlap each other, and the independent coding units may not depend on each other but may share some parallel coding unit header information.

[0106] For example, the width of an independent coding unit is w_lcu and its height is h_lcu. If a parallel coding unit is split into one independent coding unit, the size of the independent coding unit must be exactly the same as the size of the parallel coding unit; otherwise, the width of the independent coding unit must be greater than its height (unless it is an edge region).

[0107] Typically, the independent coding unit may be a fixed w_lcu×h_lcu, where w_lcu and h_lcu are both 2 to the power N (N≧0). For example, the size of the independent coding unit may be 128×4, 64×4, 32×4, 16×4, 8×4, 32×2, 16×2, or 8×2.

[0108] As a possible example, the independent coding units may be fixed at 128x4. If the size of the parallel coding units is 256x8, the parallel coding units can be divided evenly into four independent coding units. If the size of the parallel coding units is 288x10, the parallel coding units divide the first and second rows into two 128x4 independent coding units and one 32x4 independent coding unit, and divide the third row into two 128x2 independent coding units and one 32x2 independent coding unit.

[0109] Note that the independent coding unit may include three components of luminance Y, chromaticity Cb, and chromaticity Cr, or three components of red (R), green (G), and blue (B), or may include only one of these components. When the independent coding unit includes three components, the sizes of these three components may be completely the same or different, which is specifically related to the input format of the image.

[0110] Process (3): For each independent coding unit, it may be further divided into one or more non-overlapping coding units, and each coding unit within the independent coding unit may be dependent on each other, for example, multiple coding units may be cross-referenced for precoding and decoding.

[0111] If the size of the coding unit and the independent coding unit is the same (i.e., if the independent coding unit is divided into only one coding unit), the size may be any of the sizes described in process (2).

[0112] When an independent coding unit is divided into multiple non-overlapping coding units, possible examples of division include horizontal equal division (the height of the coding unit is the same as that of the independent coding unit, but the width is different, which may be 1 / 2, 1 / 4, 1 / 8, 1 / 16, etc.), vertical equal division (the width of the coding unit is the same as that of the independent coding unit, but the height is different, which may be 1 / 2, 1 / 4, 1 / 8, 1 / 16, etc.), horizontal and vertical equal division (quadtree division), etc., of which horizontal equal division is preferred.

[0113] If the width of a coding unit is w_cu and the height is h_cu, the width must be greater than the height (unless it is an edge region). Typically, a coding unit may be a fixed w_cu x h_cu, where w_cu and h_cu are both 2 to the power N (N is 0 or greater), such as 16x4, 8x4, 16x2, 8x2, 8x1, 4x1, etc.

[0114] As one possible example, the coding unit may be a fixed 16x4. If the size of the independent coding unit is 64x4, the independent coding unit may be divided evenly into four coding units, and if the size of the independent coding unit is 72x4, the coding unit is divided into four 16x4 and one 8x4.

[0115] Note that the coding unit may include three components of luminance Y, chromaticity Cb, and chromaticity Cr (or three components of red R, green G, and blue B), or may include only one of these components. When the coding unit includes three components, the sizes of the three components may be completely the same or different, which is specifically related to the image input format.

[0116] Note that process (3) is an optional step in the video encoding and decoding method, and the video encoder / decoder may encode / decode the residual coefficients (or residual values) of the independent coding units obtained in process (2).

[0117] Process (4): The coding unit may be further divided into one or more non-overlapping prediction groups (PGs). PGs may be abbreviated as Groups. Each PG is encoded and decoded according to a selected prediction mode to obtain a predicted value for each PG. The predicted value for each PG constitutes a predicted value for the entire coding unit. A residual value for the coding unit is obtained based on the predicted value and the original value of the coding unit.

[0118] Process (5): Group the coding units based on their residual values, obtain one or more non-overlapping residual blocks (RBs), and encode and decode the residual coefficients of each RB according to the selected mode to form a residual coefficient stream. Specifically, the residual coefficients can be transformed or not transformed.

[0119] Here, the selected mode of residual coefficient encoding and decoding method in process (5) includes, but is not limited to, a semi-fixed length encoding method, an exponential Golomb (Golomb) encoding method, a Golomb-Rice encoding method, a truncated unary (TU) encoding method, a run length encoding (RLE) method, a direct encoding method of the original residual value, etc.

[0120] For example, the video encoder may directly encode the coefficients within the RBs.

[0121] Also, for example, the video encoder may perform a transform such as DCT, DST, or Hadamard transform on the residual block and encode the transformed coefficients.

[0122] As a possible example, when an RB is small, the video encoder may directly quantize each coefficient in the RB uniformly to perform binarization encoding. When an RB is large, the video encoder may further divide the RB into multiple coefficient groups (CGs), uniformly quantize each CG, and perform binarization encoding. In some embodiments of the present invention, the coefficient groups (CGs) and the quantization groups (Quantization Groups) may be the same.

[0123] The following describes an example of residual coefficient coding in a semi-fixed-length coding scheme. First, the maximum absolute value of residual values in one RB block is defined as the modified maximum. Next, the number of coding bits for residual coefficients in the RB block is determined (the number of coding bits for residual coefficients in the same RB block is the same). For example, if the coding length (CL) of the current RB block is 2 and the current residual coefficient is 1, 2 bits are required to code the residual coefficient 1, which is represented as 01. If the CL of the current RB block is 7, this indicates that 8 bits of residual coefficients and 1 code bit are coded. CL is determined by finding the smallest M value that satisfies the requirement that all residuals of the current sub-block are within the range of [-2^(M-1), 2^(M-1)]. If the two boundary values of -2^(M-1) and 2^(M-1) exist simultaneously, M needs to be increased by 1, i.e., M+1 bits are required to code all residuals of the current RB block. If only one of the two boundary values, -2^(M-1) and 2^(M-1), exists, one trailing bit needs to be coded to determine whether the boundary value is -2^(M-1) or 2^(M-1). If neither -2^(M-1) nor 2^(M-1) exists in any residual, there is no need to code the trailing bit.

[0124] Also, for some special cases, the video encoder may directly encode the original values of the image rather than the residual values.

[0125] Note that a coding block (CB) in an embodiment of the present invention corresponds to one image block in an image, and the coding block may be a coding unit divided in the above process (3), or a prediction group into which the coding unit is divided.

[0126] An image decoding method and an image encoding method according to an embodiment of the present invention will be described in detail below with reference to schematic configuration diagrams of a video coding system shown in FIG. 1, a video encoder shown in FIG. 2, and a video decoder shown in FIG. 3.

[0127] 5 is a flowchart of an image decoding method according to the present invention, which may be applied to the video coding system shown in FIG. 1 and may be performed by the decoding side 20. Specifically, the decoding method may be performed by a video decoder 200 included in the decoding side 20. As shown in FIG. 5, the image decoding method according to an embodiment of the present invention includes the following steps:

[0128] In S501, a prediction residual value of the first pixel point is determined based on the prediction method of the first pixel point.

[0129] The first pixel point is a pixel point currently being processed by the video decoder. The first pixel point is any one of pixel points in a current coding block. The current coding block is a decoding block currently being processed by the video decoder. Note that each pixel point in the current coding block can be decoded or coded using the solution provided by the present invention. The embodiment of the present invention will be described taking the coding and decoding process for the first pixel point as an example, and is not particularly limited.

[0130] It can be understood that the video stream to be decoded can be decoded to obtain one or more image frames included in the video stream. One image frame includes one or more image blocks. In the embodiment of the present invention, the current coding block corresponds to one image block of the image to be processed (the image to be processed is any one of the one or more image frames), and the coding block may be a coding unit (CU).

[0131] In some embodiments, the prediction residual value is used to reflect the gradient of the first pixel point. Here, the gradient of a pixel point is used to indicate the difference between the pixel point and its surrounding pixel points, and can reflect the complexity of the pixel point (e.g., texture characteristics, etc.). For example, a large gradient value of a pixel point indicates a large difference between the pixel point and its surrounding pixel points (which, when reflected in an image, indicates that the current coding block has a lot of texture information), indicating that the complexity of the pixel point is high. A small gradient value of a pixel point indicates a small difference between the pixel point and its surrounding pixel points (which, when reflected in an image, indicates that the current coding block is smooth), indicating that the complexity of the pixel point is low.

[0132] In some embodiments, the prediction residual value of the first pixel point is determined by the gradient of the reconstructed pixel points around the first pixel point. Alternatively, the prediction residual value of the first pixel point is determined by the residual values after inverse quantization of the reconstructed pixel points around the first pixel point. It should be understood that since the first pixel point is not reconstructed, the gradient of the first pixel point cannot be directly calculated based on the pixel value of the first pixel point. However, since the gradient information of the neighboring pixel points is relatively close, the gradient of the first pixel point can be predicted based on the gradient of the reconstructed pixel points around the first pixel point or based on the residual values after inverse quantization of the reconstructed pixel points around the first pixel point (the residual values after inverse quantization of the pixel point can reflect the gradient of the pixel point), that is, the prediction residual value of the first pixel point can be determined.

[0133] Exemplarily, the prediction residual value of the first pixel point includes a target value or an average value of target values, where the target value is the gradient of the reconstructed pixel points around the first pixel point, or the target value is the absolute value of the gradient of the reconstructed pixel points around the first pixel point, or the target value is the residual value after inverse quantization of the reconstructed pixel points around the first pixel point, or the target value is the absolute value of the residual value after inverse quantization of the reconstructed pixel points around the first pixel point.

[0134] Optionally, as shown in FIG. 6, the above step S501 may be specifically realized by the following steps S5011 to S5012.

[0135] In S5011, information on the reference pixel point of the first pixel point is obtained.

[0136] Here, the reference pixel points refer to surrounding reconstructed pixel points that are referenced by the first pixel point when performing pixel-by-pixel prediction.

[0137] As one possible embodiment, the above step S5011 may be realized by the following steps.

[0138] In step 1, information on reconstructed pixel points around a first pixel point is obtained.

[0139] The reconstructed pixel points around the first pixel point may be understood as reconstructed pixel points adjacent to the first pixel point. In an embodiment of the present invention, the reconstructed pixel points around the first pixel point include pixel points in a square region having the first pixel point as its center and a side length of a first predetermined value, pixel points in a diamond region having the first pixel point as its center and a diagonal length of a second predetermined value, or pixel points in an L-shaped region having the first pixel point as its vertex, a horizontal length of a third predetermined value, and a vertical length of a fourth predetermined value.

[0140] Optionally, the first preset value, the second preset value, the third preset value, and the fourth preset value may be set according to actual requirements. Also, the first preset value, the second preset value, the third preset value, and the fourth preset value may be equal to or different from each other. For example, the first preset value, the second preset value, the third preset value, and the fourth preset value may be one of 3 or 5.

[0141] 7A is a schematic diagram illustrating the division of a square region centered on a first pixel point, showing two possible cases. In Case 1, the reconstructed pixel point refers to a pixel point within a square region centered on the first pixel point and having a side length of 3, such as surrounding pixel point 1 shown in FIG. 7A. In Case 2, the reconstructed pixel point refers to a pixel point within a square region centered on the first pixel point and having a side length of 5, such as surrounding pixel point 2 shown in FIG. 7A.

[0142] FIG. 7(b) is a schematic diagram illustrating the division of a diamond-shaped region centered on the first pixel point, showing two possible cases. In case 1, the reconstructed pixel point refers to a pixel point within the diamond-shaped region whose center is the first pixel point and whose diagonal length is 3, such as surrounding pixel point 1 shown in FIG. 7(b). In case 2, the reconstructed pixel point refers to a pixel point within the diamond-shaped region whose center is the first pixel point and whose diagonal length is 5, such as surrounding pixel point 2 shown in FIG. 7(b).

[0143] FIG. 7C is a schematic diagram illustrating the division of a semi-enclosed region having a horizontal length equal to a third predetermined value and a vertical length equal to a fourth predetermined value, showing two possible cases. In Case 1, the reconstructed pixel point refers to a pixel point in the semi-enclosed region having an upper horizontal length of 3 and a left vertical length of 3, such as surrounding pixel point 1 shown in FIG. 7C. In Case 2, the reconstructed pixel point refers to a pixel point in the semi-enclosed region having an upper horizontal length of 5, a vertical length of 2, a left vertical length of 5, and a horizontal length of 2, such as surrounding pixel point 2 shown in FIG. 7C.

[0144] 7 is merely an example of an embodiment of the present invention for describing the reconstructed pixel points around the current pixel point, and should not be understood as limiting the present invention. In some other possible examples, the reconstructed pixel points around the current pixel point may be one or two pixel points adjacent to the current pixel point on the top and bottom, or on the left and right.

[0145] Optionally, the information about the reconstructed pixel points surrounding the first pixel point includes at least one of the following information: a pixel value of the reconstructed pixel point; a reconstructed residual value of the reconstructed pixel point (the reconstructed residual value includes a residual value after inverse quantization or a difference between a reconstructed value and a predicted value); a gradient value of the reconstructed pixel point (the gradient value includes a horizontal gradient, a vertical gradient, or an average gradient); flatness information, texture information, or complexity information; background brightness, contrast, or motion amount (the motion amount can be represented by a motion vector).

[0146] Furthermore, optionally, the value of the information of the reconstructed pixel point mentioned above includes at least one of an original value, an absolute value, an average value, or a difference value.

[0147] In step 2, information on a reference pixel point for the first pixel point is determined based on the prediction method for the first pixel point and information on reconstructed pixel points around the first pixel point.

[0148] In some embodiments, the prediction scheme is used to indicate the location of the reconstructed pixel point that is referenced when predicting the pixel point.

[0149] Optionally, the prediction scheme comprises making a prediction based on reconstructed pixel points on both the left and right sides of the pixel point, and / or making a prediction based on reconstructed pixel points on both the top and bottom sides of the pixel point, and / or making a prediction based on a reconstructed pixel point on the left side of the pixel point, and / or making a prediction based on a reconstructed pixel point above the pixel point.

[0150] As an example, as shown in Figure 8, taking a current coding block as an example, where the coding block is 16x2 (width w is 16, height h is 2), the prediction mode of the coding block is a pixel-wise prediction mode, and the pixel-wise prediction mode of the coding block includes two modes, mode 1 and mode 2. As shown in (a) of Figure 8, mode 1 is a vertical prediction mode for processing a vertical type texture, and as shown in (b) of Figure 8, mode 2 is a horizontal prediction mode for processing a horizontal type texture.

[0151] Here, RL indicates that the predicted value of the current pixel point is the average of the reconstructed values of the pixel points on both the left and right sides of the pixel point. TB indicates that the predicted value of the current pixel point is the average of the reconstructed values of the pixel points above and below the pixel point. L indicates that the predicted value of the current pixel point is the reconstructed value of the pixel point to the left of the pixel point. T indicates that the predicted value of the current pixel point is the reconstructed value of the pixel point above the pixel point.

[0152] In some embodiments, for convenience of explanation, a pixel point predicted based on the reconstructed pixel points on either side thereof will be referred to as a first type pixel point, a pixel point predicted based on the reconstructed pixel points on either side thereof will be referred to as a second type pixel point, a pixel point predicted based on the reconstructed pixel points on the left side thereof will be referred to as a third type pixel point, and a pixel point predicted based on the reconstructed pixel point on the upper side thereof will be referred to as a fourth type pixel point.

[0153] In some embodiments, first, a reference pixel point for the first pixel point is selected from reconstructed pixel points surrounding the first pixel point based on a prediction method for the first pixel point, and then information of the reference pixel point for the first pixel point is determined based on information of the reconstructed pixel points surrounding the first pixel point.

[0154] Optionally, selecting a reference pixel point for the first pixel point from reconstructed pixel points around the first pixel point based on a prediction scheme for the first pixel point includes the following cases:

[0155] Case 1: When the prediction method of the first pixel point is to perform prediction based on the reconstructed pixel points on both the left and right sides of the pixel point, the reference pixel points of the first pixel point are the reconstructed pixel point on the left side of the first pixel point and the reconstructed pixel point on the right side of the first pixel point.

[0156] Case 2: When the prediction method of the first pixel point is to perform prediction based on the reconstructed pixel points above and below the pixel point, the reference pixel points of the first pixel point are the reconstructed pixel point above the first pixel point and the reconstructed pixel point below the first pixel point.

[0157] Case 3: If the prediction mode of the first pixel point is to make a prediction based on the reconstructed pixel point to the left of the pixel point, the reference pixel point of the first pixel point is the reconstructed pixel point to the left of the first pixel point.

[0158] Case 4: If the prediction method of the first pixel point is to perform prediction based on the reconstructed pixel point above the pixel point, the reference pixel point of the first pixel point is the reconstructed pixel point above the first pixel point.

[0159] Optionally, determining the information of the reference pixel point of the first pixel point based on the information of the reconstructed pixel points surrounding the first pixel point includes the following two cases:

[0160] Case 1: If the reference pixel point is a reconstructed pixel point in the current coding block, the information of the reference pixel point is taken as the information of the reconstructed pixel points surrounding the first pixel point.

[0161] Case 2: If the reference pixel point is a reconstructed pixel point other than the current encoding block, the difference value or the absolute value of the difference value between the information of the reference pixel point and the information of the reconstructed pixel points surrounding the reference pixel point is used as the information of the reconstructed pixel points surrounding the first pixel point.

[0162] The reference pixel point of the first pixel point is a reconstructed pixel point. The reconstructed pixel point may be a reconstructed pixel point in the current coding block, or may not be a reconstructed pixel point in the current coding block, i.e., may be a reconstructed pixel point other than the current coding block. For example, when the prediction mode is an intra prediction mode, the reconstructed pixel point may be a pixel point surrounding the coding block in the current image frame, and when the prediction mode is an inter prediction mode, the reconstructed pixel point may be a reconstructed block on a reference frame of the current image frame.

[0163] As another possible embodiment, the above step S5011 may be realized by the following steps.

[0164] In step a1, a reference pixel point for the first pixel point is determined based on a prediction method for the first pixel point.

[0165] As an example, when the prediction method for a first pixel point is to perform prediction based on reconstructed pixel points on both the left and right sides of the pixel point, the reconstructed pixel point on the left side of the first pixel point and the reconstructed pixel point on the right side of the first pixel point are set as the reference pixel points for the first pixel point. When the prediction method for the first pixel point is to perform prediction based on reconstructed pixel points on both the top and bottom sides of the pixel point, the reconstructed pixel point above the first pixel point and the reconstructed pixel point below the first pixel point are set as the reference pixel points for the first pixel point. When the prediction method for the first pixel point is to perform prediction based on the reconstructed pixel point on the left side of the pixel point, the reconstructed pixel point on the left side of the first pixel point is set as the reference pixel point for the first pixel point. When the prediction method for the first pixel point is to perform prediction based on the reconstructed pixel point above the pixel point, the reconstructed pixel point above the first pixel point is set as the reference pixel point for the first pixel point.

[0166] In step a2, information on the reference pixel point of the first pixel point is obtained.

[0167] As an example, the information of the reference pixel point of the acquired first pixel point includes one or more of pixel value, flatness information or texture information or complexity information, background brightness, contrast, or motion amount (the motion amount can be expressed by a motion vector), etc.

[0168] Furthermore, optionally, the value of the information of the above-mentioned reference pixel point includes at least one of an original value, an absolute value, an average value, or a difference value.

[0169] In S5012, a prediction residual value of the first pixel point is determined based on information on the reference pixel point of the first pixel point.

[0170] In some embodiments, when the prediction method of the first pixel point is to make a prediction based on the reconstructed pixel points on both the left and right sides of the pixel point, the reference pixel points of the first pixel point are the reconstructed pixel point on the left side of the first pixel point and the reconstructed pixel point on the right side of the first pixel point, and determining the prediction residual value of the first pixel point has the following two methods:

[0171] Method 1: Determine a prediction residual value of the first pixel point based on the difference between the reconstructed value of the reference pixel point of the first pixel point and the predicted value. The above step S5012 may be realized by the following steps.

[0172] In step b1-1, the difference between the pixel values of the second pixel point and the third pixel point is calculated as a first difference value.

[0173] Here, the second pixel point is the first reconstructed pixel point on the left side of the first pixel point, and the third pixel point is the first reconstructed pixel point above the second pixel point.

[0174] For example, suppose the coordinates of a first pixel point in the target channel are (pic_x, pic_y), the coordinates of a second pixel point in the target channel are (pic_x-1, pic_y), the coordinates of a third pixel point in the target channel are (pic_x-1, pic_y-1), and the first difference value can be expressed as I[pic_y][pic_x-1]-I[pic_y-1][pic_x-1], where I[pic_y][pic_x-1] represents the pixel value of the pixel point (pic_x-1, pic_y) in the target channel, and I[pic_y-1][pic_x-1] represents the pixel value of the pixel point (pic_x-1, pic_y-1) in the target channel.

[0175] For a single pixel point, I represents the pixel value of the pixel point (for a reconstructed pixel point, I represents the reconstructed pixel value of the pixel point). Note that for the current coding block, I represents the pixel value matrix of the reconstructed pixel points included in the current coding block.

[0176] In some embodiments, the target channel represents the channel in which the current coding block is located. The channel in which the current coding block is located may be either the luma channel or the chroma channel. It is understood that an image typically corresponds to one luma channel and two chroma channels, and the target image may be any one of three channels.

[0177] In other embodiments, if the channel in which the current coding block is located is a chroma channel, the target channel may be represented as a luma channel. For example, assuming that the channel in which the first pixel point (a pixel point in the current coding block) is located is a chroma channel, when obtaining the bit depth of the first pixel point in the chroma channel, the bit depth of the first pixel point in the luma channel can be obtained to replace the bit depth of the first pixel point in the chroma channel.

[0178] In step b1-2, the difference between the pixel values of the fourth pixel point and the fifth pixel point is calculated as a second difference value.

[0179] Here, the fourth pixel point is the first reconstructed pixel point to the right of the first pixel point, and the fifth pixel point is the first reconstructed pixel point above the fourth pixel point.

[0180] For example, suppose the coordinates of a first pixel point in the target channel are (pic_x, pic_y), the coordinates of a fourth pixel point in the target channel are (pic_x+1, pic_y), the coordinates of a fifth pixel point in the target channel are (pic_x+1, pic_y-1), and the second difference value can be expressed as I[pic_y][pic_x+1]-I[pic_y-1][pic_x+1], where I represents the pixel value of the pixel point (for a reconstructed pixel point, I represents the pixel value of the pixel point after reconstruction), I[pic_y][pic_x+1] represents the pixel value of the pixel point (pic_x+1, pic_y) in the target channel, and I[pic_y-1][pic_x+1] represents the pixel value of the pixel point (pic_x+1, pic_y-1) in the target channel.

[0181] In step b1-3, the average value of the absolute value of the first difference value and the absolute value of the second difference value is set as the prediction residual value of the first pixel point.

[0182] The first difference value represents the vertical gradient of the second pixel point, and the second difference value represents the vertical gradient of the fourth pixel point. Since the second pixel point and the fourth pixel point are the reconstructed pixel points closest to the left and right sides of the first pixel point, respectively, the predicted residual value of the first pixel point (i.e., the gradient of the first pixel point) can be determined based on the average value of the gradient of the second pixel point and the gradient of the fourth pixel point.

[0183] For example, the position information of the first pixel point, the second pixel point, the third pixel point, the fourth pixel point, and the fifth pixel point is as shown in Figure 9. The prediction residual value of the first pixel point may satisfy the following equation (1). PredResiTmp=(Abs(I[pic_y][pic_x-1]-I[pic_y-1][pic_x-1])+Abs(I[pic_y][pic_x+1]-I[pic_y-1][pic_x+1])+1)>>1 Equation (1)

[0184] where PredResiTmp represents the prediction residual value of the pixel point, Abs represents a function for obtaining the absolute value, >> represents a right shift in binary, +1 represents rounding off, and >>1 represents a 1-bit right shift in binary and a division by 2 in decimal.

[0185] Method 2: Determine a prediction residual value of the first pixel point based on a residual value after inverse quantization of a reference pixel point of the first pixel point. The above step S5012 may be realized by the following steps.

[0186] In step b2-1, the residual value after inverse quantization of the second pixel point is calculated as the first difference value.

[0187] Here, the second pixel point is the first reconstructed pixel point on the left side of the first pixel point.

[0188] For example, assuming that the coordinates of a first pixel point in the target channel are (pic_x, pic_y), the coordinates of a second pixel point in the target channel are (pic_x-1, pic_y), and the residual value after inverse quantization of the second pixel point can be expressed as PointCoeffData[pic_y][pic_x-1].

[0189] For a single pixel point, PointCoeffData represents the residual value after dequantization of the pixel point. Note that for the current coding block, PointCoeffData represents the residual value matrix after dequantization of the reconstructed pixel points included in the current coding block.

[0190] In step b2-2, the residual value after inverse quantization of the fourth pixel point is calculated as a second difference value.

[0191] Here, the fourth pixel point is the first reconstructed pixel point on the right side of the first pixel point.

[0192] For example, assuming that the coordinates of the first pixel point in the target channel are (pic_x, pic_y), the coordinates of the fourth pixel point in the target channel are (pic_x+1, pic_y), and the residual value after inverse quantization of the fourth pixel point can be expressed as PointCoeffData[pic_y][pic_x+1].

[0193] In step b2-3, the average value of the absolute value of the first difference value and the absolute value of the second difference value is set as the prediction residual value of the first pixel point.

[0194] The first difference value represents the residual value after inverse quantization of the second pixel point and can reflect the gradient of the second pixel point, and the second difference value represents the residual value after inverse quantization of the fourth pixel point and can reflect the gradient of the fourth pixel point. Since the second pixel point and the fourth pixel point are the reconstructed pixel points closest to the left and right sides of the first pixel point, respectively, the predicted residual value of the first pixel point (i.e., the absolute value of the gradient of the first pixel point) can be determined based on the average value of the absolute values of the gradient of the second pixel point and the gradient of the fourth pixel point.

[0195] For example, the position information of the first pixel point, the second pixel point, and the fourth pixel point is as shown in Fig. 9. The prediction residual value of the first pixel point may satisfy the following equation (2). PredResiTmp=(Abs(PointCoeffData[pic_y][pic_x-1])+Abs(PointCoeffData[pic_y][pic_x+1])+1)>>1 Equation (2)

[0196] In some embodiments, when the prediction method of the first pixel point is to perform prediction based on the reconstructed pixel points above and below the pixel point, the reference pixel points of the first pixel point are the reconstructed pixel point above the first pixel point and the reconstructed pixel point below the first pixel point, and determining the prediction residual value of the first pixel point has the following two methods:

[0197] Method 1: Determine a prediction residual value of the first pixel point based on the difference between the reconstructed value of the reference pixel point of the first pixel point and the predicted value. The above step S5012 may be realized by the following steps.

[0198] In step c1-1, the difference between the pixel values of the sixth pixel point and the seventh pixel point is calculated as a third difference value.

[0199] Here, the sixth pixel point is the first reconstructed pixel point above the first pixel point, and the seventh pixel point is the first reconstructed pixel point to the left of the sixth pixel point.

[0200] For example, suppose the coordinates of the first pixel point in the target channel are (pic_x, pic_y), the coordinates of the sixth pixel point in the target channel are (pic_x, pic_y-1), the coordinates of the seventh pixel point in the target channel are (pic_x-1, pic_y-1), and the third difference value can be expressed as I[pic_y-1][pic_x]-I[pic_y-1][pic_x-1], where I[pic_y-1][pic_x] represents the pixel value of the pixel point (pic_x, pic_y-1) in the target channel, and I[pic_y-1][pic_x-1] represents the pixel value of the pixel point (pic_x-1, pic_y-1) in the target channel.

[0201] In step c1-2, the difference between the pixel values of the eighth pixel point and the ninth pixel point is calculated as a fourth difference value.

[0202] Here, the eighth pixel point is the first reconstructed pixel point below the first pixel point, and the ninth pixel point is the first reconstructed pixel point on the left side of the eighth pixel point.

[0203] For example, suppose the coordinates of the first pixel point in the target channel are (pic_x, pic_y), the coordinates of the eighth pixel point in the target channel are (pic_x, pic_y+1), the coordinates of the ninth pixel point in the target channel are (pic_x-1, pic_y+1), and the fourth difference value can be expressed as I[pic_y+1][pic_x]-I[pic_y+1][pic_x-1], where I represents the pixel value of the pixel point (for a reconstructed pixel point, I represents the pixel value of the pixel point after reconstruction), I[pic_y+1][pic_x] represents the pixel value of the pixel point (pic_x, pic_y+1) in the target channel, and I[pic_y+1][pic_x-1] represents the pixel value of the pixel point (pic_x-1, pic_y+1) in the target channel.

[0204] In step c1-3, the average value of the absolute value of the third difference value and the absolute value of the fourth difference value is set as the prediction residual value of the first pixel point.

[0205] The third difference value represents the horizontal gradient of the sixth pixel point, and the fourth difference value represents the horizontal gradient of the eighth pixel point. Since the sixth pixel point and the eighth pixel point are the reconstructed pixel points closest to the first pixel point on both the upper and lower sides, respectively, the predicted residual value of the first pixel point (i.e., the gradient of the first pixel point) can be determined based on the average value of the gradient of the sixth pixel point and the gradient of the eighth pixel point.

[0206] For example, the position information of the first pixel point, the sixth pixel point, the seventh pixel point, the eighth pixel point, and the ninth pixel point is as shown in Figure 10. The prediction residual value of the first pixel point may satisfy the following equation (3). PredResiTmp=(Abs(I[pic_y-1][pic_x]-I[pic_y-1][pic_x-1])+Abs(I[pic_y+1][pic_x]-I[pic_y+1][pic_x-1])+1)>>1 Equation (3)

[0207] Method 2: Determine a prediction residual value of the first pixel point based on the difference between the reconstruction value and the prediction value of the reference pixel point of the first pixel point and the residual value after inverse quantization of the reference pixel point of the first pixel point. The above step S5012 may be realized by the following steps.

[0208] In step c2-1, the difference between the pixel values of the sixth pixel point and the seventh pixel point is calculated as a third difference value.

[0209] Here, the sixth pixel point is the first reconstructed pixel point above the first pixel point, and the seventh pixel point is the first reconstructed pixel point to the left of the sixth pixel point.

[0210] In step c2-2, the residual value after inverse quantization of the eighth pixel point is calculated as a fourth difference value.

[0211] Here, the eighth pixel point is the first reconstructed pixel point below the first pixel point.

[0212] For example, assuming that the coordinates of the first pixel point in the target channel are (pic_x, pic_y), the coordinates of the eighth pixel point in the target channel are (pic_x, pic_y+1), and the residual value after inverse quantization of the eighth pixel point can be expressed as PointCoeffData[pic_y+1][pic_x].

[0213] In step c2-3, the average value of the absolute value of the third difference value and the absolute value of the fourth difference value is set as the prediction residual value of the first pixel point.

[0214] As can be seen, since the sixth pixel point is a pixel point outside the current coding block, the residual value after dequantization of the sixth pixel point cannot be calculated. Therefore, the difference between the reconstructed value and the predicted value of the sixth pixel point (i.e., the third difference value) is calculated to represent the gradient of the sixth pixel point. The fourth difference value represents the residual value after dequantization of the eighth pixel point and can reflect the gradient of the eighth pixel point. Since the sixth pixel point and the eighth pixel point are the reconstructed pixel points closest to the first pixel point on both the upper and lower sides, respectively, the predicted residual value of the first pixel point (i.e., the absolute value of the gradient of the first pixel point) can be determined based on the average absolute value of the gradient of the sixth pixel point and the gradient of the eighth pixel point.

[0215] For example, the position information of the first pixel point, the sixth pixel point, and the eighth pixel point is as shown in Fig. 10. The prediction residual value of the first pixel point may satisfy the following equation (4). PredResiTmp=(Abs(I[pic_y-1][pic_x]-I[pic_y-1][pic_x-1])+Abs(PointCoeffData[pic_y+1][pic_x])+1)>>1 Equation (4)

[0216] In some embodiments, when the prediction method of the first pixel point is to make a prediction based on a reconstructed pixel point to the left of the pixel point, the reference pixel point of the first pixel point is a reconstructed pixel point to the left of the first pixel point, and determining the prediction residual value of the first pixel point has the following two methods:

[0217] Method 1: A prediction residual value of the first pixel point is determined based on the difference between the reconstructed value of the reference pixel point of the first pixel point and the predicted value.

[0218] When the method 1 is adopted to calculate the prediction residual value of the first pixel point, the above step S5012 may include the following two cases:

[0219] Case 1: If the horizontal position of the first pixel point in the target channel is not 5, 9, or 13, the absolute value of the difference between the pixel values of the tenth and eleventh pixel points is set as the prediction residual value of the first pixel point.

[0220] Here, the tenth pixel point is the first reconstructed pixel point on the left side of the first pixel point, and the eleventh pixel point is the first reconstructed pixel point on the left side of the tenth pixel point.

[0221] As can be understood, the difference between the pixel values of the 10th pixel point and the 11th pixel point represents the horizontal gradient of the 10th pixel point, and since the 10th pixel point is the reconstructed pixel point closest to the left of the first pixel point, the predicted residual value of the first pixel point (i.e., the gradient of the first pixel point) can be determined based on the horizontal gradient of the 10th pixel point.

[0222] For example, the position information of the first pixel point, the tenth pixel point, and the eleventh pixel point is as shown in (a) of Figure 11. Assuming that the coordinates of the first pixel point in the target channel are (pic_x, pic_y), the coordinates of the tenth pixel point in the target channel are (pic_x-1, pic_y), and the coordinates of the eleventh pixel point in the target channel are (pic_x-2, pic_y), the prediction residual value of the first pixel point can satisfy the following equation (5): PredResiTmp=Abs(I[pic_y][pic_x-1]-I[pic_y][pic_x-2]) Equation (5)

[0223] Here, I[pic_y][pic_x-1] represents the pixel value of the pixel point (pic_x-1, pic_y) in the target channel, and I[pic_y][pic_x-2] represents the pixel value of the pixel point (pic_x-2, pic_y) in the target channel.

[0224] Case 2: When the horizontal position of the first pixel point in the target channel is 5, 9 or 13, the absolute value of the difference between the pixel values of the tenth and twelfth pixel points is set as the prediction residual value of the first pixel point.

[0225] Here, the tenth pixel point is the first reconstructed pixel point on the left side of the first pixel point, and the twelfth pixel point is the first reconstructed pixel point above the tenth pixel point.

[0226] As can be understood, the difference between the pixel values of the 10th pixel point and the 12th pixel point represents the vertical gradient of the 10th pixel point, and since the 10th pixel point is the reconstructed pixel point closest to the left of the first pixel point, the predicted residual value of the first pixel point (i.e., the gradient of the first pixel point) can be determined based on the vertical gradient of the 10th pixel point.

[0227] As shown in Figure 8, when the pixel-by-pixel prediction mode of the current coding block is mode 2, the prediction methods for the fifth, ninth, and thirteenth pixel points in the horizontal direction in the current coding block are all "T", that is, prediction is performed based on the reconstructed pixel point above the pixel point. Therefore, for the fifth, ninth, and thirteenth pixel points in the horizontal direction in the current coding block, information on the horizontally adjacent pixel points of the pixel points cannot be obtained, and the horizontal gradient cannot be calculated, but only the vertical gradient can be calculated.

[0228] Therefore, if the horizontal position of the first pixel point in the target channel is 5, 9 or 13, the vertical gradient of the first reconstructed pixel point to the left of the first pixel point (i.e., the 10th pixel point) should be taken as the prediction residual value of the first pixel point.

[0229] For example, the position information of the first pixel point, the tenth pixel point, and the twelfth pixel point is as shown in (b) of Figure 11. Assuming that the coordinates of the first pixel point in the target channel are (pic_x, pic_y), the coordinates of the tenth pixel point in the target channel are (pic_x-1, pic_y), and the coordinates of the twelfth pixel point in the target channel are (pic_x-1, pic_y-1), the prediction residual value of the first pixel point can satisfy the following equation (6): PredResiTmp=Abs(I[pic_y][pic_x-1]-I[pic_y-1][pic_x-1]) Equation (6)

[0230] where I represents the pixel value of the pixel point (for a reconstructed pixel point, I represents the pixel value of the pixel point after reconstruction), I[pic_y][pic_x-1] represents the pixel value of the pixel point (pic_x-1, pic_y) in the target channel, and I[pic_y-1][pic_x-1] represents the pixel value of the pixel point (pic_x-1, pic_y-1) in the target channel.

[0231] Method 2: A prediction residual value of the first pixel point is determined based on a residual value after inverse quantization of a reference pixel point of the first pixel point.

[0232] Specifically, step S5012 may be implemented by setting the absolute value of the inversely quantized residual value of the tenth pixel point as the predicted residual value of the first pixel point, where the tenth pixel point is the first reconstructed pixel point on the left side of the first pixel point.

[0233] As can be understood, the residual value after inverse quantization of the above-mentioned 10th pixel point can reflect the gradient of the 10th pixel point, and since the 10th pixel point is the reconstructed pixel point closest to the left of the first pixel point, the predicted residual value of the first pixel point (i.e., the absolute value of the gradient of the first pixel point) can be determined based on the absolute value of the gradient of the 10th pixel point.

[0234] For example, the position information of the first pixel point and the tenth pixel point is as shown in (b) of Figure 11. Assuming that the coordinates of the first pixel point in the target channel are (pic_x, pic_y), the coordinates of the tenth pixel point in the target channel are (pic_x-1, pic_y), and the prediction residual value of the first pixel point can satisfy the following equation (7): PredResiTmp=Abs(PointCoeffData[pic_y][pic_x-1]) Equation (7)

[0235] Here, PointCoeffData[pic_y][pic_x-1] represents the residual value after inverse quantization of the pixel point (pic_x-1, pic_y).

[0236] In some embodiments, when the prediction method of the first pixel point is to make a prediction based on a reconstructed pixel point above the pixel point, the reference pixel point of the first pixel point is a reconstructed pixel point above the first pixel point, and determining the prediction residual value of the first pixel point has the following two methods:

[0237] Method 1: A prediction residual value of the first pixel point is determined based on the difference between the reconstructed value of the reference pixel point of the first pixel point and the predicted value.

[0238] Specifically, step S5012 may be implemented by setting the absolute value of the difference between the pixel values of the thirteenth pixel point and the fourteenth pixel point as the prediction residual value of the first pixel point.

[0239] Here, the 13th pixel point is the first reconstructed pixel point above the first pixel point, and the 14th pixel point is the first reconstructed pixel point above the 13th pixel point.

[0240] As can be understood, the absolute value of the difference between the pixel values of the 13th pixel point and the 14th pixel point represents the vertical gradient of the 13th pixel point, and since the 13th pixel point is the reconstructed pixel point closest to the upper side of the first pixel point, the predicted residual value of the first pixel point (i.e., the gradient of the first pixel point) can be determined based on the vertical gradient of the 13th pixel point.

[0241] For example, the position information of the first pixel point, the thirteenth pixel point, and the fourteenth pixel point is as shown in Figure 12. Assuming that the coordinates of the first pixel point in the target channel are (pic_x, pic_y), the coordinates of the thirteenth pixel point in the target channel are (pic_x, pic_y-1), and the coordinates of the fourteenth pixel point in the target channel are (pic_x, pic_y-2), the prediction residual value of the first pixel point can satisfy the following equation (8): PredResiTmp=Abs(I[pic_y-1][pic_x]-I[pic_y-2][pic_x]) Equation (8)

[0242] Here, I[pic_y-1][pic_x] represents the pixel value of the pixel point (pic_x, pic_y-1) in the target channel, and I[pic_y-2][pic_x] represents the pixel value of the pixel point (pic_x, pic_y-2) in the target channel.

[0243] Method 2: A prediction residual value of the first pixel point is determined based on a residual value after inverse quantization of a reference pixel point of the first pixel point.

[0244] Specifically, step S5012 may be implemented by using the absolute value of the inverse-quantized residual value of the thirteenth pixel point as the predicted residual value of the first pixel point, where the thirteenth pixel point is the first reconstructed pixel point above the first pixel point.

[0245] As can be understood, the residual value after inverse quantization of the 13th pixel point can reflect the gradient of the 13th pixel point, and since the 13th pixel point is the reconstructed pixel point closest to the upper side of the first pixel point, the predicted residual value of the first pixel point (i.e., the absolute value of the gradient of the first pixel point) can be determined based on the absolute value of the gradient of the 13th pixel point.

[0246] For example, the position information of the first pixel point and the thirteenth pixel point is as shown in Figure 12. Assuming that the coordinates of the first pixel point in the target channel are (pic_x, pic_y), the coordinates of the thirteenth pixel point in the target channel are (pic_x, pic_y-1), and the prediction residual value of the first pixel point can satisfy the following equation (9): PredResiTmp=Abs(PointCoeffData[pic_y-1][pic_x]) Equation (9)

[0247] Here, PointCoeffData[pic_y-1][pic_x] represents the residual value after inverse quantization of the pixel point (pic_x, pic_y-1).

[0248] In S502, a target QP value for the first pixel point is determined based on the prediction residual value of the first pixel point.

[0249] In some embodiments, referring to FIG. 5, as shown in FIG. 13, before determining the target QP value of the first pixel point based on the prediction residual value of the first pixel point, the method further includes the following steps S504 to S505.

[0250] In S504, the QP value of the current coding block is obtained.

[0251] In one embodiment, the QP value of a current coding block may be obtained by analyzing the bitstream. Because a small QP is more likely to appear than a large QP in near-lossless compression techniques, a video encoder can directly encode the QP value of a coding block using a truncated unary (TU), truncated rice (TR), or exponential-Golomb algorithm, so that a video decoder can analyze the bitstream to obtain the QP value of the coding block.

[0252] In another embodiment, the QP value of the current coding block may be obtained based on the predicted QP value of the current coding block and a QP offset. For example, the QP value of the current coding block may be obtained by the following Scheme 1 or Scheme 2.

[0253] The process of method 1 includes steps S1 to S3.

[0254] In S1, the predicted QP value of the current coding block is obtained.

[0255] Optionally, the predicted QP value of the current coding block may be calculated based on the QP values of the surrounding blocks of the current coding block.

[0256] For example, the predicted QP value of the current coding block may be determined based on the QP value of the reconstructed block to the left of the current coding block and the QP value of the reconstructed block above the current coding block as follows: predQP=(QPA+QPB+1)>>1 Here, predQP represents the predicted QP value of the current coding block, QPA represents the reconstructed block to the left of the current coding, QPB represents the reconstructed block above the current coding block, +1 represents rounding, and >>1 represents a 1-bit right shift, i.e., division by 2.

[0257] At S2, the bitstream is parsed to obtain the QP offset of the current coding block.

[0258] It can be understood that in a video encoding process, a video encoder determines a predicted QP value of a current encoding block, determines a difference between the true QP value of the current encoding block and the predicted QP value, obtains a QP offset (which can be denoted as deltAQP) of the current encoding block, and then encodes the QP offset using variable-length coding and transmits the QP offset to a video decoder through a bitstream, so that the video decoder can obtain the bitstream and then parse the bitstream to obtain the QP offset of the current encoding block.

[0259] In S3, the sum of the predicted QP value of the current coding block and the QP offset is set as the QP value of the current coding block.

[0260] That is, QP=predQP+deltaQP, where QP represents the currently coded QP value, predQP represents the predicted QP value of the currently coded block, and deltaQP represents the QP offset of the currently coded block.

[0261] The process of Method 2 includes S10 to S30.

[0262] At S10, the predicted QP value of the current coding block is obtained.

[0263] For the description of S10, please refer to the related description of S1, which will not be repeated here.

[0264] In S20, a QP offset value of the current coding block is determined based on the derived information of the current coding block.

[0265] Here, the derived information includes at least one of flatness information of the current coding block, remaining space in the bitstream buffer, or distortion constraint information.

[0266] In a video encoding process, a video encoder uses a code control algorithm to derive a QP offset for a current encoding block based on derivation information for the current encoding block, but the video encoder does not transmit the QP offset in a bitstream. Thus, in a video decoding process, a video decoder derives a QP offset for a current encoding block using the same method as a video encoder. The process of deriving a QP offset for a current encoding block based on at least one of flatness information for the current encoding block, remaining space in a bitstream buffer, or distortion constraint information in S20 above is a conventional technique. For specific details, reference may be made to existing content related to derivation of a QP offset, and this process will not be described in detail in the embodiments of the present invention.

[0267] In S30, the sum of the predicted QP value of the current coding block and the QP offset is set as the QP value of the current coding block.

[0268] In an embodiment of the present invention, the predicted QP value of the current coding block can be obtained based on more information, such as the QP of the previous coding block of the current coding block, the number of coding bits of the previous coding block (prevBlockRate), the target bitrate (targetRate), flatness information of the current coding block, and the current bitstream buffer fullness (rcFullness) to derive the QP offset of the current coding block.

[0269] In S505, a predicted QP value of the first pixel point is determined based on the QP value of the current coding block.

[0270] Here, the predicted QP value of the first pixel point is the QP value of the current coding block.

[0271] In an embodiment of the present invention, the obtained QP value of the current coding block is used as the initial QP value (i.e., predicted QP value) of each pixel point of the current coding block, and the QP value of each pixel is obtained by adjusting or not adjusting the predicted QP value.

[0272] Based on S503 to S504, as shown in FIG. 14, determining a target QP value of the first pixel point based on the prediction residual value of the first pixel point (ie, S502) specifically includes S5021 to S5022.

[0273] In S5021, if the first pixel point is a target pixel point in the current coding block, adjust the predicted QP value of the first pixel point according to the prediction residual value to obtain the target QP value of the first pixel point.

[0274] The target QP value is less than the predicted QP value.

[0275] Here, the target pixel point is a pixel point that is preset for adjusting the QP value. In an embodiment of the present invention, the target pixel point is one or more designated pixel points in the current coding block, and these designated pixel points can be understood as pixel points or candidate pixel points whose QP values should be adjusted. A QP adjustment policy is executed for the candidate pixel points.

[0276] In one embodiment, all pixel points in the current coding block may be target pixel points, ie, pixel points that are preset for adjusting the QP value.

[0277] For example, the current coding block includes at least pixel points in a first portion and / or pixel points in a second portion, the pixel points in the first portion are set as pixel points whose QP values do not need to be adjusted, and the pixel points in the second portion are set as pixel points whose QP values should be adjusted, and the target pixel points are any one or more of the pixel points in the second portion.

[0278] The pixel points of the first part and the pixel points of the second part are determined based on the pixel-wise prediction mode of the current coding block. For example, as shown in Figure 15, the pixel-wise prediction mode of the current coding block includes mode 1 and mode 2. Specifically, when the size of the current coding block is 16x2, the pixel-wise prediction mode of the current coding block is shown in (a) and (b) of Figure 15, respectively. When the size of the current coding block is 8x2, the pixel-wise prediction mode of the current coding block is shown in (c) and (d) of Figure 15, respectively. When the size of the current coding block is 8x1, the pixel-wise prediction mode of the current coding block is shown in (e) and (f) of Figure 15, respectively. Therefore, when the pixel-wise prediction mode of the current coding block is mode 1 and mode 2, respectively, the pixel points of the second part of the current coding block may be as shown in Table 1 below.

[0279] [Table 1]

[0280] 15, when the pixel-by-pixel prediction mode of the current coding block is mode 1, the pixel points of the second portion include at least one of a first-type pixel point (RL) and a fourth-type pixel point (T) that is not in the first row. The first-type pixel point is a pixel point predicted based on the reconstructed pixel points on both the left and right sides of the pixel point, and the fourth-type pixel point is a pixel point predicted based on the reconstructed pixel point above the pixel point.

[0281] When the pixel-by-pixel prediction mode of the current coding block is mode 2, the pixel points of the second portion include at least one of a second-type pixel point (TB) that is not in the first column, a third-type pixel point (L) that is not in the first column, and a fourth-type pixel point (T) that is not in the first row. The second-type pixel points are pixel points predicted based on the reconstructed pixel points above and below the pixel point, and the third-type pixel points are pixel points predicted based on the reconstructed pixel point to the left of the pixel point.

[0282] In this way, after the first pixel point is determined to be the target pixel point in the current coding block, the predicted QP value of the first pixel point can be adjusted based on the prediction residual value of the first pixel point to obtain the target QP value of the first pixel point.

[0283] In an embodiment of the present invention, a QP value adjustment parameter table may be set for the first pixel point of the current coding block. For example, the following Table 2 shows some parameters required to adjust the QP value:

[0284] [Table 2]

[0285] Here, info may be a reconstructed residual value of a reconstructed pixel point surrounding the current pixel point, for example, the reconstructed residual value of the reconstructed pixel point includes a residual value after inverse quantization of the reconstructed pixel point, or a difference between the reconstructed value of the reconstructed pixel point and a predicted value.

[0286] Specifically, the above step S5021 may be realized by the following steps.

[0287] In step d1, a first threshold, a second threshold, and a third threshold are determined.

[0288] Here, the first threshold is the QP value corresponding to just noticeable distortion (i.e., jndQp in Table 2), the second threshold is the adjustable maximum QP value (i.e., adjustMaxQp in Table 2), and the third threshold is the information threshold of the reconstructed pixel point (i.e., resiThres in Table 2).

[0289] Optionally, the QP value (first threshold) corresponding to the just-noticeable distortion is determined based on the bit depth of the first pixel point in the target channel or the channel index of the first pixel point. If the target channel is a chrominance channel, the QP value (first threshold) corresponding to the just-noticeable distortion may be determined based on the bit depth of the first pixel point in the luma channel. Illustratively, the QP value corresponding to the just-noticeable distortion has a positive correlation with the bit depth of the first pixel point in the target channel.

[0290] Optionally, the adjustable maximum QP value (second threshold) is determined based on one or more of the bit depth of the first pixel point in the target channel, the channel index of the first pixel point, and the fullness of the bitstream buffer. If the target channel is a chrominance channel, the adjustable maximum QP value (second threshold) may be determined based on the bit depth of the first pixel point in the luma channel. Illustratively, the adjustable maximum QP value has a positive correlation with the bit depth of the first pixel point in the luma channel and a negative correlation with the fullness of the bitstream buffer.

[0291] Optionally, the reconstructed pixel point information threshold (third threshold) is determined based on one or more of the bit depth of the first pixel point in the destination channel, the channel index of the first pixel point, and the texture complexity of the current block. If the destination channel is a chrominance channel, the reconstructed pixel point information threshold (third threshold) may be determined based on the bit depth of the first pixel point in the luma channel. Illustratively, the reconstructed pixel point information threshold has a positive correlation with the bit depth of the first pixel point in the luma channel.

[0292] In one possible embodiment, first, the bit depth of the first pixel point in the target channel is adjusted, and the bit depth index of the first pixel point in the target channel is determined; and then, based on the bit depth index of the first pixel point in the target channel, a QP value corresponding to just noticeable distortion, an adjustable maximum QP value, and an information threshold of the reconstructed pixel point are respectively determined.

[0293] For example, the bit depth index of the first pixel point in the target channel satisfies the following equation (10): bdIdx=Clip3(BitDepth[component]1-8, 0, 8) Equation (10)

[0294] where bdIdx represents the bit depth index of the first pixel point in the destination channel, BitDepth[component] represents the bit depth of the first pixel point in the destination channel, and Clip is a clipping function.

[0295] It can be seen that the index value of the bit depth of the first pixel point in the target channel can be adjusted from 8 to 16, or from 0 to 8, according to the above formula (10), to facilitate the processing.

[0296] For example, the QP value (first threshold) corresponding to just noticeable distortion is determined by the bit depth of the first pixel point in the target channel, and the QP value (first threshold) corresponding to just noticeable distortion can satisfy the following equation (11): jndQp=(bdIdx<<2)+16 Equation (11)

[0297] where << represents a left shift in binary, and <<2 represents a left shift by 2 bits in binary and a multiplication by 4 in decimal.

[0298] As an example, the adjustable maximum QP value (second threshold) is determined by the bit depth of the first pixel point in the target channel, and the adjustable maximum QP value (second threshold) can satisfy the following equation (12): adjustMaxQp=((bdIdx>>1)<<3)+32 formula (12)

[0299] where >> represents a right shift in binary, >>1 represents a right shift by 1 bit in binary and a division by 2 in decimal, and <<3 represents a left shift by 3 bits in binary and a multiplication by 8 in decimal.

[0300] For example, the information threshold (third threshold) of the reconstructed pixel point is determined by the bit depth of the first pixel point in the target channel, and the information threshold (third threshold) of the reconstructed pixel point can satisfy the following equation (13): resiThres=10<<(bdIdx>>1) Equation (13)

[0301] Furthermore, it may be determined whether to adjust the predicted QP value of the first pixel point based on the first threshold, the second threshold, and the third threshold, as specifically shown in step d2 below.

[0302] In step d2, if the predicted QP value of the first pixel point is greater than or equal to the first threshold and less than or equal to the second threshold, and the prediction residual value of the first pixel point is less than or equal to the third threshold, adjust the predicted QP value of the first pixel point to obtain the target QP value; otherwise, take the predicted QP value of the first pixel point as the target QP value of the first pixel point.

[0303] In some embodiments, if the predicted QP value of the first pixel point and the prediction residual value of the first pixel point satisfy the above condition, the step of adjusting the predicted QP value of the first pixel point to obtain a target QP value of the first pixel point may be realized by a step of setting the target QP value to the larger value of the QP value obtained by subtracting a preset offset from the predicted QP value and the QP value corresponding to just noticeable distortion.

[0304] Here, the preset offset may be determined based on one or more of the bit depth of the first pixel point in the destination channel, the channel index of the first pixel point, the fullness of the bitstream buffer, and the texture complexity of the current coding block. If the destination channel is a chrominance channel, the preset offset may be determined based on the bit depth of the first pixel point in the luma channel.

[0305] For example, the target QP value of the first pixel point satisfies the following equation (14). PixelQp=Max(QP[component]-offset,jndQp) Equation (14)

[0306] where PixelQp represents the target QP value of the first pixel point (i.e., finalQp in Table 2), QP[component] represents the QP value of the current coding block in which the first pixel point is located, QP[component] can be the predicted QP value of the first pixel point (i.e., orgQp in Table 2), offset represents a preset offset (i.e., QP offset value), jndQp represents the QP value corresponding to just noticeable distortion, and Max represents taking the maximum value.

[0307] As can be understood, the prediction residual value reflects the gradient of the first pixel point, that is, it can reflect the complexity of the first pixel point (e.g., texture features, etc.). Therefore, if the prediction residual value of the first pixel point is equal to or less than the threshold value of the information of the reconstructed pixel point, it indicates that the first pixel point is a simple pixel point. If the predicted QP value of the first pixel point is equal to or greater than the QP value corresponding to the noticeable distortion (the QP value reflects detailed information of the spatial compression of the pixel point, and the larger the QP value, the stronger the distortion of the pixel point), it indicates that the distortion of the first pixel point is large. For simple pixel points, distortion can be reduced by reducing the QP value. Therefore, in an embodiment of the present invention, the target QP value is determined based on the QP value obtained by subtracting a preset offset from the predicted QP value.

[0308] On the other hand, in order to avoid excessive adjustment of the predicted QP value, in the embodiment of the present invention, the larger of the QP value obtained by subtracting a preset offset from the predicted QP value and the QP value corresponding to just noticeable distortion is set as the target QP value. In this way, the predicted QP value can be adjusted to reduce the distortion of the first pixel point, and excessive adjustment can be avoided, thereby reducing the decoding distortion of the image frame and improving the quality of the decoded image.

[0309] In S5022, if the first pixel point is a pixel point other than the target pixel point in the current coding block, the predicted QP value of the first pixel point is set as the target QP value.

[0310] In the embodiment of the present invention, pixel points other than the target pixel point in the current coding block are not adjusted, there is no need to execute the QP adjustment policy, and the QP values of these pixel points are the QP value of the current coding block.

[0311] As can be understood, in the prior art, the same QP value is used for all pixel points in one coding block (e.g., the above-mentioned current coding block), i.e., in the process of dequantizing the current coding block, the QP value is a QP value of the coding block granularity, so that the resulting image distortion is relatively large. In contrast, in the embodiment of the present invention, for one coding block (e.g., the above-mentioned current coding block), the video decoder determines a QP value for any one pixel point of the coding block, i.e., in the process of dequantizing the current coding block, the QP value is a QP value of the pixel granularity, so that the difference between different pixel points in the same coding block is fully taken into account. By assigning an appropriate QP value to each pixel point, the decoding distortion of the image frame can be reduced, i.e., the dequantization effect of the current coding block can be improved (the dequantization effect can be evaluated by the quality of the image obtained by decoding).

[0312] In S503, the first pixel point is inverse quantized based on the target QP value of the first pixel point.

[0313] Inverse quantizing the first pixel point specifically means inverse quantizing the level value of the first pixel point, and the level value of the first pixel point is obtained by the video decoder analyzing the bitstream.

[0314] In the encoding process, a video encoder predicts a current coding block to obtain a residual value of the current coding block (the residual value is the difference between the true value of the current coding block and the predicted value of the current coding block). The residual value may be referred to as a residual coefficient of the current coding block. Then, the residual coefficient of the current coding block is transformed and then quantized to obtain the quantized coefficient of the current coding block. Alternatively, the video encoder does not transform the residual coefficient of the current coding block, but directly quantizes the residual coefficient to obtain the quantized coefficient of the current coding block. The quantized coefficient may be referred to as a level value or a quantized residual coefficient. In the embodiments of the present invention, for ease of description, the quantized value is collectively referred to as a level value.

[0315] In an embodiment of the present invention, the step of inversely quantizing the level value of the first pixel point based on the target QP value of the first pixel point specifically includes the steps of determining a quantization step Qstep of the first pixel point based on the target QP value of the first pixel point, and inversely quantizing the level value of the first pixel point using the Qstep of the first pixel point for the selected quantizer combination.

[0316] Optionally, the quantizers are uniform or non-uniform quantizers, and the combination of quantizers is determined by mark information contained in the bitstream.

[0317] The video decoder can determine the Qstep based on the QP value by at least one of formula derivation or lookup, and the following provides three possible implementation methods.

[0318] Method 1:

number

[0319] Method 2:

number

[0320] Method 3:

number

[0321] Optionally, quantization and dequantization may be achieved using conventional scalar quantization methods in H.265 as described below. Quantization:

number

number

[0322] where l is the level value obtained after quantization, c is the residual coefficient to be quantized (the residual coefficient may be a transform domain residual coefficient, i.e., a transformed residual coefficient, or a pixel domain residual coefficient, i.e., a residual value), Qstep is the quantization step, f is a parameter for controlling rounding (f∈0,1), sign represents the sign function, floor represents the truncation function, and c' is the value after inverse quantization.

[0323] It can be seen that the larger the QP value and the larger the Qstep value, the coarser the quantization, the greater the image distortion due to quantization, and the smaller the bit rate for coefficient coding.

[0324] [0,1-f) represents the quantization dead zone, and the parameter f relates to the length of the quantization dead zone. The smaller f is, the longer the quantization dead zone becomes, and the level value after quantization approaches zero. When f=0.5, the above quantization and inverse quantization formulas correspond to rounding, and quantization distortion is minimized. When f<0.5, the smaller f is, the larger the quantization distortion becomes, and the coefficient coding bit rate becomes smaller. In H.265, f=1 / 3 is selected for I frames, and f=1 / 6 is selected for B / P frames.

[0325] Illustratively, the quantization or inverse quantization formula of the uniform quantizer can refer to the above quantization and inverse quantization formula, and the parameter f can be taken in the following manner:

[0326] Method 1: f is set to 0.5 or some other fixed value.

[0327] Method 2: f is adaptively determined based on the QP value, the prediction mode, and whether or not a transform is performed.

[0328] As described above, the image decoding process is described. According to the image decoding method provided in the embodiment of the present invention, the video decoder determines a prediction residual value (reflecting the gradient of the first pixel point) for each pixel point in the coding block based on the prediction method of each pixel point (the prediction method is used to indicate the position of the reconstructed pixel point referenced when predicting the pixel point), and determines a QP of the granularity of each pixel point for the pixel points in the coding block, thereby dequantizing each pixel point based on the target QP value of each pixel point, that is, performing dequantization on a pixel-by-pixel basis. This can reduce the decoding distortion of the image frame while maintaining a certain compression rate, and improve the authenticity and accuracy of image decoding.

[0329] Accordingly, in the image encoding method, the video encoder first obtains the QP, Qstep and residual coefficient of the pixel point, adaptively selects a quantizer, quantizes the residual coefficient, and finally adjusts the quantization coefficient to obtain the final level value, thereby realizing the encoding of the image frame.

[0330] Based on the video encoder 100 shown in Fig. 2, the present invention further provides an image encoding method. Fig. 16 is a flowchart of the image encoding method according to the present invention. The image encoding method may be performed by the video encoder 100, or may be performed by an encoding side (encoding side 10 shown in Fig. 1) that supports the functions of the video encoder 100. Here, an example will be described in which the video encoder 100 performs the encoding method. The image encoding method includes the following steps S601 to S603.

[0331] In S601, a prediction residual value of the first pixel point is determined based on the prediction method of the first pixel point.

[0332] The prediction residual value is used to reflect the gradient of the first pixel point, and the prediction scheme is used to indicate the position of the reconstructed pixel point to be referenced when predicting the pixel point, where the first pixel point is any one of the pixel points in the current coding block.

[0333] For the process of determining the prediction residual value of the first pixel point, please refer to the specific implementation process of step S501.

[0334] In S602, a target QP value for the first pixel point is determined based on the prediction residual value of the first pixel point.

[0335] For the process of determining the target QP value of the first pixel point, please refer to the specific implementation process of step S502.

[0336] In S603, the first pixel point is quantized based on the target QP value of the first pixel point.

[0337] The quantization shown in Figure 16 is the inverse process of the inverse quantization shown in Figure 5, and for quantization using a target QP value in the image coding method, reference can be made to the corresponding process in the image decoding method of Figures 5 to 15 above, and a description thereof will be omitted here.

[0338] As described above, according to the image encoding method provided in the embodiment of the present invention, the video encoder determines a prediction residual value (reflecting the gradient of the first pixel point) for each pixel point in the encoding block based on the prediction method of each pixel point (the prediction method is used to indicate the position of the reconstructed pixel point referenced when predicting the pixel point), and determines a QP of the granularity of each pixel point for the pixel points in the encoding block, thereby quantizing each pixel point based on the QP value of each pixel point, i.e., performing quantization on a pixel-by-pixel basis. This can reduce the decoding distortion of the image frame while maintaining a certain compression rate, and improve the authenticity and accuracy of image decoding.

[0339] It should be understood that to realize the functions in the above embodiments, the video encoder / video decoder includes corresponding hardware structures and / or software modules for performing each function. Those skilled in the art should easily understand that the present invention can be implemented in the form of hardware or a combination of hardware and computer software with reference to the example units and method steps described with reference to the embodiments disclosed in the present invention. Whether a function is implemented by hardware or driven by computer software depends on the specific application scenario and design constraints of the technical solution.

[0340] 17 is a schematic structural diagram of an image decoding device according to an embodiment of the present invention, where the image decoding device 700 includes a decision unit 701 and an inverse quantization unit 702. The image decoding device 700 is configured to implement the functions of the video decoder or decoding side in the above-mentioned embodiment of the image decoding method, and can therefore achieve the beneficial effects of the above-mentioned embodiment of the image decoding method. In the embodiment of the present invention, the image decoding device 700 may be the decoding side 20 or the video decoder 200 shown in FIG. 1, or may be the video decoder 200 shown in FIG. 3, or may be a module (e.g., a device) applied to the decoding side 20 or the video decoder 200.

[0341] The determination unit 701 and the inverse quantization unit 702 are configured to implement an image decoding method according to any of the embodiments shown in Figures 5 to 15. For detailed descriptions of the determination unit 701 and the inverse quantization unit 702, reference can be made directly to the relevant descriptions in the method embodiments shown in Figures 5 to 15.

[0342] For example, the determination unit 701 may be configured to determine a prediction residual value of a first pixel point based on a prediction scheme of the first pixel point, the prediction residual value being used to reflect a gradient of the first pixel point, the prediction scheme being used to indicate a position of a reconstructed pixel point referenced in predicting the pixel point, and the first pixel point being any one pixel point in a current coding block.

[0343] The determining unit 701 may be further configured to determine a target quantization parameter (QP) value of the first pixel point based on a prediction residual value of the first pixel point.

[0344] The inverse quantization unit 702 may be configured to inverse quantize the first pixel point based on a target QP value of the first pixel point.

[0345] 18 is a schematic structural diagram of an image encoding device according to the present invention, where the image encoding device 800 includes a decision unit 801 and a quantization unit 802. The image encoding device 800 is configured to implement the functions of the video encoder or encoding side in the above-described image encoding method embodiment, and can thus achieve the beneficial effects of the above-described image encoding method embodiment. In the embodiment of the present invention, the image encoding device 800 may be the encoding side 10 or video encoder 100 shown in FIG. 1, or the video encoder 100 shown in FIG. 2, or may be a module (e.g., a device) applied to the encoding side 10 or video encoder 100.

[0346] The determining unit 801 and the quantizing unit 802 are configured to implement the image encoding method provided in Fig. 16. For more detailed descriptions of the determining unit 801 and the quantizing unit 802, please refer directly to the related descriptions in the method embodiments shown in Figs.

[0347] For example, the determination unit 801 may be configured to determine a prediction residual value of a first pixel point based on a prediction scheme of the first pixel point, the prediction residual value being used to reflect a gradient of the first pixel point, the prediction scheme being used to indicate a position of a reconstructed pixel point referenced in predicting the pixel point, and the first pixel point being any one pixel point in a current coding block.

[0348] The determining unit 801 may be further configured to determine a target quantization parameter (QP) value of the first pixel point based on a prediction residual value of the first pixel point.

[0349] The quantization unit 802 may be configured to quantize the first pixel point based on a target QP value for the first pixel point.

[0350] The present invention further provides an electronic device, and FIG. 19 is a schematic structural diagram of an electronic device according to the present invention. As shown in FIG. 19, the electronic device 900 includes a processor 901 and a communication interface 902. The processor 901 and the communication interface 902 are coupled to each other. It can be understood that the communication interface 902 may be a transceiver or an input / output interface. Optionally, the electronic device 900 may further include a memory 903 configured to store instructions to be executed by the processor 901, to store input data required for the processor 901 to execute the instructions, or to store data generated after the processor 901 executes the instructions.

[0351] When the electronic device 900 is used to implement the methods shown in FIGS. 5 to 15, the processor 901 and the communication interface 902 are used to perform the functions of the determining unit 701 and the inverse quantization unit 702 above.

[0352] When the electronic device 900 is used to implement the method shown in FIG. 16, the processor 901 and the communication interface 902 are used to perform the functions of the determining unit 801 and the quantizing unit 802 above.

[0353] In an embodiment of the present invention, the specific connection medium between the communication interface 902, the processor 901, and the memory 903 is not limited. In an embodiment of the present invention, in FIG. 19, the communication interface 902, the processor 901, and the memory 903 are connected via a bus 904, and the bus is shown by a bold line in FIG. 19. The connection method between other components is merely an exemplary explanation and is not limited. The bus may be classified into an address bus, a data bus, a control bus, etc. For ease of representation, only one bold line is shown in FIG. 19, but this does not indicate that only one bus or one type of bus is present.

[0354] The memory 903 may be configured to store software programs and modules, such as program instructions / modules corresponding to an image decoding method or an image encoding method provided in an embodiment of the present invention. The processor 901 executes the software programs and modules stored in the memory 903 to perform various functional applications and data processing. The communication interface 902 may also be configured to communicate signaling and data with other devices. In the present invention, the electronic device 900 may have multiple communication interfaces 902.

[0355] It may be understood that the processor in embodiments of the present invention may be a central processing unit (CPU), neural processing unit (NPU), or graphics processing unit (GPU), or may be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0356] The steps of the method in the embodiments of the present invention may be implemented by hardware or by a processor executing software instructions. The software instructions may be configured with corresponding software modules, which may be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disk, removable hard disk, CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such that the processor can read information from and write information to the storage medium. Of course, the storage medium may be a component of the processor. The processor and the storage medium may be located in an ASIC. The ASIC may also be located in a network device or a terminal device. Of course, the processor and the storage medium may reside as discrete components in the network device or the terminal device.

[0357] In the above embodiments, all or part of the invention may be implemented by software, hardware, firmware, or any combination thereof. When implemented using a software program, all or part of the invention may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the procedures or functions according to the embodiments of the present invention are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, radio, microwave, etc.) method. The computer-readable storage medium may be any available medium accessible by a computer, or a data storage device such as a server or data center in which one or more available media are integrated. The available medium may be a magnetic medium (e.g., a floppy disk, magnetic disk, or magnetic tape), an optical medium (e.g., a digital video disc (DVD)), or a semiconductor medium (e.g., a solid state drive (SSD)).

[0358] From the description of the above embodiments, it is clear to those skilled in the art that for convenience and simplicity of description, only the division of each of the above functional modules is described as an example, and in actual application, the above functions can be assigned to different functional modules to complete them as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. For the specific operation processes of the above systems, devices, and units, reference can be made to the corresponding processes in the above method embodiments, and they will not be repeated here.

[0359] It should be understood that in some embodiments provided by the present invention, the disclosed system, device, and method may be realized in other ways. For example, the above-described device embodiments are merely exemplary, and the division of modules or units is merely a division of logical functions, and may be divided in other ways in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not implemented. Meanwhile, the couplings or direct couplings or communication connections shown or discussed with each other may be indirect couplings or communication connections via some interfaces, devices, or units, which may be electrical, mechanical, or other types of couplings.

[0360] The units described as separate components may or may not be physically separated. Furthermore, components described as units may or may not be physical units. That is, they may be located in one place or distributed across multiple network units. Depending on actual needs, some or all of the units may be selected to achieve the objectives of the solution of this embodiment.

[0361] Furthermore, the functional units in each embodiment of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The integrated units may be realized in the form of hardware or in the form of a software functional unit.

[0362] The integrated unit may be realized in the form of a software functional unit and stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention may be embodied essentially, or in part, in the form of a software product, whether that part contributes to the prior art or all or part of the technical solution. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) or a processor to execute all or part of the steps of the method according to each embodiment of the present invention. The storage medium includes various media capable of storing program code, such as a flash memory, a removable hard disk, a read-only memory, a random access memory, a magnetic disk, or an optical disk.

[0363] The above are only specific embodiments of the present invention, and the protection scope of the present invention is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present invention shall be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. An image decoding method executed by a decoding side, comprising: determining a prediction residual value of the first pixel point based on a prediction scheme of the first pixel point, the prediction residual value being used to reflect a gradient of the first pixel point, the prediction scheme being used to indicate a position of a reconstructed pixel point referenced in predicting the pixel point, the first pixel point being any one pixel point in a current coding block; determining a target quantization parameter (QP) value for the first pixel point based on a prediction residual value for the first pixel point; and dequantizing the first pixel point based on a target QP value of the first pixel point.

1. An image decoding method comprising:

2. The prediction method is making a prediction based on reconstructed pixel points on both the left and right sides of the pixel point; and / or making a prediction based on a reconstructed pixel point to the left of the pixel point; and / or making a prediction based on a reconstructed pixel point above the pixel point; 2. The method of claim 1 .

3. The prediction method of the first pixel point is to perform prediction based on reconstructed pixel points on both sides of the pixel point; determining a prediction residual value of the first pixel point based on a prediction method of the first pixel point, a step of calculating a difference between pixel values of a second pixel point and a third pixel point as a first difference value, or setting a residual value after inverse quantization of the second pixel point as the first difference value, wherein the second pixel point is a first reconstructed pixel point on the left side of the first pixel point, and the third pixel point is a first reconstructed pixel point above the second pixel point; a step of calculating a difference between pixel values of a fourth pixel point and a fifth pixel point as a second difference value, or setting a residual value after inverse quantization of the fourth pixel point as the second difference value, wherein the fourth pixel point is the first reconstructed pixel point to the right of the first pixel point, and the fifth pixel point is the first reconstructed pixel point above the fourth pixel point; and determining an average value of the absolute value of the first difference value and the absolute value of the second difference value as a prediction residual value of the first pixel point.

3. The method of claim 2.

4. The prediction method of the first pixel point is to perform prediction based on a reconstructed pixel point on the left side of the pixel point; determining a prediction residual value of the first pixel point based on a prediction method of the first pixel point, a step of setting an absolute value of a difference between pixel values of a tenth pixel point and an eleventh pixel point as a prediction residual value of the first pixel point, the tenth pixel point being the first reconstructed pixel point on the left side of the first pixel point, and the eleventh pixel point being the first reconstructed pixel point on the left side of the tenth pixel point; or a step of setting the absolute value of the difference between the pixel values of a tenth pixel point and a twelfth pixel point as the prediction residual value of the first pixel point, the twelfth pixel point being the first reconstructed pixel point above the tenth pixel point, or a step of setting an absolute value of the residual value after inverse quantization of the tenth pixel point as a predicted residual value of the first pixel point, 3. The method of claim 2.

5. a prediction method for the first pixel point is to predict the first pixel point based on a reconstructed pixel point above the first pixel point; determining a prediction residual value of the first pixel point based on a prediction method of the first pixel point, a step of setting an absolute value of a difference between pixel values of a thirteenth pixel point and a fourteenth pixel point as a prediction residual value of a first pixel point, the thirteenth pixel point being the first reconstructed pixel point above the first pixel point, and the fourteenth pixel point being the first reconstructed pixel point above the thirteenth pixel point; or a step of setting an absolute value of the residual value after inverse quantization of the thirteenth pixel point as a predicted residual value of the first pixel point, 3. The method of claim 2.

6. the prediction residual value of the first pixel point includes a target value or an average value of the target values; the target value is a gradient of reconstructed pixel points around the first pixel point, or the target value is an absolute value of a gradient of reconstructed pixel points around the first pixel point, or the target value is a residual value after inverse quantization of reconstructed pixel points around the first pixel point, or the target value is an absolute value of a residual value after inverse quantization of reconstructed pixel points around the first pixel point.

2. The method of claim 1 .

7. The method further includes determining a predicted QP value for the first pixel point; the predicted QP value of the first pixel point is the QP value of the current coding block; the first pixel point is a target pixel point within the current coding block; The step of determining a target QP value of the first pixel point based on a prediction residual value of the first pixel point includes: adjusting a predicted QP value of the first pixel point based on the prediction residual value to obtain a target QP value of the first pixel point; The target pixel point is a pixel point that is preset for adjusting the QP value.

10. The method according to claim 1 or 6.

8. adjusting a predicted QP value of the first pixel point based on the prediction residual value to obtain a target QP value of the first pixel point, When the predicted QP value of the first pixel point is equal to or greater than a first threshold and equal to or less than a second threshold, and the prediction residual value of the first pixel point is equal to or less than a third threshold, adjusting the predicted QP value of the first pixel point to obtain the target QP value; otherwise, setting the predicted QP value of the first pixel point as the target QP value of the first pixel point; The target QP value is less than the predicted QP value.

8. The method of claim 7.

9. The first threshold is a QP value corresponding to just noticeable distortion, and the second threshold is an adjustable maximum QP value.

9. The method of claim 8.

10. adjusting the predicted QP value of the first pixel point to obtain the target QP value of the first pixel point, and setting the target QP value to a larger value of a QP value obtained by subtracting a preset offset from the predicted QP value and a QP value corresponding to the just noticeable distortion.

10. The method of claim 9.

11. The QP value corresponding to the just noticeable distortion is determined based on a bit depth of the first pixel point in a target channel or a channel index of the first pixel point; the adjustable maximum QP value is determined based on one or more of a bit depth of the first pixel point in a target channel, a channel index of the first pixel point, and a fullness of a bitstream buffer; The third threshold is determined based on one or more of a bit depth of the first pixel point in a target channel, a channel index of the first pixel point, and a texture complexity of a current block; the preset offset is determined based on one or more of a bit depth of the first pixel point in a target channel, a channel index of the first pixel point, a fullness of a bitstream buffer, and a texture complexity of the current coding block.

11. The method of claim 10.

12. An image coding method executed by a coding side, comprising: determining a prediction residual value of the first pixel point based on a prediction scheme of the first pixel point, the prediction residual value being used to reflect a gradient of the first pixel point, the prediction scheme being used to indicate a position of a reconstructed pixel point referenced in predicting the pixel point, the first pixel point being any one pixel point in a current coding block; determining a target quantization parameter (QP) value for the first pixel point based on a prediction residual value for the first pixel point; quantizing the first pixel point based on a target QP value for the first pixel point; 1. An image coding method comprising:

13. the prediction residual value of the first pixel point includes a target value or an average value of the target values; the target value is a gradient of reconstructed pixel points around the first pixel point, or the target value is an absolute value of a gradient of reconstructed pixel points around the first pixel point, or the target value is a residual value after inverse quantization of reconstructed pixel points around the first pixel point, or the target value is an absolute value of a residual value after inverse quantization of reconstructed pixel points around the first pixel point.

13. The method of claim 12.

14. The method further includes determining a predicted QP value for the first pixel point; the predicted QP value of the first pixel point is the QP value of the current coding block; the first pixel point is a target pixel point within the current coding block; The step of determining a target QP value of the first pixel point based on a prediction residual value of the first pixel point includes: adjusting a predicted QP value of the first pixel point based on the prediction residual value to obtain a target QP value of the first pixel point; The target pixel point is a pixel point that is preset for adjusting the QP value.

14. The method according to claim 12 or 13.

15. An image decoding device comprising a decision unit and an inverse quantization unit, The determining unit is configured to determine a prediction residual value of the first pixel point based on a prediction method of the first pixel point, the prediction residual value being used to reflect a gradient of the first pixel point, the prediction method being used to indicate a position of a reconstructed pixel point referenced in predicting the pixel point, the first pixel point being any one pixel point in a current coding block; The determining unit is further configured to determine a target quantization parameter (QP) value for the first pixel point based on a prediction residual value of the first pixel point; the inverse quantization unit is configured to inverse quantize the first pixel point based on a target QP value of the first pixel point; An image decoding device comprising:

16. An image coding device comprising a decision unit and a quantization unit, The determining unit is configured to determine a prediction residual value of the first pixel point based on a prediction method of the first pixel point, the prediction residual value being used to reflect a gradient of the first pixel point, the prediction method being used to indicate a position of a reconstructed pixel point referenced in predicting the pixel point, the first pixel point being any one pixel point in a current coding block; The determining unit is further configured to determine a target quantization parameter (QP) value for the first pixel point based on a prediction residual value of the first pixel point; the quantization unit is configured to quantize the first pixel point based on a target QP value of the first pixel point; An image encoding device comprising:

17. A video coding system comprising an encoding side and a decoding side, the encoding side being communicatively connected to the decoding side, the decoding side being configured to implement the method according to any one of claims 1 to 11, and the encoding side being configured to implement the method according to any one of claims 12 to 14. A video coding system comprising:

18. a memory configured to store computer instructions; and a processor configured to retrieve and execute the computer instructions from the memory to perform the method of any one of claims 1 to 14. An electronic device characterized by:

19. A computer readable storage medium having stored thereon a computer program or instructions, which, when executed by an electronic device, performs the method of any one of claims 1 to 14. A computer-readable storage medium comprising:

Images