Video encoding / decoding method, apparatus, and storage medium

Abstract

Embodiments of the present invention provide a video encoding / decoding method, apparatus, and storage medium, relate to the technical field of video encoding / decoding, and are useful for improving encoding / decoding efficiency. The method includes obtaining complexity information of a current block in a processing target image, where the complexity information of the current block is obtained by calculating at least one angular gradient of the current block based on at least pixel values of the current block, determining a quantization parameter of the current block based on the complexity information of the current block, and decoding the current block based on the quantization parameter.

Description

[Technical Field]

[0001] (Cross-reference to related applications) This invention claims priority based on a Chinese patent application filed with the China National Intellectual Property Administration on May 31, 2022, application number 202210612716.1, with the title of the invention "Video encoding / decoding method, apparatus and storage medium." All of its contents are incorporated herein by reference.

[0002] The present invention relates to the field of video encoding / decoding technology, and more particularly to video encoding / decoding methods, apparatus, and storage media. [Background technology]

[0003] Video coding / decoding techniques play a crucial role in the field of video processing. These techniques include the coding and decoding of video. In the video coding and decoding process, the quantization or dequantization of images within the video is key to determining image quality. Quantization primarily reduces the redundancy of the original data in the code stream by replacing some of the original data in the code stream with quantization parameters. However, the quantization process carries the risk of image distortion. Therefore, when considering image quality, determining more accurate quantization parameters for images in video is an urgent issue that needs to be addressed to improve video coding / decoding efficiency. [Overview of the project]

[0004] Embodiments of the present invention provide a video encoding / decoding method, apparatus, and storage medium, which are useful in improving video encoding / decoding efficiency.

[0005] To achieve the above objectives, the embodiments of the present invention employ the following technical solutions.

[0006] In a first embodiment, an embodiment of the present invention provides a video coding / decoding method. The method is applied to a video coding device, a video decoding device, or a chip for a video coding and decoding device. The method includes obtaining complexity information of a current block in an image to be processed, wherein the complexity information of the current block is obtained by calculating at least one angular gradient of the current block based on at least the pixel values ​​of the current block; determining quantization parameters of the current block based on the complexity information of the current block; and coding / decoding the current block based on the quantization parameters.

[0007] Quantization parameters play a crucial role in video coding and decoding processes. When using the video coding / decoding method according to the present invention, the video coding / decoding device acquires complexity information of the current block in the image being processed. This complexity information is calculated based on the current block information, and the quantization parameters of the current block are determined based on this complexity information before coding / decoding. Because the angular gradient information of the current block is taken into consideration, it helps to determine more accurate quantization parameters for the current block, thereby improving video coding / decoding efficiency while taking image quality into account. Furthermore, when video decoding is performed using the above method, the decoding side acquires complexity information of the current block from the code stream to determine the quantization parameters, which helps to reduce the resources occupied by the quantization parameters in the code stream. This allows more effective data to be transmitted within the code stream, improving transmission efficiency.

[0008] In one possible embodiment, obtaining complexity information of the current block in the image to be processed includes calculating at least one angular gradient of the current block based on the pixel value of the current block and the reconstructed value of the encoded / decoded pixel value of the current block, and obtaining complexity information of the current block based on at least one angular gradient of the current block.

[0009] In this possible embodiment, calculating the complexity of the current block using the current block's pixel values ​​and reconstruction values ​​helps determine more accurate quantization parameters for the current block, improving video encoding / decoding efficiency while taking image quality into consideration.

[0010] In one possible embodiment, obtaining complexity information of the current block in the image to be processed includes calculating at least one angular gradient of the current block based on the pixel value of the current block and the pixel values ​​adjacent to the current block in the image to be processed, and obtaining complexity information of the current block based on at least one angular gradient of the current block.

[0011] In this possible embodiment, calculating the complexity of the current block using the pixel values ​​of the current block and the pixel values ​​of the pixels adjacent to the current block helps determine more accurate quantization parameters for the current block, thereby improving video encoding / decoding efficiency while taking image quality into consideration.

[0012] In one possible embodiment, obtaining complexity information of the current block in the image to be processed includes obtaining the predicted angle used in the angle prediction mode of the current block, calculating the angle gradient based on the predicted angle to obtain the corresponding complexity information, and using the corresponding complexity information as the complexity information of the current block.

[0013] In this possible embodiment, determining the corresponding complexity by the angle prediction mode of the current block helps to determine more accurate quantization parameters for the current block, which on the decoding side helps to conserve resources in the code stream and improve video encoding / decoding efficiency.

[0014] In one possible embodiment, the current block is an N-channel image block, and obtaining complexity information of the current block in the image to be processed includes obtaining complexity information for each channel image block based on the pixel values ​​of each channel image block in the N-channel image block, where N is an integer greater than 0, and determining the complexity information of the current block based on the complexity information of each channel image block.

[0015] In this possible embodiment, a method is provided to determine the complexity of the current block based on the complexity of multiple channel image blocks, thereby improving the feasibility of the technical proposal. Furthermore, dividing the image into multiple channels and calculating each one helps to improve the accuracy of the complexity information obtained by the determination.

[0016] In one possible embodiment, obtaining complexity information for each channel image block based on the pixel values ​​of each channel image block in an N-channel image block includes dividing each channel image block into at least two subblocks, determining complexity information for at least two subblocks of each channel image block, and determining complexity information for the corresponding channel image block in each channel image block based on the complexity information for at least two subblocks of each channel image block.

[0017] In this possible embodiment, a configuration is provided for determining the complexity of a current block based on the complexity of multiple channel image blocks. Further subdividing the multiple channel image blocks and determining their complexity helps to improve the accuracy of the complexity information obtained by the determination.

[0018] In a possible embodiment, determining the complexity information of a corresponding channel image block based on the complexity information of at least two sub-blocks of each channel image block includes determining the minimum value of the complexity information of at least two sub-blocks of each channel image block as the complexity information of the corresponding channel image block.

[0019] In this possible embodiment, a form of determining the complexity of a plurality of channel image blocks based on the complexity of the plurality of divided channel image blocks is provided, improving the feasibility of implementing the technical solution.

[0020] In a possible embodiment, determining the complexity information of the current block based on the complexity information of each channel image block includes determining the minimum value of the complexity information of each channel image block as the complexity information of the current block.

[0021] In this possible embodiment, a form of determining the complexity of a plurality of channel image blocks based on the complexity information of the plurality of channel image blocks is provided, improving the feasibility of implementing the technical solution.

[0022] In a possible embodiment, determining the complexity information of the current block based on the complexity information of each channel image block includes determining the complexity level of each channel image block based on the complexity information of each channel image block, and determining the complexity information of the current block based on the complexity level of each channel image block.

[0023] In this possible embodiment, a form of determining the complexity of a plurality of channel image blocks based on the complexity of the plurality of divided channel image blocks is provided, improving the feasibility of implementing the technical solution.

[0024] In one possible embodiment, determining the quantization parameters of a current block based on the complexity information of the current block includes determining the reference quantization parameters of the current block based on the complexity information of the current block, and determining the quantization parameters of the current block based on the reference quantization parameters of the current block.

[0025] In this possible embodiment, a method is provided for determining quantization parameters based on a reference quantization parameter, thereby improving the accuracy of the quantization parameters obtained by the determination.

[0026] In one possible embodiment, when the video encoding / decoding method is a video encoding method, determining the reference quantization parameter of the current block based on the complexity information of the current block includes obtaining the buffer region state of the image to be processed, where the buffer region state is used to represent the number of bits occupied by the encoded image block in the image to be processed within the buffer region, and where the buffer region is used to control the uniform output of the code stream of the image to be processed, and determining the reference quantization parameter of the current block based on the correspondence between the buffer region state and the complexity information of the current block. Of course, when the video encoding / decoding method is a video decoding method, the reference quantization parameter can be determined by simulating the situation of buffering the code stream in the buffer region in the encoding process during the decoding process, based on the results of the simulation.

[0027] In this possible embodiment, a configuration is provided in which the reference quantization parameter of the current block is determined based on the buffer region state and the complexity information of the current block, thereby improving the feasibility of the technical proposal.

[0028] In one possible embodiment, determining the reference quantization parameter of a current block based on the complexity information of the current block includes determining the complexity level of the current block, determining the corresponding target bits based on the complexity level of the current block, where the target bits refer to the number of bits occupied by the current block in the code stream, and obtaining the reference quantization parameter of the current block based on the target bits.

[0029] In this possible embodiment, a configuration is provided in which the reference quantization parameter of the current block is determined based on the target bit, thereby improving the feasibility of the technical proposal.

[0030] In one possible embodiment, determining the quantization parameters of a current block based on the reference quantization parameters of the current block includes determining weight coefficients based on complexity information of the current block, wherein the weight coefficients are for adjusting the quantization parameters of the current block based on the degree of complexity of the current block, and determining the quantization parameters of the current block based on the weight coefficients and the reference quantization parameters of the current block.

[0031] In this possible embodiment, a configuration is provided in which the quantization parameters of the current block are determined based on a reference quantization parameter, thereby improving the feasibility of the technical proposal.

[0032] In one possible embodiment, the complexity information of the current block is obtained based on calculations by the current block's coding rate control unit, which is the basic processing unit that calculates the complexity information of the current block. The quantization parameter of the current block is the quantization parameter of the current block's coding rate control unit. Encoding / decoding the current block based on the current block's quantization parameter includes determining the quantization parameter of the current block's encoding / decoding unit based on the quantization parameter of the coding rate control unit, and encoding / decoding the current block based on the quantization parameter of the encoding / decoding unit.

[0033] In this possible embodiment, a form is provided for determining the quantization parameters of a block, where, if the size of the coding rate control unit is smaller than the size of the quantization unit, the multiple quantization parameters obtained by the corresponding calculation are used for quantization by a single quantization unit. To address this problem, the above method provides a corresponding technical proposal and improves the feasibility of the proposal.

[0034] In a second embodiment, an embodiment of the present invention provides a video encoding / decoding device. The device has a function to implement the video encoding / decoding method described in any one of the first embodiments. This function may be implemented by hardware or by running corresponding software on hardware. The hardware or software includes one or more modules corresponding to the above function.

[0035] In a third embodiment, an embodiment of the present invention provides a video encoder for performing the video encoding / decoding method described in any one of the first embodiments.

[0036] In a fourth embodiment, an embodiment of the present invention provides another video encoder comprising a processor and memory for storing computer execution instructions, wherein when the video encoder is in operation, the processor executes the computer execution instructions stored in the memory so that the video encoder performs the video encoding / decoding method described in any one of the first embodiments.

[0037] In a fifth embodiment, an embodiment of the present invention provides a video decoder for performing the video encoding / decoding method described in any one of the first embodiments.

[0038] In a sixth embodiment, an embodiment of the present invention provides another video decoder, the video decoder comprising a processor and a memory for storing computer execution instructions, and when the video decoder is in operation, the processor executes the computer execution instructions stored in the memory so that the video decoder performs the video encoding / decoding method described in any one of the first embodiments.

[0039] In a seventh embodiment, an embodiment of the present invention provides a computer-readable storage medium in which a program is stored, and when executed on a computer, the program causes the computer to execute the video encoding / decoding method described in any one of the first embodiments.

[0040] In an eighth embodiment, an embodiment of the present invention provides a computer program product including instructions, which, when executed on a computer, cause the computer to perform the video encoding / decoding method described in any one of the first embodiments.

[0041] In a ninth embodiment, an embodiment of the present invention provides an electronic device comprising a video encoding / decoding device, wherein a processing circuit is configured to perform the video encoding / decoding method described in any one of the first embodiments.

[0042] In a tenth embodiment, an embodiment of the present invention provides a chip comprising a processor, the processor being coupled with memory, the memory storing program instructions, and when the program instructions stored in memory are executed by the processor, the video encoding / decoding method described in any one of the first embodiments is realized.

[0043] In an eleventh embodiment, a video encoding and decoding system is provided, comprising a video encoder and a video decoder, wherein the video encoder is configured to perform the video encoding / decoding method described in any one of the first embodiments, and the video decoder is configured to perform the video encoding / decoding method described in any one of the first embodiments.

[0044] The technical effects of any of the implementation methods from the second to the eleventh embodiment can be explained by referring to the technical effects of the corresponding implementation methods in the first embodiment or the embodiments for carrying out the invention described below, and therefore, no explanation is provided here. [Brief explanation of the drawing]

[0045] The drawings described herein are provided to further illustrate the present invention and constitute part of it. Schematic embodiments and descriptions thereof are for interpretation purposes only and do not constitute an unwarranted limitation of the present invention. [Figure 1] Figure 1 is a system architecture diagram of a video encoding and decoding system according to an embodiment of the present invention. [Figure 2] Figure 2 is a schematic diagram of a video encoder according to an embodiment of the present invention. [Figure 3] Figure 3 is a schematic diagram of a video decoder according to an embodiment of the present invention. [Figure 4] Figure 4 is a flowchart of a video encoding and decoding method according to an embodiment of the present invention. [Figure 5] Figure 5 is a schematic diagram of a video encoding and decoding device according to an embodiment of the present invention. [Figure 6] Figure 6 is a flowchart of a video encoding / decoding method according to an embodiment of the present invention. [Figure 7] Figure 7 is a schematic diagram of a method for calculating the angular gradient according to an embodiment of the present invention. [Figure 8] Figure 8 is a schematic diagram of another method for calculating the angular gradient according to an embodiment of the present invention. [Figure 9]Figure 9 is a schematic diagram of yet another method for calculating the angular gradient according to an embodiment of the present invention. [Figure 10] Figure 10 is a schematic diagram of a method for dividing an image block and calculating an angular gradient according to an embodiment of the present invention. [Figure 11] Figure 11 is a schematic diagram showing the relationship between a dynamic threshold and an absolute threshold according to an embodiment of the present invention. [Figure 12a] Figure 12a shows a function graph of the reference quantization parameter, complexity information, and buffer region state according to an embodiment of the present invention. [Figure 12b] Figure 12b is a function graph of the reference quantization parameter and buffer region state according to an embodiment of the present invention. [Figure 12c] Figure 12c is a function graph of the reference quantization parameter and complexity information according to an embodiment of the present invention. [Figure 13a] Figure 13a is a schematic diagram of the image boundary according to an embodiment of the present invention. [Figure 13b] Figure 13b is a schematic diagram of a slice according to an embodiment of the present invention. [Figure 14a] Figure 14a is a flowchart of a code stream grouping method on the encoding side according to an embodiment of the present invention. [Figure 14b] Figure 14b is a flowchart of the code stream grouping method on the decoding side according to an embodiment of the present invention. [Figure 15] Figure 15 is a schematic diagram of chunk interleaving by the code stream grouping method according to an embodiment of the present invention. [Figure 16a] Figure 16a is a flowchart of a code stream grouping method on the encoding side according to an embodiment of the present invention. [Figure 16b] Figure 16b is a flowchart of the code stream grouping method on the decoding side according to an embodiment of the present invention. [Figure 17] Figure 17 is a schematic diagram of chunk interleaving by the code stream grouping method according to an embodiment of the present invention. [Figure 18]Figure 18 is a schematic diagram of a video encoding / decoding device according to an embodiment of the present invention. [Modes for carrying out the invention]

[0046] To further clarify the object, technical concept, and advantages of the present invention, the present invention will be described in more detail below with reference to the drawings and examples. It will be clear that the examples described are merely some examples of the present invention, and not all examples. All other examples that can be obtained by those skilled in the art based on the examples of the present invention are all within the scope of protection of the present invention.

[0047] In the description of this invention, unless otherwise specified, " / " means "or," for example, A / B means A or B. In this specification, "and / or" merely describes the correlation between related objects and indicates that there may be three relationships. For example, A and / or B may mean that A exists alone, that A and B exist simultaneously, and that B exists alone. Also, "at least one" means one or more, and "multiple" means two or more. Terms such as "first" and "second" do not limit the quantity or order of execution. Furthermore, terms such as "first" and "second" do not necessarily limit them to being different.

[0048] In this specification, terms such as “exemplary” or “for example” are used to indicate an example, illustration, or explanation. No embodiment or design described as “exemplary” or “for example” in the present invention should be construed as being preferable or advantageous to other embodiments or designs. More precisely, the use of terms such as “exemplary” or “for example” is intended to specifically illustrate the relevant concepts.

[0049] First, we will introduce the technical terms used in the embodiments of the present invention.

[0050] 1. Video encoding / decoding technology

[0051] Video encoding / decoding technology includes video encoding technology and video decoding technology, and may be collectively referred to as video encoding and decoding technology.

[0052] Among these, video sequences contain a range of redundant information, including spatial redundancy, temporal redundancy, visual redundancy, information entropy redundancy, structural redundancy, knowledge redundancy, and importance redundancy. To remove as much redundant information as possible from video sequences and reduce the amount of data representing the video, video coding techniques have been proposed to achieve the effects of reducing storage space and saving transmission bandwidth. Video coding techniques are also called video compression techniques.

[0053] To retrieve data stored or transmitted based on the above video compression technology, it is necessary to implement a corresponding video decoding technology.

[0054] To the extent that it is internationally recognized, video compression coding standards are intended to standardize video coding and decoding methods. Examples include Advanced Video Coding (AVC) in Part 10 of the MPEG-2 and MPEG-4 standards developed by the Motion Picture Experts Group (MPEG), and H.263, H.264, and H.265 (also known as High Efficiency Video Coding standard, HEVC) developed by the International Telecommunication Union Telecommunication Standardization Sector (ITU-T).

[0055] Furthermore, coding algorithms based on a hybrid coding architecture may use a combination of the above-mentioned compression coding methods.

[0056] The basic processing unit in video encoding and decoding is the image block, which is obtained by dividing one frame / image on the encoding side. Typically, one row at a time is processed from the resulting image block. The image block being processed is called the current block, and the processed image block is called the encoded image block, the decoded image block, or the encoded / decoded image block. Taking HEVC as an example, HEVC defines a Coding Tree Unit (CTU), a Coding Unit (CU), a Prediction Unit (PU), and a Transform Unit (TU). CTU, CU, PU, ​​and TU can all be divided image blocks. Of these, both PU and TU are divided based on the CU.

[0057] 2. Video sampling

[0058] Since a pixel is the smallest complete sample of a video or image, data processing for an image block is performed on a pixel-by-pixel basis. Each pixel records color information. One sampling method is to represent color using RGB, which includes three image channels: R represents red, G represents green, and B represents blue. Another sampling method is to represent color using YUV, which also includes three image channels: Y represents luminance, U represents the first chromaticity Cb, and V represents the second chromaticity Cr. Because humans are more sensitive to luminance than chromaticity, memory space is reduced by storing more data indicating luminance and less data indicating chromaticity. Specifically, in video encoding and decoding, video sampling is usually performed using the YUV format, which includes formats such as 420-sampling and 422-sampling formats. This sampling format determines the number of samples for the two chromaticities based on the number of luminance samples. For example, assuming that one CU is a 4x2 pixel, the format would be as follows: The sampling format of TIFF0007847674000001.tif14116420 indicates that sampling is performed in the YUV4:2:0 format. That is, luminance and the first or second chromaticity are selected in a 4:2 ratio, with the first and second chromaticities being selected alternately in every other row. In this case, the above CU sampling selects luminances Y0-Y3 and first chromaticities U0, U2 from the first row, and luminances Y4-Y7 and second chromaticities V4, V6 from the second row. After sampling, the CU consists of a luminance coding unit and a chromaticity coding unit. Of these, the luminance coding unit is: It is TIFF0007847674000002.tif1438, The first chromaticity coding unit is: It is TIFF0007847674000003.tif822, The second chromaticity coding unit is, The filename is TIFF0007847674000004.tif822.

[0059] As can be seen, the size of the image blocks sampled using the above sampling format has changed. Of these, the block size of the luminance coding unit remains unchanged at 4x2, but the block size of the first chromaticity coding unit becomes 2x1, and the block size of the second chromaticity coding unit also becomes 2x1. Therefore, if we assume the size of the CU is XxY, the block size of the chromaticity coding unit sampled based on the 420 sampling format will be X / 2xY / 2.

[0060] Similarly, the 422 sampling format indicates that sampling is performed in the YUV4:2:2 format. That is, luminance, first chromaticity, and second chromaticity are selected in a 4:2:2 ratio. In this case, the above CU sampled luminance coding unit is: The filename is TIFF0007847674000005.tif1440, The first chromaticity coding unit is: It is TIFF0007847674000006.tif1421, The second chromaticity coding unit is, The file is TIFF0007847674000007.tif1523.

[0061] Of these, the block size of the luminance coding unit remains unchanged at 4x2, but the block size of the first chromaticity coding unit becomes 2x2, and the block size of the second chromaticity coding unit also becomes 2x2. Therefore, if we assume the size of the CU is XxY, the block size of the chromaticity coding unit sampled based on the 422 sampling format will be X / 2xY.

[0062] The sampled luminance coding unit, the first chromaticity coding unit, and the second chromaticity coding unit described above are used as data units for each channel for subsequent processing of the current block.

[0063] The encoding / decoding method according to the present invention is applied to a video encoding and decoding system. This video encoding and decoding system is also called a video encoding / decoding system. Figure 1 shows the structure of the video encoding and decoding system.

[0064] As shown in Figure 1, the video encoding and decoding system includes a source device 10 and a destination device 11. The source device 10 generates encoded video data. The source device 10 is also called a video encoding device. The destination device 11 decodes the encoded video data generated by the source device 10. The destination device 11 is also called a video decoding device. The source device 10 and / or the destination device 11 include at least one processor and memory coupled to the at least one processor. The memory includes, but is not limited to, read-only memory (ROM), random access memory (RAM), electrically erasable programmable read-only memory (EEPROM), flash memory, or any other medium for storing desired program code in the form of computer-accessible instructions or data structures. The present invention is not particularly limited thereto.

[0065] The source device 10 and destination device 11 may include a variety of devices. For example, they may include electronic devices such as desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet personal computers, set-top boxes, telephone handsets such as so-called "smartphones," televisions, cameras, display devices, digital media players, video game consoles, in-car computers or similar devices.

[0066] The destination device 11 receives encoded video data from the source device 10 via link 12. Link 12 may include one or more media and / or devices on which the encoded video data can be transmitted from the source device 10 to the destination device 11. In one example, link 12 may include one or more communication media that enable the source device 10 to transmit the encoded video data directly to the destination device 11 in real time. In this example, the source device 10 modulates the encoded video data according to a communication standard (e.g., a wireless communication protocol) and transmits the modulated video data to the destination device 11. The one or more communication media may include, for example, a radio frequency (RF) spectrum or one or more physical transmission lines, or wireless and / or wired communication media. The one or more communication media may form part of a packet-based network, such as a local area network, a wide area network, or a global network (e.g., the Internet). The above-mentioned communication media include routers, switches, base stations, or other devices that enable communication from source device 10 to destination device 11.

[0067] In another example, the encoded video data is output from the output interface 103 to the storage device 13. Similarly, the encoded video data is accessed from the storage device 13 via the input interface 113. The storage device 13 may include various locally accessible data storage media. Examples include Blu-ray discs, Digital Video Discs (DVDs), Compact Disc Read-Only Memory (CD-ROMs), flash memory, or other suitable digital storage media for storing encoded video data.

[0068] In another example, storage device 13 may correspond to a file server or another intermediate storage device for storing encoded video data generated by source device 10. In this example, destination device 11 retrieves the video data stored in storage device 13 from storage device 13 by streaming or downloading. The file server may be any type of server that stores encoded video data and can transmit encoded video data to destination device 11. For example, the file server may include a World Wide Web server (e.g., for websites), a File Transfer Protocol (FTP) server, a Network Attached Storage (NAS) device, and a local disk drive.

[0069] The destination device 11 accesses the encoded video data via any standard data connection (e.g., an internet connection). Examples of data connections include wireless channels, wired connections (e.g., cable modems), or a combination of both, suitable for accessing the encoded video data stored on the file server. The method of transmitting the encoded video data from the file server may be streaming, downloading, or a combination of both.

[0070] The encoding / decoding method of the present invention is not limited to wireless application scenarios. Exemplarily, the encoding / decoding method of the present invention applies to encoding and decoding video to support a variety of multimedia applications, such as wireless television broadcasting, wired television transmission, satellite television transmission, streaming video transmission (e.g., over the Internet), encoding video data stored on a data storage medium, decoding video data stored on a data storage medium, or other applications. In some examples, the video encoding and decoding system is configured to support unidirectional or bidirectional video transmission to support applications such as video streaming, video playback, video broadcasting, and / or video phone calls.

[0071] Figure 1 is a system architecture diagram of a video encoding and decoding system according to an embodiment of the present invention. Figure 1 is merely one example of a video encoding and decoding system and does not limit the video encoding and decoding systems in the present invention. The encoding / decoding method according to the present invention can also be applied to scenarios where there is no data between the encoding device and the decoding device. In other examples, the video data to be encoded or the encoded video data may be retrieved from local memory or streamed over a network. The video encoding device may encode the video data to be encoded and store the encoded video data in memory. The video decoding device may retrieve the encoded video data from memory and decode the encoded video data.

[0072] In Figure 1, the source device 10 includes a video source 101, a video encoder 102, and an output interface 103. In some examples, the output interface 103 may include a modulator / demodulator (modem) and / or transmitter. The video source 101 may include a video capture device (e.g., a video camera), a video archive containing previously captured video data, a video input interface for receiving video data from a video content provider, and / or a computer graphics system for generating video data, or a combination of these video data sources.

[0073] The video encoder 102 encodes video data from the video source 101. In some examples, the source device 10 transmits the encoded video data directly to the destination device 11 via the output interface 103. In some other examples, the encoded video data may be stored in a storage device 13 so that it can be later accessed by the destination device 11 for decoding and / or playback.

[0074] In the example shown in Figure 1, the destination device 11 includes a display device 111, a video decoder 112, and an input interface 113. In some examples, the input interface 113 includes a receiver and / or modem. The input interface 113 receives encoded video data via link 12 and / or from storage device 13. The display device 111 may be integrated with the destination device 11 or installed outside the destination device 11. Generally, the display device 111 displays the decoded video data. The display device 111 may include various types of displays, such as liquid crystal displays, plasma displays, organic light-emitting diode displays, or other types of displays.

[0075] Selectively, the video encoder 102 and video decoder 112 are integrated with an audio encoder and audio decoder, respectively, and include appropriate multiplexer / demultiplexer units or other hardware and software to handle the encoding of both audio and video in a common data stream or separate data streams.

[0076] The video encoder 102 and video decoder 112 may include at least one microprocessor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), discrete logic, hardware, or any combination thereof. When the encoding / decoding method according to the present invention is implemented by software, the present invention is carried out by storing the instructions used in the software in a suitable non-volatile computer-readable storage medium and executing the instructions using at least one processor.

[0077] The video encoder 102 and video decoder 112 in this invention may operate according to a video compression standard (e.g., HEVC) or according to other industry standards. The present invention is not particularly limited.

[0078] Figure 2 is a schematic diagram of a video encoder 102 according to an embodiment of the present invention. In the video encoder 102, the prediction module 21, the transformation module 22, the quantization module 23, and the entropy coding module 24 perform the processes of prediction, transformation, quantization, and entropy coding, respectively. The video encoder 102 further includes a preprocessing module 20 and an adder 202, of which the preprocessing module 20 includes a splitting module and a coding rate control module. For the reconstruction of video blocks, the video encoder 102 further includes an inverse quantization module 25, an inverse transformation module 26, an adder 201, and a reference image memory 27.

[0079] As shown in Figure 2, the video encoder 102 receives video data. The preprocessing module 20 is for obtaining input parameters of the video data. These input parameters include information such as the resolution of the image in the video data, the sampling format of the image, the pixel depth (bits per pixel, bpp), and the bit width. bpp refers to the number of bits occupied by one pixel component in a unit pixel. Bit width refers to the number of bits occupied by a unit pixel. For example, if one pixel is represented by the values ​​of three pixel components, RGB, and each pixel component occupies 8 bits, then the pixel depth of that pixel is 8, and the bit width of that pixel is 3 × 8 = 24 bits.

[0080] The partitioning module in the preprocessing module 20 divides the image into its original blocks. This partitioning may include dividing into slices, image blocks, or other relatively large units, and (for example) partitioning video blocks based on a quadtree structure of Largest Coding Units (LCUs) and CUs. Exemplarily, the video encoder 102 is an assembly for encoding video blocks located in the video slice to be encoded. Generally, a slice can be divided into multiple original blocks (and may be divided into a set of original blocks called image blocks). Typically, the partitioning module determines the sizes of the CUs, PUs, and TUs. The partitioning module is also used to determine the size of the coding rate control unit. The coding rate control unit refers to the basic processing unit in the coding rate control module. The coding rate control unit is for calculating the quantization parameters of the current block. For example, the coding rate control module calculates complexity information for the current block using the coding rate control unit, and further calculates the quantization parameters of the current block based on the complexity information. The partitioning strategy of the partitioning module may be preset or it may be continuously adjusted based on the image during the encoding process. If the partitioning strategy is a preset strategy, the decoding side also presets the same partitioning strategy, thereby obtaining the same image processing unit. This image processing unit is one of the image blocks described above and corresponds one-to-one with the encoding side. If the partitioning strategy is continuously adjusted based on the image during the encoding process, the partitioning strategy is incorporated directly or indirectly into the code stream. Correspondingly, the decoding side obtains the corresponding parameters from the code stream, obtaining the same partitioning strategy and the same image processing unit.

[0081] The coding rate control module in the preprocessing module 20 is used to generate quantization parameters so that the quantization module 23 and the inverse quantization module 25 can perform the relevant calculations. The coding rate control module may, in the process of calculating the quantization parameters, acquire and calculate image information of the current block, such as the input information described above, or acquire and calculate reconstructed values ​​reconstructed by the adder 201, but the present invention is not limited to these.

[0082] The prediction module 21 provides a prediction block to the adder 202 to generate a residual block, and provides the prediction block to the adder 201 to obtain a reconstructed block through reconstruction, which is used as a reference pixel for later prediction. The video encoder 102 generates a pixel difference by subtracting the pixel values ​​of the prediction block from the pixel values ​​of the original block. This pixel difference is a residual block, and the data in this residual block may include luminance difference and chromaticity difference. The adder 201 represents one or more assemblies that perform this subtraction. The prediction module 21 may send the relevant syntax elements to the entropy coding module 24 for merging into the code stream.

[0083] The transformation module 22 divides the residual block into one or more TUs and performs the transformation. The transformation module 22 transforms the residual block from the pixel domain to the transformation domain (e.g., the frequency domain). For example, it uses a discrete cosine transform (DCT) or a discrete sine transform (DST) to transform the residual block and obtain transformation coefficients. The transformation module 22 transmits the obtained transformation coefficients to the quantization module 23.

[0084] The quantization module 23 performs quantization using quantization units, which may be the same as the CU, TU, and PU described above, or they may be further divided in the partitioning module. The quantization module 23 quantizes the transformation coefficients to obtain quantization coefficients by further reducing the encoded bits. The quantization process may reduce the bit depth associated with some or all of the coefficients. The degree of quantization can be changed by adjusting the quantization parameters. In some possible embodiments, the quantization module 23 may then perform a scan of the matrix containing the quantized transformation coefficients. Alternatively, the entropy coding module 24 may perform the scan.

[0085] After quantization, the entropy coding module 24 entropy codes the quantization coefficients. For example, the entropy coding module 24 may perform context-adaptive variable-length coding (CAVLC), context-based adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic decoding (SBAC), stochastic interval partitioning entropy (PIPE) decoding, or another entropy coding method or technique. After entropy coding is performed by the entropy coding module 24, the code stream is obtained and either transmitted to the video decoder 112 or archived for subsequent transmission or retrieval by the video decoder 112.

[0086] The inverse quantization module 25 and the inverse transformation module 26 apply inverse quantization and inverse transformation, respectively. The adder 201 adds the inversely transformed residual block and the predicted residual block to generate a reconstructed block, which is later used as a reference pixel to predict the original block. The reconstructed block is stored in the reference image memory 27.

[0087] Figure 3 is a schematic diagram of a video decoder 112 according to an embodiment of the present invention. As shown in Figure 3, the video decoder 112 includes an entropy decoding module 30, a prediction module 31, an inverse quantization module 32, an inverse transform module 33, an adder 301, and a reference image memory 34. Of these, the entropy decoding module 30 includes an analysis module and a coding rate control module. In some possible embodiments, the video decoder 112 performs a decoding flow that is exemplary inverse to the coding flow described for the video encoder 102 in Figure 2.

[0088] In the decoding process, the video decoder 112 receives the encoded video code stream from the video encoder 102. The analysis module in the entropy decoding module 30 of the video decoder 112 performs entropy decoding on the code stream to generate quantization coefficients and syntax elements. The entropy decoding module 30 transmits the syntax elements to the prediction module 31. The video decoder 112 receives the syntax elements at the video slice level and / or video block level.

[0089] In the entropy decoding module 30, the coding rate control module generates quantization parameters based on the information of the image to be decoded obtained by the analysis module, so that the inverse quantization module 32 performs correlation calculations. The coding rate control module may also calculate the quantization parameters based on the reconstruction blocks reconstructed by the adder 301.

[0090] The inverse quantization module 32 performs inverse quantization (e.g., dequantization) on the quantization coefficients and generated quantization parameters provided from the code stream and decoded by the entropy decoding module 30. The inverse quantization process may include a process of determining the degree of quantization using quantization parameters calculated by the video encoder 102 for each video block in the video slice. Similarly, the inverse quantization process may include a process of determining the degree of application of inverse quantization. The inverse transform module 33 applies an inverse transform (e.g., a transform method such as DCT, DST, etc.) to the transformed coefficients after inverse quantization and inverse transforms the transformed coefficients after inverse quantization to obtain an inverse transform unit, i.e., a residual block. The size of the inverse transform unit may be the same as the size of the TU. The inverse transform method and transform method employ the corresponding forward and inverse transforms in the same transform method. For example, the inverse transform of DCT, DST is an inverse DCT, inverse DST, or a conceptually similar inverse transform process.

[0091] After the prediction module 31 generates prediction blocks, the video decoder 112 forms decoded image blocks by adding the prediction blocks to the residual blocks from the inverse transformation module 33. The adder 301 represents one or more assemblies that perform this addition operation. If necessary, a deblocking filter may be used to filter the image of the decoded blocks to remove block artifacts. The decoded image blocks in a given frame or image are stored in the reference image memory 34 as reference pixels for later prediction.

[0092] The present invention provides a possible video encoding / decoding implementation method as shown in Figure 4. Figure 4 is a flowchart of the video encoding and decoding method according to the present invention. The video encoding / decoding implementation method includes processes 1 to 5. Processes 1 to 5 may be performed by one or more of the above source device 10, video encoder 102, destination device 11, or video decoder 112.

[0093] The following explains processes 1 through 5 above, using the video encoding process as an example.

[0094] Process 1: Divide the image of one frame into one or more parallel encoding units that do not overlap with each other. There are no dependencies between these one or more parallel encoding units, and encoding and decoding are performed completely in parallel / independently, as shown in parallel encoding unit 1 and parallel encoding unit 2 in Figure 4.

[0095] Process 2: Each parallel coding unit may be further divided into one or more independent coding units that do not overlap with each other. The independent coding units are independent of each other, but may share header information from some of the parallel coding units.

[0096] An independent coding unit may contain three components: luminance Y, first chromaticity Cb, and second chromaticity Cr, or three components of RGB, or only one of these components. If an independent coding unit contains three components, the sizes of these three components may be exactly the same or different, specifically relating to the image input format. The independent coding unit can also be understood as one or more processing units composed of N channels contained in each parallel coding unit. For example, the three components Y, Cb, and Cr are the three channels that constitute the parallel coding unit, each of which may be an independent coding unit. Alternatively, if Cb and Cr are collectively referred to as chromaticity channels, the parallel coding unit may include an independent coding unit composed of luminance channels and an independent coding unit composed of chromaticity channels.

[0097] Process 3: Each independent coding unit may be further divided into one or more non-overlapping coding units. Each coding unit within an independent coding unit may be dependent on one another. For example, multiple coding units may refer to each other to perform pre-coding and pre-decoding.

[0098] If the size of the coding unit and the independent coding unit are the same (i.e., the independent coding unit is divided into only one coding unit), then the size may be any of the sizes described in process 2.

[0099] The encoding unit may contain three components (or the three components of RGB): luminance Y, first chromaticity Cb, and second chromaticity Cr, or it may contain only one of these components. If it contains three components, the sizes of these components may be exactly the same or different, specifically related to the image input format.

[0100] It should be noted that Process 3 is one optional step in the video encoding and decoding method, and the video encoder / decoder may perform encoding / decoding on the residual coefficients (or residual values) of the independent encoding units obtained in Process 2.

[0101] Process 4: The coding unit may be further divided into one or more non-overlapping prediction groups (PGs). A PG may be abbreviated as a Group. Each PG performs coding and decoding according to the selected prediction mode to obtain the predicted value of the PG, which constitutes the predicted value of the entire coding unit, and the residual value of the coding unit is obtained based on the predicted value and the original value of the coding unit. For example, in Figure 4, one coding unit in an independent coding unit is divided into PG-1, PG-2, and PG-3.

[0102] Process 5: Based on the residual values ​​of the encoded units, the encoded units are grouped to obtain one or more non-overlapping residual blocks (RBs). The residual coefficients of each RB are encoded and decoded according to the selected mode to form a residual coefficient stream. Specifically, there are two types: one in which a transformation is performed on the residual coefficients, and one in which no transformation is performed. As shown in Figure 4, one encoded unit is grouped to obtain RB-1 and RB-2.

[0103] Of these, the selection modes for the encoding and decoding methods for residual coefficients in process 5 may include, but are not limited to, semi-fixed-length encoding, exponential Golomb encoding, Golomb-Rice encoding, truncated unary encoding, run-length encoding, and direct encoding of the original residual values. For example, if the exponential Golomb encoding method is selected to encode the residual coefficients of each RB, then when decoding the residual coefficients of each RB, it is necessary to select a decoding method corresponding to the exponential Golomb encoding method and decode accordingly.

[0104] For example, a video encoder directly encodes the coefficients in RB.

[0105] Furthermore, for example, the video encoder may perform a transformation on the residual block, such as DCT, DST, or Hadamard transform, and then encode the transformed coefficients.

[0106] As a possible example, if RB is small, the video encoder may directly uniformly quantize each coefficient in RB and then perform binary coding. If RB is large, it may be further divided into multiple coefficient groups (CG), each CG may be uniformly quantized, and then binary coding may be performed. In some embodiments of the present invention, the sizes of the coefficient groups (CG) and the quantization groups (QG) may be the same.

[0107] The following provides an illustrative explanation of how residual coefficients are encoded using a semi-fixed-length coding scheme. First, the maximum absolute value of residuals in a given RB is defined as the modified maximum (mm). Next, the number of bits used to encode the residual coefficients in that RB (the number of bits used to encode residual coefficients in the same RB must match) is determined by the above mm. For example, if the code length (CL) of the current RB is 2 and the current residual coefficient is 1, then 2 bits are needed to encode a residual coefficient of 1, which is represented as 01. In a special case, if the CL of the current RB is 7, it means encoding 8-bit residual coefficients and 1-bit sign bit. The method for determining the CL is to find the smallest M value that satisfies the condition that all residuals in the current RB are within the range [-2^(M-1), 2^(M-1)], and the found M is taken as the CL of the current RB. If the current RB contains both boundary values ​​-2^(M-1) and 2^(M-1), then M should be increased by 1; that is, M+1 bits are needed to encode all residuals in the current RB. If the current RB contains only one of the two boundary values ​​-2^(M-1) and 2^(M-1), then the trailing bit needs to be encoded to determine whether that boundary value is -2^(M-1) or 2^(M-1). If the current RB contains neither -2^(M-1) nor 2^(M-1) in any of the residuals, then the trailing bit does not need to be encoded.

[0108] In special circumstances, the video encoder may directly encode the original values ​​of the image instead of the residual values.

[0109] The video encoder 102 and video decoder 112 described above may be implemented by other means, for example, by using a general-purpose digital processor system. Figure 5 provides a schematic configuration diagram of a video encoding and decoding device. As shown in Figure 5, the encoding and decoding device 50 may be a part of the video encoder 102 or a part of the video decoder 112.

[0110] The encoding and decoding device 50 may be applied to the encoding side or to the decoding side. The encoding and decoding device 50 includes a processor 501 and a memory 502. The processor 501 is connected to the memory 502 (for example, connected to each other via a bus 504). Optionally, the encoding and decoding device 50 may further include a communication interface 503. The communication interface 503 is connected to the processor 501 and the memory 502 and is used to send and receive data.

[0111] The memory 502 may be Random Access Memory (RAM), Read-Only Memory (ROM), Erasable Programmable Read-Only Memory (EPROM), or Compact Disc Read-Only Memory (CD-ROM). The memory 502 is for storing the associated program code and video data.

[0112] The processor 501 may be one or more central processing units (CPUs), such as CPU 0 and CPU 1 shown in Figure 5. If the processor 501 is a single CPU, it may be a single-core CPU or a multi-core CPU.

[0113] The processor 501 reads program code stored in memory 502 and executes the operations of any embodiment corresponding to Figure 6 and various executable embodiments thereof.

[0114] The encoding / decoding method according to the present invention may be applied to a video encoder 102 or a video decoder 112. For example, in some cases, the video encoder 102 may not perform encoding using the encoding / decoding method of the present invention and may not transmit quantization parameter information to the video decoder 112, in which case the video decoder 112 may perform decoding using the encoding / decoding method according to the present invention. In other cases, the video encoder 102 may perform encoding using the encoding / decoding method of the present invention and transmit quantization parameter information to the video decoder 112, in which case the video decoder 112 may obtain quantization parameter information from the code stream and perform decoding.

[0115] The encoding / decoding method according to the present invention will be described in detail below, combining the video encoding and decoding system shown in Figure 1, the video encoder 102 shown in Figure 2, and the video decoder 112 shown in Figure 3.

[0116] As shown in Figure 6, Figure 6 is a flowchart of the video encoding / decoding method according to the present invention. This method includes S601.

[0117] In S601, the video encoding / decoding device acquires complexity information of the current block in the image to be processed. This complexity information represents the degree of difference in the pixel values ​​of the current block, and is obtained by calculating at least one angular gradient of the current block based on at least the pixel values ​​of the current block.

[0118] It is generally understood that the information in an image block is represented by the pixel points contained within that image block. If the difference in pixel values ​​between each pixel point in an image block is small, i.e., low complexity, then the image block is considered relatively simple if it indicates that the color changes in that image block are small. Similarly, if the difference in pixel values ​​between each pixel point in an image block is large, i.e., high complexity, then the image block is considered relatively complex if it indicates that the color changes in that image block are large.

[0119] Specifically, the current block complexity information (block_complexity) is obtained by calculating at least one angular gradient of the current block based on at least the pixel values ​​of the current block. The angular gradient of the current block refers to calculating the difference in pixel values ​​of the current block based on a certain angular gradient direction. Angular gradients include horizontal gradients, vertical gradients, and other angular gradients.

[0120] The current block's horizontal gradient refers to the set of differences between the pixel values ​​in column t and column t-1 in the current block, calculated based on the horizontal gradient direction (leftward or rightward). Here, t is an integer greater than 1. The formula is as follows:

[0121] Horizontal gradient H = Pixel value in column t - Pixel value in column t-1

[0122] Figure 7 provides a schematic diagram of a method for calculating the angular gradient. As shown in Figure 7(a), using a 4x2 image block as an example, the above formula is used to calculate the difference for the image block according to the direction shown in the figure, yielding a 3x2 difference. In this case, the horizontal gradient of the image block is the sum of these 3x2 differences.

[0123] The horizontal complexity is calculated based on the horizontal gradient described above using the following formula. Hereafter, `complexity_hor` will represent the horizontal complexity. complexity_hor = sum of elements in the horizontal gradient (gradH) / number of elements (grad_block_size)

[0124] Combining this with the example above, if a 3x2 difference is obtained, the current block's complexity_hor is the sum of 6 differences divided by 6. Among these, the elements in the horizontal gradient above may be each horizontal gradient calculated for the image block according to the horizontal direction.

[0125] Similarly, the vertical gradient of the current block refers to the set of differences between the pixel value in row s and the pixel value in row s-1 of the current block, calculated based on the vertical gradient direction (up or down), where s is an integer greater than 1. The formula is as follows:

[0126] Vertical gradient V = pixel value in row s - pixel value in row s - 1

[0127] As shown in Figure 7(b), using a 4x2 image block as an example, a 4x1 difference is obtained by calculating the difference for the image block according to the direction shown in the figure using the above formula. In this case, the horizontal slope of the image block is the sum of the above 4x1 differences.

[0128] The vertical complexity is calculated based on the vertical gradient described above using the following formula. Hereafter, `complexity_ver` will represent the vertical complexity. complexity_ver = sum of elements in the vertical gradient (gradV) / number of elements (grad_block_size)

[0129] When combined with the above example, it is the sum of the differences of complexity_ver=4 divided by 4. Among these, the elements in the vertical gradient above may be each vertical gradient calculated for the image block according to the vertical direction.

[0130] Similarly, other angular gradients in the current block may include 45°, 135°, 225°, or 315° gradients. Refer to the schematic diagram of another method for calculating angular gradients provided in Figure 8, and the directions of the other angular gradients mentioned above are shown in Figures 8(a) to 8(d), respectively.

[0131] Calculating complexity information for the current block based on the pixel values ​​of the current block using the method described above is advantageous for more accurately determining encoding / decoding parameters, such as quantization parameters, for that current block. This improves the quality of the video encoded / decoded image and enhances the image encoding / decoding efficiency.

[0132] The following describes implementations for determining the complexity information of the current block using the following four possible implementations.

[0133] In the above scheme, the number of elements included in the gradient of the current block is less than the number of pixels in the current block. Considering a more accurate method for calculating the complexity of the current block, the video encoding / decoding device may determine the complexity information of the current block by referring to the reconstructed value of the encoded / decoded pixel values ​​of the current block.

[0134] In the first possible implementation, the video encoding / decoding device calculates at least one angular gradient of the current block based on the pixel value of the current block and the reconstructed value of the encoded / decoded pixel value of the current block, and obtains complexity information of the current block based on at least one angular gradient of the current block. Refer to the schematic diagrams of other methods for calculating the angular gradient provided in Figure 9, as shown in Figures 9 a) to f), where the blank areas represent the original pixels, i.e., the pixel values ​​of the current block, and the shaded areas represent the reconstructed pixels, i.e., the reconstructed values ​​of the encoded / decoded pixel values ​​of the current block. Figure 9 a) is a schematic diagram of a configuration in which the gradient of the current block is calculated row by row based on the pixels in the current block and the reconstructed values ​​of the edge pixel values ​​in the direction of the pixel gradient. In this way, for each original pixel, the reconstructed value of the corresponding edge pixel value or other original pixel values ​​in the current block are searched along the gradient direction of the current block so as to calculate the elements in the gradient direction contained in the current block by the method described above. For example, the shadow pixels in the first row are marked from left to right as pixel 1-1, pixel 1-2...pixel 1-16, the pixels in the second row are marked from left to right as pixel 2-1, pixel 2-2...pixel 2-17, and the pixels in the third row are marked from left to right as pixel 3-1, pixel 3-2...pixel 3-16. The calculated gradient of the current block may include the gradient obtained by subtracting pixel 1-1 from pixel 2-1, the gradient obtained by subtracting pixel 1-2 from pixel 2-2...the gradient obtained by subtracting pixel 1-16 from pixel 2-16, the gradient obtained by subtracting pixel 2-2 from pixel 3-1, the gradient obtained by subtracting pixel 2-3 from pixel 3-2...the gradient obtained by subtracting pixel 2-17 from pixel 3-16. In this way, 32 gradient values ​​are obtained for the original 2x16 pixels. Similarly, the method for calculating the angular gradient in the gradient direction shown in Figures 9b) to f) may be referred to the method for calculating the gradient shown in Figure 9a) above. The only difference is that the position and angular direction of the pixels used are different, but a detailed explanation is omitted here.

[0135] An implementation method for obtaining complexity information of a block based on at least one angular gradient of the block may be described in the following embodiment, but a detailed explanation is omitted here.

[0136] In this way, for each original pixel, one element is calculated along the same gradient direction, and the number of elements included in each gradient of the current block is equal to the number of pixels in the current block. The above elements represent the change along the gradient direction of the pixel in the current block corresponding to that element. And each element has a one-to-one correspondence with the pixel in the current block. Therefore, since each element uniformly represents the change along the gradient direction of the pixel in the current block, each element obtained based on the above helps to obtain more accurate complexity information of the current block.

[0137] In the second possible implementation, the video encoding / decoding device calculates complexity information for the current block based on the pixel values ​​of the current block and the pixel values ​​adjacent to the current block in the image being processed. The difference from the first possible implementation described above is as follows: In the first possible implementation, the reconstructed value of the encoded / decoded pixel value of the current block, i.e., the reconstructed value, is used, and the reconstructed value is the reconstructed value of the pixel in that current block. On the other hand, in the second possible implementation, the pixel values ​​adjacent to the current block in the image being processed, i.e., values ​​related to the pixels in the blocks adjacent to the current block, are used, and these values ​​may be reconstructed values ​​or original values.

[0138] It can be seen that this is similar to the method of calculating complexity information based on the reconstructed values ​​of the encoded / decoded pixel values ​​of the current block described above. As shown in Figure 9, the blank areas in Figure 9 may be considered as pixels of the 16x2 current block, but the difference is that the shaded areas in Figure 9 may be considered as pixels adjacent to the current block, representing the pixel values ​​of the current block and the adjacent pixel values.

[0139] For example, if the horizontal gradient is calculated using Figure 9e), the current block contains 16 × 2 pixels. The horizontal gradient calculated using the value obtained by subtracting the pixel value of the pixel in the first column of the current block from the reconstructed value of the pixel in the previous column of the current block, the value obtained by subtracting the pixel value of the pixel in the second column of the current block from the reconstructed value of the pixel in the first column of the current block, ... and the value obtained by subtracting the pixel value of the pixel in the 16th column of the current block from the reconstructed value of the pixel in the 15th column of the current block includes a difference of 16 × 2 and is equal to the sum of the differences of the complexity_hor = 32 of the current block divided by 32. A detailed explanation of the specific calculation method is omitted here.

[0140] The above selective method is merely an example, and in practice, multiple reconstructed pixels and adjacent pixels may be selected. For example, the reconstructed pixels are the average of the reconstructed values ​​n rows, n columns, or n pixels prior to the current block, and the adjacent pixels are the average of the pixel values ​​n rows, n columns, or n pixels prior to the current block, but the present invention is not limited to these.

[0141] Selectively, the complexity information of the current block is obtained based on at least one angular gradient of the current block. Specifically, the video encoding / decoding device uses the minimum value among the complexity information obtained based on at least one angular gradient as the complexity information of the current block. In other words, if the complexity information of the current block calculated based on a certain angular gradient is the minimum, then that minimum complexity information is used as the complexity information of the current block.

[0142] In a third possible implementation, the video encoding / decoding process samples the image to be processed in multiple channels, so the current block consists of multiple channel image blocks. If the current block is an N-channel image block, where N is an integer greater than 0, the video encoding / decoding device determines the complexity information of each channel image block based on the pixel values ​​of each channel image block in the above N-channel image block, and then determines the complexity information of the current block based on the complexity information of each channel image block. For example, the minimum value in the complexity information of the N-channel image block is determined as the complexity information of the current block. In this way, the complexity information calculated for a single-channel image block becomes more accurate, which helps to improve the accuracy of determining the complexity information of the current block and the accuracy of video encoding / decoding.

[0143] In one possible implementation, at least one angular gradient can be calculated for each channel image block, one complexity information can be obtained for the channel image block based on each angular gradient, and the complexity information for the channel image block can be determined based on the obtained complexity information, for example, the minimum value of the obtained complexity information can be determined as the complexity information for the channel image block.

[0144] In another possible implementation, the video encoding / decoding device divides each channel image block in a group of channel image blocks that currently constitute a block, and determines the complexity information of each channel image block based on the subblocks obtained after the division of each channel image block. Specifically, the video encoding / decoding device divides each channel image block into at least two subblocks, obtains the complexity information of at least two subblocks of each channel image block, and determines the complexity information of the corresponding channel image block in each channel image block based on the complexity information of at least two subblocks of each channel image block.

[0145] Exemplary, Figure 10 provides a schematic diagram of a method for dividing an image block and calculating the angular gradient. As shown in Figure 10, a 4x2 channel image block in the current block is divided according to the vertical direction, and the angular gradient is further calculated for the divided 8x2 sub-image blocks to obtain complexity information of the channel image block.

[0146] The implementation method for determining the complexity information of at least two subblocks of each channel image block may refer to the implementation method provided in Figure 7 above, but a detailed explanation is omitted here.

[0147] Selectively, the video encoding / decoding device determines the minimum complexity information of at least two subblocks of each channel image block as the complexity information of the corresponding channel image block.

[0148] The division rules for each channel image block relative to the subblocks may be the same or different. The size of each channel image block may vary depending on the image sampling format. For example, if sampling is performed in YUV format according to a 420 sampling format as described above, the block sizes after sampling luminance, first chromaticity, and second chromaticity will differ. Therefore, relatively large channel image blocks may be divided to determine the complexity information of those channel image blocks. Since the complexity information calculated for smaller subblocks in a channel image block is more accurate, the complexity information of the channel image block obtained based on the subblock complexity information is more accurate, which helps improve the accuracy of determining the complexity information of the current block and the accuracy of video encoding / decoding.

[0149] The multi-channel of the present invention is not limited to the three RGB channels described above, but may have more channels. For example, if the image sensor is a 4-channel sensor, the corresponding image to be processed will contain 4 channels of image information. For example, if the image sensor is a 5-channel sensor, the corresponding image to be processed will contain 5 channels of image information.

[0150] The multichannel in the present invention includes, for example, at least one or more of the following: Y channel, U channel, V channel, Co channel, Cg channel, R channel, G channel, B channel, alpha channel, IR channel, D channel, and W channel. For example, the multichannel may include the Y channel, U channel, and V channel. Alternatively, the multichannel may include the R channel, G channel, and B channel. Alternatively, the multichannel may include the R channel, G channel, B channel, and alpha channel. Alternatively, the multichannel may include the R channel, G channel, B channel, and IR channel. Alternatively, the multichannel may include the R channel, G channel, B channel, and W channel. Alternatively, the multichannel may include the R channel, G channel, B channel, IR channel, and W channel. Alternatively, the multichannel may include the R channel, G channel, B channel, and D channel. Alternatively, the multichannel may include the R channel, G channel, B channel, D channel, and W channel. In addition to the RGB color light-sensitive channels, the sensor may also have an IR channel (infrared or near-infrared light-sensitive channel), a D channel (dark channel mainly composed of infrared or near-infrared light), and a W channel (full-color light-sensitive channel). Different sensors may have different channels; for example, the sensor type may be an RGB sensor, RGBIR sensor, RGBW sensor, RGBIRW sensor, RGBD sensor, RGBDW sensor, etc.

[0151] In the fourth possible implementation, the video encoding / decoding device obtains the predicted angle used in the angle prediction mode of the current block, calculates the angle gradient based on the predicted angle to obtain the corresponding complexity information, and uses this corresponding complexity information as the complexity information of the current block.

[0152] Among these, the angle prediction mode is a general prediction mode used to determine the residual between the current block's pixel value and the reconstructed value based on a specified angle. The predicted angle, also called the predicted direction, is similar to the angles related to the other angular gradients mentioned above, such as 45° and 135°.

[0153] An implementation method for calculating the angular gradient based on the predicted angle and obtaining the corresponding complexity information can be found in the implementation method described above, but a detailed explanation is omitted here.

[0154] The above method establishes a correlation between the prediction mode of the current block and the calculation of complexity information, making the calculation of complexity information of the current block more purposeful, improving the accuracy of complexity information, and thus improving the accuracy of video encoding / decoding.

[0155] Selectively, the prediction mode may include multiple prediction modes, such as a DC prediction mode and a Planar prediction mode. The video encoding / decoding device may set up multiple complexity information for the current block in order to determine the complexity information of the current block by the prediction mode. In some cases, the multiple complexity information may be calculated based on the above multiple angular gradients. In other cases, the video encoding / decoding device may have multiple complexity information presets. Among these, there is a correspondence between the preset multiple complexity information and the prediction mode. For example, the multiple complexity information has a first complexity information and a second complexity information, where the first complexity information corresponds to an angular prediction mode (which may include one or more angular prediction modes), and the second complexity information corresponds to a DC prediction mode and a Planar prediction mode. When prediction is made for the current block, it is possible to obtain multiple prediction results based on the multiple prediction modes, and the optimal prediction mode is determined based on the Rate-Distortion Optimized (RDO) cost and set as the prediction mode for the current block. The complexity information of a current block is obtained through the correspondence between the prediction mode, which is determined based on the cost of the RDO, and the preset complexity information. Combining the above example, if the prediction mode of the current block is determined to be DC prediction mode based on the cost of the RDO, then the complexity information of the current block should become the second complexity information according to the above correspondence.

[0156] Selectively, the above prediction modes may be divided into multiple categories, each corresponding to one complexity information. These categories include intra-mode categories, dot prediction mode categories, screen content coding (SCC) mode categories, original value mode categories, and fallback mode categories. For example, the angle prediction mode belongs to the intra-mode category. The categories to which the above prediction modes belong may be further divided depending on whether or not a transformation is performed. For example, the intra-mode category may be divided into intra-mode + transformation category and intra-mode + non-transformation category. The correspondence between complexity information and categories is similar to the above, where one of the first complexity information corresponds to multiple prediction modes. Once the category to which the prediction mode of the current block belongs is determined, the quantization parameters are calculated using the corresponding complexity information.

[0157] Selectively, as can be seen from the third possible implementation described above, the determination of complexity information of the current block based on the angular gradient of at least one of the N-channel image blocks of the current block is achieved by the following steps S11-S12.

[0158] In S11, the video encoding / decoding device obtains complexity information for each channel image block in the N channel image block based on the pixel values ​​of each channel image block, where N is an integer greater than 0.

[0159] In S12, the video encoding / decoding device obtains the complexity information of the current block based on the complexity information of each channel image block.

[0160] As can be seen from the above explanation, for the current block, at least one angular gradient can be obtained, and based on the obtained angular gradient, complexity information for the current block can be obtained. For channel image blocks, at least one angular gradient can be obtained, and based on the obtained angular gradient, complexity information for the channel image block can be obtained. For subblocks of channel image blocks, at least one angular gradient can be obtained, and based on the obtained angular gradient, complexity information for the subblocks can be obtained. In addition to the implementation methods for obtaining complexity information described above, other implementation methods also exist. Below, we will explain using implementation method 1-4 as an example of obtaining complexity information for the current block.

[0161] In implementation method 1, the minimum value of each complexity information calculated based on the angular gradient is used as the complexity information of the current block. The formula is as follows: TIFF0007847674000008.tif11149

[0162] Of these, complexity_45 is complexity information calculated based on a 45° angular gradient, complexity_225 is complexity information calculated based on a 225° angular gradient, complexity_135 is complexity information calculated based on a 135° angular gradient, and complexity_315 is complexity information calculated based on a 315° angular gradient.

[0163] In implementation method 2, the complexity information of the current block is obtained based on the weighted values ​​of each complexity information calculated based on the angular gradient. The formula is as follows: TIFF0007847674000009.tif11149

[0164] Eventually, w v This represents the weighted value corresponding to the vertical slope, w h w represents the weighted value corresponding to the horizontal slope, 225 w represents the weighted value corresponding to a 225° gradient, 315represents the weighting value corresponding to a 315° gradient, where 0 ≤ w v , w h , w 225 , w 315 ≤ 1 and w v + w h + w 225 + w 315 = 1.

[0165] In implementation method 3, the complexity level (complexity_level) of the original image block of each channel is calculated, and the block_complexity of the current block is determined based on the complexity_level of each channel image block. It includes the following steps S21 - S23.

[0166] In step S21, the video encoding / decoding device determines the complexity_level of each channel image block based on the complexity information (complexity) of each channel image block.

[0167] In case 1, considering the subjective model classification of the human eye, A - 1 absolute thresholds are set, and the image blocks are divided into A levels from simple to complex.

[0168] For example, for an 8-bit image block with a sampling format of YUV444, the complexity_level of each channel image block is divided based on the following method, including that complexity ≦ 4 belongs to level 1, 4 < complexity < 16 belongs to level 2, and complexity ≧ 16 belongs to level 3.As another example, for a 10-bit image block with a sampling format of YUV444, the complexity_level of each channel image block is divided based on the following method, including that complexity ≦ 8 belongs to level 1, 8 < complexity < 64 belongs to level 2, and complexity ≧ 64 belongs to level 3.As another example, for a 12-bit image block with a sampling format of YUV444, the complexity_level of each channel image block is divided based on the following method, including that complexity ≦ 16 belongs to level 1, 16 < complexity < 256 belongs to level 2, and complexity ≧ 256 belongs to level 3.

[0169] In case 2, considering the subjective model classification of the human eye, B - 1 absolute thresholds are set, and C dynamic thresholds updated in the image block are installed, and the image block is divided into B + C levels in total.

[0170] FIG. 11 is a schematic diagram of the relationship between the dynamic threshold and the absolute threshold according to an embodiment of the present invention. As shown in FIG. 11, assuming that the absolute threshold 1 is 4 and the absolute threshold 2 is 16, then 0 < dynamic threshold (thread) 1 < 4 and 4 < thread2 < 16. Assume that complexity ≦ thread1 belongs to level 1, thread1 < complexity ≦ 4 belongs to level 2, 4 < complexity ≦ thread2 belongs to level 3, thread2 < complexity ≦ 16 belongs to level 4, and complexity > 16 belongs to level 5. Among them, when the above dynamic threshold is updated with an image block, if the complexity information of the current block is less weighted than the complexity information of some previous image blocks, the dynamic threshold becomes smaller. If the complexity information of the current block is equally weighted as the complexity information of some previous image blocks, the dynamic threshold remains unchanged. If the complexity information of the current block is more weighted than the complexity information of some previous image blocks, the dynamic threshold becomes larger.

[0171] In step S22, the video encoding / decoding device determines the complexity level (block_complexity_level) of the current block based on the complexity_level of each channel image block.

[0172] Regarding the complexity level of each channel image block, a large weight is assigned to the channel to which the human eye is sensitive, and conversely, a small weight is assigned. The complexity levels of each channel image block are weighted to obtain the complexity level of the current block.

[0173] Taking the case where the number of channels of the image block is 3 as an example, the complexity level of the current block is determined by the following formula. TIFF0007847674000010.tif7149

[0174] Among these, w1, w2, or w3 may be 0. w1, w2, and w3 each represent the weights of the complexity level of the 3-channel image block, and complexity_level1, complexity_level2, and complexity_level3 each represent the complexity level of the 3-channel image block. For example, for an image block with a sampling format of YUV444, a large weight can be assigned to the Y channel coefficient, and w1=2, w2=w3=1 can be set.

[0175] In step S23, the video encoding / decoding device represents the complexity information of the current block based on block_complexity_level.

[0176] Steps S21-S23 flexibly adjust the complexity level weights of image blocks from different channels, thereby flexibly adjusting the complexity information of the current block.

[0177] Regarding implementation method 3, the video encoding / decoding device may determine the complexity information of the current block (block_complexity) directly based on the complexity of each channel image block without performing step S21 above. That is, implementation method 4 is as follows.

[0178] In implementation method 4, the video encoding / decoding device directly weights the complexity of each channel image block to obtain complexity information for the current block. The formula is as follows: TIFF0007847674000011.tif8145

[0179] Of these, w4, w5, or w6 may be 0. w4, w5, and w6 each represent the weights of the complexity information for a 3-channel image block, while complexity1, complexity2, and complexity3 each represent the complexity information for a 3-channel image. The weights represented by w4, w5, and w6 may be related to a subjective model of the human eye. For example, a larger weight may be assigned to channels that the human eye is sensitive to.

[0180] Alternatively, based on the obtained block_complexity, the block_complexity may be leveled and the block_complexity_level obtained by step S22 above.

[0181] Furthermore, if an N-channel image block is divided into at least two sub-blocks, the video encoding / decoding device may determine the complexity information of the N-channel image block using the method described above, and then determine the complexity information of the current block.

[0182] Specifically, taking the example of dividing one channel image block into two subblocks, if the complexity information of the two subblocks is sub_complexity1 and sub_complexity2 respectively, then determining the complexity information (complexity1) of the channel image block based on the complexity information of the two subblocks includes the following implementation methods 5 to 7.

[0183] In implementation method 5, the minimum value of the complexity information for each subblock is used as the complexity information for the channel image block. TIFF0007847674000012.tif9108

[0184] In implementation method 6, the complexity information of each subblock is weighted to obtain the complexity information of the channel image block.

[0185] In TIFF0007847674000013.tif8112, w7 and w8 each represent the weights of two subblock complexity information, and 0≦w7≦1 and 0≦w8≦1.

[0186] In implementation method 7, complexity information for the channel image block is determined based on the complexity level of each subblock.

[0187] In (1) and (2) below, the implementation method will be explained using the example that the complexity levels of the two subblocks are sub_complexity_level1 and sub_complexity_level2, respectively, the complexity information of the channel image block is complexity1, and the complexity level of the channel image block is complexity_level1. The method for determining the complexity level of each subblock is to determine the complexity level of each subblock based on the complexity information of each subblock. Specifically, you may refer to the explanation of step S21 above, but the difference is that the names of each subblock and each channel image block are different, but a detailed explanation will be omitted here.

[0188] (1) The complexity level of the channel image block may be determined based on the complexity level of each subblock.

[0189] Alternatively, the complexity level (e.g., complexity_level1) of the channel image block may be obtained by selectively weighting the complexity levels of each subblock (e.g., sub_complexity_level1 and sub_complexity_level2). The formula is as follows:

[0190] In TIFF0007847674000014.tif8148, w9 and w10 represent the weights of the complexity levels of two subblocks, respectively, and 0 ≤ w9 ≤ 1 and 0 ≤ w10 ≤ 1.

[0191] Selectively, the minimum complexity level of each subblock (e.g., sub_complexity_level1 and sub_complexity_level2) is set as the complexity level of the channel image block (e.g., complexity_level1). The formula is as follows: TIFF0007847674000015.tif8133

[0192] (2) Based on the determined block complexity level of the channel image block, the complexity information of the channel image block is represented.

[0193] Specifically, this step is similar to S23, and a detailed explanation will be omitted here.

[0194] The following describes other implementation methods for step S601.

[0195] In some cases, the video encoder 102 may acquire complexity information and transmit it to the video decoder 112. In this case, if the encoding / decoding device is the video decoder 112, the selective implementation of step S601 above includes analyzing the code stream and acquiring complexity information for the current block in the image to be processed from the code stream.

[0196] In S602, the video encoding / decoding unit determines the quantization parameters of the current block based on the complexity information of the current block.

[0197] Selectively, the video encoding / decoding device determines the quantization parameters of the current block based on complexity information, which includes the following steps S31-S32.

[0198] In S31, the video encoding / decoding device determines the reference quantization parameter (ref_qp) of the current block based on the complexity information of the current block.

[0199] In S32, the quantization parameters of the current block are determined based on the reference quantization parameters of the current block.

[0200] Of these, the reference quantization parameter is used to instruct the generation of the quantization parameter.

[0201] Selectively, the video encoding / decoding device obtains the buffer area state of the image to be processed, and the correspondence between the buffer area state and the complexity information of the current block. Based on the correspondence between the buffer area state and the complexity information, it determines the reference quantization parameter of the current block.

[0202] In the video encoding process, the encoding rates of different image blocks vary, causing the output code stream to fluctuate and affecting transmission stability. To address this, a video encoder includes a buffer module, and the buffer area within this module controls the output of the code stream at a uniform rate. Outputting the code stream at a uniform rate means that the bits occupied by the code stream are output at a uniform rate, and it can be understood that this means the buffer area allows the code stream, which flows in at an uneven rate after encoding, to flow out in a uniform rate format in order to achieve stable output. Furthermore, the buffer area does not allow overruns. Overruns include overflows and underflows; an overflow occurs when the buffer area state exceeds the maximum value (max_buffer), and an underflow occurs when it falls below the minimum value (0).

[0203] The buffer area state described above represents the number of bits that the encoded image blocks in the image being processed occupy within the buffer area. It can be understood that this buffer area state is updated by the image blocks. For example, if an image block, after encoding, flows into the buffer area at a rate of 100 bits / second and flows out of the preset buffer area at a uniform rate of 50 bits / second, then the number of bits in the buffer area of ​​that image block per second will be 100 - 50 = 50 bits.

[0204] The buffer area state described above may also be called the physical buffer state (physical_buffer).

[0205] Specifically, the video encoding / decoding device may directly read the state information of the corresponding buffer area and obtain the buffer area state of the image to be processed. The video encoding / decoding device obtains the correspondence between the buffer area state of the image to be processed and the complexity information of the current block by the following steps S41-S42.

[0206] In S41, the video encoding / decoding device determines the fullness based on the buffer area state.

[0207] The term "filling level" refers to the linear mapping of buffer area states for each category, and indicates the degree to which the buffer area is full. See the following formula. TIFF0007847674000016.tif972

[0208] Of these, a and b are parameters for linear mapping of fullness segments by physical_buffer, where a represents scaling relative to physical_buffer and b represents the degree of offset relative to physical_buffer. These parameters may be adjusted based on buffer region state, image information, and complexity information. This will be explained below by case.

[0209] In Case 1, parameters a and b are determined based on image information.

[0210] For example, in some blocks starting with slice, a = 0.8 and b = (-bpp) / (2 × block_size), where block_size represents the size of the block.

[0211] It can be seen that in several blocks starting from a slice, the buffer area state is small, and by a and b above, the fullness is further reduced, resulting in a smaller ref_qp and improved encoding / decoding accuracy.

[0212] Alternatively, in a block at a slice boundary (e.g., the first row and first column), a=0 and b=-bpp x block_size.

[0213] It is understandable that the quality of the prediction results is low because there are no reference pixels in the boundary blocks. By determining a relatively small ref_qp, we can reduce the fullness.

[0214] In Case 2, parameters a and b are determined based on image information and buffer area state.

[0215] Parameters a and b are adjusted based on the relationship between bpp and the buffer area state.

[0216] For example, if bpp is 8 and (physical_buffer) / (max_buffer)>0.85, then a=1 and b=0. Also, for example, if bpp is 6 and (physical_buffer) / (max_buffer)>0.85, then a=1.1 and b=block_size.

[0217] As can be seen from this, when the buffer area state is relatively satisfied with a low bpp, a relatively large ref_qp is obtained, and by further increasing the fullness using a and b above, it is possible to prevent the buffer area state from exceeding the maximum number of bits in the buffer area.

[0218] In Case 3, parameters a and b are determined based on image information and complexity information.

[0219] For example, for an image block with high complexity information, a=1 and b=bpp×block_size. For an image block with general complexity information, a=1 and b=0. For an image block with low complexity information, a=1 and b=-bpp×block_size.

[0220] Selectively, the degree of complexity information may correspond to the complexity information levels described above.

[0221] In Case 4, parameters a and b are determined by the buffer area state.

[0222] For example, if physical_buffer < (max_buffer) / 2, then a = 0.9 and b = 0. If (max_buffer) / 2 ≤ physical_buffer < (3 × max_buffer) / 4, then a = 1.0 and b = 0. If physical_buffer ≥ (3 × max_buffer) / 4, then a = 1.2 and b = 0.

[0223] As can be seen from this, when the buffer area is relatively free, reducing the fullness using a and b above can lower ref_qp. Similarly, when the buffer area is relatively full, increasing the fullness using a and b above can increase ref_qp.

[0224] The above four cases are merely examples for determining the related parameters of parameters a and b, and the magnitude of the parameter values ​​may be other values, but are not limited to these.

[0225] In S42, the video encoding / decoding device calculates ref_qp based on the saturation level.

[0226] The implementation method for determining ref_qp is described below.

[0227] In the possible implementation methods, ref_qp is calculated by referring to the following formula. TIFF0007847674000017.tif952

[0228] Of these, c and d are parameters that may be adjusted based on buffer region state, image information, and complexity information. Optionally, parameter c may be determined based on max_qp, where max_qp is the maximum quantization parameter.

[0229] Of these, c and d are parameters for linear mapping of each ref_qp segment by fullness, where c represents scaling relative to fullness and d represents the degree of offset relative to fullness. These parameters may be adjusted based on image information, complexity information, and fullness. This will be explained below by case.

[0230] In Case 1, parameters c and d are determined based on image information.

[0231] For example, fixed parameters such as c=1 and d=1 may be used depending on the image information. Different c and d may exist for different bit widths.

[0232] Alternatively, special processing can be performed on the slice boundary block (e.g., the first row and first column) so that ref_qp becomes smaller than its original value, i.e., relatively small c and d are used.

[0233] In Case 2, parameters c and d are determined based on the level of filling and image information.

[0234] For example, parameters c and d are determined based on the bit width and target pixel depth (target_bpp) in the image information.

[0235] For example, if target_bpp is 8 bits and fullness < 0.1, then for an 8-bit image, c=0, d=2; for a 10-bit image, c=0, d=4; and for a 12-bit image, c=0, d=6. If target_bpp is 8 bits and fullness > 0.8, then for an 8-bit image, c=1, d=2; for a 10-bit image, c=1, d=4; and for a 12-bit image, c=1, d=6. Of these, c and d are updated in the image block.

[0236] Among these, `target_bpp` refers to a parameter specified on the encoding side, representing the number of bits required per pixel point after compression. For example, for a 10-bit original image with a YUV444 sampling format, if the original image's bpp is 30 bits, a `target_bpp` of 5 bits indicates a 6x compression.

[0237] In other possible implementations, ref_qp may be determined based on the bit width and pixel depth in the image information. Specifically, a range may be determined for ref_qp based on the image information. Of these, the range of ref_qp will differ when the image information changes. Specifically, the range is an interval consisting of the minimum reference quantization parameter (min_ref_qp) and the maximum reference quantization parameter (max_ref_qp). For example, for an image with a bpp of 8 bits and a sampling format of YUV444, if fullness < 0.25, then min_ref_qp = 0 and max_ref_qp = 8, and ref_qp is selected from (0, 8). If fullness > 0.85, then min_ref_qp = 4 and max_ref_qp = 56, and ref_qp is selected from (4, 56). Otherwise, min_ref_qp=0 and max_ref_qp=32, and ref_qp is selected from (0, 32).

[0238] In Case 3, parameters c and d are determined based on the buffer area state.

[0239] Specifically, parameters c and d are determined based on the relationship between physical_buffer and max_buffer.

[0240] If physical_buffer < (max_buffer) / 4, then c = 0.5 and d = 0. If (max_buffer) / 4 ≤ physical_buffer < (max_buffer) / 2, then c = 1.0 and d = 0. If physical_buffer ≥ (max_buffer) / 2, then c = 1.3 and d = 0.

[0241] In Case 4, parameters c and d are determined based on complexity information and occupancy.

[0242] For blocks with high complexity information, the mapping results for different levels of fullness will differ. For example, if fullness=0.5, a complex block ref_qp will be mapped to 32, while a simple block ref_qp will be mapped to 16.

[0243] Alternatively, for example, for a simple block, if fullness < 0.2, then c > 0 and d = 0, meaning ref_qp increases with increasing fullness. If 0.2 ≤ fullness ≤ 0.8, then c = 0 and d > 0, and the value of d is updated along with the image block, and as fullness increases, ref_qp is kept constant within a portion of fullness. If fullness > 0.8, then c > 0, meaning ref_qp increases with increasing fullness. Similar behavior occurs for normal and complex blocks. Of these, simple blocks are understood as blocks whose complexity information is less than or equal to a first preset value, complex blocks are understood as blocks whose complexity information is greater than or equal to a second preset value, and normal blocks are understood as blocks whose complexity information lies between the first and second preset values.

[0244] In Case 5, parameter c is determined based on complexity information.

[0245] As can be seen from the formula for calculating ref_qp above, setting a large parameter c results in a larger change in ref_qp in response to changes in fullness, and consequently, a larger change in qp determined based on ref_qp in response to changes in fullness. It can be seen that parameter c can be considered as the "slope" in the mapping relationship between qp and fullness. Therefore, by adjusting the size of parameter c, the sensitivity of qp in response to changes in fullness can be adjusted. When encoding blocks with relatively low complexity, the computational resources such as the number of bits in the buffer area consumed by encoding are relatively small, so the sensitivity of qp in response to changes in fullness can be appropriately reduced. In other words, to prevent qp from being adjusted to be too small due to changes in fullness, the size of parameter c can be appropriately reduced, thus ensuring that the encoding quality of blocks with relatively low complexity is not too low. Conversely, when encoding blocks with relatively high complexity, the computational resources such as the number of bits in the buffer area consumed by encoding are relatively large, so the sensitivity of qp in response to changes in fullness can be appropriately increased. In other words, the size of parameter c can be appropriately increased, further improving the control capability over the encoding rate. In summary, a relatively high parameter c may be determined for a block with relatively high complexity, but a relatively low parameter c may be determined for a block with relatively low complexity.

[0246] For example, c = 1.1 for blocks with relatively high complexity, and c = 0.9 for blocks with relatively low complexity.

[0247] In Case 6, parameters e and f are determined based on complexity information, maximum complexity (max_complexity), and maximum quantization parameter (max_qp), of which e and f are parameters related to the filling level, buffer state, and image information. TIFF0007847674000018.tif9127

[0248] Determining the reference quantization parameter based on the complexity information and fullness or buffer region state described above may refer to the function images of the reference quantization parameter, complexity, and buffer region state provided in Figure 12a. Of these, the reference quantization parameter increases as the buffer region state increases, and complexity information can have a different effect on the increase of the reference quantization parameter relative to the buffer region state. For example, in image blocks with relatively high complexity, the reference quantization parameter increases with increasing buffer region state. In image blocks with relatively low complexity, the increase in the reference quantization parameter with respect to the increase in buffer region state may be smaller than that in image blocks with relatively high complexity. Of course, the influence of complexity and buffer region state on the reference quantization parameter is always positively correlated.

[0249] Specifically, as shown in the function image of the reference quantization parameter and buffer area state provided in Figure 12b, assuming that the influence of the buffer area state on the reference quantization parameter is dominant in the intervals (0, max_buffer × 0.15) and (max_buffer × 0.85, maximum number of bits in the buffer area), that is, when the buffer area state is relatively empty or relatively full, the influence of complexity is relatively small. In the intervals [max_buffer × 0.15, max_buffer × 0.85], the influence of complexity is large, and considering the influence of complexity information on the reference quantization parameter, five possibilities can arise as shown in the figure. If the influence of complexity is relatively small, the reference quantization parameter does not need to change within that interval, corresponding to L3. If the influence of complexity is general, the reference quantization parameter may increase slowly within that interval, corresponding to L2 or L4. If the influence of complexity is relatively large, the reference quantization parameter may change suddenly within that interval, corresponding to L1 or L5. Of these, the starting point for changes in complexity is determined based on the complexity information itself.

[0250] For the influence of complexity information on each interval in Figure 12b, you may refer to the function image of the reference quantization parameter and complexity provided in Figure 12c.

[0251] As can be seen from this, the input and output of the code stream in the buffer area of ​​the image being processed can be dynamically controlled based on the state of the physical buffer area of ​​the image being processed, further stabilizing the output of the code stream.

[0252] Selectively, the video encoder / decoder determines the complexity level of the current block, determines the target cost based on the complexity level of the current block, and obtains the reference quantization parameters of the current block based on the target cost. The target cost refers to the predicted number of bits encoded / decoded by the current block. The actual number of bits encoded / decoded by the current block may be greater than, less than, or equal to the target cost. Determining the complexity level of the current block can be described in the examples section on complexity levels above, but is omitted here. The target cost refers to the number of bits occupied by the current block in the code stream.

[0253] Specifically, video encoding / decoding devices determine the target bit based on the complexity level of the current block through several of the following cases:

[0254] In Case 1, target_cost is determined based on image information and complexity.

[0255] Specifically, each complexity piece of information stores one reference bit (ref_cost) and is updated along with the image block, where ref_cost = 0.75 × ref_cost t-1+0.25 × real_cost, where real_cost represents the predicted lossless bit consumption of the current block, related to the quantization parameter (qp) and the actual number of encoded bits. ref_cost is the reference bit count used to determine target_cost. ref_cost t-1 `ref_cost` is the reference bit count corresponding to the previous image block. `ref_cost` represents the reference bit count corresponding to the current image block. A piecewise linear transformation is performed using `ref_cost` at the complexity level of the current block to obtain the target bits. See the following formula. TIFF0007847674000019.tif863

[0256] Of these, g and h are parameters for a piecewise linear mapping of target_cost based on ref_cost, where g represents the scaling relative to target_cost and h represents the degree of offset relative to ref_cost. The parameters g and h are related to the image information; for example, for an image with 8-bit BPP and a YUV444 sampling format, if ref_cost > 1.1 × average number of bits (ave_cost), then g = 1.8 and h = -9. If ref_cost < 0.9 × ave_cost, then g = 1.2 and h = -6. Of these, g and h are used to modify ref_cost based on ave_cost to obtain target_cost.

[0257] In Case 2, target_cost is determined based on complexity information.

[0258] Specifically, there are I complexity information and J types of modes, which are divided into I × J categories, and each category corresponds to one target_cost. For example, the complexity information indicates whether the current block is simple or complex, the modes include intra-block copy (IBC) mode and non-IBC mode, and there are a total of four categories: simple IBC, simple non-IBC, complex IBC and complex non-IBC, each category corresponds to one target_cost. The predicted number of bits for each category is updated by estimating the reference bit count for each category (ref_cost = real_bit + qp / 8) ((pred_cost t ) = 0.75 × pred_cost t-1 Calculate the coefficient (scale=bpp / (avg_complexity-offset)) by adding +0.25 × ref_cost). Of these, avg_complexity represents the sliding average complexity of all blocks up to the current block's position, and offset is related to the image format and bpp. Finally, obtain target_cost=scale × (pred_cost-offset). Of these, pred_cost t-1 This is the predicted number of bits corresponding to the previous image block.

[0259] In Case 3, target_cost is determined based on image information. Image information includes bit width, image sampling format, or other information.

[0260] Specifically, there are K bit widths and L image sampling formats, which are totaled into K × L categories, with each category corresponding to one target_cost. For example, there are two bit widths (8-bit and 12-bit) and two image sampling formats (YUV and RGB), resulting in a total of four target_costs, including 8-bit YUV, 8-bit RGB, 12-bit YUV, and 12-bit RGB.

[0261] In Case 4, obtain the fullness, and in combination with the complexity information, buffer area state, or image information of the current block, determine the target_cost based on the fullness.

[0262] Among them, the process of obtaining the fullness is the same as that in Step S41.

[0263] In Case 4.1, determine the target_cost based on the fullness and the buffer area state. ]

[0264] Specifically, target_cost = m × ref_cost + n × physical_buffer + o, where m, n, and o are parameters.

[0265] When fullness > 0.85, at this point, the setting of target_cost is dominated by fullness, that is, the value of n is greater than m. When fullness < 0.25, at this point, m is greater than n.

[0266] In Case 4.2, determine the target_cost based on the fullness, image information, and complexity information.

[0267] Among them, different fullness corresponds to different minimum target bits (min_target_cost) and maximum target bits (max_target_cost), which limit the target_cost.

[0268] For example, for an image with 8 bits and a sampling format of YUV444, min_target_cost = bpp × fullness × p1 + q1, and max_target_cost = bpp × fullness × p2 + q2. Among them, when fullness < 0.25, p1 and q1 make min_target_cost smaller, and when fullness > 0.75, p1 and q1 make min_target_cost larger. Among them, when fullness < 0.25, p2 and q2 make max_target_cost smaller, and when fullness > 0.75, p2 and q2 make max_target_cost larger. <(

[0269] In Case 4.3, target_cost is determined based on fullness and complexity information.

[0270] For a simple block with relatively low complexity, when fullness < 0.1, m > 0 and n = 0, that is, ref_qp increases as fullness increases. When 0.1 ≤ fullness ≤ 0.9, m = 0 and n > 0, and the value of n is updated with the image block. As fullness increases, ref_qp is maintained constant within a part of fullness. When fullness > 0.9, m > 0, that is, ref_qp increases as fullness increases. Similar operations are also performed on normal blocks and complex blocks. For a certain part of simple blocks and normal blocks, when the actual encoded bits are larger than the target bits, the value of n increases and conversely decreases, that is, by adjusting n, the actual bit consumption is made less than or equal to target_cost. For complex blocks, by adjusting n, the actual bit consumption is made greater than or equal to target_cost (if simple blocks and normal blocks do not save bits, there are no extra bits available for complex blocks, and it is strictly made less than or equal to target_cost).

[0271] It should be noted that there seems to be a minor formatting issue in the original text where "<(

[0269] " is likely a typo and should probably be "

[0269] ". This has been maintained as is in the translation for the sake of following the instruction to preserve all 7 - digit tags exactly.Selectively, the video encoder / decoder determines the reference quantization parameter for the current block based on the target bit. See the following formula. TIFF0007847674000020.tif1070

[0272] Of these, u and v are parameters. u and v are parameters for the linear mapping of ref_qp for each segment by target_cost, where u represents the scaling relative to target_cost and v represents the degree of offset relative to target_cost. For example, u = 8 / 3 and v = ref_cost × 8.

[0273] As can be seen, the reference quantization parameters obtained based on the target bits corresponding to the complexity level can be flexibly adjusted based on the magnitude of the target bits. In this way, the quantization parameters can be adjusted more flexibly.

[0274] The implementation method for step S32 will be explained below.

[0275] Selectively, the video encoder / decoder determines the quantization parameters of the current block based on the reference quantization parameters of the current block. Specifically, one can refer to the equation qp = ref_qp × x + y, where x and y are parameters of the linear mapping of qp segment by segment according to ref_qp, where x represents the scaling relative to ref_qp and y represents the degree of offset relative to ref_qp.

[0276] Specifically, the determination of quantization parameters by a video encoding / decoding device based on the current reference quantization parameters is achieved through several of the following cases:

[0277] In Case 1, parameters x and y are determined based on image information.

[0278] When x is 1 and y is 0, the reference quantization parameter is the quantization parameter.

[0279] Alternatively, for an image with a YUV444 sampling format, each channel is x=1 / 3, y=0. Or, the Y channel is x=1 / 2, y=0, and the chromaticity channel is x=1, y=0.

[0280] Alternatively, if ref_qp is in (0, 16), the Y channel is x=1 / 4, y=0 and the chromaticity channel is x=1 / 2, y=0. If ref_qp is in (17, 32), the Y channel is x=1 / 2, y=2 and the chromaticity channel is x=1 / 2, y=4. If ref_qp is in (33, 63), the Y channel is x=1, y=0 and the chromaticity channel is x=1, y=0.

[0281] In Case 2, parameters x and y are determined based on complexity information.

[0282] Specifically, the video encoding / decoding device determines weight coefficients based on the complexity information of the current block, and these weight coefficients are used to adjust the quantization parameters of the current block based on the degree of complexity of the current block. Based on the weight coefficients and the reference quantization parameters of the current block, the quantization parameters of the current block are determined.

[0283] For example, the above weighting coefficient may be considered as x below, and the weighting coefficient may be determined based on the following formula. TIFF0007847674000021.tif6149

[0284] Of these, w represents the weights, including w11, w12, and w13, where 0 ≤ w11, w12, and w13 ≤ 1. block_complexity1, block_complexity2, and block_complexity3 represent the complexity information of the three channels of the current block, respectively.

[0285] By this possible implementation method, by using the weight coefficient determined by the complexity information of the current block to adjust the quantization parameter of the current block, the quantization parameter of the current block can be adaptively adjusted based on the complexity information of the current block, improving the accuracy of determining the quantization parameter of the current block and improving the accuracy of video encoding / decoding.

[0286] In Case 3, a weight coefficient is determined based on the complexity information of M encoded / decoded image blocks and the complexity information of the current block. Based on the weight coefficient and the reference quantization parameter of the current block, the quantization parameter of the current block is determined.

[0287] For example, the above weight coefficient may be regarded as the following x, and the weight coefficient may be determined based on the following formula.

[0288] Among them, window_complexity represents the complexity information of the image blocks included in the sliding window. As the image blocks in the sliding window change, the window_complexity is updated accordingly. Specifically, it is calculated based on the following formula. TIFF0007847674000022.tif8149TIFF0007847674000023.tif11149

[0289] Among them, w10, w11, and w12 represent weights, and 0 ≤ w10, w11, w12 ≤ 1. window_complexity z represents the complexity information of the M encoded / decoded image blocks starting from the z block. Similarly, window_complexit y z-1 represents the complexity information of the M encoded / decoded image blocks starting from the z-1 block. window_complexity1 z , window_complexity2 z and window_complexity3 zEach of these represents complexity information for three different image blocks contained within the sliding window. y = 0.

[0290] An implementation method for determining the quantization parameters of the current block based on the weight coefficients and the reference quantization parameters of the current block may refer to the equation qp = ref_qp × x + y, but a detailed explanation is omitted here.

[0291] In S603, the video encoding / decoding unit encodes / decodes the current block based on the quantization parameters.

[0292] Selectively, the complexity information of the current block is obtained by calculating it using the current block's coding rate control unit, and the quantization parameter of the current block is the quantization parameter of the current block's coding rate control unit. The video encoding / decoding of the current block by the video encoding / decoding device based on the current block's quantization parameter includes determining the quantization parameter of the current block's encoding / decoding unit based on the quantization parameter of the coding rate control unit, and performing encoding / decoding on the current block based on the quantization parameter of the encoding unit. When encoding is performed on the current block, the above encoding / decoding unit is the encoding unit, and when decoding is performed on the current block, the above encoding / decoding unit is the decoding unit.

[0293] The coding rate control module is understood to be calculated by the coding rate control unit when determining the quantization parameters. If the size of the coding rate control unit is larger than the size of the basic coding unit (quantization unit), it means that multiple basic coding units use the same quantization parameters. If the size of the coding rate control unit is equal to the size of the quantization unit, one-to-one correspondences of quantization parameters can be obtained. If the size of the coding rate control unit is smaller than the size of the quantization unit, it means that one quantization unit corresponds to multiple quantization parameters, and in this case, the quantization unit needs to use some strategy to determine the final quantization parameters based on those multiple quantization parameters.

[0294] In the first possible implementation, the video encoding / decoding device divides the quantization units based on the coding rate control unit, that is, it assigns a one-to-one correspondence between multiple quantization parameters and multiple quantization units.

[0295] In the second possible implementation, multiple quantization parameters are weighted, or a minimum value is selected to obtain one quantization parameter, which is then associated with that single quantization unit.

[0296] In a third possible implementation, multiple quantization parameters are aggregated based on complexity information and buffer region states. Exemplarily, quantization parameters with similar complexity information are aggregated into one, and similar complexity information may be multiple pieces of complexity information that satisfy a specific difference range.

[0297] Through the possible implementation methods described above, the quantization parameters of the coding unit in the current block are determined based on the quantization parameters of the coding rate control unit. In this way, the quantization parameters of the coding block can be matched with the coding rate control strategy, and the coding / decoding results take into account image quality while considering the coding rate control requirements, thereby improving coding / decoding efficiency.

[0298] The above aggregation method may involve obtaining a single quantization parameter by weighting similar quantization parameters, or it may involve selecting the minimum value among similar quantization parameters to obtain the aggregated quantization parameter.

[0299] For one or more accumulated quantization parameters, multiple quantization units corresponding to the quantization parameters are obtained by the first possible implementation method described above, or a single quantization parameter corresponding to a single quantization unit is obtained by the second possible implementation method described above, but is not limited to these.

[0300] Step S603 can be understood to include the video encoding / decoding device performing encoding or decoding on the current block based on quantization parameters.

[0301] Selectively, during encoding, the video encoder encodes either the complexity information of the current block into the code stream, or the quantization parameters of the current block into the code stream. Accordingly, the decoding side obtains the complexity information from the code stream, calculates the quantization parameters, and performs decoding, or the decoding side obtains the quantization parameters from the code stream and performs decoding. Of course, the video encoder may encode both of the above information into the code stream.

[0302] Furthermore, when a video encoder encodes the complexity information of the current block into the code stream, the video decoder obtains the complexity information and calculates the quantization parameters accordingly, but the video decoder does not need to update other parameters using this complexity information. For example, the method for determining target_cost described above relates to updating based on the complexity information of the current block. However, in concrete implementation, the result of updating based on complexity may differ from the result of updating based on historical information (e.g., the number of bits occupied by the encoded / decoded image block and the quantization parameters of the encoded / decoded image block). In this case, the original parameter update method is retained instead of using a method that updates the parameter using complexity information.

[0303] Calculating the complexity information of the current block using the method described above helps to determine encoding / decoding parameters for that current block more accurately, such as quantization parameters, thereby improving the encoding / decoding efficiency of the image.

[0304] Selectively, the code streams obtained by the above encoding / decoding methods may undergo the following code stream grouping methods before being encoded into code streams.

[0305] As can be seen from the above explanation of slices, an image may be divided into multiple slices based on the image width (image_width) and image height (image_height). `image_width` specifies the width of the image's luminance components, i.e., the number of horizontal samples, and is a 16-bit unsigned integer. The unit of `image_width` should be the number of samples per row of the image. The sample at the top-left corner of the displayable area should be aligned with the sample at the top-left corner of the decoded image. The value of `ImageWidth` is equal to the value of `image_width`. The value of `ImageWidth` must be a multiple of 16, not 0. `image_height` specifies the height of the image's luminance components, i.e., the number of vertical scan rows, and is a 16-bit unsigned integer. The unit of `image_height` should be the number of rows of image samples. The value of `ImageHeight` is equal to the value of `image_height`. The value of `ImageHeight` must be a multiple of 2, not 0.

[0306] A slice is a fixed rectangular region in an image, and is therefore also called a rectangular slice. It contains portions of several encoding units within the image, and the slices do not overlap. The division method is not limited, but the slice may be further divided into CUs. When dividing a slice based on an image, it is possible to adjust the current image width or height to divide it into an integer number of slices. As shown in the schematic diagram of the image boundary provided in Figure 13a, real_width is the actual width of the image, and real_height is the actual height of the image, i.e., it indicates the boundary of the displayable area of ​​the image. To divide the slice, the image width and height are adaptively increased to obtain image_width and image_height in the figure.

[0307] A slice has a width (slice_width) and a height (slice_height). For example, as shown in the schematic diagram of slices provided in Figure 13b, SliceNum X represents the number of slices in the horizontal direction of an image, and SliceNum Y represents the number of slices in the vertical direction of an image.

[0308] In this code stream grouping method, the length of the code stream of each slice is fixed, and the length of the R-1 chunk prior is fixed, but the length of the last chunk is not fixed. Figures 14a and 14b are flowcharts showing this code stream grouping method on the encoding and decoding sides, respectively.

[0309] Figure 14a is a flowchart of a code stream grouping method on the encoding side according to an embodiment of the present invention, and includes steps S1401a-S1406a.

[0310] In S1401a, the image is divided into sliceNumX * sliceNumY rectangular slices in the horizontal and vertical directions.

[0311] By dividing the image horizontally and vertically, sliceNumX rectangular slices are obtained horizontally, and sliceNumY rectangular slices are obtained vertically. The image referred to above is the image to be processed. After dividing the image to be processed horizontally and vertically in this way, horizontal slices and vertical slices can be obtained, thereby obtaining sliceNumX and sliceNumY, i.e., the number of first slices and the number of second slices.

[0312] In S1402a, the total resources are calculated, and the number of chunks (chunkNum) for each slice is determined.

[0313] `total_resoure` refers to the resources occupied by the slice, calculated based on the number of bits required for each pixel point.

[0314] total_resourec=((slice_width×slice_height×target pixel depth (target_bpp)+7)>>3)<<3.

[0315] The number of chunks (chunkNum) is determined based on the total resources (total_resource).

[0316] TIFF0007847674000024.tif973

[0317] Of these, n = total_resource%size == 0?0:1, and size is an integer multiple of 8, for example, size = target_bpp × 32 × block_num.

[0318] Of these, block_num is a preset configuration parameter, and block_num is an integer multiple of 4.

[0319] In S1403a, sliceNumX slices in each slice row are encoded sequentially to generate sliceNumX bitstream buffer areas.

[0320] The bitstream buffer area may be represented by slicebuffer[sliceNumX], and the bitstream buffer area may be padded with zeros and byte alignment may be performed.

[0321] In this step, encoding each slice is understood to mean encoding each image block in the slice according to the method described in the above embodiment and obtaining the encoded data for each image block in the slice.

[0322] In S1404a, each bitstream buffer area is further divided into N bit chunks, the length of the N-1 previous chunk is the first number, and the length of the last chunk is the second number.

[0323] In this step, the above bit chunk may also be called a code stream chunk. The above N represents the number of chunks, i.e., chunkNum.

[0324] Specifically, each bitstream buffer area is further divided into chunkNum chunks. The chunksize is calculated as follows: The length of the chunkNum-1 chunk prior is chunksize=size1, and the length of the last chunk is size2, where size1=size and size2=total_resource-(chunkNum-1)×size. The first number may be size1, and the second number may be size2.

[0325] In S1405a, each bit chunk of sliceNumX slices is sequentially interleaved and encoded to form the final bitstream.

[0326] For each sliceNum slice in a single slice row, encoding is performed sequentially Num times, and each time the chunk of each slice is interleaved to form the final code stream.

[0327] In S1406a, it is determined whether the encoding of the slice is finished. If it is not finished, the process returns to S1403a and encodes the next slice line.

[0328] Figure 14b is a flowchart of a code stream grouping method on the decoding side according to an embodiment of the present invention, and includes steps S1401b-S1406b as shown in Figure 14b.

[0329] In S1401b, the image is divided into sliceNumX * sliceNumY rectangular slices in the horizontal and vertical directions.

[0330] Since this step is the same as step S1401a above, the number of first horizontal slices and the number of second vertical slices in the image can be obtained. Similarly, in this step, the image is the image to be processed.

[0331] Until decoding is complete on the decoding side, the decoding side cannot obtain the image to be processed itself. Therefore, in this step, the actual image width and actual image height of the image to be processed are obtained from the video header information or image header information in the received code stream, and then sliceNumX (first slice count) and sliceNumY (second slice count) are calculated based on the actual image width and actual image height.

[0332] Of course, you can also directly obtain sliceNumX and sliceNumY from the video header information or image header information in the code stream.

[0333] In S1402b, the total resources are calculated, and the number of chunks (chunkNum) for each slice is determined.

[0334] This step is the same as step S1402a described above.

[0335] In S1403b, the code stream is received, and the code stream chunks of sliceNumX slices in each slice row are sequentially analyzed. The code stream of each chunk is deinterleaved into the bitstream buffer area of ​​each slice, and the length of the N-1 previous chunk is the first number, and the length of the last chunk is the second number.

[0336] The decryption side may receive the code streams sequentially, but of course, it may also directly obtain all the code streams to be decrypted.

[0337] The number of code stream chunks in each slice is given by the chunkNum above.

[0338] During deinterleaving, the code stream chunks of each slice are deinterleaved on a slice-by-slice basis, and the result of the deinterleaving is stored in the bitstream buffer area corresponding to that condition.

[0339] Selectively, the code stream is received, and chunkNum times are parsed sequentially. Each time, slicenumX code stream chunks that have been parsed are deinterleaved into the code stream buffer area of ​​each slice. The chunk size is calculated as follows: the length of the chunkNum-1 chunk before is chunksize=size1, and the length of the last chunk is size2, where size1=size and size2=total_resource-(chunkNum-1)×size.

[0340] In S1404b, each slice is decoded using the bitstream buffer area of ​​each slice.

[0341] In S1405b, the reconstructed image of each rectangular slice is obtained.

[0342] In S1406b, the system determines whether the parsing of the slice code stream is complete. If not, it returns to S1403b and parses the next slice lines in order.

[0343] As shown in the schematic diagram of chunk interleaving using the code stream grouping method provided in Figure 15, when sliceNumX is 2, the schematic diagram of chunk interleaving using the code stream grouping method described above is shown.

[0344] In Figure 15, chunk R represents the R-th chunk. The length of the R-th chunk is not fixed, but the lengths of the other R-1 chunks are fixed. In the same figure, size 1 represents the lengths of the other R-1 chunks, and size 2 represents the length of the R-th chunk.

[0345] The code streams shown in Figure 15(c) are grouped according to size 1 and size 2, respectively. When size 1 is smaller than size 2, a schematic diagram of the interleaving of the grouped chunks is shown in Figure 15(a). When size 1 is larger than size 2, a schematic diagram of the interleaving of the grouped chunks is shown in Figure 15(b).

[0346] In other code stream grouping methods, the length of the code stream of each slice is fixed, while the length of the r-th chunk is not fixed, but the lengths of the other r-1 chunks are fixed. Figures 16a and 16b are flowcharts showing this code stream grouping method on the encoding and decoding sides, respectively.

[0347] Figure 16a is a flowchart of the code stream grouping method on the encoding side. As shown in Figure 16a, it includes steps S1601a-S1606a.

[0348] In S1601a, the image is divided into sliceNumX * sliceNumY rectangular slices in the horizontal and vertical directions.

[0349] This step is the same as step S1401a described above.

[0350] In S1602a, the total resources are calculated, and the number of chunks (chunkNum) for each slice is determined.

[0351] This step is the same as step S1402a described above.

[0352] In S1603a, sliceNumX slices in each slice row are encoded sequentially, and slicenumX bitstream buffer areas are generated.

[0353] In S1604a, the bitstream buffer area of ​​each slice is further divided into N bit chunks, the length of the Kth chunk is the first number, and the lengths of the other N-1 chunks are the second number.

[0354] In this step, the above bit chunk may also be called a code stream chunk. The above N represents the number of chunks, i.e., chunkNum.

[0355] For example, each bitstream buffer area may be divided into chunkNum chunks, and the length of each chunk, chunksize, is not fixed. The first number may be size1, and the second number may be size2. The method for calculating chunksize is as follows: the length of the k-th chunk is size1, and the length of the other chunkNum-1 chunks is chunksize=size2, where size2=size, size1=total_resource-(chunkNum-1)×size, and the range of k is 1 to chunkNum.

[0356] In S1605a, interleaving and encoding are performed sequentially on each bit chunk of sliceNumX slices to form the final bitstream.

[0357] For each sliceNum slice in a single slice row, chunkNum times are encoded sequentially, and each time the chunk of each slice is interleaved to form the final code stream.

[0358] In S1606a, it is determined whether the encoding of the slice is finished. If it is not finished, the process returns to S1603a and encodes the next slice line.

[0359] Figure 16b is a flowchart of the code stream grouping method on the decoding side. As shown in Figure 16b, it includes steps S1601b-S1606b.

[0360] In S1601b, the image is divided into sliceNumX * sliceNumY rectangular slices in the horizontal and vertical directions.

[0361] This step is the same as step S1401b described above.

[0362] In S1602b, the total resources are calculated, and the number of chunks (chunkNum) for each slice is determined.

[0363] This step is the same as step S1402b described above.

[0364] In S1603b, the code stream is received, and the code stream chunks of sliceNumX slices in each slice row are sequentially analyzed. The code stream of each chunk is deinterleaved into the bitstream buffer area of ​​each slice, and the length of the K-th chunk is the first number, and the lengths of the other N-1 chunks are the second number.

[0365] The code stream is received, and for each slice row, sliceNumX slices are parsed sequentially for chunkNum times. Each time, sliceNumX code stream chunks that have been parsed are deinterleaved into the code stream buffer area of ​​each slice. The chunksize is calculated as follows: The length of the k-th chunk is size1, and the length of the other chunkNum-1 chunks is chunksize=size2. Of these, size2=size, size1=total_resource-(chunkNum-1)×size, and the range of k is 1 to chunkNum.

[0366] In S1604b, each slice is decoded using the bitstream buffer area of ​​each slice.

[0367] In S1605b, the reconstructed image of each rectangular slice is obtained.

[0368] In S1606b, it is determined whether the parsing of the slice code stream is complete. If not, the process returns to S1603b and the next slice lines are parsed sequentially.

[0369] As shown in the schematic diagram of chunk interleaving using the code stream grouping method provided in Figure 17, when sliceNumX is 2, the schematic diagram of chunk interleaving using the other code stream grouping method described above is shown.

[0370] In Figure 17, chunk r represents the r-th chunk, and chunk R represents the R-th chunk. The length of the r-th chunk is not fixed, but the lengths of the other R-1 chunks are fixed. In the same figure, size 1 represents the lengths of the other R-1 chunks, and size 2 represents the length of the r-th chunk.

[0371] The code streams shown in Figure 17(c) are grouped according to size 1 and size 2, respectively. When size 1 is smaller than size 2, a schematic diagram of the interleaving of the grouped chunks is shown in Figure 17(a). When size 1 is larger than size 2, a schematic diagram of the interleaving of the grouped chunks is shown in Figure 17(b).

[0372] Furthermore, any configurations not specifically described in the above proposal can be implemented on either the decoding or encoding side.

[0373] If no conflicts occur, some or all of the above-mentioned implementation methods can constitute a new implementation method.

[0374] Embodiments of the present invention provide a video encoding / decoding device, which may be a video encoding / decoding apparatus, a video encoder, or a video decoder. Specifically, the video encoding / decoding device is for performing steps performed by the video encoding / decoding apparatus in the video encoding / decoding method described above. The video encoding / decoding device according to embodiments of the present invention may include modules corresponding to the corresponding steps.

[0375] Embodiments of the present invention may divide a video encoding / decoding device into functional modules by the methods and examples described above, for example, by dividing it into a separate functional module corresponding to each function, or by integrating two or more functions into a single processing module. The integrated modules may be implemented in hardware form or in software functional module form. In embodiments of the present invention, the division into modules is schematic and merely a logical division of functions, and other division methods may be used in actual implementation.

[0376] When the device is divided into separate functional modules corresponding to each function, Figure 18 is a schematic diagram of the video encoding / decoding device according to an embodiment of the present invention. As shown in Figure 18, the video encoding / decoding device 180 includes an acquisition module 1801, a determination module 1802, and an encoding / decoding module 1803.

[0377] The acquisition module 1801 is for acquiring complexity information of the current block in the image to be processed. The complexity information of the current block is acquired by calculating at least one angular gradient of the current block based on at least the pixel value of the current block, and corresponds to, for example, step S601 above.

[0378] The decision module 1802 determines the quantization parameters of the current block based on the complexity information of the current block, and corresponds, for example, to step S602 above.

[0379] The encoding / decoding module 1803 is for encoding / decoding the current block based on quantization parameters, and corresponds, for example, to step S603 above.

[0380] In one example, the acquisition module 1801 is used to calculate at least one angular gradient of the current block based on the pixel values ​​of the current block and the reconstructed values ​​of the encoded / decoded pixel values ​​of the current block, and to acquire complexity information of the current block based on at least one angular gradient of the current block.

[0381] In one example, the acquisition module 1801 is specifically used to calculate at least one angular gradient of the current block based on the pixel values ​​of the current block and the pixel values ​​of blocks adjacent to the current block in the image to be processed, and to acquire complexity information of the current block based on at least one angular gradient of the current block.

[0382] In one example, acquisition module 1801 specifically acquires the predicted angle used in the angle prediction mode of the current block, calculates the angle gradient based on the predicted angle, acquires the corresponding complexity information, and uses the corresponding complexity information as the complexity information of the current block.

[0383] In one example, the current block is an N-channel image block, and the acquisition module 1801 specifically acquires complexity information for each channel image block based on the pixel values ​​of each channel image block in the N-channel image block, where N is an integer greater than zero, and is used to determine the complexity information of the current block based on the complexity information of each channel image block.

[0384] In one example, the acquisition module 1801 is specifically used to divide each channel image block into at least two subblocks, determine the complexity information of at least two subblocks of each channel image block, and determine the complexity information of the corresponding channel image block in each channel image block based on the complexity information of at least two subblocks of each channel image block.

[0385] In one example, the acquisition module 1801 is specifically used to determine the minimum value in the complexity information of at least two subblocks of each channel image block as the complexity information of the corresponding channel image block.

[0386] In one example, the acquisition module 1801 is specifically used to determine the minimum value in the complexity information of each channel image block as the complexity information of the current block.

[0387] In one example, the acquisition module 1801 is specifically used to determine the complexity level of each channel image block based on the complexity information of each channel image block, and to determine the complexity information of the current block based on the complexity level of each channel image block.

[0388] In one example, the decision module 1802 is specifically used to determine the reference quantization parameter of the current block based on the complexity information of the current block, and to determine the quantization parameter of the current block based on the reference quantization parameter of the current block.

[0389] In one example, when the video encoding / decoding method is a video encoding method, the decision module 1802 specifically obtains the buffer area state of the image to be processed. The buffer area state represents the number of bits occupied by the encoded image block in the image to be processed in the buffer area, and the buffer area itself controls the output of the code stream of the image to be processed at a uniform rate. The reference quantization parameter of the current block is determined based on the correspondence between the buffer area state and the complexity information of the current block.

[0390] In one example, the decision module 1802 is specifically used to determine the complexity level of the current block. Based on the complexity level of the current block, the corresponding target bit is determined, which is the number of bits that the current block occupies in the code stream. Based on the target bit, the reference quantization parameter of the current block is obtained.

[0391] In one example, the decision module 1802 specifically determines weight coefficients based on the complexity information of the current block, and these weight coefficients are used to adjust the quantization parameters of the current block based on the degree of complexity of the current block. The weight coefficients and the reference quantization parameters of the current block are used to determine the quantization parameters of the current block.

[0392] In one example, the complexity information of the current block is obtained by the current block's coding rate control unit, which is the basic processing unit that calculates the complexity information of the current block, and the quantization parameters of the current block are the quantization parameters of the current block's coding rate control unit. Specifically, the coding / decoding module 1803 is used to determine the quantization parameters of the current block's coding / decoding unit based on the quantization parameters of the coding rate control unit, and to code / decode the current block based on the quantization parameters of the coding / decoding unit.

[0393] Of these, all relevant details for each step in the embodiment of the above method can be explained by referring to the functional description of the corresponding functional block, and therefore will be omitted here.

[0394] Of course, the video encoding / decoding device according to the embodiment of the present invention includes the above-mentioned modules, but is not limited thereto. For example, the video encoding / decoding device may further include a storage module 1804.

[0395] The memory module 1804 is for storing the program code and data of the video encoding / decoding device.

[0396] Embodiments of the present invention also provide a video decoder including a processor and memory.

[0397] The processor stores instructions that it can execute in memory.

[0398] The processor is configured to execute instructions and cause the video decoder to implement the video image decoding method in the above embodiment.

[0399] Embodiments of the present invention also provide a video encoder including a processor and memory.

[0400] The processor stores executable instructions in memory.

[0401] When the processor executes an instruction, it is configured to cause the video encoder to implement the video image encoding method in the above embodiment.

[0402] Embodiments of the present invention also provide a video encoding and decoding system including a video encoder and a video decoder. The video encoder is for performing any of the video encoding / decoding methods according to the above embodiments, and the video decoder is for performing any of the video encoding / decoding methods according to the above embodiments.

[0403] Embodiments of the present invention also provide electronic equipment, which includes the video encoding / decoding device 180 described above, and which performs a method performed by any of the video encoding / decoding devices provided in the above description.

[0404] Embodiments of the present invention provide a computer-readable storage medium on which a computer program is stored, and when the computer program is executed on a computer, the computer is made to perform a method performed by one of the video decoders provided in the above description.

[0405] For any of the computer-readable storage media provided above, you can refer to the corresponding implementation methods described above for explanations of relevant content and beneficial effects; therefore, we will omit the explanation here.

[0406] Embodiments of the present invention also provide a chip, which integrates a control circuit and one or more ports for realizing the functions of the video encoding / decoding device 100 described above. The functions that may be selectively supported by the chip can be described in the above description, which is omitted here. Those skilled in the art will understand that all or part of the steps for realizing the above-described implementation may be programmed to the relevant hardware. The program may be stored in a computer-readable storage medium, such as read-only memory or random-access memory. The processing unit or processor may be a central processing unit, a general-purpose processor, an application-specific integrated circuit (ASIC), a microprocessor (digital signal processor, DSP), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof.

[0407] Embodiments of the present invention provide a computer program product including instructions, which, when executed on a computer, cause the computer to perform one of the methods described in the embodiments. The computer program product includes one or more computer instructions. When the instructions of the computer program are loaded onto a computer and executed, all or part of them realize the flow or function of the embodiments of the present invention. The computer may be a general-purpose computer, a dedicated computer, a computer network, or other programmable device. The computer instructions may be stored on a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, radio, microwave, etc.). The computer-readable storage medium may be any available medium accessible to the computer, or it may be a data storage device such as a server or data center that integrates one or more available media. The usable media may include magnetic media (e.g., floppy disks, hard disks, tapes), optical media (e.g., DVDs), or semiconductor media (e.g., SSDs).

[0408] It should be noted that the device for storing the above-mentioned computer instructions or computer programs according to the embodiments of the present invention is, for example, the above-mentioned memory, computer-readable storage medium, and communication chip, and is non-transitory, but is not limited thereto.

[0409] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware, or any combination thereof. When implemented using a software program, all or part of the embodiments are implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded into a computer and executed, all or part of them implement the flow or function of the embodiments of the present invention. The computer may be a general-purpose computer, a dedicated computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, radio, microwave, etc.) method. The computer-readable storage medium may be any available medium accessible to the computer, or it may be a data storage device such as a server or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

[0410] While the present invention has been described here with reference to each embodiment, those skilled in the art can understand and implement other variations of the disclosed embodiments by referring to the drawings, disclosures, and claims in the process of carrying out the requested claims. In the claims, the word “comprising” does not exclude other components or steps, and “one” or “one” does not exclude multiple cases. A single processor or other unit can implement some of the functions listed in the claims. Although different dependent claims describe several measures, this does not mean that these measures cannot be combined to produce a good effect.

[0411] While the present invention has been described with reference to specific features and embodiments, it is clear that various modifications and combinations are possible without departing from the spirit and scope of the invention. Accordingly, this specification and drawings are merely illustrative descriptions of the invention as defined by the claims and are deemed to include any and all modifications, changes, combinations, or equivalents within the scope of the invention. Clearly, those skilled in the art can make various changes and variations to the invention without departing from the spirit and scope of the invention. Thus, the invention also includes these changes and variations if they fall within the scope of the claims and the equivalent art of the invention. The above description is a preferred embodiment of the invention and is not intended to limit the invention, and modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be considered to be within the scope of the protection of the invention.

Claims

1. A video decoding method applied to the decoding side, The complexity level of the current block in the image to be processed is obtained by analyzing the code stream, The target bit of the current block is determined based on the complexity level of the current block, The quantization parameters of the current block are determined based on the target bits of the current block, This includes decoding the current block based on the quantization parameters, A video decoding method characterized by the following:

2. The current block is an N-channel image block, and obtaining the complexity level of the current block in the image to be processed is: Obtain the complexity level of each channel image block, This includes obtaining the complexity level of the current block based on the complexity level of each channel image block, The video decoding method according to feature 1.

3. Determining the quantization parameters of the current block based on the target bits of the current block is: Based on the target bits of the current block, the reference quantization parameter of the current block is determined, This includes determining the quantization parameters of the current block based on the reference quantization parameters of the current block, The video decoding method according to claim 1 or 2, characterized by the features described above.

4. The current block is an N-channel image block, and the complexity level of the current block in the image to be processed is obtained. Obtaining the complexity level of each channel image block in the N-channel image block of the current block within the image to be processed from the code stream, wherein the N-channel image block includes at least a luminance channel image block and a chromaticity channel image block, and N is an integer greater than 0. This includes obtaining the complexity level of the current block based on the weighted complexity levels of each channel image block, The video decoding method according to feature 1.

5. Decoding the current block based on the quantization parameters is Based on the quantization parameters of the current block, the current block is dequantized, and the transformation coefficients after the dequantization of the current block are obtained. The transformation coefficients after inverse quantization of the current block are inversely transformed to obtain the residual block after the inverse transformation, This includes obtaining the current block after decoding based on the residual block after the inverse transformation, The video decoding method according to claim 1 or 2, characterized by the features described above.

6. After the step of decoding the current block based on the quantization parameters, further, The process involves obtaining the number of code stream chunks n divided into slices in the image to be processed, wherein n is a positive integer. Deinterleaving the code stream chunks corresponding to each slice in each slice row of the code stream, and obtaining n code stream chunks corresponding to each slice, wherein the n code stream chunks have at least two sizes, and at least one size is an integer multiple of 8, The video decoding method according to claim 1 or 2, characterized by the features described above.

7. Obtaining the number n of code stream chunks divided into slices in the image to be processed is: Obtain the slice width and slice height of each slice in the image to be processed, This includes determining the target number of transmission bits based on the slice width, slice height, and target number of bits occupied per pixel, and determining the number of code stream chunks based on the target number of transmission bits and chunk size value. The video decoding method according to feature 6.

8. The chunk size value is derived based on the number of coding units in the code stream chunk, the coding unit size, and the target pixel depth, and the chunk size value is an integer multiple of 8. The video decoding method according to feature 7.

9. Deinterleaving the code stream chunks corresponding to each slice in each slice row of the code stream and obtaining n code stream chunks corresponding to each slice is: The process includes sequentially deinterleaving code stream chunks corresponding to X slices in the same slice row based on the horizontal orientation of each slice row in the code stream, and obtaining n code stream chunks corresponding to each slice in the X slices, wherein X is a positive integer. The video decoding method according to feature 7.

10. When the actual length of the number of encoded bits is fixed, for the number of code stream chunks, the length of all code stream chunks except the kth code stream chunk is a first size, the length of the kth code stream chunk is a second size, the first size is an integer multiple of 8, the second size is determined based on the target number of transmitted bits, the first size, and the n code stream chunks, and k is a positive integer greater than 0 and less than or equal to n. The video decoding method according to feature 7.

11. The k-th code stream chunk is the last code stream chunk out of all code stream chunks. The video decoding method according to feature 10.

12. A video encoding method applied to the encoding side, The process involves obtaining complexity information of the current block in the image to be processed, wherein the complexity information of the current block is obtained by calculating at least one angular gradient of the current block based on the pixel values ​​of the current block. The quantization parameters of the current block are determined based on the complexity information of the current block, The process includes encoding the current block based on the quantization parameters and incorporating the complexity level of the current block into the code stream, wherein the complexity level of the current block is determined based on the complexity information of the current block. A video encoding method characterized by the following features.

13. A video decoding device, An acquisition module for obtaining the complexity level of the current block in an image to be processed, wherein the complexity level of the current block is obtained by analyzing a code stream; A decision module for determining the target bits of the current block based on the complexity level of the current block, and for determining the quantization parameters of the current block based on the target bits of the current block, Includes an encoding / decoding module for decoding the current block based on the quantization parameters, A video decoding device characterized by the following features.

14. The current block is an N-channel image block, The acquisition module is further used to acquire the complexity level of each channel image block in the N-channel image block of the current block within the image to be processed from the code stream, wherein the N-channel image block includes at least a luminance channel image block and a chromaticity channel image block, and N is an integer greater than 0. The aforementioned decision module further, Based on the weighted complexity levels of each channel image block, the complexity level of the current block is obtained. The target bit of the current block is determined based on the complexity level of the current block, The quantization parameters of the current block are determined based on the target bits of the current block, Used in The video decoding device according to feature 13.

15. The acquisition module is further used to acquire the number n of code stream chunks divided into slices in the image to be processed, wherein n is a positive integer. The encoding / decoding module is further used to deinterleave the code stream chunks corresponding to each slice in each slice row of the code stream, and to obtain n code stream chunks corresponding to each slice, wherein the n code stream chunks include at least two sizes, and at least one size is an integer multiple of 8. The video decoding device according to feature 13.

16. A video encoding device, An acquisition module for obtaining complexity information of the current block in an image to be processed, wherein the complexity information of the current block is obtained by calculating at least one angular gradient of the current block based on at least the pixel values ​​of the current block; A decision module for determining the quantization parameters of the current block based on the complexity information of the current block, An encoding / decoding module for encoding the current block based on the quantization parameters and incorporating the complexity level of the current block into a code stream, wherein the complexity level of the current block is determined based on the complexity information of the current block, A video encoding device characterized by the following features.

17. The current block is an N-channel image block, The aforementioned acquisition module further, The process involves obtaining the pixel value of the current block in the image to be processed, wherein the pixel value of the current block includes the pixel value of each channel image block in an N-channel image block, the N-channel image block includes at least a luminance channel image block and a chromaticity channel image block, and N is an integer greater than 0. The complexity information of each channel image block is obtained based on the pixel values ​​of each channel image block in the aforementioned N channel image block, Used in, The aforementioned decision module further, The complexity level of each channel image block is obtained based on the complexity information of each channel image block, The complexity level of the current block is obtained based on the weighted complexity levels of each channel image block. The target number of bits for the current block is determined based on the complexity level of the current block, This is used to determine the quantization parameters of the current block based on the target number of bits of the current block, The code stream obtained after encoding includes the complexity level of each channel image block. The video encoding apparatus according to feature 16.

18. The acquisition module further, This is used to obtain the number n of code stream chunks divided into slices in the image to be processed, where n code stream chunks have at least two sizes, at least one size is an integer multiple of 8, and n is a positive integer. The encoding / decoding module is further used to obtain a code stream by interleaving code stream chunks corresponding to each strip in each strip row within the image to be encoded. The video encoding apparatus according to feature 16.

19. A video decoder, Including the processor and memory, The memory stores instructions that can be executed by the processor. The processor is configured such that, upon executing the instruction, it causes the video decoder to implement the video decoding method described in claim 1 or 2. A video decoder characterized by the following features.

20. video encoder, Including the processor and memory, The memory stores instructions that can be executed by the processor. The processor is configured such that, upon executing the instruction, it causes the video encoder to implement the video encoding method described in claim 12. A video encoder characterized by the following features.

21. A video encoding and decoding system, Includes a video encoder and a video decoder, The video encoder is for performing the video encoding method described in claim 12, and the video decoder is for performing the video decoding method described in claim 1 or 2. A video encoding and decoding system characterized by the following:

22. A computer-readable storage medium, The program is stored in the computer-readable storage medium. When the program is executed on the computer, the computer is instructed to execute the video decoding method described in claim 1 or 2, or the video encoding method described in claim 12. A computer-readable storage medium characterized by the following features.

23. A computer program that includes instructions, When executed on a computer, the computer is instructed to perform the steps of the video decoding method described in claim 1 or 2, or the video encoding method described in claim 12. A computer program containing instructions characterized by the following features.

Images