Image encoding and decoding method, device and storage medium

Abstract

The present application provides an image encoding and decoding method, device and storage medium, which relates to the field of video encoding and decoding. The encoding method includes: obtaining an encoding unit; the encoding unit is an image block in the image to be processed; the encoding unit includes encoding blocks of multiple channels; the multiple channels include a first channel; the first channel is any one of the multiple channels; the encoding block of the first channel is encoded according to a first encoding mode; the first encoding mode is a mode for encoding sample values ​​in the encoding block of the first channel according to a first fixed-length code; the code length of the first fixed-length code is less than or equal to the image bit width of the image to be processed; the image bit width is used to represent the number of bits required to store each sample in the image to be processed. The method is applicable to the image encoding and decoding process, and is used to solve the problem of too large a substream buffer setting.

Description

Technical Field

[0001] The present application relates to the field of video coding and decoding, and in particular to an image coding and decoding method, device and storage medium. Background Art

[0002] In order to improve the performance of the encoder, a technique called sub-stream parallelism is proposed.

[0003] Substream parallelism refers to using multiple entropy encoders to encode syntax elements of different channels to obtain multiple substreams, filling the multiple substreams into their respective corresponding substream buffers, and interleaving the substreams in the substream buffers into bit streams (or code streams) according to preset interleaving rules.

[0004] However, considering the dependencies between sub-streams, different sub-stream buffers are filled with sub-streams at different speeds. At the same time, a sub-stream buffer with a faster filling speed has more sub-stream bits filled than a sub-stream buffer with a slower filling speed. In order to ensure the integrity of the filled data, all sub-stream buffers need to be set larger, which increases hardware costs. Summary of the invention

[0005] Based on the above technical problems, the present application provides an image encoding and decoding method, device and storage medium, which can be encoded through a variety of improved encoding and decoding modes, and the speed difference of different sub-streams during encoding can be reduced by reducing the theoretical expansion rate and actual expansion rate of the encoding unit, so as to reasonably configure the space of the sub-stream buffer and reduce the hardware cost.

[0006] Among them, the expansion rate of the coding unit may include a theoretical expansion rate and a current (CU) expansion rate (actual expansion rate). The theoretical expansion rate can be obtained through theoretical deduction after the codec is determined, and its value is greater than 1. If the number of bits of the substream encoded by the coding block of a certain channel in the coding unit is 0, this situation needs to be excluded when calculating the current expansion rate. Theoretical expansion rate = the number of bits of the CB with the largest theoretical bit number in the CU / the number of bits of the CB with the smallest theoretical bit number in the CU. Current (CU) expansion rate = the number of bits of the CB with the largest actual bit number in the current CU / the number of bits of the CB with the smallest actual bit number in the current CU.

[0007] Optionally, the related expansion rate may also include the current substream expansion rate: Current substream expansion rate = the number of data bits in the substream buffer with the largest number of data bits in the current multiple substream buffers / the number of data bits in the substream buffer with the smallest number of data bits in the current multiple substream buffers.

[0008] In a first aspect, the present application provides an image encoding method, the method comprising: obtaining a coding unit; the coding unit is an image block in an image to be processed; the coding unit comprises coding blocks of multiple channels; the multiple channels include a first channel; the first channel is any one of the multiple channels; encoding the coding block of the first channel according to a first coding mode; the first coding mode is a mode of encoding sample values ​​in the coding block of the first channel according to a first fixed-length code; the code length of the first fixed-length code is less than or equal to the image bit width of the image to be processed; the image bit width is used to represent the number of bits required to store each sample in the image to be processed.

[0009] It should be understood that the anti-expansion mode refers to a mode for directly encoding the original pixel values ​​in the coding blocks of multiple channels. Compared with other coding modes, the number of bits consumed by directly encoding the original pixel values ​​is usually larger. Therefore, the coding block with the largest theoretical number of bits in the theoretical expansion rate usually comes from the coding block encoded according to the anti-expansion mode. The embodiment of the present application reduces the theoretical number of bits (that is, the numerator in the fraction) of the coding block with the largest theoretical number of bits in the theoretical expansion rate by reducing the code length of the fixed-length code in the anti-expansion mode, thereby reducing the theoretical expansion rate. The theoretical expansion rate is low, and the speed difference of the coding block encoding sub-streams of each channel is small. Therefore, when setting the sub-stream buffer, the sub-stream buffer will neither be larger nor cause waste of sub-stream buffer space, thereby achieving a reasonable configuration of the sub-stream buffer.

[0010] In a second aspect, the present application provides an image decoding method, the method comprising: obtaining a code stream after encoding a coding unit; the coding unit is an image block in an image to be processed; the coding unit includes coding blocks of multiple channels; the multiple channels include a first channel; the first channel is any one of the multiple channels; the code stream includes multiple sub-streams corresponding to the multiple channels after the coding blocks of the multiple channels are encoded; the sub-stream corresponding to the first channel is decoded according to a first decoding mode; the first decoding mode is a mode of parsing sample values ​​from the sub-stream corresponding to the first channel according to a first fixed-length code; the code length of the first fixed-length code is less than or equal to the image bit width of the image to be processed; the image bit width is used to represent the number of bits required to store each sample in the image to be processed.

[0011] In a third aspect, the present application provides an image encoding method, which includes: obtaining a coding unit; the coding unit is an image block in an image to be processed; the coding unit includes coding blocks of multiple channels; determining a first total code length; the first total code length is the total code length of a first code stream obtained after the coding blocks of multiple channels are encoded according to their respective corresponding target coding modes; the target coding mode includes a first coding mode; the first coding mode is a mode for encoding sample values ​​in the coding block according to a first fixed-length code; the code length of the first fixed-length code is less than or equal to the image bit width of the image to be processed; the image bit width is used to characterize the number of bits required to store each sample in the image to be processed; when the first total code length is greater than or equal to the remaining size of the code stream buffer, the coding blocks of multiple channels are encoded in a fallback mode; the fallback mode and the mode flag of the first coding mode are the same.

[0012] It should be understood that when other coding modes are used for encoding, the residual after encoding may be too large, resulting in too large a number of bits of the coding block using the other coding modes, while the anti-expansion mode is encoded using fixed-length codes, and the total code length after encoding is fixed. The anti-expansion mode can avoid the above-mentioned situation of excessive residual errors. The embodiment of the present application provides a solution for informing the decoding end of the adopted coding mode by judging the remaining size of the code stream buffer when the mode flags of the fallback mode and the first coding mode (anti-expansion mode) are the same, thereby ensuring that the anti-expansion mode can always be selected, thereby reducing the theoretical expansion rate. The theoretical expansion rate is low, and the speed difference of the coding block encoding substreams of each channel is small. Therefore, when setting the substream buffer, it will neither cause the substream buffer to be larger nor cause waste of substream buffer space, thereby achieving reasonable configuration of the substream buffer.

[0013] Optionally, the method further comprises: encoding a mode flag in a plurality of sub-streams obtained by encoding the coding blocks of the plurality of channels; the mode flag is used to indicate the coding mode adopted by each of the coding blocks of the plurality of channels.

[0014] In one possible implementation, the multiple channels include a first channel; the first channel is any one of the multiple channels; a mode flag is encoded in a sub-stream obtained by encoding the coding blocks of the multiple channels, including: encoding a sub-mode flag in a sub-stream obtained by encoding the coding blocks of the first channel; the sub-mode flag is used to indicate the type of fallback mode adopted by the coding blocks of the multiple channels.

[0015] In another possible implementation, the multiple channels include a first channel; the first channel is an encoding mode flag in a sub-stream obtained by encoding the coding blocks of the multiple channels for any one channel among the multiple channels, including: encoding a first flag, a second flag, and a third flag in the sub-stream obtained by encoding the coding blocks of the first channel; the first flag is used to indicate that the coding blocks of the multiple channels are encoded in a first coding mode or a fallback mode; the second flag is used to indicate that the coding blocks of the multiple channels are encoded in a target mode, and the target mode is any one of the first coding mode and the fallback mode; when the second flag indicates that the target mode adopted by the coding blocks of the multiple channels is the fallback mode, the third flag is used to indicate the type of fallback mode adopted by the coding blocks of the multiple channels.

[0016] Optionally, the method also includes: determining a target encoding code length of the coding unit based on the remaining size of the code stream buffer and the target pixel depth BPP; the target encoding code length is used to indicate the code length required for encoding the coding unit; based on the encoding code length, determining the allocated code lengths of multiple channels; the allocated code length is used to indicate the code length required for encoding the residual of the coding blocks of multiple channels; and determining the encoding code lengths allocated to each of the multiple channels based on the average value of the allocated code lengths in the multiple channels.

[0017] In a fourth aspect, the present application provides an image decoding method, the method comprising: parsing a code stream after encoding a coding unit; the coding unit comprises coding blocks of multiple channels; if a mode flag is parsed from a substream obtained by encoding the coding blocks of multiple channels, and a first total code length is greater than the remaining size of the code stream buffer, determining that a target decoding mode of the substream is a fallback mode; the mode flag is used to indicate that the coding blocks of multiple channels are encoded using a first coding mode or a fallback mode; the first total code length is a total code length of a first code stream obtained after the coding blocks of multiple component channels are encoded according to their respective corresponding target coding modes and first coding modes; the target coding mode comprises a first coding mode; the first coding mode is a mode for encoding a sample value coding unit in a coding block according to a first fixed-length code; the code length of the first fixed-length code is less than or equal to the image bit width of the image to be processed; The image bit width is used to characterize the number of bits required to encode and store each sample in the image to be processed; the preset flag in the sub-stream is parsed to determine the target fallback mode; the target fallback mode is one of the fallback modes; the preset flag is used to indicate the position of the sub-mode flag; the sub-mode flag is used to indicate the type of fallback mode used when encoding the coding blocks of multiple channels; the fallback mode includes a first fallback mode and a second fallback mode; the sub-stream is decoded according to the target fallback mode.

[0018] Optionally, the method further includes: determining a target decoding code length of the coding unit based on the remaining size of the code stream buffer and the target pixel depth BPP; the target decoding code length is used to indicate the code length required for decoding the code stream of the coding unit; based on the decoding code length; determining the allocated code lengths of multiple channels; the allocated code length is used to indicate the code length required for decoding the residual of the code stream of the coding blocks of multiple channels; and determining the decoding code lengths allocated to each of the multiple channels according to the average value of the allocated code lengths in the multiple channels.

[0019] It should be understood that when the remaining code length is small, the current code length allocation mode based on the number of pixels of the coding unit may result in the chrominance channel being unable to be allocated a code length, resulting in a smaller number of bits after encoding the coding block of the chrominance channel, which increases the expansion rate. The image encoding method provided in the embodiment of the present application can allocate by converting the allocation unit from the number of pixels to the number of bits of the allocated code length, so that each channel can be allocated a code length for encoding, thereby reducing the theoretical expansion rate.

[0020] In a fifth aspect, the present application provides an image decoding method, the method comprising: parsing a code stream after encoding a coding unit; the coding unit comprises coding blocks of multiple channels; the multiple channels include a first channel; the first channel is any one of the multiple channels; parsing a first flag from a substream obtained by encoding the coding block of the first channel; the first flag is used to indicate that the coding blocks of the multiple channels are encoded using a first coding mode or a fallback mode; the first coding mode is a mode for encoding sampling values ​​in the coding block of the first channel according to a first fixed-length code; the code length of the first fixed-length code is less than or equal to the image bit width of the image to be processed; The image bit width is used to characterize the number of bits required to encode and store each sample in the image to be processed; the second flag is parsed from the substream obtained from the coding block encoding of the first channel; the second flag is used to indicate the target mode encoding of the coding blocks of multiple channels; the target mode is any one of the first encoding mode and the fallback mode; when the second flag indicates that the target mode adopted by the coding blocks of multiple channels is the fallback mode, the third flag is parsed from the substream obtained from the coding block encoding of the first channel; the target decoding mode of the multiple channels is determined according to the type of the fallback mode indicated by the third flag, and the substreams obtained from the coding blocks of the multiple channels are decoded according to the target decoding mode.

[0021] Optionally, the method further includes: determining a target decoding code length of the coding unit based on the remaining size of the code stream buffer and the target pixel depth BPP; the target decoding code length is used to indicate the code length required for decoding the code stream of the coding unit; based on the decoding code length; determining the allocated code lengths of multiple channels; the allocated code length is used to indicate the code length required for decoding the residual of the code stream of the coding blocks of multiple channels; and determining the decoding code lengths allocated to each of the multiple channels according to the average value of the allocated code lengths in the multiple channels.

[0022] In a sixth aspect, the present application provides an image encoding method, the method comprising: obtaining a coding unit; the coding unit comprises coding blocks of multiple channels; encoding the coding blocks of at least one channel among the multiple channels according to the intra-frame block copy IBC mode; obtaining a block vector BV of a reference prediction block; the BV of the reference prediction block is used to indicate the position of the reference prediction block in the encoded image block; the reference prediction block is used to represent the prediction value of the coding block encoded according to the IBC mode; encoding the BV of the reference prediction block in at least one sub-stream obtained by encoding the coding blocks of at least one channel by the IBC mode.

[0023] It should be understood that the number of bits of the substream corresponding to the luminance channel is usually larger, and the number of bits of the substream corresponding to the chrominance channel is usually smaller. In the embodiment of the present application, by transmitting the BV of the reference prediction block in the substream corresponding to the coding block of at least one channel when encoding in the IBC mode, the transmission data can be increased in the substream corresponding to the chrominance channel with a smaller number of bits, thereby increasing the denominator in the formula for calculating the theoretical expansion rate, thereby reducing the theoretical expansion rate.

[0024] In a possible implementation, encoding a coding block of at least one channel among multiple channels according to an intra-block copy (IBC) mode includes: determining a target coding mode of at least one channel using a rate-distortion optimization decision; the target coding mode includes an IBC mode; encoding a coding block of at least one channel according to the target coding mode.

[0025] In a possible implementation, encoding a coding block of at least one channel among multiple channels according to an intra-block copy (IBC) mode includes: encoding the coding blocks of multiple channels according to the IBC mode. Encoding a BV of a reference prediction block in at least one substream obtained by encoding the coding block of at least one channel in the IBC mode includes: encoding a BV of a reference prediction block in a substream obtained by encoding the coding block of each channel among the multiple channels in the IBC mode.

[0026] In a possible implementation, encoding the BV of the reference prediction block in a substream obtained by encoding the coding block of each channel in multiple channels in IBC mode includes: encoding the BV of the reference prediction block in a code stream obtained by encoding the coding block of each channel in multiple channels in IBC mode according to a preset ratio.

[0027] In a seventh aspect, the present application provides an image decoding method, the method comprising: parsing a code stream after encoding a coding unit; the coding unit comprises coding blocks of multiple channels; the code stream comprises multiple sub-streams corresponding one-to-one to the multiple channels after the coding blocks of the multiple channels are encoded; based on a block vector BV of a reference prediction block parsed from at least one of the multiple sub-streams, determining a position of a reference prediction block in the multiple sub-streams; the reference prediction block is used to represent a prediction value of a coding block encoded in an intra-block copy IBC mode; the BV of the reference prediction block is used to indicate a position of the reference prediction block in an encoded image block; based on the position information of the reference prediction block, determining a prediction value of a decoding block decoded in the IBC mode; based on the prediction value, reconstructing the decoding block decoded in the IBC mode.

[0028] Optionally, the BV of the reference prediction block is encoded in multiple sub-streams; the BV of the reference prediction block is obtained when encoding coding blocks of multiple channels according to the IBC mode.

[0029] In an eighth aspect, the present application provides an image encoding method, the method comprising: obtaining a processing coefficient corresponding to a coding unit; the processing coefficient comprises one or more of a residual coefficient and a transform coefficient; dividing the processing coefficient into multiple groups according to a number threshold, the number of processing coefficients in each group of the multiple groups of processing coefficients being less than or equal to the number threshold.

[0030] It should be understood that compared with the residual skip mode, the image encoding method provided in the embodiment of the present application can group the processing coefficients during encoding. Each group of processing coefficients after grouping needs to add the header information of the group of processing coefficients to describe the details of the group of processing coefficients during transmission. Compared with the current use of 1-bit data to represent the residual skip, the denominator in the calculation formula of the theoretical expansion rate is increased, and the theoretical expansion rate is reduced.

[0031] In a ninth aspect, the present application provides an image decoding method, the method comprising: parsing a code stream after encoding a coding unit, determining a processing coefficient corresponding to the coding unit; the processing coefficient comprises one or more of a residual coefficient and a transform coefficient; the processing coefficient comprises multiple groups; the number of processing coefficients in each group is less than or equal to a number threshold; and decoding the code stream based on the processing coefficient.

[0032] In a tenth aspect, the present application provides an image encoding method, the method comprising: obtaining a coding unit; the coding unit comprises coding blocks of P channels; P is an integer greater than or equal to 2; obtaining complexity information of the coding blocks of each of the P channels; the complexity information is used to characterize the degree of difference in pixel values ​​of the coding blocks of each channel; encoding the complexity information of the coding blocks of each channel in a sub-stream obtained by encoding the coding blocks of the P channels.

[0033] It should be understood that in an optional implementation scheme, the complexity information is not transmitted in the third substream corresponding to the coding block of the second chrominance channel. The image coding method provided in the embodiment of the present application increases the number of bits in the substream with a smaller number of bits by adding coding complexity information in the third substream, thereby increasing the denominator in the above theoretical expansion rate calculation formula and reducing the theoretical expansion rate.

[0034] In one possible implementation, the complexity information includes a complexity level; encoding the complexity information of the coding block of each channel in a sub-stream obtained by encoding the coding blocks of P channels includes: encoding the complexity level of the coding block of each channel in the sub-stream obtained by encoding the coding blocks of P channels.

[0035] In a possible implementation, the complexity information includes a complexity level and a first reference coefficient; the first reference coefficient is used to characterize the proportional relationship between the complexity levels of coding blocks of different channels; the complexity information of the coding blocks of each channel is encoded in a sub-stream obtained by encoding the coding blocks of P channels, including: in a sub-stream obtained by encoding the coding blocks of Q channels among the P channels, the complexity levels of the coding blocks of Q channels are respectively encoded; Q is an integer less than P; and the first reference coefficient is encoded in a sub-stream obtained by encoding the coding blocks of PQ channels among the P channels.

[0036] In a possible implementation, the complexity information includes a complexity level, a reference complexity level, and a second reference coefficient; the reference complexity level includes any one of the following: a first complexity level, a second complexity level, and a third complexity level; the first complexity level is the maximum value of the complexity levels of the coding blocks of PQ channels among P channels; Q is an integer less than P; the second complexity level is the minimum value of the complexity levels of the coding blocks of PQ channels among P channels; the third complexity level is the average value of the complexity levels of the coding blocks of PQ channels among P channels; the second reference coefficient is used to characterize the size relationship, and / or proportional relationship, between the complexity levels of the coding blocks of PQ channels among P channels; the complexity information of the coding blocks of each channel encoded in the sub-stream obtained by encoding the coding blocks of P channels includes: the complexity levels of the coding blocks of Q channels respectively encoded in the sub-stream obtained by encoding the coding blocks of Q channels among P channels; the reference complexity level and the second reference coefficient are encoded in the sub-stream obtained by encoding the coding blocks of PQ channels among P channels.

[0037] In an eleventh aspect, the present application provides an image decoding method, the method comprising: parsing a code stream after encoding a coding unit; the coding unit comprises coding blocks of P channels; P is an integer greater than or equal to 2; the code stream comprises a plurality of sub-streams corresponding one to one to the P channels after the coding blocks of the P channels are encoded; parsing complexity information of the coding blocks of each channel in the sub-streams obtained by encoding the coding blocks of the P channels; the complexity information is used to characterize the degree of difference in pixel values ​​of the coding blocks of each channel; based on the complexity information of the coding blocks of each channel, determining a quantization parameter of the coding blocks of each channel; based on the quantization parameter of the coding blocks of each channel, decoding the code stream.

[0038] In one possible implementation, the complexity information includes a complexity level; parsing the complexity information of the coding blocks of each channel in the sub-streams obtained by encoding the coding blocks of the P channels includes: parsing the complexity level of the coding blocks of each channel in the sub-streams obtained by encoding the coding blocks of the P channels.

[0039] In a possible implementation, complexity information includes complexity level and a first reference coefficient; the first reference coefficient is used to characterize the proportional relationship between complexity levels of coding blocks of different channels; the complexity information of the coding blocks of each channel is parsed in the sub-stream obtained by encoding the coding blocks of P channels, including: parsing the complexity level of the coding blocks of Q channels in the sub-stream obtained by encoding the coding blocks of Q channels in the P channels respectively; Q is an integer less than P; parsing the first reference coefficient in the sub-stream obtained by encoding the coding blocks of PQ channels in the P channels; based on the first reference coefficient and the complexity level of the coding blocks of the Q channels, determining the complexity level of the coding blocks of the PQ channels.

[0040] In a possible implementation, the complexity information includes a complexity level, a reference complexity level, and a second reference coefficient; the reference complexity level includes any one of the following: a first complexity level, a second complexity level, and a third complexity level; the first complexity level is the maximum value of the complexity levels of the coding blocks of PQ channels among the P channels; Q is an integer less than P; the second complexity level is the minimum value of the complexity levels of the coding blocks of PQ channels among the P channels; the third complexity level is the average value of the complexity levels of the coding blocks of PQ channels among the P channels; the second reference coefficient is used to characterize the size relationship and / or proportional relationship between the complexity levels of the coding blocks of PQ channels among the P channels; parsing the complexity information of the coding blocks of each channel in the substream obtained by encoding the coding blocks of the P channels includes: The complexity levels of the coding blocks of the Q channels in the P channels are respectively parsed in the substreams obtained by encoding the coding blocks of the Q channels; the reference complexity level and the second reference coefficient are parsed in the substreams obtained by encoding the coding blocks of the PQ channels in the P channels; and the complexity levels of the coding blocks of the PQ channels are determined based on the complexity levels of the coding blocks of the Q channels and the second reference coefficient.

[0041] In a twelfth aspect, the present application provides an image encoding method, the method comprising: obtaining a coding unit; the coding unit comprises coding blocks of multiple channels; the coding unit is an image block in an image to be processed; when the image format of the image to be processed is a preset format, the sub-streams obtained by encoding the coding blocks of at least two preset channels among the multiple channels are merged into a merged sub-stream.

[0042] It should be understood that the number of bits after encoding the syntax elements and quantized transform coefficients of the two chrominance channels is usually smaller than that of the luminance channel. Fusion of the syntax elements of the two chrominance channels in the second substream for transmission can reduce the difference in the number of bits between the first substream and the second substream, thereby reducing the theoretical expansion rate.

[0043] In a thirteenth aspect, the present application provides an image decoding method, the method comprising: parsing a code stream after encoding a coding unit; the coding unit comprises coding blocks of multiple channels; the coding unit is an image block in an image to be processed; determining a merged substream; the merged substream is obtained by merging substreams encoded by coding blocks of at least two preset channels in multiple channels when the image format of the image to be processed is a preset format; decoding the code stream based on the merging of at least two substreams to be merged.

[0044] In a fourteenth aspect, the present application provides an image encoding method, the method comprising: obtaining a coding unit; the coding unit comprises coding blocks of multiple channels; the channel encodes a preset codeword in a target substream that meets a preset condition until the target substream does not meet the preset condition; the target substream is a substream among multiple substreams; the multiple substreams are code streams obtained by encoding the coding blocks of multiple channels.

[0045] It should be understood that filling the preset codeword in the substream with a smaller number of bits can directly increase the denominator of the expansion rate, thereby reducing the actual expansion rate of the coding unit.

[0046] Optionally, the preset expansion rate includes a first preset expansion rate; the current expansion rate is equal to the quotient of the number of bits of the maximum sub-stream and the number of bits of the minimum sub-stream; the maximum sub-stream is the sub-stream with the largest number of bits among the multiple sub-streams obtained by encoding the coding blocks of multiple channels; the minimum sub-stream is the sub-stream with the smallest number of bits among the multiple sub-streams obtained by encoding the coding blocks of multiple channels.

[0047] Optionally, the method further includes: determining a current expansion ratio; when the current expansion ratio is greater than a first preset expansion ratio, encoding a preset codeword in the smallest substream so that the current expansion ratio is less than or equal to the first preset expansion ratio.

[0048] Optionally, the preset expansion rate includes a second preset expansion rate; the current expansion rate is equal to the quotient of the number of bits of the coding block with the largest number of bits and the number of bits of the coding block with the smallest number of bits in the coded blocks of the multiple channels.

[0049] Optionally, the method further includes: determining a current expansion rate; when the current expansion rate is greater than a second preset expansion rate, encoding a preset codeword in a coding block with a minimum number of bits, so that the current expansion rate is less than or equal to the second preset expansion rate.

[0050] In a fifteenth aspect, the present application provides an image decoding method, the method comprising: parsing a code stream after encoding a coding unit; the coding unit comprises coding blocks of multiple channels; based on the code stream, determining the number of preset code words; based on the number of preset code words, decoding the code stream.

[0051] In a sixteenth aspect, the present application provides an image encoding and decoding device, which includes various modules used for the methods described in the first to fifteenth aspects above.

[0052] In the seventeenth aspect, the present application provides an image encoding device, which includes a processor and a memory; the memory stores instructions executable by the processor; when the processor is configured to execute the instructions, the image encoding and decoding device implements the image encoding method in the above-mentioned first to fifteenth aspects.

[0053] In an eighteenth aspect, the present application provides an image decoding device, which includes a processor and a memory; the memory stores instructions executable by the processor; when the processor is configured to execute the instructions, the image encoding and decoding device implements the image decoding method in the above-mentioned first to fifteenth aspects.

[0054] In the nineteenth aspect, the present application provides a computer program product. When the computer program product runs on an image coding and decoding device, the image coding and decoding device executes the steps of the related method described in any one of the first to fifteenth aspects above, so as to implement the method described in any one of the first to fifteenth aspects above.

[0055] In the twentieth aspect, the present application provides a readable storage medium, which includes: software instructions; when the software instructions are executed in an image coding and decoding device, the image coding and decoding device implements the method described in any one of the first to fifteenth aspects above.

[0056] In the twenty-first aspect, the present application provides a chip comprising a processor and an interface, wherein the processor is coupled to a memory through the interface, and when the processor executes a computer program in the memory or an image encoding and decoding device executes instructions, the method described in any one of the first to fifteenth aspects is executed. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0058] Figure 1 It is a schematic diagram of the framework of sub-stream parallel technology;

[0059] Figure 2 It is a schematic diagram of the encoding terminal stream interleaving process;

[0060] Figure 3 is a schematic diagram of the format of a sub-stream interleaving unit;

[0061] Figure 4 Schematic diagram of the reverse sub-stream interleaving process at the decoding end;

[0062] Figure 5 Another schematic diagram of the encoding terminal stream interleaving process;

[0063] Figure 6 A schematic diagram of the composition of a video encoding and decoding system provided in an embodiment of the present application;

[0064] Figure 7 A schematic diagram of the composition of the video encoder 102 provided in an embodiment of the present application;

[0065] Figure 8 A schematic diagram of the structure of the video decoder 112 provided in an embodiment of the present application;

[0066] Fig. 9 A schematic diagram of a video encoding and decoding process provided in an embodiment of the present application;

[0067] Fig.10 A schematic diagram of the composition of the encoding and decoding device provided in an embodiment of the present application;

[0068] Fig.11 A schematic diagram of a flow chart of an image encoding method provided in an embodiment of the present application;

[0069] Fig.12 A flowchart of an image decoding method provided in an embodiment of the present application;

[0070] Fig.13 A schematic diagram of a flow chart of another image decoding method provided in an embodiment of the present application;

[0071] Fig.14 A schematic diagram of a flow chart of another image decoding method provided in an embodiment of the present application;

[0072] Fig.15 A flowchart of another image decoding method provided in an embodiment of the present application;

[0073] Fig.16 A flowchart of another image encoding method provided in an embodiment of the present application;

[0074] Fig.17 A flowchart of another image encoding method provided in an embodiment of the present application;

[0075] Fig.18 A flowchart of another image decoding method provided in an embodiment of the present application;

[0076] Fig.19 A flowchart of another image encoding method provided in an embodiment of the present application;

[0077] Fig. 20 A flowchart of another image decoding method provided in an embodiment of the present application;

[0078] Fig.21 A flowchart of another image encoding method provided in an embodiment of the present application;

[0079] Fig. 22A flowchart of another image decoding method provided in an embodiment of the present application;

[0080] Fig.23 A flowchart of another image encoding method provided in an embodiment of the present application;

[0081] Fig.24 A flowchart of another image decoding method provided in an embodiment of the present application;

[0082] Fig.25 A flowchart of another image encoding method provided in an embodiment of the present application;

[0083] Fig.26 A flowchart of another image decoding method provided in an embodiment of the present application;

[0084] Fig. 27 A flowchart of another image encoding method provided in an embodiment of the present application;

[0085] Fig.28 A flowchart of another image decoding method provided in an embodiment of the present application;

[0086] Fig.29 A schematic diagram of the composition of an image encoding and decoding device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0087] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0088] In the description of this application, unless otherwise specified, " / " means "or", for example, A / B can mean A or B. "And / or" in this article is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, "at least one" means one or more, and "plurality" means two or more. The words "first", "second", etc. do not limit the quantity and execution order, and the words "first", "second", etc. do not limit them to be different.

[0089] It should be noted that, in this application, words such as "exemplarily" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplarily" or "for example" in this application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplarily" or "for example" is intended to present related concepts in a specific way.

[0090] In order to improve the performance of the encoder, a technology called sub-stream parallelism (or also called sub-stream interleaving) is proposed.

[0091] For the encoding end, substream parallelism means that the encoding end encodes the syntax elements of the coding blocks (CB) of different channels (such as the luminance channel, the first chrominance channel, and the second chrominance channel) of the coding unit (CU) using multiple entropy encoders to obtain multiple substreams, and interleaves the multiple substreams into a compressed bit stream with a fixed-size data packet. Correspondingly, for the decoding end, substream parallelism means that the decoding end uses different entropy decoders to decode different substreams in parallel.

[0092] For example, Figure 1 This is a schematic diagram of the framework of sub-stream parallel technology. Figure 1 As shown, taking the encoding end as an example, the specific application timing of the sub-stream parallel technology is after the syntax elements (such as transform coefficients and quantization coefficients, etc.) are encoded. Figure 1 The encoding process of other parts in the video encoding and decoding system provided in the following embodiment of the present application can be referred to, and will not be repeated here.

[0093] For example, Figure 2 Figure 1 is a schematic diagram of the coding terminal stream interleaving process. Figure 2 As shown, taking the image block to be encoded including three channels as an example, the encoding module (such as the prediction module, the transform module, and the quantization module, etc.) can output the syntax elements and quantized transform coefficients of the three channels, and then the entropy encoder 1, the entropy encoder 2, and the entropy encoder 3 respectively encode the syntax elements and quantized transform coefficients of the three channels to obtain the substream corresponding to each channel, and the substreams corresponding to the three channels are pressed into the encoding substream buffer 1, the encoding substream buffer 2, and the encoding substream buffer 3. The substream interleaving module can interleave the substreams in the encoding substream buffer 1, the encoding substream buffer 2, and the encoding substream buffer 3, and finally output a bit stream (or also called a code stream) after the interleaving of multiple substreams.

[0094] For example, Figure 3 FIG. 1 is a schematic diagram of the format of a sub-stream interleaving unit. Figure 3As shown, a substream may be composed of a substream interleaving unit, which may also be called a substream segment. The length of a substream segment is N bits, and the substream segment includes an M-bit data header and an NM-bit data body.

[0095] The data header is used to indicate the substream to which the current substream slice belongs. N can be 512, and M can be 2.

[0096] For example, Figure 4 Figure 1 is a schematic diagram of the reverse sub-stream interleaving process at the decoding end. Figure 4 As shown, taking the case where the image block to be encoded by the encoder includes three channels, after the bit stream output by the encoder is input to the decoder, the substream interleaving module in the decoder can first perform a reverse substream interleaving process on the bit stream, decompose the bit stream into substreams corresponding to the three channels, and push the substreams corresponding to the three channels into decoding substream buffer 1, decoding substream buffer 2, and decoding substream buffer 3. Figure 3 Taking the substream slice shown in the figure as an example, the decoding end can extract N-bit-long data packets from the bit stream each time, parse the M-bit data header, obtain the target substream to which the current substream slice belongs, and put the remaining data body in the current substream slice into the decoding substream buffer corresponding to the target substream.

[0097] Entropy decoder 1 can decode the substream in decoding substream buffer 1 to obtain syntax elements and quantized transform coefficients of one channel; entropy decoder 2 can decode the substream in decoding substream buffer 2 to obtain syntax elements and quantized transform coefficients of another channel; entropy decoder 3 can decode the substream in decoding substream buffer 3 to obtain syntax elements and quantized transform coefficients of yet another channel. Finally, the syntax elements and quantized transform coefficients of each of the three channels are input into the subsequent decoding module for decoding processing to obtain a decoded image.

[0098] The following introduces the process of sub-stream interleaving by taking the encoding end as an example.

[0099] Each substream slice in the coding substream buffer includes coding bits generated by encoding at least one image block. When interleaving substreams, the image blocks corresponding to the initial bits of the data body of each substream slice can be sequentially marked, and in the substream interleaving process, different substreams are interleaved according to the sequential markings.

[0100] Optionally, the sub-stream interleaving process may mark image blocks via a block count queue.

[0101] For example, the block count queue is implemented in a first-in-first-out queue. The encoder sets the count of the currently encoded image block, and sets a block count queue counter queue [ss_idx] for each substream. When encoding each stripe starts, the block count is initialized to 0, each counter queue [ss_idx] is initialized to empty, and then a 0 is pushed into each counter queue [ss_idx].

[0102] After each image block (or coding unit (CU)) completes the encoding operation, the block count queue is updated. The update process is as follows:

[0103] Step 1: Set the count of the currently encoded image block to +1, that is, block count +=1.

[0104] Step 2: Select a substream ss_idx.

[0105] Step 3: Calculate the number of substream slices num_in_buffer[ss_idx] that can be constructed in the coded substream buffer corresponding to the substream ss_idx. Assume that the coded substream buffer corresponding to the substream ss_idx is buffer[ss_idx], and the amount of data included in buffer[ss_idx] is buffer[ss_idx].fullness. Similarly, taking the substream slice size as N bits and the substream slice including an M-bit data header as shown in the above-mentioned step 3 as an example, num_in_buffer[ss_idx] can be calculated according to the following formula (1).

[0106] num_in_buffer[ss_idx]=buffer[ss_idx].fullness / (N–M) Formula (1)

[0107] Here, “ / ” indicates integer division.

[0108] Step 4: Compare the current block count queue length num_in_queue[ss_idx] with the number of substream slices that can be constructed in the coded substream buffer num_in_buffer[ss_idx]. If the two are equal, push the count of the currently coded image block into the block count queue, that is, counter_queue[ss_idx].push(block_count).

[0109] Step 4: Return to step 2 to process the next sub-flow until all sub-flows are processed.

[0110] After the block count queue is updated, the encoder can interleave the substreams in each encoded substream buffer. The interleaving process is as follows:

[0111] Step 1. Select a substream ss_idx.

[0112] Step 2: Determine whether the amount of data buffer[ss_idx].fullness included in the coded substream buffer buffer[ss_idx] corresponding to the substream ss_idx is greater than or equal to NM. If yes, execute step 3; if no, execute step 6.

[0113] Step 3: Determine whether the head element value in the block count queue of the substream ss_idx is the minimum value in the block count queues of all substreams. If yes, execute step 4; if no, execute step 6.

[0114] Step 4: Construct a substream slice with the data in the current encoding substream buffer. For example, extract NM-bit data from the encoding substream buffer buffer[ss_idx], add an M-bit data header, the data in the header is ss_idx, concatenate the M-bit data header and the extracted NM-bit data into an N-bit substream slice, and send the substream slice to the bit stream finally output by the encoding end.

[0115] Step 5: Pop (or delete) the head element of the block counter queue of the substream ss_idx, that is, counter_queue[ss_idx].pop().

[0116] Step 6: Return to step 1 and process the next sub-stream until all sub-streams are processed.

[0117] Optionally, if the current image block is the last image block of a slice, the encoder may further perform the following steps after the above interleaving process to pack the remaining data in the encoding substream buffer:

[0118] Step 1: Determine whether there is at least one non-empty buffer in all the current coded substreams. If yes, execute step 2; if no, end.

[0119] Step 2: Select a substream ss_idx.

[0120] Step 3: Determine whether the head element value of the block count queue of the substream ss_idx is the minimum value of the block count queues of all substreams. If yes, execute step 4; if no, execute step 6.

[0121] Step 4: If the amount of data in the encoding substream buffer buffer[ss_idx] corresponding to the substream ss_idx is less than NM bits, fill 0 into the encoding substream buffer buffer[ss_idx] until the data in the encoding substream buffer buffer[ss_idx] reaches NM bits. At the same time, pop (or delete) the head element value of the block counter queue of the substream, that is, counter_queue[ss_idx].pop(), and push a MAX_INT representing the maximum value in the data range, that is, counter_queue[ss_idx].push(MAX_INT).

[0122] Step 5: Construct a sub-stream slice. Step 5 here can refer to step 4 in the above sub-stream interleaving process, and will not be described in detail.

[0123] Step 6: Return to step 2 to process the next sub-flow. If all sub-flows have been processed, return to step 1.

[0124] For example, Figure 5 Another schematic diagram of the encoding terminal stream interleaving process is shown in FIG. Figure 5 As shown, taking the image block to be encoded including three channels as an example, the encoding sub-stream buffer can include encoding sub-stream buffer 1, encoding sub-stream buffer 2, and encoding sub-stream buffer 3 respectively.

[0125] The substream slices in the coding substream buffer 1 are 1_1, 1_2, 1_3, and 1_4 from front to back. The marks in the block count queue 1 of the substream 1 corresponding to the coding substream buffer 1 are 12, 13, 27, and 28. The substream slices in the coding substream buffer 2 are 2_1 and 2_2 from front to back. The marks in the block count queue 2 of the substream 2 corresponding to the coding substream buffer 2 are 5 and 71. The substream slices in the coding substream buffer 3 are 3_1, 3_2, and 3_3 from front to back. The marks in the block count queue 3 of the substream 3 corresponding to the coding substream buffer 3 are 6, 13, and 25. Then the substream interleaving module can interleave the substream slices in the coding substream buffer 1, the coding substream buffer 2, and the coding substream buffer 3 according to the mark order in the block count queue 1, the block count queue 2, and the block count queue 3.

[0126] The order of sub-stream slices in the interleaved code stream is: sub-stream slice 2_1 corresponding to the minimum mark 5, sub-stream slice 3_1 corresponding to mark 6, sub-stream slice 1_1 corresponding to mark 12, sub-stream slice 1_2 corresponding to mark 13, sub-stream slice 3_2 corresponding to mark 13, sub-stream slice 3_3 corresponding to mark 25, sub-stream slice 1_3 corresponding to mark 27, sub-stream slice 1_4 corresponding to mark 28, and sub-stream slice 2_2 corresponding to mark 71.

[0127] However, there are dependencies when multiple sub-streams are interleaved. Taking the encoding end as an example, the filling speeds of sub-streams in different sub-stream buffers are different. At the same time, the sub-stream buffer with a faster filling speed has more sub-stream bits than the sub-stream buffer with a slower filling speed. The sub-stream buffer with a faster filling speed continues to fill data while waiting for the sub-stream buffer with a slower filling speed to fill enough sub-stream slices. In order to ensure the integrity of the filled data, all sub-stream buffers need to be set larger, which increases the hardware cost.

[0128] In this case, the embodiments of the present application provide an image encoding and decoding method, device and storage medium, which can be encoded through a variety of improved encoding modes, so as to control the number of bits of the maximum encoding block and the minimum encoding block, so that the theoretical expansion rate of the encoded block to be encoded is reduced, and the filling speed difference between different sub-stream buffers is reduced, thereby reducing the preset space size of the sub-stream buffer and reducing the hardware cost.

[0129] The following is an introduction with reference to the accompanying drawings.

[0130] Figure 6 The following is a schematic diagram of the composition of the video encoding and decoding system provided in the embodiment of the present application. Figure 6 As shown, the video coding and decoding system includes a source device 10 and a destination device 11 .

[0131] The source device 10 generates encoded video data, and the source device 10 may also be referred to as an encoding end, a video encoding end, a video encoding device, or a video encoding apparatus, etc. The destination device 11 may decode the encoded video data generated by the source device 10, and the destination device 11 may also be referred to as a decoding end, a video decoding end, a video decoding device, or a video decoding apparatus, etc. The source device 10 and / or the destination device 11 may include at least one processor and a memory coupled to the at least one processor. The memory may include but is not limited to a read-only memory (ROM), a random access memory (RAM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, or any other medium that can be used to store the desired program code in the form of instructions or data structures that can be accessed by a computer, and the embodiments of the present application do not specifically limit this.

[0132] The source device 10 and the destination device 11 may include a variety of devices, including electronic devices such as desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, telephone handsets such as so-called "smart" phones, televisions, cameras, display devices, digital media players, video game consoles, in-vehicle computers, or the like.

[0133] The destination device 11 may receive the encoded video data from the source device 10 via the link 12. The link 12 may include one or more media and / or devices capable of moving the encoded video data from the source device 10 to the destination device 11. In one example, the link 12 may include one or more communication media that enable the source device 10 to transmit the encoded video data directly to the destination device 11 in real time. In this example, the source device 10 may modulate the encoded video data according to a communication standard (e.g., a wireless communication protocol) and may transmit the modulated video data to the destination device 11. The one or more communication media may include wireless and / or wired communication media, such as a radio frequency (RF) spectrum, one or more physical transmission lines. The one or more communication media may form part of a packet-based network, such as a local area network, a wide area network, or a global network (e.g., the Internet). The one or more communication media may include routers, switches, base stations, or other devices that enable communication from the source device 10 to the destination device 11.

[0134] In another example, the source device 10 may output the encoded video data from the output interface 103 to the storage device 13. Similarly, the destination device 11 may access the encoded video data from the storage device 13 through the input interface 113. The storage device 13 may include a variety of locally accessible data storage media, such as a Blu-ray disc, a high-density digital video disc (DVD), a compact disc read-only memory (CD-ROM), a flash memory, or other suitable digital storage media for storing encoded video data.

[0135] In another example, the storage device 13 may correspond to a file server or another intermediate storage device storing the encoded video data generated by the source device 10. In this example, the destination device 11 may obtain the video data stored therein from the storage device 13 via streaming or downloading. The file server may be any type of server capable of storing the encoded video data and transmitting the encoded video data to the destination device 11. For example, the file server may include a World Wide Web (Web) server (e.g., for a website), a File Transfer Protocol (FTP) server, a Network Attached Storage (NAS) device, and a local disk drive.

[0136] The destination device 11 can access the encoded video data through any standard data connection (e.g., an Internet connection). Example types of data connections include wireless channels suitable for accessing the encoded video data stored on a file server, wired connections (e.g., cable modems, etc.), or a combination of both. The encoded video data can be transmitted from the file server in the form of streaming, downloading, or a combination of both.

[0137] It should be noted that the image encoding and decoding method provided in the embodiment of the present application is not limited to wireless application scenarios.

[0138] Exemplarily, the image encoding and decoding method provided in the embodiments of the present application can be applied to video encoding and decoding supporting the following multimedia applications: over-the-air television broadcasting, cable television transmission, satellite television transmission, streaming video transmission (e.g., via the Internet), encoding of video data stored on a data storage medium, decoding of video data stored on a data storage medium, or other applications. In some examples, the video encoding and decoding system can be configured to support one-way or two-way video transmission to support applications such as video streaming, video playback, video broadcasting and / or video telephony.

[0139] It should be noted that Figure 6 The video coding and decoding system shown is only an example of a video coding and decoding system, and is not a limitation of the video coding and decoding system in the present application. The image coding and decoding method provided in the present application can also be applied to a scenario where there is no data communication between the coding device and the decoding device. In other embodiments, the video data to be encoded or the encoded video data can be retrieved from a local memory, or can be streamed on a network, etc. The video encoding device can encode the video data to be encoded and store the encoded video data in a memory, and the video decoding device can also obtain the encoded video data from the memory and decode the encoded video data.

[0140] exist Figure 6 In an embodiment of the present invention, source device 10 includes a video source 101, a video encoder 102, and an output interface 103. In some embodiments, output interface 103 may include a modulator / demodulator (modem) and / or a transmitter. Video source 101 may include a video capture device (e.g., a camera), a video archive containing previously captured video data, a video input interface for receiving video data from a video content provider, and / or a computer graphics system for generating video data, or a combination of such sources of video data.

[0141] The video encoder 102 may encode the video data from the video source 101. In some embodiments, the source device 10 transmits the encoded video data directly to the destination device 11 via the output interface 103. In other embodiments, the encoded video data may also be stored on the storage device 13 for later access by the destination device 11 for decoding and / or playback.

[0142] exist Figure 6 In an embodiment of the present invention, the destination device 11 includes a display device 111, a video decoder 112, and an input interface 113. In some embodiments, the input interface 113 includes a receiver and / or a modem. The input interface 113 can receive the encoded video data via the link 12 and / or from the storage device 13. The display device 111 can be integrated with the destination device 11 or can be external to the destination device 11. In general, the display device 111 displays the decoded video data. The display device 111 can include a variety of display devices, for example, a liquid crystal display, a plasma display, an organic light emitting diode display, or other types of display devices.

[0143] Alternatively, the video encoder 102 and the video decoder 112 may each be integrated with an audio encoder and decoder, and may include appropriate multiplexer-demultiplexer units or other hardware and software to handle the encoding of both audio and video in a common data stream or in separate data streams.

[0144] The video encoder 102 and the video decoder 112 may include at least one microprocessor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), discrete logic, hardware, or any combination thereof. If the decoding method provided in the present application is implemented in software, the instructions for the software may be stored in a suitable non-volatile computer-readable storage medium, and at least one processor may be used to execute the instructions to implement the present application.

[0145] The video encoder 102 and the video decoder 112 in the present application may operate according to a video compression standard (eg, HEVC), or may operate according to other industry standards, which are not specifically limited in the present application.

[0146] Figure 7 FIG. 1 is a schematic diagram of the composition of the video encoder 102 provided in an embodiment of the present application. Figure 7 As shown, the video encoder 102 can perform prediction, transformation, quantization, entropy coding, and substream interleaving processes in the prediction module 21, the transformation module 22, the quantization module 23, the entropy coding module 24, the coded substream buffer 25, and the substream interleaving module 26, respectively. Figure 1 The video encoder 102 further includes a pre-processing module 20 and a summer 202, wherein the pre-processing module 20 includes a segmentation module and a bit rate control module. For video block reconstruction, the video encoder 102 also includes an inverse quantization module 27, an inverse transform module 28, a summer 201 and a reference image memory 29.

[0147] like Figure 7 As shown, the video encoder 102 receives video data, and the preprocessing module 20 receives input parameters of the video data. Among them, the input parameters include information such as the resolution of the image in the video data, the sampling format of the image, the pixel depth (bits perpixel, BPP), and the bit width (or it can also be called the image bit width). Among them, BPP refers to the number of bits occupied by a unit pixel. The bit width refers to the number of bits occupied by a pixel channel in a unit pixel. For example, if the values ​​of three YUV pixel channels represent a pixel, and each pixel channel occupies 8 bits (bits), the bit width of the pixel is 8, and the BPP of the pixel is 3 × 8 = 24 bits.

[0148] The segmentation module in the preprocessing module 20 divides the image into original blocks (or may also be referred to as coding units (CU)). The original block (or may also be referred to as coding unit (CU)) may include coding blocks of multiple channels. For example, the multiple channels may be RGB channels or YUV channels, etc. The embodiment of the present application is not limited to this. Optionally, this segmentation may also include segmentation into slices, image blocks or other larger units, and segmentation of video blocks according to the quadtree structure of the largest coding unit (LCU) and CU. Exemplarily, the video encoder 102 encodes the components of the video blocks in the video slice to be encoded. Generally, the slice can be divided into multiple original blocks (and may be divided into a set of original blocks called image blocks). The sizes of CU, PU and TU are usually determined in the segmentation module. In addition, the segmentation module is also used to determine the size of the rate control unit. The rate control unit refers to a basic processing unit in the rate control module, for example, the rate control module calculates complexity information for the original block based on the rate control unit, and then calculates the quantization parameter of the original block based on the complexity information. Among them, the segmentation strategy of the segmentation module can be preset or continuously adjusted based on the image during the encoding process. When the segmentation strategy is a preset strategy, the same segmentation strategy is also preset in the decoding end accordingly, so as to obtain the same image processing unit. The image processing unit is any of the above-mentioned image blocks and corresponds one-to-one with the encoding side. When the segmentation strategy is continuously adjusted based on the image during the encoding process, the segmentation strategy can be directly or indirectly encoded into the bitstream, and accordingly, the decoding end obtains the corresponding parameters from the bitstream, obtains the same segmentation strategy, and obtains the same image processing unit.

[0149] The rate control module in the preprocessing module 20 is used to generate quantization parameters so that the quantization module 23 and the inverse quantization module 27 perform related calculations. In the process of calculating the quantization parameters, the rate control module can obtain the image information of the original block for calculation, such as the above-mentioned input information; it can also obtain the reconstructed value obtained by the summer 201 after reconstruction for calculation, and this application does not limit this.

[0150] The prediction module 21 may provide the prediction block to the summer 202 to generate a residual block, and provide the prediction block to the summer 201 for reconstruction to obtain a reconstructed block, which is used as a reference pixel for subsequent prediction. The video encoder 102 forms a pixel difference by subtracting the pixel value of the prediction block from the pixel value of the original block. The pixel difference is the residual block, and the data in the residual block may include brightness difference and chrominance difference. The summer 201 represents one or more components that perform this subtraction operation. The prediction module 21 may also send related syntax elements to the entropy coding module 24 for merging into the bitstream.

[0151] The transform module 22 may divide the residual block into one or more TUs for transform. The transform module 22 may convert the residual block from the pixel domain to the transform domain (e.g., the frequency domain). For example, the residual block may be transformed using a discrete cosine transform (DCT) or a discrete sine transform (DST) to obtain a transform coefficient. The transform module 32 may send the obtained transform coefficient to the quantization module 23.

[0152] The quantization module 23 may perform quantization based on the quantization unit. The quantization unit may be the same as the CU, TU, and PU described above, or may be further divided in the segmentation module. The quantization module 23 quantizes the transform coefficients to further reduce the bit rate to obtain quantized coefficients. The quantization process may reduce the bit depth associated with some or all of the coefficients. The degree of quantization may be modified by adjusting the quantization parameter. In some feasible embodiments, the quantization module 23 may then perform a scan of the matrix containing the quantized transform coefficients. Alternatively, the entropy coding module 24 may perform the scan.

[0153] After quantization, the entropy coding module 24 may entropy encode the quantized coefficients. For example, the entropy coding module 24 may perform context-adaptive variable-length coding (CAVLC), context-based adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic decoding (SBAC), probability interval partitioning entropy (PIPE) decoding, or another entropy coding method or technology. After entropy coding by the entropy coding module 24, a substream is obtained. After entropy coding by multiple entropy coding modules 24, multiple substreams can be obtained. The multiple substreams are passed through the coded substream buffer 25 and the substream interleaving module 26 to obtain a bitstream after substream interleaving. The bitstream is transmitted to the video decoder 112 or archived for later transmission or retrieval by the video decoder 112.

[0154] The process of sub-stream interleaving can refer to the above Figures 1 to 5 It has been described in detail and will not be repeated here.

[0155] The inverse quantization module 27 and the inverse transform module 28 apply inverse quantization and inverse transform respectively, and the summer 201 adds the inverse transformed residual block to the predicted residual block to generate a reconstructed block, which is used as a reference pixel for subsequent prediction of the original block. The reconstructed block is stored in the reference image memory 29.

[0156] Figure 8FIG. 1 is a schematic diagram of the structure of the video decoder 112 provided in the embodiment of the present application. Figure 8 As shown, the video decoder 112 includes a substream interleaving module 30 , a decoding substream buffer 31 , an entropy decoding module 32 , a prediction module 33 , an inverse quantization module 34 , an inverse transform module 35 , a summer 301 and a reference image memory 36 .

[0157] The entropy decoding module 32 includes a parsing module and a bit rate control module. In some feasible implementations, the video decoder 112 may perform the Figure 7 The decoding process of the video encoder 102 is an exemplary reciprocal of the encoding process described.

[0158] During the decoding process, the video decoder 112 receives a bitstream of the encoded video from the video encoder 102. The bitstream is reversely interleaved through the substream interleaving module 30 to obtain multiple substreams, and the multiple substreams pass through the corresponding decoded substream buffers and flow into the corresponding entropy decoding modules 32. Taking one substream as an example, the parsing module in the entropy decoding module 32 of the video decoder 112 performs entropy decoding on the substream to generate quantization coefficients and syntax elements. The entropy decoding module 32 transfers the syntax elements to the prediction module 33. The video decoder 112 can receive syntax elements at the video slice level and / or the video block level.

[0159] The rate control module in the entropy decoding module 32 generates quantization parameters based on the information of the image to be decoded obtained by the analysis module so that the inverse quantization module 34 can perform relevant calculations. The rate control module can also calculate the quantization parameters based on the reconstructed block obtained by the summer 301 through reconstruction.

[0160] The inverse quantization module 34 inversely quantizes (e.g., dequantizes) the quantized coefficients and the generated quantization parameters provided in the substream and decoded by the entropy decoding module 32. The inverse quantization process may include determining the degree of quantization using the quantization parameters calculated by the video encoder 102 for each video block in the video slice, and similarly determining the degree of inverse quantization to be applied. The inverse transform module 35 applies an inverse transform (e.g., a transform method such as DCT, DST, etc.) to the inverse quantized transform coefficients, and generates an inverse transformed residual block in the pixel domain according to an inverse transform unit. The size of the inverse transform unit is the same as the size of the TU, and the inverse transform method and the transform method use the corresponding forward transform and inverse transform in the same transform method, for example, the inverse transform of DCT, DST is an inverse DCT, an inverse DST, or a conceptually similar inverse transform process.

[0161] After the prediction module 33 generates the prediction block, the video decoder 112 forms a decoded video block by summing the inverse transformed residual block from the inverse transform module 35 with the prediction block. Summer 301 represents one or more components that perform this summing operation. If necessary, a deblocking filter may also be applied to filter the decoded block to remove blocking artifacts. The decoded image blocks in a given frame or image are stored in the reference image memory 36 as reference pixels for subsequent prediction.

[0162] The present application embodiment provides a possible implementation method of video (image) encoding and decoding, see Fig. 9 , Fig. 9 A schematic diagram of a video encoding and decoding process provided in an embodiment of the present application, wherein the video encoding and decoding implementation method includes process ① to process ⑤, and process ① to process ⑤ can be executed by any one or more of the above-mentioned source device 10, video encoder 102, destination device 11 or video decoder 112.

[0163] Process ①: Divide a frame of image into one or more non-overlapping parallel coding units. The one or more parallel coding units have no dependencies and can be completely parallel / independently encoded and decoded, such as Figure 4 A parallel encoding unit 1 and a parallel encoding unit 2 are shown.

[0164] Process ②: Each parallel coding unit can be further divided into one or more non-overlapping independent coding units. The independent coding units may not depend on each other, but may share some parallel coding unit header information.

[0165] The independent coding unit may include three channels of brightness Y, first chromaticity Cb, and second chromaticity Cr, or three channels of RGB, or may include only one of the channels. If the independent coding unit includes three channels, the sizes of the three channels may be exactly the same or different, which is related to the input format of the image. The independent coding unit can also be understood as one or more processing units formed by the N channels contained in each parallel coding unit. For example, the three channels Y, Cb, and Cr mentioned above are the three channels constituting the parallel coding unit, which can be an independent coding unit respectively, or Cb and Cr can be collectively referred to as chromaticity channels, then the parallel coding unit includes an independent coding unit consisting of a brightness channel, and an independent coding unit consisting of a chromaticity channel.

[0166] Process ③: Each independent coding unit can be further divided into one or more non-overlapping coding units. The coding units within the independent coding unit can be interdependent, such as multiple coding units can perform mutual reference pre-coding and decoding.

[0167] If the coding unit and the independent coding unit have the same size (ie, the independent coding unit is divided into only one coding unit), the size thereof can be any size described in process ②.

[0168] The coding unit may include three channels (or three channels of RGB) of brightness Y, first chrominance Cb, and second chrominance Cr, or only one of the channels. If three channels are included, the sizes of the channels may be exactly the same or different, depending on the image input format.

[0169] It is worth noting that process ③ is an optional step in the video coding and decoding method, and the video codec can encode / decode the residual coefficients (or residual values) of the independent coding units obtained in process ②.

[0170] Process ④: For the coding unit, it can be further divided into one or more non-overlapping prediction groups (prediction groups, PG), PG can also be referred to as Group, each PG is encoded and decoded according to the selected prediction mode, and the prediction value of PG is obtained to constitute the prediction value of the entire coding unit. Based on the prediction value and the original value of the coding unit, the residual value of the coding unit is obtained.

[0171] Process ⑤: Based on the residual values ​​of the coding units, the coding units are grouped to obtain one or more non-overlapping residual blocks (RBs). The residual coefficients of each RB are encoded and decoded according to the selected mode to form a residual coefficient stream. Specifically, it can be divided into two categories: transforming the residual coefficients and not transforming them.

[0172] Among them, the selected mode of the residual coefficient encoding and decoding method in process ⑤ may include, but is not limited to any of the following: semi-fixed length coding method, exponential Golomb coding method, Golomb-Rice coding method, truncated unary code coding method, run-length coding method, direct encoding of the original residual value, etc.

[0173] For example, a video encoder may directly encode coefficients within an RB.

[0174] For another example, the video encoder may also transform the residual block, such as DCT, DST, Hadamard transform, etc., and then encode the transformed coefficients.

[0175] As a possible example, when RB is small, the video encoder can directly perform uniform quantization on each coefficient in RB, and then perform binary encoding. If RB is large, it can be further divided into multiple coefficient groups (coefficient groups, CG), and then each CG is uniformly quantized and then binary encoded. In some embodiments of the present application, the coefficient group (CG) and the quantization group (QG) can be the same.

[0176] The following is an exemplary explanation of the residual coefficient encoding part in a semi-fixed length coding method. First, the maximum value of the absolute value of the residual in an RB block is defined as the modified maximum value (modified maximum, mm). Secondly, the number of coding bits for the residual coefficient in the RB block is determined (the number of coding bits for the residual coefficient in the same RB block is consistent). For example, if the critical limit (CL) of the current RB block is 2 and the current residual coefficient is 1, then 2 bits are required to encode the residual coefficient 1, which is expressed as 01. If the CL of the current RB block is 7, it means encoding an 8-bit residual coefficient and a 1-bit sign bit. The determination of CL is to find the minimum M value that satisfies that all residuals of the current sub-block are within the range of [-2^(M-1), 2^(M-1)]. If both -2^(M-1) and 2^(M-1) boundary values ​​exist at the same time, M should be increased by 1, that is, M+1 bits are required to encode all residuals of the current RB block; if only one of the two boundary values ​​of -2^(M-1) and 2^(M-1) exists, a Trailing bit needs to be encoded to determine whether the boundary value is -2^(M-1) or 2^(M-1); if all residuals do not exist in either -2^(M-1) or 2^(M-1), there is no need to encode the Trailing bit.

[0177] In addition, for some special cases, the video encoder can also directly encode the original value of the image instead of the residual value.

[0178] The video encoder 102 and the video decoder 112 may also be implemented in another implementation form, for example, by using a general digital processing system, see Fig.10 , Fig.10 The schematic diagram of the composition of the coding and decoding device provided in the embodiment of the present application is as follows. The coding and decoding device may be a part of the device in the above-mentioned video encoder 102, or a part of the device in the above-mentioned video decoder 112. The coding and decoding device may be applied to the encoding side (or encoding end), or to the decoding side (or decoding end). Fig.10As shown, the coding and decoding device includes a processor 41 and a memory 42. The processor 41 is connected to the processor 42 (for example, connected to each other through a bus 43). Optionally, the coding and decoding device may also include a communication interface 44, which connects the processor 41 and the memory 42 for receiving / sending data.

[0179] The processor 41 is used to execute the instructions stored in the memory 42 to implement the image encoding and decoding method provided in the following embodiments of the present application. The processor 41 can be a central processing unit (CPU), a general-purpose processor network processor (NP), a digital signal processor (DSP), a microprocessor, a microcontroller, a programmable logic device (PLD), or any combination thereof. The processor 41 can also be any other device with processing functions, such as a circuit, a device, or a software module, which is not limited in the embodiments of the present application. In one example, the processor 41 can include one or more CPUs, such as Fig.10 As an optional implementation, the electronic device may include multiple processors, for example, in addition to the processor 41, it may also include a processor 45 ( Fig.10 The dashed line is used as an example.

[0180] The memory 42 is used to store instructions. For example, the instructions may be computer programs. Optionally, the memory 42 may be a read-only memory (ROM) or other types of static storage devices that can store static information and / or instructions, or a random access memory (RAM) or other types of dynamic storage devices that can store information and / or instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, etc., and the embodiments of the present application are not limited to this.

[0181] It should be noted that the memory 42 may exist independently of the processor 41, or may be integrated with the processor 41. The memory 42 may be located in the codec device, or may be located outside the codec device, which is not limited in the embodiment of the present application.

[0182] The bus 43 is used to transmit information between the various components included in the codec device. The bus 43 can be an industry standard architecture (ISA) line, a peripheral component interconnect (PCI) line, or an extended industry standard architecture (EISA) line. The bus 43 can be divided into an address line, a data line, a control line, etc. For ease of representation, Fig.10 Although only one solid line is used in the diagram, it does not mean that there is only one line or one type of line.

[0183] The communication interface 44 is used to communicate with other devices or other communication networks. The other communication networks may be Ethernet, radio access network (RAN), wireless local area network (WLAN), etc. The communication interface 44 may be a module, a circuit, a transceiver, or any device capable of achieving communication. The embodiments of the present application are not limited to this.

[0184] It should be noted that Fig.10 The structure shown in the figure does not constitute a limitation on the encoding and decoding device, except Fig.10 In addition to the components shown, the coding and decoding device may include more or fewer components than shown, or a combination of certain components, or a different arrangement of components.

[0185] It should be noted that the execution subject of the image encoding and decoding method provided in the embodiment of the present application can be the above-mentioned encoding and decoding device, or an application (application, APP) installed in the encoding and decoding device that provides encoding and decoding functions; or a CPU in the encoding and decoding device; or a functional module in the encoding and decoding device for executing the image encoding and decoding method. The embodiment of the present application does not limit this. For the sake of simplicity of description, the following description is unified in terms of the encoding end or the decoding end.

[0186] The image encoding and decoding method provided by the embodiment of the present application is introduced below with reference to the accompanying drawings.

[0187] As the background technology and the above Figures 1 to 5As described above, different sub-stream buffers fill sub-streams at different speeds. At the same time, a sub-stream buffer with a faster filling speed fills more sub-stream bits than a sub-stream buffer with a slower filling speed. In order to ensure the integrity of the filled data, all sub-stream buffers need to be set larger, which increases hardware costs.

[0188] In order to reasonably configure the sub-stream buffer, the embodiment of the present application proposes a series of improved encoding methods (such as encoding mode / prediction mode, complexity information transmission, coefficient grouping, code stream arrangement, etc.) to reduce the expansion rate of the coding unit, thereby reducing the speed difference when encoding the coding blocks of each channel of the coding unit, and reasonably configuring the space of the sub-stream buffer to reduce hardware costs.

[0189] Among them, the expansion rate of the coding unit may include a theoretical expansion rate and a current (CU) expansion rate (actual expansion rate). The theoretical expansion rate can be obtained through theoretical deduction after the codec is determined, and its value is greater than 1. If the number of bits of the substream encoded by the coding block of a certain channel in the coding unit is 0, this situation needs to be excluded when calculating the current expansion rate. Theoretical expansion rate = the number of bits of the CB with the largest theoretical bit number in the CU / the number of bits of the CB with the smallest theoretical bit number in the CU. Current (CU) expansion rate = the number of bits of the CB with the largest actual bit number in the current CU / the number of bits of the CB with the smallest actual bit number in the current CU.

[0190] Optionally, the related expansion rate may also include the current substream expansion rate: Current substream expansion rate = the number of data bits in the substream buffer with the largest number of data bits in the current multiple substream buffers / the number of data bits in the substream buffer with the smallest number of data bits in the current multiple substream buffers.

[0191] It should be noted that, for the coding blocks of each channel or the entire coding unit, the coding end can select the coding mode (or prediction mode) based on the following strategies. Each improved coding mode in the following embodiments can be selected and determined according to the following strategies.

[0192] Strategy 1: Calculate the bit consumption cost and select the encoding mode with the lowest bit consumption cost.

[0193] The bit consumption cost refers to the number of bits required for encoding / decoding a CU. The bit consumption cost mainly includes the code length of the mode flag (or mode codeword), the code length of the encoding / decoding tool information codeword, and the code length of the residual codeword.

[0194] Strategy 2: Calculate the distortion after encoding and select the encoding mode with the smallest distortion.

[0195] The distortion is used to indicate the difference between the reconstructed value and the original value. The distortion can be calculated by using any one or more of the sum of squared difference (SSD), mean squared error (MSE), sum of absolute difference (SAD), sum of absolute transformed difference (SATD), and peak signal to noise ratio (PSNR). The embodiments of the present application are not limited to this.

[0196] Strategy 3: Calculate the rate-distortion cost and select the coding mode with the minimum rate-distortion cost.

[0197] The rate distortion cost refers to the weighted sum of the bit consumption cost and the distortion. When performing weighted calculation, the weight coefficient of the bit consumption cost and the weight coefficient of the encoded distortion can be preset in the encoding end. The embodiment of the present application does not limit the specific value of the weight coefficient.

[0198] The following will introduce the improved encoding mode in the image encoding and decoding method provided in the embodiment of the present application through a series of embodiments.

[0199] 1. Modification of anti-expansion mode.

[0200] Optional implementation scheme: The encoding end encodes each pixel value in the encoding blocks of multiple channels of the encoding unit using the image bit width as a fixed-length code.

[0201] Modification scheme: The encoding end encodes each pixel value in the encoding blocks of multiple channels of the encoding unit with a fixed-length code that is less than or equal to the image bit width.

[0202] Embodiment 1:

[0203] For example, Fig.11 The following is a flow chart of an image encoding method provided in an embodiment of the present application. Fig.11 As shown, the image encoding method includes S101 to S102.

[0204] S101: The encoding end obtains a coding unit.

[0205] Among them, the encoding end is also the above Figure 6 The source device 10 in the source device 10, or the video encoder 102 in the source device 10, or the above Fig.10The encoding and decoding device in the present application is not limited to this. The encoding unit is an image block in the image to be processed (that is, the original block mentioned above). The encoding unit includes encoding blocks of multiple channels; the multiple channels include a first channel; the first channel is any one of the multiple channels. For example, the size of the encoding unit can be 16×2×3, then the size of the encoding block of the first channel of the encoding unit is 16×2.

[0206] S102: The encoding end encodes the encoding block of the first channel according to the first encoding mode.

[0207] The first coding mode is a mode for encoding the sample values ​​in the coding block of the first channel according to the first fixed-length code. The code length of the first fixed-length code is less than or equal to the image bit width of the image to be processed. The image bit width is used to characterize the number of bits required to store each sample in the image to be processed. The first fixed-length code can be preset in the encoding / decoding end, or determined by the encoding end and written into the code stream header information for transmission to the decoding end.

[0208] It should be understood that the anti-expansion mode refers to a mode for directly encoding the original pixel values ​​in the coding blocks of multiple channels. Compared with other coding modes, the number of bits consumed by directly encoding the original pixel values ​​is usually larger. Therefore, the coding block with the largest theoretical number of bits in the theoretical expansion rate usually comes from the coding block encoded according to the anti-expansion mode. The embodiment of the present application reduces the theoretical number of bits (that is, the numerator in the fraction) of the coding block with the largest theoretical number of bits in the theoretical expansion rate by reducing the code length of the fixed-length code in the anti-expansion mode, thereby reducing the theoretical expansion rate. The theoretical expansion rate is low, and the speed difference of the coding block encoding sub-streams of each channel is small. Therefore, when setting the sub-stream buffer, the sub-stream buffer will neither be larger nor cause waste of sub-stream buffer space, thereby achieving a reasonable configuration of the sub-stream buffer.

[0209] Optionally, if the image to be processed is an RGB image, the encoding end may convert it into a YUV format for encoding, or if the image to be processed is a YUV image, the encoding end may also convert it into an RGB format for encoding. The present application embodiment does not limit this.

[0210] Correspondingly, an embodiment of the present application also provides an image decoding method. Fig.12 The following is a flow chart of an image decoding method provided in an embodiment of the present application. Fig.12 As shown, the image decoding method includes S201 to S202.

[0211] S201. A decoding end obtains a code stream after encoding a coding unit.

[0212] The bitstream after encoding the coding unit may include multiple substreams corresponding to the multiple channels after the coding blocks of the multiple channels are encoded. For example, the multiple substreams may include the substream corresponding to the first channel (that is, the substream after the coding blocks of the first channel are encoded).

[0213] S202: The decoding end decodes the substream corresponding to the first channel according to the first decoding mode.

[0214] The first decoding mode is a mode of parsing sample values ​​from a substream corresponding to the first channel according to a first fixed-length code.

[0215] Optionally, S202 may also specifically include: when the first fixed-length code is equal to the image bit width, the decoding end directly decodes the sub-stream corresponding to the first channel according to the first decoding mode; when the first fixed-length code is smaller than the image bit width, the decoding end dequantizes the pixel values ​​in the parsed coding block of the first channel.

[0216] The quantization step is 1<<(bitdepth-fixed_length). bitdepth indicates the image bit width. fixed_length indicates the length of the first fixed-length code. 1<< indicates a left shift of one bit.

[0217] 2. Modify the fallback mode.

[0218] Optional implementation scheme 1: If the current code stream buffer cannot satisfy the requirement that all channels of the coding unit are encoded in the anti-expansion mode, the anti-expansion mode is turned off and the encoding end selects a mode other than the original value. At this time, even if other coding modes are used for encoding, expansion will occur (the number of bits of the sub-stream after encoding the coding block of a certain channel is too large), and the anti-expansion mode cannot be used.

[0219] Improvement plan 1: Ensure that the anti-expansion mode is turned on.

[0220] Embodiment 2:

[0221] Fig.13 A flowchart of another image decoding method provided in an embodiment of the present application is shown below. Fig.13 As shown, the image decoding method includes S301 to S303.

[0222] S301: The encoding end obtains a coding unit.

[0223] The coding unit is an image block in the image to be processed, and the coding unit includes coding blocks of multiple channels.

[0224] S302: The encoding end determines a first total code length.

[0225] The first total code length is the total code length of the first code stream obtained after the coding blocks of multiple channels are encoded according to their corresponding target coding modes. The target coding mode includes a first coding mode, which is a mode for encoding pixel values ​​in the coding block according to a first fixed-length code, and the code length of the first fixed-length code is less than or equal to the image bit width of the image to be processed, and the image bit width is used to represent the number of bits required to store each sample in the image to be processed.

[0226] For example, for each of the multiple channels, the encoder may determine the rate-distortion cost of each coding mode according to the above strategy 3, and determine the mode with the lowest rate-distortion cost as the target coding mode corresponding to the coding block of the channel.

[0227] S303: When the first total code length is greater than or equal to the remaining size of the code stream buffer, the encoding end encodes the coding blocks of the multiple channels in a fallback mode.

[0228] Among them, the fallback mode includes a first fallback mode and a second fallback mode. The first fallback mode refers to using the IBC mode to obtain the block vector of the reference prediction block, and then calculating the residual and quantizing the residual. The quantization step size is determined based on the remaining size of the code stream buffer and the target pixel depth (bite per pixel, BPP). The second fallback mode refers to directly quantizing the pixels, and the quantization step size is determined based on the remaining size of the code stream buffer and the target BPP. The fallback mode has the same mode flag as the first coding mode.

[0229] It should be understood that when other coding modes are used for encoding, the residual after encoding may be too large, resulting in too large a number of bits of the coding block using the other coding modes, while the anti-expansion mode is encoded using fixed-length codes, and the total code length after encoding is fixed. The anti-expansion mode can avoid the above-mentioned situation of excessive residual errors. The embodiment of the present application provides a solution for informing the decoding end of the adopted coding mode by judging the remaining size of the code stream buffer when the mode flags of the fallback mode and the first coding mode (anti-expansion mode) are the same, thereby ensuring that the anti-expansion mode can always be selected, thereby reducing the theoretical expansion rate. The theoretical expansion rate is low, and the speed difference of the coding block encoding substreams of each channel is small. Therefore, when setting the substream buffer, it will neither cause the substream buffer to be larger nor cause waste of substream buffer space, thereby achieving reasonable configuration of the substream buffer.

[0230] In some possible embodiments, the image encoding method may further include: the encoding end encodes a mode flag in a plurality of sub-streams obtained by encoding the encoding blocks of the plurality of channels.

[0231] The mode flag is used to indicate the coding modes respectively adopted by the coding blocks of multiple channels, and the mode flag of the first coding mode is the same as the mode flag of the fallback mode.

[0232] In a possible implementation, the mode flag of the substream after the coding block of multiple channels is encoded can be encoded. Taking the case where the multiple channels include a first channel, and the first channel is any one of the multiple channels as an example, in this case, the encoding end encodes the mode flag in the multiple substreams obtained by encoding the coding blocks of the multiple channels, which may include: encoding the submode flag in the substream obtained by encoding the coding block of the first channel.

[0233] The sub-mode flag is used to indicate the type of fallback mode used by the coding block of the first channel, or to indicate the type of fallback mode used by the coding blocks of multiple channels. As described above, the fallback mode may include a first fallback mode and a second fallback mode. No further details are given here.

[0234] In another possible implementation, also taking the above-mentioned first component as an example, in this case, the encoding end encodes mode flags in multiple sub-streams obtained by encoding the coding blocks of multiple channels, which may include: encoding a first flag, a second flag, and a third flag in a sub-stream obtained by encoding the coding block of the brightness channel.

[0235] Among them, the first flag is used to indicate that the coding blocks of multiple channels are encoded in the first coding mode or the fallback mode; the second flag is used to indicate that the coding blocks of multiple channels are encoded in the target mode, and the target mode is any one of the first coding mode and the fallback mode; when the second flag indicates that the target mode adopted by the coding blocks of multiple channels is the fallback mode, the third flag is used to indicate the type of fallback mode adopted by the coding blocks of multiple channels.

[0236] Correspondingly, the embodiments of the present application also provide two image decoding methods. Fig.14 A flowchart of another image decoding method provided in an embodiment of the present application is shown below. Fig.14 As shown, the image decoding method includes S401 to S404.

[0237] S401: The decoding end parses the bit stream encoded by the encoding unit.

[0238] The encoding unit includes encoding blocks of multiple channels. The multiple channels include a first channel, and the first channel is any one of the multiple channels.

[0239] S402: If a mode flag is parsed from a substream obtained by encoding the coding blocks of the multiple channels, and the second total code length is greater than the remaining size of the codestream buffer, the decoding end determines that the target decoding mode of the substream is a fallback mode.

[0240] The second total code length is a total code length of a first code stream obtained after coding blocks of multiple channels are encoded according to the first coding mode or the fallback mode.

[0241] S403: The decoding end parses the preset flag bit in the substream to determine a target fallback mode.

[0242] The target fallback mode is one of the fallback modes. The sub-mode flag is used to indicate the type of fallback mode used when encoding the coding blocks of multiple channels. The fallback mode includes a first fallback mode and a second fallback mode.

[0243] S404: The decoding end decodes the substream according to the target fallback mode.

[0244] Fig.15 A flowchart of another image decoding method provided in an embodiment of the present application is shown below. Fig.15 As shown, the image decoding method includes S501 to S505.

[0245] S501: The decoding end parses the bit stream encoded by the encoding unit.

[0246] S502: The decoding end parses a first flag from a substream obtained by encoding a coding block of a first channel.

[0247] The first channel is any one of the multiple channels. The first flag can refer to the above encoding method, which will not be described here.

[0248] S503: The decoding end parses the second flag from the substream obtained by encoding the encoding block of the first channel.

[0249] Among them, the second flag can refer to the description in the above encoding method, which will not be repeated here.

[0250] S504: When the second flag indicates that the target mode adopted by the coding blocks of the multiple channels is the fallback mode, the decoding end parses a third flag from a substream obtained by encoding the coding blocks of the first channel.

[0251] Among them, the third flag can refer to the description in the above encoding method, which will not be repeated here.

[0252] S505: The decoding end determines target decoding modes of the multiple channels according to the type of the fallback mode indicated by the third flag, and decodes substreams obtained by encoding the coding blocks of the multiple channels according to the target decoding mode.

[0253] For example, the decoding end may use the type of fallback mode indicated by the third flag as the target decoding mode of multiple channels.

[0254] Optionally, the method may further include: the decoding end determines a target decoding code length of the coding unit based on the remaining size of the code stream buffer and the target pixel depth BPP; the target decoding code length is used to indicate the code length required for decoding the code stream of the coding unit; the decoding end determines the allocated code lengths of multiple channels based on the decoding code length; the allocated code length is used to indicate the code length required for decoding the residual of the code stream of the coding blocks of multiple channels; the decoding end determines the decoding code lengths allocated to each of the multiple channels according to the average value of the allocated code lengths in the multiple channels.

[0255] Based on the understanding of the above-mentioned embodiment 2, taking the case where multiple channels include a brightness (Y) channel, a first chroma (U) channel, and a second chroma (V) channel as an example, two solutions mainly included in the improved solution 1 of the fallback mode are introduced:

[0256] Scenario 1:

[0257] Encoding end: The encoding end controls that at any time, the bit consumption cost of encoding in the anti-expansion mode (original value mode) is the largest, and the encoding end cannot select a mode with a bit consumption cost greater than the original value. Therefore, encoding modes are required for all components. Even in fallback mode, not only the sub-stream corresponding to the Y channel (the first sub-stream) requires an encoding mode, but also the sub-stream corresponding to the U / V channel (the second sub-stream and the third sub-stream). Keep the fallback mode codeword the same as the original value mode codeword. The specific type of fallback mode (the first fallback mode and the second fallback mode) can be based on a certain channel encoding.

[0258] Decoding end: First, parse the encoding modes of the three channels. When the three channels Y / U / V are all target encoding modes selected based on the rate-distortion cost, then determine the remaining size of the code stream buffer to determine whether the code stream buffer allows the three channels to be decoded according to their respective target encoding modes. If not allowed, the current decoding mode is the fallback mode. If it is a fallback mode, a flag bit can be parsed based on a certain component to indicate whether the current fallback mode belongs to the first fallback mode or the second fallback mode (the three channels use the same fallback mode). Therefore, when one channel on the decoding end does not select the anti-expansion mode, the current CU will not select the fallback mode.

[0259] Scenario 2:

[0260] Encoding end: The encoding end controls that at any time, the bit consumption cost of encoding under the anti-expansion mode (original value mode) is the largest, and the encoding end cannot select a mode with a bit consumption cost greater than the original value. The original value mode and the fallback mode still use the same mode flag (that is, the first flag mentioned above), but when encoding the anti-expansion mode and the fallback mode, an additional flag (that is, the second flag mentioned above) is encoded to indicate that the current encoding mode is one of the anti-expansion mode or the fallback mode. When the additional flag indicates that the current encoding mode is the fallback mode, a flag bit (that is, the third flag mentioned above) is further encoded to indicate that the current fallback mode is one of the first fallback mode or the second fallback mode. When multiple channels all select the fallback mode, the types of fallback modes of the multiple channels remain consistent. When a channel selects the anti-expansion mode, the mode flag is encoded in the substream corresponding to the channel, but there is no need to encode the flag whether it is the fallback mode (that is, the second flag mentioned above).

[0261] Decoding end: parse the mode flag from the bitstream. If the mode flag is a mode flag shared by the anti-expansion mode and the fallback mode, continue to parse a flag (that is, the second flag mentioned above) to indicate whether the current mode is the fallback mode. If it is the fallback mode, further parse a flag (that is, the third flag mentioned above) to indicate that the current fallback mode is one of the first fallback mode and the second fallback mode. When the type of fallback mode of a channel is parsed, other channels that use the fallback mode use the same type of fallback mode as the channel. Channels that do not use the fallback mode each parse their own mode.

[0262] As described above, Improvement Scheme 1 can reduce the upper limit of the numerator in the expansion rate formula through the above-mentioned Embodiment 2. Optionally, for the fallback mode, the lower limit of the denominator in the expansion rate formula can also be increased. The following is introduced through optional Implementation Scheme 2 and Improvement Scheme 2.

[0263] Optional implementation scheme 2: When it is determined to use the fallback mode for encoding, the encoding end will give a target BPP. In the fallback mode, the sum of the number of bits of the substreams corresponding to the coding blocks of the three channels must be less than or equal to the target BPP. Therefore, when performing code rate distribution, the public information (block vector and mode flag) is encoded on the brightness channel. After the allocated code length is subtracted from the code length occupied by the public information, the remaining code length is used to encode the residuals of multiple channels. Therefore, the remaining code length needs to be distributed. The distribution method is to divide the number of pixels of multiple channels by the remaining code length (that is, the code length allocated to each channel needs to be an integer multiple of the number of pixels of the coding unit). If it cannot be divided, the part of the code length that cannot be divided is allocated to the brightness channel. When the remaining code length is small, it cannot be divided by the number of pixels of multiple channels. The substreams corresponding to the first chrominance channel and the second chrominance channel cannot be allocated code lengths, resulting in a small number of corresponding substream bits and a large expansion rate.

[0264] Improvement plan 2: Evenly distribute the remaining code length to multiple channels according to the number of bits to improve the allocation accuracy.

[0265] Embodiment three:

[0266] Fig.16 A flowchart of another image encoding method provided in an embodiment of the present application is shown below. Fig.15 As shown, based on the above S301 to S303, the image encoding method may further include S601 to S603.

[0267] S601: The encoder determines a target encoding code length of a coding unit based on the remaining size of the code stream buffer and the target BPP.

[0268] The target encoding code length is used to indicate the code length required for encoding the encoding unit. The target BPP can be obtained by the encoding end, for example, by receiving the target BPP input by the user.

[0269] S602: Determine the allocated code lengths of multiple channels based on the target encoding code length.

[0270] The allocated code length is used to indicate a code length required for encoding residuals of coding blocks of multiple channels.

[0271] For example, as described above, the encoding end may allocate code lengths minus the code lengths occupied by the public information to obtain allocated code lengths for multiple channels.

[0272] S603: Determine the encoding code lengths allocated to each of the multiple channels according to the average values ​​of the allocated code lengths in the multiple channels.

[0273] Optionally, the encoding end may further encode the block vector and mode flag in sub-streams corresponding to multiple channels.

[0274] The mode flag can refer to the above, which will not be described in detail here. The block vector can refer to the IBC mode described below, which will not be described in detail here.

[0275] Optionally, if when encoding the residual, there is a channel whose bits allocated cannot divide the number of pixels of the channel, that is, the residual of each pixel cannot be encoded using a fixed-length code, then the encoder can group the pixels of the channel and encode only one residual value in each group.

[0276] It should be understood that when the remaining code length is small, the current code length allocation mode based on the number of pixels of the coding unit may result in the chrominance channel being unable to be allocated a code length, resulting in a smaller number of bits after encoding the coding block of the chrominance channel, which increases the expansion rate. The image encoding method provided in the embodiment of the present application can allocate by converting the allocation unit from the number of pixels to the number of bits of the allocated code length, so that each channel can be allocated a code length for encoding, thereby reducing the theoretical expansion rate.

[0277] Correspondingly, an embodiment of the present application also provides an image decoding method. Based on the above S401 to S404 or S501 to S505, the method may also include: the decoding end determines the target decoding code length of the coding unit based on the remaining size of the code stream buffer and the target pixel depth BPP; the target decoding code length is used to indicate the code length required for decoding the code stream of the coding unit; the decoding end determines the allocated code lengths of multiple channels based on the decoding code length; the allocated code length is used to indicate the code length required for decoding the residual of the code stream of the coding block of multiple channels; the decoding end determines the decoding code lengths allocated to each of the multiple channels according to the average value of the allocated code lengths in the multiple channels.

[0278] 3. Intra-block copy (IBC) mode modification.

[0279] Optional implementation: Mode identification CU level, BV is also CU level, mode identification and BV are both transmitted in brightness

[0280] Improvement plan: The mode identifier is changed to CB level, BV is still CU level, each channel transmits the mode identifier, and each channel transmits BV.

[0281] Embodiment 4:

[0282] Fig.17 A flowchart of another image encoding method provided in an embodiment of the present application is shown below. Fig.17 As shown, the method includes S701 to S704.

[0283] S701: The encoding end obtains a coding unit.

[0284] The encoding unit includes encoding blocks of multiple channels.

[0285] S702: The encoding end encodes a coding block of at least one channel among the multiple channels according to the IBC mode.

[0286] Optionally, S702 may specifically include: the encoding end determines a target encoding mode of the at least one channel using a rate-distortion optimization decision.

[0287] The target coding mode includes the IBC mode. The rate-distortion optimization decision can refer to the above strategy 3, which will not be described here.

[0288] S703: The encoding end obtains the BV of the reference prediction block.

[0289] The BV of the reference prediction block is used to indicate the position of the reference prediction block in the encoded image block. The reference prediction block is used to represent the prediction value of the coding block encoded according to the IBC mode.

[0290] S704: The encoder encodes the BV of the reference prediction block in at least one substream obtained by encoding the coding block of at least one channel in the IBC mode.

[0291] Optionally, for the BV of the reference prediction block of the coding block of each channel, the BV may be entirely encoded in the substream corresponding to the luma channel.

[0292] Optionally, for the BV of the reference prediction block of the coding block of each channel, the BV can also be encoded in the sub-stream corresponding to a certain chrominance channel (for example, the first chrominance channel or the second chrominance channel), or the number of BVs can be equally divided in the sub-streams corresponding to the two chrominance channels for encoding. If it cannot be equally divided, the encoding end can encode one more BV in the sub-stream corresponding to any chrominance channel.

[0293] Optionally, for the BV of the reference prediction block of the coding block of each channel, the BV can also be equally divided and encoded in the sub-streams corresponding to the coding blocks of different channels. If it cannot be equally divided, the encoding end can allocate it according to a preset ratio. For example, taking the number of BVs as 8 as an example, the preset ratio can be Y channel: U channel: V channel = 2:3:3, or Y channel: U channel: V channel = 4:2:2, etc. The embodiment of the present application does not limit the specific value of the preset ratio. In this case, the above S602 may specifically include: encoding the coding blocks of multiple channels in accordance with the IBC mode; the above S704 may specifically include: encoding the BV of the reference prediction block in the sub-stream obtained by encoding the coding blocks of each channel in the multiple channels in the IBC mode.

[0294] Optionally, the BV of the reference prediction block includes multiple BVs. Encoding the BV of the reference prediction block in the substream obtained by encoding the coding block of each channel in the multiple channels in the IBC mode may include: encoding the BV of the reference prediction block in the code stream obtained by encoding the coding block of each channel in the multiple channels in the IBC mode according to a preset ratio.

[0295] It should be understood that the number of bits of the substream corresponding to the luminance channel is usually larger, and the number of bits of the substream corresponding to the chrominance channel is usually smaller. In the embodiment of the present application, by transmitting the BV of the reference prediction block in the substream corresponding to the coding block of at least one channel when encoding in the IBC mode, the transmission data can be increased in the substream corresponding to the chrominance channel with a smaller number of bits, thereby increasing the denominator in the formula for calculating the theoretical expansion rate, thereby reducing the theoretical expansion rate.

[0296] Optionally, header information may be generated during the encoding process of the CU, and the header information of the CU may also be allocated and encoded in a substream according to the above-mentioned method of allocating BVs.

[0297] Correspondingly, an embodiment of the present application also provides an image decoding method. Fig.18 A flowchart of another image decoding method provided in an embodiment of the present application is shown below. Fig.18 As shown, the method includes S801 to S804.

[0298] S801: The decoding end parses the bit stream encoded by the encoding unit.

[0299] The coding unit includes coding blocks of multiple channels; the code stream includes multiple sub-streams corresponding to the multiple channels one by one after the coding blocks of the multiple channels are encoded.

[0300] S802: The decoding end determines a position of a reference prediction block in the multiple substreams based on a block vector BV of the reference prediction block parsed from at least one substream among the multiple substreams.

[0301] S803: The decoding end determines a prediction value of a decoding block to be decoded according to the IBC mode based on the position information of the reference prediction block.

[0302] S804: The decoding end reconstructs the decoding block decoded according to the IBC mode based on the prediction value.

[0303] S803 and S804 may refer to the above-mentioned video encoding and decoding system, which will not be described in detail here.

[0304] Optionally, for the BV of the reference prediction block of the coding block of each channel, the encoding end may parse all of them in the substream corresponding to the luma channel.

[0305] Optionally, for the BV of the reference prediction block of the coding block of each channel, the BV can also be parsed in the sub-stream corresponding to a certain chrominance channel (for example, the first chrominance channel or the second chrominance channel), or the number of BVs can be equally divided in the sub-streams corresponding to the two chrominance channels for parsing. If it cannot be equally divided, the encoding end can parse one more BV in the sub-stream corresponding to any chrominance channel.

[0306] Optionally, for the BV of the reference prediction block of the coding block of each channel, the BV can also be equally divided and parsed in the sub-streams corresponding to the coding blocks of different channels. If it cannot be equally divided, the decoding end can allocate it according to a preset ratio. For example, taking the number of BVs as 8 as an example, the preset ratio can be Y channel: U channel: V channel = 2:3:3, or Y channel: U channel: V channel = 4:2:2, etc. The embodiment of the present application does not limit the specific value of the preset ratio.

[0307] Optionally, the coding blocks encoded according to the IBC mode include coding blocks of at least two channels; the coding blocks of at least two channels share the BV of the reference prediction block.

[0308] Optionally, the method may further include: when an IBC mode identifier is parsed from any one of the multiple substreams, the decoding end determines that the target decoding mode corresponding to the multiple substreams is the IBC mode.

[0309] Optionally, the method may further include: the decoding end parses the IBC mode identifiers from the multiple substreams one by one, and determines that the target decoding mode corresponding to the substream from which the IBC mode identifier is parsed among the multiple substreams is the IBC mode.

[0310] Based on the understanding of the fourth embodiment, taking the case where multiple channels include a brightness (Y) channel, a first chroma (U) channel, and a second chroma (V) channel as an example, two solutions mainly included in the improved solution of the IBC mode are introduced:

[0311] Scenario 1:

[0312] Encoding side:

[0313] Step 1: Get the reference prediction block based on multiple channels. For a multi-channel coding unit, use the multi-channel at a certain position in the search area as the reference prediction block under the current multi-channel coding unit. This position is recorded as BV, that is, multiple channels share one BV. If the input image is YUV400, there is only one channel, otherwise there are 3 channels. After obtaining the prediction block, calculate the residual under each channel, quantize the residual (transform), and complete the reconstruction after inverse quantization (inverse transformation).

[0314] Step 2: Encode auxiliary information (including information such as coding block complexity level), mode information, and quantized coefficients of the encoded brightness channel on the brightness channel.

[0315] Step 3: If a chroma channel exists, encode auxiliary information (including information such as coding block complexity) on the chroma channel, and encode the quantized coefficients of the chroma channel.

[0316] 3.1. For each coding unit, the BV of the reference prediction block can be encoded entirely on the luminance channel.

[0317] 3.2. For the BV of the reference prediction block of each coding unit, all of it can be encoded on a certain chrominance channel, or the BV can be equally divided and encoded on two chrominance channels. If it cannot be equally divided, a certain chrominance channel can encode more part of the BV.

[0318] 3.3. For the BV of the reference prediction block of each coding unit, the BV can also be divided equally and encoded on different components. If it cannot be divided equally, it is allocated according to a preset ratio.

[0319] Decoding side:

[0320] Step 1: parse the auxiliary information and mode information of the coding block of the brightness channel, and parse the quantized coefficients of the brightness channel.

[0321] Step 2: If there is a chroma channel, parse the auxiliary information of the coding block of the U channel. If the prediction mode of the luminance channel is the IBC mode, the prediction mode of the current chroma channel does not need to be parsed and is directly defaulted to the IBC mode. Parse the quantized coefficients of the current chroma channel.

[0322] Step 3: The parsing of BV is the same as that of the encoding end, that is:

[0323] 3.1. For each coding unit, the BV of the reference prediction block can be parsed entirely on the luminance channel.

[0324] 3.2. For the BV of the reference prediction block of each parsing unit, all of it can be parsed on one chrominance channel, or the BV can be equally divided on two chrominance channels for parsing. If it cannot be equally divided, a part of the BV can be encoded more on one chrominance channel.

[0325] 3.3. For the BV of the reference prediction block of each parsing unit, the BV can also be divided equally and parsed on different components. If it cannot be divided equally, it is allocated according to a preset ratio.

[0326] Step 4: According to the BV shared by the three channels, the predicted value of each coding block under each channel is obtained, the coefficients obtained by analysis under each channel are inversely quantized (inversely transformed) to obtain the residual value, and each coding block is reconstructed according to the residual value and the predicted value.

[0327] Scenario 2:

[0328] Encoding side:

[0329] Step 1. For the same coding unit, that is, each IBC mode under three channels only needs to be trained to obtain a set of BVs. This set of BVs can be obtained based on the search area and original value of the three channels, based on the search area and original value of one of the channels, or calculated based on the search area and original value of any two channels.

[0330] Step 2: For the coding block of each channel, the target coding mode is determined using the rate-distortion cost, where the BV of the three channels in the IBC mode uses the BV calculated in step 1. It is allowed that after one component selects the IBC mode, the other components do not select the IBC mode.

[0331] Step 3: For each channel's coding block, its own optimal mode needs to be encoded. If the target mode of one or more channels selects the IBC mode, then BV can be encoded based on a channel that selects the IBC mode, or it can be encoded based on two channels that select the IBC mode and the number of BVs is equally divided. It can also be encoded based on all channels that select the IBC mode and the number of BVs is equally divided. When the number of BVs cannot be divided evenly, it is allocated according to the preset ratio.

[0332] Decoding side:

[0333] Each channel parses a target mode. If it is parsed that the target mode of one or more channels selects the IBC mode (only the same IBC mode can be selected), then the BV parsing is consistent with the allocation scheme of the BV encoding on the encoding side.

[0334] 4. Coefficient grouping.

[0335] Optional implementation scheme: Figure 1 As shown in the flowchart in, there may be a residual skip mode in the encoding process. If the encoder selects this skip mode during encoding, there is no need to encode the residual during encoding. Only 1 bit of data needs to be encoded to indicate the residual skip.

[0336] Improved solution: group the processing coefficients (residual coefficients and / or transform coefficients).

[0337] Embodiment five:

[0338] Fig.19A flowchart of another image encoding method provided in an embodiment of the present application is shown below. Fig.19 As shown, the image encoding method includes S901 to S902.

[0339] S901: The encoding end obtains a processing coefficient corresponding to a coding unit.

[0340] The processing coefficients include one or more of residual coefficients and transform coefficients.

[0341] S902: The encoding end divides the processing coefficients into multiple groups according to the number threshold.

[0342] The number threshold may be preset in the encoding end. The number threshold is related to the size of the encoding block. For example, for a 16×2 encoding block, the number threshold may be set to 16. The number of each group of processing coefficients in the plurality of groups of processing coefficients is less than or equal to the number threshold.

[0343] It should be understood that compared with the residual skip mode in the optional implementation scheme, the image encoding method provided in the embodiment of the present application can group the processing coefficients during encoding. Each group of processing coefficients after grouping needs to add the header information of the group of processing coefficients to describe the details of the group of processing coefficients during transmission. Compared with the current use of 1-bit data to represent the residual skip, the denominator in the calculation formula of the theoretical expansion rate is increased, thereby reducing the theoretical expansion rate.

[0344] Correspondingly, an embodiment of the present application also provides an image decoding method. Fig. 20 A flowchart of another image decoding method provided in an embodiment of the present application is shown below. Fig. 20 As shown, the image decoding method includes S1001 to S1002.

[0345] S1001: A decoding end parses a bit stream after encoding a coding unit to determine a processing coefficient corresponding to the coding unit.

[0346] The processing coefficients include one or more of the residual coefficients and the transformation coefficients; the processing coefficients include multiple groups; and the number of processing coefficients in each group is less than or equal to the number threshold.

[0347] S1002: The decoding end decodes the bit stream based on the processing coefficient.

[0348] S1002 may refer to the above-mentioned video encoding and decoding system, which will not be described in detail here.

[0349] 5. Complexity transmission.

[0350] Optional implementation scheme: The coding unit includes a coding block for a luminance channel, a coding block for a first chrominance channel, and a coding block for a second chrominance channel. The substream corresponding to the coding block for the luminance channel is the first substream, the substream corresponding to the coding block for the first chrominance channel is the second substream, and the substream corresponding to the coding block for the second chrominance channel is the third substream. The first substream uses 1 or 3 bits to transmit the complexity level of the luminance channel, the second substream uses 1 or 3 bits to transmit the average value of the two chrominance channels, and the complexity level is not transmitted in the third substream.

[0351] For example, the CU-level complexity is calculated as:

[0352] if(image_format=='000'){ / *YUV400* /

[0353] CuComplexityLevel=ComplexityLevel[0]

[0354] }elseif(image_format=='001'){ / *YUV420* /

[0355] CuComplexityLevel=ComplexityDivide3Table[ComplexityLevel[1]+(ComplexityLevel[0]<<1)]

[0356] }elseif(image_format=='010'){ / *YUV422* /

[0357] CuComplexityLevel=(ComplexityLevel[0]+ComplexityLevel[1])>>1

[0358] }elseif(image_format=='011'||image_format=='100'){ / *YUV444 orRGB444* /

[0359] CuComplexityLevel=ComplexityDivide3Table[ComplexityLevel[0]+(ComplexityLevel[1]<<1)]

[0360] }

[0361] BiasInit is obtained by looking up the table according to the complexity level ComplexityLevel[0] of the luminance channel and the complexity level ComplexityLevel[1] of the chrominance channel, and then the quantization parameter Qp[0] of the luminance channel and the quantization parameters Qp[1] and Qp[2] of the two chrominance channels are calculated.

[0362] The table for the lookup table may refer to the following Table 1, which will not be described in detail here.

[0363] Taking the first sub-stream as an example, the specific implementation of the first sub-stream is as follows:

[0364]

[0365] Among them, complexity_level_flag[0] is the level update flag of the complexity of the luma channel, which is a binary variable. The value of '1' indicates that the complexity level of the luma channel of the coding unit needs to be updated; the value of '0' indicates that the complexity level of the luma channel of the coding unit does not need to be updated. The value of ComplexityLevelFlag[0] is equal to the value of complexity_level_flag[0]. delta_level[0] is the change in the complexity level of the luma channel, which is a 2-bit unsigned integer. Determine the change in the luma complexity level. The value of DeltaLevel[0] is equal to the value of delta_level[0]. If delta_level[0] does not exist in the bitstream, the value of DeltaLevel[0] is equal to 0. PrevComplexityLevel indicates the complexity level of the luma channel of the previous coding unit; ComplexityLevel[0] indicates the complexity level of the luma channel.

[0366] Improvement scheme: Add transmission complexity information in the third sub-stream.

[0367] Embodiment 6

[0368] Fig.21 A flowchart of another image encoding method provided in an embodiment of the present application is shown below. Fig.21 As shown, the image encoding method includes S1101 to S1103.

[0369] S1101. The encoding end obtains a coding unit.

[0370] The coding unit includes coding blocks of P channels, where P is an integer greater than or equal to 2.

[0371] S1102: The encoding end obtains complexity information of the encoding block of each channel among the P channels.

[0372] The complexity information is used to characterize the degree of difference in pixel values ​​of the coding blocks of each channel. For example, taking the case where P channels include a luminance channel, a first chrominance channel, and a second chrominance channel, the complexity information of the coding block of the luminance channel is used to characterize the degree of difference in pixel values ​​of the coding block of the luminance channel; the complexity information of the coding block of the first chrominance channel is used to characterize the degree of difference in pixel values ​​of the coding block of the first chrominance channel; the complexity information of the coding block of the second chrominance channel is used to characterize the degree of difference in pixel values ​​of the coding block of the second chrominance channel;

[0373] S1103 . The encoder encodes complexity information of the coding block of each channel in a substream obtained by encoding the coding blocks of the P channels.

[0374] In a possible implementation, S1103 may specifically include: the encoding end encodes the complexity level of the coding block of each channel in the sub-streams obtained by encoding the coding blocks of the P channels.

[0375] Exemplarily, taking the second substream corresponding to the first chrominance component as an example, the second substream is specifically implemented as follows:

[0376]

[0377] Wherein, complexity_level_flag[1] indicates the complexity level update flag of the first chroma channel, which is a binary variable. A value of '1' indicates that the complexity level of the coding block of the first chroma channel of the coding unit is consistent with the complexity level of the coding block of the luma channel; a value of '0' indicates that the complexity level of the coding block of the first chroma channel of the coding unit is inconsistent with the complexity level of the coding block of the luma channel. The value of ComplexityLevelFlag[1] is equal to the value of complexity_level_flag[1]. delta_level[1] indicates the change in complexity level of the first chroma channel, which is a 2-bit unsigned integer that determines the change in complexity level of the coding block of the first chroma channel. The value of DeltaLevel[1] is equal to the value of delta_level[1]. If delta_level[1] does not exist in the bitstream, the value of DeltaLevel[1] is equal to 0. ComplexityLevel[1] indicates the complexity level of the coding block of the first chroma channel.

[0378] Exemplarily, taking the third substream corresponding to the second chrominance component as an example, the third substream is specifically implemented as follows:

[0379]

[0380]

[0381] Wherein, complexity_level_flag[2] indicates the second chroma channel complexity level update flag, which is a binary variable. A value of '1' indicates that the complexity level of the coding block of the second chroma channel of the coding unit is consistent with the complexity level of the coding block of the first chroma channel; a value of '0' indicates that the complexity level of the coding block of the second chroma channel of the coding unit is inconsistent with the complexity level of the coding block of the first chroma channel. The value of ComplexityLevelFlag[2] is equal to the value of complexity_level_flag[2]. delta_level[2] indicates the change in complexity level of the second chroma channel, which is a 2-bit unsigned integer that determines the change in complexity level of the coding block of the second chroma channel. The value of DeltaLevel[2] is equal to the value of delta_level[2]. If delta_level[2] does not exist in the bitstream, the value of DeltaLevel[2] is equal to 0. ComplexityLevel[2] indicates the complexity level of the coding block of the second chroma channel.

[0382] In another possible implementation, the complexity information includes a complexity level and a first reference coefficient, and the first reference coefficient is used to characterize the proportional relationship between the complexity levels of the coding blocks of different channels. S1103 may specifically include: the encoding end encodes the complexity level of the coding blocks of Q channels in the sub-streams obtained by encoding the coding blocks of Q channels in the P channels; Q is an integer less than P; the encoding end encodes the first reference coefficient in the sub-stream obtained by encoding the coding blocks of PQ channels in the P channels.

[0383] For example, taking the example of P channels including a luma channel, a first chroma channel, and a second chroma channel, the complexity information of the coding block of the luma channel is the complexity level of the coding block of the luma channel; the complexity information of the coding block of the first chroma channel is the complexity level of the coding block of the first chroma channel; the complexity information of the coding block of the second chroma channel is the reference coefficient; the reference coefficient is used to characterize the proportional relationship between the complexity level of the coding block of the first chroma channel and the complexity level of the coding block of the second chroma channel.

[0384] For example, the meaning of the value assigned to the above variable complexity_level_flag[2] is changed. A value of '1' indicates that the complexity level of the coding block of the second chroma channel of the coding unit is greater than the complexity level of the coding block of the first chroma channel; a value of '0' indicates that the complexity level of the coding block of the second chroma channel of the coding unit is less than or equal to the complexity level of the coding block of the first chroma channel. The value of ComplexityLevelFlag[2] is equal to the value of complexity_level_flag[2].

[0385] In another possible implementation, the complexity information includes a complexity level, a reference complexity level, and a second reference coefficient; the reference complexity level includes any one of the following: a first complexity level, a second complexity level, and a third complexity level; the first complexity level is the maximum value of the complexity levels of the coding blocks of PQ channels among P channels; Q is an integer less than P; the second complexity level is the minimum value of the complexity levels of the coding blocks of PQ channels among P channels; the third complexity level is the average value of the complexity levels of the coding blocks of PQ channels among P channels; the second reference coefficient is used to characterize the size relationship and / or the proportional relationship between the complexity levels of the coding blocks of PQ channels among P channels. In this case, the above S1103 may specifically include: the encoding end encodes the complexity level of the coding blocks of Q channels in the substreams obtained by encoding the coding blocks of Q channels among P channels; the encoding end encodes the reference complexity level and the second reference coefficient in the substreams obtained by encoding the coding blocks of PQ channels among P channels.

[0386] For example, taking the example that P channels include a luminance channel, a first chrominance channel, and a second chrominance channel, the complexity information of the coding block of the luminance channel is the complexity level of the coding block of the luminance channel; the complexity information of the coding block of the first chrominance channel is the reference complexity level; the reference complexity level includes any one of the following: the first complexity level, the second complexity level, and the third complexity level; the first complexity level is the maximum value of the complexity level of the coding block of the first chrominance channel and the complexity level of the coding block of the second chrominance channel; the second complexity level is the minimum value of the complexity level of the coding block of the first chrominance channel and the complexity level of the coding block of the second chrominance channel; the third complexity level is the average value of the complexity level of the coding block of the first chrominance channel and the complexity level of the coding block of the second chrominance channel; the complexity information of the coding block of the second chrominance channel is the reference coefficient; the reference coefficient is used to characterize the size relationship, and / or proportional relationship, between the complexity level of the coding block of the first chrominance channel and the complexity level of the coding block of the second chrominance channel.

[0387] It should be understood that in an optional implementation scheme, the complexity information is not transmitted in the third substream corresponding to the coding block of the second chrominance channel. The image coding method provided in the embodiment of the present application increases the number of bits in the substream with a smaller number of bits by adding coding complexity information in the third substream, thereby increasing the denominator in the above theoretical expansion rate calculation formula and reducing the theoretical expansion rate.

[0388] Correspondingly, an embodiment of the present application also provides an image decoding method. Fig. 22 Another image decoding method provided in the embodiment of the present application. Fig. 22 As shown, the decoding method includes S1201 to S1204.

[0389] S1201: The decoding end parses the bit stream after encoding the encoding unit.

[0390] The coding unit includes coding blocks of P channels; P is an integer greater than or equal to 2; and the code stream includes a plurality of sub-streams corresponding to the P channels in a one-to-one manner after the coding blocks of the P channels are encoded.

[0391] S1202: The decoding end parses complexity information of the coding block of each channel in the substream obtained by encoding the coding blocks of P channels.

[0392] In a possible implementation, S1202 may specifically include: the decoding end analyzes the complexity level of the coding block of each channel in the sub-streams obtained by encoding the coding blocks of the P channels.

[0393] S1203: The decoding end determines a quantization parameter of the coding block of each channel based on the complexity information of the coding block of each channel.

[0394] S1204: The decoding end decodes the bitstream based on the quantization parameter of the coding block of each channel.

[0395] Optionally, when the complexity level of each channel is transmitted in the substream corresponding to each channel, the complexity level (CuComplexityLevel) of the CU level can be calculated according to the following process:

[0396] if(image_format=='000'){ / *YUV400* /

[0397] CuComplexityLevel=ComplexityLevel[0]

[0398] }else if(image_format=='001'){ / *YUV420* /

[0399] CuComplexityLevel=ComplexityDivide3Table[((ComplexityLevel[0]<<2)+ComplexityLevel[1]+ComplexityLevel[2])>>1]

[0400] }else if(image_format=='010'){ / *YUV422* /

[0401] CuComplexityLevel=((ComplexityLevel[0]<<1)+ComplexityLevel[1]+ComplexityLevel[2])>>2

[0402] }else if(image_format=='011'||image_format=='100'){ / *YUV444 orRGB444* /

[0403] CuComplexityLevel=ComplexityDivide3Table[ComplexityLevel[0]+ComplexityLevel[1]+ComplexityLevel[2]]

[0404] }

[0405] Among them, ComplexityDivide3Table is defined as: ComplexityDivide3Table={0,0,0,1,1,1,2,2,2,3,3,3,4}; image_format represents the image format of the image to be processed where the coding unit is located.

[0406] In another possible implementation, as described above, the complexity information includes a complexity level and a first reference coefficient; the first reference coefficient is used to characterize the proportional relationship between the complexity levels of the coding blocks of different channels. In this case, S1202 may specifically include: the decoding end parses the complexity levels of the coding blocks of Q channels in the substreams obtained by encoding the coding blocks of Q channels in the P channels respectively; Q is an integer less than P; the decoding end parses the first reference coefficient in the substream obtained by encoding the coding blocks of PQ channels in the P channels; the decoding end determines the complexity levels of the coding blocks of PQ channels based on the first reference coefficient and the complexity levels of the coding blocks of Q channels.

[0407] Optionally, when there is no transmission complexity level in the substream (that is, when the transmission size relationship is in the third substream mentioned above), the complexity level (CuComplexityLevel) of the CU level can be calculated according to the following process:

[0408] if(image_format=='000'){ / *YUV400* /

[0409] CuComplexityLevel=ComplexityLevel[0]

[0410] }else if(image_format=='001'){ / *YUV420* /

[0411] CuComplexityLevel=ComplexityDivide3Table[ChromaComplexityLevel+(ComplexityLevel[0]<<1)]

[0412] }else if(image_format=='010'){ / *YUV422* /

[0413] CuComplexityLevel=(ComplexityLevel[0]+ChromaComplexityLevel)>>1

[0414] }else if(image_format=='011'||image_format=='100'){ / *YUV444 orRGB444* /

[0415] CuComplexityLevel=ComplexityDivide3Table[ComplexityLevel[0]+(ChromaComplexityLevel<<1)]

[0416] }

[0417] Where ChromaComplexityLevel represents the complexity level of the chroma channel.

[0418] ChromaComplexityLevel needs to be calculated separately on the encoding side (the decoding side obtains it directly from the bitstream).

[0419] ChromaComplexityLevel=(ComplexityLevel[1]+ComplexityLevel[2])>>1 or ChromaComplexityLevel=max(ComplexityLevel[1],ComplexityLevel[2]) or ChromaCo complexityLevel=min(ComplexityLevel[1],ComplexityLevel[2]) or ChromaComplexityLevel=ComplexityLevel[1] or ChromaComplexityLevel=ComplexityLevel[2]

[0420] In another possible implementation, as described above, the complexity information includes a complexity level, a reference complexity level, and a second reference coefficient; the reference complexity level includes any one of the following: a first complexity level, a second complexity level, and a third complexity level; the first complexity level is the maximum value of the complexity levels of the coding blocks of PQ channels among P channels; Q is an integer less than P; the second complexity level is the minimum value of the complexity levels of the coding blocks of PQ channels among P channels; the third complexity level is the average value of the complexity levels of the coding blocks of PQ channels among P channels; the second reference coefficient is used to characterize the size relationship, and / or proportional relationship, between the complexity levels of the coding blocks of PQ channels among P channels. In this case, the above S1202 may specifically include: the decoding end respectively analyzes the complexity level of the coding block of the Q channels in the substreams obtained by encoding the coding blocks of the Q channels in the P channels; the decoding end analyzes the reference complexity level and the second reference coefficient in the substreams obtained by encoding the coding blocks of the PQ channels in the P channels; the decoding end determines the complexity level of the coding blocks of the PQ channels based on the complexity level of the coding blocks of the Q channels and the second reference coefficient.

[0421] Optionally, when the complexity level of each channel is transmitted in the sub-stream corresponding to each channel, the decoding end can refer to the following Table 2 to obtain BiasInit1 according to the complexity level ComplexityLevel[0] of the coding block of the luminance channel and the complexity level ComplexityLevel[1] of the coding block of the first chrominance channel, and refer to the following Table 2 to obtain BiasInit2 according to the complexity level ComplexityLevel[0] of the coding block of the luminance channel and the complexity level ComplexityLevel[2] of the coding block of the second chrominance channel, and then calculate the quantization parameter Qp[0] of the luminance channel, the quantization parameter Qp[1] of the first chrominance channel, and the quantization parameter Qp[2] of the second chrominance channel.

[0422] For example, the decoder can calculate the quantization parameter according to the following process:

[0423] Bias1=(BiasInit1×FormatBias)>>1

[0424] Bias2=(BiasInit2×FormatBias)>>1

[0425] tmp=(ChromaSampleRate>>1)×(Bias1+Bias2)

[0426] tmp=((tmp<<7)+256)>>9

[0427] Qp[0]=Cl ip3(0,MaxQp[0],(MasterQp–tmp)>>7)

[0428] Qp[1]=Clip3(0,MaxQp[1],(MasterQp+Bias1)>>7)

[0429] Qp[2]=Clip3(0,MaxQp[2],(MasterQp+Bias2)>>7)

[0430] Table 1 Correspondence between ChromaSampleRate, FormatBias and ImageFormat

[0431]

[0432] Table 2 Definition of BiasInit

[0433]

[0434] Optionally, the decoding end may also look up Table 2 to obtain BiasInit according to the complexity level ComplexityLevel[0] of the coding block of the chrominance channel and the complexity level ChromaComplexityLevel of the coding block of the chrominance channel, and then calculate Qp[0], Qp[1], and Qp[2] through the following process.

[0435] Bias=(BiasInit×FormatBias)>>1

[0436] tmp=ChromaSampleRate×Bias

[0437] tmp=((tmp<<7)+128)>>8

[0438] Qp[0]=Clip3(0,MaxQp[0],(MasterQp–tmp)>>7)

[0439] Qp[1]=Clip3(0,MaxQp[1],(MasterQp+Bias)>>7)

[0440] Qp[2]=Qp[1]

[0441] Qp[1]=Clip3(0,MaxQp[1],Qp[1]+ComplexityLevel[1]-ChromaComplexityLevel)

[0442] Qp[2]=Clip3(0,MaxQp[2],Qp[2]+ComplexityLevel[2]-ChromaComplexityLevel)

[0443] Optionally, as described above, the sub-stream corresponding to the coding block of the first chroma channel can transmit the first complexity level and the second complexity level. In this case, the decoding end can obtain BiasInit by looking up Table 2 according to the complexity level ComplexityLevel[0] of the coding block of the luminance channel and the complexity level ChromaComplexityLevel of the coding block of the chroma channel (taking the first complexity level / complexity level of the coding block of the first chroma channel), and then calculate Qp[0], Qp[1], and Qp[2] through the following process.

[0444] Bias=(BiasInit×FormatBias)>>1

[0445] tmp=ChromaSampleRate×Bias

[0446] tmp=((tmp<<7)+128)>>8

[0447] Qp[0]=Clip3(0,MaxQp[0],(MasterQp–tmp)>>7)

[0448] if(0==complexity_level_flag[2]){

[0449] Qp[1]=Clip3(0,MaxQp[1],(MasterQp+Bias)>>7)

[0450] Qp[2]=Clip3(0,MaxQp[2],(MasterQp+Bias-32)>>7)

[0451] }elseif(1==complexity_level_flag[2]){

[0452] Qp[1]=Clip3(0,MaxQp[1],(MasterQp+Bias-32)>>7)

[0453] Qp[2]=Clip3(0,MaxQp[2],(MasterQp+Bias)>>7)

[0454] }

[0455] Optionally, as described above, the substream corresponding to the coding block of the first chroma channel may be transmitted at a third complexity level. In this case, the ChromaComplexityLevel may take the third complexity level, and Qp[0], Qp[1], and Qp[2] may be calculated by the following process.

[0456] Bias=(BiasInit×FormatBias)>>1

[0457] tmp=ChromaSampleRate×Bias

[0458] tmp=((tmp<<7)+128)>>8

[0459] Qp[0]=Clip3(0,MaxQp[0],(MasterQp–tmp)>>7)

[0460] Qp[1]=Clip3(0,MaxQp[1],(MasterQp+Bias)>>7)

[0461] Qp[2]=Qp[1]

[0462] if(0==complexity_level_flag[2]){

[0463] Qp[1]=Clip3(0,MaxQp[1],Qp[1]+2)

[0464] }elseif(1==complexity_level_flag[2]){

[0465] Qp[2]=Clip3(0,MaxQp[2],Qp[2]+1)

[0466] }

[0467] 6. Chroma channels share a substream.

[0468] Optional implementation scheme: For the image to be processed in the YUV420 / YUV422 format, three substreams are transmitted in total, namely, the first substream, the second substream, and the third substream. The first substream includes the syntax elements and transform coefficients / residual coefficients of the luma channel, the second substream includes the syntax elements and transform coefficients / residual coefficients of the first chroma channel, and the third substream includes the syntax elements and transform coefficients / residual coefficients of the second chroma channel.

[0469] Improved solution: For the to-be-processed image in YUV420 / YUV422 format, the syntax elements and transform coefficients / residual coefficients of the first chroma channel and the syntax elements and transform coefficients / residual coefficients of the second chroma channel are transmitted in the second substream, and the third substream is cancelled.

[0470] Embodiment seven:

[0471] Fig.23 A flowchart of another image encoding method provided in an embodiment of the present application is shown below. Fig.23 As shown, the image encoding method includes S1301 to S1302.

[0472] S1301. The encoding end obtains a coding unit.

[0473] The coding unit is an image block in the image to be processed, and includes coding blocks of multiple channels.

[0474] S1302: When the image format of the to-be-processed image is a preset format, the encoding end merges substreams obtained by encoding coding blocks of at least two preset channels among the multiple channels into a merged substream.

[0475] The preset format may be preset in the encoding / decoding end, for example, the preset format may be YUV420 or YUV422, etc., and the preset channel may be a first chroma channel and a second chroma channel. Exemplarily, the definition of the coding block of the first substream may be as follows:

[0476]

[0477] Exemplarily, the definition of the coding block of the second sub-stream may be as follows:

[0478]

[0479]

[0480] It should be understood that the number of bits after encoding the syntax elements and quantized transform coefficients of the two chrominance channels is usually smaller than that of the luminance channel. Fusion of the syntax elements of the two chrominance channels in the second substream for transmission can reduce the difference in the number of bits between the first substream and the second substream, thereby reducing the theoretical expansion rate.

[0481] Correspondingly, the embodiment of the present application also provides an image decoding method, Fig.24 A flowchart of another image decoding method provided in an embodiment of the present application is shown as follows: Fig.24 As shown, the method includes S1401 to S1403.

[0482] S1401: The decoding end parses the bit stream encoded by the encoding unit.

[0483] S1402: The decoding end determines a merged sub-stream.

[0484] The merged substream is obtained by merging substreams encoded by encoding blocks of at least two preset channels among the multiple channels when the image format of the image to be processed is a preset format.

[0485] S1403: The decoding end decodes the bitstream based on the merging of at least two substreams to be merged.

[0486] 7. Sub-stream filling.

[0487] Improved solution: fill the substream whose bit number is less than the bit number threshold with a preset codeword.

[0488] Embodiment eight:

[0489] Fig.25 A flowchart of another image encoding method provided in an embodiment of the present application is shown as follows: Fig.25 As shown, the method includes S1501 to S1502.

[0490] S1501. The encoding end obtains a coding unit.

[0491] The encoding unit includes encoding blocks of multiple channels.

[0492] S1502: The encoder encodes a preset codeword in a target substream that meets a preset condition until the target substream does not meet the preset condition.

[0493] The target substream is a substream in the multiple substreams. The multiple substreams are code streams obtained by encoding the coding blocks of multiple channels. The preset codeword may be "0" or other codewords, etc. This embodiment of the application does not limit this.

[0494] In a possible implementation manner, the preset condition includes: the number of bits of the substream is less than a preset first bit number threshold.

[0495] The first bit number threshold may be preset at the encoder / decoder, or transmitted by the encoder / decoder in a bit stream. This embodiment of the present application does not limit this. The first bit number threshold is used to indicate the minimum number of bits of the CB allowed in the CU.

[0496] In another possible implementation manner, the preset condition includes: there is an encoded coding block having a bit number less than a preset second bit number threshold in the code stream of the coding unit.

[0497] The second bit number threshold may also be preset at the encoder / decoder, or transmitted by the encoder / decoder in a bit stream. This embodiment of the present application does not limit this. The second bit number threshold is used to indicate the minimum number of bits allowed in a substream among multiple substreams.

[0498] It should be understood that filling the preset codeword in the substream with a smaller number of bits can directly increase the denominator of the expansion rate, thereby reducing the actual expansion rate of the coding unit.

[0499] Correspondingly, an embodiment of the present application also provides an image decoding method. Fig.26 A flowchart of another image decoding method provided in an embodiment of the present application is shown as follows: Fig.26 As shown, the image decoding method includes S1601 to S1603.

[0500] S1601: The decoding end parses the bit stream after encoding the encoding unit.

[0501] The encoding unit includes encoding blocks of multiple channels.

[0502] S1602: The decoding end determines the number of code words.

[0503] The number of codewords is used to indicate the number of preset codewords encoded in the target substream that meets the preset condition. The preset codeword is encoded into the target substream when there is a target substream that meets the preset condition.

[0504] S1603: The decoding end decodes the code stream based on the number of code words.

[0505] For example, the decoding end may delete the preset codeword based on the number of codewords, and decode the code stream after the preset codeword is deleted.

[0506] 8. Preset expansion rate.

[0507] Improvement plan: Use the preset expansion rate as a threshold to control the current actual expansion rate.

[0508] Embodiment nine:

[0509] Fig. 27 A flowchart of another image encoding method provided in an embodiment of the present application is shown as follows: Fig. 27 As shown, the image encoding method includes S1701 to S1703

[0510] S1701. The encoding end obtains a coding unit.

[0511] The encoding unit includes encoding blocks of multiple channels.

[0512] S1702: The encoding end determines a target encoding mode corresponding to each encoding block of each channel in the encoding blocks of the multiple channels based on a preset expansion rate.

[0513] The preset expansion rate can be

[0514] The preset expansion rate is preset in the encoding end and the decoding end. Alternatively, the preset expansion rate can also be encoded by the encoding end into a substream and transmitted to the decoding end. The embodiment of the present application does not limit this.

[0515] S1703: The encoding end encodes the coding block of each channel according to the target coding mode, so that the current expansion rate is less than or equal to the preset expansion rate.

[0516] In a possible implementation, the preset expansion rate includes a first preset expansion rate; the current expansion rate is equal to the quotient of the number of bits of the maximum sub-stream and the number of bits of the minimum sub-stream; the maximum sub-stream is a sub-stream with the largest number of bits among multiple sub-streams obtained by encoding coding blocks of multiple channels; the minimum sub-stream is a sub-stream with the smallest number of bits among multiple sub-streams obtained by encoding coding blocks of multiple channels.

[0517] Optionally, as described above, the encoder may preset a first preset expansion rate as the maximum threshold allowed in the coding unit, and before encoding the coding unit, obtain all current substream states, obtain the substream with the most fixed-length code streams sent, plus the substream state with the largest total amount of the remaining amount in the current substream, and obtain the substream with the least fixed-length code streams sent, plus the substream state with the smallest total amount of the remaining amount in the current substream. If the ratio of the maximum state to the minimum state, or the difference is greater than the preset difference threshold, the encoder may no longer use rate-distortion optimization as the target mode selection criterion, and only select a coding mode with a lower code stream for the substream in the maximum state, and select a mode with a higher code rate for the substream in the minimum state.

[0518] Exemplarily, with a preset expansion rate of B_th and B_delta set as an auxiliary threshold, the coding blocks of the three channels Y / U / V are encoded to obtain three sub-streams. When encoding a coding unit, the encoding end can obtain the states of the three sub-streams. If the maximum bit rate corresponding to the first sub-stream (the sub-stream corresponding to the Y channel) is set to bit_stream1, and the minimum bit rate corresponding to the second sub-stream (the sub-stream corresponding to the U channel) is set to bit_stream1, then when bit_stream1 / bit_stream2>B_th-B_delta, the encoding end can select a mode with a larger bit rate for the coding block of the Y channel and a mode with a smaller bit rate for the coding block of the U channel.

[0519] In another possible implementation, the preset expansion rate includes a second preset expansion rate; the current expansion rate is equal to the quotient of the number of bits of the coding block with the largest bit number and the number of bits of the coding block with the smallest bit number among the coded coding blocks of the multiple channels.

[0520] Optionally, as described above, the encoder may preset a second preset expansion rate as the maximum threshold allowed in the encoding unit, and obtain the code rate cost in each mode before encoding a certain coding block. If the coding mode corresponding to the optimal code rate cost obtained according to the rate-distortion cost satisfies that the actual expansion rate of each substream encoded is less than the preset expansion rate, the coding mode corresponding to the optimal code rate cost is used as the target coding mode of the coding block, and the target coding mode is encoded into the substream corresponding to the coding block; if the coding mode corresponding to the optimal code rate cost cannot satisfy that the actual expansion rate of each substream encoded is less than the preset expansion rate, the encoder changes the maximum code rate mode to a code rate mode with a code rate less than the maximum code rate mode, or changes the minimum code rate mode to a code rate mode with a code rate greater than the minimum code rate mode, or changes the maximum code rate mode to a code rate mode with a code rate less than the maximum code rate mode, and changes the minimum code rate mode to a code rate mode with a code rate greater than the minimum code rate mode, or changes the maximum code rate mode to a code rate mode with a code rate less than the maximum code rate mode, and changes the minimum code rate mode to a code rate mode with a code rate greater than the minimum code rate mode.

[0521] Exemplarily, taking the preset expansion rate A_th as an example, the coding blocks of the three channels Y / U / V are encoded to obtain three sub-streams. Assume that the optimal bit rate costs of the three channels Y / U / V are rate_y, rate_u, and rate_v, respectively, and rate_y has the largest bit rate and rate_u has the smallest bit rate. If rate_y / rate_u≥A_th, the encoding end can modify the target encoding mode of the Y channel to satisfy the bit rate cost less than rate_y, or modify the target encoding mode of the U channel to satisfy the bit rate cost greater than rate_u, so that rate_y / rate_u<A_th, and rate_y / rate_v<A_th.

[0522] The image encoding method provided in the embodiment of the present application can also intervene in the target encoding mode selected for the encoding block of each channel by presetting the expansion rate, so that when the encoding end encodes each channel according to the target encoding mode, the actual expansion rate of the encoding unit is less than the preset expansion rate, thereby reducing the actual expansion rate.

[0523] In some possible embodiments, the image encoding method may further include: the encoding end determines a current expansion rate; when the current expansion rate is greater than a first preset expansion rate, encoding a preset codeword in a minimum substream so that the current expansion rate is less than or equal to the first preset expansion rate.

[0524] For example, the encoder may fill the end of the smallest substream with a preset codeword.

[0525] In some other possible embodiments, the image encoding method may further include: the encoding end determines a current expansion; when the current expansion rate is greater than a second preset expansion, the encoding end encodes a preset codeword in a coding block with a minimum number of bits so that the current expansion rate is less than or equal to the second preset expansion rate.

[0526] Correspondingly, an embodiment of the present application also provides an image decoding method. Fig.28 A flowchart of another image decoding method provided in an embodiment of the present application is shown below. Fig. 27 As shown, the method includes S1801 to S1803.

[0527] S1801: The decoding end parses the bit stream after encoding the encoding unit.

[0528] S1802: The decoding end determines the number of preset code words based on the bit stream.

[0529] S1803: The decoding end decodes the code stream based on the number of preset code words.

[0530] S1801 to S1803 may refer to the above-mentioned S1601 to S1603, and will not be repeated here.

[0531] It should be noted that the above-mentioned image encoding and decoding method provided by the embodiment of the present application is introduced in a series of separate embodiments. In actual use, the above-mentioned series of embodiments and the optional solutions in the embodiments can be used in combination with each other. The embodiment of the present application does not limit the specific combination.

[0532] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of the method. In order to achieve the above functions, it includes hardware structures and / or software modules corresponding to the execution of each function. It should be easy to realize that the technical goals in this field are combined with the units and algorithm steps of each example described in the embodiments disclosed in this article, and the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical goals can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0533] In an exemplary embodiment, the present application also provides an image encoding device, and any of the above-mentioned image encoding methods can be executed by the image encoding device. The image encoding device provided in the present application embodiment can be the above-mentioned source device 10 or video encoder 102 or destination device 11 or video decoder 112. Fig.29 The following is a schematic diagram of the composition of the image encoding device provided in the embodiment of the present application. Fig.29 As shown, the image encoding device includes an acquisition module 29012901 and a processing module 29022902.

[0534] Optionally, the acquisition module 2901 is used to acquire a coding unit; the coding unit is an image block in the image to be processed; the coding unit includes coding blocks of multiple channels; the multiple channels include a first channel; the first channel is any one of the multiple channels. The processing module 2902 is used to encode the coding block of the first channel according to a first coding mode; the first coding mode is a mode for encoding sample values ​​in the coding block of the first channel according to a first fixed-length code; the code length of the first fixed-length code is less than or equal to the image bit width of the image to be processed; the image bit width is used to represent the number of bits required to store each sample in the image to be processed.

[0535] Optionally, the acquisition module 2901 is further used to obtain a code stream after encoding the coding unit; the coding unit is an image block in the image to be processed; the coding unit includes coding blocks of multiple channels; the multiple channels include a first channel; the first channel is any one of the multiple channels; the code stream includes multiple sub-streams corresponding to the multiple channels after the coding blocks of the multiple channels are encoded. The processing module 2902 is also used to decode the sub-stream corresponding to the first channel according to the first decoding mode; the first decoding mode is a mode of parsing sample values ​​from the sub-stream corresponding to the first channel according to the first fixed-length code; the code length of the first fixed-length code is less than or equal to the image bit width of the image to be processed; the image bit width is used to represent the number of bits required to store each sample in the image to be processed.

[0536] Optionally, the acquisition module 2901 is further used to acquire a coding unit; the coding unit is an image block in the image to be processed; the coding unit includes coding blocks of multiple channels. The processing module 2902 is also used to determine a first total code length; the first total code length is the total code length of the first code stream obtained after the coding blocks of multiple channels are encoded according to their respective corresponding target coding modes; the target coding mode includes a first coding mode; the first coding mode is a mode for encoding sample values ​​in the coding block according to a first fixed-length code; the code length of the first fixed-length code is less than or equal to the image bit width of the image to be processed; the image bit width is used to represent the number of bits required to store each sample in the image to be processed; when the first total code length is greater than or equal to the remaining size of the code stream buffer, the coding blocks of multiple channels are encoded according to the fallback mode; the fallback mode and the mode flag of the first coding mode are the same.

[0537] Optionally, the processing module 2902 is further configured to encode a mode flag in a plurality of sub-streams obtained by encoding the coding blocks of the plurality of channels; the mode flag is used to indicate the coding modes respectively adopted by the coding blocks of the plurality of channels.

[0538] Optionally, the multiple channels include a first channel; the first channel is any one of the multiple channels; the processing module 2902 is specifically used to encode a sub-mode flag in a sub-stream obtained by encoding the coding block of the first channel; the sub-mode flag is used to indicate the type of fallback mode adopted by the coding blocks of multiple channels.

[0539] Optionally, the multiple channels include a first channel; the first channel is any one of the multiple channels; the processing module 2902 is specifically used to encode a first flag, a second flag, and a third flag in a sub-stream obtained by encoding the coding block of the first channel; the first flag is used to indicate that the coding blocks of the multiple channels are encoded using a first coding mode or a fallback mode; the second flag is used to indicate that the coding blocks of the multiple channels are encoded using a target mode, and the target mode is any one of the first coding mode and the fallback mode; when the second flag indicates that the target mode adopted by the coding blocks of the multiple channels is the fallback mode, the third flag is used to indicate the type of fallback mode adopted by the coding blocks of the multiple channels.

[0540] Optionally, the processing module 2902 is also used to determine the target encoding code length of the coding unit based on the remaining size of the code stream buffer and the target pixel depth BPP; the target encoding code length is used to indicate the code length required for encoding the coding unit; based on the encoding code length, the allocated code lengths of multiple channels are determined; the allocated code length is used to indicate the code length required for encoding the residual of the coding blocks of multiple channels; and the encoding code lengths allocated to each of the multiple channels are determined according to the average value of the allocated code lengths in the multiple channels.

[0541] Optionally, the processing module 2902 is further used to parse the bitstream after encoding the coding unit; the coding unit includes coding blocks of multiple channels; if a mode flag is parsed from a substream obtained by encoding the coding blocks of the multiple channels, and the first total code length is greater than the remaining size of the codestream buffer, then determine that the target decoding mode of the substream is the fallback mode; the mode flag is used to indicate that the coding blocks of the multiple channels are encoded using the first coding mode or the fallback mode; the first total code length is the total code length of the first bitstream obtained after the coding blocks of the multiple component channels are encoded according to the first coding mode corresponding to each of the target coding modes; the target coding mode includes the first coding mode; The first coding mode is a mode for encoding the sample value coding unit in the coding block according to the first fixed-length code; the code length of the first fixed-length code is less than or equal to the image bit width of the image to be processed; the image bit width is used to characterize the number of bits required to encode and store each sample in the image to be processed; the preset flag bit in the sub-stream is parsed to determine the target fallback mode; the target fallback mode is one of the fallback modes; the preset flag bit is used to indicate the position of the sub-mode flag; the sub-mode flag is used to indicate the type of fallback mode adopted when encoding the coding blocks of multiple channels; the fallback mode includes a first fallback mode and a second fallback mode; the sub-stream is decoded according to the target fallback mode.

[0542] Optionally, the processing module 2902 is further used to determine a target decoding code length of the coding unit based on the remaining size of the code stream buffer and the target pixel depth BPP; the target decoding code length is used to indicate the code length required for decoding the code stream of the coding unit; based on the decoding code length; determine the allocated code lengths of multiple channels; the allocated code length is used to indicate the code length required for decoding the residual of the code stream of the coding blocks of multiple channels; and determine the decoding code lengths allocated to each of the multiple channels according to the average value of the allocated code lengths in the multiple channels.

[0543] Optionally, the processing module 2902 is further used to parse the bit stream after encoding the coding unit; the coding unit includes coding blocks of multiple channels; the multiple channels include a first channel; the first channel is any one of the multiple channels; a first flag is parsed from a substream encoded by the coding block of the first channel; the first flag is used to indicate that the coding blocks of the multiple channels are encoded using a first coding mode or a fallback mode; the first coding mode is a mode for encoding the sampling values ​​in the coding block of the first channel according to a first fixed-length code; the code length of the first fixed-length code is less than or equal to the image bit width of the image to be processed; the image bit width is used to characterize the coding storage The number of bits required for each sample in the image to be processed is stored; a second flag is parsed from a substream obtained by encoding the coding block of the first channel; the second flag is used to indicate the target mode encoding of the coding blocks of multiple channels; the target mode is any one of the first encoding mode and the fallback mode; when the second flag indicates that the target mode adopted by the coding blocks of multiple channels is the fallback mode, a third flag is parsed from a substream obtained by encoding the coding blocks of the first channel; the target decoding modes of the multiple channels are determined according to the type of the fallback mode indicated by the third flag, and the substreams obtained by encoding the coding blocks of the multiple channels are decoded according to the target decoding mode.

[0544] Optionally, the processing module 2902 is further used to determine a target decoding code length of the coding unit based on the remaining size of the code stream buffer and the target pixel depth BPP; the target decoding code length is used to indicate the code length required for decoding the code stream of the coding unit; based on the decoding code length; determine the allocated code lengths of multiple channels; the allocated code length is used to indicate the code length required for decoding the residual of the code stream of the coding blocks of multiple channels; and determine the decoding code lengths allocated to each of the multiple channels according to the average value of the allocated code lengths in the multiple channels.

[0545] Optionally, the acquisition module 2901 is further used to acquire a coding unit; the coding unit includes coding blocks of multiple channels. The processing module 2902 is further used to encode the coding blocks of at least one channel among the multiple channels according to the intra-frame block copy IBC mode; obtain a block vector BV of a reference prediction block; the BV of the reference prediction block is used to indicate the position of the reference prediction block in the encoded image block; the reference prediction block is used to represent the prediction value of the coding block encoded according to the IBC mode; encode the BV of the reference prediction block in at least one substream obtained by encoding the coding blocks of at least one channel in the IBC mode.

[0546] Optionally, the processing module 2902 is specifically configured to determine a target coding mode of at least one channel by using a rate-distortion optimization decision; the target coding mode includes an IBC mode; and encode a coding block of at least one channel according to the target coding mode.

[0547] Optionally, the processing module 2902 is specifically configured to encode the coding blocks of multiple channels according to the IBC mode; and encode the BV of the reference prediction block in a substream obtained by encoding the coding blocks of each channel in the multiple channels in the IBC mode.

[0548] Optionally, the processing module 2902 is specifically configured to encode the BV of the reference prediction block in a bitstream obtained by encoding the coding block of each channel in the multiple channels in the IBC mode according to a preset ratio.

[0549] Optionally, the processing module 2902 is also used to parse the code stream after encoding the coding unit; the coding unit includes coding blocks of multiple channels; the code stream includes multiple sub-streams corresponding to the multiple channels after the coding blocks of multiple channels are encoded; based on the block vector BV of the reference prediction block parsed from at least one of the multiple sub-streams, the position of the reference prediction block is determined in the multiple sub-streams; the reference prediction block is used to represent the prediction value of the coding block encoded according to the intra-frame block copy IBC mode; the BV of the reference prediction block is used to indicate the position of the reference prediction block in the encoded image block; based on the position information of the reference prediction block, the prediction value of the decoding block decoded according to the IBC mode is determined; based on the prediction value, the decoding block decoded according to the IBC mode is reconstructed.

[0550] Optionally, the BV of the reference prediction block is encoded in multiple sub-streams; the BV of the reference prediction block is obtained when encoding coding blocks of multiple channels according to the IBC mode.

[0551] Optionally, the acquisition module 2901 is further used to acquire processing coefficients corresponding to the coding unit; the processing coefficients include one or more of residual coefficients and transform coefficients. The processing module 2902 is further used to divide the processing coefficients into multiple groups according to the number threshold, and the number of processing coefficients in each group of the multiple groups of processing coefficients is less than or equal to the number threshold.

[0552] Optionally, the processing module 2902 is further used to parse the bitstream after encoding the coding unit to determine the processing coefficient corresponding to the coding unit; the processing coefficient includes one or more of the residual coefficient and the transform coefficient; the processing coefficient includes multiple groups; the number of processing coefficients in each group is less than or equal to the number threshold; and the bitstream is decoded based on the processing coefficient.

[0553] Optionally, the acquisition module 2901 is further used to acquire a coding unit; the coding unit includes coding blocks of P channels; P is an integer greater than or equal to 2; the complexity information of the coding block of each channel in the P channels is acquired; the complexity information is used to characterize the degree of difference of the pixel values ​​of the coding block of each channel. The processing module 2902 is further used to encode the complexity information of the coding block of each channel in the substream obtained by encoding the coding blocks of the P channels.

[0554] Optionally, the complexity information includes a complexity level; the processing module 2902 is specifically configured to encode the complexity level of the coding block of each channel in the sub-streams obtained by encoding the coding blocks of the P channels.

[0555] Optionally, the complexity information includes a complexity level and a first reference coefficient; the first reference coefficient is used to characterize the proportional relationship between the complexity levels of the coding blocks of different channels; the processing module 2902 is specifically used to encode the complexity level of the coding blocks of Q channels in the sub-streams obtained by encoding the coding blocks of Q channels among the P channels; Q is an integer less than P; and the first reference coefficient is encoded in the sub-stream obtained by encoding the coding blocks of PQ channels among the P channels.

[0556] Optionally, the complexity information includes a complexity level, a reference complexity level, and a second reference coefficient; the reference complexity level includes any one of the following: a first complexity level, a second complexity level, and a third complexity level; the first complexity level is the maximum value of the complexity levels of the coding blocks of PQ channels among P channels; Q is an integer less than P; the second complexity level is the minimum value of the complexity levels of the coding blocks of PQ channels among P channels; the third complexity level is the average value of the complexity levels of the coding blocks of PQ channels among P channels; the second reference coefficient is used to characterize the size relationship, and / or proportional relationship, between the complexity levels of the coding blocks of PQ channels among P channels; the processing module 2902 is specifically used to encode the complexity levels of the coding blocks of Q channels in the sub-streams obtained by encoding the coding blocks of Q channels among P channels; and encode the reference complexity level and the second reference coefficient in the sub-streams obtained by encoding the coding blocks of PQ channels among P channels.

[0557] Optionally, the processing module 2902 is also used to parse the code stream after encoding the coding unit; the coding unit includes coding blocks of P channels; P is an integer greater than or equal to 2; the code stream includes multiple sub-streams corresponding to the P channels after the coding blocks of P channels are encoded; the complexity information of the coding blocks of each channel is parsed in the sub-streams obtained by encoding the coding blocks of P channels; the complexity information is used to characterize the degree of difference in pixel values ​​of the coding blocks of each channel; based on the complexity information of the coding blocks of each channel, the quantization parameters of the coding blocks of each channel are determined; based on the quantization parameters of the coding blocks of each channel, the code stream is decoded.

[0558] Optionally, the complexity information includes a complexity level; the processing module 2902 is further configured to parse the complexity level of the coding block of each channel in the sub-streams obtained by encoding the coding blocks of the P channels.

[0559] Optionally, the complexity information includes a complexity level and a first reference coefficient; the first reference coefficient is used to characterize the proportional relationship between the complexity levels of the coding blocks of different channels; the processing module 2902 is specifically used to parse the complexity levels of the coding blocks of Q channels in the sub-streams obtained by encoding the coding blocks of Q channels among the P channels; Q is an integer less than P; parse the first reference coefficient in the sub-stream obtained by encoding the coding blocks of PQ channels among the P channels; based on the first reference coefficient and the complexity levels of the coding blocks of the Q channels, determine the complexity levels of the coding blocks of the PQ channels.

[0560] Optionally, the complexity information includes a complexity level, a reference complexity level, and a second reference coefficient; the reference complexity level includes any one of the following: a first complexity level, a second complexity level, and a third complexity level; the first complexity level is the maximum value of the complexity levels of the coding blocks of PQ channels among P channels; Q is an integer less than P; the second complexity level is the minimum value of the complexity levels of the coding blocks of PQ channels among P channels; the third complexity level is the average value of the complexity levels of the coding blocks of PQ channels among P channels; the second reference coefficient is used to characterize the size relationship and / or proportional relationship between the complexity levels of the coding blocks of PQ channels among P channels; the processing module 2902 is specifically used to parse the complexity levels of the coding blocks of Q channels in the sub-streams obtained by encoding the coding blocks of Q channels among P channels; parse the reference complexity level and the second reference coefficient in the sub-streams obtained by encoding the coding blocks of PQ channels among P channels; based on the complexity levels of the coding blocks of Q channels. and a second reference coefficient to determine the complexity level of the coding blocks of the PQ channels.

[0561] Optionally, the acquisition module 2901 is further used to acquire a coding unit; the coding unit includes coding blocks of multiple channels; the coding unit is an image block in the image to be processed. The processing module 2902 is further used to merge substreams obtained by encoding coding blocks of at least two preset channels in the multiple channels into a merged substream when the image format of the image to be processed is a preset format.

[0562] Optionally, the processing module 2902 is also used to parse the code stream after encoding the coding unit; the coding unit includes coding blocks of multiple channels; the coding unit is an image block in the image to be processed; determine the merged sub-stream; the merged sub-stream is obtained by merging the sub-streams encoded by the coding blocks of at least two preset channels in the multiple channels when the image format of the image to be processed is a preset format; based on the merging of at least two sub-streams to be merged, decode the code stream.

[0563] Optionally, the acquisition module 2901 is further used to acquire a coding unit; the coding unit includes coding blocks of multiple channels. The processing module 2902 is further used to encode a preset codeword in a target substream that meets a preset condition until the target substream does not meet the preset condition; the target substream is a substream in multiple substreams; and the multiple substreams are code streams obtained by encoding coding blocks of multiple channels.

[0564] Optionally, the preset condition includes: the number of bits of the substream is less than a preset first bit number threshold.

[0565] Optionally, the preset condition includes: there is an encoded coding block with a bit number less than a preset second bit number threshold in the code stream of the coding unit.

[0566] Optionally, the processing module 2902 is also used to parse the code stream after encoding the coding unit; the coding unit includes coding blocks of multiple channels; determine the number of codewords; the number of codewords is used to indicate the number of preset codewords encoded in the target substream that meets the preset conditions; the preset codeword is encoded into the target substream when there is a target substream that meets the preset conditions; based on the number of codewords, decode the code stream.

[0567] Optionally, the acquisition module 2901 is further used to acquire a coding unit; the coding unit includes coding blocks of multiple channels. The processing module 2902 is further used to determine, based on a preset expansion rate, a target coding mode corresponding to each coding block of each channel in the coding blocks of the multiple channels; and encode the coding block of each channel according to the target coding mode so that the current expansion rate is less than or equal to the preset expansion rate.

[0568] Optionally, the preset expansion rate includes a first preset expansion rate; the current expansion rate is equal to the quotient of the number of bits of the maximum sub-stream and the number of bits of the minimum sub-stream; the maximum sub-stream is the sub-stream with the largest number of bits among the multiple sub-streams obtained by encoding the coding blocks of multiple channels; the minimum sub-stream is the sub-stream with the smallest number of bits among the multiple sub-streams obtained by encoding the coding blocks of multiple channels.

[0569] Optionally, the preset expansion rate includes a second preset expansion rate; the current expansion rate is equal to the quotient of the number of bits of the coding block with the largest number of bits and the number of bits of the coding block with the smallest number of bits in the coded blocks of the multiple channels.

[0570] Optionally, the processing module 2902 is further configured to determine a current expansion rate; when the current expansion rate is greater than a first preset expansion rate, encode a preset codeword in the smallest substream so that the current expansion rate is less than or equal to the first preset expansion rate.

[0571] Optionally, the processing module 2902 is further configured to determine a current expansion rate; when the current expansion rate is greater than a second preset expansion rate, encode a preset codeword in a coding block with a minimum number of bits so that the current expansion rate is less than or equal to the second preset expansion rate.

[0572] Optionally, the processing module 2902 is further used to parse the code stream after encoding the coding unit; the coding unit includes coding blocks of multiple channels; based on the code stream, determine the number of preset code words; based on the number of preset code words, decode the code stream.

[0573] It should be noted that Fig.28The division of modules in the example is schematic and is only a logical function division. There may be other division methods in actual implementation. For example, two or more functions may be integrated into one processing module. The above integrated modules may be implemented in the form of hardware or software function modules.

[0574] In an exemplary embodiment, the present application also provides a readable storage medium, including execution instructions, which, when executed on an image coding and decoding device, enable the image coding and decoding device to execute any one of the methods provided in the above embodiments.

[0575] In an exemplary embodiment, the present application also provides a computer program product including execution instructions, which, when executed on an image coding and decoding device, enables the image coding and decoding device to execute any one of the methods provided in the above embodiments.

[0576] In an exemplary embodiment, the embodiment of the present application further provides a chip, including: a processor and an interface, the processor is coupled to a memory through the interface, when the processor executes a computer program in the memory or the image encoding and decoding device executes instructions, any one of the methods provided in the above embodiments is executed.

[0577] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using a software program, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer-executable instructions. When the computer-executable instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer-executable instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer-executable instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains one or more servers that can be integrated with the medium. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium (eg, a solid state disk (SSD)).

[0578] Although the present application is described herein in conjunction with various embodiments, in the process of implementing the claimed application, those skilled in the art may understand and implement other variations of the disclosed embodiments by viewing the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "one" or "an" does not exclude multiple situations. A single processor or other unit may implement several functions listed in the claims. Certain measures are recorded in different dependent claims, but this does not mean that these measures cannot be combined to produce good results.

[0579] Although the present application has been described in conjunction with specific features and embodiments thereof, it is obvious that various modifications and combinations may be made thereto without departing from the spirit and scope of the present application. Accordingly, this specification and the drawings are merely exemplary illustrations of the present application as defined by the appended claims, and are deemed to have covered any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, those skilled in the art may make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

[0580] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An image coding method, characterized in that: The method comprises: Acquire a coding unit; the coding unit includes coding blocks of multiple channels; Encoding a coding block of at least one of the multiple channels according to an intra block copy (IBC) mode; Obtaining a block vector BV of a reference prediction block; the BV of the reference prediction block is used to indicate a position of the reference prediction block in an encoded image block; the reference prediction block is used to represent a prediction value of a coding block encoded according to the IBC mode; Encoding the BV of the reference prediction block in at least one substream obtained by encoding the coding block of the at least one channel in the IBC mode; The step of encoding the coding block of at least one of the multiple channels according to the intra block copy (IBC) mode includes: Encoding the coding blocks of the multiple channels according to the IBC mode; The encoding of the BV of the reference prediction block in at least one substream obtained by encoding the coding block of the at least one channel in the IBC mode comprises: The BV of the reference prediction block is encoded in a code stream obtained by encoding the coding block of each channel in the multiple channels through the IBC mode according to a preset ratio.

2. The method according to claim 1, characterized in that: The encoding of the coding block of at least one channel among the multiple channels according to the intra block copy (IBC) mode includes: A target coding mode of the at least one channel is determined by using a rate-distortion optimization decision; the target coding mode includes the IBC mode; and a coding block of the at least one channel is encoded according to the target coding mode.

3. The method according to claim 1, characterized in that The method further comprises: Obtain the residual coefficient corresponding to the coding unit; The residual coefficients are divided into multiple groups according to a number threshold, and the number of residual coefficients in each group of the multiple groups of residual coefficients is less than or equal to the number threshold.

4. An image decoding method, characterized in that: The method comprises: Parsing a bitstream after encoding a coding unit; the coding unit includes coding blocks of multiple channels; the bitstream includes multiple substreams corresponding to the multiple channels after the coding blocks of the multiple channels are encoded; Based on a block vector BV of a reference prediction block parsed from at least one of the multiple substreams, determining a position of the reference prediction block in the multiple substreams; the reference prediction block is used to represent a prediction value of a decoded block decoded according to an intra block copy (IBC) mode; the BV of the reference prediction block is used to indicate a position of the reference prediction block in a reconstructed image block; Determining a prediction value of the decoded block decoded according to the IBC mode based on the position information of the reference prediction block; Reconstructing the decoded block decoded according to the IBC mode based on the predicted value; Among them, the coding blocks encoded according to the IBC mode include coding blocks of at least two channels; the coding blocks of the at least two channels share the BV of the reference prediction block; the BV of the reference prediction block is encoded in the multiple sub-streams; the BV of the reference prediction block is obtained when the coding blocks of multiple channels are encoded according to the IBC mode according to a preset ratio.

5. The method according to claim 4, characterized in that The method further comprises: When an IBC mode identifier is parsed from any one of the multiple substreams, it is determined that the target decoding mode corresponding to the multiple substreams is the IBC mode.

6. The method according to claim 4, characterized in that The method further comprises: The IBC mode identifiers are parsed one by one from the multiple substreams, and a target decoding mode corresponding to a substream from which the IBC mode identifier is parsed is determined to be the IBC mode.

7. The method according to claim 4, characterized in that The method further comprises: Parsing a bitstream after encoding a coding unit to determine a residual coefficient corresponding to the coding unit; the residual coefficient includes multiple groups; the number of residual coefficients in each group is less than or equal to a number threshold; The code stream is decoded based on the residual coefficients.

8. An image encoding device, characterized in that The device comprises: An acquisition module, used to acquire a coding unit; the coding unit includes coding blocks of multiple channels; A processing module, configured to encode a coding block of at least one of the multiple channels according to an intra-block copy (IBC) mode; obtain a block vector BV of a reference prediction block; the BV of the reference prediction block is used to indicate a position of the reference prediction block in an encoded image block; the reference prediction block is used to represent a prediction value of the coding block encoded according to the IBC mode; and encode the BV of the reference prediction block in at least one substream obtained by encoding the coding block of the at least one channel in the IBC mode; The processing module is further used to encode the coding blocks of the multiple channels according to the IBC mode; The processing module is further configured to encode the BV of the reference prediction block in a code stream obtained by encoding the coding block of each channel in the multiple channels through the IBC mode according to a preset ratio.

9. An image decoding device, characterized in that: The device comprises: A processing module, configured to parse a bitstream after encoding a coding unit; the coding unit comprises coding blocks of multiple channels; the bitstream comprises multiple substreams corresponding to the multiple channels after the coding blocks of the multiple channels are encoded; based on a block vector BV of a reference prediction block parsed from at least one of the multiple substreams, determining a position of the reference prediction block in the multiple substreams; the reference prediction block is used to represent a prediction value of a coding block encoded in an intra-block copy (IBC) mode; the BV of the reference prediction block is used to indicate a position of the reference prediction block in an encoded image block; based on the position information of the reference prediction block, determining a prediction value of a decoded block decoded in the IBC mode; and based on the prediction value, reconstructing the decoded block decoded in the IBC mode; Among them, the coding blocks encoded according to the IBC mode include coding blocks of at least two channels; the coding blocks of the at least two channels share the BV of the reference prediction block; the BV of the reference prediction block is encoded in the multiple sub-streams; the BV of the reference prediction block is obtained when the coding blocks of multiple channels are encoded according to the IBC mode according to a preset ratio.

10. An image encoding device, characterized in that: The device comprises: a processor and a memory; The memory stores instructions executable by the processor; When the processor is configured to execute the instructions, the image encoding device implements the method according to any one of claims 1 to 3.

11. An image decoding device, characterized in that: The device comprises: a processor and a memory; The memory stores instructions executable by the processor; When the processor is configured to execute the instructions, the image decoding device implements the method according to any one of claims 4 to 7.

12. A readable storage medium, characterized in that: The readable storage medium includes: software instructions; When the software instructions are executed in an image coding and decoding device, the image coding and decoding device implements the method according to any one of claims 1 to 7.

Images