Image decoding method, image coding method, image decoding device, image coding device and equipment

Abstract

The invention provides an image decoding method, an image coding method, an image decoding device, an image coding device and equipment, and the image decoding method comprises the steps that when permission protection is started, code streams of one or more permission levels corresponding to a target image are acquired, the target image is a patch-level image or a frame-level image, and the target image comprises at least one CU; wherein if the target image comprises a CU of a first permission level and a CU of a second permission level, the code streams of the multiple permission levels comprise a code stream of the first permission level and a code stream of the second permission level; wherein the first permission level is the lowest permission level, and the second permission level is the non-lowest permission level; and decoding CU division information and CU permission information of the target image based on the code streams of the plurality of permission levels, and decoding each CU of the target image based on the CU division information and the CU permission information. Through the technical scheme of the invention, permission protection of various information of the image can be carried out, and the data security is improved.

Description

Technical Field

[0001] The present application relates to the field of coding and decoding technology, and in particular to an image decoding method, device and equipment thereof, and also to an image encoding method, device and equipment thereof. Background Art

[0002] In order to save space, video images are transmitted after being encoded. The complete video encoding includes prediction, transformation, quantization, entropy coding, filtering and other processes. For the prediction process, the prediction process includes intra-frame prediction and inter-frame prediction. Inter-frame prediction refers to the use of the correlation in the video time domain to predict the current pixel using the pixels of the adjacent encoded image to effectively remove the redundancy in the video time domain. Intra-frame prediction refers to the use of the correlation in the video space domain to predict the current pixel using the pixels of the encoded block of the current frame image to remove the redundancy in the video space domain. For intra-frame prediction and inter-frame prediction, the decoding end can reconstruct the complete video image, and the user can view the complete video image. However, after the decoding end reconstructs the complete video image, the complete video image will present all the permission information, such as license plate information, local area information of the human body, screen information, etc., which poses a security risk and requires permission protection. Summary of the invention

[0003] In view of this, the present application provides an image decoding, image encoding method, device and equipment, which can protect various information rights of images (such as license plates, local areas of the human body, screens, text, etc.).

[0004] The present application provides an image decoding method, which is applied to a decoding end, and the method includes:

[0005] When enabling permission protection, obtaining one or more bitstreams of permission levels corresponding to a target image, the target image being a patch-level image or a frame-level image, and the target image including at least one CU; wherein, if the target image includes a CU of a first permission level and a CU of a second permission level, the bitstreams of the multiple permission levels include a bitstream of the first permission level and a bitstream of the second permission level; wherein, the first permission level is a minimum permission level, and the second permission level is a non-minimum permission level;

[0006] The CU partition information and the CU permission information of the target image are decoded based on the code stream of the one or more permission levels, and each CU of the target image is decoded based on the CU partition information and the CU permission information.

[0007] The present application provides an image encoding method, which is applied to an encoding end, and the method includes:

[0008] When enabling permission protection, obtaining one or more bitstreams of permission levels corresponding to a target image, the target image being a patch-level image or a frame-level image, and the target image including at least one CU; wherein, if the target image includes a CU of a first permission level and a CU of a second permission level, the bitstreams of the multiple permission levels include a bitstream of the first permission level and a bitstream of the second permission level; wherein, the first permission level is a minimum permission level, and the second permission level is a non-minimum permission level;

[0009] Encoding CU partition information and CU permission information of the target image in the bitstream of the one or more permission levels; the CU permission information is used to indicate the permission level corresponding to each CU in the target image;

[0010] Based on the CU partition information and the CU permission information, encode each CU of the target image in the bitstreams of the multiple permission levels;

[0011] The code streams of multiple permission levels corresponding to the target image are sent to the decoding end.

[0012] The present application provides an image decoding method, which is applied to a decoding end, and the method includes:

[0013] When enabling permission protection, obtaining a first permission level code stream and a second permission level code stream corresponding to a target image, wherein the target image is a patch-level image or a frame-level image, and the target image corresponds to at least two frames of spatial domain images, and the at least two frames of spatial domain images include a base layer image and an enhancement layer image; wherein the first permission level code stream is a minimum permission level code stream, and the second permission level code stream is a non-minimum permission level code stream;

[0014] Decoding the first permission level code stream to obtain first reconstruction information corresponding to the target image, and determining a base layer reconstructed image corresponding to the target image based on the first reconstruction information;

[0015] The second permission level code stream is decoded to obtain second reconstruction information corresponding to the target image, and an enhanced layer reconstructed image corresponding to the target image is determined based on the first reconstruction information and the second reconstruction information, wherein the resolution of the enhanced layer reconstructed image is the same as the resolution of the basic layer reconstructed image.

[0016] The present application provides an image encoding method, which is applied to an encoding end, and the method includes:

[0017] When enabling permission protection, obtaining a first permission level code stream and a second permission level code stream corresponding to a target image, wherein the target image is a patch-level image or a frame-level image, and the target image corresponds to at least two frames of spatial domain images, and the at least two frames of spatial domain images include a base layer image and an enhancement layer image; wherein the first permission level code stream is a minimum permission level code stream, and the second permission level code stream is a non-minimum permission level code stream;

[0018] Encoding first reconstruction information corresponding to the target image in the first permission level code stream, where the first reconstruction information is used to determine a base layer reconstructed image corresponding to the target image;

[0019] The second reconstruction information corresponding to the target image is encoded in the second permission level code stream, and the first reconstruction information and the second reconstruction information are used to determine the enhanced layer reconstructed image corresponding to the target image; wherein the resolution of the enhanced layer reconstructed image is the same as the resolution of the basic layer reconstructed image.

[0020] The present application provides an image decoding method, which is applied to a decoding end, and the method includes:

[0021] When permission protection is enabled, a first permission level code stream and a second permission level code stream corresponding to a target image are obtained, where the target image is a patch-level image or a frame-level image; wherein the target image corresponds to at least two time domain images, where the at least two time domain images include a k-th time domain image and a (k+i)-th time domain image, where the value of i is 1-M, where M represents the total number of second permission level code streams, where the first permission level code stream is a lowest permission level code stream, and where the second permission level code stream is a non-lowest permission level code stream;

[0022] Decoding the first permission level code stream to obtain first reconstruction information corresponding to the target image, and determining a k-th frame of time-domain reconstructed image corresponding to the target image based on the first reconstruction information;

[0023] The i-th second authority level code stream among the M second authority level code streams is decoded to obtain second reconstruction information corresponding to the target image, and the (k+i)-th frame time domain image corresponding to the target image is determined based on the first reconstruction information and the second reconstruction information.

[0024] The present application provides an image encoding method, which is applied to an encoding end, and the method includes:

[0025] When permission protection is enabled, a first permission level code stream and a second permission level code stream corresponding to a target image are obtained, where the target image is a patch-level image or a frame-level image; wherein the target image corresponds to at least two time domain images, where the at least two time domain images include a k-th time domain image and a (k+i)-th time domain image, where the value of i is 1-M, where M represents the total number of second permission level code streams, where the first permission level code stream is a lowest permission level code stream, and where the second permission level code stream is a non-lowest permission level code stream;

[0026] Encoding first reconstruction information corresponding to the target image in the first permission level code stream, where the first reconstruction information is used to determine a k-th frame of time-domain reconstructed image corresponding to the target image;

[0027] The second reconstruction information corresponding to the target image is encoded in the i-th second authority level code stream among the M second authority level code streams; wherein the first reconstruction information and the second reconstruction information are used to determine the (k+i)-th frame time domain image corresponding to the target image.

[0028] The present application provides an image decoding device, which is applied to a decoding end, and the device includes:

[0029] an acquisition module, configured to acquire, when permission protection is enabled, a code stream of one or more permission levels corresponding to a target image, wherein the target image is a patch-level image or a frame-level image, and the target image includes at least one CU; wherein, if the target image includes a CU of a first permission level and a CU of a second permission level, the code streams of the multiple permission levels include a code stream of the first permission level and a code stream of the second permission level; wherein, the first permission level is a minimum permission level, and the second permission level is a non-minimum permission level;

[0030] A decoding module is used to decode the CU partition information and CU permission information of the target image based on the code stream of the one or more permission levels, and decode each CU of the target image based on the CU partition information and the CU permission information.

[0031] The present application provides an image encoding device, which is applied to an encoding end, and the device includes:

[0032] an acquisition module, configured to acquire, when permission protection is enabled, a code stream of one or more permission levels corresponding to a target image, wherein the target image is a patch-level image or a frame-level image, and the target image includes at least one CU; wherein, if the target image includes a CU of a first permission level and a CU of a second permission level, the code streams of the multiple permission levels include a code stream of the first permission level and a code stream of the second permission level; wherein, the first permission level is a minimum permission level, and the second permission level is a non-minimum permission level;

[0033] an encoding module, configured to encode CU partition information and CU permission information of the target image in the bitstream of the one or more permission levels; the CU permission information is used to indicate the permission level corresponding to each CU in the target image; and based on the CU partition information and the CU permission information, encode each CU of the target image in the bitstream of the multiple permission levels;

[0034] The sending module is used to send the code streams of multiple permission levels corresponding to the target image to the decoding end.

[0035] The present application provides an image decoding device, which is applied to a decoding end, and the device includes:

[0036] an acquisition module, configured to acquire, when permission protection is enabled, a first permission level code stream and a second permission level code stream corresponding to a target image, wherein the target image is a patch-level image or a frame-level image, and the target image corresponds to at least two frames of spatial domain images, and the at least two frames of spatial domain images include a base layer image and an enhancement layer image; wherein the first permission level code stream is a minimum permission level code stream, and the second permission level code stream is a non-minimum permission level code stream;

[0037] A decoding module is used to decode the first permission level code stream to obtain first reconstruction information corresponding to the target image, and determine a basic layer reconstructed image corresponding to the target image based on the first reconstruction information; decode the second permission level code stream to obtain second reconstruction information corresponding to the target image, and determine an enhanced layer reconstructed image corresponding to the target image based on the first reconstruction information and the second reconstruction information, wherein the resolution of the enhanced layer reconstructed image is the same as the resolution of the basic layer reconstructed image.

[0038] The present application provides an image encoding device, which is applied to an encoding end, and the device includes:

[0039] an acquisition module, configured to acquire, when permission protection is enabled, a first permission level code stream and a second permission level code stream corresponding to a target image, wherein the target image is a patch-level image or a frame-level image, and the target image corresponds to at least two frames of spatial domain images, and the at least two frames of spatial domain images include a base layer image and an enhancement layer image; wherein the first permission level code stream is a minimum permission level code stream, and the second permission level code stream is a non-minimum permission level code stream;

[0040] A coding module, used to encode first reconstruction information corresponding to the target image in the first permission level code stream, the first reconstruction information is used to determine the base layer reconstructed image corresponding to the target image; encode second reconstruction information corresponding to the target image in the second permission level code stream, the first reconstruction information and the second reconstruction information are used to determine the enhancement layer reconstructed image corresponding to the target image; wherein the resolution of the enhancement layer reconstructed image is the same as the resolution of the base layer reconstructed image.

[0041] The present application provides an image decoding device, which is applied to a decoding end, and the device includes:

[0042] an acquisition module, configured to acquire, when permission protection is enabled, a first permission level code stream and a second permission level code stream corresponding to a target image, wherein the target image is a patch level image or a frame level image; wherein the target image corresponds to at least two time domain image frames, wherein the at least two time domain image frames include a k-th time domain image and a (k+i)-th time domain image, wherein the value of i is 1-M, and M represents the total number of second permission level code streams, wherein the first permission level code stream is a lowest permission level code stream, and the second permission level code stream is a non-lowest permission level code stream;

[0043] a decoding module, configured to decode the first permission level code stream to obtain first reconstruction information corresponding to the target image, and determine a k-th time domain reconstructed image corresponding to the target image based on the first reconstruction information; decode the i-th second permission level code stream among the M second permission level code streams to obtain second reconstruction information corresponding to the target image, and determine a (k+i)-th time domain image corresponding to the target image based on the first reconstruction information and the second reconstruction information.

[0044] The present application provides a device for use at an encoding end, the device comprising:

[0045] an acquisition module, configured to acquire, when permission protection is enabled, a first permission level code stream and a second permission level code stream corresponding to a target image, wherein the target image is a patch level image or a frame level image; wherein the target image corresponds to at least two time domain image frames, wherein the at least two time domain image frames include a k-th time domain image and a (k+i)-th time domain image, wherein the value of i is 1-M, and M represents the total number of second permission level code streams, wherein the first permission level code stream is a lowest permission level code stream, and the second permission level code stream is a non-lowest permission level code stream;

[0046] A coding module, configured to encode first reconstruction information corresponding to the target image in the first permission level code stream, the first reconstruction information being used to determine the k-th time domain reconstructed image corresponding to the target image; and to encode second reconstruction information corresponding to the target image in the i-th second permission level code stream among M second permission level code streams; wherein the first reconstruction information and the second reconstruction information are used to determine the (k+i)-th time domain image corresponding to the target image.

[0047] The present application provides a decoding end device, including: a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the above-mentioned example image decoding method.

[0048] The present application provides an encoding end device, including: a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the above-mentioned example image encoding method.

[0049] The present application provides an electronic device, comprising: a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the above-mentioned example image decoding method and image encoding method.

[0050] The present application provides a machine-readable storage medium, characterized in that a number of computer instructions are stored on the machine-readable storage medium, and when the computer instructions are executed by a processor, the above-mentioned example image decoding method and image encoding method are implemented.

[0051] It can be seen from the above technical solutions that in the embodiments of the present application, when permission protection is enabled, various types of information permission protection of images and videos (such as license plates, local areas of the human body, screens, text, etc.) can be performed to improve data security. For example, when there are information security issues with images, permission protection can be performed on the images so that users with high permission levels can view a large amount or even all of the image information, and users with low permission levels can only view a small amount of image information. For example, the image is divided into low-authority areas and high-authority areas, low-authority users can only view low-authority areas, and high-authority users can view both low-authority areas and high-authority areas. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is a schematic diagram of the video coding framework;

[0053] Figure 2 is a flowchart of an image decoding method in one embodiment of the present application;

[0054] Figure 3 is a flowchart of an image encoding method in one embodiment of the present application;

[0055] Figure 4A This is a patch image rights protection coding flow chart in one implementation of the present application;

[0056] Figure 4B This is a flowchart of patch image permission protection with a high permission level in one implementation of the present application;

[0057] Figure 4C This is a flowchart of patch image permission protection at a low permission level in one implementation of the present application;

[0058] Figure 5 is a flowchart of an image decoding method in one embodiment of the present application;

[0059] Figure 6 is a flowchart of an image encoding method in one embodiment of the present application;

[0060] Fig. 7A is a hardware structure diagram of a decoding end device in one embodiment of the present application;

[0061] Figure 7B It is a hardware structure diagram of an encoding end device in one implementation of the present application. DETAILED DESCRIPTION

[0062] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, rather than for limiting the present application. The singular forms of "a", "said" and "the" used in the embodiments of the present application and the claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to any or all possible combinations of one or more associated listed items. It should be understood that although the embodiments of the present application may use the terms first, second, etc. to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the embodiments of the present application, the first information may also be referred to as the second information, and the second information may also be referred to as the first information, depending on the context. In addition, the word "if" used can be interpreted as "at ... ", or "when ... ", or "in response to determination".

[0063] In the embodiments of the present application, a method for image decoding and image encoding is proposed, which may involve the following concepts:

[0064] Prediction Signal: Prediction signal refers to the pixel value derived from the pixel that has been encoded and decoded. The residual is obtained by the difference between the original pixel and the predicted pixel, and then the residual transform quantization and coefficient encoding are performed. The inter-frame prediction pixel refers to the pixel value derived from the reference frame (reconstructed pixel frame) of the current block. Since the pixel position is discrete, interpolation operation is required to obtain the final prediction pixel. The closer the predicted pixel is to the original pixel, the smaller the residual energy obtained by subtracting the two, and the higher the coding compression performance.

[0065] Rate-Distortion Optimized: There are two major indicators for evaluating coding efficiency: bit rate and PSNR (Peak Signal to Noise Ratio). The smaller the bit stream, the greater the compression rate, the greater the PSNR, and the better the quality of the reconstructed image. When selecting a mode, the discriminant formula is essentially a comprehensive evaluation of the two. For example, the cost corresponding to the mode: J(mode) = D + λ*R, D represents Distortion, which is usually measured using the SSE indicator. SSE refers to the mean square sum of the difference between the reconstructed image block and the source image. In order to consider the cost, the SAD indicator can also be used. SAD is the sum of the absolute values ​​of the difference between the reconstructed image block and the source image; λ is the Lagrange multiplier, and R is the actual number of bits required for encoding the image block in this mode, including the sum of bits required for encoding mode information, motion information, residuals, etc. When selecting a mode, if the rate-distortion principle is used to make a comparison decision on the encoding mode, the best encoding performance can usually be guaranteed.

[0066] Video Coding Framework: See Figure 1 As shown, it is a schematic diagram of a video encoding framework of the encoding end, and the video encoding framework can be used to implement the encoding end processing flow of the present application. The schematic diagram of the video decoding framework can be similar to Figure 1 Similarly, the video decoding framework can be used to implement the decoding end processing flow of the present application.

[0067] For example, see Figure 1 As shown, the video encoding framework may include modules such as prediction, transformation, quantization, entropy encoder, inverse quantization, inverse transformation, reconstruction, and filtering. At the encoding end, through the cooperation between these modules, the encoding process flow of the encoding end can be realized. In addition, the video decoding framework may include modules such as prediction, transformation, quantization, entropy decoder, inverse quantization, inverse transformation, reconstruction, and filtering. At the decoding end, through the cooperation between these modules, the decoding process flow of the decoding end can be realized.

[0068] A large number of coding tools are proposed for each module of the encoding end, and each tool often has multiple modes. For different video sequences, the coding tools that can achieve the best coding performance are often different. Therefore, in the encoding process, RDO (Rate-Distortion Optimize) is usually used to compare the coding performance of different tools or modes to select the best mode. After determining the optimal tool or mode, the decision information of the tool or mode is transmitted by encoding the marker information in the bitstream. Although this method brings higher coding complexity, it can adaptively select the optimal mode combination for different content to obtain the best coding performance. The decoding end obtains relevant mode information by directly parsing the flag information, and the complexity impact is small.

[0069] Bitstream: A bitstream is a bit sequence consisting of encoded audio and video and related data. A bitstream can be used to represent a NAL (Network Abstraction Layer) unit stream or a byte stream. Therefore, both NAL unit streams and byte streams are called bitstreams.

[0070] The NAL unit stream format consists of a series of syntax structures called NAL units, which are sorted in decoding order. The decoding order and content of NAL units in a NAL unit stream are constrained.

[0071] The byte stream can be constructed using a NAL unit stream, arranging the NAL units in decoding order and adding a start code prefix and several zero-valued bytes to each NAL unit to form a byte stream. The NAL unit stream format can be extracted from the byte stream format by searching for a unique start code prefix in the byte stream. In addition to the byte stream format, there can be other ways to construct NAL units.

[0072] NumBytesInNALunit is used to indicate the length of the NAL unit in bytes. The NAL unit consists of a unit header and a unit payload. The unit payload contains a RBSP (Raw Byte Sequence Payload) syntax structure and possible authentication data payload, and may also contain some emulation_prevention_three_byte (anti-emulation three bytes). NumBytesInNALunit (byte unit in NAL unit) is needed when decoding the NAL unit. In order to derive NumBytesInNALunit, the boundaries of the NAL unit need to be divided.

[0073] The NAL unit stream is generated as an ordered byte stream, and the NAL unit boundary position can be identified from the byte stream. For bit-oriented transmission, the bit order in the byte stream starts with the MSB of the first byte, processed to the LSB of the first byte, then the MSB of the second byte, and so on.

[0074] The byte stream format consists of a series of byte stream NAL unit syntax structures, each of which contains a start code prefix followed by a NAL unit.

[0075] The byte stream NAL unit syntax structure may contain an additional zero_byte syntax element and may also contain one or more additional trailing_zero_8bits syntax elements. The first byte stream NAL unit syntax structure may also contain one or more additional leading_zero_8bits syntax elements.

[0076] NAL unit: NAL unit is a syntactic structure. NAL unit contains the type indication of the subsequent data and the number of bytes contained. The data appears in the form of RBSP and includes interspersed security bytes when necessary. NAL unit stream is a sequence of NAL units.

[0077] Coded video sequence: A coded video sequence is a picture sequence consisting of an IDR picture arranged in decoding order and zero or more non-IDR pictures following it.

[0078] Emulation prevention byte: The emulation prevention byte is a byte whose value can be equal to 0x03 and may appear in the NAL unit. The presence of the emulation prevention byte can ensure that the start code prefix will not be included in the subsequent byte-aligned byte stream of the NAL unit.

[0079] Block: A block is an M*N (M columns and N rows) sample matrix in the video signal space, or a block is an M*N transform coefficient matrix in the video signal space.

[0080] Start code prefix: The start code prefix is ​​a unique sequence of 3 bytes equal to 0x000001 in the byte stream. The start code prefix is ​​used as the prefix of each NAL unit. The decoding end (such as a decoder) can use the position of the start code prefix to determine the beginning of a new NAL unit and the end of the previous NAL unit. Anti-counterfeiting bytes are added to the NAL unit to prevent the appearance of fake start code prefixes.

[0081] Secure front end device: A secure front end device is a front end device with secure cryptographic components. The secure front end device can use technologies such as certificate storage and management, key storage and management, data signature verification, and data encryption to achieve device identity authentication, video signature, video encryption and other functions.

[0082] Patch partitioning: Patch partitioning is used to indicate how to divide a frame into patches and tree coding units (CTUs). When patch_enable is equal to 0, the whole frame has only one patch. When tile_enable is equal to 1, the whole frame may be divided into multiple patches. A patch can be composed of a series of tree coding units (CTUs). The tree coding unit is the basic unit of encoding. Each tree coding unit can contain a luminance array and two chrominance arrays. In other words, a patch can be understood as an image block containing an integer number of CTUs or LCUs.

[0083] The index of the CTU in the upper left corner of the image is equal to 0, and the index of the CTU in the image increases in raster scanning order. An image is divided into several rectangular areas in the horizontal and vertical directions. Each rectangular area is called a patch. Each patch contains several CTUs and can be encoded and decoded in parallel and independently.

[0084] When the image width is greater than or equal to 8 maximum tree coding unit sizes, it can be divided into multiple patch columns in the horizontal direction. When the image width exceeds 64 maximum tree coding unit sizes, it needs to be divided into multiple patch columns. Each patch contains at least 4 CTUs in the horizontal direction and a maximum of 64 CTUs. The image can be divided into multiple patch rows in the vertical direction, and the number of patch rows can be equal to 1, 2, or 4.

[0085] Region Of Interest partitioning: An image can be divided into several regions of interest (ROIs). The minimum unit of ROI is 8*8, and the samples in the same prediction unit belong to the same ROI area. If there is ROI partitioning in the image, the value of the parameter segmentation_enable in the corresponding image parameter set is equal to 1. If there is no ROI partitioning in the image, the value of the parameter segmentation_enable in the corresponding image parameter set is equal to 0. Among them, the ROI area in an image can be divided into 8 different levels, which are indicated by the segment_id of the block where the sample point is located. Among them, if there is no ROI partitioning in the image, the segment_id of the block where all the sample points are located should be equal to 0.

[0086] Raw byte sequence payload (RBSP): A raw byte sequence payload is a syntax structure containing an integer number of bytes encapsulated in a NAL unit. An RBSP can be empty. An RBSP can contain syntax elements in the form of a data bit string, followed by an RBSP stop bit and zero or more consecutive 0-valued bits. The raw byte sequence payload stop bit is a bit with a value of 1 that appears after the data bit string in the RBSP. The end position of the data bit string in the RBSP can be obtained by searching for the RBSP stop bit in the RBSP.

[0087] Encapsulation and constraints of NAL units: Use emulation_prevention_three_byte to encapsulate an RBSP into a NAL unit. During the encapsulation process, any SODB is allowed to appear in the NAL unit. However, it is necessary to prevent the appearance of a pseudo start code in the NAL unit. By looking for the RBSP bit rbsp_stop_one_bit at the end of the RBSP, the end of the SODB in the NAL unit can be identified.

[0088] The encoding end (such as the encoder) generates a NAL unit from an RBSP through the following steps: searching for the following byte-aligned binary bit string '00000000000000000000000xx' from the RBSP, where xx represents any 2-bit string '00', '01', '10' or '11', and inserting a byte equal to 0x03 therein to form '000000000000000000000011000000xx'; the resulting byte sequence plus the unit header part of the NAL unit that identifies the RBSP data structure type forms the entire NAL unit.

[0089] Based on the above process of generating NAL units, the process allows any SODB to appear in a NAL unit, and at the same time ensures that: there is no byte-aligned pseudo start code in the NAL unit; regardless of whether it is byte-aligned, there is no sequence of 8 bits with a value of 0 followed by a start code in the NAL unit.

[0090] The order of NAL units and the relationship between NAL units and coded images and video sequences, the order and effectiveness of sequence parameter set RBSP and image parameter set RBSP: The parameters included in a sequence parameter set RBSP can be used by one or more images or SEI NAL units containing cache period SEI messages. Each sequence parameter set RBSP takes effect at the same time it is received by the decoder, and will cause the previously valid sequence parameter set RBSP (if any) to become invalid. At most one sequence parameter set RBSP is valid at a specified time during the decoding process. When a sequence parameter set RBSP is used by a SEI NAL unit containing a cache period SEI message, the SEI NAL unit can be located after the sequence parameter set RBSP.

[0091] The parameters included in the picture parameter set RBSP can be used by the coded slice NAL unit of a coded picture. Each picture parameter set RBSP takes effect when it is received by the decoder and will cause the previously valid picture parameter set RBSP (if any) to become invalid. For a certain layer of SVC pictures, at most one picture parameter set RBSP is valid at a specified time during the decoding process.

[0092] The relationship between the syntax element values ​​in the sequence parameter set and the picture parameter set and other syntax elements is specified only for the valid sequence parameter set and the valid picture parameter set. During the decoding process, the parameter values ​​of the valid picture parameter set and the valid sequence parameter set may remain valid.

[0093] Region of Interest Syntax Elements: An image can be divided into several regions of interest (ROIs), the minimum unit of which is 8*8, and samples in the same prediction unit can belong to the same ROI. The syntax elements of the region of interest can be shown in Table 1.

[0094] Table 1

[0095]

[0096] Decoding of the region of interest: If the current sequence header Roi_flag is turned on, a set of probability parameters is parsed in the image header, and the segments in the current frame that need to be forcibly skipped for parsing are obtained. When parsing the segment_id of each block, if the current segment_id is a segment that needs to be forcibly skipped for parsing, the parsing of the current block is skipped.

[0097] During the inter-frame prediction process, for blocks that process the region of interest and select the skip mode (each block is a CU, also called a CU block), the default corresponding reference frame is DYNAMIC_REF (the 0th reference frame), the MV is (0,0), and the corresponding reference block is obtained in the base layer reference frame.

[0098] In the DB process, the cu_filter_level corresponding to the current block can be calculated based on the segment_id, reference frame type ref_frame, and prediction mode mode corresponding to the current block as indexes. Different filtering parameters can be used according to different cu_filter_level values ​​corresponding to the current block.

[0099] In a possible implementation, the decoding end can reconstruct the complete video image, and the user can view the complete video image. However, after the complete video image is reconstructed, the complete video image may present all permissions, such as license plate information, human body local area information, text information, screen information, etc., which poses a security risk and requires permission protection. To this end, permission protection based on NAL units can be used.

[0100] For NAL unit-based rights protection, the input of the NAL unit decoding process is the NAL unit, and the output is the RBSP syntax structure encapsulated in the NAL unit. The RBSP syntax structure can be extracted from the NAL unit. If encryption_idc is equal to 1, the RBSP syntax structure needs to be decrypted when extracting the RBSP syntax structure from the NAL unit to obtain the unencrypted RBSP.

[0101] The RBSP syntax structure in the NAL unit is decoded as follows: When the value of the syntax nal_unit_type in the NAL unit is 1 / 2 / 3 / 4, the code stream in the current NAL unit is parsed, and the corresponding tiles are reconstructed using the parsed value. When the value of the syntax nal_unit_type in the NAL unit is 1 / 2 / 3 / 4, for RBSPs that cannot be decrypted, this information is discarded, and these tiles cannot be reconstructed.

[0102] For NAL unit-based rights protection, when encoding, the image is divided into tiles, and the tiles cannot refer to each other. The bitstream information inside each tile is written together and packaged into an RBSP and placed in the NAL unit. Each RBSP bitstream can be processed differently, such as encrypting certain RBSPs. In fact, the size of a tile contains an integer number of CTUs, so only large areas of image content can be processed specially. For very small targets, it is not possible to flexibly process the bitstream of small area blocks.

[0103] In view of the above findings, in this embodiment, for a Tile code stream, the Tile code stream is divided into n RBSPs, and all different contents inside the Tile are written into different RBSPs. Then, different RBSPs are packaged into different NAL units for transmission. The n RBSP code streams obtained from a Tile content contain code streams of n permissions, and different processing can be performed on code streams of different permissions, such as different encryption permission processing.

[0104] In this embodiment, an image decoding method and an image encoding method are proposed, and a code stream processing method based on authority protection is given. In the encoding and decoding process, different authorities are assigned to different image areas, and the code streams based on different authorities are stored separately. When decoding a low-authority area, the high-authority code stream cannot be accessed. On this basis, when there are information security issues with the image, the image can be protected by authority, and the authority areas of the image at all levels can be expressed efficiently and losslessly, so that users with high authority levels can view a large amount or even all of the image information, and users with low authority levels can only view a small amount of image information. For example, an image can be divided into a low-authority area and a high-authority area, and a low-authority user can only view the low-authority area, while a high-authority user can view both the low-authority area and the high-authority area, thereby improving data security.

[0105] The image decoding method and the image encoding method are described in detail below in conjunction with several specific embodiments.

[0106] Embodiment 1: In the embodiment of the present application, an image decoding method is proposed, see Figure 2 FIG. 1 is a flow chart of the method, which is applied to a decoding end (also called a video decoder), and includes:

[0107] Step 201: When enabling permission protection, obtain a bitstream of one or more permission levels corresponding to a target image, where the target image is a patch-level image or a frame-level image, and the target image includes at least one CU. If the target image includes a CU of a first permission level and a CU of a second permission level, the bitstreams of multiple permission levels include a bitstream of the first permission level and a bitstream of the second permission level; the first permission level is the lowest permission level, and the second permission level is a non-lowest permission level. The second permission level may be at least one permission level, that is, the non-lowest permission level may be at least one non-lowest permission level.

[0108] Exemplarily, if all CUs of the target image correspond to the same permission level, a bitstream of a permission level corresponding to the target image can be obtained. In this embodiment, taking the example that all CUs of the target image correspond to multiple permission levels, the bitstreams of multiple permission levels corresponding to the target image can be obtained.

[0109] Step 202: Decode the CU partition information and CU permission information of the target image based on the bitstream of one or more permission levels. For example, decode the CU partition information and CU permission information of the target image based on the bitstream of the first permission level and / or the bitstream of the second permission level.

[0110] Step 203: decode each CU of the target image based on the CU partition information and the CU permission information, that is, decode each CU of the target image one by one based on the CU partition information and the CU permission information.

[0111] In a possible implementation, decoding each CU of the target image based on the CU partition information and the CU permission information may include but is not limited to: for the current CU to be decoded in the target image, then: determining the current permission level corresponding to the current CU based on the CU permission information. If it is determined based on the current permission level that the target permission level has access rights to the current CU, high-dimensional reconstruction information corresponding to the current CU is decoded from the target bitstream based on the CU partition information, and a reconstructed image block corresponding to the current CU is determined based on the high-dimensional reconstruction information. Among them, the target permission level may be the permission level of the user accessing the decoding end, and the target bitstream may be a bitstream with a permission level less than or equal to the target permission level.

[0112] In a possible implementation, if it is determined based on the current permission level that the target permission level does not have access rights to the current CU, then decoding the reconstruction information corresponding to the current CU from the target bitstream is skipped; low-dimensional reconstruction information corresponding to the current CU is obtained, and the reconstructed image block corresponding to the current CU is determined based on the low-dimensional reconstruction information; wherein the low-dimensional reconstruction information is a fixed reconstruction value, or the low-dimensional reconstruction information is a reconstruction value obtained based on intra-frame prediction, or the low-dimensional reconstruction information is a reconstruction value obtained based on inter-frame prediction.

[0113] Exemplarily, the above execution order is only for the convenience of describing the examples given. In practical applications, the execution order between the steps can also be changed, and there is no limitation on this execution order. Moreover, in other embodiments, the steps of the corresponding method are not necessarily executed in the order shown and described in this specification, and the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; multiple steps described in this specification may also be combined into a single step for description in other embodiments.

[0114] In the embodiment of the present application, an image encoding method is proposed, see Figure 3 FIG. 1 is a flow chart of the method, which is applied to an encoding end (also called a video encoder), and includes:

[0115] Step 301: When enabling permission protection, obtain one or more permission level code streams corresponding to a target image, the target image is a patch level image or a frame level image, and the target image includes at least one CU. If the target image includes a CU of a first permission level and a CU of a second permission level, the code streams of multiple permission levels may include a code stream of the first permission level and a code stream of the second permission level; the first permission level may be the lowest permission level, and the second permission level may be a non-lowest permission level.

[0116] Step 302: Encode CU partition information and CU permission information of the target image in a bitstream of one or more permission levels; wherein the CU permission information is used to indicate the permission level corresponding to each CU in the target image.

[0117] Step 303: Encode each CU of the target image in code streams of multiple permission levels based on the CU partition information and the CU permission information.

[0118] For example, for each CU in the target image, when encoding the CU, a candidate bitstream corresponding to the permission level corresponding to the CU is determined (that is, the permission level of the candidate bitstream is the same as the permission level corresponding to the CU), and high-dimensional reconstruction information corresponding to the CU is encoded in the candidate bitstream based on the CU partitioning information, and encoding the high-dimensional reconstruction information corresponding to the CU in bitstreams of other permission levels is skipped.

[0119] Step 304: Send the code streams of multiple permission levels corresponding to the target image to the decoding end.

[0120] Exemplarily, the above execution order is only for the convenience of describing the examples given. In practical applications, the execution order between the steps can also be changed, and there is no limitation on this execution order. Moreover, in other embodiments, the steps of the corresponding method are not necessarily executed in the order shown and described in this specification, and the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; multiple steps described in this specification may also be combined into a single step for description in other embodiments.

[0121] It can be seen from the above technical solutions that in the embodiments of the present application, when permission protection is enabled, various types of information permission protection of images and videos (such as license plates, local areas of the human body, screens, text, etc.) can be performed to improve data security. For example, when there are information security issues with images, permission protection can be performed on the images so that users with high permission levels can view a large amount or even all of the image information, and users with low permission levels can only view a small amount of image information. For example, the image is divided into low-authority areas and high-authority areas, low-authority users can only view low-authority areas, and high-authority users can view both low-authority areas and high-authority areas.

[0122] Embodiment 2: To implement code stream processing based on rights protection, the following method may be used:

[0123] First, when enabling permission protection for a target image, the decoder needs to obtain code streams of multiple permission levels corresponding to the target image. For example, if there are K permission levels in total, K is a positive integer greater than 1, then obtain code streams of K permission levels, such as code streams of permission level 1, code streams of permission level 2, ..., code streams of permission level K. Assuming that permission level 1 is the lowest permission level (i.e., the code stream of the first permission level), permission level K is the highest permission level, and the code streams of permission level 2 to permission level K are all code streams of the second permission level. Corresponding to the processing process of the decoder, the encoder needs to obtain code streams of multiple permission levels corresponding to the target image, and send the code streams of multiple permission levels corresponding to the target image to the decoder. For example, if the target image includes a CU of the first permission level and a CU of the second permission level, the code streams of multiple permission levels include code streams of the first permission level and code streams of the second permission level; wherein the first permission level is the lowest permission level, and the second permission level is a non-lowest permission level. For example, permission level 1 is the first permission level, and permission levels 2 to K are the second permission levels.

[0124] Among them, the target image can be a patch-level image, and the target image can also be a frame-level image, and there is no restriction on the type of the target image. For example, if the target image is a patch-level image, the code stream format can be ABCABC, and the first group ABC represents the code streams of multiple permission levels corresponding to the first patch-level image, and the second group ABC represents the code streams of multiple permission levels corresponding to the second patch-level image. For another example, if the target image is a frame-level image, the code stream format can be AAABBBCCC, and the first group AAA represents the code stream of the first permission level corresponding to all frame-level images, the second group BBB represents the code stream of the second permission level corresponding to all frame-level images, and the third group CCC represents the code stream of the third permission level corresponding to all frame-level images. Of course, the above are just examples of code stream formats. For example, the code stream format of patch-level images can also be AAABBBCCC, and the code stream format of frame-level images can also be ABCABC, and there is no restriction on this.

[0125] Among them, for the target image, the target image includes at least one CU, CU can also be called an image block, abbreviated as a block, that is, a CU block, and the current CU can also be called a current block (current image block).

[0126] Then, the decoding end needs to decode the CU division information and CU permission information of the target image based on the bitstreams of multiple permission levels. Corresponding to the processing process of the decoding end, the encoding end needs to encode the CU division information and CU permission information of the target image in the bitstreams of multiple permission levels. For example, the encoding end encodes the CU division information and CU permission information of the target image in the bitstream of the lowest permission level (i.e., the first permission level), and the decoding end decodes the CU division information and CU permission information of the target image from the bitstream of the lowest permission level (i.e., the first permission level). Of course, the above is only an example of decoding the CU division information and CU permission information. For the decoding method of this CU division information and CU permission information, please refer to the subsequent embodiments.

[0127] Among them, the CU division information represents the position information of each CU in the target image. For each CU, the position information of the CU represents the starting position and length of the CU in the bitstream. Based on the position information of the CU, the reconstruction information of the CU can be parsed from the bitstream without any restriction.

[0128] The CU permission information indicates permission information of each CU in the target image. For each CU, the permission information of the CU indicates the permission level of the CU. For example, if there are K permission levels in total, the permission level of the CU may be permission level 1, permission level 2, ..., permission level K. The CU of permission level 1 is a CU of the first permission level, that is, a CU of the lowest permission level, and the CUs of permission level 2, ..., permission level K are CUs of the second permission level, that is, CUs of non-lowest permission levels.

[0129] Regarding the permission level of each CU, the encoder obtains the permission level of each CU without limiting the acquisition method, and encodes the permission level of each CU in the target image, that is, CU permission information, in the bitstream.

[0130] In addition to the CU division information and CU authority information, the decoding end can also decode the filtering information of each CU from the bitstream. The filtering information is used to filter the CU, and there is no restriction on this filtering information.

[0131] Then, the decoding end performs decoding based on the CU granularity. For the current CU (i.e., the current CU to be decoded), the decoding end determines the current permission level corresponding to the current CU based on the CU permission information. For example, since the CU permission information indicates the permission level of each CU in the target image, the decoding end can determine the current permission level corresponding to the current CU based on the CU permission information, such as permission level 1, permission level 2, etc.

[0132] Then, based on the current permission level corresponding to the current CU, the decoding end determines whether the target permission level has the access rights of the current CU based on the current permission level. For example, if the target permission level is greater than or equal to the current permission level, it is determined that the target permission level has the access rights of the current CU. If the target permission level is less than the current permission level, the target permission level does not have the access rights of the current CU.

[0133] The target permission level is the permission level of the user accessing the decoding end. When permission protection is enabled, the decoding end can determine the target permission level. For example, if user A is currently viewing the image reconstructed by the decoding end, the permission level of user A is used as the target permission level. If user B is currently viewing the image reconstructed by the decoding end, the permission level of user B is used as the target permission level, and so on.

[0134] Then, if the target permission level has access rights to the current CU, the decoding end decodes the high-dimensional reconstruction information corresponding to the current CU from the target bitstream based on the CU partition information, and determines the reconstructed image block corresponding to the current CU based on the high-dimensional reconstruction information. The target bitstream is a bitstream whose permission level is less than or equal to the target permission level.

[0135] Among them, the decoding end can obtain the code stream of each permission level, the code stream of the lowest permission level can be an unencrypted stream (the code stream of the lowest permission level can also be an encrypted stream, taking the unencrypted stream as an example), and the code stream of a non-lowest permission level can be an encrypted stream. For example, the encoding end encrypts the code stream of a non-lowest permission level, but the encoding end does not encrypt the code stream of the lowest permission level.

[0136] When the permission level of the code stream is less than or equal to the target permission level, when the user of the target permission level accesses the decoding end, the decoding end can parse the code stream. For example, the code stream of the lowest permission level is an unencrypted code stream, and the decoding end can directly parse the code stream of the lowest permission level. For another example, although the code stream of a non-lowest permission level is an encrypted code stream, when the permission level of the code stream is less than or equal to the target permission level, the user of the target permission level has the decryption information, and the decoding end can successfully decrypt the code stream based on the decryption information, and then can parse the decrypted code stream.

[0137] When the permission level of the code stream is greater than the target permission level, the code stream is an encrypted code stream. Although the user of the target permission level has the decryption information, the decoding end cannot successfully decrypt the code stream based on the decryption information, resulting in the inability to parse the code stream, which can be understood as the inability to obtain the code stream.

[0138] In summary, the code stream with a permission level less than or equal to the target permission level can be called the target code stream, and the decoding end can parse the target code stream, and the target code stream is a code stream of at least one permission level. For example, when permission level 1 is the lowest permission level, if the target permission level is permission level 1, the target code stream is the code stream of permission level 1, if the target permission level is permission level 2, the target code stream is the code stream of permission level 1 and the code stream of permission level 2, if the target permission level is permission level 3, the target code stream is the code stream of permission level 1, the code stream of permission level 2 and the code stream of permission level 3, and so on.

[0139] Among them, when the decoding end decodes the high-dimensional reconstruction information corresponding to the current CU from the target bitstream, it can select a candidate bitstream corresponding to the permission level corresponding to the current CU from the target bitstream, decode the high-dimensional reconstruction information corresponding to the current CU from the candidate bitstream based on the CU partition information, and skip decoding the high-dimensional reconstruction information corresponding to the current CU from target bitstreams of other permission levels. Corresponding to the processing process of the decoding end, for each CU in the target image, the encoding end determines the candidate bitstream corresponding to the permission level corresponding to the CU, encodes the high-dimensional reconstruction information corresponding to the CU in the candidate bitstream based on the CU partition information, and skips encoding the high-dimensional reconstruction information corresponding to the CU in bitstreams of other permission levels, that is, does not encode the high-dimensional reconstruction information.

[0140] For example, if the permission level corresponding to the current CU is permission level 1, the encoder only encodes the high-dimensional reconstruction information corresponding to the current CU in the bitstream of permission level 1 (i.e., the candidate bitstream), and does not encode the high-dimensional reconstruction information corresponding to the current CU in the bitstreams of other permission levels (such as permission level 2, permission level 3, etc.). Based on this, assuming that the target permission level is permission level 2, the target bitstream is the bitstream of permission level 1 and the bitstream of permission level 2. The decoder decodes the high-dimensional reconstruction information corresponding to the current CU from the bitstream of permission level 1, and does not decode the high-dimensional reconstruction information corresponding to the current CU from the bitstream of permission level 2.

[0141] For another example, if the permission level corresponding to the current CU is permission level 2, the encoder only encodes the high-dimensional reconstruction information corresponding to the current CU in the bitstream of permission level 2 (i.e., the candidate bitstream), and does not encode the high-dimensional reconstruction information corresponding to the current CU in the bitstreams of other permission levels (such as permission level 1, permission level 3, etc.). Based on this, assuming that the target permission level is permission level 2, the target bitstream is a bitstream of permission level 2, and the decoder decodes the high-dimensional reconstruction information corresponding to the current CU from the bitstream of permission level 2.

[0142] Among them, when the decoding end decodes the high-dimensional reconstruction information corresponding to the current CU, since the CU partition information includes the position information of the current CU, the position information indicates the starting position and length of the current CU in the bitstream. Therefore, the high-dimensional reconstruction information corresponding to the current CU can be parsed from the bitstream based on the CU partition information.

[0143] Among them, after obtaining the high-dimensional reconstruction information corresponding to the current CU, the reconstructed image block corresponding to the current CU can be determined based on the high-dimensional reconstruction information. For example, the high-dimensional reconstruction information can also be called open reconstruction information. The high-dimensional reconstruction information includes the reconstruction values ​​of all pixels of the current CU, and the image content can be completely restored through the high-dimensional reconstruction information. For example, for the current CU of the local area of ​​the human body, license plate, screen, text and other areas, if the current CU is reconstructed based on the high-dimensional reconstruction information, the real picture of the local area of ​​the human body, license plate, screen, text and other areas can be viewed, thereby obtaining the reconstructed image block of the real picture.

[0144] Then, if the target permission level does not have access rights to the current CU, the decoding end skips decoding the reconstruction information corresponding to the current CU from the target bitstream. Also, the decoding end can obtain the low-dimensional reconstruction information corresponding to the current CU, and determine the reconstructed image block corresponding to the current CU based on the low-dimensional reconstruction information. The low-dimensional reconstruction information can be a fixed reconstruction value, or the low-dimensional reconstruction information can be a reconstruction value obtained based on intra-frame prediction, or the low-dimensional reconstruction information can be a reconstruction value obtained based on inter-frame prediction.

[0145] Among them, if the target permission level does not have access rights to the current CU, although the decoding end can also parse the target bitstream, the target bitstream will not have high-dimensional reconstruction information corresponding to the current CU. Therefore, the decoding end needs to skip decoding the reconstruction information corresponding to the current CU from the target bitstream.

[0146] For example, if the permission level corresponding to the current CU is permission level 2, the encoder only encodes the high-dimensional reconstruction information corresponding to the current CU in the bitstream of permission level 2, and does not encode the high-dimensional reconstruction information corresponding to the current CU in the bitstreams of other permission levels (such as permission level 1, permission level 3, etc.). Based on this, assuming that the target permission level is permission level 1 (permission level 1 does not have access rights to the current CU), the target bitstream is a bitstream of permission level 1, and the high-dimensional reconstruction information corresponding to the current CU does not exist in the bitstream of permission level 1. Therefore, the decoder skips decoding the reconstruction information corresponding to the current CU from the target bitstream.

[0147] For another example, if the permission level corresponding to the current CU is permission level 3, the encoder only encodes the high-dimensional reconstruction information corresponding to the current CU in the bitstream of permission level 3. Assuming that the target permission level is permission level 2, the target bitstream is the bitstream of permission level 1 and the bitstream of permission level 2. The high-dimensional reconstruction information corresponding to the current CU does not exist in the target bitstream, and decoding of the reconstruction information corresponding to the current CU from the target bitstream is skipped.

[0148] Among them, if the target permission level does not have the access rights of the current CU, the decoding end can obtain the low-dimensional reconstruction information corresponding to the current CU. After obtaining the low-dimensional reconstruction information, the reconstructed image block corresponding to the current CU can be determined based on the low-dimensional reconstruction information. Low-dimensional reconstruction information can also be called hidden reconstruction information. The low-dimensional reconstruction information does not include the reconstruction values ​​of all pixels. It can only include the reconstruction values ​​of some pixels (the decoding end only parses the reconstruction values ​​of these pixels, and the remaining pixels can use the default values). It can also include only one pixel value (that is, all pixels are this pixel value, such as 255, 0, etc.), and it can also include several pixel values ​​(for example, the average value of some pixels, such as 137, 245, etc.), that is, the reconstruction value of the pixel is hidden in the low-dimensional reconstruction information, and the image content cannot be completely restored through the low-dimensional reconstruction information. For example, for the current CU of the local area of ​​the human body, license plate, screen, text and other areas, if the current CU is reconstructed based on the low-dimensional reconstruction information, then the real picture of the local area of ​​the human body, license plate, screen, text and other areas cannot be viewed.

[0149] Among them, the low-dimensional reconstruction information can be a fixed reconstruction value. For example, the fixed reconstruction value is used as the reconstruction value of each pixel in the current CU. Alternatively, the low-dimensional reconstruction information can be a reconstruction value obtained based on intra-frame prediction. For example, the intra-frame prediction mode can be used to obtain the reconstruction value of each pixel in the current CU, and there is no restriction on this intra-frame prediction process. Alternatively, the low-dimensional reconstruction information can be a reconstruction value obtained based on inter-frame prediction. For example, the inter-frame prediction mode can be used to obtain the reconstruction value of each pixel in the current CU, and there is no restriction on this inter-frame prediction process. Of course, the above are just a few examples and there is no restriction on this.

[0150] In a possible implementation, for the encoding and decoding process of rights protection, a multi-authority area encoding and decoding scheme is designed to ensure that users with low authority levels at the decoding end cannot correctly decode code streams with higher authority levels, and users with high authority levels can correctly decode code streams with high authority levels and code streams with authority levels less than or equal to their own. In order to achieve the above functions, the processing process of the encoding end may include:

[0151] Each area is marked as a CU, and the permission level of all CUs is recorded. Based on the patch code stream packaging transmission, the area information of different permission levels in the patch is packaged in different code streams. For the encoding within the patch, the reference of the low permission level area to the high permission level area is restricted. At the same time, different permission levels are encoded using different entropy encoders to obtain code streams of different permission levels.

[0152] Take two permission levels as an example. These two permission levels are high permission level and low permission level. Under high permission level and low permission level, the patch image permission protection encoding process can be found in Figure 4A shown.

[0153] The patch image is divided into blocks to obtain multiple CU blocks. Based on the high-authority area in the patch, the permission level corresponding to each CU block is determined. The permission level corresponding to the CU block can be a high-authority level and a low-authority level. For example, if the CU block is in the high-authority area in the patch, the permission level corresponding to the CU block can be a high-authority level (which can be referred to as high-authority). If the CU block is not in the high-authority area in the patch, the permission level corresponding to the CU block can be a low-authority level (which can be referred to as low-authority).

[0154] For each CU block, if the permission level corresponding to the CU block is high permission, the CU block is predicted, transformed, quantized, and entropy encoded (entropy encoding is performed using entropy encoder 1) to obtain a high permission code stream. For each CU block, if the permission level corresponding to the CU block is low permission, the CU block is restricted predicted, transformed, quantized, and entropy encoded (entropy encoding is performed using entropy encoder 2) to obtain a low permission code stream.

[0155] In a possible implementation, for the decoding end, different users have different permission levels. The decoding end can only parse the bitstream within the permission level range, and cannot parse the bitstream beyond the permission level range. The decoding end obtains bitstreams of different permission levels. If the permission level of the bitstream is not parseable, the bitstream parsing is skipped. Otherwise, the bitstream is parsed. In order to achieve the above functions, the processing process of the decoding end may include:

[0156] Parse the sequence header parameters and image header parameters from the bitstream of the lowest permission level. The sequence header parameters contain a flag indicating whether the current sequence has permission protection enabled, and the image header parameters contain a flag indicating whether the current image has permission protection enabled. Then, parse the bitstream patch by patch. If the sequence header or image header indicates that permission protection is not enabled, the NAL unit packaged by each patch bitstream contains the parameters of the entire patch. Otherwise, the NAL unit packaged by each patch bitstream only contains the parameters of a certain permission level of the entire patch image. The reconstructed image is obtained based on the parameters parsed for each NAL unit. Take two permission levels as an example for explanation. These two permission levels are high permission level and low permission level. Under high permission level and low permission level, the patch image permission protection decoding process can be referred to. Figure 4B and Figure 4C shown.

[0157] See also Figure 4B As shown in the figure, the patch image permission protection process for users with a high permission level is performed. The low permission code stream is entropy decoded (entropy decoding is performed using entropy decoder 2) to obtain CU division and permission (i.e., CU division information and CU permission information). For the current CU, if it is determined based on the CU permission information that the current CU is of a low permission level (referred to as low permission), the low permission code stream is entropy decoded, restricted prediction, inverse transformation and inverse quantization are performed to obtain reconstruction information, and a patch image is generated based on the reconstruction information.

[0158] For the current CU, if it is determined based on the CU permission information that the current CU is at a high permission level (referred to as high permission), the high permission code stream is entropy decoded (such as entropy decoding using entropy decoder 1), predicted, inversely transformed, and inversely quantized to obtain reconstruction information, and a patch image is generated based on the reconstruction information.

[0159] See also Figure 4CAs shown in the figure, the patch image permission protection process for users with low permission levels is performed. The low permission code stream is entropy decoded (entropy decoding is performed using entropy decoder 2) to obtain CU division and permissions (i.e., CU division information and CU permission information). For the current CU, if it is determined based on the CU permission information that the current CU is of a low permission level (referred to as low permission), the low permission code stream is entropy decoded, restricted prediction, inverse transformation and inverse quantization are performed to obtain reconstruction information, and a patch image is generated based on the reconstruction information.

[0160] For the current CU, if it is determined based on the CU permission information that the current CU is at a high permission level (referred to as high permission), the reconstruction information corresponding to the current CU is obtained, and the reconstruction information is a fixed reconstruction value, or a reconstruction value obtained based on intra-frame prediction, or a reconstruction value obtained based on inter-frame prediction. Figure 4C In the example, a fixed reconstruction value is taken as an example, that is, the reconstruction information is obtained by filling with a fixed value, and a patch image is generated based on the reconstruction information.

[0161] Embodiment 3: In Embodiment 1 and Embodiment 2, it involves enabling permission protection for a target image. As to whether permission protection (also referred to as information protection) is enabled, a mark bit may be transmitted at the sequence level, sequence-level extended data, image level, image-level extended data, Slice level, Tile level, Patch level, etc., and the mark bit is used to indicate whether permission protection is enabled. If the mark bit is a first value, permission protection is allowed to be enabled; if the mark bit is a second value, permission protection is not allowed to be enabled. Among them, the syntax for managing or controlling a number of images may be referred to as a sequence level or sequence-level extended data; the syntax for managing or controlling a single-frame image may be referred to as an image level or image-level extended data; the syntax for managing or controlling a parallel unit (i.e., an image area) including a number of coding units may be referred to as a Slice level, a Tile level, or a Patch level.

[0162] Exemplarily, if the switch control information allows the permission protection to be enabled, it is determined that the permission protection is enabled. If the switch control information does not allow the permission protection to be enabled, it is determined that the permission protection is prohibited to be enabled. For example, the switch control information may include but is not limited to: switch control information at the sequence level or sequence level extended data; or switch control information at the image level or image level extended data; or switch control information at the slice level; or switch control information at the tile level; or switch control information at the patch level.

[0163] For example, if the switch control information at the sequence level allows the permission protection to be enabled, it is determined that the permission protection is enabled. If the switch control information at the sequence level does not allow the permission protection to be enabled, it is determined that the permission protection is prohibited from being enabled. The switch control information at the sequence level may include a sequence level flag bit, and if the sequence level flag bit is a first value, the permission protection is allowed to be enabled, and if the sequence level flag bit is a second value, the permission protection is not allowed to be enabled.

[0164] For another example, if the switch control information of the sequence-level extended data allows the activation of permission protection, it is determined that permission protection is activated. If the switch control information of the sequence-level extended data does not allow the activation of permission protection, it is determined that the activation of permission protection is prohibited. The switch control information of the sequence-level extended data may include a sequence-level extended data flag bit, and if the sequence-level extended data flag bit is a first value, the activation of permission protection is allowed, and if the sequence-level extended data flag bit is a second value, the activation of permission protection is not allowed.

[0165] For another example, if the image-level switch control information allows the permission protection to be enabled, it is determined that the permission protection is enabled. If the image-level switch control information does not allow the permission protection to be enabled, it is determined that the permission protection is prohibited from being enabled. The image-level switch control information may include an image-level flag bit. If the image-level flag bit is a first value, the permission protection is allowed to be enabled. If the image-level flag bit is a second value, the permission protection is not allowed to be enabled.

[0166] For another example, if the switch control information of the image-level extended data allows the permission protection to be enabled, it is determined that the permission protection is enabled. If the switch control information of the image-level extended data does not allow the permission protection to be enabled, it is determined that the permission protection is prohibited from being enabled. The switch control information of the image-level extended data may include an image-level extended data flag bit. If the image-level extended data flag bit is a first value, the permission protection is allowed to be enabled. If the image-level extended data flag bit is a second value, the permission protection is not allowed to be enabled.

[0167] For another example, if the switch control information at the Slice level allows permission protection to be enabled, it is determined that permission protection is enabled. If the switch control information at the Slice level does not allow permission protection to be enabled, it is determined that permission protection is prohibited from being enabled. The switch control information at the Slice level may include a Slice level flag bit. If the Slice level flag bit is a first value, permission protection is allowed to be enabled. If the Slice level flag bit is a second value, permission protection is not allowed to be enabled.

[0168] For another example, if the Tile-level switch control information allows permission protection to be enabled, it is determined that permission protection is enabled. If the Tile-level switch control information does not allow permission protection to be enabled, it is determined that permission protection is prohibited from being enabled. The Tile-level switch control information may include a Tile-level flag bit. If the Tile-level flag bit is a first value, permission protection is allowed to be enabled. If the Tile-level flag bit is a second value, permission protection is not allowed to be enabled.

[0169] For another example, if the switch control information at the Patch level allows the permission protection to be enabled, it is determined that the permission protection is enabled. If the switch control information at the Patch level does not allow the permission protection to be enabled, it is determined that the permission protection is prohibited from being enabled. The switch control information at the Patch level may include a Patch level flag bit, and if the Patch level flag bit is a first value, the permission protection is allowed to be enabled, and if the Patch level flag bit is a second value, the permission protection is not allowed to be enabled.

[0170] Exemplarily, the switch control information of the sequence level / sequence level extended data allows the current sequence to enable permission protection, that is, permission protection is enabled for all patch-level images or frame-level images (i.e., Slice-level images) in the current sequence. The switch control information of the image level / image-level extended data allows the current image to enable permission protection, that is, permission protection is enabled for all patch-level images or frame-level images in the current image. The switch control information of the slice level allows the current slice to enable permission protection, that is, permission protection is enabled for the current frame-level image (i.e., Slice-level image). The switch control information of the tile level allows the current tile to enable permission protection, that is, permission protection is enabled for all patch-level images in the current tile. The switch control information of the patch level allows the current patch to enable permission protection, that is, permission protection is enabled for the current patch-level image.

[0171] Embodiment 4: In Embodiment 1 and Embodiment 2, the encoder may encode the CU partition information and CU permission information of the target image in the bitstream, and the decoder may decode the CU partition information and CU permission information of the target image based on the bitstream. Based on this process, the position of the CU partition information and the CU permission information in the bitstream may be designed, that is, the syntax element for permission protection may be designed. The process may include the following implementation modes:

[0172] Mode 1: The CU partition information and CU permission information of the target image are located in the bitstream of the lowest permission level (i.e., the bitstream of the first permission level). For example, the encoder encodes the CU partition information and CU permission information of the target image in the bitstream of the lowest permission level, and the decoder decodes the CU partition information and the CU permission information from the bitstream of the lowest permission level.

[0173] Exemplarily, CU partition information (i.e., partition information of each CU) and CU permission information (i.e., permission information of each CU) can be placed only in the bitstream of the lowest permission level. Since users of all permission levels can parse the bitstream of the lowest permission level, no matter which target permission level the user adopts, the decoding end can decode the CU partition information and CU permission information from the bitstream of the lowest permission level.

[0174] For decoding-end users with higher than or equal to CU authority (i.e., the target authority level is greater than or equal to the current authority level of the current CU, and they have access rights to the current CU), the reconstruction information corresponding to the current CU can be obtained from the high-authority bitstream (i.e., the high-dimensional reconstruction information corresponding to the current CU is decoded from the target bitstream), and the reconstruction of the current CU is completed based on the reconstruction information. For decoding-end users with lower than CU authority (i.e., the target authority level is less than the current authority level of the current CU, and they do not have access rights to the current CU), the parsing of the current CU is skipped by default, and the reconstruction of the current CU can be completed using the low-dimensional reconstruction information corresponding to the current CU. For example, the current CU can be filled with a fixed value, or the current CU can be filled with the surrounding reconstructed pixels. For the surrounding reconstructed pixels, a prediction mode (such as an intra-frame prediction mode or an inter-frame prediction mode) can be obtained, and the reference pixels in the prediction mode can be filled into the current CU.

[0175] For example, the CU permission information is decoded from the bitstream with the lowest permission level, and the permission level of the current CU is determined to be permission level 2 based on the CU permission information. The decoding users with permission levels less than 2 (that is, the target permission level is less than 2) skip the parsing of the current CU, and the decoding users with permission levels greater than or equal to 2 (that is, the target permission level is greater than or equal to 2) can parse the reconstruction information of the current CU from the bitstream with permission level 2.

[0176] Method 2: The CU division information of the target image is located in the bitstream of the lowest permission level (i.e., the bitstream of the first permission level), and the CU permission information of the target image is located in the bitstream of which the permission level is less than or equal to that of the CU, that is, for the permission information of each CU, the permission information is located in the bitstream of which the permission level is less than or equal to that of the CU. For example, the encoder encodes the CU division information of the target image in the bitstream of the lowest permission level, and the decoder decodes the CU division information from the bitstream of the lowest permission level. And / or, the encoder encodes the permission information of the first CU in the bitstream of which the permission level is less than or equal to that of the first CU, and the decoder decodes the permission information of the first CU from the bitstream of which the permission level is less than or equal to that of the first CU. Wherein, the first CU is any CU in the target image.

[0177] Exemplarily, the CU division information can be placed only in the bitstream of the lowest permission level. Since users of all permission levels can parse the bitstream of the lowest permission level, no matter which target permission level the user adopts, the decoding end can decode the CU division information from the bitstream of the lowest permission level.

[0178] Exemplarily, the permission information of the CU can be placed in a bitstream with a permission level less than or equal to the CU, that is, it is marked in the bitstream with a permission level less than or equal to the CU whether the reconstruction information of the current CU is in the current bitstream. When parsing the CU permissions, it is necessary to parse in the order of the permission levels.

[0179] For example, for each CU, the permission information of the CU can be placed in a bitstream that is less than or equal to the permission level of the CU. In this way, as long as the target permission level adopted by the user is greater than or equal to the permission level of the CU, the decoding end can decode the permission information of the CU from the bitstream.

[0180] For example, each CU among all CUs can be called the first CU, the permission level A of the first CU is determined, and all permission levels less than or equal to permission level A are called permission level A'. Assuming that permission level A is permission level 1, then permission level A' is permission level 1. Assuming that permission level A is permission level 2, then permission level A' is permission level 1 and permission level 2, and so on.

[0181] The encoding end may encode the permission information of the first CU in the bitstream of permission level A', and the decoding end may decode the permission information of the first CU from the bitstream of permission level A' (i.e., the bitstream whose permission level is less than the permission level of the first CU). For example, if the permission level of the first CU is permission level 2, the decoding end may decode the permission information of the first CU from the bitstream of permission level 1, and may decode the permission information of the first CU from the bitstream of permission level 2.

[0182] Exemplarily, when the decoding end decodes the permission information of the first CU from the bitstream of the permission level A', if the permission level A' is less than the permission level of the first CU, that is, for the bitstream whose permission level is less than the permission level of the first CU, the permission information of the first CU in the bitstream is a first value, and the first value indicates that the bitstream does not include high-dimensional reconstruction information corresponding to the first CU; or, if the permission level A' is equal to the permission level of the first CU, that is, for the bitstream whose permission level is equal to the permission level of the first CU, the permission information of the first CU in the bitstream is a second value, and the second value indicates that the bitstream includes high-dimensional reconstruction information corresponding to the first CU.

[0183] For example, if the permission level of the first CU is permission level 2, when the decoding end decodes the permission information of the first CU from the bitstream of permission level 1, the permission information of the first CU is a first value (such as 0), and the first value indicates that the bitstream of permission level 1 does not contain the high-dimensional reconstruction information corresponding to the first CU. When the decoding end decodes the permission information of the first CU from the bitstream of permission level 2, the permission information of the first CU is a second value (such as 1), and the second value indicates that the bitstream of permission level 2 contains the high-dimensional reconstruction information corresponding to the first CU.

[0184] For decoding end users with higher than or equal to CU authority (i.e., the target authority level is greater than or equal to the current authority level of the current CU), the reconstruction information corresponding to the current CU can be obtained from the bitstream of CU authority (i.e., the high-dimensional reconstruction information corresponding to the current CU is decoded from the target bitstream), and the reconstruction of the current CU is completed based on the reconstruction information. For decoding end users with lower than CU authority (i.e., the target authority level is less than the current authority level of the current CU), the parsing of the current CU is skipped by default, and the low-dimensional reconstruction information corresponding to the current CU can be used to complete the reconstruction of the current CU. For example, the current CU can be filled with a fixed value.

[0185] For example, if the permission level of the current CU is 2 and the minimum permission level is 0, the encoder encodes the first value (such as 0) in the bitstreams of permission level 0 and permission level 1 respectively, indicating that the current bitstream does not contain the reconstruction information of the current CU. The encoder encodes the second value (such as 1) in the bitstream of permission level 2, indicating that the current bitstream contains the reconstruction information of the current CU, and the reconstruction information corresponding to the current CU can be parsed from the current bitstream. After the decoder parses the bitstream of permission level 2, the decoding user with the target permission level greater than or equal to 2 can continue to decode the reconstruction information of the current CU from the bitstream of permission level 2.

[0186] Mode 3: In order to increase the parallelism of code stream parsing, that is, so that the code stream of each permission level can be decoded independently, the CU partition information (that is, the partition information of each CU) is placed in the code streams of all permission levels, and the CU permission information (that is, the permission information of each CU) is placed in the code streams of all permission levels. The CU permission mark is whether the current code stream contains the information of the current CU. For example, the encoder encodes the CU partition information of the target image in the code stream of each permission level (such as the first permission level and each second permission level), and the decoder decodes the CU partition information from the code stream of any permission level. For example, the decoder decodes the CU partition information from the code stream of the first permission level, or decodes the CU partition information from the code stream of any second permission level. The encoder encodes the CU permission information of the target image in the code stream of each permission level (such as the first permission level and each second permission level), and the decoder decodes the CU permission information from the code stream of any permission level. For example, the decoder decodes the CU permission information from the code stream of the first permission level, or decodes the CU permission information from the code stream of any second permission level.

[0187] Exemplarily, the CU partition information may be placed in a bitstream of each permission level, and the CU permission information may be placed in a bitstream of each permission level, so that when the decoding end decodes a bitstream of any permission level, the CU partition information and the CU permission information may be decoded from the bitstream. For example, when the decoding end decodes a bitstream of permission level 1, the CU partition information and the CU permission information may be decoded from the bitstream of permission level 1, and when the decoding end decodes a bitstream of permission level 2, the CU partition information and the CU permission information may be decoded from the bitstream of permission level 2, and so on.

[0188] Exemplarily, if the permission information of the second CU of the target image is located in a bitstream of a first permission level and a second permission level, and the second CU is any CU, then: for a bitstream whose permission level is less than the permission level of the second CU, the permission information of the second CU in the bitstream is a first value, and the first value is used to indicate that the bitstream does not contain high-dimensional reconstruction information corresponding to the second CU; or, for a bitstream whose permission level is greater than or equal to the permission level of the second CU, the permission information of the second CU in the bitstream is a second value, and the second value is used to indicate that the bitstream contains high-dimensional reconstruction information corresponding to the second CU.

[0189] For example, if the permission level of the second CU is permission level 2, assuming that there are bitstreams of permission level 1, permission level 2, and permission level 3, then the permission information of the second CU is located in the bitstreams of permission level 1, permission level 2, and permission level 3. Based on this, when the decoding end decodes the permission information of the second CU from the bitstream of permission level 1, the permission information of the second CU is a first value (such as 0), and the first value indicates that the bitstream of permission level 1 does not contain the high-dimensional reconstruction information corresponding to the second CU. When the decoding end decodes the permission information of the second CU from the bitstream of permission level 2, the permission information of the second CU is a second value (such as 1), and the second value indicates that the bitstream of permission level 2 contains the high-dimensional reconstruction information corresponding to the second CU. When the decoding end decodes the permission information of the second CU from the bitstream of permission level 3, the permission information of the second CU is a second value, and the second value indicates that the bitstream of permission level 3 contains the high-dimensional reconstruction information corresponding to the second CU.

[0190] For decoding end users with higher than or equal to CU authority (i.e., the target authority level is greater than or equal to the current authority level of the current CU), the reconstruction information corresponding to the current CU can be obtained from the bitstream of CU authority (i.e., the high-dimensional reconstruction information corresponding to the current CU is decoded from the target bitstream), and the reconstruction of the current CU is completed based on the reconstruction information. For decoding end users with lower than CU authority (i.e., the target authority level is less than the current authority level of the current CU), the parsing of the current CU is skipped by default, and the low-dimensional reconstruction information corresponding to the current CU can be used to complete the reconstruction of the current CU. For example, the current CU can be filled with a fixed value.

[0191] For example, if the permission level of the current CU is 2 and the minimum permission level is 0, the encoder encodes the first value (such as 0) in the bitstreams of permission level 0 and permission level 1, respectively, indicating that the current bitstream does not contain the reconstruction information of the current CU. The encoder encodes the second value (such as 1) in the bitstreams of permission level 2 and permission level 3, indicating that the current bitstream contains the reconstruction information of the current CU, and the reconstruction information corresponding to the current CU can be parsed from the current bitstream. When the decoder parses the bitstream of permission level 2, the decoding user with the target permission level greater than or equal to 2 can continue to decode the reconstruction information of the current CU from the bitstream of permission level 2. When the decoder parses the bitstream of permission level 3, the decoding user with the target permission level greater than or equal to 3 can decode the bitstream of permission level 3. Although the bitstream of permission level 3 does not include the reconstruction information of the current CU, the reconstruction information of the current CU can be decoded from the bitstream of permission level 2.

[0192] Mode 4: In order to increase the parallelism of code stream parsing, that is, so that the code stream of each permission level can be decoded independently, the CU partition information (that is, the partition information of each CU) is placed in the code streams of all permission levels, and the CU permission information (that is, the permission information of each CU) is placed in the code streams of all permission levels. The CU permission mark is whether the current code stream contains the information of the current CU. For example, the encoder encodes the CU partition information of the target image in the code stream of each permission level (such as the first permission level and each second permission level), and the decoder decodes the CU partition information from the code stream of any permission level. For example, the decoder decodes the CU partition information from the code stream of the first permission level, or decodes the CU partition information from the code stream of any second permission level. The encoder encodes the CU permission information of the target image in the code stream of each permission level (such as the first permission level and each second permission level), and the decoder decodes the CU permission information from the code stream of any permission level. For example, the decoder decodes the CU permission information from the code stream of the first permission level, or decodes the CU permission information from the code stream of any second permission level.

[0193] Exemplarily, the CU partition information may be placed in a bitstream of each permission level, and the CU permission information may be placed in a bitstream of each permission level, so that when the decoding end decodes a bitstream of any permission level, the CU partition information and the CU permission information may be decoded from the bitstream.

[0194] Exemplarily, if the permission information of the second CU of the target image is located in a bitstream of a first permission level and a second permission level, and the second CU is any CU, then: for a bitstream whose permission level is not equal to the permission level of the second CU (such as a bitstream whose permission level is greater than the permission level of the second CU, or a bitstream whose permission level is less than the permission level of the second CU), the permission information of the second CU in the bitstream is a first value, and the first value is used to indicate that the bitstream does not contain high-dimensional reconstruction information corresponding to the second CU; for a bitstream whose permission level is equal to the permission level of the second CU, the permission information of the second CU in the bitstream is a second value, and the second value is used to indicate that the bitstream contains high-dimensional reconstruction information corresponding to the second CU.

[0195] If the permission level of the second CU is permission level 2, assuming that there are bitstreams of permission level 1, permission level 2, and permission level 3, the permission information of the second CU is located in the bitstreams of permission level 1, permission level 2, and permission level 3. Based on this, when the decoding end decodes the permission information of the second CU from the bitstream of permission level 1, the permission information of the second CU is a first value (such as 0), and the first value indicates that the bitstream of permission level 1 does not contain the high-dimensional reconstruction information corresponding to the second CU. When the decoding end decodes the permission information of the second CU from the bitstream of permission level 2, the permission information of the second CU is a second value (such as 1), and the second value indicates that the bitstream of permission level 2 contains the high-dimensional reconstruction information corresponding to the second CU. When the decoding end decodes the permission information of the second CU from the bitstream of permission level 3, the permission information of the second CU is a first value, and the first value indicates that the bitstream of permission level 3 does not contain the high-dimensional reconstruction information corresponding to the second CU.

[0196] For decoding end users with higher than or equal to CU authority (i.e., the target authority level is greater than or equal to the current authority level of the current CU), the reconstruction information corresponding to the current CU can be obtained from the bitstream of CU authority (i.e., the high-dimensional reconstruction information corresponding to the current CU is decoded from the target bitstream), and the reconstruction of the current CU is completed based on the reconstruction information. For decoding end users with lower than CU authority (i.e., the target authority level is less than the current authority level of the current CU), the parsing of the current CU is skipped by default, and the low-dimensional reconstruction information corresponding to the current CU can be used to complete the reconstruction of the current CU. For example, the current CU can be filled with a fixed value.

[0197] For example, if the permission level of the current CU is 2 and the minimum permission level is 0, the encoder encodes the first value in the bitstreams of permission level 0 and permission level 1, respectively, indicating that the current bitstream does not contain the reconstruction information of the current CU. The encoder encodes the second value in the bitstream of permission level 2, indicating that the current bitstream contains the reconstruction information of the current CU, and the reconstruction information corresponding to the current CU can be parsed from the current bitstream. The encoder encodes the first value in the bitstream of permission level 3, indicating that the current bitstream does not contain the reconstruction information of the current CU. When the decoder parses the bitstream of permission level 2, since the bitstream of permission level 2 contains the second value, the decoding user with the target permission level greater than or equal to 2 can continue to decode the reconstruction information of the current CU from the bitstream of permission level 2. When the decoder parses the bitstream of permission level 2, since the bitstream contains the first value, this CU is skipped during reconstruction and the parsing of the next CU is entered.

[0198] Embodiment 5: In order to design the syntax elements of rights protection, the syntax elements of rights protection and the syntax elements of the region of interest are combined. Before obtaining the code streams of multiple rights levels corresponding to the target image, the rights protection parameters corresponding to the target image and the region of interest parameters corresponding to the target image can also be decoded from the high-level syntax corresponding to the target image. The high-level syntax may include but is not limited to: sequence-level high-level syntax, sequence-level extended data high-level syntax, image-level high-level syntax, image-level extended data high-level syntax, slice-level high-level syntax, tile-level high-level syntax, and patch-level high-level syntax, which are not limited to this.

[0199] For the convenience of distinction, the high-level syntax adopted by the rights protection parameter may be referred to as the first syntax, and the high-level syntax adopted by the region of interest parameter may be referred to as the second syntax. The first syntax adopted by the rights protection parameter may be different from the second syntax adopted by the region of interest parameter, or the first syntax adopted by the rights protection parameter may be partially the same as the second syntax adopted by the region of interest parameter, or the first syntax adopted by the rights protection parameter may be completely the same as the second syntax adopted by the region of interest parameter.

[0200] In a possible implementation, the permission protection parameters may include but are not limited to at least one of the following: a sequence header switch identifier, an image header switch identifier, an enhancement layer switch identifier, permission level classification information, whether the permission level is predictable identifier, whether the probability model is updated identifier, and CU boundary filtering parameters. The region of interest parameters may include but are not limited to at least one of the following: a sequence header switch identifier, an image header switch identifier, an enhancement layer switch identifier, region category classification information, whether the region category is predictable identifier, whether the probability model is updated identifier, and CU boundary filtering parameters. Of course, the above are just examples of parameters and are not limited to this.

[0201] Exemplarily, the syntax elements of the region of interest parameters may include: a sequence header switch flag of the region of interest, an image header switch flag, an enhancement layer switch flag, region category classification information (such as block-level / CU-level classification information), and whether the region category of each CU block is predictable. On this basis, the image header may also include some additional syntax elements, for example, when encoding and decoding the CU category flag, whether the probability model is updated, and CU boundary filtering parameters (CU boundary filtering parameters are used to perform different filtering processes on different categories of CU boundaries). Of course, the above are just a few examples of region of interest parameters.

[0202] Exemplarily, the syntax elements of the rights protection parameters may include: sequence header switch flag, picture header switch flag, enhancement layer switch flag, rights level classification information (i.e., block-level / CU-level rights), whether the rights level is predictable flag, etc. On this basis, some additional syntax elements may be designed for rights protection, such as the probability model update of the rights level (i.e., whether the probability model is updated), CU boundary filtering parameters (CU boundary filtering parameters are used to perform different filtering processes on CU boundaries of different rights levels), mosaic information of different rights levels, etc., and there is no restriction on the rights protection parameters.

[0203] In order to combine the permission protection and the syntax elements of the region of interest, the following method can be used:

[0204] Mode 1: The first syntax adopted by the rights protection parameter is different from the second syntax adopted by the region of interest parameter. For example, the difference between the first syntax and the second syntax may include at least one of the following: the sequence header switch identifier of the rights protection parameter and the sequence header switch identifier of the region of interest parameter adopt different syntax. The image header switch identifier of the rights protection parameter and the image header switch identifier of the region of interest parameter adopt different syntax. The enhancement layer switch identifier of the rights protection parameter and the enhancement layer switch identifier of the region of interest parameter adopt different syntax. The permission level classification information of the rights protection parameter and the region category classification information of the region of interest parameter adopt different syntax. The permission level predictable flag of the rights protection parameter and the region category predictable flag of the region of interest parameter adopt different syntax. The probability model update flag of the rights protection parameter and the probability model update flag of the region of interest parameter adopt different syntax. The CU boundary filter parameters of the rights protection parameter and the CU boundary filter parameters of the region of interest parameters adopt different syntax.

[0205] For example, the first syntax used by the right protection parameters and the second syntax used by the region of interest parameters are independent of each other, that is, the syntax elements used by the right protection parameters and the region of interest parameters are completely independent. At this time, when the two tools (that is, the right protection function and the region of interest processing function) are turned on at the same time, when parsing the syntax elements of the CU, both the region of interest parameters and the right protection parameters of the CU must be parsed.

[0206] Exemplarily, when two tools are turned on at the same time, during the parsing process, the parameters of the second parsing can refer to the parameters of the first parsing. For example, if the region of interest parameters are parsed first and the permission protection parameters are parsed later, the parsing results of the region of interest parameters can be referred to when parsing the permission protection parameters. For another example, if the permission protection parameters are parsed first and the region of interest parameters are parsed later, the parsing results of the permission protection parameters can be referred to when parsing the region of interest parameters. For example, when the region of interest category of the CU is 5 and the CU permission level is 5, at this time, parse whether the CU level is equal to the CU category. If so, directly equate the CU level to the CU category. Otherwise, parse the CU permissions, or parse the difference between the CU permissions and the category permissions.

[0207] Mode 2: The first syntax adopted by the rights protection parameter is partially identical to the second syntax adopted by the region of interest parameter. For example, the first syntax being partially identical to the second syntax may include but is not limited to at least one of the following: the sequence header switch identifier of the rights protection parameter and the sequence header switch identifier of the region of interest parameter adopt different syntaxes, or the sequence header switch identifier of the rights protection parameter and the sequence header switch identifier of the region of interest parameter adopt the same syntax. The image header switch identifier of the rights protection parameter and the image header switch identifier of the region of interest parameter adopt different syntaxes, or the image header switch identifier of the rights protection parameter and the image header switch identifier of the region of interest parameter adopt the same syntax. The enhancement layer switch identifier of the rights protection parameter and the enhancement layer switch identifier of the region of interest parameter adopt different syntaxes, or the enhancement layer switch identifier of the rights protection parameter and the enhancement layer switch identifier of the region of interest parameter adopt the same syntax. The permission level classification information of the rights protection parameter and the region category classification information of the region of interest parameter adopt different syntaxes, or the permission level classification information of the rights protection parameter and the region category classification information of the region of interest parameter adopt the same syntax. The whether permission level predictable flag of the right protection parameter and the whether region category predictable flag of the region of interest parameter use different syntax, or the whether permission level predictable flag of the right protection parameter and the whether region category predictable flag of the region of interest parameter use the same syntax. The whether probability model update flag of the right protection parameter and the whether probability model update flag of the region of interest parameter use different syntax, or the whether probability model update flag of the right protection parameter and the whether probability model update flag of the region of interest parameter use the same syntax. The CU boundary filtering parameters of the right protection parameter and the CU boundary filtering parameters of the region of interest parameters use different syntax, or the CU boundary filtering parameters of the right protection parameter and the CU boundary filtering parameters of the region of interest parameters use the same syntax.

[0208] In a possible implementation, the first grammar and the second grammar part being identical may include but are not limited to at least one of the following: the sequence header switch identifier of the rights protection parameter and the sequence header switch identifier of the region of interest parameter use different syntax; the image header switch identifier of the rights protection parameter and the image header switch identifier of the region of interest parameter use different syntax; the enhancement layer switch identifier of the rights protection parameter and the enhancement layer switch identifier of the region of interest parameter use the same syntax; the permission level classification information of the rights protection parameter and the region category classification information of the region of interest parameter use the same syntax; the permission level predictability identifier of the rights protection parameter and the region category predictability identifier of the region of interest parameter use the same syntax; the probability model update identifier of the rights protection parameter and the probability model update identifier of the region of interest parameter use the same syntax; the CU boundary filtering parameters of the rights protection parameter and the CU boundary filtering parameters of the region of interest parameters use the same syntax. Of course, the above are just examples and are not limited to this.

[0209] For example, the first syntax used by the rights protection parameters and the second syntax used by the region of interest parameters share an independent part, and the independent syntax elements may include a sequence header switch and an image header switch. The remaining syntax elements except the independent syntax elements may be common syntax elements. At this time, when the two tools (i.e., the rights protection function and the region of interest processing function) are turned on at the same time, when parsing the syntax elements of the CU, for the independent syntax elements, both the region of interest parameters of the CU and the rights protection parameters of the CU must be parsed. For the common syntax elements, only one set of parameters needs to be parsed, based on which the region of interest parameters of the CU and the rights protection parameters of the CU can be obtained.

[0210] Method 3: The first syntax adopted by the right protection parameter is exactly the same as the second syntax adopted by the region of interest parameter, and the first syntax and the second syntax are exactly the same including but not limited to at least one of the following: the sequence header switch identifier of the right protection parameter and the sequence header switch identifier of the region of interest parameter adopt the same syntax; the image header switch identifier of the right protection parameter and the image header switch identifier of the region of interest parameter adopt the same syntax; the enhancement layer switch identifier of the right protection parameter and the enhancement layer switch identifier of the region of interest parameter adopt the same syntax; the permission level classification information of the right protection parameter and the area category classification information of the region of interest parameter adopt the same syntax; the permission level predictable flag of the right protection parameter and the area category predictable flag of the region of interest parameter adopt the same syntax; the probability model update flag of the right protection parameter and the probability model update flag of the region of interest parameter adopt the same syntax; the CU boundary filtering parameters of the right protection parameter and the CU boundary filtering parameters of the region of interest parameters adopt the same syntax.

[0211] For example, the first syntax used by the rights protection parameter is exactly the same as the second syntax used by the region of interest parameter. When the two tools (i.e., the rights protection function and the region of interest processing function) are enabled at the same time, only one set of parameters needs to be parsed when parsing the syntax elements of the CU. Based on this set of parameters, the region of interest parameters of the CU and the rights protection parameters of the CU can be obtained. For example, the rights protection parameters and the region of interest parameters are completely shared, that is, the two tools are strongly bound, and the two tools are enabled and disabled together.

[0212] In Mode 2 and Mode 3, if the sequence header switch identifier of the permission protection parameter and the sequence header switch identifier of the region of interest parameter use the same syntax, when the syntax is a first value, it is determined that the sequence-level switch control information allows the permission protection to be enabled and the region of interest processing to be enabled, and when the syntax is a second value, it is determined that the sequence-level switch control information prohibits the permission protection from being enabled and the region of interest processing from being enabled. The first value and the second value can be arbitrarily configured, such as the first value is 1 and the second value is 0.

[0213] In Mode 2 and Mode 3, if the image header switch flag of the permission protection parameter and the image header switch flag of the region of interest parameter use the same syntax, when the syntax is a first value, it is determined that the image-level switch control information allows the permission protection to be enabled and the region of interest processing to be enabled, and when the syntax is a second value, it is determined that the image-level switch control information prohibits the permission protection from being enabled and the region of interest processing from being enabled. The first value and the second value can be arbitrarily configured, such as the first value is 1 and the second value is 0.

[0214] In mode 2 and mode 3, if the enhancement layer switch flag of the rights protection parameter and the enhancement layer switch flag of the region of interest parameter use the same syntax, when the syntax is a first value, it is determined that the enhancement layer turns on rights protection and region of interest processing, and when the syntax is a second value, it is determined that the enhancement layer turns off rights protection and region of interest processing. For example, the first value can be 1, and the second value can be 0.

[0215] In Mode 2 and Mode 3, if the predictable flag of the permission level of the permission protection parameter and the predictable flag of the region of interest parameter use the same syntax, when the syntax is the first value, the permission level can be predicted based on the permission level of the previous CU, and the region category can be predicted based on the region category of the previous CU. When the syntax is the second value, the permission level and region category are unpredictable, that is, the permission level cannot be predicted based on the permission level of the previous CU, and the region category cannot be predicted based on the region category of the previous CU. For example, the first value can be 1, and the second value can be 0.

[0216] In Mode 2 and Mode 3, if the flag whether the probability model of the rights protection parameter is updated and the flag whether the probability model of the region of interest parameter is updated use the same syntax, when the syntax is the first value, it is determined that the probability model in the rights protection process and the probability model in the region of interest processing process are updated, and when the syntax is the second value, it is determined that the probability model in the rights protection process and the probability model in the region of interest processing process are not updated. For example, the first value can be 1, and the second value can be 0.

[0217] In Method 2 and Method 3, if the permission level classification information of the permission protection parameter and the area category classification information of the area of ​​interest parameter use the same syntax, the classification identifier is parsed from the syntax, and the permission level and area category are determined based on the classification identifier; or, the classification identifier is parsed from the syntax, the permission level is determined based on the classification identifier, and the area category is determined based on the permission level and a mapping relationship; or, the classification identifier is parsed from the syntax, the area category is determined based on the classification identifier, and the permission level is determined based on the area category and a mapping relationship; wherein the mapping relationship represents a mapping relationship between the permission level and the area category.

[0218] For example, if the block-level CU permission parsing and the block-level (CU-level) classification use the same syntax element, that is, the permission level classification information of the permission protection parameter and the region category classification information of the region of interest parameter use the same syntax, then, if the number of permission levels is the same as the number of region categories, the classification identifier is parsed from the syntax, and the permission level and region category are determined based on the classification identifier. For example, if the classification identifier is 0, the permission level is the first permission level and the region category is the first region category. If the classification identifier is 1, the permission level is the second permission level and the region category is the second region category, and so on.

[0219] If the number of permission levels is different from the number of area categories, a mapping relationship can also be obtained, which represents the mapping relationship between permission levels and area categories. Exemplarily, if the number of area categories is greater than the number of permission levels, the mapping relationship can represent mapping area categories to permission levels. For example, there are 8 area categories [0, 1, 2, 3, 4, 5, 6, 7], and different area categories represent different degrees of interest areas. If the number of permission levels is 2, the mapping relationship is [0, 0, 0, 0, 1, 1, 1, 1], indicating that area categories 0, 1, 2, 3 correspond to permission level 0, and area categories 4, 5, 6, 7 correspond to permission level 1. If the number of permission levels is greater than the number of area categories, the mapping relationship can represent the mapping of permission levels to area categories. For example, there are 8 permission levels [0, 1, 2, 3, 4, 5, 6, 7]. If the number of area categories is 2, the mapping relationship is [0, 0, 0, 0, 1, 1, 1, 1], indicating that permission levels 0, 1, 2, 3 correspond to area category 0, and permission levels 4, 5, 6, 7 correspond to area category 1.

[0220] If the number of area categories is greater than the number of permission levels, the classification identifier is parsed from the syntax, the area category is determined based on the classification identifier, and the permission level is determined based on the area category and the mapping relationship. For example, if the classification identifier is 0, the area category is the first area category 0, and the mapping relationship is queried through the area category 0 to obtain permission level 0. If the classification identifier is 5, the area category is the fifth area category 4, and the mapping relationship is queried through the area category 4 to obtain permission level 1, and so on.

[0221] If the number of permission levels is greater than the number of regional categories, the classification identifier is parsed from the grammar, the permission level is determined based on the classification identifier, and the regional category is determined based on the permission level and the mapping relationship. For example, if the classification identifier is 0, the permission level is the first permission level 0, and the mapping relationship is queried through the permission level 0 to obtain regional category 0. If the classification identifier is 5, the permission level is the fifth permission level 4, and the mapping relationship is queried through the permission level 4 to obtain regional category 1, and so on.

[0222] Regarding this mapping relationship, the decoding end may decode it from the current syntax, or may decode it from other syntaxes, or may be pre-configured at the decoding end, and there is no restriction on the source of this mapping relationship. Exemplarily, if the number of permission levels is not equal to the number of region categories, a syntax element may be added to the image header to mark the mapping relationship between the permission level and the region category. After the region category (or permission level) is obtained through parsing, the permission level (or region category) is obtained according to the mapping relationship.

[0223] In Method 2 and Method 3, if the CU boundary filter parameters of the permission protection parameters and the CU boundary filter parameters of the region of interest parameters use the same syntax, the candidate filter parameters are parsed from the syntax, and the target filter parameters for permission protection and the target filter parameters for the region of interest are determined based on the candidate filter parameters; or, the first candidate filter parameters are determined based on the permission level and the first mapping relationship obtained, and the second candidate filter parameters are determined based on the region category and the second mapping relationship obtained, the target filter parameters for permission protection are determined based on the first candidate filter parameters, and the target filter parameters for the region of interest are determined based on the second candidate filter parameters; wherein the first mapping relationship includes a mapping relationship between the permission level and the filter parameters, and the second mapping relationship includes a mapping relationship between the region category and the filter parameters.

[0224] For example, when filtering, the same filtering process is used, and a syntax element of a mapping relationship is added to the image header to indicate the mapping relationship between the permission level and the region category filtering process. When filtering, two sets of filtering parameters may be derived, and one of them may be selected for use, or the weighted combination of the two parameters may be used as the final filtering parameter.

[0225] For example, the decoding end can directly parse the candidate filtering parameters from the syntax, and the decoding end can also determine the reference filtering parameters based on the image content of the current CU, and there is no restriction on this process. The target filtering parameters for rights protection and the target filtering parameters for the region of interest can be determined based on the candidate filtering parameters and the reference filtering parameters, the region to be filtered for rights protection can be filtered based on the target filtering parameters for rights protection, and the region to be filtered for the region of interest processing can be filtered based on the target filtering parameters for the region of interest.

[0226] For another example, the decoding end may obtain the first mapping relationship and the second mapping relationship, which may be obtained by decoding from the current syntax, or from other syntaxes, or may be pre-configured at the decoding end, and there is no restriction on the source of the first mapping relationship and the second mapping relationship. For example, the image header adds a syntax element of the mapping relationship, and the first mapping relationship and the second mapping relationship are decoded from the syntax element.

[0227] The first mapping relationship may include a mapping relationship between an authority level and a filtering parameter. After obtaining the authority level of the authority protection, the decoding end may query the first mapping relationship through the authority level to obtain the first candidate filtering parameter for the authority protection. The second mapping relationship may include a mapping relationship between an area category and a filtering parameter. After obtaining the area category of the region of interest, the decoding end may query the second mapping relationship through the area category to obtain the second candidate filtering parameter for the region of interest processing.

[0228] The decoding end can also determine the reference filtering parameters based on the image content of the current CU. The decoding end can determine the target filtering parameters for rights protection based on the first candidate filtering parameters and the reference filtering parameters, or the decoding end can perform operations (such as weighted operations) on the first candidate filtering parameters and the second candidate filtering parameters to obtain weighted candidate filtering parameters, and determine the target filtering parameters for rights protection based on the weighted candidate filtering parameters and the reference filtering parameters. Then, the rights-protected area to be filtered is filtered based on the rights-protected target filtering parameters. In addition, the decoding end can determine the target filtering parameters of the area of ​​interest based on the second candidate filtering parameters and the reference filtering parameters, or the decoding end can perform operations (such as weighted operations) on the first candidate filtering parameters and the second candidate filtering parameters to obtain weighted candidate filtering parameters, and determine the target filtering parameters of the area of ​​interest based on the weighted candidate filtering parameters and the reference filtering parameters. Then, the area to be filtered processed by the area of ​​interest is filtered based on the target filtering parameters of the area of ​​interest.

[0229] In one possible implementation, for Mode 1, Mode 2, and Mode 3, during the decoding process of the current CU, if the current CU belongs to both the region of interest and the high-authority region, the reconstruction information of the current CU is parsed from the bitstream of the high-authority level. If the current CU only belongs to the region of interest but not the high-authority region, during the reconstruction process, the current CU is restricted from referring to other CUs. If the current CU does not belong to the region of interest but belongs to the high-authority region, if the current CU chooses to skip parsing at this time, then, if the current CU is intra-predicted, the predicted pixels are obtained from the current frame based on the prediction mode in the high-authority bitstream, and if the current CU is inter-predicted, the prediction mode (such as the default prediction mode or the prediction mode obtained by some method) is used to obtain the predicted pixels from the reference frame, and the predicted pixels are used as the reconstructed pixels.

[0230] Embodiment 6: In Embodiment 1 and Embodiment 2, the encoding end sends the code streams of multiple permission levels corresponding to the target image to the decoding end, and the decoding end obtains the code streams of multiple permission levels corresponding to the target image. The packaging method of the code streams of different permission levels can be based on patch-level packaging. For example, if there are K permission levels, K is a positive integer greater than 1, and the code streams of multiple permission levels corresponding to the target image (such as a patch-level image) include RBSP code streams of K permission levels corresponding to the target image, and the RBSP code streams of K permission levels are located in K NAL units, and the RBSP code streams of different permission levels are located in different NAL units. For example, when the frame-level permission protection switch is turned on, each patch-level image corresponds to at least two RBSP code stream packets, and the at least two RBSP code stream packets include a low permission level RBSP code stream packet and at least one high permission level RBSP code stream packet. For example, assuming that K permission levels are 3 permission levels, each patch-level image corresponds to 3 permission levels of RBSP code stream packets, and these 3 permission levels of RBSP code stream packets are located in 3 NAL units. For example, the RBSP code stream packet of permission level 1 is located in NAL unit 1, the RBSP code stream packet of permission level 2 is located in NAL unit 2, and the RBSP code stream packet of permission level 3 is located in NAL unit 3. The information in the RBSP code stream packet can be specified as at least one of the following:

[0231] The RBSP code stream of a high authority level includes the partitioning information (such as position, shape, etc.) of each CU.

[0232] Each RBSP codestream only contains codestreams with the same permissions in one patch.

[0233] The RBSP code stream with a low authority level includes the division information of each CU (such as position, shape, etc.), while the RBSP code stream with a high authority level does not include the division information of the CU. In this case, before parsing the RBSP code stream with a high authority level, the RBSP code stream with a low authority level is parsed to obtain the division information of each CU.

[0234] The patch header information is included only in the RBSP code stream of the lowest permission level. The header information content may include the CU partition information and filtering information of the entire patch. There is no restriction on the content of this header information.

[0235] The high-privilege-level RBSP code stream and the low-privilege-level RBSP code stream obtained based on the patch image are placed in different NAL units. When the low-privilege-level RBSP code stream and the high-privilege-level RBSP code stream are interleaved and transmitted, the high-privilege-level RBSP code stream can be transmitted first, or the low-privilege-level RBSP code stream can be transmitted first. Therefore, the interleaving order can be: privilege 0 NAL-> privilege 1 NAL-> privilege 2 NAL-> ...-> privilege n NAL, or, the interleaving order can be: privilege n NAL-> privilege n-1 NAL-> ...-> privilege 1 NAL-> privilege 0 NAL, or, any order of interleaving. Among them, privilege 0 NAL indicates the NAL unit where the RBSP code stream of the lowest privilege level is located, privilege 1 NAL indicates the NAL unit where the RBSP code stream of privilege level 1 is located, and so on, privilege n NAL indicates the NAL unit where the RBSP code stream of the highest privilege level is located. Based on the above interleaving order, when the decoding end obtains the code streams of multiple permission levels corresponding to the target image, it can obtain the NAL units corresponding to the RBSP code streams of multiple permission levels in sequence according to the interleaving order from high to low permission levels; or, obtain the NAL units corresponding to the RBSP code streams of multiple permission levels in sequence according to the interleaving order from low to high permission levels; or, obtain the NAL units corresponding to the RBSP code streams of multiple permission levels in sequence according to the preset interleaving order of the permission levels.

[0236] For RBSP bitstreams with low permission levels, if the permission identifier of the current CU is a high permission level, parsing is skipped by default. For RBSP bitstreams with high permission levels, the information of each CU is parsed. For example, if the user at the decoding end is a low permission level (that is, the target permission level is a low permission level), when reconstructing a high permission CU, the default value is used to fill. If the user at the decoding end is a high permission level, when reconstructing a high permission CU, the RBSP bitstream of the corresponding high permission level is parsed to obtain the reconstructed value. For another example, if the user at the decoding end is a low permission level, when reconstructing a high permission CU, intra-frame prediction uses a certain fixed prediction mode to obtain the reconstructed value, and inter-frame prediction uses the reconstructed value at the same position of the reference frame as the reconstructed value. If the user at the decoding end is a high permission level, when reconstructing a high permission CU, the RBSP bitstream of the corresponding high permission level is parsed to obtain the reconstructed value.

[0237] For the RBSP code stream of each permission level, the RBSP code stream is encrypted using a patch-level encryption method, and / or the RBSP code stream is encrypted using a CU-level encryption method.

[0238] If the RBSP codestream is encrypted using a patch-level encryption method, and the RBSP codestream is encrypted using a CU-level encryption method, then: when the target permission level (i.e., the permission level of the decoding end user) is less than the current patch-level permission, it is prohibited to parse the information in the RBSP codestream, that is, the entire patch cannot obtain high-dimensional reconstruction information. When the target permission level is greater than or equal to the current patch-level permission, continue to parse the CU information in the RBSP codestream. When parsing the CU information in the RBSP codestream, since the RBSP codestream is encrypted using a CU-level encryption method, if the target permission level is less than the current CU-level permission, it is prohibited to parse the high-dimensional reconstruction information of the current CU, that is, the current CU cannot obtain high-dimensional reconstruction information; when the target permission level is greater than or equal to the current CU-level permission, continue to parse the high-dimensional reconstruction information of the current CU, that is, the high-dimensional reconstruction information of the current CU can be parsed. In summary, after obtaining all resolvable codestreams, the CU partition information and CU permission information in the patch can be parsed from a certain permission codestream, and the high-dimensional reconstruction information can be parsed from the corresponding permission codestream according to the CU partition information and CU permission information. For example, the encapsulation package of each RBSP code stream contains two encryption methods, patch-level encryption and CU-level encryption, and each encryption can contain multiple levels. If based on patch encryption, the decoder with a permission level lower than the permission level of the current patch cannot parse any information at all, that is, the lowest permission information cannot be parsed. If the decoder user obtains patch-level permissions, it will continue to parse the internal CU information of each RBSP code stream. After obtaining patch-level permissions, it is assumed that the permission level of the RBSP code stream with the lowest permission level has been obtained.

[0239] If the RBSP codestream is encrypted using a patch-level encryption method, but the RBSP codestream is not encrypted using a CU-level encryption method, then: when the target permission level is less than the current patch-level permission, it is prohibited to parse the information in the RBSP codestream, that is, the entire patch cannot obtain high-dimensional reconstruction information. When the target permission level is greater than or equal to the current patch-level permission, continue to parse the CU information in the RBSP codestream. When parsing the CU information in the RBSP codestream, since the RBSP codestream is not encrypted using a CU-level encryption method, the high-dimensional reconstruction information of the current CU can be directly parsed without comparing the relationship between the target permission level and the current CU-level permission, that is, the high-dimensional reconstruction information of the current CU can be parsed.

[0240] If the RBSP code stream is not encrypted using the patch-level encryption method, but the RBSP code stream is encrypted using the CU-level encryption method, then: there is no need to compare the target permission level (i.e., the permission level of the decoding end user) with the current patch-level permission, and the CU information in the RBSP code stream can be directly parsed. When parsing the CU information in the RBSP code stream, since the RBSP code stream is encrypted using the CU-level encryption method, if the target permission level is less than the current CU-level permission, parsing the high-dimensional reconstruction information of the current CU is prohibited, that is, the current CU cannot obtain the high-dimensional reconstruction information; when the target permission level is greater than or equal to the current CU-level permission, the high-dimensional reconstruction information of the current CU continues to be parsed, that is, the high-dimensional reconstruction information of the current CU can be parsed.

[0241] Embodiment 7: In Embodiment 1 and Embodiment 2, the encoding end sends code streams of multiple permission levels corresponding to the target image to the decoding end, and the decoding end obtains code streams of multiple permission levels corresponding to the target image. The packaging method of code streams of different permission levels can also be combined with the knowledge base frame code stream.

[0242] For example, if there are K permission levels, the code streams of multiple permission levels corresponding to the target image (such as a patch-level image) include the RBSP code streams of the K permission levels corresponding to the target image, the RBSP code streams of the K permission levels are located in K NAL units, and the RBSP code streams of different permission levels are located in different NAL units. On this basis, if there are multiple target images, the knowledge base frame code stream referenced by the i-th target image is interleaved with the main bit stream of the i-th target image, the main bit stream of the i-th target image includes the RBSP code streams of the K permission levels corresponding to the i-th target image, and the i-th target image is any target image.

[0243] For example, if the knowledge base frame is transmitted as a knowledge bit stream separately, the RBSP code stream of the high permission level and the RBSP code stream of the low permission level are transmitted as the main bit stream and have nothing to do with the knowledge bit stream. However, if the knowledge bit stream is transmitted interleaved with the main bit stream, an interleaving method can be specified. An example of the interleaving method is: the knowledge base frame code stream referenced by the i-th target image is interleaved with the main bit stream of the i-th target image, for example, the knowledge base frame code stream referenced by the first target image is interleaved with the main bit stream of the first target image, the knowledge base frame code stream referenced by the second target image is interleaved with the main bit stream of the second target image, and so on.

[0244] For example, the main bit stream of the ith target image is interleaved with the jth knowledge base frame code stream, and the jth knowledge base frame is the reference knowledge base frame of the ith target image. Alternatively, one knowledge base frame code stream is interleaved with the main bit streams of N target images, where N is greater than or equal to 1, and the one knowledge base frame serves as the reference knowledge base frame of the N target images, that is, the same knowledge base frame can be referenced by N target images. In summary, the main bit stream of the ith target image is interleaved with the knowledge base frame code stream referenced by the ith target image. For example, if the same knowledge base frame is referenced by one target image, the knowledge bit stream of the first patch is interleaved with the main bit stream of the first frame image, the knowledge bit stream of the second patch is interleaved with the main bit stream of the second frame image, and so on.

[0245] In a possible implementation, the following interleaving scheme may be used when combined with the knowledge base frame code stream:

[0246] Interleaving scheme 1: The knowledge base frame code stream referenced by the i-th target image is located behind the last RBSP code stream among the K permission level RBSP code streams corresponding to the i-th target image.

[0247] For example, each patch of the knowledge base image is packaged as an RBSP, which is processed to obtain a VCL NAL unit, which is interleaved with the NAL unit generated by each frame of the main bitstream in the following way: the current frame image header NAL-> the permission 0 information NAL in the first patch of the current frame-> the permission 1 information NAL in the first patch of the current frame->... (multiple permission information, multiple patches)-> knowledge base NAL (that is, placed after each frame NAL). In summary, the knowledge base frame code stream is located after the RBSP code stream of the last permission level.

[0248] Exemplarily, VCL NAL refers to a code stream obtained by encoding the patch content, that is, NAL packaged with header information and NAL packaged with extended information do not belong to VCL NAL.

[0249] Interleaving scheme 2: The knowledge base frame code stream referenced by the i-th target image is located in front of the first RBSP code stream among the K permission level RBSP code streams corresponding to the i-th target image.

[0250] For example, each patch of the knowledge base is packaged as an independent VCL NAL and placed before each frame of VCL NAL. For example, each patch of the knowledge base image is packaged as an RBSP, which is processed to obtain a VCL NAL unit, and the method of interleaving with the NAL unit generated by the main bitstream of each frame is: current frame image header NAL->knowledge base NAL->permission 0 information NAL in the first patch of the current frame->permission 1 information NAL in the first patch of the current frame->... (multiple permission information, multiple patches). In summary, it can be seen that the knowledge base frame code stream is located in front of the RBSP code stream of the first permission level.

[0251] Interleaving scheme 3: If the i-th target image is not the last target image of all target images, the knowledge base frame code stream referenced by the i-th target image is located behind the last RBSP code stream of the K permission levels corresponding to the i-th target image. Alternatively, if the i-th target image is the last target image of all target images, the knowledge base frame code stream referenced by the i-th target image is located in front of the first RBSP code stream of the K permission levels corresponding to the i-th target image.

[0252] For example, each patch of the knowledge base is packaged as an independent VCL NAL. When the VCL NAL generated by the knowledge base other than the last patch is interleaved, it is placed after all NALs of the corresponding image, such as after the last RBSP code stream of the K permission levels corresponding to the target image, see interleaving scheme 1. When the VCL NAL generated by the last patch of the knowledge base is interleaved, it is placed before the corresponding image header NAL, such as before the first RBSP code stream of the K permission levels corresponding to the target image, see interleaving scheme 2. After decoding and obtaining the last patch information, the complete knowledge base frame can be obtained, and the current frame can refer to the knowledge base frame.

[0253] Interleaving scheme 4: Place the NAL generated by the knowledge base patch before or after the image header NAL position of each frame, or place the NAL generated by the knowledge base patch before or after a certain extended information NAL of each frame.

[0254] It can be seen from the above embodiments that when permission protection is enabled, various types of information permission protection of images and videos (such as license plates, local areas of the human body, screens, text, etc.) can be performed to improve data security. For example, when there are information security issues with images, permission protection can be performed on the images so that users with high permission levels can view a large amount or even all of the image information, and users with low permission levels can only view a small amount of image information. For example, the image is divided into low-authority areas and high-authority areas, and low-authority users can only view low-authority areas, and high-authority users can view both low-authority areas and high-authority areas.

[0255] Embodiment 8: In the embodiment of the present application, an image decoding method is proposed, see Figure 5 FIG. 1 is a flow chart of the method, which is applied to a decoding end (also called a video decoder), and includes:

[0256] Step 501: When permission protection is enabled, obtain a minimum permission level code stream and at least one non-minimum permission level code stream corresponding to a target image. The target image may be a patch-level image or a frame-level image.

[0257] Step 502: Decode the lowest authority level code stream to obtain first reconstruction information corresponding to the target image, and determine a first reconstructed image corresponding to the target image based on the first reconstruction information.

[0258] Step 503: For each non-minimum authority level code stream, decode the non-minimum authority level code stream to obtain second reconstruction information corresponding to the target image, and determine the second reconstructed image corresponding to the target image based on the first reconstruction information corresponding to the target image and the second reconstruction information corresponding to the target image.

[0259] In a possible implementation, when permission protection is enabled, the first permission level code stream and the second permission level code stream corresponding to the target image can be obtained, the target image can be a patch-level image or a frame-level image, the target image corresponds to at least two frames of spatial domain images, and the at least two frames of spatial domain images can include a basic layer image and an enhanced layer image; wherein the first permission level code stream is a minimum permission level code stream, and the second permission level code stream is a non-minimum permission level code stream. Then, the first permission level code stream is decoded to obtain the first reconstruction information corresponding to the target image, and the basic layer reconstructed image corresponding to the target image is determined based on the first reconstruction information; the second permission level code stream is decoded to obtain the second reconstruction information corresponding to the target image, and the enhanced layer reconstructed image corresponding to the target image is determined based on the first reconstruction information and the second reconstruction information, and the resolution of the enhanced layer reconstructed image can be the same as the resolution of the basic layer reconstructed image.

[0260] In a possible implementation, when permission protection is enabled, a first permission level code stream and a second permission level code stream corresponding to a target image are obtained, where the target image is a patch-level image or a frame-level image; wherein the target image corresponds to at least two time domain image frames, and the at least two time domain image frames include a k-th time domain image and a (k+i)-th time domain image, where i is 1-M, where M represents the total number of second permission level code streams, the first permission level code stream is a lowest permission level code stream, and the second permission level code stream is a non-lowest permission level code stream; the first permission level code stream is decoded to obtain first reconstruction information corresponding to the target image, and the k-th time domain reconstructed image corresponding to the target image is determined based on the first reconstruction information; the i-th second permission level code stream among the M second permission level code streams is decoded to obtain second reconstruction information corresponding to the target image, and the (k+i)-th time domain image corresponding to the target image is determined based on the first reconstruction information and the second reconstruction information.

[0261] Exemplarily, the above execution order is only for the convenience of describing the examples given. In practical applications, the execution order between the steps can also be changed, and there is no limitation on this execution order. Moreover, in other embodiments, the steps of the corresponding method are not necessarily executed in the order shown and described in this specification, and the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; multiple steps described in this specification may also be combined into a single step for description in other embodiments.

[0262] In the embodiment of the present application, an image encoding method is proposed, see Figure 6 FIG. 1 is a flow chart of the method, which is applied to an encoding end (also called a video encoder), and includes:

[0263] Step 601: When permission protection is enabled, obtain a minimum permission level code stream and at least one non-minimum permission level code stream corresponding to a target image. The target image may be a patch-level image or a frame-level image.

[0264] Step 602: Encode first reconstruction information corresponding to the target image in a bitstream at the lowest authority level; wherein the first reconstruction information is used to determine a first reconstructed image corresponding to the target image.

[0265] Step 603: For each non-minimum authority level code stream, encode second reconstruction information corresponding to the target image in the non-minimum authority level code stream; wherein the first reconstruction information corresponding to the target image and the second reconstruction information corresponding to the target image are used to determine the second reconstructed image corresponding to the target image.

[0266] In a possible implementation, when permission protection is enabled, a first permission level code stream and a second permission level code stream corresponding to a target image may be obtained, the target image may be a patch-level image or a frame-level image, the target image may correspond to at least two frames of spatial domain images, and the at least two frames of spatial domain images may include a base layer image and an enhancement layer image; wherein the first permission level code stream may be a minimum permission level code stream, and the second permission level code stream may be a non-minimum permission level code stream; first reconstruction information corresponding to the target image is encoded in the first permission level code stream, and the first reconstruction information is used to determine the base layer reconstructed image corresponding to the target image; second reconstruction information corresponding to the target image is encoded in the second permission level code stream, and the first reconstruction information and the second reconstruction information are used to determine the enhancement layer reconstructed image corresponding to the target image; wherein the resolution of the enhancement layer reconstructed image is the same as the resolution of the base layer reconstructed image.

[0267] In a possible implementation, when permission protection is enabled, a first permission level code stream and a second permission level code stream corresponding to a target image are obtained, and the target image is a patch-level image or a frame-level image; wherein the target image corresponds to at least two time domain image frames, and the at least two time domain image frames include a k-th time domain image and a (k+i)-th time domain image, and the value of i is 1-M, and M represents the total number of second permission level code streams, the first permission level code stream is a lowest permission level code stream, and the second permission level code stream is a non-lowest permission level code stream; first reconstruction information corresponding to the target image is encoded in the first permission level code stream, and the first reconstruction information is used to determine the k-th time domain reconstructed image corresponding to the target image; and second reconstruction information corresponding to the target image is encoded in the i-th second permission level code stream among the M second permission level code streams; wherein the first reconstruction information and the second reconstruction information are used to determine the (k+i)-th time domain image corresponding to the target image.

[0268] Exemplarily, the above execution order is only for the convenience of describing the examples given. In practical applications, the execution order between the steps can also be changed, and there is no limitation on this execution order. Moreover, in other embodiments, the steps of the corresponding method are not necessarily executed in the order shown and described in this specification, and the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; multiple steps described in this specification may also be combined into a single step for description in other embodiments.

[0269] As can be seen from the above technical solutions, in the embodiments of the present application, when permission protection is enabled, the code streams of different permissions can be divided into multiple frames for decoding, and various information permission protections of images and videos (such as license plates, local areas of the human body, screens, text, etc.) can be performed to improve data security. For example, when there are information security issues with images, permission protection can be performed on the images so that users with high permission levels can view a large amount or even all of the image information, while users with low permission levels can only view a small amount of image information.

[0270] Example 9: In Example 8, for the same target image (such as a patch-level image or a frame-level image), a frame-level permission protection code stream distribution based on the spatial domain can be designed. When implementing permission protection, the code streams of different permissions can be divided into multiple frames for decoding, and non-minimum permissions can be processed as a spatial domain enhancement layer.

[0271] Exemplarily, the encoding end may obtain the minimum permission level code stream (i.e., the first permission level code stream) and at least one non-minimum permission level code stream (i.e., the second permission level code stream) corresponding to the target image, and send the minimum permission level code stream and the at least one non-minimum permission level code stream to the decoding end, and the decoding end obtains the minimum permission level code stream (i.e., the first permission level code stream) and at least one non-minimum permission level code stream (i.e., the second permission level code stream) corresponding to the target image. Among them, the target image may correspond to at least two frames of spatial domain images (i.e., at least two frames of spatial domain images generated based on the same target image), and the minimum permission level code stream and the at least one non-minimum permission level code stream are code streams corresponding to the same target image, that is, each frame of the spatial domain image corresponds to one code stream. Among them, the at least two frames of spatial domain images may include a base layer image and an enhancement layer image.

[0272] Exemplarily, if the target image corresponds to at least two frames of spatial domain images, that is, at least two frames of spatial domain images that are different in the spatial domain are generated based on the target image, the decoding end can decode the lowest authority level code stream to obtain first reconstruction information, and determine a first reconstructed image based on the first reconstruction information. The first reconstructed image can be a basic layer reconstructed image, that is, the decoding end decodes the lowest authority code stream to obtain a basic layer reconstructed image.

[0273] The decoding end may decode the non-minimum permission level code stream to obtain the second reconstruction information, and determine the second reconstructed image based on the first reconstruction information and the second reconstruction information, and the second reconstructed image may be an enhanced layer reconstructed image, that is, the decoding end decodes the non-minimum permission level code stream to obtain an enhanced layer reconstructed image. Wherein, when the target image corresponds to M non-minimum permission level code streams, an enhanced layer reconstructed image may be obtained by decoding each non-minimum permission level code stream, that is, M enhanced layer reconstructed images may be obtained, and M is a positive integer.

[0274] Exemplarily, the resolution of the enhanced layer reconstructed image is the same as the resolution of the base layer reconstructed image, and the CU division information of the enhanced layer reconstructed image is the same as the CU division information of the base layer reconstructed image.

[0275] Exemplarily, each permission codestream (such as a minimum permission level codestream and at least one non-minimum permission level codestream) includes CU partition information (i.e., partition information of each CU) and CU permission information (i.e., permission information of each CU), and each permission codestream includes all image headers and patch header information.

[0276] Exemplarily, during prediction, the reconstruction process of the enhancement layer is allowed to refer to the base layer, while the reconstruction process of the base layer is not allowed to refer to the enhancement layer, and the division of the base layer and the enhancement layer is required to be completely consistent. For example, when decoding the lowest authority level bitstream, the base layer reconstructed image can only be determined based on the first reconstruction information, and the base layer reconstructed image cannot be determined by referring to the second reconstruction information, that is, the base layer reconstruction process is not allowed to refer to the enhancement layer. When decoding a bitstream not at the lowest authority level, the enhancement layer reconstructed image can be determined based on the first reconstruction information and the second reconstruction information, that is, the enhancement layer reconstruction process refers to the base layer.

[0277] Exemplarily, during reconstruction, the reconstructed image of the enhancement layer can be merged with the reconstructed image of the base layer, and the reconstructed content of the enhancement layer parsed CU can replace the corresponding position of the base layer reconstructed image, and these positions are marked. When the base layer of the next frame is reconstructed, reference to the reconstructed image generated by the enhancement layer is not allowed.

[0278] For example, the enhanced layer reconstructed image and the base layer reconstructed image may be merged to obtain a merged image. The target image may include at least one CU. For each CU, if the enhanced layer reconstructed image includes the reconstructed image block corresponding to the CU, the image content corresponding to the CU in the base layer reconstructed image is replaced by the reconstructed image block; if the enhanced layer reconstructed image does not include the reconstructed image block corresponding to the CU, the image content corresponding to the CU in the base layer reconstructed image is retained. On this basis, the modified base layer reconstructed image is used as the merged image.

[0279] After obtaining the merged image, the first type CU in the merged image can be marked, where the first type CU is a CU whose image content comes from the enhanced layer reconstructed image; wherein, when generating a subsequent basic layer reconstructed image, it is prohibited to refer to the first type CU in the merged image to generate the subsequent basic layer reconstructed image.

[0280] Exemplarily, during reconstruction, the enhanced layer reconstructed image and the base layer reconstructed image may be retained simultaneously, that is, two frames of images may be retained simultaneously. When reconstructing the base layer of the next frame, the base layer reconstructed image of the previous frame may be referenced. When reconstructing the enhanced layer of the next frame, it is allowed to reference the enhanced layer reconstructed image of the previous frame, the base layer reconstructed image of the current frame, and the base layer reconstructed image of the previous frame.

[0281] Example 10: In Example 8, for the same target image (such as a patch-level image or a frame-level image), a frame-level permission protection code stream distribution based on the time domain can be designed. When implementing permission protection, the code streams with different permissions can be divided into multiple frames for decoding, and non-minimum permissions can be processed as time domain images.

[0282] Exemplarily, the encoding end may obtain the minimum permission level code stream (i.e., the first permission level code stream) and at least one non-minimum permission level code stream (i.e., the second permission level code stream) corresponding to the target image, and send the minimum permission level code stream and the at least one non-minimum permission level code stream to the decoding end, and the decoding end obtains the minimum permission level code stream (i.e., the first permission level code stream) and at least one non-minimum permission level code stream (i.e., the second permission level code stream) corresponding to the target image. The target image may correspond to at least two time domain image frames (i.e., at least two time domain image frames generated based on the same target image), and the minimum permission level code stream and the at least one non-minimum permission level code stream are code streams corresponding to the same target image, that is, each time domain image frame corresponds to one code stream. The at least two time domain image frames may include the kth time domain image frame and the (k+i)th time domain image frame, i is 1-M, and M represents the total number of non-minimum permission level code streams.

[0283] Exemplarily, if the target image corresponds to at least two time domain image frames, that is, at least two time domain image frames that are different in the time domain are generated based on the target image, the decoding end can decode the lowest authority level code stream to obtain first reconstruction information, and determine the first reconstructed image based on the first reconstruction information. The first reconstructed image can be the k-th frame time domain reconstructed image, that is, the decoding end decodes the lowest authority code stream to obtain the k-th frame time domain reconstructed image.

[0284] The decoding end decodes the i-th non-minimum privilege level code stream among the M non-minimum privilege level code streams to obtain the second reconstruction information, and determines the second reconstructed image based on the first reconstruction information and the second reconstruction information. The second reconstructed image may be the (k+i)-th frame time domain reconstructed image, that is, the decoding end decodes the i-th non-minimum privilege level code stream to obtain the (k+i)-th frame time domain reconstructed image, where i is a positive integer. When the target image corresponds to M non-minimum privilege level code streams, the first non-minimum privilege level code stream is decoded to obtain the (k+1)-th frame time domain reconstructed image, the second non-minimum privilege level code stream is decoded to obtain the (k+2)-th frame time domain reconstructed image, and so on, to obtain M frames of time domain reconstructed images of the M non-minimum privilege level code streams.

[0285] Exemplarily, the resolution of the time domain reconstructed image of the (k+i)th frame is the same as the resolution of the time domain reconstructed image of the kth frame, and the CU division information of the time domain reconstructed image of the (k+i)th frame is the same as the CU division information of the time domain reconstructed image of the kth frame. For example, the resolutions of the time domain reconstructed image of the kth frame, the time domain reconstructed image of the (k+1)th frame, the time domain reconstructed image of the (k+2)th frame, ..., the time domain reconstructed image of the (k+M)th frame may all be the same, and the CU division information of the time domain reconstructed image of the kth frame, the time domain reconstructed image of the (k+1)th frame, the time domain reconstructed image of the (k+2)th frame, ..., the time domain reconstructed image of the (k+M)th frame may all be the same.

[0286] Exemplarily, the lowest permission code stream is decoded to obtain the kth frame time domain reconstructed image, and the image obtained by decoding the non-lowest permission code stream is used as the k+1th, k+2th, ..., k+Mth frame time domain reconstructed image. Each permission code stream (such as the lowest permission level code stream and at least one non-lowest permission level code stream) includes CU partition information (i.e., the partition information of each CU), and each permission code stream includes all image headers and patch header information.

[0287] Exemplarily, during prediction, the reconstruction process of the time domain reconstructed image of the (k+i)th frame is allowed to refer to the time domain reconstructed image of the kth frame, but the reconstruction process of the time domain reconstructed image of the kth frame is not allowed to refer to the time domain reconstructed image of the (k+i)th frame. For example, when decoding the bitstream of the lowest authority level, the time domain reconstructed image of the kth frame can only be determined based on the first reconstruction information, but the time domain reconstructed image of the kth frame cannot be determined by referring to the time domain reconstructed image of the (k+i)th frame, that is, the reconstruction process of the time domain reconstructed image of the kth frame is not allowed to refer to the time domain reconstructed image of the (k+i)th frame.

[0288] When decoding a non-minimum privilege level code stream, the (k+i)th frame time domain reconstructed image can be determined based on the first reconstruction information and the second reconstruction information, that is, the reconstruction process of the (k+i)th frame time domain reconstructed image refers to the kth frame time domain reconstructed image. For example, the target image may include at least one CU. For each CU, the i-th non-minimum privilege level code stream (that is, the second privilege level code stream) can be decoded to obtain the second reconstruction information corresponding to the CU, and the reconstructed image block corresponding to the CU can be determined based on the first reconstruction information and the second reconstruction information corresponding to the CU; the (k+i)th frame time domain reconstructed image is generated based on the reconstructed image block corresponding to each CU.

[0289] Exemplarily, during prediction, the reconstruction process of the (k+1)th time-domain reconstructed image is allowed to refer to the kth time-domain reconstructed image, and the kth time-domain reconstructed image is not allowed to refer to the (k+1)th time-domain reconstructed image or the (k-1)th time-domain reconstructed image. However, the kth time-domain reconstructed image is allowed to refer to the (k+M+1) or (kM)th frame image, that is, it is allowed to refer to the time-domain reconstructed images corresponding to other target images. The reconstruction process of the (k+M)th time-domain reconstructed image is allowed to refer to the kth, k+1, ..., k+M-1th frames.

[0290] Exemplarily, during reconstruction, only the time domain reconstructed image of the kth frame is retained. For the time domain reconstructed images of the k+1 to k+M frames, the reconstructed pixels of the CU directly marked as permission-protected replace the content of the corresponding position in the kth frame (the CU division information of the kth frame and the (k+i)th frame are required to be completely consistent). When predicting the reference, it is stipulated that the kth frame cannot refer to the reconstructed pixels generated by the k+1 to k+M frames. Therefore, the image information positions generated by the k+1 to k+M frames are recorded and saved to restrict low-authority references to high-authority.

[0291] Example 11: In Example 8, for the same target image (such as a patch-level image or a frame-level image), a frame-level permission protection code stream distribution based on the time domain can be designed. When implementing permission protection, the code streams with different permissions can be divided into multiple frames for decoding, and non-minimum permissions can be processed as time domain images.

[0292] Exemplarily, the encoding end may obtain the minimum permission level code stream (i.e., the first permission level code stream) and at least one non-minimum permission level code stream (i.e., the second permission level code stream) corresponding to the target image, and send the minimum permission level code stream and the at least one non-minimum permission level code stream to the decoding end, and the decoding end obtains the minimum permission level code stream (i.e., the first permission level code stream) and at least one non-minimum permission level code stream (i.e., the second permission level code stream) corresponding to the target image. The target image may correspond to at least two time domain image frames (i.e., at least two time domain image frames generated based on the same target image), and the minimum permission level code stream and the at least one non-minimum permission level code stream are code streams corresponding to the same target image, that is, each time domain image frame corresponds to one code stream. The at least two time domain image frames may include the kth time domain image frame and the (k+i)th time domain image frame, i is 1-M, and M represents the total number of non-minimum permission level code streams.

[0293] Exemplarily, if the target image corresponds to at least two time domain image frames, that is, at least two time domain image frames that are different in the time domain are generated based on the target image, the decoding end can decode the lowest authority level code stream to obtain first reconstruction information, and determine the first reconstructed image based on the first reconstruction information. The first reconstructed image can be the k-th frame time domain reconstructed image, that is, the decoding end decodes the lowest authority code stream to obtain the k-th frame time domain reconstructed image.

[0294] The decoding end decodes the i-th non-minimum privilege level code stream among the M non-minimum privilege level code streams to obtain the second reconstruction information, and determines the second reconstructed image based on the first reconstruction information and the second reconstruction information. The second reconstructed image may be the (k+i)-th frame time domain reconstructed image, that is, the decoding end decodes the i-th non-minimum privilege level code stream to obtain the (k+i)-th frame time domain reconstructed image, where i is a positive integer. When the target image corresponds to M non-minimum privilege level code streams, the first non-minimum privilege level code stream is decoded to obtain the (k+1)-th frame time domain reconstructed image, the second non-minimum privilege level code stream is decoded to obtain the (k+2)-th frame time domain reconstructed image, and so on, to obtain M frames of time domain reconstructed images of the M non-minimum privilege level code streams.

[0295] Exemplarily, the resolution of the time domain reconstructed image of the (k+i)th frame is the same as the resolution of the time domain reconstructed image of the kth frame, and the CU division information of the time domain reconstructed image of the (k+i)th frame is the same as the CU division information of the time domain reconstructed image of the kth frame. For example, the resolutions of the time domain reconstructed image of the kth frame, the time domain reconstructed image of the (k+1)th frame, the time domain reconstructed image of the (k+2)th frame, ..., the time domain reconstructed image of the (k+M)th frame may all be the same, and the CU division information of the time domain reconstructed image of the kth frame, the time domain reconstructed image of the (k+1)th frame, the time domain reconstructed image of the (k+2)th frame, ..., the time domain reconstructed image of the (k+M)th frame may all be the same.

[0296] Exemplarily, the kth frame time domain reconstructed image is obtained by decoding the minimum authority code stream, and the images obtained by decoding the non-minimum authority code stream are used as the k+1th, k+2th, ..., k+Mth frame time domain reconstructed images.

[0297] Exemplarily, each permission code stream (such as the lowest permission level code stream and at least one non-lowest permission level code stream) includes CU partition information (i.e., partition information of each CU) and CU permission information (i.e., permission information of each CU). Each permission code stream includes all image headers and patch header information.

[0298] Exemplarily, during prediction, the reconstruction process of the time domain reconstructed image of the (k+i)th frame is allowed to refer to the time domain reconstructed image of the kth frame, but the reconstruction process of the time domain reconstructed image of the kth frame is not allowed to refer to the time domain reconstructed image of the (k+i)th frame. For example, when decoding the bitstream of the lowest authority level, the time domain reconstructed image of the kth frame can only be determined based on the first reconstruction information, but the time domain reconstructed image of the kth frame cannot be determined by referring to the time domain reconstructed image of the (k+i)th frame, that is, the reconstruction process of the time domain reconstructed image of the kth frame is not allowed to refer to the time domain reconstructed image of the (k+i)th frame.

[0299] When decoding a non-minimum privilege level bitstream, the (k+i)th frame time domain reconstructed image can be determined based on the first reconstruction information and the second reconstruction information, that is, the reconstruction process of the (k+i)th frame time domain reconstructed image refers to the kth frame time domain reconstructed image. For example, the target image includes a first type CU and a second type CU, the privilege level of the first type CU is greater than the minimum privilege level, and the privilege level of the second type CU is the minimum privilege level. The i-th non-minimum privilege level bitstream (that is, the second privilege level bitstream) is decoded to obtain the second reconstruction information corresponding to the first type CU, and the reconstructed image block corresponding to the first type CU is determined based on the first reconstruction information and the second reconstruction information corresponding to the first type CU. The reconstructed image block corresponding to the second type CU is obtained from the first reconstructed image (that is, the reconstructed image block corresponding to the second type CU is not obtained by decoding the non-minimum privilege level bitstream). On this basis, the (k+i)th frame time domain reconstructed image can be generated based on the reconstructed image block corresponding to each first type CU and the reconstructed image block corresponding to each second type CU.

[0300] Exemplarily, during reconstruction, only the time domain reconstructed image of the kth frame is retained. For the time domain reconstructed images of the k+1 to k+M frames, the reconstructed pixels of the CU directly marked as permission-protected replace the content of the corresponding position in the kth frame (the CU division information of the kth frame and the (k+i)th frame are required to be completely consistent). When predicting the reference, it is stipulated that the kth frame cannot refer to the reconstructed pixels generated by the k+1 to k+M frames. Therefore, the image information positions generated by the k+1 to k+M frames are recorded and saved to restrict low-authority references to high-authority.

[0301] Exemplarily, after generating the (k+i)th time domain reconstructed image based on the reconstructed image blocks corresponding to each first-class CU and the reconstructed image blocks corresponding to each second-class CU, the first-class CU in the (k+i)th time domain image is marked, and when generating the next kth time domain reconstructed image, it is prohibited to generate the next kth time domain reconstructed image with reference to the first-class CU in the (k+i)th time domain reconstructed image.

[0302] Exemplarily, the above embodiments can be implemented separately or in combination. For example, each embodiment in Embodiment 1 to Embodiment 11 can be implemented separately, and at least two embodiments in Embodiment 1 to Embodiment 11 can be implemented in combination. Exemplarily, in the above embodiments, the content of the encoding end can also be applied to the decoding end, that is, the decoding end can be processed in the same way, and the content of the decoding end can also be applied to the encoding end, that is, the encoding end can be processed in the same way, which will not be repeated here.

[0303] Based on the same application concept as the above method, an image decoding device is also proposed in the embodiment of the present application, and the device is applied to the decoding end, and the device includes: a memory, which is configured to store video data; a decoder, which is configured to implement the decoding method in the above embodiment 1 to embodiment 11, that is, the processing flow of the decoding end. For example, in a possible implementation, the decoder is configured to implement:

[0304] When enabling permission protection, obtaining one or more bitstreams of permission levels corresponding to a target image, the target image being a patch-level image or a frame-level image, and the target image including at least one CU; wherein, if the target image includes a CU of a first permission level and a CU of a second permission level, the bitstreams of the multiple permission levels include a bitstream of the first permission level and a bitstream of the second permission level; wherein, the first permission level is a minimum permission level, and the second permission level is a non-minimum permission level;

[0305] The CU partition information and the CU permission information of the target image are decoded based on the code stream of the one or more permission levels, and each CU of the target image is decoded based on the CU partition information and the CU permission information.

[0306] Alternatively, when permission protection is enabled, a first permission level code stream and a second permission level code stream corresponding to a target image are obtained, the target image is a patch-level image or a frame-level image, and the target image corresponds to at least two frames of spatial domain images, and the at least two frames of spatial domain images include a basic layer image and an enhancement layer image; wherein the first permission level code stream is a minimum permission level code stream, and the second permission level code stream is a non-minimum permission level code stream; the first permission level code stream is decoded to obtain first reconstruction information corresponding to the target image, and the basic layer reconstructed image corresponding to the target image is determined based on the first reconstruction information; the second permission level code stream is decoded to obtain second reconstruction information corresponding to the target image, and the enhancement layer reconstructed image corresponding to the target image is determined based on the first reconstruction information and the second reconstruction information, and the resolution of the enhancement layer reconstructed image is the same as the resolution of the basic layer reconstructed image.

[0307] Alternatively, when permission protection is enabled, a first permission level code stream and a second permission level code stream corresponding to a target image are obtained, where the target image is a patch-level image or a frame-level image; wherein the target image corresponds to at least two time domain image frames, the at least two time domain image frames include a k-th time domain image frame and a (k+i)-th time domain image frame, i is 1-M, M represents the total number of second permission level code streams, the first permission level code stream is a minimum permission level code stream, and the second permission level code stream is a non-minimum permission level code stream; the first permission level code stream is decoded to obtain first reconstruction information corresponding to the target image, and the k-th time domain reconstructed image corresponding to the target image is determined based on the first reconstruction information; the i-th second permission level code stream among the M second permission level code streams is decoded to obtain second reconstruction information corresponding to the target image, and the (k+i)-th time domain image corresponding to the target image is determined based on the first reconstruction information and the second reconstruction information.

[0308] Based on the same application concept as the above method, an image encoding device is also proposed in the embodiment of the present application, and the device is applied to the encoding end, and the device includes: a memory, which is configured to store video data; an encoder, which is configured to implement the encoding method in the above embodiment 1 to embodiment 11, that is, the processing flow of the encoding end. For example, in a possible implementation, the encoder is configured to implement:

[0309] When enabling permission protection, obtaining one or more bitstreams of permission levels corresponding to a target image, the target image being a patch-level image or a frame-level image, and the target image including at least one CU; wherein, if the target image includes a CU of a first permission level and a CU of a second permission level, the bitstreams of the multiple permission levels include a bitstream of the first permission level and a bitstream of the second permission level; wherein, the first permission level is a minimum permission level, and the second permission level is a non-minimum permission level;

[0310] Encoding CU partition information and CU permission information of the target image in the bitstream of the one or more permission levels; the CU permission information is used to indicate the permission level corresponding to each CU in the target image;

[0311] Based on the CU partition information and the CU permission information, encode each CU of the target image in the bitstreams of the multiple permission levels;

[0312] The code streams of multiple permission levels corresponding to the target image are sent to the decoding end.

[0313] Alternatively, when permission protection is enabled, a plurality of bitstreams of permission levels corresponding to a target image are obtained, the target image being a patch-level image or a frame-level image, and the target image including at least one CU; wherein, if the target image includes a CU of a first permission level and a CU of a second permission level, the bitstreams of the plurality of permission levels include a bitstream of the first permission level and a bitstream of the second permission level; wherein, the first permission level is a minimum permission level, and the second permission level is a non-minimum permission level;

[0314] Encoding CU partition information and CU permission information of the target image in the bitstreams of the multiple permission levels; the CU permission information is used to indicate the permission level corresponding to each CU in the target image;

[0315] Based on the CU partition information and the CU permission information, encode each CU of the target image in the bitstreams of the multiple permission levels;

[0316] The code streams of multiple permission levels corresponding to the target image are sent to the decoding end.

[0317] Alternatively, when permission protection is enabled, a first permission level code stream and a second permission level code stream corresponding to a target image are obtained, where the target image is a patch-level image or a frame-level image; wherein the target image corresponds to at least two time domain image frames, where the at least two time domain image frames include a k-th time domain image frame and a (k+i)-th time domain image frame, where i is 1-M, where M represents the total number of second permission level code streams, where the first permission level code stream is a minimum permission level code stream, and where the second permission level code stream is a non-minimum permission level code stream; and first reconstruction information corresponding to the target image is encoded in the first permission level code stream, where the first reconstruction information is used to determine the k-th time domain reconstructed image frame corresponding to the target image;

[0318] The second reconstruction information corresponding to the target image is encoded in the i-th second authority level code stream among the M second authority level code streams; wherein the first reconstruction information and the second reconstruction information are used to determine the (k+i)-th frame time domain image corresponding to the target image.

[0319] Based on the same application concept as the above method, the decoding end device (also referred to as a video decoder) provided in the embodiment of the present application, from the hardware level, its hardware architecture diagram can be specifically referred to as Fig. 7A As shown. It includes: a processor 711 and a machine-readable storage medium 712, the machine-readable storage medium 712 stores machine-executable instructions (image decoding instructions) that can be executed by the processor 711; the processor 711 is used to execute the machine-executable instructions to implement the image decoding methods of the above-mentioned embodiments 1 to 11 of the present application.

[0320] Based on the same application concept as the above method, the encoding end device (also referred to as a video encoder) provided in the embodiment of the present application, from the hardware level, its hardware architecture diagram can be specifically referred to as Figure 7B As shown. It includes: a processor 721 and a machine-readable storage medium 722, the machine-readable storage medium 722 stores machine-executable instructions (image coding instructions) that can be executed by the processor 721; the processor 721 is used to execute the machine-executable instructions to implement the image coding methods of the above-mentioned embodiments 1 to 11 of the present application.

[0321] Based on the same application concept as the above method, an embodiment of the present application also provides a machine-readable storage medium, on which a number of computer instructions are stored. When the computer instructions are executed by a processor, the method disclosed in the above example of the present application can be implemented, such as the image decoding method or the image encoding method in the above embodiments.

[0322] Based on the same application concept as the above method, an embodiment of the present application also provides a computer application, which, when executed by a processor, can implement the image decoding method or image encoding method disclosed in the above example of the present application.

[0323] Based on the same application concept as the above method, an image decoding device is also proposed in an embodiment of the present application, and the device is applied to a decoding end, and the device includes: an acquisition module, which is used to acquire one or more permission level code streams corresponding to a target image when permission protection is enabled, and the target image is a patch-level image or a frame-level image, and the target image includes at least one CU; wherein, if the target image includes a CU of a first permission level and a CU of a second permission level, the multiple permission level code streams include a first permission level code stream and a second permission level code stream; wherein the first permission level is the lowest permission level, and the second permission level is a non-lowest permission level; a decoding module, which is used to decode the CU partition information and CU permission information of the target image based on the one or more permission level code streams, and decode each CU of the target image based on the CU partition information and the CU permission information.

[0324] Exemplarily, when the decoding module decodes each CU of the target image based on the CU partition information and the CU permission information, it is specifically used to: for the current CU to be decoded in the target image, determine the current permission level corresponding to the current CU based on the CU permission information; if the target permission level is determined to have access rights to the current CU based on the current permission level, decode the high-dimensional reconstruction information corresponding to the current CU from the target bitstream based on the CU partition information, and determine the reconstructed image block corresponding to the current CU based on the high-dimensional reconstruction information; wherein, the target permission level is the permission level of the user accessing the decoding end, and the target bitstream is a bitstream whose permission level is less than or equal to the target permission level.

[0325] Exemplarily, when the decoding module decodes each CU of the target image based on the CU partition information and the CU permission information, it is specifically used to: skip decoding the reconstruction information corresponding to the current CU from the target bitstream if it is determined based on the current permission level that the target permission level does not have access rights to the current CU; obtain low-dimensional reconstruction information corresponding to the current CU, and determine the reconstructed image block corresponding to the current CU based on the low-dimensional reconstruction information; wherein the low-dimensional reconstruction information is a fixed reconstruction value, or the low-dimensional reconstruction information is a reconstruction value obtained based on intra-frame prediction, or the low-dimensional reconstruction information is a reconstruction value obtained based on inter-frame prediction.

[0326] Exemplarily, when the decoding module decodes the CU partition information and the CU permission information of the target image based on the code streams of the multiple permission levels, it is specifically used to: if the CU partition information and the CU permission information of the target image are located in the code stream of the first permission level, then decode the CU partition information and the CU permission information from the code stream of the first permission level.

[0327] Exemplarily, when the decoding module decodes the CU partition information and CU permission information of the target image based on the code streams of the multiple permission levels, it is specifically used to: if the CU partition information of the target image is located in the code stream of the first permission level, decode the CU partition information from the code stream of the first permission level; and / or, if the permission information of the first CU of the target image is located in the code stream of the permission level less than or equal to the permission level of the first CU, decode the permission information of the first CU from the code stream of the permission level less than or equal to the permission level of the first CU; wherein the first CU is any CU.

[0328] Exemplarily, for a bitstream whose permission level is less than the permission level of the first CU, the permission information of the first CU in the bitstream is a first value, and the first value is used to indicate that the bitstream does not include high-dimensional reconstruction information corresponding to the first CU; or, for a bitstream whose permission level is equal to the permission level of the first CU, the permission information of the first CU in the bitstream is a second value, and the second value is used to indicate that the bitstream includes high-dimensional reconstruction information corresponding to the first CU.

[0329] Exemplarily, when the decoding module decodes the CU partition information and the CU permission information of the target image based on the code streams of the multiple permission levels, it is specifically used to: if the CU partition information and the CU permission information of the target image are located in the code streams of the first permission level and the second permission level, then the CU partition information and the CU permission information are decoded from the code stream of the first permission level, or the CU partition information and the CU permission information are decoded from the code stream of the second permission level.

[0330] Exemplarily, if the permission information of the second CU of the target image is located in a bitstream of a first permission level and a second permission level, and the second CU is any CU, then: for a bitstream whose permission level is less than the permission level of the second CU, the permission information of the second CU in the bitstream is a first value, and the first value is used to indicate that the bitstream does not contain high-dimensional reconstruction information corresponding to the second CU; or, for a bitstream whose permission level is greater than or equal to the permission level of the second CU, the permission information of the second CU in the bitstream is a second value, and the second value is used to indicate that the bitstream contains high-dimensional reconstruction information corresponding to the second CU; or, for a bitstream whose permission level is not equal to the permission level of the second CU, the permission information of the second CU in the bitstream is a first value, and the first value is used to indicate that the bitstream does not contain high-dimensional reconstruction information corresponding to the second CU; or, for a bitstream whose permission level is equal to the permission level of the second CU, the permission information of the second CU in the bitstream is a second value, and the second value is used to indicate that the bitstream contains high-dimensional reconstruction information corresponding to the second CU.

[0331] Exemplarily, the decoding module is also used to decode the permission protection parameters corresponding to the target image and the area of ​​interest parameters corresponding to the target image from the high-level syntax corresponding to the target image; wherein the first syntax adopted by the permission protection parameters is different from the second syntax adopted by the area of ​​interest parameters, or the first syntax is partially the same as the second syntax, or the first syntax is completely the same as the second syntax.

[0332] Exemplarily, if there are K permission levels, the code streams of multiple permission levels corresponding to the target image include RBSP code streams of the K permission levels corresponding to the target image, the RBSP code streams of the K permission levels are located in K NAL units, and RBSP code streams of different permission levels are located in different NAL units.

[0333] Exemplarily, the decoding module is further used for encrypting an RBSP code stream of each permission level using a patch-level encryption method, and / or encrypting the RBSP code stream using a CU-level encryption method; if the RBSP code stream is encrypted using a patch-level encryption method, then when the target permission level is less than the current patch-level permission, parsing the information in the RBSP code stream is prohibited; when the target permission level is greater than or equal to the current patch-level permission, continuing to parse the CU information in the RBSP code stream; when parsing the CU information in the RBSP code stream, if the RBSP code stream is encrypted using a CU-level encryption method, then when the target permission level is less than the current CU-level permission, parsing the high-dimensional reconstruction information of the current CU is prohibited; when the target permission level is greater than or equal to the current CU-level permission, continuing to parse the high-dimensional reconstruction information of the current CU.

[0334] Exemplarily, when the acquisition module acquires the code streams of multiple permission levels corresponding to the target image, it is specifically used to: sequentially acquire the NAL units corresponding to the RBSP code streams of multiple permission levels in an interleaving order from high to low; or, sequentially acquire the NAL units corresponding to the RBSP code streams of multiple permission levels in an interleaving order from low to high; or, sequentially acquire the NAL units corresponding to the RBSP code streams of multiple permission levels in an interleaving order of the permission levels.

[0335] Based on the same application concept as the above method, an image encoding device is also proposed in an embodiment of the present application, and the device is applied to an encoding end, and the device includes: an acquisition module, which is used to acquire one or more permission level code streams corresponding to a target image when permission protection is enabled, and the target image is a patch-level image or a frame-level image, and the target image includes at least one CU; wherein, if the target image includes a CU of a first permission level and a CU of a second permission level, the multiple permission level code streams include a first permission level code stream and a second permission level code stream; the first permission level is the lowest permission level, and the second permission level is a non-lowest permission level; an encoding module, which is used to encode CU division information and CU permission information of the target image in the one or more permission level code streams; the CU permission information is used to indicate the permission level corresponding to each CU in the target image; based on the CU division information and the CU permission information, each CU of the target image is encoded in the multiple permission level code streams; a sending module, which is used to send the multiple permission level code streams corresponding to the target image to a decoding end.

[0336] Based on the same application concept as the above method, an image decoding device is also proposed in an embodiment of the present application, which is applied to a decoding end, and the device includes: an acquisition module, which is used to acquire a first permission level code stream and a second permission level code stream corresponding to a target image when permission protection is enabled, the target image is a patch-level image or a frame-level image, and the target image corresponds to at least two frames of spatial domain images, and the at least two frames of spatial domain images include a basic layer image and an enhancement layer image; wherein the first permission level code stream is a minimum permission level code stream, and the second permission level code stream is a non-minimum permission level code stream; a decoding module, which is used to decode the first permission level code stream to obtain first reconstruction information corresponding to the target image, and determine the basic layer reconstructed image corresponding to the target image based on the first reconstruction information; decode the second permission level code stream to obtain second reconstruction information corresponding to the target image, and determine the enhancement layer reconstructed image corresponding to the target image based on the first reconstruction information and the second reconstruction information, and the resolution of the enhancement layer reconstructed image is the same as the resolution of the basic layer reconstructed image.

[0337] Exemplarily, the decoding module is also used to merge the enhanced layer reconstructed image with the basic layer reconstructed image to obtain a merged image; wherein the target image includes at least one CU, and for each CU, if the enhanced layer reconstructed image includes a reconstructed image block corresponding to the CU, then the image content corresponding to the CU in the basic layer reconstructed image is replaced by the reconstructed image block; if the enhanced layer reconstructed image does not include the reconstructed image block corresponding to the CU, then the image content corresponding to the CU in the basic layer reconstructed image is retained; wherein the modified basic layer reconstructed image is used as the merged image.

[0338] Exemplarily, the decoding module is also used to mark the first type CU in the merged image, the first type CU being a CU whose image content comes from the enhanced layer reconstructed image; wherein, when generating a subsequent basic layer reconstructed image, it is prohibited to generate a subsequent basic layer reconstructed image with reference to the first type CU in the merged image.

[0339] Based on the same application concept as the above method, an image encoding device is also proposed in an embodiment of the present application, which is applied to an encoding end, and the device includes: an acquisition module, which is used to acquire a first permission level code stream and a second permission level code stream corresponding to a target image when permission protection is enabled, the target image is a patch-level image or a frame-level image, and the target image corresponds to at least two frames of spatial domain images, and the at least two frames of spatial domain images include a basic layer image and an enhancement layer image; wherein the first permission level code stream is a minimum permission level code stream, and the second permission level code stream is a non-minimum permission level code stream; an encoding module, which is used to encode first reconstruction information corresponding to the target image in the first permission level code stream, and the first reconstruction information is used to determine the basic layer reconstructed image corresponding to the target image; and encode second reconstruction information corresponding to the target image in the second permission level code stream, and the first reconstruction information and the second reconstruction information are used to determine the enhancement layer reconstructed image corresponding to the target image; wherein the resolution of the enhancement layer reconstructed image is the same as the resolution of the basic layer reconstructed image.

[0340] Based on the same application concept as the above method, an image decoding device is also proposed in an embodiment of the present application. The device is applied to a decoding end, and the device includes: an acquisition module, which is used to acquire a first permission level code stream and a second permission level code stream corresponding to a target image when permission protection is enabled, and the target image is a patch-level image or a frame-level image; wherein the target image corresponds to at least two time domain images, and the at least two time domain images include a k-th time domain image and a (k+i)-th time domain image, and the value of i is 1-M, and M represents the total number of second permission level code streams, the first permission level code stream is a minimum permission level code stream, and the second permission level code stream is a non-minimum permission level code stream; a decoding module, which is used to decode the first permission level code stream to obtain first reconstruction information corresponding to the target image, and determine the k-th time domain reconstructed image corresponding to the target image based on the first reconstruction information; decode the i-th second permission level code stream among the M second permission level code streams to obtain the second reconstruction information corresponding to the target image, and determine the (k+i)-th time domain image corresponding to the target image based on the first reconstruction information and the second reconstruction information.

[0341] Exemplarily, the target image includes at least one CU, and for each CU, the decoding module is specifically used to decode the i-th second authority level code stream to obtain the second reconstruction information corresponding to the CU, determine the reconstructed image block corresponding to the CU based on the first reconstruction information and the second reconstruction information corresponding to the CU; and generate the (k+i)-th frame time domain image based on the reconstructed image block corresponding to each CU.

[0342] Exemplarily, the target image includes a first-class CU and a second-class CU, the permission level of the first-class CU is greater than the minimum permission level, and the permission level of the second-class CU is the minimum permission level; the decoding module is specifically used to decode the i-th second permission level code stream to obtain the second reconstruction information corresponding to the first-class CU, determine the reconstructed image block corresponding to the first-class CU based on the first reconstruction information and the second reconstruction information corresponding to the first-class CU; obtain the reconstructed image block corresponding to the second-class CU from the first reconstructed image; generate the (k+i)-th frame time domain image based on the reconstructed image block corresponding to each first-class CU and the reconstructed image block corresponding to each second-class CU.

[0343] Exemplarily, the decoding module generates the (k+i)th frame time domain image based on the reconstructed image blocks corresponding to each first-class CU and the reconstructed image blocks corresponding to each second-class CU, and is also used to: mark the first-class CU in the (k+i)th frame time domain image, and when generating a subsequent k-th frame time domain image, it is prohibited to refer to the first-class CU in the (k+i)th frame time domain image.

[0344] Based on the same application concept as the above method, an image encoding device is also proposed in an embodiment of the present application. The device is applied to an encoding end, and the device includes: an acquisition module, which is used to acquire a first permission level code stream and a second permission level code stream corresponding to a target image when permission protection is enabled, and the target image is a patch-level image or a frame-level image; wherein the target image corresponds to at least two time domain images, and the at least two time domain images include a k-th time domain image and a (k+i)-th time domain image, and the value of i is 1-M, and M represents the total number of second permission level code streams, the first permission level code stream is a minimum permission level code stream, and the second permission level code stream is a non-minimum permission level code stream; an encoding module, which is used to encode first reconstruction information corresponding to the target image in the first permission level code stream, and the first reconstruction information is used to determine the k-th time domain reconstructed image corresponding to the target image; and encode the second reconstruction information corresponding to the target image in the i-th second permission level code stream among the M second permission level code streams; wherein the first reconstruction information and the second reconstruction information are used to determine the (k+i)-th time domain image corresponding to the target image.

[0345] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. The present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. The embodiments of the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0346] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. An image decoding method, characterized in that: Applied to a decoding end, the method comprises: When enabling permission protection, obtaining one or more bitstreams of permission levels corresponding to a target image, the target image being a patch-level image or a frame-level image, and the target image including at least one CU; wherein, if the target image includes a CU of a first permission level and a CU of a second permission level, the bitstreams of the multiple permission levels include a bitstream of the first permission level and a bitstream of the second permission level; wherein, the first permission level is a minimum permission level, and the second permission level is a non-minimum permission level; The CU partition information and the CU permission information of the target image are decoded based on the code stream of the one or more permission levels, and each CU of the target image is decoded based on the CU partition information and the CU permission information.

2. The method according to claim 1, characterized in that The decoding of each CU of the target image based on the CU partition information and the CU permission information includes: For the current CU to be decoded in the target image, Determine a current permission level corresponding to the current CU based on the CU permission information; If it is determined based on the current permission level that the target permission level has the access right to the current CU, high-dimensional reconstruction information corresponding to the current CU is decoded from the target bitstream based on the CU partition information, and a reconstructed image block corresponding to the current CU is determined based on the high-dimensional reconstruction information; The target permission level is the permission level of the user accessing the decoding end, and the target code stream is a code stream with a permission level less than or equal to the target permission level.

3. The method according to claim 2, characterized in that The decoding of each CU of the target image based on the CU partition information and the CU permission information further includes: If it is determined based on the current permission level that the target permission level does not have access rights to the current CU, skip decoding the reconstruction information corresponding to the current CU from the target bitstream; obtain low-dimensional reconstruction information corresponding to the current CU, and determine the reconstructed image block corresponding to the current CU based on the low-dimensional reconstruction information; wherein the low-dimensional reconstruction information is a fixed reconstruction value, or the low-dimensional reconstruction information is a reconstruction value obtained based on intra-frame prediction, or the low-dimensional reconstruction information is a reconstruction value obtained based on inter-frame prediction.

4. The method according to any one of claims 1 to 3, characterized in that: The decoding of the CU partition information and the CU permission information of the target image based on the code stream of the multiple permission levels includes: If the CU partition information and the CU permission information of the target image are located in a bitstream of a first permission level, the CU partition information and the CU permission information are decoded from the bitstream of the first permission level.

5. The method according to any one of claims 1 to 3, characterized in that: The decoding of the CU partition information and the CU permission information of the target image based on the code stream of the multiple permission levels includes: If the CU partition information of the target image is located in a bitstream of a first permission level, decoding the CU partition information from the bitstream of the first permission level; and / or, If the permission information of the first CU of the target image is located in a bitstream whose permission level is less than or equal to the permission level of the first CU, the permission information of the first CU is decoded from the bitstream whose permission level is less than or equal to the permission level of the first CU; wherein the first CU is any CU.

6. The method according to claim 5, characterized in that For a bitstream whose permission level is less than the permission level of the first CU, the permission information of the first CU in the bitstream is a first value, and the first value is used to indicate that the bitstream does not include high-dimensional reconstruction information corresponding to the first CU; or, for a bitstream whose permission level is equal to the permission level of the first CU, the permission information of the first CU in the bitstream is a second value, and the second value is used to indicate that the bitstream includes high-dimensional reconstruction information corresponding to the first CU.

7. The method according to any one of claims 1 to 3, characterized in that: The decoding of the CU partition information and the CU permission information of the target image based on the code stream of the multiple permission levels includes: If the CU partition information and the CU permission information of the target image are located in the bitstreams of the first permission level and the second permission level, the CU partition information and the CU permission information are decoded from the bitstream of the first permission level, or the CU partition information and the CU permission information are decoded from the bitstream of the second permission level.

8. The method according to claim 7, characterized in that If the permission information of the second CU of the target image is located in the bitstream of the first permission level and the second permission level, and the second CU is any CU, then: For a bitstream whose permission level is less than the permission level of the second CU, the permission information of the second CU in the bitstream is a first value, and the first value is used to indicate that the bitstream does not include high-dimensional reconstruction information corresponding to the second CU; or, for a bitstream whose permission level is greater than or equal to the permission level of the second CU, the permission information of the second CU in the bitstream is a second value, and the second value is used to indicate that the bitstream includes high-dimensional reconstruction information corresponding to the second CU; Alternatively, for a bitstream whose permission level is not equal to the permission level of the second CU, the permission information of the second CU in the bitstream is a first value, and the first value is used to indicate that the bitstream does not include high-dimensional reconstruction information corresponding to the second CU; or, for a bitstream whose permission level is equal to the permission level of the second CU, the permission information of the second CU in the bitstream is a second value, and the second value is used to indicate that the bitstream includes high-dimensional reconstruction information corresponding to the second CU.

9. The method according to any one of claims 1 to 3, characterized in that: Before acquiring the code streams of multiple permission levels corresponding to the target image, the method further includes: Decode the rights protection parameters corresponding to the target image and the region of interest parameters corresponding to the target image from the high-level syntax corresponding to the target image; wherein a first syntax adopted by the rights protection parameters is different from a second syntax adopted by the region of interest parameters, or the first syntax is partially identical to the second syntax, or the first syntax is completely identical to the second syntax.

10. The method according to claim 9, characterized in that The permission protection parameters include at least one of the following: a sequence header switch identifier, an image header switch identifier, an enhancement layer switch identifier, permission level classification information, whether the permission level is predictable identifier, whether the probability model is updated identifier, and CU boundary filtering parameters; the region of interest parameters include at least one of the following: a sequence header switch identifier, an image header switch identifier, an enhancement layer switch identifier, region category classification information, whether the region category is predictable identifier, whether the probability model is updated identifier, and CU boundary filtering parameters.

11. The method according to claim 10, characterized in that The first grammar and the second grammar are different from each other including at least one of the following: the sequence header switch identifier of the right protection parameter and the sequence header switch identifier of the region of interest parameter use different grammars; the image header switch identifier of the right protection parameter and the image header switch identifier of the region of interest parameter use different grammars; the enhancement layer switch identifier of the right protection parameter and the enhancement layer switch identifier of the region of interest parameter use different grammars; the permission level classification information of the right protection parameter and the region category classification information of the region of interest parameter use different grammars; the permission level predictability identifier of the right protection parameter and the region category predictability identifier of the region of interest parameter use different grammars; the probability model update identifier of the right protection parameter and the probability model update identifier of the region of interest parameter use different grammars; the CU boundary filtering parameters of the right protection parameter and the CU boundary filtering parameters of the region of interest parameters use different grammars; The first grammar and the second grammar part are identical and include at least one of the following: the sequence header switch identifier of the right protection parameter and the sequence header switch identifier of the region of interest parameter use different syntax; the image header switch identifier of the right protection parameter and the image header switch identifier of the region of interest parameter use different syntax; the enhancement layer switch identifier of the right protection parameter and the enhancement layer switch identifier of the region of interest parameter use the same syntax; the permission level classification information of the right protection parameter and the region category classification information of the region of interest parameter use the same syntax; the permission level predictability identifier of the right protection parameter and the region category predictability identifier of the region of interest parameter use the same syntax; the probability model update identifier of the right protection parameter and the probability model update identifier of the region of interest parameter use the same syntax; the CU boundary filtering parameters of the right protection parameter and the CU boundary filtering parameters of the region of interest parameters use the same syntax; The first grammar and the second grammar are completely identical and include at least one of the following: the sequence header switch identifier of the right protection parameter and the sequence header switch identifier of the region of interest parameter use the same syntax; the image header switch identifier of the right protection parameter and the image header switch identifier of the region of interest parameter use the same syntax; the enhancement layer switch identifier of the right protection parameter and the enhancement layer switch identifier of the region of interest parameter use the same syntax; the permission level classification information of the right protection parameter and the region category classification information of the region of interest parameter use the same syntax; the permission level predictable flag of the right protection parameter and the region category predictable flag of the region of interest parameter use the same syntax; the probability model update flag of the right protection parameter and the probability model update flag of the region of interest parameter use the same syntax; the CU boundary filtering parameters of the right protection parameter and the CU boundary filtering parameters of the region of interest parameters use the same syntax.

12. The method according to claim 11, characterized in that If the sequence header switch identifier of the permission protection parameter and the sequence header switch identifier of the region of interest parameter use the same syntax, when the syntax is a first value, it is determined that the sequence-level switch control information allows the permission protection to be enabled and the region of interest processing to be enabled, and when the syntax is a second value, it is determined that the sequence-level switch control information prohibits the permission protection from being enabled and the region of interest processing from being enabled; If the image header switch identifier of the permission protection parameter and the image header switch identifier of the region of interest parameter use the same syntax, when the syntax is a first value, it is determined that the image-level switch control information allows the permission protection to be enabled and the region of interest processing to be enabled, and when the syntax is a second value, it is determined that the image-level switch control information prohibits the permission protection from being enabled and the region of interest processing from being enabled; If the enhancement layer switch flag of the rights protection parameter and the enhancement layer switch flag of the region of interest parameter use the same syntax, when the syntax is a first value, it is determined that the enhancement layer turns on rights protection and region of interest processing, and when the syntax is a second value, it is determined that the enhancement layer turns off rights protection and region of interest processing; If the permission level classification information of the permission protection parameter and the area category classification information of the area of ​​interest parameter use the same syntax, then parse the classification identifier from the syntax, and determine the permission level and area category based on the classification identifier; Or, parsing the classification identifier from the grammar, determining the permission level based on the classification identifier, and determining the area category based on the permission level and the acquired mapping relationship; Or, parsing the classification identifier from the grammar, determining the area category based on the classification identifier, and determining the authority level based on the area category and the acquired mapping relationship; wherein the mapping relationship represents the mapping relationship between the authority level and the area category; If the whether the permission level of the permission protection parameter is predictable and the whether the region category of the region of interest parameter is predictable use the same syntax, when the syntax is a first value, it is determined that the permission level can be predicted based on the permission level of the previous CU, and the region category can be predicted based on the region category of the previous CU, and when the syntax is a second value, it is determined that the permission level and the region category are unpredictable; If the flag whether the probability model of the rights protection parameter is updated and the flag whether the probability model of the region of interest parameter is updated use the same syntax, when the syntax is a first value, it is determined that the probability model in the rights protection process and the probability model in the region of interest processing process are updated, and when the syntax is a second value, it is determined that the probability model in the rights protection process and the probability model in the region of interest processing process are not updated; If the CU boundary filter parameters of the permission protection parameters and the CU boundary filter parameters of the region of interest parameters use the same syntax, the candidate filter parameters are parsed from the syntax, and the target filter parameters for permission protection and the target filter parameters for the region of interest are determined based on the candidate filter parameters; or, the first candidate filter parameters are determined based on the permission level and the first mapping relationship obtained, and the second candidate filter parameters are determined based on the region category and the second mapping relationship obtained, the target filter parameters for permission protection are determined based on the first candidate filter parameters, and the target filter parameters for the region of interest are determined based on the second candidate filter parameters; wherein the first mapping relationship includes a mapping relationship between the permission level and the filter parameters, and the second mapping relationship includes a mapping relationship between the region category and the filter parameters.

13. The method according to any one of claims 1 to 3, characterized in that: If there are K permission levels, K is a positive integer greater than 1, the code streams of multiple permission levels corresponding to the target image include RBSP code streams of the K permission levels corresponding to the target image, and the RBSP code streams of the K permission levels are located in K NAL units, wherein the RBSP code streams of different permission levels are located in different NAL units.

14. The method according to claim 13, characterized in that For each permission level RBSP code stream, the RBSP code stream is encrypted using a patch-level encryption method, and / or the RBSP code stream is encrypted using a CU-level encryption method; If the RBSP code stream is encrypted using a patch-level encryption method, then when the target permission level is less than the current patch-level permission, parsing the information in the RBSP code stream is prohibited; When the target authority level is greater than or equal to the current patch-level authority, continue to parse the CU information in the RBSP codestream; when parsing the CU information in the RBSP codestream, if the RBSP codestream is encrypted using a CU-level encryption method, then when the target authority level is less than the current CU-level authority, parsing the high-dimensional reconstruction information of the current CU is prohibited; when the target authority level is greater than or equal to the current CU-level authority, continue to parse the high-dimensional reconstruction information of the current CU.

15. The method according to claim 13, characterized in that The step of obtaining code streams of multiple permission levels corresponding to the target image includes: According to the interleaving order from high to low of the permission level, the NAL units corresponding to the RBSP code streams of multiple permission levels are obtained in sequence; or, according to the interleaving order from low to high of the permission level, the NAL units corresponding to the RBSP code streams of multiple permission levels are obtained in sequence; or, according to the preset interleaving order of the permission level, the NAL units corresponding to the RBSP code streams of multiple permission levels are obtained in sequence.

16. The method according to claim 13, characterized in that If there are multiple target images, the knowledge base frame code stream referenced by the i-th target image is interleaved with the main bit stream of the i-th target image, and the main bit stream of the i-th target image includes the RBSP code streams of K permission levels corresponding to the i-th target image, and the i-th target image is any target image.

17. The method according to claim 16, characterized in that The knowledge base frame code stream referenced by the i-th target image is located behind the last RBSP code stream among the K permission level RBSP code streams corresponding to the i-th target image; or, The knowledge base frame code stream referenced by the i-th target image is located in front of the first RBSP code stream among the RBSP code streams of K permission levels corresponding to the i-th target image; or, If the i-th target image is not the last target image of all target images, the knowledge base frame code stream referenced by the i-th target image is located behind the last RBSP code stream of the K permission levels corresponding to the i-th target image; or, if the i-th target image is the last target image of all target images, the knowledge base frame code stream referenced by the i-th target image is located in front of the first RBSP code stream of the K permission levels corresponding to the i-th target image.

18. An image encoding method, characterized in that: Applied to the encoding end, the method includes: When enabling permission protection, obtaining one or more bitstreams of permission levels corresponding to a target image, the target image being a patch-level image or a frame-level image, and the target image including at least one CU; wherein, if the target image includes a CU of a first permission level and a CU of a second permission level, the bitstreams of the multiple permission levels include a bitstream of the first permission level and a bitstream of the second permission level; wherein, the first permission level is a minimum permission level, and the second permission level is a non-minimum permission level; Encoding CU partition information and CU permission information of the target image in the bitstream of the one or more permission levels; the CU permission information is used to indicate the permission level corresponding to each CU in the target image; Based on the CU partition information and the CU permission information, encode each CU of the target image in the bitstreams of the multiple permission levels; The code streams of multiple permission levels corresponding to the target image are sent to the decoding end.

19. An image decoding method, characterized in that: Applied to a decoding end, the method comprises: When enabling permission protection, obtaining a first permission level code stream and a second permission level code stream corresponding to a target image, wherein the target image is a patch-level image or a frame-level image, and the target image corresponds to at least two frames of spatial domain images, and the at least two frames of spatial domain images include a base layer image and an enhancement layer image; wherein the first permission level code stream is a minimum permission level code stream, and the second permission level code stream is a non-minimum permission level code stream; Decoding the first permission level code stream to obtain first reconstruction information corresponding to the target image, and determining a base layer reconstructed image corresponding to the target image based on the first reconstruction information; The second permission level code stream is decoded to obtain second reconstruction information corresponding to the target image, and an enhanced layer reconstructed image corresponding to the target image is determined based on the first reconstruction information and the second reconstruction information, wherein the resolution of the enhanced layer reconstructed image is the same as the resolution of the basic layer reconstructed image.

20. The method according to claim 19, characterized in that The method further comprises: Merging the enhanced layer reconstructed image with the base layer reconstructed image to obtain a merged image; The target image includes at least one CU, and for each CU, if the enhanced layer reconstructed image includes a reconstructed image block corresponding to the CU, the image content corresponding to the CU in the base layer reconstructed image is replaced by the reconstructed image block; if the enhanced layer reconstructed image does not include the reconstructed image block corresponding to the CU, the image content corresponding to the CU in the base layer reconstructed image is retained; The modified base layer reconstructed image is used as the merged image.

21. The method according to claim 20, characterized in that The method further comprises: Marking a first type CU in the merged image, where the first type CU is a CU whose image content comes from the enhanced layer reconstructed image; wherein, when generating a subsequent base layer reconstructed image, it is prohibited to refer to the first type CU in the merged image to generate a subsequent base layer reconstructed image.

22. The method according to any one of claims 19 to 21, characterized in that: The CU partition information of the base layer reconstructed image is the same as the CU partition information of the enhancement layer reconstructed image.

23. An image encoding method, characterized in that: Applied to the encoding end, the method includes: When enabling permission protection, obtaining a first permission level code stream and a second permission level code stream corresponding to a target image, wherein the target image is a patch-level image or a frame-level image, and the target image corresponds to at least two frames of spatial domain images, and the at least two frames of spatial domain images include a base layer image and an enhancement layer image; wherein the first permission level code stream is a minimum permission level code stream, and the second permission level code stream is a non-minimum permission level code stream; Encoding first reconstruction information corresponding to the target image in the first permission level code stream, where the first reconstruction information is used to determine a base layer reconstructed image corresponding to the target image; The second reconstruction information corresponding to the target image is encoded in the second permission level code stream, and the first reconstruction information and the second reconstruction information are used to determine the enhanced layer reconstructed image corresponding to the target image; wherein the resolution of the enhanced layer reconstructed image is the same as the resolution of the basic layer reconstructed image.

24. An image decoding method, characterized in that: Applied to a decoding end, the method comprises: When permission protection is enabled, a first permission level code stream and a second permission level code stream corresponding to a target image are obtained, where the target image is a patch-level image or a frame-level image; wherein the target image corresponds to at least two time domain images, where the at least two time domain images include a k-th time domain image and a (k+i)-th time domain image, where the value of i is 1-M, where M represents the total number of second permission level code streams, where the first permission level code stream is a lowest permission level code stream, and where the second permission level code stream is a non-lowest permission level code stream; Decoding the first permission level code stream to obtain first reconstruction information corresponding to the target image, and determining a k-th frame of time-domain reconstructed image corresponding to the target image based on the first reconstruction information; The i-th second authority level code stream among the M second authority level code streams is decoded to obtain second reconstruction information corresponding to the target image, and the (k+i)-th frame time domain image corresponding to the target image is determined based on the first reconstruction information and the second reconstruction information.

25. The method according to claim 24, characterized in that The decoding of the i-th second authority level code stream among the M second authority level code streams to obtain second reconstruction information corresponding to the target image, and determining the (k+i)-th frame of time domain image corresponding to the target image based on the first reconstruction information and the second reconstruction information, comprises: The target image includes at least one CU. For each CU, the i-th second authority level code stream is decoded to obtain second reconstruction information corresponding to the CU, and a reconstructed image block corresponding to the CU is determined based on the first reconstruction information and the second reconstruction information corresponding to the CU; and the (k+i)-th frame time domain image is generated based on the reconstructed image block corresponding to each CU.

26. The method according to claim 24, characterized in that The decoding of the i-th second authority level code stream among the M second authority level code streams to obtain second reconstruction information corresponding to the target image, and determining the (k+i)-th frame of time domain image corresponding to the target image based on the first reconstruction information and the second reconstruction information, comprises: The target image includes a first-class CU and a second-class CU, the permission level of the first-class CU is greater than the lowest permission level, and the permission level of the second-class CU is the lowest permission level; decoding the i-th second permission level code stream to obtain second reconstruction information corresponding to the first-class CU, and determining a reconstructed image block corresponding to the first-class CU based on the first reconstruction information and the second reconstruction information corresponding to the first-class CU; Obtaining a reconstructed image block corresponding to a second type of CU from the first reconstructed image; The (k+i)th frame time domain image is generated based on the reconstructed image block corresponding to each first-category CU and the reconstructed image block corresponding to each second-category CU.

27. The method according to claim 26, characterized in that After generating the (k+i)th frame of time domain image based on the reconstructed image block corresponding to each first-type CU and the reconstructed image block corresponding to each second-type CU, the method further includes: The first type CU in the (k+i)th frame of the time domain image is marked, and when a subsequent kth frame of the time domain image is generated, it is prohibited to refer to the first type CU in the (k+i)th frame of the time domain image.

28. The method according to any one of claims 24 to 27, characterized in that: The CU division information of the k-th frame time domain image is the same as the CU division information of the (k+i)-th frame time domain image.

29. An image encoding method, characterized in that: Applied to the encoding end, the method includes: When permission protection is enabled, a first permission level code stream and a second permission level code stream corresponding to a target image are obtained, where the target image is a patch-level image or a frame-level image; wherein the target image corresponds to at least two time domain images, where the at least two time domain images include a k-th time domain image and a (k+i)-th time domain image, where the value of i is 1-M, where M represents the total number of second permission level code streams, where the first permission level code stream is a lowest permission level code stream, and where the second permission level code stream is a non-lowest permission level code stream; Encoding first reconstruction information corresponding to the target image in the first permission level code stream, where the first reconstruction information is used to determine a k-th frame of time-domain reconstructed image corresponding to the target image; The second reconstruction information corresponding to the target image is encoded in the i-th second authority level code stream among the M second authority level code streams; wherein the first reconstruction information and the second reconstruction information are used to determine the (k+i)-th frame time domain image corresponding to the target image.

30. An image decoding device, characterized in that: Applied to a decoding end, the device comprises: an acquisition module, configured to acquire, when permission protection is enabled, a code stream of one or more permission levels corresponding to a target image, wherein the target image is a patch-level image or a frame-level image, and the target image includes at least one CU; wherein, if the target image includes a CU of a first permission level and a CU of a second permission level, the code streams of the multiple permission levels include a code stream of the first permission level and a code stream of the second permission level; wherein, the first permission level is a minimum permission level, and the second permission level is a non-minimum permission level; A decoding module is used to decode the CU partition information and CU permission information of the target image based on the code stream of the one or more permission levels, and decode each CU of the target image based on the CU partition information and the CU permission information.

31. An image encoding device, characterized in that: Applied to the encoding end, the device includes: an acquisition module, configured to acquire, when permission protection is enabled, a code stream of one or more permission levels corresponding to a target image, wherein the target image is a patch-level image or a frame-level image, and the target image includes at least one CU; wherein, if the target image includes a CU of a first permission level and a CU of a second permission level, the code streams of the multiple permission levels include a code stream of the first permission level and a code stream of the second permission level; wherein, the first permission level is a minimum permission level, and the second permission level is a non-minimum permission level; an encoding module, configured to encode CU partition information and CU permission information of the target image in the bitstream of the one or more permission levels; the CU permission information is used to indicate the permission level corresponding to each CU in the target image; and based on the CU partition information and the CU permission information, encode each CU of the target image in the bitstream of the multiple permission levels; The sending module is used to send the code streams of multiple permission levels corresponding to the target image to the decoding end.

32. An image decoding device, characterized in that: Applied to a decoding end, the device comprises: an acquisition module, configured to acquire, when permission protection is enabled, a first permission level code stream and a second permission level code stream corresponding to a target image, wherein the target image is a patch-level image or a frame-level image, and the target image corresponds to at least two frames of spatial domain images, and the at least two frames of spatial domain images include a base layer image and an enhancement layer image; wherein the first permission level code stream is a minimum permission level code stream, and the second permission level code stream is a non-minimum permission level code stream; A decoding module is used to decode the first permission level code stream to obtain first reconstruction information corresponding to the target image, and determine a basic layer reconstructed image corresponding to the target image based on the first reconstruction information; decode the second permission level code stream to obtain second reconstruction information corresponding to the target image, and determine an enhanced layer reconstructed image corresponding to the target image based on the first reconstruction information and the second reconstruction information, wherein the resolution of the enhanced layer reconstructed image is the same as the resolution of the basic layer reconstructed image.

33. An image encoding device, characterized in that: Applied to the encoding end, the device includes: an acquisition module, configured to acquire, when permission protection is enabled, a first permission level code stream and a second permission level code stream corresponding to a target image, wherein the target image is a patch-level image or a frame-level image, and the target image corresponds to at least two frames of spatial domain images, and the at least two frames of spatial domain images include a base layer image and an enhancement layer image; wherein the first permission level code stream is a minimum permission level code stream, and the second permission level code stream is a non-minimum permission level code stream; A coding module, used to encode first reconstruction information corresponding to the target image in the first permission level code stream, the first reconstruction information is used to determine the base layer reconstructed image corresponding to the target image; encode second reconstruction information corresponding to the target image in the second permission level code stream, the first reconstruction information and the second reconstruction information are used to determine the enhancement layer reconstructed image corresponding to the target image; wherein the resolution of the enhancement layer reconstructed image is the same as the resolution of the base layer reconstructed image.

34. An image decoding device, characterized in that: Applied to a decoding end, the device comprises: an acquisition module, configured to acquire, when permission protection is enabled, a first permission level code stream and a second permission level code stream corresponding to a target image, wherein the target image is a patch level image or a frame level image; wherein the target image corresponds to at least two time domain image frames, wherein the at least two time domain image frames include a k-th time domain image and a (k+i)-th time domain image, wherein the value of i is 1-M, and M represents the total number of second permission level code streams, wherein the first permission level code stream is a lowest permission level code stream, and the second permission level code stream is a non-lowest permission level code stream; a decoding module, configured to decode the first permission level code stream to obtain first reconstruction information corresponding to the target image, and determine a k-th time domain reconstructed image corresponding to the target image based on the first reconstruction information; decode the i-th second permission level code stream among the M second permission level code streams to obtain second reconstruction information corresponding to the target image, and determine a (k+i)-th time domain image corresponding to the target image based on the first reconstruction information and the second reconstruction information.

35. An image encoding device, characterized in that: Applied to the encoding end, the device includes: an acquisition module, configured to acquire, when permission protection is enabled, a first permission level code stream and a second permission level code stream corresponding to a target image, wherein the target image is a patch level image or a frame level image; wherein the target image corresponds to at least two time domain image frames, wherein the at least two time domain image frames include a k-th time domain image and a (k+i)-th time domain image, wherein the value of i is 1-M, and M represents the total number of second permission level code streams, wherein the first permission level code stream is a lowest permission level code stream, and the second permission level code stream is a non-lowest permission level code stream; A coding module, configured to encode first reconstruction information corresponding to the target image in the first permission level code stream, the first reconstruction information being used to determine the k-th time domain reconstructed image corresponding to the target image; and to encode second reconstruction information corresponding to the target image in the i-th second permission level code stream among M second permission level code streams; wherein the first reconstruction information and the second reconstruction information are used to determine the (k+i)-th time domain image corresponding to the target image.

36. A decoding end device, characterized in that: include: a processor and a machine-readable storage medium storing machine-executable instructions executable by the processor; The processor is used to execute machine executable instructions to implement any of the methods described in claims 1-17, or to implement any of the methods described in claims 19-22, or to implement any of the methods described in claims 24-28.

37. A coding end device, characterized in that: include: a processor and a machine-readable storage medium storing machine-executable instructions executable by the processor; The processor is configured to execute machine executable instructions to implement the method of claim 18, 23 or 29.

38. An electronic device, characterized in that: include: a processor and a machine-readable storage medium storing machine-executable instructions executable by the processor; The processor is used to execute machine executable instructions to implement any method described in claims 1-29.

39. A machine-readable storage medium, characterized in that: The machine-readable storage medium stores a plurality of computer instructions, and when the computer instructions are executed by a processor, the method described in any one of claims 1 to 29 is implemented.

Images