Image decoding method, image coding method, image decoding device, image coding device and equipment

Abstract

The invention provides an image decoding method, an image coding method, an image decoding device, an image coding device and equipment, and the image decoding method comprises the steps that when permission protection is started, code streams of one or more permission levels corresponding to a target image are acquired, the target image is a patch-level image or a frame-level image, and the target image comprises at least one CU; wherein if the target image comprises a CU of a first permission level and a CU of a second permission level, the code streams of the multiple permission levels comprise a code stream of the first permission level and a code stream of the second permission level; wherein the first permission level is the lowest permission level, and the second permission level is the non-lowest permission level; and decoding CU division information and CU permission information of the target image based on the code streams of the plurality of permission levels, and decoding each CU of the target image based on the CU division information and the CU permission information. Through the technical scheme of the invention, permission protection of various information of the image can be carried out, and the data security is improved.

Description

Technical Field

[0001] The present application relates to the field of coding and decoding technology, and in particular to an image decoding method, apparatus and equipment thereof, as well as an image encoding method, apparatus and equipment thereof. Background Art

[0002] To save space, video images are encoded before transmission. Complete video encoding involves prediction, transformation, quantization, entropy coding, and filtering. The prediction process includes intra-frame prediction and inter-frame prediction. Inter-frame prediction leverages correlations in the temporal domain of the video, using pixels from adjacent coded images to predict the current pixel, effectively removing temporal redundancy. Intra-frame prediction leverages correlations in the spatial domain of the video, using pixels from coded blocks of the current frame to predict the current pixel, removing spatial redundancy. Both intra-frame and inter-frame prediction methods can reconstruct the complete video image at the decoder, allowing users to view the entire image. However, after the decoder reconstructs the complete video image, it contains all authorized information, such as license plate information, information about local human areas, and screen information. This poses a security risk and requires authorization protection. Summary of the Invention

[0003] In view of this, the present application provides an image decoding, image encoding method, device and equipment thereof, which can protect various types of information rights of images (such as license plates, local areas of the human body, screens, text, etc.).

[0004] The present application provides an image decoding method, applied to a decoding end, the method comprising:

[0005] When rights protection is enabled, obtaining codestreams of one or more permission levels corresponding to a target image, the target image being a patch-level image or a frame-level image, and the target image including at least one CU; wherein, if the target image includes a CU of a first permission level and a CU of a second permission level, the codestreams of the multiple permission levels include a codestream of the first permission level and a codestream of the second permission level; wherein, the first permission level is a minimum permission level, and the second permission level is a non-minimum permission level;

[0006] The CU partition information and CU permission information of the target image are decoded based on the code stream of the one or more permission levels, and each CU of the target image is decoded based on the CU partition information and the CU permission information.

[0007] The present application provides an image encoding method, applied to an encoding end, the method comprising:

[0008] When rights protection is enabled, obtaining codestreams of one or more permission levels corresponding to a target image, the target image being a patch-level image or a frame-level image, and the target image including at least one CU; wherein, if the target image includes a CU of a first permission level and a CU of a second permission level, the codestreams of the multiple permission levels include a codestream of the first permission level and a codestream of the second permission level; wherein, the first permission level is a minimum permission level, and the second permission level is a non-minimum permission level;

[0009] Encoding CU partition information and CU permission information of the target image in the bitstream of the one or more permission levels; the CU permission information is used to indicate the permission level corresponding to each CU in the target image;

[0010] Encoding each CU of the target image in the codestreams of the multiple permission levels based on the CU partition information and the CU permission information;

[0011] The code streams of multiple permission levels corresponding to the target image are sent to the decoding end.

[0012] The present application provides an image decoding method, applied to a decoding end, the method comprising:

[0013] When rights protection is enabled, obtaining a first permission level codestream and a second permission level codestream corresponding to a target image, the target image being a patch-level image or a frame-level image, the target image corresponding to at least two spatial domain image frames, the at least two spatial domain image frames including a base layer image and an enhancement layer image; wherein the first permission level codestream is a lowest permission level codestream, and the second permission level codestream is a non-lowest permission level codestream;

[0014] Decoding the first permission level code stream to obtain first reconstruction information corresponding to the target image, and determining a base layer reconstructed image corresponding to the target image based on the first reconstruction information;

[0015] The second permission level code stream is decoded to obtain second reconstruction information corresponding to the target image, and an enhanced layer reconstructed image corresponding to the target image is determined based on the first reconstruction information and the second reconstruction information, where the resolution of the enhanced layer reconstructed image is the same as the resolution of the base layer reconstructed image.

[0016] The present application provides an image encoding method, applied to an encoding end, the method comprising:

[0017] When rights protection is enabled, obtaining a first permission level codestream and a second permission level codestream corresponding to a target image, the target image being a patch-level image or a frame-level image, the target image corresponding to at least two spatial domain image frames, the at least two spatial domain image frames including a base layer image and an enhancement layer image; wherein the first permission level codestream is a lowest permission level codestream, and the second permission level codestream is a non-lowest permission level codestream;

[0018] Encoding first reconstruction information corresponding to the target image in the first permission level codestream, where the first reconstruction information is used to determine a base layer reconstructed image corresponding to the target image;

[0019] Second reconstruction information corresponding to the target image is encoded in the second permission level code stream, and the first reconstruction information and the second reconstruction information are used to determine the enhanced layer reconstructed image corresponding to the target image; wherein the resolution of the enhanced layer reconstructed image is the same as the resolution of the base layer reconstructed image.

[0020] The present application provides an image decoding method, applied to a decoding end, the method comprising:

[0021] When rights protection is enabled, obtaining a first-rights-level codestream and a second-rights-level codestream corresponding to a target image, where the target image is a patch-level image or a frame-level image; wherein the target image corresponds to at least two time-domain image frames, the at least two time-domain image frames include a k-th time-domain image and a (k+i)-th time-domain image, where i is 1-M, and M represents the total number of second-rights-level codestreams; the first-rights-level codestream is a lowest-rights-level codestream, and the second-rights-level codestream is a non-lowest-rights-level codestream;

[0022] Decoding the first permission level code stream to obtain first reconstruction information corresponding to the target image, and determining a k-th time-domain reconstructed image corresponding to the target image based on the first reconstruction information;

[0023] Decode an i-th second permission level code stream among the M second permission level code streams to obtain second reconstruction information corresponding to the target image, and determine a (k+i)-th frame time domain image corresponding to the target image based on the first reconstruction information and the second reconstruction information.

[0024] The present application provides an image encoding method, applied to an encoding end, the method comprising:

[0025] When rights protection is enabled, obtaining a first-rights-level codestream and a second-rights-level codestream corresponding to a target image, where the target image is a patch-level image or a frame-level image; wherein the target image corresponds to at least two time-domain image frames, the at least two time-domain image frames include a k-th time-domain image and a (k+i)-th time-domain image, where i is 1-M, and M represents the total number of second-rights-level codestreams; the first-rights-level codestream is a lowest-rights-level codestream, and the second-rights-level codestream is a non-lowest-rights-level codestream;

[0026] Encoding first reconstruction information corresponding to the target image in the first permission level code stream, where the first reconstruction information is used to determine a k-th time-domain reconstructed image corresponding to the target image;

[0027] Second reconstruction information corresponding to the target image is encoded in the i-th second authority level code stream among M second authority level code streams; wherein the first reconstruction information and the second reconstruction information are used to determine the (k+i)-th frame time domain image corresponding to the target image.

[0028] The present application provides an image decoding device, applied to a decoding end, the device comprising:

[0029] an acquisition module, configured to, when rights protection is enabled, acquire codestreams of one or more permission levels corresponding to a target image, the target image being a patch-level image or a frame-level image, the target image including at least one CU; wherein, if the target image includes a CU of a first permission level and a CU of a second permission level, the codestreams of the multiple permission levels include a codestream of the first permission level and a codestream of the second permission level; wherein, the first permission level is a minimum permission level, and the second permission level is a non-minimum permission level;

[0030] A decoding module is configured to decode the CU partition information and the CU permission information of the target image based on the code stream of the one or more permission levels, and decode each CU of the target image based on the CU partition information and the CU permission information.

[0031] The present application provides an image encoding device, applied to an encoding end, the device comprising:

[0032] an acquisition module, configured to, when rights protection is enabled, acquire codestreams of one or more permission levels corresponding to a target image, the target image being a patch-level image or a frame-level image, the target image including at least one CU; wherein, if the target image includes a CU of a first permission level and a CU of a second permission level, the codestreams of the multiple permission levels include a codestream of the first permission level and a codestream of the second permission level; wherein, the first permission level is a minimum permission level, and the second permission level is a non-minimum permission level;

[0033] an encoding module, configured to encode CU partition information and CU permission information of the target image in the codestreams of the one or more permission levels, wherein the CU permission information is used to indicate the permission level corresponding to each CU in the target image; and encode each CU of the target image in the codestreams of the multiple permission levels based on the CU partition information and the CU permission information;

[0034] The sending module is used to send the code streams of multiple permission levels corresponding to the target image to the decoding end.

[0035] The present application provides an image decoding device, applied to a decoding end, the device comprising:

[0036] an acquisition module, configured to, when rights protection is enabled, acquire a first permission level codestream and a second permission level codestream corresponding to a target image, the target image being a patch-level image or a frame-level image, the target image corresponding to at least two spatial domain image frames, the at least two spatial domain image frames including a base layer image and an enhancement layer image; wherein the first permission level codestream is a lowest permission level codestream, and the second permission level codestream is a non-lowest permission level codestream;

[0037] a decoding module, configured to decode the first permission level code stream to obtain first reconstruction information corresponding to the target image, and determine a base layer reconstructed image corresponding to the target image based on the first reconstruction information; and decode the second permission level code stream to obtain second reconstruction information corresponding to the target image, and determine an enhancement layer reconstructed image corresponding to the target image based on the first reconstruction information and the second reconstruction information, wherein the resolution of the enhancement layer reconstructed image is the same as the resolution of the base layer reconstructed image.

[0038] The present application provides an image encoding device, applied to an encoding end, the device comprising:

[0039] an acquisition module, configured to, when rights protection is enabled, acquire a first permission level codestream and a second permission level codestream corresponding to a target image, the target image being a patch-level image or a frame-level image, the target image corresponding to at least two spatial domain image frames, the at least two spatial domain image frames including a base layer image and an enhancement layer image; wherein the first permission level codestream is a lowest permission level codestream, and the second permission level codestream is a non-lowest permission level codestream;

[0040] an encoding module configured to encode first reconstruction information corresponding to the target image in the first permission level codestream, the first reconstruction information being used to determine a base layer reconstructed image corresponding to the target image; and to encode second reconstruction information corresponding to the target image in the second permission level codestream, the first reconstruction information and the second reconstruction information being used to determine an enhancement layer reconstructed image corresponding to the target image; wherein a resolution of the enhancement layer reconstructed image is the same as a resolution of the base layer reconstructed image.

[0041] The present application provides an image decoding device, applied to a decoding end, the device comprising:

[0042] an acquisition module, configured to, when permission protection is enabled, acquire a first permission level codestream and a second permission level codestream corresponding to a target image, the target image being a patch-level image or a frame-level image; wherein the target image corresponds to at least two time-domain image frames, the at least two time-domain image frames including a k-th time-domain image frame and a (k+i)-th time-domain image frame, where i is 1-M, M representing the total number of second permission level codestreams, the first permission level codestream being a lowest permission level codestream, and the second permission level codestream being a non-lowest permission level codestream;

[0043] a decoding module configured to decode the first permission level codestream to obtain first reconstruction information corresponding to the target image, and determine a k-th time-domain reconstructed image corresponding to the target image based on the first reconstruction information; decode the i-th second permission level codestream among the M second permission level codestreams to obtain second reconstruction information corresponding to the target image, and determine a (k+i)-th time-domain image corresponding to the target image based on the first reconstruction information and the second reconstruction information.

[0044] The present application provides a device for use at an encoding end, the device comprising:

[0045] an acquisition module, configured to, when permission protection is enabled, acquire a first permission level codestream and a second permission level codestream corresponding to a target image, the target image being a patch-level image or a frame-level image; wherein the target image corresponds to at least two time-domain image frames, the at least two time-domain image frames including a k-th time-domain image frame and a (k+i)-th time-domain image frame, where i is 1-M, M representing the total number of second permission level codestreams, the first permission level codestream being a lowest permission level codestream, and the second permission level codestream being a non-lowest permission level codestream;

[0046] an encoding module configured to encode first reconstruction information corresponding to the target image in the first permission level codestream, the first reconstruction information being used to determine the k-th time-domain reconstructed image corresponding to the target image; and to encode second reconstruction information corresponding to the target image in the i-th second permission level codestream among the M second permission level codestreams; wherein the first reconstruction information and the second reconstruction information are used to determine the (k+i)-th time-domain image corresponding to the target image.

[0047] The present application provides a decoding end device, comprising: a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the above-mentioned example image decoding method.

[0048] The present application provides an encoding end device, comprising: a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the above-mentioned example image encoding method.

[0049] The present application provides an electronic device, comprising: a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the above-mentioned example image decoding method and image encoding method.

[0050] The present application provides a machine-readable storage medium, characterized in that a plurality of computer instructions are stored on the machine-readable storage medium, and when the computer instructions are executed by a processor, the above-mentioned example image decoding method and image encoding method are implemented.

[0051] As can be seen from the above technical solutions, in the embodiments of the present application, when permission protection is enabled, various types of information permission protection of images and videos (such as license plates, local areas of the human body, screens, text, etc.) can be performed to improve data security. For example, when there are information security issues with images, permission protection can be performed on the images so that users with high permission levels can view a large amount or even all of the image information, while users with low permission levels can only view a small amount of image information. For example, the image is divided into low permission areas and high permission areas, and low permission users can only view low permission areas, while high permission users can view both low permission areas and high permission areas. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is a schematic diagram of the video coding framework;

[0053] Figure 2 is a flowchart of an image decoding method in one embodiment of the present application;

[0054] Figure 3 is a flowchart of an image encoding method in one embodiment of the present application;

[0055] Figure 4A This is a flowchart of patch image rights protection coding in one embodiment of the present application;

[0056] Figure 4B This is a flowchart of high-level patch image permission protection in one embodiment of the present application;

[0057] Figure 4C This is a flowchart of low-level patch image permission protection in one embodiment of the present application;

[0058] Figure 5 is a flowchart of an image decoding method in one embodiment of the present application;

[0059] Figure 6 is a flowchart of an image encoding method in one embodiment of the present application;

[0060] Figure 7A This is a hardware structure diagram of a decoding end device in one embodiment of the present application;

[0061] Figure 7B This is a hardware structure diagram of an encoding terminal device in one embodiment of the present application. DETAILED DESCRIPTION

[0062] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, rather than for limiting the present application. The singular forms of "a", "said" and "the" used in the embodiments of the present application and the claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to any or all possible combinations of one or more associated listed items. It should be understood that although the embodiments of the present application may use the terms first, second, etc. to describe various information, these information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of the embodiments of the present application, the first information may also be referred to as the second information, and the second information may also be referred to as the first information, depending on the context. In addition, the word "if" used can be interpreted as "at the time of...", or "when...", or "in response to determination".

[0063] In the embodiments of the present application, a method for image decoding and encoding is proposed, which may involve the following concepts:

[0064] Prediction Signal: A prediction signal is a pixel value derived from an already coded pixel. The difference between the original pixel and the predicted pixel yields a residual, which is then transformed, quantized, and coefficient encoded. Inter-frame prediction signals are pixel values ​​derived from a reference frame (reconstructed pixel frame) for the current block. Due to the discrete pixel positions, interpolation is required to obtain the final predicted pixel. The closer the predicted pixel is to the original pixel, the smaller the residual energy obtained by subtracting the two, resulting in higher coding compression performance.

[0065] Rate-Distortion Optimized (RDO): Coding efficiency is evaluated using two key metrics: bit rate and Peak Signal to Noise Ratio (PSNR). A smaller bitrate results in a higher compression ratio, and a higher PSNR results in better reconstructed image quality. When selecting a mode, the discriminant formula is essentially a comprehensive evaluation of these two metrics. For example, the cost of a mode is: J(mode) = D + λ*R, where D represents distortion, typically measured using the SSE metric (mean squared sum of squared differences between the reconstructed image block and the source image). For cost considerations, the SAD metric (the sum of absolute differences between the reconstructed image block and the source image) can also be used. λ is the Lagrange multiplier, and R is the actual number of bits required to encode the image block in that mode, including the bits required for coding mode information, motion information, and residual information. Using the rate-distortion principle to compare coding modes during mode selection generally ensures optimal coding performance.

[0066] Video Coding Framework: See Figure 1 As shown in FIG, a schematic diagram of a video encoding framework of an encoding end can be used to implement the encoding end processing flow of the present application, and a schematic diagram of a video decoding framework can be used with FIG. Figure 1 Similarly, the video decoding framework can be used to implement the decoding end processing flow of this application.

[0067] For example, see Figure 1 As shown, the video encoding framework can include modules such as prediction, transformation, quantization, entropy encoder, inverse quantization, inverse transformation, reconstruction, and filtering. On the encoding side, the coordination between these modules enables the encoding process. Furthermore, the video decoding framework can include modules such as prediction, transformation, quantization, entropy decoder, inverse quantization, inverse transformation, reconstruction, and filtering. On the decoding side, the coordination between these modules enables the decoding process.

[0068] Numerous coding tools have been proposed for various modules on the encoding side, and each tool often has multiple modes. Different coding tools often achieve optimal coding performance for different video sequences. Therefore, during the encoding process, RDO (Rate-Distortion Optimize) is often used to compare the coding performance of different tools or modes to select the optimal mode. After determining the optimal tool or mode, the decision information for the tool or mode is transmitted by encoding marker information in the bitstream. Although this method brings higher coding complexity, it can adaptively select the optimal mode combination for different content to achieve optimal coding performance. The decoding side obtains relevant mode information by directly parsing the marker information, which has a minimal impact on complexity.

[0069] Bitstream: A bitstream is a sequence of bits consisting of encoded audio, video, and related data. A bitstream can be used to represent a NAL (Network Abstraction Layer) unit stream or a byte stream. Therefore, both NAL unit streams and byte streams are called bitstreams.

[0070] The NAL unit stream format consists of a series of syntax structures called NAL units, which are sorted in decoding order. The decoding order and content of NAL units in the NAL unit stream are constrained.

[0071] A byte stream can be constructed using a NAL unit stream. NAL units are arranged in decoding order, and a start code prefix and several zero-valued bytes are added to each NAL unit to form a byte stream. The NAL unit stream format can be extracted from the byte stream format by searching for a unique start code prefix in the byte stream. In addition to the byte stream format, other methods of constructing NAL units are possible.

[0072] NumBytesInNALunit is used to indicate the length of the NAL unit in bytes. The NAL unit consists of a unit header and a unit payload. The unit payload contains an RBSP (Raw Byte Sequence Payload) syntax structure and possible authentication data payload. It may also contain some emulation_prevention_three_byte (anti-emulation three bytes). NumBytesInNALunit (byte unit in NAL unit) is needed when decoding the NAL unit. In order to derive NumBytesInNALunit, the boundaries of the NAL unit need to be demarcated.

[0073] Generate the NAL unit stream into an ordered byte stream, and the NAL unit boundary position can be identified from the byte stream. For bit-oriented transmission, the bit order in the byte stream starts from the MSB of the first byte, processed to the LSB of the first byte, then the MSB of the second byte, and so on.

[0074] The byte stream format consists of a series of byte stream NAL unit syntax structures, each of which contains a start code prefix followed by a NAL unit.

[0075] The byte stream NAL unit syntax structure may contain an additional zero_byte syntax element and may also contain one or more additional trailing_zero_8bits syntax elements. The first byte stream NAL unit syntax structure may also contain one or more additional leading_zero_8bits syntax elements.

[0076] NAL unit: A NAL unit is a syntactic structure that contains an indication of the type of subsequent data and the number of bytes it contains. The data appears in the form of RBSP and, if necessary, includes interspersed security bytes. A NAL unit stream is a sequence of NAL units.

[0077] Coded video sequence: A coded video sequence is a picture sequence consisting of an IDR picture arranged in decoding order and followed by zero or more non-IDR pictures.

[0078] Emulation prevention byte: The emulation prevention byte is a byte whose value can be 0x03 and may appear in a NAL unit. The presence of the emulation prevention byte ensures that the start code prefix is ​​not included in the subsequent byte-aligned byte stream of the NAL unit.

[0079] Block: A block is an M*N (M columns and N rows) sample matrix in the video signal space, or a block is an M*N transform coefficient matrix in the video signal space.

[0080] Start code prefix: A start code prefix is ​​a unique three-byte sequence equal to 0x000001 in the byte stream. It serves as the prefix for each NAL unit. The decoder (e.g., decoder) can use the position of the start code prefix to determine the beginning of a new NAL unit and the end of the previous one. Anti-counterfeiting bytes are added to NAL units to prevent the appearance of counterfeit start code prefixes.

[0081] Secure front-end device: A secure front-end device is a front-end device equipped with secure cryptographic components. It can use technologies such as certificate storage and management, key storage and management, data signature verification, and data encryption to implement functions such as device identity authentication, video signature, and video encryption.

[0082] Patch partitioning: Patch partitioning is used to indicate how to divide an image into patches and tree-shaped coding units (CTUs). When patch_enable is 0, the entire image has only one patch. When tile_enable is 1, the entire image may be divided into multiple patches. A patch can be composed of a series of tree-shaped coding units (CTUs). The tree-shaped coding unit is the basic unit of encoding. Each tree-shaped coding unit can contain a luma array and two chroma arrays. In other words, a patch can be understood as an image block containing an integer number of CTUs or LCUs.

[0083] The CTU index in the upper left corner of the image is 0, and the CTU index in the image increases in raster scan order. An image is divided into several rectangular areas in the horizontal and vertical directions. Each rectangular area is called a patch. Each patch contains several CTUs and can be independently encoded and decoded in parallel.

[0084] When the image width is greater than or equal to 8 maximum tree coding unit sizes, it can be divided into multiple patch columns in the horizontal direction. When the image width exceeds 64 maximum tree coding unit sizes, it needs to be divided into multiple patch columns. Each patch contains at least 4 CTUs in the horizontal direction and a maximum of 64 CTUs. The image can be divided into multiple patch rows in the vertical direction, and the number of patch rows can be equal to 1, 2, or 4.

[0085] Region of Interest partitioning: An image can be divided into several regions of interest (ROIs). The minimum unit of ROI is 8*8, and the samples in the same prediction unit belong to the same ROI area. If there is ROI partitioning in the image, the value of the parameter segmentation_enable in the corresponding image parameter set is equal to 1. If there is no ROI partitioning in the image, the value of the parameter segmentation_enable in the corresponding image parameter set is equal to 0. Among them, the ROI area in an image can be divided into 8 different levels, which are indicated by the segment_id of the block where the sample is located. Among them, if there is no ROI partitioning in the image, the segment_id of the block where all samples are located should be equal to 0.

[0086] Raw byte sequence payload (RBSP): A raw byte sequence payload is a syntax structure consisting of an integer number of bytes encapsulated in a NAL unit. An RBSP can be empty or contain syntax elements in the form of a data bit string, followed by an RBSP stop bit and zero or more consecutive zero-valued bits. The raw byte sequence payload stop bit is a single bit with a value of 1 that appears after the data bit string in the RBSP. The end position of the data bit string in the RBSP can be determined by searching for the RBSP stop bit in the RBSP.

[0087] Encapsulation and constraints of NAL units: Use emulation_prevention_three_byte to encapsulate an RBSP into a NAL unit. During the encapsulation process, any SODB is allowed to appear in the NAL unit. However, it is necessary to prevent the appearance of a pseudo start code in the NAL unit. By searching for the RBSP bit rbsp_stop_one_bit at the end of the RBSP, the end of the SODB in the NAL unit can be identified.

[0088] The encoding end (such as the encoder) generates a NAL unit from an RBSP through the following steps: searching for the following byte-aligned binary bit string '00000000000000000000000xx' from the RBSP, where xx represents any 2-bit string '00', '01', '10', or '11', and inserting a byte equal to 0x03 therein to form '000000000000000000000011000000xx'. The resulting byte sequence plus the unit header part of the NAL unit that identifies the RBSP data structure type forms the entire NAL unit.

[0089] Based on the above process of generating NAL units, this process allows any SODB to appear in a NAL unit, and at the same time ensures that: there is no byte-aligned pseudo start code in the NAL unit; regardless of whether it is byte-aligned, there is no sequence of 8 bits with a value of 0 followed by a start code in the NAL unit.

[0090] The order of NAL units and the relationship between NAL units and coded images and video sequences, the order and effectiveness of sequence parameter set RBSP and picture parameter set RBSP: The parameters included in a sequence parameter set RBSP can be used by one or more images or SEI NAL units containing a buffering period SEI message. Each sequence parameter set RBSP takes effect at the same time it is received by the decoder and will cause the previously valid sequence parameter set RBSP (if any) to become invalid. At most one sequence parameter set RBSP is valid at a specified time during the decoding process. When a sequence parameter set RBSP is used by an SEI NAL unit containing a buffering period SEI message, the SEI NAL unit can be located after the sequence parameter set RBSP.

[0091] The picture parameter set RBSP contains parameters that can be used by the coded slice NAL unit of a coded picture. Each picture parameter set RBSP takes effect when it is received by the decoder and will cause the previously valid picture parameter set RBSP (if any) to become invalid. For a picture in a certain layer of SVC, at most one picture parameter set RBSP is valid at a given time during the decoding process.

[0092] The relationship between syntax element values ​​in sequence parameter sets and picture parameter sets and other syntax elements is specified only for active sequence parameter sets and active picture parameter sets. During decoding, the parameter values ​​of active picture parameter sets and active sequence parameter sets may remain valid.

[0093] Region of Interest (ROI) syntax elements: An image can be divided into several regions of interest (ROIs). The minimum unit of an ROI is 8*8, and samples within the same prediction unit can belong to the same ROI. The syntax elements of the ROI can be found in Table 1.

[0094] Table 1

[0095]

[0096] Decoding of the region of interest: If the current sequence header Roi_flag is turned on, a set of probability parameters is parsed in the image header, and the segments in the current frame that need to be forced to skip parsing are obtained. When parsing the segment_id of each block, if the current segment_id is a segment that needs to be forced to skip parsing, the parsing of the current block is skipped.

[0097] During the inter-frame prediction process, for blocks that process the region of interest and select the skip mode (each block is a CU, also called a CU block), the default corresponding reference frame is DYNAMIC_REF (the 0th reference frame), the MV is (0,0), and the corresponding reference block is obtained in the base layer reference frame.

[0098] In the DB process, the cu_filter_level corresponding to the current block can be calculated based on the segment_id, reference frame type ref_frame, and prediction mode mode corresponding to the current block as indexes. Different filtering parameters can be used according to different cu_filter_level values ​​corresponding to the current block.

[0099] In one possible implementation, the decoding end can reconstruct a complete video image, and the user can view the complete video image. However, after the complete video image is reconstructed, all permissions may be displayed, such as license plate information, body area information, text information, screen information, etc., which poses a security risk and requires permission protection. To this end, permission protection based on NAL units can be used.

[0100] For NAL unit-based rights protection, the NAL unit decoding process takes the NAL unit as input and outputs the RBSP syntax structure encapsulated in the NAL unit. The RBSP syntax structure can be extracted from the NAL unit. If encryption_idc is 1, the RBSP syntax structure needs to be decrypted when extracting it from the NAL unit to obtain the unencrypted RBSP.

[0101] The RBSP syntax structure in the NAL unit is decoded as follows: When the value of the syntax nal_unit_type in the NAL unit is 1 / 2 / 3 / 4, the code stream in the current NAL unit is parsed and the corresponding tiles are reconstructed using the parsed value. When the value of the syntax nal_unit_type in the NAL unit is 1 / 2 / 3 / 4, for RBSPs that cannot be decrypted, this information is discarded and these tiles cannot be reconstructed.

[0102] For NAL unit-based rights protection, the image is divided into tiles during encoding. Tiles cannot reference each other, and the bitstream information within each tile is packaged together into an RBSP and placed in the NAL unit. Each RBSP bitstream can be specially processed, such as encryption of certain RBSPs. However, the actual tile size is an integer number of CTUs, so special processing can only be applied to large image areas. For smaller objects, special processing of the bitstream of small blocks is not flexible.

[0103] In response to the above findings, this embodiment divides a tile stream into n RBSPs, and writes all different contents within the tile into different RBSPs. These different RBSPs are then packaged into separate NAL units for transmission. The n RBSP streams derived from a tile's content contain streams for n permissions, and streams with different permissions can be processed differently, such as using different encryption permissions.

[0104] In this embodiment, an image decoding method and an image encoding method are proposed, and a code stream processing method based on permission protection is given. During the encoding and decoding process, different permissions are assigned to different image areas, and the code streams based on different permissions are stored separately. When decoding a low-permission area, the high-permission code stream cannot be accessed. On this basis, when there are information security issues with the image, the image can be protected by permissions, and the permission areas of each level of the image can be expressed efficiently and losslessly, so that users with high permission levels can view a large amount or even all of the image information, and users with low permission levels can only view a small amount of image information. For example, the image can be divided into low-permission areas and high-permission areas. Low-permission users can only view low-permission areas, while high-permission users can view both low-permission areas and high-permission areas, thereby improving data security.

[0105] The image decoding method and the image encoding method are described in detail below with reference to several specific embodiments.

[0106] Example 1: In the present application, an image decoding method is proposed. Figure 2 FIG. 1 is a flow chart of the method, which is applied to a decoding end (also called a video decoder), and includes:

[0107] Step 201: When rights protection is enabled, obtain one or more codestreams of permission levels corresponding to a target image, where the target image is a patch-level image or a frame-level image and includes at least one CU. If the target image includes a CU of a first permission level and a CU of a second permission level, the multiple codestreams of permission levels include codestreams of the first permission level and codestreams of the second permission level; the first permission level is the lowest permission level, and the second permission level is a non-lowest permission level. The second permission level can be at least one permission level, i.e., the non-lowest permission level can be at least one non-lowest permission level.

[0108] For example, if all CUs of the target image correspond to the same permission level, a code stream of a permission level corresponding to the target image can be obtained. In this embodiment, taking the example that all CUs of the target image correspond to multiple permission levels, code streams of multiple permission levels corresponding to the target image can be obtained.

[0109] Step 202: Decode the CU partition information and CU permission information of the target image based on the bitstream of one or more permission levels. For example, decode the CU partition information and CU permission information of the target image based on the bitstream of the first permission level and / or the bitstream of the second permission level.

[0110] Step 203: Decode each CU of the target image based on the CU partition information and the CU permission information, that is, decode each CU of the target image one by one based on the CU partition information and the CU permission information.

[0111] In one possible implementation, decoding each CU of a target image based on CU partition information and CU permission information may include, but is not limited to: for a current CU to be decoded in the target image, determining a current permission level corresponding to the current CU based on the CU permission information. If it is determined based on the current permission level that the target permission level has access rights to the current CU, high-dimensional reconstruction information corresponding to the current CU is decoded from the target bitstream based on the CU partition information, and a reconstructed image block corresponding to the current CU is determined based on the high-dimensional reconstruction information. The target permission level may be the permission level of a user accessing the decoding end, and the target bitstream may be a bitstream having a permission level less than or equal to the target permission level.

[0112] In one possible implementation, if it is determined based on the current permission level that the target permission level does not have access rights to the current CU, decoding the reconstruction information corresponding to the current CU from the target bitstream is skipped; low-dimensional reconstruction information corresponding to the current CU is obtained, and the reconstructed image block corresponding to the current CU is determined based on the low-dimensional reconstruction information; wherein the low-dimensional reconstruction information is a fixed reconstruction value, or the low-dimensional reconstruction information is a reconstruction value obtained based on intra-frame prediction, or the low-dimensional reconstruction information is a reconstruction value obtained based on inter-frame prediction.

[0113] For example, the above execution order is only for the convenience of describing the examples given. In actual applications, the execution order between the steps can also be changed, and this execution order is not limited. Moreover, in other embodiments, the steps of the corresponding method are not necessarily executed in the order shown and described in this specification, and the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; multiple steps described in this specification may also be combined into a single step for description in other embodiments.

[0114] In the embodiment of the present application, an image coding method is proposed. Figure 3 FIG. 1 is a flow chart of the method, which is applied to an encoding end (also called a video encoder), and includes:

[0115] Step 301: When rights protection is enabled, obtain one or more codestreams of permission levels corresponding to a target image. The target image is a patch-level image or a frame-level image, and the target image includes at least one CU. If the target image includes a CU of a first permission level and a CU of a second permission level, the codestreams of the multiple permission levels may include both the codestream of the first permission level and the codestream of the second permission level. The first permission level may be the lowest permission level, and the second permission level may be a non-lowest permission level.

[0116] Step 302: Encode CU partition information and CU permission information of the target image in a bitstream of one or more permission levels; wherein the CU permission information is used to indicate the permission level corresponding to each CU in the target image.

[0117] Step 303: Encode each CU of the target image in code streams of multiple permission levels based on the CU partition information and the CU permission information.

[0118] For example, for each CU in the target image, when encoding the CU, a candidate codestream corresponding to the permission level corresponding to the CU is determined (that is, the permission level of the candidate codestream is the same as the permission level corresponding to the CU), and the high-dimensional reconstruction information corresponding to the CU is encoded in the candidate codestream based on the CU partitioning information, and encoding the high-dimensional reconstruction information corresponding to the CU in codestreams of other permission levels is skipped.

[0119] Step 304: Send the code streams of multiple permission levels corresponding to the target image to the decoding end.

[0120] For example, the above execution order is only for the convenience of describing the examples given. In actual applications, the execution order between the steps can also be changed, and this execution order is not limited. Moreover, in other embodiments, the steps of the corresponding method are not necessarily executed in the order shown and described in this specification, and the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; multiple steps described in this specification may also be combined into a single step for description in other embodiments.

[0121] As can be seen from the above technical solutions, in the embodiments of the present application, when permission protection is enabled, various types of information permission protection of images and videos (such as license plates, local areas of the human body, screens, text, etc.) can be performed to improve data security. For example, when there are information security issues with images, permission protection can be performed on the images so that users with high permission levels can view a large amount or even all of the image information, while users with low permission levels can only view a small amount of image information. For example, the image is divided into low permission areas and high permission areas, and low permission users can only view low permission areas, while high permission users can view both low permission areas and high permission areas.

[0122] Example 2: To implement code stream processing based on rights protection, the following method can be used:

[0123] First, when enabling rights protection for a target image, the decoder needs to obtain codestreams corresponding to multiple permission levels for the target image. For example, if there are K permission levels, where K is a positive integer greater than 1, then codestreams for K permission levels are obtained, such as a codestream for permission level 1, a codestream for permission level 2, and so on. Assuming permission level 1 is the lowest permission level (i.e., the first permission level codestream), and permission level K is the highest permission level, the codestreams from permission level 2 through permission level K are all second permission level codestreams. Similar to the decoding process, the encoder needs to obtain codestreams corresponding to multiple permission levels for the target image and send them to the decoder. For example, if the target image includes CUs of the first permission level and CUs of the second permission level, the multiple codestreams for the first permission level and the second permission level codestreams are included; the first permission level is the lowest permission level, and the second permission level is a non-lowest permission level. For example, permission level 1 is the first permission level, and permission levels 2 through K are the second permission levels.

[0124] Among them, the target image can be a patch-level image, and the target image can also be a frame-level image, and there is no restriction on the type of the target image. For example, if the target image is a patch-level image, the code stream format can be ABCABC, and the first group ABC represents the code streams of multiple permission levels corresponding to the first patch-level image, and the second group ABC represents the code streams of multiple permission levels corresponding to the second patch-level image. For another example, if the target image is a frame-level image, the code stream format can be AAABBBCCC, and the first group AAA represents the code stream of the first permission level corresponding to all frame-level images, the second group BBB represents the code stream of the second permission level corresponding to all frame-level images, and the third group CCC represents the code stream of the third permission level corresponding to all frame-level images. Of course, the above are just examples of code stream formats. For example, the code stream format of patch-level images can also be AAABBBCCC, and the code stream format of frame-level images can also be ABCABC, and there is no restriction on this.

[0125] For a target image, the target image includes at least one CU, where the CU may also be referred to as an image block, or simply referred to as a block, ie, a CU block, and a current CU may also be referred to as a current block (current image block).

[0126] Then, the decoding end needs to decode the CU partition information and CU permission information of the target image based on the code streams of multiple permission levels. Corresponding to the processing process of the decoding end, the encoding end needs to encode the CU partition information and CU permission information of the target image in the code streams of multiple permission levels. For example, the encoding end encodes the CU partition information and CU permission information of the target image in the code stream of the lowest permission level (i.e., the first permission level), and the decoding end decodes the CU partition information and CU permission information of the target image from the code stream of the lowest permission level (i.e., the first permission level). Of course, the above is just an example of decoding the CU partition information and CU permission information. For the decoding method of this CU partition information and CU permission information, please refer to the subsequent embodiments.

[0127] Among them, the CU division information represents the position information of each CU in the target image. For each CU, the CU position information represents the starting position and length of the CU in the bitstream. Based on the CU position information, the reconstruction information of the CU can be parsed from the bitstream without any restriction.

[0128] The CU permission information represents the permission information of each CU in the target image. For each CU, the permission information represents the permission level of the CU. For example, if there are K permission levels in total, the permission levels of the CU may be permission level 1, permission level 2, ..., permission level K. A CU of permission level 1 is a CU of the first permission level, i.e., the lowest permission level. CUs of permission level 2, ..., permission level K are CUs of the second permission level, i.e., CUs of non-lowest permission levels.

[0129] Regarding the permission level of each CU, the encoder obtains the permission level of each CU without any restriction on the acquisition method, and encodes the permission level of each CU in the target image, that is, the CU permission information, in the bitstream.

[0130] In addition to the CU partition information and CU permission information, the decoding end can also decode the filtering information of each CU from the bitstream. The filtering information is used to filter the CU, and there is no restriction on this filtering information.

[0131] The decoder then performs decoding at the CU level. For the current CU (i.e., the CU currently to be decoded), the decoder determines the current permission level corresponding to the current CU based on the CU permission information. For example, since the CU permission information indicates the permission level of each CU within the target image, the decoder can determine the current permission level corresponding to the current CU based on the CU permission information, such as permission level 1 or permission level 2.

[0132] Then, based on the current permission level corresponding to the current CU, the decoding end determines whether the target permission level has access rights to the current CU based on the current permission level. For example, if the target permission level is greater than or equal to the current permission level, it is determined that the target permission level has access rights to the current CU. If the target permission level is less than the current permission level, the target permission level does not have access rights to the current CU.

[0133] The target permission level is the permission level of the user accessing the decoder. When permission protection is enabled, the decoder can determine the target permission level. For example, if user A is currently viewing the image reconstructed by the decoder, user A's permission level is used as the target permission level. If user B is currently viewing the image reconstructed by the decoder, user B's permission level is used as the target permission level, and so on.

[0134] Then, if the target permission level has access rights to the current CU, the decoding end decodes the high-dimensional reconstruction information corresponding to the current CU from the target bitstream based on the CU partition information, and determines the reconstructed image block corresponding to the current CU based on the high-dimensional reconstruction information. The target bitstream is a bitstream whose permission level is less than or equal to the target permission level.

[0135] Among them, the decoding end can obtain the code stream of each permission level. The code stream of the lowest permission level can be an unencrypted stream (the code stream of the lowest permission level can also be an encrypted stream, taking the unencrypted stream as an example), and the code stream of non-lowest permission levels can be an encrypted stream. For example, the encoding end encrypts the code stream of non-lowest permission levels, but the encoding end does not encrypt the code stream of the lowest permission level.

[0136] When the permission level of a code stream is less than or equal to the target permission level, the decoding end can parse the code stream when a user of the target permission level accesses the decoding end. For example, if the code stream of the lowest permission level is unencrypted, the decoding end can directly parse the code stream of the lowest permission level. For another example, although the code stream of a non-lowest permission level is encrypted, when the permission level of the code stream is less than or equal to the target permission level, the user of the target permission level has the decryption information. The decoding end can successfully decrypt the code stream based on the decryption information and then parse the decrypted code stream.

[0137] When the permission level of the code stream is greater than the target permission level, the code stream is encrypted. Although the user of the target permission level has the decryption information, the decoding end cannot successfully decrypt the code stream based on the decryption information, resulting in the inability to parse the code stream. It can be understood that the code stream cannot be obtained.

[0138] In summary, a bitstream with a permission level less than or equal to the target permission level can be called a target bitstream. The decoding end can parse the target bitstream, which is a bitstream of at least one permission level. For example, if permission level 1 is the lowest permission level, and the target permission level is permission level 1, then the target bitstream is a bitstream of permission level 1. If the target permission level is permission level 2, then the target bitstream is a bitstream of permission level 1 and a bitstream of permission level 2. If the target permission level is permission level 3, then the target bitstream is a bitstream of permission level 1, a bitstream of permission level 2, and a bitstream of permission level 3, and so on.

[0139] When the decoder decodes the high-dimensional reconstruction information corresponding to the current CU from the target bitstream, it can select a candidate bitstream corresponding to the permission level corresponding to the current CU from the target bitstream, decode the high-dimensional reconstruction information corresponding to the current CU from the candidate bitstream based on the CU partition information, and skip decoding the high-dimensional reconstruction information corresponding to the current CU from target bitstreams of other permission levels. Corresponding to the processing process at the decoder, for each CU in the target image, the encoder determines a candidate bitstream corresponding to the permission level corresponding to the CU, encodes the high-dimensional reconstruction information corresponding to the CU in the candidate bitstream based on the CU partition information, and skips encoding the high-dimensional reconstruction information corresponding to the CU in bitstreams of other permission levels, that is, does not encode the high-dimensional reconstruction information.

[0140] For example, if the permission level corresponding to the current CU is permission level 1, the encoder only encodes the high-dimensional reconstruction information corresponding to the current CU in the permission level 1 bitstream (i.e., the candidate bitstream), and does not encode the high-dimensional reconstruction information corresponding to the current CU in the bitstreams of other permission levels (such as permission level 2, permission level 3, etc.). Based on this, assuming that the target permission level is permission level 2, the target bitstream is the permission level 1 bitstream and the permission level 2 bitstream. The decoder decodes the high-dimensional reconstruction information corresponding to the current CU from the permission level 1 bitstream and does not decode the high-dimensional reconstruction information corresponding to the current CU from the permission level 2 bitstream.

[0141] For another example, if the permission level corresponding to the current CU is permission level 2, the encoder encodes the high-dimensional reconstruction information corresponding to the current CU only in the permission level 2 bitstream (i.e., the candidate bitstream), and does not encode the high-dimensional reconstruction information corresponding to the current CU in bitstreams of other permission levels (such as permission level 1, permission level 3, etc.). Based on this, assuming that the target permission level is permission level 2, the target bitstream is a permission level 2 bitstream, and the decoder decodes the high-dimensional reconstruction information corresponding to the current CU from the permission level 2 bitstream.

[0142] Among them, when the decoding end decodes the high-dimensional reconstruction information corresponding to the current CU, since the CU partition information includes the position information of the current CU, the position information indicates the starting position and length of the current CU in the bitstream. Therefore, the high-dimensional reconstruction information corresponding to the current CU can be parsed from the bitstream based on the CU partition information.

[0143] Among them, after obtaining the high-dimensional reconstruction information corresponding to the current CU, the reconstructed image block corresponding to the current CU can be determined based on the high-dimensional reconstruction information. For example, the high-dimensional reconstruction information can also be called open reconstruction information. The high-dimensional reconstruction information includes the reconstruction values ​​of all pixels of the current CU, and the image content can be completely restored through the high-dimensional reconstruction information. For example, for the current CU of the local area of ​​the human body, license plate, screen, text and other areas, if the current CU is reconstructed based on the high-dimensional reconstruction information, the real picture of the local area of ​​the human body, license plate, screen, text and other areas can be viewed, thereby obtaining the reconstructed image block of the real picture.

[0144] Then, if the target permission level does not have access rights to the current CU, the decoding end skips decoding the reconstruction information corresponding to the current CU from the target bitstream. Furthermore, the decoding end may obtain the low-dimensional reconstruction information corresponding to the current CU and determine the reconstructed image block corresponding to the current CU based on the low-dimensional reconstruction information. The low-dimensional reconstruction information may be a fixed reconstruction value, or a reconstruction value obtained based on intra-frame prediction, or a reconstruction value obtained based on inter-frame prediction.

[0145] Among them, if the target permission level does not have access rights to the current CU, although the decoding end can also parse the target bitstream, the target bitstream will not have the high-dimensional reconstruction information corresponding to the current CU. Therefore, the decoding end needs to skip decoding the reconstruction information corresponding to the current CU from the target bitstream.

[0146] For example, if the permission level corresponding to the current CU is permission level 2, the encoder only encodes the high-dimensional reconstruction information corresponding to the current CU in the bitstream of permission level 2, and does not encode the high-dimensional reconstruction information corresponding to the current CU in the bitstreams of other permission levels (such as permission level 1, permission level 3, etc.). Based on this, assuming that the target permission level is permission level 1 (permission level 1 does not have access to the current CU), the target bitstream is a bitstream of permission level 1. The high-dimensional reconstruction information corresponding to the current CU does not exist in the bitstream of permission level 1. Therefore, the decoder skips decoding the reconstruction information corresponding to the current CU from the target bitstream.

[0147] For another example, if the permission level corresponding to the current CU is permission level 3, the encoder only encodes the high-dimensional reconstruction information corresponding to the current CU in the bitstream of permission level 3. Assuming that the target permission level is permission level 2, the target bitstream is the bitstream of permission level 1 and the bitstream of permission level 2. The high-dimensional reconstruction information corresponding to the current CU does not exist in the target bitstream, and decoding of the reconstruction information corresponding to the current CU from the target bitstream is skipped.

[0148] Among them, if the target permission level does not have access rights to the current CU, the decoding end can obtain the low-dimensional reconstruction information corresponding to the current CU. After obtaining the low-dimensional reconstruction information, the reconstructed image block corresponding to the current CU can be determined based on the low-dimensional reconstruction information. The low-dimensional reconstruction information can also be called hidden reconstruction information. The low-dimensional reconstruction information does not include the reconstruction values ​​of all pixels. It can only include the reconstruction values ​​of some pixels (the decoding end only parses the reconstruction values ​​of these pixels, and the remaining pixels can use the default values). It can also include only one pixel value (that is, all pixels are this pixel value, such as 255, 0, etc.), or it can include several pixel values ​​(for example, the average value of some pixels, such as 137, 245, etc.), that is, the reconstruction value of the pixel is hidden in the low-dimensional reconstruction information, and the image content cannot be fully restored through the low-dimensional reconstruction information. For example, for the current CU of the local area of ​​the human body, license plate, screen, text and other areas, if the current CU is reconstructed based on the low-dimensional reconstruction information, then the real picture of the local area of ​​the human body, license plate, screen, text and other areas cannot be viewed.

[0149] Among them, the low-dimensional reconstruction information can be a fixed reconstruction value, for example, the fixed reconstruction value is used as the reconstruction value of each pixel in the current CU. Alternatively, the low-dimensional reconstruction information can be a reconstruction value obtained based on intra-frame prediction. For example, the reconstruction value of each pixel in the current CU can be obtained using an intra-frame prediction mode, and there is no restriction on this intra-frame prediction process. Alternatively, the low-dimensional reconstruction information can be a reconstruction value obtained based on inter-frame prediction. For example, the reconstruction value of each pixel in the current CU can be obtained using an inter-frame prediction mode, and there is no restriction on this inter-frame prediction process. Of course, the above are just a few examples and there is no restriction on this.

[0150] In one possible implementation, a multi-authority-zone encoding and decoding solution is designed for the rights-protected encoding and decoding process. This solution ensures that users with lower privilege levels on the decoding end cannot correctly decode streams with higher privilege levels, while users with higher privilege levels can correctly decode streams with higher privilege levels and streams with privilege levels less than or equal to their own. To achieve this, the encoding process may include:

[0151] Each region is labeled as a CU, and the permission levels of all CUs are recorded. Based on the patch stream packaging and transmission, the information of regions with different permission levels within the patch is packaged into different streams. For encoding within the patch, the reference of regions with lower permission levels to regions with higher permission levels is restricted. At the same time, different permission levels are encoded using different entropy encoders to generate different permission level bitstreams.

[0152] Take two permission levels as an example, these two permission levels are high permission level and low permission level. Under high permission level and low permission level, the patch image permission protection encoding process can be found in Figure 4A shown.

[0153] The patch image is divided into blocks to obtain multiple CU blocks. The permission level corresponding to each CU block is determined based on the high-privilege area in the patch. The permission level corresponding to the CU block can be a high permission level or a low permission level. For example, if the CU block is in the high-privilege area of ​​the patch, the permission level corresponding to the CU block can be a high permission level (which can be simply referred to as high permission). If the CU block is not in the high-privilege area of ​​the patch, the permission level corresponding to the CU block can be a low permission level (which can be simply referred to as low permission).

[0154] For each CU block, if the permission level corresponding to the CU block is high, the CU block is predicted, transformed, quantized, and entropy coded (using entropy encoder 1 for entropy coding) to obtain a high-privilege bitstream. For each CU block, if the permission level corresponding to the CU block is low, the CU block is restrictedly predicted, transformed, quantized, and entropy coded (using entropy encoder 2 for entropy coding) to obtain a low-privilege bitstream.

[0155] In one possible implementation, different users have different permission levels at the decoding end. The decoding end can only parse bitstreams within the permission level range and cannot parse bitstreams outside the permission level range. The decoding end obtains bitstreams of different permission levels. If the permission level of the bitstream cannot be parsed, the bitstream parsing is skipped. Otherwise, the bitstream is parsed. To achieve the above function, the processing process of the decoding end may include:

[0156] Parse the sequence header parameters and image header parameters from the bitstream of the lowest permission level. The sequence header parameters contain a flag indicating whether permission protection is enabled for the current sequence, and the image header parameters contain a flag indicating whether permission protection is enabled for the current image. Then, parse the bitstream patch by patch. If the sequence header or image header indicates that permission protection is not enabled, the NAL unit packaged by each patch bitstream contains the parameters of the entire patch. Otherwise, the NAL unit packaged by each patch bitstream only contains the parameters of a certain permission level of the entire patch image. The reconstructed image is obtained based on the parameters parsed for each NAL unit. Take two permission levels as an example for explanation. These two permission levels are high permission level and low permission level. Under high permission level and low permission level, the patch image permission protection decoding process can be found in Figure 4B and Figure 4C shown.

[0157] See also Figure 4B The figure shows the patch image permission protection process for users with a high permission level. The low permission code stream is entropy decoded (using entropy decoder 2 for entropy decoding) to obtain CU partitioning and permissions (i.e., CU partitioning information and CU permission information). For the current CU, if it is determined based on the CU permission information that the current CU is of a low permission level (referred to as low permission), the low permission code stream is entropy decoded, restricted prediction, inverse transform and dequantization are performed to obtain reconstruction information, and the patch image is generated based on the reconstruction information.

[0158] For the current CU, if it is determined based on the CU permission information that the current CU is of a high permission level (referred to as high permission), the high permission code stream is entropy decoded (such as entropy decoding using entropy decoder 1), predicted, inversely transformed and inversely quantized to obtain reconstruction information, and a patch image is generated based on the reconstruction information.

[0159] See also Figure 4CThe figure shows the patch image permission protection process for users with low permission levels. The low permission code stream is entropy decoded (using entropy decoder 2 for entropy decoding) to obtain CU partitioning and permissions (i.e., CU partitioning information and CU permission information). For the current CU, if it is determined based on the CU permission information that the current CU is of low permission level (referred to as low permission), the low permission code stream is entropy decoded, restricted prediction, inverse transform and dequantization are performed to obtain reconstruction information, and the patch image is generated based on the reconstruction information.

[0160] For the current CU, if it is determined based on the CU permission information that the current CU is of high permission level (referred to as high permission), the reconstruction information corresponding to the current CU is obtained, and the reconstruction information is a fixed reconstruction value, or a reconstruction value obtained based on intra-frame prediction, or a reconstruction value obtained based on inter-frame prediction. Figure 4C In the example above, a fixed reconstruction value is used to obtain reconstruction information, and a patch image is generated based on the reconstruction information.

[0161] Example 3: In Example 1 and Example 2, it involves enabling permission protection for the target image. As to whether permission protection (also referred to as information protection) is enabled, a flag bit can be transmitted at the sequence level, sequence-level extended data, image level, image-level extended data, Slice level, Tile level, Patch level, etc., and the flag bit is used to indicate whether permission protection is enabled. If the flag bit is the first value, permission protection is allowed to be enabled. If the flag bit is the second value, permission protection is not allowed to be enabled. Among them, the syntax used to manage or control several images can be called sequence level or sequence-level extended data; the syntax used to manage or control a single-frame image can be called image level or image-level extended data; the syntax used to manage or control a parallel unit (i.e., an image area) including several coding units can be called Slice level, Tile level, or Patch level.

[0162] Exemplarily, if the switch control information allows permission protection to be enabled, permission protection is determined to be enabled. If the switch control information does not allow permission protection to be enabled, permission protection is determined to be prohibited. For example, the switch control information may include, but is not limited to: switch control information at the sequence level or sequence-level extended data; or switch control information at the image level or image-level extended data; or switch control information at the slice level; or switch control information at the tile level; or switch control information at the patch level.

[0163] For example, if the sequence-level switch control information allows the activation of permission protection, it is determined that permission protection is activated. If the sequence-level switch control information does not allow the activation of permission protection, it is determined that the activation of permission protection is prohibited. The sequence-level switch control information may include a sequence-level flag bit. If the sequence-level flag bit has a first value, the activation of permission protection is permitted. If the sequence-level flag bit has a second value, the activation of permission protection is prohibited.

[0164] For another example, if the switch control information of the sequence-level extended data allows the activation of permission protection, it is determined that permission protection is activated. If the switch control information of the sequence-level extended data does not allow the activation of permission protection, it is determined that the activation of permission protection is prohibited. The switch control information of the sequence-level extended data may include a sequence-level extended data flag bit. If the sequence-level extended data flag bit has a first value, the activation of permission protection is permitted. If the sequence-level extended data flag bit has a second value, the activation of permission protection is prohibited.

[0165] For another example, if the image-level switch control information allows permission protection to be enabled, it is determined that permission protection is enabled. If the image-level switch control information does not allow permission protection to be enabled, it is determined that permission protection is prohibited. The image-level switch control information may include an image-level flag bit. If the image-level flag bit has a first value, permission protection is allowed to be enabled. If the image-level flag bit has a second value, permission protection is not allowed to be enabled.

[0166] For another example, if the switch control information of the image-level extended data allows the activation of rights protection, then it is determined that rights protection is enabled. If the switch control information of the image-level extended data does not allow the activation of rights protection, then it is determined that the activation of rights protection is prohibited. The switch control information of the image-level extended data may include an image-level extended data flag bit. If the image-level extended data flag bit has a first value, then the activation of rights protection is allowed. If the image-level extended data flag bit has a second value, then the activation of rights protection is not allowed.

[0167] For another example, if the slice-level switch control information allows permission protection to be enabled, it is determined that permission protection is enabled. If the slice-level switch control information does not allow permission protection to be enabled, it is determined that permission protection is prohibited from being enabled. The slice-level switch control information may include a slice-level flag bit. If the slice-level flag bit is a first value, permission protection is allowed to be enabled; if the slice-level flag bit is a second value, permission protection is not allowed to be enabled.

[0168] For another example, if the tile-level switch control information allows permission protection to be enabled, it is determined that permission protection is enabled. If the tile-level switch control information does not allow permission protection to be enabled, it is determined that permission protection is prohibited. The tile-level switch control information may include a tile-level flag bit. If the tile-level flag bit is a first value, permission protection is allowed to be enabled. If the tile-level flag bit is a second value, permission protection is not allowed to be enabled.

[0169] For another example, if the patch-level switch control information allows permission protection to be enabled, it is determined that permission protection is enabled. If the patch-level switch control information does not allow permission protection to be enabled, it is determined that permission protection is prohibited. The patch-level switch control information may include a patch-level flag bit. If the patch-level flag bit has a first value, permission protection is allowed to be enabled. If the patch-level flag bit has a second value, permission protection is not allowed to be enabled.

[0170] Exemplarily, the switch control information of the sequence level / sequence level extended data allows the current sequence to enable permission protection, that is, permission protection is enabled for all patch-level images or frame-level images (i.e., Slice-level images) in the current sequence. The switch control information of the image level / image-level extended data allows the current image to enable permission protection, that is, permission protection is enabled for all patch-level images or frame-level images in the current image. The switch control information of the slice level allows the current slice to enable permission protection, that is, permission protection is enabled for the current frame-level image (i.e., Slice-level image). The switch control information of the tile level allows the current tile to enable permission protection, that is, permission protection is enabled for all patch-level images in the current tile. The switch control information of the patch level allows the current patch to enable permission protection, that is, permission protection is enabled for the current patch-level image.

[0171] Embodiment 4: In Embodiments 1 and 2, the encoder may encode the CU partition information and CU permission information of the target image in the bitstream, and the decoder may decode the CU partition information and CU permission information of the target image based on the bitstream. Based on this process, the location of the CU partition information and CU permission information in the bitstream may be designed, that is, the syntax elements for permission protection may be designed. This process may include the following implementation methods:

[0172] Method 1: The CU partition information and CU permission information of the target image are located in the bitstream of the lowest permission level (i.e., the bitstream of the first permission level). For example, the encoder encodes the CU partition information and CU permission information of the target image in the bitstream of the lowest permission level, and the decoder decodes the CU partition information and CU permission information from the bitstream of the lowest permission level.

[0173] For example, the CU partition information (i.e., the partition information of each CU) and the CU permission information (i.e., the permission information of each CU) can be placed only in the bitstream of the lowest permission level. Since users of all permission levels can parse the bitstream of the lowest permission level, no matter which target permission level the user adopts, the decoding end can decode the CU partition information and CU permission information from the bitstream of the lowest permission level.

[0174] For decoding-end users with higher than or equal to CU authority (i.e., the target authority level is greater than or equal to the current authority level of the current CU, and they have access rights to the current CU), the reconstruction information corresponding to the current CU can be obtained from the high-authority code stream (i.e., the high-dimensional reconstruction information corresponding to the current CU is decoded from the target code stream), and the reconstruction of the current CU is completed based on the reconstruction information. For decoding-end users with lower than CU authority (i.e., the target authority level is less than the current authority level of the current CU, and they do not have access rights to the current CU), the parsing of the current CU is skipped by default, and the reconstruction of the current CU can be completed using the low-dimensional reconstruction information corresponding to the current CU. For example, the current CU can be filled with a fixed value, or the current CU can be filled with the surrounding reconstructed pixels. For the surrounding reconstructed pixels, a prediction mode (such as intra-frame prediction mode or inter-frame prediction mode) can be obtained, and the reference pixels under the prediction mode are filled into the current CU.

[0175] For example, CU permission information is decoded from the bitstream with the lowest permission level, and the permission level of the current CU is determined to be permission level 2 based on the CU permission information. Decoding users with permission levels less than 2 (i.e., target permission level less than 2) skip parsing the current CU. Decoding users with permission levels greater than or equal to 2 (i.e., target permission level greater than or equal to 2) can parse the reconstruction information of the current CU from the bitstream with permission level 2.

[0176] Method 2: The CU partition information of the target image is located in the bitstream with the lowest permission level (i.e., the bitstream with the first permission level), and the CU permission information of the target image is located in the bitstream with a permission level less than or equal to that of the CU, that is, for the permission information of each CU, the permission information is located in the bitstream with a permission level less than or equal to that of the CU. For example, the encoder encodes the CU partition information of the target image in the bitstream with the lowest permission level, and the decoder decodes the CU partition information from the bitstream with the lowest permission level. And / or, the encoder encodes the permission information of the first CU in the bitstream with a permission level less than or equal to that of the first CU, and the decoder decodes the permission information of the first CU from the bitstream with a permission level less than or equal to that of the first CU. Wherein, the first CU is any CU in the target image.

[0177] For example, the CU partition information can be placed only in the bitstream of the lowest permission level. Since users of all permission levels can parse the bitstream of the lowest permission level, no matter which target permission level the user adopts, the decoding end can decode the CU partition information from the bitstream of the lowest permission level.

[0178] For example, the permission information of the CU can be placed in a bitstream with a permission level less than or equal to the CU, that is, it is marked in the bitstream with a permission level less than or equal to the CU whether the reconstruction information of the current CU is in the current bitstream. When parsing the CU permission, it needs to be parsed in the order of the permission level.

[0179] For example, for each CU, the permission information of the CU can be placed in a bitstream that is less than or equal to the permission level of the CU. In this way, as long as the target permission level adopted by the user is greater than or equal to the permission level of the CU, the decoding end can decode the permission information of the CU from the bitstream.

[0180] For example, each CU among all CUs can be called the first CU, the permission level A of the first CU is determined, and all permission levels less than or equal to permission level A are called permission level A'. Assuming that permission level A is permission level 1, then permission level A' is permission level 1. Assuming that permission level A is permission level 2, then permission level A' is permission level 1 and permission level 2, and so on.

[0181] The encoder can encode the permission information of the first CU in a bitstream of permission level A', and the decoder can decode the permission information of the first CU from a bitstream of permission level A' (i.e., a bitstream with a permission level lower than the permission level of the first CU). For example, if the permission level of the first CU is permission level 2, the decoder can decode the permission information of the first CU from a bitstream of permission level 1, and can decode the permission information of the first CU from a bitstream of permission level 2.

[0182] Exemplarily, when a decoding end decodes permission information of a first CU from a bitstream of permission level A', if the permission level A' is less than the permission level of the first CU, that is, for a bitstream whose permission level is less than the permission level of the first CU, the permission information of the first CU in the bitstream has a first value, and the first value indicates that the bitstream does not include high-dimensional reconstruction information corresponding to the first CU; or, if the permission level A' is equal to the permission level of the first CU, that is, for a bitstream whose permission level is equal to the permission level of the first CU, the permission information of the first CU in the bitstream has a second value, and the second value indicates that the bitstream includes high-dimensional reconstruction information corresponding to the first CU.

[0183] For example, if the permission level of the first CU is permission level 2, when the decoding end decodes the permission information of the first CU from the bitstream of permission level 1, the permission information of the first CU has a first value (e.g., 0), which indicates that the bitstream of permission level 1 does not contain the high-dimensional reconstruction information corresponding to the first CU. When the decoding end decodes the permission information of the first CU from the bitstream of permission level 2, the permission information of the first CU has a second value (e.g., 1), which indicates that the bitstream of permission level 2 contains the high-dimensional reconstruction information corresponding to the first CU.

[0184] For decoding end users with higher than or equal CU authority (i.e., the target authority level is greater than or equal to the current authority level of the current CU), the reconstruction information corresponding to the current CU can be obtained from the code stream of the CU authority (i.e., the high-dimensional reconstruction information corresponding to the current CU is decoded from the target code stream), and the reconstruction of the current CU is completed based on the reconstruction information. For decoding end users with lower than CU authority (i.e., the target authority level is less than the current authority level of the current CU), the parsing of the current CU is skipped by default, and the low-dimensional reconstruction information corresponding to the current CU can be used to complete the reconstruction of the current CU. For example, the current CU can be filled with a fixed value.

[0185] For example, if the permission level of the current CU is 2 and the lowest permission level is 0, the encoder encodes the first value (such as 0) in the bitstreams of permission level 0 and permission level 1 respectively, indicating that the current bitstream does not contain the reconstruction information of the current CU. The encoder encodes the second value (such as 1) in the bitstream of permission level 2, indicating that the current bitstream contains the reconstruction information of the current CU, and the reconstruction information corresponding to the current CU can be parsed from the current bitstream. After the decoder parses the bitstream of permission level 2, the decoding user with the target permission level greater than or equal to 2 can continue to decode the reconstruction information of the current CU from the bitstream of permission level 2.

[0186] Method 3: To increase parallelism in bitstream parsing, that is, to ensure that each permission level bitstream can be decoded independently, the CU partitioning information (i.e., the partitioning information for each CU) is placed in the bitstreams of all permission levels, and the CU permission information (i.e., the permission information for each CU) is placed in the bitstreams of all permission levels. The CU permission flag indicates whether the current bitstream contains information about the current CU. For example, the encoder encodes the CU partitioning information of the target image in the bitstream of each permission level (e.g., the first permission level and each second permission level), and the decoder decodes the CU partitioning information from the bitstream of any permission level. For example, the decoder decodes the CU partitioning information from the bitstream of the first permission level, or from the bitstream of any second permission level. The encoder encodes the CU permission information of the target image in the bitstream of each permission level (e.g., the first permission level and each second permission level), and the decoder decodes the CU permission information from the bitstream of any permission level. For example, the decoder decodes the CU permission information from the bitstream of the first permission level, or from the bitstream of any second permission level.

[0187] For example, the CU partition information can be placed in the bitstream of each permission level, and the CU permission information can be placed in the bitstream of each permission level. In this way, when a decoding end decodes a bitstream of any permission level, it can decode the CU partition information and CU permission information from the bitstream. For example, when a decoding end decodes a bitstream of permission level 1, it can decode the CU partition information and CU permission information from the bitstream of permission level 1. When a decoding end decodes a bitstream of permission level 2, it can decode the CU partition information and CU permission information from the bitstream of permission level 2, and so on.

[0188] Exemplarily, if the permission information of the second CU of the target image is located in a codestream of a first permission level and a second permission level, and the second CU is any CU, then: for a codestream having a permission level less than the permission level of the second CU, the permission information of the second CU in the codestream is a first value, where the first value is used to indicate that the codestream does not include high-dimensional reconstruction information corresponding to the second CU; or, for a codestream having a permission level greater than or equal to the permission level of the second CU, the permission information of the second CU in the codestream is a second value, where the second value is used to indicate that the codestream includes high-dimensional reconstruction information corresponding to the second CU.

[0189] For example, if the permission level of the second CU is permission level 2, assuming that there are bitstreams of permission level 1, permission level 2, and permission level 3, the permission information of the second CU is located in the bitstreams of permission level 1, permission level 2, and permission level 3. Based on this, when the decoding end decodes the permission information of the second CU from the bitstream of permission level 1, the permission information of the second CU is a first value (such as 0), and the first value indicates that the bitstream of permission level 1 does not contain the high-dimensional reconstruction information corresponding to the second CU. When the decoding end decodes the permission information of the second CU from the bitstream of permission level 2, the permission information of the second CU is a second value (such as 1), and the second value indicates that the bitstream of permission level 2 contains the high-dimensional reconstruction information corresponding to the second CU. When the decoding end decodes the permission information of the second CU from the bitstream of permission level 3, the permission information of the second CU is a second value, and the second value indicates that the bitstream of permission level 3 contains the high-dimensional reconstruction information corresponding to the second CU.

[0190] For decoding end users with higher than or equal CU authority (i.e., the target authority level is greater than or equal to the current authority level of the current CU), the reconstruction information corresponding to the current CU can be obtained from the code stream of the CU authority (i.e., the high-dimensional reconstruction information corresponding to the current CU is decoded from the target code stream), and the reconstruction of the current CU is completed based on the reconstruction information. For decoding end users with lower than CU authority (i.e., the target authority level is less than the current authority level of the current CU), the parsing of the current CU is skipped by default, and the low-dimensional reconstruction information corresponding to the current CU can be used to complete the reconstruction of the current CU. For example, the current CU can be filled with a fixed value.

[0191] For example, if the permission level of the current CU is 2 and the lowest permission level is 0, the encoder encodes the first value (such as 0) in the bitstreams of permission level 0 and permission level 1 respectively, indicating that the current bitstream does not contain the reconstruction information of the current CU. The encoder encodes the second value (such as 1) in the bitstreams of permission level 2 and permission level 3, indicating that the current bitstream contains the reconstruction information of the current CU, and the reconstruction information corresponding to the current CU can be parsed from the current bitstream. After the decoder parses the bitstream of permission level 2, the decoding user with the target permission level greater than or equal to 2 can continue to decode the reconstruction information of the current CU from the bitstream of permission level 2. When the decoder parses the bitstream of permission level 3, the decoding user with the target permission level greater than or equal to 3 can decode the bitstream of permission level 3. Although the bitstream of permission level 3 does not include the reconstruction information of the current CU, the reconstruction information of the current CU can be decoded from the bitstream of permission level 2.

[0192] Method 4: To increase parallelism in bitstream parsing, that is, to ensure that each permission level bitstream can be decoded independently, the CU partitioning information (i.e., the partitioning information for each CU) is placed in the bitstreams of all permission levels, and the CU permission information (i.e., the permission information for each CU) is placed in the bitstreams of all permission levels. The CU permission flag indicates whether the current bitstream contains information about the current CU. For example, the encoder encodes the CU partitioning information of the target image in the bitstream of each permission level (e.g., the first permission level and each second permission level), and the decoder decodes the CU partitioning information from the bitstream of any permission level. For example, the decoder decodes the CU partitioning information from the bitstream of the first permission level, or from the bitstream of any second permission level. The encoder encodes the CU permission information of the target image in the bitstream of each permission level (e.g., the first permission level and each second permission level), and the decoder decodes the CU permission information from the bitstream of any permission level. For example, the decoder decodes the CU permission information from the bitstream of the first permission level, or from the bitstream of any second permission level.

[0193] For example, the CU partition information can be placed in the bitstream of each permission level, and the CU permission information can be placed in the bitstream of each permission level. In this way, when the decoding end decodes the bitstream of any permission level, the CU partition information and CU permission information can be decoded from the bitstream.

[0194] Exemplarily, if the permission information of the second CU of the target image is located in a codestream of a first permission level and a second permission level, and the second CU is any CU, then: for a codestream whose permission level is not equal to the permission level of the second CU (such as a codestream whose permission level is greater than the permission level of the second CU, or a codestream whose permission level is less than the permission level of the second CU), the permission information of the second CU in the codestream is a first value, where the first value is used to indicate that the codestream does not include high-dimensional reconstruction information corresponding to the second CU; for a codestream whose permission level is equal to the permission level of the second CU, the permission information of the second CU in the codestream is a second value, where the second value is used to indicate that the codestream includes high-dimensional reconstruction information corresponding to the second CU.

[0195] If the permission level of the second CU is permission level 2, assuming that there are bitstreams of permission level 1, permission level 2, and permission level 3, the permission information of the second CU is located in the bitstreams of permission level 1, permission level 2, and permission level 3. Based on this, when the decoding end decodes the permission information of the second CU from the bitstream of permission level 1, the permission information of the second CU is a first value (such as 0), and the first value indicates that the bitstream of permission level 1 does not contain the high-dimensional reconstruction information corresponding to the second CU. When the decoding end decodes the permission information of the second CU from the bitstream of permission level 2, the permission information of the second CU is a second value (such as 1), and the second value indicates that the bitstream of permission level 2 contains the high-dimensional reconstruction information corresponding to the second CU. When the decoding end decodes the permission information of the second CU from the bitstream of permission level 3, the permission information of the second CU is a first value, and the first value indicates that the bitstream of permission level 3 does not contain the high-dimensional reconstruction information corresponding to the second CU.

[0196] For decoding end users with higher than or equal CU authority (i.e., the target authority level is greater than or equal to the current authority level of the current CU), the reconstruction information corresponding to the current CU can be obtained from the code stream of the CU authority (i.e., the high-dimensional reconstruction information corresponding to the current CU is decoded from the target code stream), and the reconstruction of the current CU is completed based on the reconstruction information. For decoding end users with lower than CU authority (i.e., the target authority level is less than the current authority level of the current CU), the parsing of the current CU is skipped by default, and the low-dimensional reconstruction information corresponding to the current CU can be used to complete the reconstruction of the current CU. For example, the current CU can be filled with a fixed value.

[0197] For example, if the permission level of the current CU is 2 and the lowest permission level is 0, the encoder encodes the first value in the bitstreams of permission level 0 and permission level 1 respectively, indicating that the current bitstream does not contain the reconstruction information of the current CU. The encoder encodes the second value in the bitstream of permission level 2, indicating that the current bitstream contains the reconstruction information of the current CU, and the reconstruction information corresponding to the current CU can be parsed from the current bitstream. The encoder encodes the first value in the bitstream of permission level 3, indicating that the current bitstream does not contain the reconstruction information of the current CU. When the decoder parses the bitstream of permission level 2, since the bitstream of permission level 2 contains the second value, the decoding user with the target permission level greater than or equal to 2 can continue to decode the reconstruction information of the current CU from the bitstream of permission level 2. When the decoder parses the bitstream of permission level 2, since the bitstream contains the first value, this CU is skipped during reconstruction and the parsing of the next CU is entered.

[0198] Example 5: To design rights protection syntax elements, the rights protection syntax elements are combined with region of interest syntax elements. Before obtaining the codestreams corresponding to the target image at multiple rights levels, rights protection parameters and region of interest parameters corresponding to the target image can also be decoded from the high-level syntax corresponding to the target image. The high-level syntax may include, but is not limited to, sequence-level high-level syntax, sequence-level extended data high-level syntax, picture-level high-level syntax, picture-level extended data high-level syntax, slice-level high-level syntax, tile-level high-level syntax, and patch-level high-level syntax, without limitation.

[0199] For ease of distinction, the high-level syntax used by the rights protection parameters may be referred to as a first syntax, and the high-level syntax used by the region of interest parameters may be referred to as a second syntax. The first syntax used by the rights protection parameters and the second syntax used by the region of interest parameters may be different, or the first syntax used by the rights protection parameters and the second syntax used by the region of interest parameters may be partially identical, or the first syntax used by the rights protection parameters and the second syntax used by the region of interest parameters may be completely identical.

[0200] In one possible implementation, the rights protection parameters may include, but are not limited to, at least one of the following: a sequence header switch identifier, an image header switch identifier, an enhancement layer switch identifier, permission level classification information, whether the permission level is predictable identifier, whether the probability model is updated identifier, and CU boundary filtering parameters. The region of interest parameters may include, but are not limited to, at least one of the following: a sequence header switch identifier, an image header switch identifier, an enhancement layer switch identifier, region category classification information, whether the region category is predictable identifier, whether the probability model is updated identifier, and CU boundary filtering parameters. Of course, the above are only examples of parameters and are not limiting.

[0201] Exemplarily, the syntax elements of the region of interest parameters may include: a sequence header switch flag of the region of interest, a picture header switch flag, an enhancement layer switch flag, region category classification information (such as block-level / CU-level classification information), and a flag indicating whether the region category of each CU block is predictable. On this basis, the picture header may also include some additional syntax elements, such as a flag indicating whether the probability model is updated when encoding and decoding the CU category flag, and CU boundary filter parameters (CU boundary filter parameters are used to perform different filtering processes on CU boundaries of different categories). Of course, the above are just a few examples of region of interest parameters.

[0202] Exemplary syntax elements of rights protection parameters may include: a sequence header switch flag, a picture header switch flag, an enhancement layer switch flag, permission level classification information (i.e., block-level / CU-level permissions), a flag indicating whether the permission level is predictable, etc. On this basis, additional syntax elements may be designed for rights protection, such as permission level probability model updates (i.e., whether the probability model is updated), CU boundary filtering parameters (CU boundary filtering parameters are used to perform different filtering processes on CU boundaries of different permission levels), and mosaic information of different permission levels. There are no restrictions on these rights protection parameters.

[0203] In order to combine the permission protection and the syntax elements of the region of interest, the following approach can be adopted:

[0204] Mode 1: The first syntax used by the rights protection parameter is different from the second syntax used by the region of interest parameter. For example, the difference between the first syntax and the second syntax may include at least one of the following: the sequence header switch identifier of the rights protection parameter and the sequence header switch identifier of the region of interest parameter use different syntax. The image header switch identifier of the rights protection parameter and the image header switch identifier of the region of interest parameter use different syntax. The enhancement layer switch identifier of the rights protection parameter and the enhancement layer switch identifier of the region of interest parameter use different syntax. The permission level classification information of the rights protection parameter and the region category classification information of the region of interest parameter use different syntax. The permission level predictability flag of the rights protection parameter and the region category predictability flag of the region of interest parameter use different syntax. The probability model update flag of the rights protection parameter and the probability model update flag of the region of interest parameter use different syntax. The CU boundary filter parameters of the rights protection parameter and the CU boundary filter parameters of the region of interest parameter use different syntax.

[0205] For example, the first syntax used by the rights protection parameters and the second syntax used by the region of interest parameters are independent of each other, that is, the syntax elements used by the rights protection parameters and the region of interest parameters are completely independent. At this time, when the two tools (that is, the rights protection function and the region of interest processing function) are turned on at the same time, when parsing the syntax elements of the CU, both the region of interest parameters of the CU and the rights protection parameters of the CU must be parsed.

[0206] Exemplarily, when two tools are turned on at the same time, during the parsing process, the parameters of the second parsing can refer to the parameters of the first parsing. For example, if the region of interest parameters are parsed first and the permission protection parameters are parsed later, the parsing results of the region of interest parameters can be referred to when parsing the permission protection parameters. For another example, if the permission protection parameters are parsed first and the region of interest parameters are parsed later, the parsing results of the permission protection parameters can be referred to when parsing the region of interest parameters. For example, when the region of interest category of the CU is 5 and the CU permission level is 5, at this time, parse whether the CU level is equal to the CU category. If so, directly equate the CU level to the CU category. Otherwise, parse the CU permission, or parse the difference between the CU permission and the category permission.

[0207] Mode 2: The first syntax used by the rights protection parameter and the second syntax used by the region of interest parameter are partially identical. For example, the first syntax and the second syntax being partially identical may include, but are not limited to, at least one of the following: the sequence header switch identifier of the rights protection parameter and the sequence header switch identifier of the region of interest parameter use different syntaxes, or the sequence header switch identifier of the rights protection parameter and the sequence header switch identifier of the region of interest parameter use the same syntax. The image header switch identifier of the rights protection parameter and the image header switch identifier of the region of interest parameter use different syntaxes, or the image header switch identifier of the rights protection parameter and the image header switch identifier of the region of interest parameter use the same syntax. The enhancement layer switch identifier of the rights protection parameter and the enhancement layer switch identifier of the region of interest parameter use different syntaxes, or the enhancement layer switch identifier of the rights protection parameter and the enhancement layer switch identifier of the region of interest parameter use the same syntax. The permission level classification information of the rights protection parameter and the region category classification information of the region of interest parameter use different syntaxes, or the permission level classification information of the rights protection parameter and the region category classification information of the region of interest parameter use the same syntax. The whether permission level predictable flag of the rights protection parameter and the whether region category predictable flag of the region of interest parameter use different syntax, or the whether permission level predictable flag of the rights protection parameter and the whether region category predictable flag of the region of interest parameter use the same syntax. The whether probability model update flag of the rights protection parameter and the whether probability model update flag of the region of interest parameter use different syntax, or the whether probability model update flag of the rights protection parameter and the whether probability model update flag of the region of interest parameter use the same syntax. The CU boundary filtering parameters of the rights protection parameter and the CU boundary filtering parameters of the region of interest parameters use different syntax, or the CU boundary filtering parameters of the rights protection parameter and the CU boundary filtering parameters of the region of interest parameters use the same syntax.

[0208] In a possible embodiment, the first grammar and the second grammar being partially identical may include, but are not limited to, at least one of the following: the sequence header switch identifier of the rights protection parameter and the sequence header switch identifier of the region of interest parameter use different syntax; the image header switch identifier of the rights protection parameter and the image header switch identifier of the region of interest parameter use different syntax; the enhancement layer switch identifier of the rights protection parameter and the enhancement layer switch identifier of the region of interest parameter use the same syntax; the permission level classification information of the rights protection parameter and the region category classification information of the region of interest parameter use the same syntax; the permission level predictability identifier of the rights protection parameter and the region category predictability identifier of the region of interest parameter use the same syntax; the probability model update identifier of the rights protection parameter and the probability model update identifier of the region of interest parameter use the same syntax; the CU boundary filter parameters of the rights protection parameter and the CU boundary filter parameters of the region of interest parameter use the same syntax. Of course, the above are only examples and are not limiting.

[0209] For example, the first syntax used by the rights protection parameter and the second syntax used by the region of interest parameter share an independent part, and the independent syntax elements may include a sequence header switch and an image header switch. The remaining syntax elements except the independent syntax elements may be shared syntax elements. At this time, when the two tools (i.e., the rights protection function and the region of interest processing function) are turned on at the same time, when parsing the syntax elements of the CU, for the independent syntax elements, both the region of interest parameters of the CU and the rights protection parameters of the CU must be parsed. For the shared syntax elements, only one set of parameters needs to be parsed, based on which the region of interest parameters of the CU and the rights protection parameters of the CU can be obtained.

[0210] Method 3: The first syntax adopted by the right protection parameter is exactly the same as the second syntax adopted by the region of interest parameter. The first syntax and the second syntax being exactly the same include but are not limited to at least one of the following: the sequence header switch identifier of the right protection parameter and the sequence header switch identifier of the region of interest parameter adopt the same syntax; the image header switch identifier of the right protection parameter and the image header switch identifier of the region of interest parameter adopt the same syntax; the enhancement layer switch identifier of the right protection parameter and the enhancement layer switch identifier of the region of interest parameter adopt the same syntax; the permission level classification information of the right protection parameter and the region category classification information of the region of interest parameter adopt the same syntax; the permission level predictability identifier of the right protection parameter and the region category predictability identifier of the region of interest parameter adopt the same syntax; the probability model update whether identifier of the right protection parameter and the probability model update whether identifier of the region of interest parameter adopt the same syntax; the CU boundary filter parameters of the right protection parameter and the CU boundary filter parameters of the region of interest parameter adopt the same syntax.

[0211] For example, the first syntax used by the rights protection parameter is exactly the same as the second syntax used by the region of interest parameter. When both tools (i.e., the rights protection function and the region of interest processing function) are enabled at the same time, only one set of parameters needs to be parsed when parsing the syntax elements of the CU. Based on this set of parameters, both the region of interest parameters and the rights protection parameters of the CU can be obtained. For example, the rights protection parameters and the region of interest parameters are completely shared, that is, the two tools are strongly bound together and can be enabled and disabled together.

[0212] In Methods 2 and 3, if the sequence header switch flag of the rights protection parameter and the sequence header switch flag of the region of interest parameter use the same syntax, when the syntax is a first value, the sequence-level switch control information is determined to allow the activation of rights protection and the activation of region of interest processing. When the syntax is a second value, the sequence-level switch control information is determined to prohibit the activation of rights protection and the activation of region of interest processing. The first value and the second value can be arbitrarily configured, such as the first value being 1 and the second value being 0.

[0213] In Methods 2 and 3, if the image header switch flag of the rights protection parameter and the image header switch flag of the region of interest parameter use the same syntax, when the syntax is a first value, it is determined that the image-level switch control information allows the activation of rights protection and the activation of region of interest processing. When the syntax is a second value, it is determined that the image-level switch control information prohibits the activation of rights protection and the activation of region of interest processing. The first value and the second value can be arbitrarily configured, such as the first value being 1 and the second value being 0.

[0214] In Methods 2 and 3, if the enhancement layer switch flag of the rights protection parameter and the enhancement layer switch flag of the region of interest parameter use the same syntax, when the syntax is a first value, it is determined that rights protection and region of interest processing are enabled for the enhancement layer, and when the syntax is a second value, it is determined that rights protection and region of interest processing are disabled for the enhancement layer. For example, the first value can be 1, and the second value can be 0.

[0215] In Methods 2 and 3, if the permission level predictability flag of the permission protection parameter and the region category predictability flag of the region of interest parameter use the same syntax, when the syntax takes a first value, the permission level is determined to be predictable based on the permission level of the previous CU, and the region category is determined to be predictable based on the region category of the previous CU. When the syntax takes a second value, the permission level and region category are determined to be unpredictable, that is, the permission level cannot be predicted based on the permission level of the previous CU, and the region category cannot be predicted based on the region category of the previous CU. For example, the first value can be 1, and the second value can be 0.

[0216] In Methods 2 and 3, if the flag indicating whether to update the probability model of the rights protection parameter and the flag indicating whether to update the probability model of the region of interest parameter use the same syntax, when the syntax takes a first value, the probability model in the rights protection process and the probability model in the region of interest processing process are determined to be updated, and when the syntax takes a second value, the probability model in the rights protection process and the probability model in the region of interest processing process are determined not to be updated. For example, the first value may be 1, and the second value may be 0.

[0217] In Method 2 and Method 3, if the permission level classification information of the permission protection parameter and the area category classification information of the area of ​​interest parameter use the same syntax, the classification identifier is parsed from the syntax, and the permission level and area category are determined based on the classification identifier; or, the classification identifier is parsed from the syntax, the permission level is determined based on the classification identifier, and the area category is determined based on the permission level and the mapping relationship; or, the classification identifier is parsed from the syntax, the area category is determined based on the classification identifier, and the permission level is determined based on the area category and the mapping relationship; wherein the mapping relationship represents the mapping relationship between the permission level and the area category.

[0218] For example, if block-level CU permission parsing and block-level (CU-level) classification use the same syntax elements, that is, the permission level classification information of the permission protection parameter and the region category classification information of the region of interest parameter use the same syntax, then if the number of permission levels is the same as the number of region categories, the classification identifier is parsed from the syntax, and the permission level and region category are determined based on the classification identifier. For example, if the classification identifier is 0, the permission level is the first permission level and the region category is the first region category. If the classification identifier is 1, the permission level is the second permission level and the region category is the second region category, and so on.

[0219] If the number of permission levels is different from the number of area categories, a mapping relationship can also be obtained, which represents the mapping relationship between permission levels and area categories. For example, if the number of area categories is greater than the number of permission levels, the mapping relationship can represent mapping area categories to permission levels. For example, there are 8 area categories [0, 1, 2, 3, 4, 5, 6, 7], and different area categories represent different degrees of interest areas. If the number of permission levels is 2, the mapping relationship is [0, 0, 0, 0, 1, 1, 1, 1], indicating that area categories 0, 1, 2, 3 correspond to permission level 0, and area categories 4, 5, 6, 7 correspond to permission level 1. If the number of permission levels is greater than the number of area categories, the mapping relationship can represent mapping permission levels to area categories. For example, there are 8 permission levels [0, 1, 2, 3, 4, 5, 6, 7]. If the number of area categories is 2, the mapping relationship is [0, 0, 0, 0, 1, 1, 1, 1], which means that permission levels 0, 1, 2, 3 correspond to area category 0, and permission levels 4, 5, 6, 7 correspond to area category 1.

[0220] If the number of area categories is greater than the number of permission levels, the classification identifier is parsed from the syntax, the area category is determined based on the classification identifier, and the permission level is determined based on the area category and the mapping relationship. For example, if the classification identifier is 0, the area category is the first area category 0. The mapping relationship is queried through area category 0 to obtain permission level 0. If the classification identifier is 5, the area category is the fifth area category 4. The mapping relationship is queried through area category 4 to obtain permission level 1, and so on.

[0221] If the number of permission levels is greater than the number of area categories, the classification identifier is parsed from the syntax, the permission level is determined based on the classification identifier, and the area category is determined based on the permission level and the mapping relationship. For example, if the classification identifier is 0, the permission level is the first permission level 0. By querying the mapping relationship through permission level 0, the area category is 0. If the classification identifier is 5, the permission level is the fifth permission level 4. By querying the mapping relationship through permission level 4, the area category is 1, and so on.

[0222] This mapping relationship can be obtained by the decoder from the current syntax, from other syntaxes, or pre-configured at the decoder. There is no restriction on the source of this mapping relationship. For example, if the number of permission levels is not equal to the number of region categories, a syntax element can be added to the image header to mark the mapping relationship between permission levels and region categories. After the region category (or permission level) is parsed and obtained, the permission level (or region category) is obtained based on the mapping relationship.

[0223] In Method 2 and Method 3, if the CU boundary filter parameters of the permission protection parameters and the CU boundary filter parameters of the region of interest parameters use the same syntax, the candidate filter parameters are parsed from the syntax, and the target filter parameters for permission protection and the target filter parameters for the region of interest are determined based on the candidate filter parameters; or, the first candidate filter parameters are determined based on the permission level and the obtained first mapping relationship, and the second candidate filter parameters are determined based on the region category and the obtained second mapping relationship, the target filter parameters for permission protection are determined based on the first candidate filter parameters, and the target filter parameters for the region of interest are determined based on the second candidate filter parameters; wherein the first mapping relationship includes a mapping relationship between the permission level and the filter parameters, and the second mapping relationship includes a mapping relationship between the region category and the filter parameters.

[0224] For example, during filtering, the same filtering process is used, and a syntax element of a mapping relationship is added to the image header to indicate the mapping relationship between the permission level and the region category filtering process. During filtering, two sets of filtering parameters may be derived, and one of them may be selected for use, or the weighted combination of the two parameters may be used as the final filtering parameter.

[0225] For example, the decoding end can directly parse candidate filter parameters from the syntax, and the decoding end can also determine reference filter parameters based on the image content of the current CU, without any restrictions on this process. The rights-protected target filter parameters and the target filter parameters of the region of interest can be determined based on the candidate filter parameters and the reference filter parameters, the rights-protected area to be filtered can be filtered based on the rights-protected target filter parameters, and the area to be filtered processed by the region of interest can be filtered based on the target filter parameters of the region of interest.

[0226] For another example, the decoding end may obtain the first mapping relationship and the second mapping relationship by decoding the first mapping relationship from the current syntax, decoding the first mapping relationship from another syntax, or pre-configured at the decoding end. There is no restriction on the source of the first mapping relationship and the second mapping relationship. For example, a syntax element for the mapping relationship may be added to the picture header, and the first mapping relationship and the second mapping relationship may be decoded from the syntax element.

[0227] The first mapping relationship may include a mapping relationship between permission levels and filtering parameters. After obtaining the permission level for permission protection, the decoding end may query the first mapping relationship based on the permission level to obtain the first candidate filtering parameters for permission protection. The second mapping relationship may include a mapping relationship between region categories and filtering parameters. After obtaining the region category of the region of interest, the decoding end may query the second mapping relationship based on the region category to obtain the second candidate filtering parameters for the region of interest processing.

[0228] The decoding end can also determine the reference filtering parameters based on the image content of the current CU. The decoding end can determine the target filtering parameters for rights protection based on the first candidate filtering parameters and the reference filtering parameters, or the decoding end can perform operations (such as weighted operations) on the first candidate filtering parameters and the second candidate filtering parameters to obtain weighted candidate filtering parameters, and determine the target filtering parameters for rights protection based on the weighted candidate filtering parameters and the reference filtering parameters. Then, the rights-protected area to be filtered is filtered based on the target filtering parameters for rights protection. In addition, the decoding end can determine the target filtering parameters for the area of ​​interest based on the second candidate filtering parameters and the reference filtering parameters, or the decoding end can perform operations (such as weighted operations) on the first candidate filtering parameters and the second candidate filtering parameters to obtain weighted candidate filtering parameters, and determine the target filtering parameters for the area of ​​interest based on the weighted candidate filtering parameters and the reference filtering parameters. Then, the area to be filtered processed by the area of ​​interest is filtered based on the target filtering parameters of the area of ​​interest.

[0229] In one possible implementation, for Mode 1, Mode 2, and Mode 3, during the decoding process of the current CU, if the current CU belongs to both the region of interest and the high-authority region, the reconstruction information of the current CU is parsed from the bitstream of the high-authority level. If the current CU only belongs to the region of interest but not the high-authority region, during the reconstruction process, the current CU is restricted from referencing other CUs. If the current CU does not belong to the region of interest but belongs to the high-authority region, if the current CU chooses to skip parsing at this time, then, if the current CU is intra-predicted, the predicted pixels are obtained from the current frame based on the prediction mode in the high-authority bitstream. If the current CU is inter-predicted, the prediction mode (such as the default prediction mode or the prediction mode obtained by some method) is used to obtain the predicted pixels from the reference frame, and the predicted pixels are used as the reconstructed pixels.

[0230] Example 6: In Example 1 and Example 2, the encoding end sends the code streams of multiple permission levels corresponding to the target image to the decoding end, and the decoding end obtains the code streams of multiple permission levels corresponding to the target image. The packaging method of the code streams of different permission levels can be based on patch-level packaging. For example, if there are K permission levels, K is a positive integer greater than 1, and the code streams of multiple permission levels corresponding to the target image (such as a patch-level image) include the RBSP code streams of the K permission levels corresponding to the target image, and the RBSP code streams of the K permission levels are located in K NAL units, and the RBSP code streams of different permission levels are located in different NAL units. For example, when the frame-level permission protection switch is turned on, each patch-level image corresponds to at least two RBSP code stream packets, and the at least two RBSP code stream packets include a low-privilege-level RBSP code stream packet and at least one high-privilege-level RBSP code stream packet. For example, assuming that K permission levels are 3 permission levels, each patch-level image corresponds to 3 permission level RBSP codestream packets. These 3 permission level RBSP codestream packets are located in 3 NAL units. For example, the RBSP codestream packet of permission level 1 is located in NAL unit 1, the RBSP codestream packet of permission level 2 is located in NAL unit 2, and the RBSP codestream packet of permission level 3 is located in NAL unit 3. The information in the RBSP codestream packet can be specified as at least one of the following:

[0231] The RBSP code stream of a high permission level includes the partitioning information of each CU (such as position, shape, etc.).

[0232] Each RBSP codestream only contains codestreams with the same permissions in one patch.

[0233] The RBSP code stream of the low permission level contains the division information of each CU (such as position, shape, etc.), while the RBSP code stream of the high permission level does not contain the division information of the CU. In this case, before parsing the RBSP code stream of the high permission level, the RBSP code stream of the low permission level is parsed to obtain the division information of each CU.

[0234] The patch header information is included only in the RBSP code stream with the lowest permission level. The header information content can include the CU partition information and filtering information of the entire patch. There is no restriction on the content of this header information.

[0235] The high-privilege-level RBSP codestream and the low-privilege-level RBSP codestream obtained based on the patch image are placed in different NAL units. When the low-privilege-level RBSP codestream and the high-privilege-level RBSP codestream are interleaved, the high-privilege-level RBSP codestream can be transmitted first, or the low-privilege-level RBSP codestream can be transmitted first. Therefore, the interleaving order can be: privilege 0 NAL -> privilege 1 NAL -> privilege 2 NAL -> ... -> privilege n NAL, or the interleaving order can be: privilege n NAL -> privilege n-1 NAL -> ... -> privilege 1 NAL -> privilege 0 NAL, or any other interleaving order. The privilege 0 NAL represents the NAL unit containing the RBSP codestream with the lowest privilege level, the privilege 1 NAL represents the NAL unit containing the RBSP codestream with privilege level 1, and so on. The privilege n NAL represents the NAL unit containing the RBSP codestream with the highest privilege level. Based on the above interleaving order, when the decoding end obtains the code streams of multiple permission levels corresponding to the target image, it can obtain the NAL units corresponding to the RBSP code streams of multiple permission levels in sequence according to the interleaving order from high to low permission levels; or, obtain the NAL units corresponding to the RBSP code streams of multiple permission levels in sequence according to the interleaving order from low to high permission levels; or, obtain the NAL units corresponding to the RBSP code streams of multiple permission levels in sequence according to the preset interleaving order of the permission levels.

[0236] For RBSP bitstreams with low permission levels, if the permission identifier of the current CU is a high permission level, parsing is skipped by default. For RBSP bitstreams with high permission levels, the information of each CU is parsed. For example, if the decoding end user has a low permission level (that is, the target permission level is a low permission level), when reconstructing a high permission CU, the default value is used to fill in. If the decoding end user has a high permission level, when reconstructing a high permission CU, the RBSP bitstream of the corresponding high permission level is parsed to obtain the reconstructed value. For another example, if the decoding end user has a low permission level, when reconstructing a high permission CU, intra-frame prediction uses a certain fixed prediction mode to obtain the reconstructed value, and inter-frame prediction uses the reconstructed value at the same position of the reference frame as the reconstructed value. If the decoding end user has a high permission level, when reconstructing a high permission CU, the RBSP bitstream of the corresponding high permission level is parsed to obtain the reconstructed value.

[0237] For the RBSP code stream of each permission level, the RBSP code stream is encrypted using a patch-level encryption method, and / or the RBSP code stream is encrypted using a CU-level encryption method.

[0238] If the RBSP codestream is encrypted using a patch-level encryption method, and the RBSP codestream is also encrypted using a CU-level encryption method, then: when the target permission level (i.e., the permission level of the decoding end user) is less than the current patch-level permission, parsing of the information in the RBSP codestream is prohibited, that is, the entire patch cannot obtain high-dimensional reconstruction information. When the target permission level is greater than or equal to the current patch-level permission, parsing of the CU information in the RBSP codestream continues. When parsing the CU information in the RBSP codestream, since the RBSP codestream is encrypted using a CU-level encryption method, if the target permission level is less than the current CU-level permission, parsing of the high-dimensional reconstruction information of the current CU is prohibited, that is, the high-dimensional reconstruction information of the current CU cannot be obtained; when the target permission level is greater than or equal to the current CU-level permission, parsing of the high-dimensional reconstruction information of the current CU continues, that is, the high-dimensional reconstruction information of the current CU can be parsed. In summary, after obtaining all parseable codestreams, the CU partition information and CU permission information within the patch can be parsed from a certain permission codestream, and the high-dimensional reconstruction information can be parsed from the corresponding permission codestream based on the CU partition information and CU permission information. For example, each RBSP stream encapsulation package includes two encryption methods: patch-level encryption and CU-level encryption, each of which can have multiple levels. If patch-based encryption is used, a decoder with a permission level lower than the current patch's permission level will be completely unable to parse any information, even information with the lowest permission level. If the decoder user obtains patch-level permission, it will continue to parse the CU information within each RBSP stream. Acquiring patch-level permission automatically assumes the permission level of the lowest-privileged RBSP stream.

[0239] If the RBSP codestream is encrypted using a patch-level encryption method, but the RBSP codestream is not encrypted using a CU-level encryption method, then: when the target permission level is less than the current patch-level permission, parsing of the information in the RBSP codestream is prohibited, that is, the entire patch cannot obtain high-dimensional reconstruction information. When the target permission level is greater than or equal to the current patch-level permission, parsing of the CU information in the RBSP codestream continues. When parsing the CU information in the RBSP codestream, since the RBSP codestream is not encrypted using a CU-level encryption method, the high-dimensional reconstruction information of the current CU can be directly parsed without comparing the relationship between the target permission level and the current CU-level permission, that is, the high-dimensional reconstruction information of the current CU can be parsed.

[0240] If the RBSP codestream is not encrypted using a patch-level encryption method, but is encrypted using a CU-level encryption method, then: there is no need to compare the target permission level (i.e., the permission level of the decoding end user) with the current patch-level permission, and the CU information in the RBSP codestream can be directly parsed. When parsing the CU information in the RBSP codestream, since the RBSP codestream is encrypted using a CU-level encryption method, if the target permission level is less than the current CU-level permission, parsing the high-dimensional reconstruction information of the current CU is prohibited, that is, the current CU cannot obtain high-dimensional reconstruction information; when the target permission level is greater than or equal to the current CU-level permission, parsing the high-dimensional reconstruction information of the current CU continues, that is, the high-dimensional reconstruction information of the current CU can be parsed.

[0241] Example 7: In Example 1 and Example 2, the encoding end sends code streams of multiple permission levels corresponding to the target image to the decoding end, and the decoding end obtains code streams of multiple permission levels corresponding to the target image. The packaging method of code streams of different permission levels can also be combined with the knowledge base frame code stream.

[0242] For example, if there are K permission levels, the codestreams for multiple permission levels corresponding to a target image (e.g., a patch-level image) include the RBSP codestreams for the K permission levels corresponding to the target image. The K RBSP codestreams for the K permission levels are located in K NAL units, and the RBSP codestreams for different permission levels are located in different NAL units. Based on this, if there are multiple target images, the knowledge base frame codestream referenced by the i-th target image is interleaved with the main bitstream of the i-th target image. The main bitstream of the i-th target image includes the RBSP codestreams for the K permission levels corresponding to the i-th target image, and the i-th target image is any target image.

[0243] For example, if the knowledge base frame is transmitted as a separate knowledge bitstream, then the high-privilege-level RBSP codestream and the low-privilege-level RBSP codestream are transmitted as the main bitstream and are independent of the knowledge bitstream. However, if the knowledge bitstream is interleaved with the main bitstream, an interleaving method can be specified. An example of an interleaving method is: the knowledge base frame codestream referenced by the i-th target image is interleaved with the main bitstream of the i-th target image. For example, the knowledge base frame codestream referenced by the first target image is interleaved with the main bitstream of the first target image, the knowledge base frame codestream referenced by the second target image is interleaved with the main bitstream of the second target image, and so on.

[0244] For example, the main bit stream of the i-th target image is interleaved with the j-th knowledge base frame code stream, and the j-th knowledge base frame is the reference knowledge base frame of the i-th target image. Alternatively, one frame of knowledge base frame code stream is interleaved with the main bit streams of N target images, where N is greater than or equal to 1, and the one frame of knowledge base frame serves as the reference knowledge base frame of the N target images, that is, the same knowledge base frame can be referenced by N target images. In summary, the main bit stream of the i-th target image is interleaved with the knowledge base frame code stream referenced by the i-th target image. For example, if the same knowledge base frame is referenced by one target image, the knowledge bit stream of the first patch is interleaved with the main bit stream of the first frame image, the knowledge bit stream of the second patch is interleaved with the main bit stream of the second frame image, and so on.

[0245] In a possible implementation, the following interleaving scheme may be used when combined with the knowledge base frame code stream:

[0246] Interleaving scheme 1: The knowledge base frame code stream referenced by the i-th target image is located after the last RBSP code stream among the K permission level RBSP code streams corresponding to the i-th target image.

[0247] For example, each patch of the knowledge base image is packaged as an RBSP, which is processed into a VCL NAL unit. This is interleaved with the NAL units generated by each frame of the main bitstream in the following manner: current frame image header NAL -> current frame's first patch's permission 0 information NAL -> current frame's first patch's permission 1 information NAL -> ... (multiple permission information, multiple patches) -> knowledge base NAL (that is, placed after each frame's NAL). In summary, the knowledge base frame codestream is placed after the RBSP codestream of the last permission level.

[0248] For example, VCL NAL refers to the code stream obtained by encoding the patch content, that is, the NAL packaged with header information and the NAL packaged with extended information do not belong to VCL NAL.

[0249] Interleaving scheme 2: The knowledge base frame code stream referenced by the i-th target image is located before the first RBSP code stream among the K permission level RBSP code streams corresponding to the i-th target image.

[0250] For example, each patch in the knowledge base is packaged as a separate VCL NAL and placed before each frame's VCL NAL. For example, each patch in the knowledge base image is packaged as an RBSP, which is processed to produce a VCL NAL unit. This is interleaved with the NAL units generated by each frame's main bitstream in the following order: current frame image header NAL -> knowledge base NAL -> permission 0 information NAL in the first patch of the current frame -> permission 1 information NAL in the first patch of the current frame -> ... (multiple permission information, multiple patches). As can be seen from the above, the knowledge base frame codestream precedes the RBSP codestream of the first permission level.

[0251] Interleaving Scheme 3: If the i-th target image is not the last target image among all target images, the knowledge base frame codestream referenced by the i-th target image is located after the last RBSP codestream among the K permission level RBSP codestreams corresponding to the i-th target image. Alternatively, if the i-th target image is the last target image among all target images, the knowledge base frame codestream referenced by the i-th target image is located before the first RBSP codestream among the K permission level RBSP codestreams corresponding to the i-th target image.

[0252] For example, each patch in the knowledge base is packaged as an independent VCL NAL. When interleaving, the VCL NAL generated by a non-last patch in the knowledge base is placed after all NALs of the corresponding image, such as after the last RBSP codestream of the K permission levels corresponding to the target image. See Interleaving Scheme 1. When interleaving, the VCL NAL generated by the last patch in the knowledge base is placed before the corresponding image header NAL, such as before the first RBSP codestream of the K permission levels corresponding to the target image. See Interleaving Scheme 2. After decoding and obtaining the last patch information, the complete knowledge base frame can be obtained, and the current frame can refer to the knowledge base frame.

[0253] Interleaving scheme 4: Place the NAL generated by the knowledge base patch before or after the image header NAL position of each frame, or place the NAL generated by the knowledge base patch before or after a certain extended information NAL of each frame.

[0254] As can be seen from the above embodiments, when permission protection is enabled, various types of information permission protection can be performed on images and videos (such as license plates, local areas of the human body, screens, text, etc.), thereby improving data security. For example, when there are information security issues with images, permission protection can be performed on the images so that users with high permission levels can view a large amount or even all of the image information, while users with low permission levels can only view a small amount of image information. For example, the image is divided into low permission areas and high permission areas, and low permission users can only view low permission areas, while high permission users can view both low permission areas and high permission areas.

[0255] Example 8: In the present application, an image decoding method is proposed. Figure 5 FIG. 1 is a flow chart of the method, which is applied to a decoding end (also called a video decoder), and includes:

[0256] Step 501: When permission protection is enabled, obtain a minimum permission level code stream and at least one non-minimum permission level code stream corresponding to a target image. The target image may be a patch-level image or a frame-level image.

[0257] Step 502: Decode the lowest permission level code stream to obtain first reconstruction information corresponding to the target image, and determine a first reconstructed image corresponding to the target image based on the first reconstruction information.

[0258] Step 503: For each non-minimum privilege level code stream, decode the non-minimum privilege level code stream to obtain second reconstruction information corresponding to the target image, and determine a second reconstructed image corresponding to the target image based on the first reconstruction information corresponding to the target image and the second reconstruction information corresponding to the target image.

[0259] In one possible implementation, when rights protection is enabled, a first permission level codestream and a second permission level codestream corresponding to a target image can be obtained. The target image can be a patch-level image or a frame-level image, and the target image corresponds to at least two spatial domain image frames, which can include a base layer image and an enhancement layer image. The first permission level codestream is a codestream of the lowest permission level, and the second permission level codestream is a codestream of a non-lowest permission level. The first permission level codestream is then decoded to obtain first reconstruction information corresponding to the target image, and the base layer reconstructed image corresponding to the target image is determined based on the first reconstruction information. The second permission level codestream is decoded to obtain second reconstruction information corresponding to the target image, and the enhancement layer reconstructed image corresponding to the target image is determined based on the first reconstruction information and the second reconstruction information. The resolution of the enhancement layer reconstructed image can be the same as that of the base layer reconstructed image.

[0260] In one possible implementation, when rights protection is enabled, a first permission level codestream and a second permission level codestream corresponding to a target image are obtained, where the target image is a patch-level image or a frame-level image. The target image corresponds to at least two time-domain image frames, where the at least two time-domain image frames include a k-th time-domain image frame and a (k+i)-th time-domain image frame, where i ranges from 1 to M, and M represents the total number of second permission level codestreams. The first permission level codestream is a lowest permission level codestream, and the second permission level codestream is a non-lowest permission level codestream. The first permission level codestream is decoded to obtain first reconstruction information corresponding to the target image, and the k-th time-domain reconstructed image corresponding to the target image is determined based on the first reconstruction information. The i-th second permission level codestream among the M second permission level codestreams is decoded to obtain second reconstruction information corresponding to the target image, and the (k+i)-th time-domain image corresponding to the target image is determined based on the first reconstruction information and the second reconstruction information.

[0261] For example, the above execution order is only for the convenience of describing the examples given. In actual applications, the execution order between the steps can also be changed, and this execution order is not limited. Moreover, in other embodiments, the steps of the corresponding method are not necessarily executed in the order shown and described in this specification, and the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; multiple steps described in this specification may also be combined into a single step for description in other embodiments.

[0262] In the embodiment of the present application, an image coding method is proposed. Figure 6 FIG. 1 is a flow chart of the method, which is applied to an encoding end (also called a video encoder), and includes:

[0263] Step 601: When permission protection is enabled, obtain a minimum permission level code stream and at least one non-minimum permission level code stream corresponding to a target image. The target image may be a patch-level image or a frame-level image.

[0264] Step 602: Encode first reconstruction information corresponding to the target image in a lowest-privilege-level bitstream; wherein the first reconstruction information is used to determine a first reconstructed image corresponding to the target image.

[0265] Step 603: For each non-minimum privilege level codestream, encode second reconstruction information corresponding to the target image in the non-minimum privilege level codestream; wherein the first reconstruction information corresponding to the target image and the second reconstruction information corresponding to the target image are used to determine the second reconstructed image corresponding to the target image.

[0266] In a possible implementation, when rights protection is enabled, a first permission level codestream and a second permission level codestream corresponding to a target image may be obtained. The target image may be a patch-level image or a frame-level image. The target image may correspond to at least two spatial domain image frames, and the at least two spatial domain image frames may include a base layer image and an enhancement layer image. The first permission level codestream may be a lowest permission level codestream, and the second permission level codestream may be a non-lowest permission level codestream. First reconstruction information corresponding to the target image is encoded in the first permission level codestream, and the first reconstruction information is used to determine a base layer reconstructed image corresponding to the target image. Second reconstruction information corresponding to the target image is encoded in the second permission level codestream, and the first reconstruction information and the second reconstruction information are used to determine an enhancement layer reconstructed image corresponding to the target image. The resolution of the enhancement layer reconstructed image is the same as that of the base layer reconstructed image.

[0267] In one possible implementation, when rights protection is enabled, a first permission level codestream and a second permission level codestream corresponding to a target image are obtained, where the target image is a patch-level image or a frame-level image. The target image corresponds to at least two time-domain image frames, where the at least two time-domain image frames include a k-th time-domain image frame and a (k+i)-th time-domain image frame, where i ranges from 1 to M, and M represents the total number of second permission level codestreams. The first permission level codestream is a lowest permission level codestream, and the second permission level codestream is a non-lowest permission level codestream. First reconstruction information corresponding to the target image is encoded in the first permission level codestream, where the first reconstruction information is used to determine the k-th time-domain reconstructed image frame corresponding to the target image. Second reconstruction information corresponding to the target image is encoded in the i-th second permission level codestream among the M second permission level codestreams. The first reconstruction information and the second reconstruction information are used to determine the (k+i)-th time-domain image frame corresponding to the target image.

[0268] For example, the above execution order is only for the convenience of describing the examples given. In actual applications, the execution order between the steps can also be changed, and this execution order is not limited. Moreover, in other embodiments, the steps of the corresponding method are not necessarily executed in the order shown and described in this specification, and the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; multiple steps described in this specification may also be combined into a single step for description in other embodiments.

[0269] As can be seen from the above technical solution, in the embodiments of the present application, when permission protection is enabled, the code streams of different permissions can be divided into multiple frames for decoding, which can protect the permissions of various types of information in images and videos (such as license plates, local areas of the human body, screens, text, etc.), thereby improving data security. For example, when there is an information security issue with an image, permission protection can be implemented on the image so that users with high permission levels can view a large amount or even all of the image information, while users with low permission levels can only view a small amount of image information.

[0270] Example 9: In Example 8, for the same target image (such as a patch-level image or a frame-level image), a frame-level permission protection code stream distribution based on the spatial domain can be designed. When implementing permission protection, the code streams of different permissions can be divided into multiple frames for decoding, and the non-minimum permission can be treated as a spatial domain enhancement layer.

[0271] Exemplarily, the encoder can obtain the lowest privilege level codestream (i.e., the first privilege level codestream) and at least one non-lowest privilege level codestream (i.e., the second privilege level codestream) corresponding to the target image, and send the lowest privilege level codestream and the at least one non-lowest privilege level codestream to the decoder. The decoder then obtains the lowest privilege level codestream (i.e., the first privilege level codestream) and the at least one non-lowest privilege level codestream (i.e., the second privilege level codestream) corresponding to the target image. The target image can correspond to at least two spatial domain image frames (i.e., at least two spatial domain image frames generated based on the same target image), and the lowest privilege level codestream and the at least one non-lowest privilege level codestream correspond to the same target image, i.e., each spatial domain image frame corresponds to one codestream. The at least two spatial domain image frames can include a base layer image and an enhancement layer image.

[0272] Exemplarily, if the target image corresponds to at least two frames of spatial domain images, that is, at least two frames of spatial domain images that are different in the spatial domain are generated based on the target image, the decoding end can decode the lowest authority level code stream to obtain first reconstruction information, and determine the first reconstructed image based on the first reconstruction information. The first reconstructed image can be a base layer reconstructed image, that is, the decoding end decodes the lowest authority code stream to obtain the base layer reconstructed image.

[0273] The decoding end may decode the non-minimum privilege level codestream to obtain second reconstruction information, and determine a second reconstructed image based on the first reconstruction information and the second reconstruction information. The second reconstructed image may be an enhancement layer reconstructed image. That is, the decoding end decodes the non-minimum privilege level codestream to obtain an enhancement layer reconstructed image. When the target image corresponds to M non-minimum privilege level codestreams, an enhancement layer reconstructed image may be obtained by decoding each non-minimum privilege level codestream, i.e., M enhancement layer reconstructed images may be obtained, where M is a positive integer.

[0274] Exemplarily, the resolution of the enhanced layer reconstructed image is the same as the resolution of the base layer reconstructed image, and the CU partitioning information of the enhanced layer reconstructed image is the same as the CU partitioning information of the base layer reconstructed image.

[0275] Exemplarily, each permission code stream (such as the lowest permission level code stream and at least one non-lowest permission level code stream) includes CU partition information (i.e., partition information of each CU) and CU permission information (i.e., permission information of each CU), and each permission code stream includes all image headers and patch header information.

[0276] For example, during prediction, the enhancement layer reconstruction process is allowed to reference the base layer, but the base layer reconstruction process is not allowed to reference the enhancement layer. Furthermore, the division of the base layer and enhancement layer must be completely consistent. For example, when decoding a minimum privilege level bitstream, the base layer reconstructed image can only be determined based on the first reconstruction information, and the second reconstruction information cannot be used to determine the base layer reconstructed image. That is, the base layer reconstruction process is not allowed to reference the enhancement layer. When decoding a non-minimum privilege level bitstream, the enhancement layer reconstructed image can be determined based on the first reconstruction information and the second reconstruction information. That is, the enhancement layer reconstruction process references the base layer.

[0277] For example, during reconstruction, the reconstructed image of the enhancement layer can be merged with the reconstructed image of the base layer, and the reconstructed content of the enhanced layer parsed CU can replace the corresponding position of the base layer reconstructed image, and these positions are marked. When the base layer of the next frame is reconstructed, reference to the reconstructed image generated by the enhancement layer is not allowed.

[0278] For example, the enhanced layer reconstructed image can be merged with the base layer reconstructed image to obtain a merged image. The target image may include at least one CU. For each CU, if the enhanced layer reconstructed image includes the reconstructed image block corresponding to the CU, the image content corresponding to the CU in the base layer reconstructed image is replaced by the reconstructed image block; if the enhanced layer reconstructed image does not include the reconstructed image block corresponding to the CU, the image content corresponding to the CU in the base layer reconstructed image is retained. On this basis, the modified base layer reconstructed image is used as the merged image.

[0279] After obtaining the merged image, the first type CU in the merged image can be marked, where the first type CU is a CU whose image content comes from the enhancement layer reconstructed image; wherein, when generating the next base layer reconstructed image, it is prohibited to refer to the first type CU in the merged image to generate the next base layer reconstructed image.

[0280] For example, during reconstruction, the enhancement layer reconstructed image and the base layer reconstructed image can be retained simultaneously, that is, two frames of images can be retained simultaneously. When reconstructing the next base layer frame, the base layer reconstructed image of the previous frame can be referenced. When reconstructing the next enhancement layer frame, reference can be made to the enhancement layer reconstructed image of the previous frame, the base layer reconstructed image of the current frame, and the base layer reconstructed image of the previous frame.

[0281] Example 10: In Example 8, for the same target image (such as a patch-level image or a frame-level image), a frame-level permission protection code stream distribution based on the time domain can be designed. When implementing permission protection, the code streams of different permissions can be divided into multiple frames for decoding, and non-minimum permissions can be processed as time domain images.

[0282] Exemplarily, the encoder can obtain the lowest privilege level codestream (i.e., the first privilege level codestream) and at least one non-lowest privilege level codestream (i.e., the second privilege level codestream) corresponding to the target image, and send the lowest privilege level codestream and the at least one non-lowest privilege level codestream to the decoder. The decoder then obtains the lowest privilege level codestream (i.e., the first privilege level codestream) and the at least one non-lowest privilege level codestream (i.e., the second privilege level codestream) corresponding to the target image. The target image can correspond to at least two time-domain image frames (i.e., at least two time-domain image frames generated based on the same target image). The lowest privilege level codestream and the at least one non-lowest privilege level codestream correspond to the same target image, i.e., each time-domain image frame corresponds to one codestream. The at least two time-domain image frames can include the kth time-domain image frame and the (k+i)th time-domain image frame, where i ranges from 1 to M, and M represents the total number of non-lowest privilege level codestreams.

[0283] Exemplarily, if the target image corresponds to at least two frames of time domain images, that is, at least two frames of time domain images that are different in the time domain are generated based on the target image, the decoding end can decode the lowest authority level code stream to obtain first reconstruction information, and determine the first reconstructed image based on the first reconstruction information. The first reconstructed image can be the k-th frame time domain reconstructed image, that is, the decoding end decodes the lowest authority code stream to obtain the k-th frame time domain reconstructed image.

[0284] The decoding end decodes the i-th non-minimum privilege level codestream among the M non-minimum privilege level codestreams to obtain second reconstruction information, and determines a second reconstructed image based on the first reconstruction information and the second reconstruction information. The second reconstructed image may be the (k+i)-th time-domain reconstructed image. That is, the decoding end decodes the i-th non-minimum privilege level codestream to obtain the (k+i)-th time-domain reconstructed image, where i is a positive integer. When the target image corresponds to M non-minimum privilege level codestreams, the first non-minimum privilege level codestream is decoded to obtain the (k+1)-th time-domain reconstructed image, the second non-minimum privilege level codestream is decoded to obtain the (k+2)-th time-domain reconstructed image, and so on, to obtain M time-domain reconstructed images for the M non-minimum privilege level codestreams.

[0285] Exemplarily, the resolution of the (k+i)th temporal reconstructed image is the same as the resolution of the kth temporal reconstructed image, and the CU partitioning information of the (k+i)th temporal reconstructed image is the same as the CU partitioning information of the kth temporal reconstructed image. For example, the resolutions of the kth temporal reconstructed image, the (k+1)th temporal reconstructed image, the (k+2)th temporal reconstructed image, ..., the (k+M)th temporal reconstructed image may all be the same, and the CU partitioning information of the kth temporal reconstructed image, the (k+1)th temporal reconstructed image, the (k+2)th temporal reconstructed image, ..., the (k+M)th temporal reconstructed image may all be the same.

[0286] Exemplarily, the lowest-privilege codestream is decoded to obtain the kth frame time-domain reconstructed image, and the image obtained by decoding the non-lowest-privilege codestream is used as the k+1th, k+2th, ..., k+Mth frames time-domain reconstructed image. Each permission codestream (such as the lowest-privilege level codestream and at least one non-lowest-privilege level codestream) includes CU partition information (i.e., the partition information of each CU), and each permission codestream includes all picture headers and patch header information.

[0287] For example, during prediction, the reconstruction process of the (k+i)th frame time-domain reconstructed image is allowed to refer to the kth frame time-domain reconstructed image, but the reconstruction process of the kth frame time-domain reconstructed image is not allowed to refer to the (k+i)th frame time-domain reconstructed image. For example, when decoding a bitstream at the lowest privilege level, the kth frame time-domain reconstructed image can only be determined based on the first reconstruction information, and cannot be determined by referring to the (k+i)th frame time-domain reconstructed image. That is, the reconstruction process of the kth frame time-domain reconstructed image is not allowed to refer to the (k+i)th frame time-domain reconstructed image.

[0288] When decoding a non-minimum privilege level codestream, a (k+i)th frame time-domain reconstructed image can be determined based on the first reconstruction information and the second reconstruction information. That is, the reconstruction process of the (k+i)th frame time-domain reconstructed image refers to the kth frame time-domain reconstructed image. For example, the target image may include at least one CU. For each CU, the i-th non-minimum privilege level codestream (i.e., the second privilege level codestream) can be decoded to obtain the second reconstruction information corresponding to the CU. The reconstructed image block corresponding to the CU is determined based on the first reconstruction information and the second reconstruction information corresponding to the CU. The (k+i)th frame time-domain reconstructed image is generated based on the reconstructed image block corresponding to each CU.

[0289] For example, during prediction, the reconstruction process of the (k+1)th temporal reconstructed image is allowed to refer to the kth temporal reconstructed image, but the kth temporal reconstructed image is not allowed to refer to the (k+1)th temporal reconstructed image or the (k-1)th temporal reconstructed image. However, the kth temporal reconstructed image is allowed to refer to the (k+M+1)th or (kM)th frame, that is, it is allowed to refer to the temporal reconstructed images corresponding to other target images. The reconstruction process of the (k+M)th temporal reconstructed image is allowed to refer to the kth, k+1th, ..., k+M-1th temporal reconstructed images.

[0290] Exemplarily, during reconstruction, only the time domain reconstructed image of the kth frame is retained. For the time domain reconstructed images of the k+1 to k+M frames, the reconstructed pixels of the CU directly marked as permission-protected replace the content of the corresponding position in the kth frame (the CU division information of the kth frame and the (k+i)th frame are required to be completely consistent). When predicting the reference, it is stipulated that the kth frame cannot refer to the reconstructed pixels generated by the k+1 to k+M frames. Therefore, the image information positions generated by the k+1 to k+M frames are recorded and saved to restrict low-privilege references to high-privilege.

[0291] Example 11: In Example 8, for the same target image (such as a patch-level image or a frame-level image), a frame-level permission protection code stream distribution based on the time domain can be designed. When implementing permission protection, the code streams of different permissions can be divided into multiple frames for decoding, and non-minimum permissions can be processed as time domain images.

[0292] Exemplarily, the encoder can obtain the lowest privilege level codestream (i.e., the first privilege level codestream) and at least one non-lowest privilege level codestream (i.e., the second privilege level codestream) corresponding to the target image, and send the lowest privilege level codestream and the at least one non-lowest privilege level codestream to the decoder. The decoder then obtains the lowest privilege level codestream (i.e., the first privilege level codestream) and the at least one non-lowest privilege level codestream (i.e., the second privilege level codestream) corresponding to the target image. The target image can correspond to at least two time-domain image frames (i.e., at least two time-domain image frames generated based on the same target image). The lowest privilege level codestream and the at least one non-lowest privilege level codestream correspond to the same target image, i.e., each time-domain image frame corresponds to one codestream. The at least two time-domain image frames can include the kth time-domain image frame and the (k+i)th time-domain image frame, where i ranges from 1 to M, and M represents the total number of non-lowest privilege level codestreams.

[0293] Exemplarily, if the target image corresponds to at least two frames of time domain images, that is, at least two frames of time domain images that are different in the time domain are generated based on the target image, the decoding end can decode the lowest authority level code stream to obtain first reconstruction information, and determine the first reconstructed image based on the first reconstruction information. The first reconstructed image can be the k-th frame time domain reconstructed image, that is, the decoding end decodes the lowest authority code stream to obtain the k-th frame time domain reconstructed image.

[0294] The decoding end decodes the i-th non-minimum privilege level codestream among the M non-minimum privilege level codestreams to obtain second reconstruction information, and determines a second reconstructed image based on the first reconstruction information and the second reconstruction information. The second reconstructed image may be the (k+i)-th time-domain reconstructed image. That is, the decoding end decodes the i-th non-minimum privilege level codestream to obtain the (k+i)-th time-domain reconstructed image, where i is a positive integer. When the target image corresponds to M non-minimum privilege level codestreams, the first non-minimum privilege level codestream is decoded to obtain the (k+1)-th time-domain reconstructed image, the second non-minimum privilege level codestream is decoded to obtain the (k+2)-th time-domain reconstructed image, and so on, to obtain M time-domain reconstructed images for the M non-minimum privilege level codestreams.

[0295] Exemplarily, the resolution of the (k+i)th temporal reconstructed image is the same as the resolution of the kth temporal reconstructed image, and the CU partitioning information of the (k+i)th temporal reconstructed image is the same as the CU partitioning information of the kth temporal reconstructed image. For example, the resolutions of the kth temporal reconstructed image, the (k+1)th temporal reconstructed image, the (k+2)th temporal reconstructed image, ..., the (k+M)th temporal reconstructed image may all be the same, and the CU partitioning information of the kth temporal reconstructed image, the (k+1)th temporal reconstructed image, the (k+2)th temporal reconstructed image, ..., the (k+M)th temporal reconstructed image may all be the same.

[0296] Exemplarily, the kth frame time domain reconstructed image is obtained by decoding the minimum authority code stream, and the images obtained by decoding the non-minimum authority code stream are used as the k+1th, k+2th, ..., k+Mth frame time domain reconstructed images.

[0297] Exemplarily, each permission codestream (e.g., the lowest permission level codestream and at least one non-lowest permission level codestream) includes CU partition information (i.e., the partition information of each CU) and CU permission information (i.e., the permission information of each CU). Each permission codestream includes all picture headers and patch header information.

[0298] For example, during prediction, the reconstruction process of the (k+i)th frame time-domain reconstructed image is allowed to refer to the kth frame time-domain reconstructed image, but the reconstruction process of the kth frame time-domain reconstructed image is not allowed to refer to the (k+i)th frame time-domain reconstructed image. For example, when decoding a bitstream at the lowest privilege level, the kth frame time-domain reconstructed image can only be determined based on the first reconstruction information, and cannot be determined by referring to the (k+i)th frame time-domain reconstructed image. That is, the reconstruction process of the kth frame time-domain reconstructed image is not allowed to refer to the (k+i)th frame time-domain reconstructed image.

[0299] When decoding a non-minimum privilege level codestream, the (k+i)th frame's temporal reconstructed image can be determined based on the first reconstruction information and the second reconstruction information. That is, the reconstruction process of the (k+i)th frame's temporal reconstructed image refers to the kth frame's temporal reconstructed image. For example, the target image includes first-class CUs and second-class CUs, where the privilege level of the first-class CUs is greater than the minimum privilege level, and the privilege level of the second-class CUs is the minimum privilege level. The i-th non-minimum privilege level codestream (i.e., the second privilege level codestream) is decoded to obtain the second reconstruction information corresponding to the first-class CUs. Based on the first reconstruction information and the second reconstruction information corresponding to the first-class CUs, the reconstructed image blocks corresponding to the first-class CUs are determined. The reconstructed image blocks corresponding to the second-class CUs are obtained from the first reconstructed image (i.e., the reconstructed image blocks corresponding to the second-class CUs are not obtained by decoding the non-minimum privilege level codestream). On this basis, the (k+i)th frame's temporal reconstructed image can be generated based on the reconstructed image blocks corresponding to each first-class CU and each second-class CU.

[0300] Exemplarily, during reconstruction, only the time domain reconstructed image of the kth frame is retained. For the time domain reconstructed images of the k+1 to k+M frames, the reconstructed pixels of the CU directly marked as permission-protected replace the content of the corresponding position in the kth frame (the CU division information of the kth frame and the (k+i)th frame are required to be completely consistent). When predicting the reference, it is stipulated that the kth frame cannot refer to the reconstructed pixels generated by the k+1 to k+M frames. Therefore, the image information positions generated by the k+1 to k+M frames are recorded and saved to restrict low-privilege references to high-privilege.

[0301] Exemplarily, after generating the (k+i)th frame time domain reconstructed image based on the reconstructed image blocks corresponding to each first-class CU and the reconstructed image blocks corresponding to each second-class CU, the first-class CU in the (k+i)th frame time domain image is marked, and when generating the next kth frame time domain reconstructed image, it is prohibited to generate the next kth frame time domain reconstructed image with reference to the first-class CU in the (k+i)th frame time domain reconstructed image.

[0302] Illustratively, the above embodiments can be implemented individually or in combination. For example, each of Embodiments 1 to 11 can be implemented individually, and at least two of Embodiments 1 to 11 can be implemented in combination. Illustratively, in the above embodiments, the content of the encoding end can also be applied to the decoding end, that is, the decoding end can be processed in the same manner, and the content of the decoding end can also be applied to the encoding end, that is, the encoding end can be processed in the same manner. These details will not be repeated here.

[0303] Based on the same application concept as the above method, an embodiment of the present application further proposes an image decoding device, which is applied to a decoding end and includes: a memory configured to store video data; a decoder configured to implement the decoding methods in the above embodiments 1 to 11, that is, the processing flow of the decoding end. For example, in one possible implementation, the decoder is configured to implement:

[0304] When rights protection is enabled, obtaining codestreams of one or more permission levels corresponding to a target image, the target image being a patch-level image or a frame-level image, and the target image including at least one CU; wherein, if the target image includes a CU of a first permission level and a CU of a second permission level, the codestreams of the multiple permission levels include a codestream of the first permission level and a codestream of the second permission level; wherein, the first permission level is a minimum permission level, and the second permission level is a non-minimum permission level;

[0305] The CU partition information and CU permission information of the target image are decoded based on the code stream of the one or more permission levels, and each CU of the target image is decoded based on the CU partition information and the CU permission information.

[0306] Alternatively, when permission protection is enabled, a first permission level codestream and a second permission level codestream corresponding to a target image are obtained, where the target image is a patch-level image or a frame-level image, and the target image corresponds to at least two frames of spatial domain images, and the at least two frames of spatial domain images include a base layer image and an enhancement layer image; wherein the first permission level codestream is a lowest permission level codestream, and the second permission level codestream is a non-lowest permission level codestream; the first permission level codestream is decoded to obtain first reconstruction information corresponding to the target image, and a base layer reconstructed image corresponding to the target image is determined based on the first reconstruction information; the second permission level codestream is decoded to obtain second reconstruction information corresponding to the target image, and an enhancement layer reconstructed image corresponding to the target image is determined based on the first reconstruction information and the second reconstruction information, where the resolution of the enhancement layer reconstructed image is the same as the resolution of the base layer reconstructed image.

[0307] Alternatively, when rights protection is enabled, a first permission level codestream and a second permission level codestream corresponding to a target image are obtained, where the target image is a patch-level image or a frame-level image; wherein the target image corresponds to at least two time-domain image frames, the at least two time-domain image frames including a k-th time-domain image frame and a (k+i)-th time-domain image frame, where i ranges from 1 to M, and M represents the total number of second permission level codestreams; the first permission level codestream is a lowest permission level codestream, and the second permission level codestream is a non-lowest permission level codestream; the first permission level codestream is decoded to obtain first reconstruction information corresponding to the target image, and the k-th time-domain reconstructed image corresponding to the target image is determined based on the first reconstruction information; the i-th second permission level codestream among the M second permission level codestreams is decoded to obtain second reconstruction information corresponding to the target image, and the (k+i)-th time-domain image corresponding to the target image is determined based on the first reconstruction information and the second reconstruction information.

[0308] Based on the same application concept as the above method, an embodiment of the present application further proposes an image encoding device, which is applied to an encoding end and includes: a memory configured to store video data; an encoder configured to implement the encoding methods in the above embodiments 1 to 11, that is, the processing flow of the encoding end. For example, in one possible implementation, the encoder is configured to implement:

[0309] When rights protection is enabled, obtaining codestreams of one or more permission levels corresponding to a target image, the target image being a patch-level image or a frame-level image, and the target image including at least one CU; wherein, if the target image includes a CU of a first permission level and a CU of a second permission level, the codestreams of the multiple permission levels include a codestream of the first permission level and a codestream of the second permission level; wherein, the first permission level is a minimum permission level, and the second permission level is a non-minimum permission level;

[0310] Encoding CU partition information and CU permission information of the target image in the bitstream of the one or more permission levels; the CU permission information is used to indicate the permission level corresponding to each CU in the target image;

[0311] Encoding each CU of the target image in the codestreams of the multiple permission levels based on the CU partition information and the CU permission information;

[0312] The code streams of multiple permission levels corresponding to the target image are sent to the decoding end.

[0313] Alternatively, when rights protection is enabled, obtaining codestreams of multiple permission levels corresponding to a target image, the target image being a patch-level image or a frame-level image, and the target image including at least one CU; wherein, if the target image includes a CU of a first permission level and a CU of a second permission level, the codestreams of the multiple permission levels include a codestream of the first permission level and a codestream of the second permission level; wherein, the first permission level is a minimum permission level, and the second permission level is a non-minimum permission level;

[0314] Encoding CU partition information and CU permission information of the target image in the code streams of the multiple permission levels; the CU permission information is used to indicate the permission level corresponding to each CU in the target image;

[0315] Encoding each CU of the target image in the codestreams of the multiple permission levels based on the CU partition information and the CU permission information;

[0316] The code streams of multiple permission levels corresponding to the target image are sent to the decoding end.

[0317] Alternatively, when rights protection is enabled, obtaining a first permission level codestream and a second permission level codestream corresponding to a target image, where the target image is a patch-level image or a frame-level image; wherein the target image corresponds to at least two time-domain image frames, the at least two time-domain image frames including a k-th time-domain image frame and a (k+i)-th time-domain image frame, where i is 1-M, and M represents the total number of second permission level codestreams; the first permission level codestream is a lowest permission level codestream, and the second permission level codestream is a non-lowest permission level codestream; and encoding first reconstruction information corresponding to the target image in the first permission level codestream, where the first reconstruction information is used to determine the k-th time-domain reconstructed image frame corresponding to the target image.

[0318] Second reconstruction information corresponding to the target image is encoded in the i-th second authority level code stream among M second authority level code streams; wherein the first reconstruction information and the second reconstruction information are used to determine the (k+i)-th frame time domain image corresponding to the target image.

[0319] Based on the same application concept as the above method, the decoding end device (also called video decoder) provided in the embodiment of the present application, from the hardware level, its hardware architecture diagram can be specifically referred to as Figure 7A As shown. It includes: a processor 711 and a machine-readable storage medium 712, the machine-readable storage medium 712 storing machine-executable instructions (image decoding instructions) that can be executed by the processor 711; the processor 711 is used to execute the machine-executable instructions to implement the image decoding methods of the above-mentioned embodiments 1 to 11 of the present application.

[0320] Based on the same application concept as the above method, the encoding end device (also called video encoder) provided in the embodiment of the present application, from the hardware level, its hardware architecture diagram can be specifically referred to as Figure 7B As shown. It includes: a processor 721 and a machine-readable storage medium 722, the machine-readable storage medium 722 storing machine-executable instructions (image encoding instructions) that can be executed by the processor 721; the processor 721 is used to execute the machine-executable instructions to implement the image encoding methods of the above-mentioned embodiments 1 to 11 of the present application.

[0321] Based on the same application concept as the above method, an embodiment of the present application also provides a machine-readable storage medium, on which a number of computer instructions are stored. When the computer instructions are executed by a processor, the method disclosed in the above example of the present application can be implemented, such as the image decoding method or the image encoding method in the above embodiments.

[0322] Based on the same application concept as the above method, an embodiment of the present application further provides a computer application, which, when executed by a processor, can implement the image decoding method or image encoding method disclosed in the above example of the present application.

[0323] Based on the same application concept as the above method, an embodiment of the present application further proposes an image decoding device, which is applied to a decoding end, and the device includes: an acquisition module, used to obtain one or more permission level code streams corresponding to a target image when permission protection is enabled, the target image is a patch-level image or a frame-level image, and the target image includes at least one CU; wherein, if the target image includes a CU of a first permission level and a CU of a second permission level, the multiple permission level code streams include a first permission level code stream and a second permission level code stream; wherein the first permission level is the lowest permission level and the second permission level is a non-lowest permission level; a decoding module, used to decode CU partition information and CU permission information of the target image based on the one or more permission level code streams, and decode each CU of the target image based on the CU partition information and the CU permission information.

[0324] Exemplarily, when the decoding module decodes each CU of the target image based on the CU partition information and the CU permission information, it is specifically used to: for the current CU to be decoded in the target image, determine the current permission level corresponding to the current CU based on the CU permission information; if it is determined based on the current permission level that the target permission level has access rights to the current CU, decode the high-dimensional reconstruction information corresponding to the current CU from the target code stream based on the CU partition information, and determine the reconstructed image block corresponding to the current CU based on the high-dimensional reconstruction information; wherein, the target permission level is the permission level of the user accessing the decoding end, and the target code stream is a code stream whose permission level is less than or equal to the target permission level.

[0325] Exemplarily, when the decoding module decodes each CU of the target image based on the CU partition information and the CU permission information, it is specifically used to: if it is determined based on the current permission level that the target permission level does not have access rights to the current CU, then skip decoding the reconstruction information corresponding to the current CU from the target code stream; obtain the low-dimensional reconstruction information corresponding to the current CU, and determine the reconstructed image block corresponding to the current CU based on the low-dimensional reconstruction information; wherein the low-dimensional reconstruction information is a fixed reconstruction value, or the low-dimensional reconstruction information is a reconstruction value obtained based on intra-frame prediction, or the low-dimensional reconstruction information is a reconstruction value obtained based on inter-frame prediction.

[0326] Exemplarily, when the decoding module decodes the CU partition information and CU permission information of the target image based on the code streams of the multiple permission levels, it is specifically used to: if the CU partition information and CU permission information of the target image are located in the code stream of the first permission level, then decode the CU partition information and the CU permission information from the code stream of the first permission level.

[0327] Exemplarily, when the decoding module decodes the CU partition information and CU permission information of the target image based on the code streams of the multiple permission levels, it is specifically used to: if the CU partition information of the target image is located in the code stream of the first permission level, decode the CU partition information from the code stream of the first permission level; and / or, if the permission information of the first CU of the target image is located in the code stream of the permission level less than or equal to the permission level of the first CU, decode the permission information of the first CU from the code stream of the permission level less than or equal to the permission level of the first CU; wherein, the first CU is any CU.

[0328] Exemplarily, for a bitstream having a permission level lower than the permission level of the first CU, the permission information of the first CU in the bitstream is a first value, where the first value is used to indicate that the bitstream does not include high-dimensional reconstruction information corresponding to the first CU; or, for a bitstream having a permission level equal to the permission level of the first CU, the permission information of the first CU in the bitstream is a second value, where the second value is used to indicate that the bitstream includes high-dimensional reconstruction information corresponding to the first CU.

[0329] Exemplarily, when the decoding module decodes the CU partition information and CU permission information of the target image based on the code streams of the multiple permission levels, it is specifically used to: if the CU partition information and CU permission information of the target image are located in the code streams of the first permission level and the second permission level, then the CU partition information and the CU permission information are decoded from the code stream of the first permission level, or the CU partition information and the CU permission information are decoded from the code stream of the second permission level.

[0330] Exemplarily, if the permission information of a second CU of the target image is in a codestream having a first permission level and a second permission level, and the second CU is any CU, then: for a codestream having a permission level less than the permission level of the second CU, the permission information of the second CU in the codestream has a first value, where the first value indicates that the codestream does not include high-dimensional reconstruction information corresponding to the second CU; or, for a codestream having a permission level greater than or equal to the permission level of the second CU, the permission information of the second CU in the codestream has a second value, where the second value indicates that the codestream includes high-dimensional reconstruction information corresponding to the second CU; or, for a codestream having a permission level not equal to the permission level of the second CU, the permission information of the second CU in the codestream has a first value, where the first value indicates that the codestream does not include high-dimensional reconstruction information corresponding to the second CU; or, for a codestream having a permission level equal to the permission level of the second CU, the permission information of the second CU in the codestream has a second value, where the second value indicates that the codestream includes high-dimensional reconstruction information corresponding to the second CU.

[0331] Exemplarily, the decoding module is further used to decode the rights protection parameters corresponding to the target image and the region of interest parameters corresponding to the target image from the high-level syntax corresponding to the target image; wherein the first syntax used by the rights protection parameters is different from the second syntax used by the region of interest parameters, or the first syntax is partially the same as the second syntax, or the first syntax is completely the same as the second syntax.

[0332] Exemplarily, if there are K permission levels, the code streams of multiple permission levels corresponding to the target image include the RBSP code streams of the K permission levels corresponding to the target image, the RBSP code streams of the K permission levels are located in K NAL units, and the RBSP code streams of different permission levels are located in different NAL units.

[0333] Exemplarily, the decoding module is further configured to encrypt, for each permission level, an RBSP code stream using a patch-level encryption method, and / or encrypt the RBSP code stream using a CU-level encryption method; if the RBSP code stream is encrypted using a patch-level encryption method, then when the target permission level is less than the current patch-level permission, parsing of information in the RBSP code stream is prohibited; when the target permission level is greater than or equal to the current patch-level permission, parsing of CU information in the RBSP code stream is continued; when parsing the CU information in the RBSP code stream, if the RBSP code stream is encrypted using a CU-level encryption method, then when the target permission level is less than the current CU-level permission, parsing of high-dimensional reconstruction information of the current CU is prohibited; when the target permission level is greater than or equal to the current CU-level permission, parsing of high-dimensional reconstruction information of the current CU is continued.

[0334] Exemplarily, when the acquisition module acquires the code streams of multiple permission levels corresponding to the target image, it is specifically used to: sequentially acquire the NAL units corresponding to the RBSP code streams of multiple permission levels in an interleaving order from high to low permission levels; or, sequentially acquire the NAL units corresponding to the RBSP code streams of multiple permission levels in an interleaving order from low to high permission levels; or, sequentially acquire the NAL units corresponding to the RBSP code streams of multiple permission levels in a preset interleaving order of the permission levels.

[0335] Based on the same application concept as the above method, an embodiment of the present application further proposes an image encoding device, which is applied to an encoding end, and includes: an acquisition module, for acquiring, when permission protection is enabled, one or more permission-level codestreams corresponding to a target image, the target image being a patch-level image or a frame-level image, and the target image including at least one CU; wherein, if the target image includes a CU of a first permission level and a CU of a second permission level, the multiple permission-level codestreams include a first permission level codestream and a second permission level codestream; the first permission level is a lowest permission level, and the second permission level is a non-lowest permission level; an encoding module, for encoding CU partition information and CU permission information of the target image in the one or more permission-level codestreams; the CU permission information is used to indicate the permission level corresponding to each CU in the target image; based on the CU partition information and the CU permission information, encoding each CU of the target image in the multiple permission-level codestreams; and a sending module, for sending the multiple permission-level codestreams corresponding to the target image to a decoding end.

[0336] Based on the same application concept as the above method, an embodiment of the present application further provides an image decoding device, which is applied to a decoding end and includes: an acquisition module, configured to, when permission protection is enabled, acquire a first permission level codestream and a second permission level codestream corresponding to a target image, wherein the target image is a patch-level image or a frame-level image, and the target image corresponds to at least two spatial domain image frames, and the at least two spatial domain image frames include a base layer image and an enhancement layer image; wherein the first permission level codestream is a lowest permission level codestream, and the second permission level codestream is a non-lowest permission level codestream; a decoding module, configured to decode the first permission level codestream to obtain first reconstruction information corresponding to the target image, and determine a base layer reconstructed image corresponding to the target image based on the first reconstruction information; and decode the second permission level codestream to obtain second reconstruction information corresponding to the target image, and determine an enhancement layer reconstructed image corresponding to the target image based on the first reconstruction information and the second reconstruction information, wherein the resolution of the enhancement layer reconstructed image is the same as that of the base layer reconstructed image.

[0337] Exemplarily, the decoding module is further used to merge the enhancement layer reconstructed image with the base layer reconstructed image to obtain a merged image; wherein, the target image includes at least one CU, and for each CU, if the enhancement layer reconstructed image includes the reconstructed image block corresponding to the CU, then the image content corresponding to the CU in the base layer reconstructed image is replaced by the reconstructed image block; if the enhancement layer reconstructed image does not include the reconstructed image block corresponding to the CU, then the image content corresponding to the CU in the base layer reconstructed image is retained; wherein, the modified base layer reconstructed image is used as the merged image.

[0338] Exemplarily, the decoding module is also used to mark the first type CU in the merged image, where the first type CU is a CU whose image content comes from the enhancement layer reconstructed image; wherein, when generating a subsequent base layer reconstructed image, it is prohibited to generate a subsequent base layer reconstructed image with reference to the first type CU in the merged image.

[0339] Based on the same application concept as the above-mentioned method, an embodiment of the present application further provides an image encoding device, which is applied to an encoding end and includes: an acquisition module, configured to, when permission protection is enabled, acquire a first permission level codestream and a second permission level codestream corresponding to a target image, wherein the target image is a patch-level image or a frame-level image, and the target image corresponds to at least two spatial domain image frames, wherein the at least two spatial domain image frames include a base layer image and an enhancement layer image; wherein the first permission level codestream is a lowest permission level codestream, and the second permission level codestream is a non-lowest permission level codestream; an encoding module, configured to encode first reconstruction information corresponding to the target image in the first permission level codestream, wherein the first reconstruction information is used to determine a base layer reconstructed image corresponding to the target image; and encode second reconstruction information corresponding to the target image in the second permission level codestream, wherein the first reconstruction information and the second reconstruction information are used to determine an enhancement layer reconstructed image corresponding to the target image; wherein the resolution of the enhancement layer reconstructed image is the same as that of the base layer reconstructed image.

[0340] Based on the same application concept as the above method, an embodiment of the present application further provides an image decoding device, which is applied to a decoding end and includes: an acquisition module, configured to, when permission protection is enabled, acquire a first permission level codestream and a second permission level codestream corresponding to a target image, wherein the target image is a patch-level image or a frame-level image; wherein the target image corresponds to at least two time-domain image frames, the at least two time-domain image frames including a k-th time-domain image frame and a (k+i)-th time-domain image frame, where i ranges from 1 to M, and M represents the total number of second permission level codestreams; the first permission level codestream is a lowest permission level codestream, and the second permission level codestream is a non-lowest permission level codestream; a decoding module, configured to decode the first permission level codestream to obtain first reconstruction information corresponding to the target image, and determine the k-th time-domain reconstructed image corresponding to the target image based on the first reconstruction information; and decode the i-th second permission level codestream among the M second permission level codestreams to obtain second reconstruction information corresponding to the target image, and determine the (k+i)-th time-domain image corresponding to the target image based on the first reconstruction information and the second reconstruction information.

[0341] Exemplarily, the target image includes at least one CU. For each CU, the decoding module is specifically used to decode the i-th second authority level code stream to obtain the second reconstruction information corresponding to the CU, determine the reconstructed image block corresponding to the CU based on the first reconstruction information and the second reconstruction information corresponding to the CU; and generate the (k+i)-th frame time domain image based on the reconstructed image block corresponding to each CU.

[0342] Exemplarily, the target image includes a first-class CU and a second-class CU, the permission level of the first-class CU is greater than the minimum permission level, and the permission level of the second-class CU is the minimum permission level; the decoding module is specifically used to decode the i-th second-privilege-level code stream to obtain the second reconstruction information corresponding to the first-class CU, determine the reconstructed image block corresponding to the first-class CU based on the first reconstruction information and the second reconstruction information corresponding to the first-class CU; obtain the reconstructed image block corresponding to the second-class CU from the first reconstructed image; and generate the (k+i)-th frame time domain image based on the reconstructed image block corresponding to each first-class CU and the reconstructed image block corresponding to each second-class CU.

[0343] Exemplarily, the decoding module generates the (k+i)th frame time domain image based on the reconstructed image block corresponding to each first-class CU and the reconstructed image block corresponding to each second-class CU, and is also used to: mark the first-class CU in the (k+i)th frame time domain image, and when generating the next k-th frame time domain image, it is prohibited to refer to the first-class CU in the (k+i)th frame time domain image.

[0344] Based on the same application concept as the above method, an embodiment of the present application further provides an image encoding device, which is applied to an encoding end and includes: an acquisition module, configured to, when rights protection is enabled, acquire a first permission level codestream and a second permission level codestream corresponding to a target image, wherein the target image is a patch-level image or a frame-level image; wherein the target image corresponds to at least two time-domain image frames, the at least two time-domain image frames including a k-th time-domain image frame and a (k+i)-th time-domain image frame, where i ranges from 1 to M, and M represents the total number of second permission level codestreams; the first permission level codestream is a lowest permission level codestream, and the second permission level codestream is a non-lowest permission level codestream; an encoding module, configured to encode first reconstruction information corresponding to the target image in the first permission level codestream, the first reconstruction information being used to determine the k-th time-domain reconstructed image corresponding to the target image; and encode second reconstruction information corresponding to the target image in the i-th second permission level codestream among the M second permission level codestreams; wherein the first reconstruction information and the second reconstruction information are used to determine the (k+i)-th time-domain image corresponding to the target image.

[0345] Those skilled in the art will appreciate that embodiments of the present application may be provided as methods, systems, or computer program products. The present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Embodiments of the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0346] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. An image decoding method, characterized in that: Applied to a decoding end, the method includes: When rights protection is enabled, obtaining code streams of multiple rights levels corresponding to a target image, the target image including at least one CU; if there are K rights levels, the code streams of the multiple rights levels corresponding to the target image include RBSP code streams of the K rights levels corresponding to the target image, the RBSP code streams of the K rights levels being located in K NAL units; wherein the RBSP code streams of different rights levels are located in different NAL units, and the NAL units corresponding to the RBSP code streams of the K rights levels are interleaved; The CU partition information and CU permission information of the target image are decoded based on the code streams of the multiple permission levels, and each CU of the target image is decoded based on the CU partition information and the CU permission information.

2. The method according to claim 1, characterized in that The step of obtaining code streams of multiple permission levels corresponding to the target image includes: Obtaining NAL units corresponding to the RBSP code streams of the K permission levels in sequence according to the interleaving order from high to low permission levels; or, Obtaining the NAL units corresponding to the RBSP code streams of the K permission levels in sequence according to the interleaving order from low to high permission levels; or According to the preset interleaving order of the permission levels, the NAL units corresponding to the RBSP code streams of the K permission levels are obtained in sequence.

3. The method according to claim 1, characterized in that For each permission level RBSP code stream, the RBSP code stream is encrypted using a patch-level encryption method; wherein: When the target permission level is less than the current patch-level permission, parsing of the information in the RBSP code stream is prohibited; when the target permission level is greater than or equal to the current patch-level permission, parsing of the CU information in the RBSP code stream continues; wherein, the target permission level is the permission level of the user accessing the decoding end.

4. The method according to claim 1, wherein The decoding each CU of the target image based on the CU partition information and the CU permission information includes: For the current CU to be decoded in the target image, Determine a current permission level corresponding to the current CU based on the CU permission information; If it is determined based on the current permission level that the target permission level has access rights to the current CU, decoding high-dimensional reconstruction information corresponding to the current CU from the target bitstream based on the CU partition information, and determining a reconstructed image block corresponding to the current CU based on the high-dimensional reconstruction information; The target permission level is the permission level of the user accessing the decoding terminal, and the target code stream is a code stream with a permission level less than or equal to the target permission level.

5. The method according to claim 4, characterized in that The decoding of each CU of the target image based on the CU partition information and the CU permission information further includes: If it is determined based on the current permission level that the target permission level does not have the access rights to the current CU, skipping decoding the reconstruction information corresponding to the current CU from the target bitstream; Obtain low-dimensional reconstruction information corresponding to a current CU, and determine a reconstructed image block corresponding to the current CU based on the low-dimensional reconstruction information; wherein the low-dimensional reconstruction information is a fixed reconstruction value.

6. The method according to any one of claims 1 to 5, characterized in that Before obtaining the code streams of multiple permission levels corresponding to the target image, the method further includes: Decoding rights protection parameters corresponding to the target image from the high-level syntax corresponding to the target image; wherein the rights protection parameters include at least one of the following: a sequence header switch identifier, a picture header switch identifier, and an enhancement layer switch identifier.

7. An image coding method, characterized in that: The method comprises: When rights protection is enabled, obtaining code streams of multiple rights levels corresponding to a target image, the target image including at least one CU; if there are K rights levels, the code streams of the multiple rights levels corresponding to the target image include RBSP code streams of the K rights levels corresponding to the target image, the RBSP code streams of the K rights levels being located in K NAL units; wherein the RBSP code streams of different rights levels are located in different NAL units, and the NAL units corresponding to the RBSP code streams of the K rights levels are interleaved; Encoding CU partition information and CU permission information of the target image in the code streams of the multiple permission levels; the CU permission information is used to indicate the permission level corresponding to each CU in the target image; Encoding each CU of the target image in the codestreams of the multiple permission levels based on the CU partition information and the CU permission information; The code streams of multiple permission levels corresponding to the target image are sent to the decoding end.

8. An image decoding device, characterized in that: The device comprises: An acquisition module is configured to, when rights protection is enabled, acquire code streams of multiple permission levels corresponding to a target image, the target image including at least one CU; if there are K permission levels, the code streams of the multiple permission levels corresponding to the target image include RBSP code streams of the K permission levels corresponding to the target image, the RBSP code streams of the K permission levels being located in K NAL units; wherein the RBSP code streams of different permission levels are located in different NAL units, and the NAL units corresponding to the RBSP code streams of the K permission levels are interleaved; A decoding module is configured to decode the CU partition information and CU permission information of the target image based on the code streams of the multiple permission levels, and decode each CU of the target image based on the CU partition information and the CU permission information.

9. An image coding device, characterized in that The device comprises: An acquisition module is configured to, when rights protection is enabled, acquire code streams of multiple permission levels corresponding to a target image, the target image including at least one CU; if there are K permission levels, the code streams of the multiple permission levels corresponding to the target image include RBSP code streams of the K permission levels corresponding to the target image, the RBSP code streams of the K permission levels being located in K NAL units; wherein the RBSP code streams of different permission levels are located in different NAL units, and the NAL units corresponding to the RBSP code streams of the K permission levels are interleaved; an encoding module, configured to encode CU partition information and CU permission information of the target image in the codestreams of the multiple permission levels, wherein the CU permission information is used to indicate the permission level corresponding to each CU in the target image; and to encode each CU of the target image in the codestreams of the multiple permission levels based on the CU partition information and the CU permission information; The sending module is used to send the code streams of multiple permission levels corresponding to the target image to the decoding end.

10. An image decoding device, characterized in that: include: a processor and a machine-readable storage medium storing machine-executable instructions capable of being executed by the processor; The processor is configured to execute machine-executable instructions to implement the method according to any one of claims 1 to 6.

11. An image encoding device, characterized in that include: a processor and a machine-readable storage medium storing machine-executable instructions capable of being executed by the processor; The processor is configured to execute machine-executable instructions to implement the method of claim 7.

12. A machine-readable storage medium, characterized in that The machine-readable storage medium stores a plurality of computer instructions, and when the computer instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented, or the method according to claim 7 is implemented.

Images