Image decoding, image encoding method, apparatus and device thereof

Abstract

The application provides an image decoding method, an image encoding method, an image decoding device, an image encoding device, and an image decoding apparatus and an image encoding apparatus. The image decoding method comprises: when permission protection is enabled, obtaining a code stream of one or more permission levels corresponding to a target image, the target image being a patch-level image or a frame-level image, and the target image comprising at least one CU; wherein if the target image comprises a CU of a first permission level and a CU of a second permission level, the code stream of the multiple permission levels comprises a code stream of the first permission level and a code stream of the second permission level; wherein the first permission level is the lowest permission level, and the second permission level is a non-lowest permission level; decoding CU partition information and CU permission information of the target image based on the code stream of the multiple permission levels, and decoding each CU of the target image based on the CU partition information and the CU permission information. Through the technical solution of the application, the permission protection of various types of information of an image can be performed, and the data security is improved.

Description

Technical Field

[0001] This application relates to the field of encoding and decoding technology, and in particular to an image decoding method, apparatus and device, as well as an image encoding method, apparatus and device. Background Technology

[0002] To save space, video images are encoded before transmission. Complete video encoding includes processes such as prediction, transform, quantization, entropy coding, and filtering. The prediction process includes intra-frame prediction and inter-frame prediction. Inter-frame prediction utilizes temporal correlation to predict the current pixel using pixels from neighboring encoded images, effectively removing temporal redundancy. Intra-frame prediction utilizes spatial correlation to predict the current pixel using pixels from encoded blocks in the current frame, removing spatial redundancy. Both intra-frame and inter-frame prediction methods allow the decoder to reconstruct a complete video image, which the user can view. However, after reconstructing the complete video image, it displays all access control information, such as license plate information, partial human body areas, and screen information, posing a security risk and requiring access control protection. Summary of the Invention

[0003] In view of this, this application provides an image decoding and image encoding method, apparatus and device, which can protect various types of information rights of images (such as license plates, partial areas of the human body, screen, text, etc.).

[0004] This application provides an image decoding method applied at a decoding end, the method comprising:

[0005] When permission protection is enabled, the bitstreams of one or more permission levels corresponding to the target image are obtained. The target image is a patch-level image or a frame-level image, and the target image includes at least one CU. If the target image includes a CU of a first permission level and a CU of a second permission level, the bitstreams of the multiple permission levels include the bitstreams of the first permission level and the bitstreams of the second permission level. The first permission level is the lowest permission level, and the second permission level is not the lowest permission level.

[0006] Based on the bitstream of the one or more permission levels, decode the CU partitioning information and CU permission information of the target image, and decode each CU of the target image based on the CU partitioning information and CU permission information.

[0007] This application provides an image encoding method applied at an encoding end, the method comprising:

[0008] When permission protection is enabled, the bitstreams of one or more permission levels corresponding to the target image are obtained. The target image is a patch-level image or a frame-level image, and the target image includes at least one CU. If the target image includes a CU of a first permission level and a CU of a second permission level, the bitstreams of the multiple permission levels include the bitstreams of the first permission level and the bitstreams of the second permission level. The first permission level is the lowest permission level, and the second permission level is not the lowest permission level.

[0009] The CU partitioning information and CU permission information of the target image are encoded in the bitstream of the one or more permission levels; the CU permission information is used to indicate the permission level corresponding to each CU in the target image;

[0010] Based on the CU partitioning information and the CU permission information, each CU of the target image is encoded in the bitstream of the multiple permission levels;

[0011] Send the bitstreams corresponding to multiple permission levels of the target image to the decoding end.

[0012] This application provides an image decoding method applied at a decoding end, the method comprising:

[0013] When permission protection is enabled, the first permission level bitstream and the second permission level bitstream corresponding to the target image are obtained. The target image is a patch-level image or a frame-level image. The target image corresponds to at least two spatial domain images, which include a base layer image and an enhancement layer image. The first permission level bitstream is the lowest permission level bitstream, and the second permission level bitstream is a non-lowest permission level bitstream.

[0014] Decode the first permission level bitstream to obtain the first reconstruction information corresponding to the target image, and determine the basic layer reconstruction image corresponding to the target image based on the first reconstruction information;

[0015] The second permission level bitstream is decoded to obtain the second reconstruction information corresponding to the target image. Based on the first reconstruction information and the second reconstruction information, the enhancement layer reconstruction image corresponding to the target image is determined. The resolution of the enhancement layer reconstruction image is the same as the resolution of the base layer reconstruction image.

[0016] This application provides an image encoding method applied at an encoding end, the method comprising:

[0017] When permission protection is enabled, the first permission level bitstream and the second permission level bitstream corresponding to the target image are obtained. The target image is a patch-level image or a frame-level image. The target image corresponds to at least two spatial domain images, which include a base layer image and an enhancement layer image. The first permission level bitstream is the lowest permission level bitstream, and the second permission level bitstream is a non-lowest permission level bitstream.

[0018] The first reconstruction information corresponding to the target image is encoded in the first permission level bitstream, and the first reconstruction information is used to determine the base layer reconstruction image corresponding to the target image.

[0019] The second reconstruction information corresponding to the target image is encoded in the second permission level bitstream. The first reconstruction information and the second reconstruction information are used to determine the enhancement layer reconstruction image corresponding to the target image. The resolution of the enhancement layer reconstruction image is the same as the resolution of the base layer reconstruction image.

[0020] This application provides an image decoding method applied at a decoding end, the method comprising:

[0021] When permission protection is enabled, the first permission level bitstream and the second permission level bitstream corresponding to the target image are obtained. The target image is a patch-level image or a frame-level image. The target image corresponds to at least two temporal images, which include the k-th temporal image and the (k+i)-th temporal image, where i takes the value 1-M and M represents the total number of second permission level bitstreams. The first permission level bitstream is the lowest permission level bitstream, and the second permission level bitstream is a non-lowest permission level bitstream.

[0022] The first permission level bitstream is decoded to obtain the first reconstruction information corresponding to the target image, and the k-th frame temporal reconstruction image corresponding to the target image is determined based on the first reconstruction information;

[0023] The second reconstruction information corresponding to the target image is obtained by decoding the i-th second permission level bitstream among the M second permission level bitstreams. The (k+i)-th frame temporal image corresponding to the target image is determined based on the first reconstruction information and the second reconstruction information.

[0024] This application provides an image encoding method applied at an encoding end, the method comprising:

[0025] When permission protection is enabled, the first permission level bitstream and the second permission level bitstream corresponding to the target image are obtained. The target image is a patch-level image or a frame-level image. The target image corresponds to at least two temporal images, which include the k-th temporal image and the (k+i)-th temporal image, where i takes the value 1-M and M represents the total number of second permission level bitstreams. The first permission level bitstream is the lowest permission level bitstream, and the second permission level bitstream is a non-lowest permission level bitstream.

[0026] The first reconstruction information corresponding to the target image is encoded in the first permission level bitstream, and the first reconstruction information is used to determine the k-th frame temporal reconstruction image corresponding to the target image.

[0027] The second reconstruction information corresponding to the target image is encoded in the i-th second permission level bitstream among the M second permission level bitstreams; wherein, the first reconstruction information and the second reconstruction information are used to determine the (k+i)-th frame temporal image corresponding to the target image.

[0028] This application provides an image decoding device applied at a decoding end, the device comprising:

[0029] The acquisition module is used to acquire bitstreams of one or more permission levels corresponding to a target image when permission protection is enabled. The target image is a patch-level image or a frame-level image, and the target image includes at least one CU. If the target image includes a CU of a first permission level and a CU of a second permission level, the bitstreams of the multiple permission levels include the bitstreams of the first permission level and the bitstreams of the second permission level. The first permission level is the lowest permission level, and the second permission level is a non-lowest permission level.

[0030] The decoding module is used to decode the CU partitioning information and CU permission information of the target image based on the bitstream of the one or more permission levels, and to decode each CU of the target image based on the CU partitioning information and the CU permission information.

[0031] This application provides an image encoding device applied at an encoding end, the device comprising:

[0032] The acquisition module is used to acquire bitstreams of one or more permission levels corresponding to a target image when permission protection is enabled. The target image is a patch-level image or a frame-level image, and the target image includes at least one CU. If the target image includes a CU of a first permission level and a CU of a second permission level, the bitstreams of the multiple permission levels include the bitstreams of the first permission level and the bitstreams of the second permission level. The first permission level is the lowest permission level, and the second permission level is a non-lowest permission level.

[0033] An encoding module is configured to encode CU partitioning information and CU permission information of the target image in the bitstream of one or more permission levels; the CU permission information is used to indicate the permission level corresponding to each CU in the target image; and based on the CU partitioning information and the CU permission information, each CU of the target image is encoded in the bitstream of the multiple permission levels.

[0034] The sending module is used to send the bitstreams corresponding to multiple permission levels of the target image to the decoding end.

[0035] This application provides an image decoding device applied at a decoding end, the device comprising:

[0036] The acquisition module is used to acquire a first permission level bitstream and a second permission level bitstream corresponding to a target image when permission protection is enabled. The target image is a patch-level image or a frame-level image, and the target image corresponds to at least two spatial domain images, which include a base layer image and an enhancement layer image. The first permission level bitstream is the lowest permission level bitstream, and the second permission level bitstream is a non-lowest permission level bitstream.

[0037] The decoding module is used to decode the first permission level bitstream to obtain the first reconstruction information corresponding to the target image, and determine the base layer reconstruction image corresponding to the target image based on the first reconstruction information; decode the second permission level bitstream to obtain the second reconstruction information corresponding to the target image, and determine the enhancement layer reconstruction image corresponding to the target image based on the first reconstruction information and the second reconstruction information, wherein the resolution of the enhancement layer reconstruction image is the same as the resolution of the base layer reconstruction image.

[0038] This application provides an image encoding device applied at an encoding end, the device comprising:

[0039] The acquisition module is used to acquire a first permission level bitstream and a second permission level bitstream corresponding to a target image when permission protection is enabled. The target image is a patch-level image or a frame-level image, and the target image corresponds to at least two spatial domain images, which include a base layer image and an enhancement layer image. The first permission level bitstream is the lowest permission level bitstream, and the second permission level bitstream is a non-lowest permission level bitstream.

[0040] An encoding module is configured to encode first reconstruction information corresponding to the target image in the first permission level bitstream, wherein the first reconstruction information is used to determine the base layer reconstruction image corresponding to the target image; and to encode second reconstruction information corresponding to the target image in the second permission level bitstream, wherein the first reconstruction information and the second reconstruction information are used to determine the enhancement layer reconstruction image corresponding to the target image; wherein the resolution of the enhancement layer reconstruction image is the same as the resolution of the base layer reconstruction image.

[0041] This application provides an image decoding device applied at a decoding end, the device comprising:

[0042] The acquisition module is used to acquire the first permission level bitstream and the second permission level bitstream corresponding to the target image when permission protection is enabled. The target image is a patch-level image or a frame-level image. The target image corresponds to at least two temporal images, which include the k-th temporal image and the (k+i)-th temporal image, where i takes the value 1-M, M represents the total number of second permission level bitstreams, the first permission level bitstream is the lowest permission level bitstream, and the second permission level bitstream is a non-lowest permission level bitstream.

[0043] The decoding module is used to decode the first permission level bitstream to obtain the first reconstruction information corresponding to the target image, and determine the k-th frame temporal reconstruction image corresponding to the target image based on the first reconstruction information; decode the i-th second permission level bitstream among M second permission level bitstreams to obtain the second reconstruction information corresponding to the target image, and determine the (k+i)-th frame temporal image corresponding to the target image based on the first reconstruction information and the second reconstruction information.

[0044] This application provides an apparatus for use at an encoding end, the apparatus comprising:

[0045] The acquisition module is used to acquire the first permission level bitstream and the second permission level bitstream corresponding to the target image when permission protection is enabled. The target image is a patch-level image or a frame-level image. The target image corresponds to at least two temporal images, which include the k-th temporal image and the (k+i)-th temporal image, where i takes the value 1-M, M represents the total number of second permission level bitstreams, the first permission level bitstream is the lowest permission level bitstream, and the second permission level bitstream is a non-lowest permission level bitstream.

[0046] An encoding module is configured to encode first reconstruction information corresponding to the target image in the first permission level bitstream, wherein the first reconstruction information is used to determine the k-th frame temporal reconstruction image corresponding to the target image; and to encode second reconstruction information corresponding to the target image in the i-th second permission level bitstream among M second permission level bitstreams; wherein the first reconstruction information and the second reconstruction information are used to determine the (k+i)-th frame temporal image corresponding to the target image.

[0047] This application provides a decoding end device, including: a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the image decoding method of the example above.

[0048] This application provides an encoding end device, including: a processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the image encoding method of the example above.

[0049] This application provides an electronic device, including: a processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; the processor is configured to execute the machine-executable instructions to implement the image decoding method and image encoding method of the above examples.

[0050] This application provides a machine-readable storage medium, characterized in that the machine-readable storage medium stores a plurality of computer instructions, which, when executed by a processor, implement the image decoding method and image encoding method of the above example.

[0051] As can be seen from the above technical solutions, in the embodiments of this application, when permission protection is enabled, various information permission protections for images and videos (such as license plates, partial areas of the human body, screen, text, etc.) can be implemented, improving data security. For example, when there are information security issues with images, permission protection can be implemented for the images, allowing users with high-level permissions to view a large amount or even all of the image information, while users with low-level permissions can only view a small amount of image information. For example, the image can be divided into low-permission areas and high-permission areas, where low-permission users can only view the low-permission areas, and high-permission users can view both the low-permission areas and the high-permission areas. Attached Figure Description

[0052] Figure 1 This is a schematic diagram of a video coding framework;

[0053] Figure 2 This is a flowchart of an image decoding method according to one embodiment of this application;

[0054] Figure 3 This is a flowchart of an image encoding method according to one embodiment of this application;

[0055] Figure 4A This is a flowchart of the patch image permission protection encoding process in one embodiment of this application;

[0056] Figure 4B This is a flowchart of a high-privilege-level patch image permission protection method in one embodiment of this application;

[0057] Figure 4C This is a flowchart of low-privilege level patch image permission protection in one embodiment of this application;

[0058] Figure 5 This is a flowchart of an image decoding method according to one embodiment of this application;

[0059] Figure 6 This is a flowchart of an image encoding method according to one embodiment of this application;

[0060] Figure 7A This is a hardware structure diagram of the decoding end device in one embodiment of this application;

[0061] Figure 7B This is a hardware structure diagram of the encoding end device in one embodiment of this application. Detailed Implementation

[0062] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the application. The singular forms “a,” “the,” and “the” used in the embodiments and claims of this application are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to any or all possible combinations including one or more of the associated listed items. It should be understood that although the embodiments of this application may use terms such as first, second, etc., to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, first information may also be referred to as second information, and second information may also be referred to as first information, depending on the context, without departing from the scope of the embodiments of this application. Furthermore, the word “if” as used can be interpreted as “when,” “when,” or “in response to a determination.”

[0063] This application proposes an image decoding and image encoding method, which may involve the following concepts:

[0064] Prediction Signal: A prediction signal is a pixel value derived from already encoded and decoded pixels. It is obtained by subtracting the original pixel from the prediction signal to obtain a residual, which is then used for residual transform quantization and coefficient encoding. Inter-frame prediction signals refer to the pixel values ​​derived from the reference frame (reconstructed pixel frame) for the current block. Due to the discrete pixel positions, interpolation is required to obtain the final prediction signal. The closer the prediction signal is to the original pixel, the smaller the residual energy obtained by subtracting the two, and the higher the encoding compression performance.

[0065] Rate-Distortion Optimized (RDBEM) principle: Two main metrics for evaluating coding efficiency are bitrate and PSNR (Peak Signal-to-Noise Ratio). A smaller bitrate results in a higher compression ratio, a higher PSNR, and better reconstructed image quality. In mode selection, the decision formula is essentially a comprehensive evaluation of both. For example, the cost corresponding to a mode is: J(mode) = D + λ*R, where D represents Distortion, usually measured using the SSE metric (Sum of Mean Squares of Differences between the Reconstructed Image Patch and the Source Image). To consider the cost, the SAD metric can also be used, which is the sum of the absolute differences between the Reconstructed Image Patch and the Source Image. λ is a Lagrange multiplier, and R is the actual number of bits required to encode the image patch in that mode, including the total number of bits needed for encoding mode information, motion information, residuals, etc. Using the RDBEM principle to compare and decide on coding modes during mode selection usually ensures optimal coding performance.

[0066] Video coding framework: see Figure 1 The diagram shown is a schematic of the video encoding framework at the encoding end. This video encoding framework can be used to implement the encoding end processing flow of this application. The schematic diagram of the video decoding framework can be compared with... Figure 1 Similarly, the decoding process of this application can be implemented using a video decoding framework.

[0067] For example, see Figure 1 As shown, a video coding framework can include modules such as prediction, transform, quantization, entropy encoder, inverse quantization, inverse transform, reconstruction, and filtering. At the encoding end, the cooperation between these modules enables the encoding processing flow. Similarly, a video decoding framework can include modules such as prediction, transform, quantization, entropy decoder, inverse quantization, inverse transform, reconstruction, and filtering. At the decoding end, the cooperation between these modules enables the decoding processing flow.

[0068] Numerous encoding tools have been proposed for various modules at the encoding end, and each tool often has multiple modes. The optimal encoding tool for different video sequences often differs. Therefore, during encoding, Rate-Distortion Optimization (RDO) is typically used to compare the encoding performance of different tools or modes to select the best mode. After determining the optimal tool or mode, the decision information is transmitted by encoding marker information in the bitstream. Although this method introduces higher encoding complexity, it can adaptively select the optimal mode combination for different content to achieve the best encoding performance. At the decoding end, the relevant mode information is obtained by directly parsing the marker information, thus minimizing the impact of complexity.

[0069] Bitstream: A bitstream is a sequence of bits consisting of encoded audio and video data and related data. Bitstreams can be used to represent NAL (Network Abstraction Layer) unit streams or byte streams. Therefore, both NAL unit streams and byte streams are called bitstreams.

[0070] The NAL unit stream format consists of a series of syntax structures called NAL units, ordered in decoding order. The decoding order and content of the NAL units in the NAL unit stream are constrained.

[0071] Byte streams can be constructed using NAL unit streams. This involves arranging NAL units in decoding order and adding a start code prefix and several zero-value bytes to each NAL unit to form a byte stream. The NAL unit stream format can be extracted from the byte stream format by searching for a unique start code prefix within the byte stream. Besides byte stream formats, there are other methods for constructing NAL units.

[0072] NumBytesInNALunit indicates the length of a NAL unit, in bytes. A NAL unit consists of a header and a payload. The payload contains an RBSP (Raw Byte Sequence Payload) syntax structure and possible authentication data payload, and may also contain some emulation_prevention_three_bytes. NumBytesInNALunit (the byte unit within the NAL unit) is needed when decoding NAL units. To derive NumBytesInNALunit, the boundaries of the NAL units also need to be defined.

[0073] The NAL cell stream is generated into an ordered byte stream, and the NAL cell boundary positions can be identified from the byte stream. For bit-oriented transmission, the bit order in the byte stream starts from the MSB of the first byte, is processed up to the LSB of the first byte, then the MSB of the second byte, and so on.

[0074] The byte stream format consists of a series of byte stream NAL unit syntax structures. Each byte stream NAL unit syntax structure contains a start code prefix followed by a NAL unit.

[0075] The byte stream NAL unit syntax structure may contain an additional zero_byte syntax element, and may also contain one or more additional trailing_zero_8bits syntax elements. The first byte stream NAL unit syntax structure may also contain one or more additional leading_zero_8bits syntax elements.

[0076] NAL unit: A NAL unit is a syntax structure that contains a type indicator for the subsequent data and the number of bytes it contains. The data appears in RBSP format and may include scattered anti-spoofing bytes if necessary. A NAL unit stream is a sequence of NAL units.

[0077] A coded video sequence is an image sequence consisting of IDR images arranged in decoding order and followed by zero or more non-IDR images.

[0078] Emulation prevention byte: The emulation prevention byte is a single byte with a value that can be 0x03. It may appear in the NAL unit. The presence of the emulation prevention byte ensures that the byte stream aligned to subsequent bytes of the NAL unit will not contain a start code prefix.

[0079] A block is an M*N (M columns and N rows) sample matrix in the video signal space, or an M*N transform coefficient matrix in the video signal space.

[0080] Start code prefix: The start code prefix is ​​a unique 3-byte sequence in the byte stream that equals 0x000001. The start code prefix serves as the prefix for each NAL unit. The decoding end (e.g., a decoder) can use the position of the start code prefix to determine the beginning of a new NAL unit and the end of the previous NAL unit. Anti-spoofing bytes are added to the NAL unit to prevent the appearance of fake start code prefixes.

[0081] Secure front end device: A secure front end device is a front-end device equipped with secure cryptographic components. Secure front end devices can use technologies such as certificate storage and management, key storage and management, data signature verification, and data encryption to achieve functions such as device identity authentication, video signing, and video encryption.

[0082] Patch partitioning: Patch partitioning represents how an image frame is divided into patches and coding units (CTUs). When `patch_enable` equals 0, the entire frame contains only one patch; when `patch_enable` equals 1, the entire frame may be divided into multiple patches. A patch can consist of a series of coding units (CTUs), which are the basic units of coding. Each coding unit can contain one luma array and two chroma arrays. In other words, a patch can be understood as an image block containing an integer number of CTUs or LCUs.

[0083] The index of the CTU in the top left corner of the image is equal to 0. The CTU indices in the image increase sequentially according to the raster scan order. An image is divided into several rectangular regions in the horizontal and vertical directions. Each rectangular region is called a patch. Each patch contains several CTUs, which can be encoded and decoded in parallel and independently.

[0084] When the image width is greater than or equal to 8 maximum tree coding unit (CTU) sizes, it can be divided into multiple patch columns horizontally. When the image width exceeds 64 CTU sizes, it needs to be divided into multiple patch columns. Each patch must contain at least 4 CTUs and at most 64 CTUs horizontally. The image can be divided into multiple patch rows vertically, with the number of patch rows being 1, 2, or 4.

[0085] Region of Interest (ROI) partitioning: An image can be divided into several ROIs. The smallest unit of an ROI is 8x8, and samples within the same prediction unit belong to the same ROI. If an ROI exists in the image, the parameter `segmentation_enable` in the corresponding image parameter set is equal to 1; if no ROI exists, the parameter `segmentation_enable` is equal to 0. An ROI in an image can be divided into 8 different levels, indicated by the `segment_id` of the block containing the sample. If no ROI exists in the image, then the `segment_id` of all blocks containing the sample should be equal to 0.

[0086] Raw byte sequence payload (RBSP): The raw byte sequence payload is a syntax structure containing an integer number of bytes encapsulated in NAL units. An RBSP can be empty. An RBSP can contain syntax elements in the form of a data bit string, followed by an RBSP stop bit and zero or more consecutive 0 bits. The raw byte sequence payload stop bit is a single bit with a value of 1, appearing after the data bit string in the RBSP. The end position of the data bit string in the RBSP can be obtained by searching for the RBSP stop bit.

[0087] Encapsulation and constraints of NAL units: An RBSP is encapsulated into a NAL unit using emulation_prevention_three_byte. During the encapsulation process, arbitrary SODBs are allowed in the NAL unit. However, it is necessary to prevent the appearance of false start codes in the NAL unit. This is done by searching for the RBSP bit rbsp_stop_one_bit at the end of the RBSP to identify the end of the SODB in the NAL unit.

[0088] The encoding end (such as the encoder) generates a NAL unit from an RBSP by the following steps: finding the byte-aligned binary bit string '0000000000000000000000xx' in the RBSP, where xx represents any 2-bit string '00', '01', '10', or '11', and inserting a byte equal to 0x03 into it to form '00000000000000000000011000000xx'. The resulting byte sequence, plus the unit header containing the NAL unit that identifies the RBSP data structure type, forms the entire NAL unit.

[0089] Based on the above process for generating NAL units, this process allows any SODB to appear in a NAL unit while ensuring that: there are no byte-aligned pseudo-start codes in the NAL unit; and regardless of whether they are byte-aligned, there is no sequence of 8 bits with a value of 0 followed by a start code in the NAL unit.

[0090] The order of NAL units and their relationship to encoded images and video sequences; the order and effectiveness of sequence parameter sets (RBSPs) and image parameter sets (RBSPs): A sequence parameter set (RBSP) can be used by one or more images, or by SEI NAL units containing buffered periodic SEI messages. Each sequence parameter set (RBSP) becomes effective upon receipt by the decoder and invalidates previously valid sequence parameter set (RBSPs) (if any). At most one sequence parameter set (RBSP) is valid at a specified point in the decoding process. When a sequence parameter set (RBSP) is used by an SEI NAL unit containing buffered periodic SEI messages, that SEI NAL unit can be located after the sequence parameter set (RBSP).

[0091] The parameters included in an Image Parameter Set (RBSP) can be used by the NAL units of a coded image. Each RBSP becomes effective as soon as it is received by the decoder and invalidates any previously valid RBSPs (if any). For a given layer of an SVC image, at most one RBSP is valid at a specified moment during the decoding process.

[0092] The rules governing the relationships between syntax element values ​​and other syntax elements in the sequence parameter set and image parameter set apply only to the valid sequence parameter set and valid image parameter set. During decoding, the parameter values ​​in the valid image parameter set and valid sequence parameter set remain valid.

[0093] Region of Interest (ROI) syntax elements: An image can be divided into several ROIs. The smallest unit of an ROI is 8*8, and samples within the same prediction unit can belong to the same ROI. The syntax elements of ROIs are shown in Table 1.

[0094] Table 1

[0095]

[0096] Decoding of Region of Interest: If Roi_flag is enabled in the current sequence header, a set of probability parameters is parsed from the image header, and the segments in the current frame that need to be skipped during parsing are also obtained. When parsing the segment_id of each block, if the current segment_id is a segment that needs to be skipped during parsing, the parsing of the current block is skipped.

[0097] During inter-frame prediction, for blocks that undergo region of interest processing and select the skip mode (each block is a CU, also known as a CU block), the default corresponding reference frame is DYNAMIC_REF (the 0th reference frame), and the MV is (0,0). At the same time, the corresponding reference block is obtained in the base layer reference frame.

[0098] During the database layering process, the `cu_filter_level` corresponding to the current block can be calculated using the `segment_id`, `ref_frame` (reference frame type), and `mode` (prediction mode) as indices. Different `cu_filter_level` values ​​can be used depending on the current block.

[0099] In one possible implementation, the decoding end can reconstruct the complete video image, which the user can then view. However, after reconstructing the complete video image, it may display all permissions, such as license plate information, partial human body information, text information, and screen information, posing a security risk and requiring permission protection. Therefore, permission protection based on NAL units can be adopted.

[0100] For access control based on NAL units, the input to the NAL unit decoding process is the NAL unit, and the output is the RBSP syntax structure encapsulated in the NAL unit. The RBSP syntax structure can be extracted from the NAL unit. If encryption_idc equals 1, then the RBSP needs to be decrypted when extracting the RBSP syntax structure from the NAL unit to obtain the unencrypted RBSP.

[0101] The RBSP syntax structure in the NAL unit is decoded as follows: When the value of nal_unit_type in the NAL unit is 1 / 2 / 3 / 4, the bitstream within the current NAL unit is parsed, and the corresponding tile is reconstructed using the parsed value. When the value of nal_unit_type in the NAL unit is 1 / 2 / 3 / 4, for RBSPs that cannot be decrypted, this information is discarded, and these tiles cannot be reconstructed.

[0102] For access control based on NAL units, during encoding, the image is divided into tiles, which cannot reference each other. The bitstream information within each tile is written together and packaged into an RBSP within the NAL unit. Each RBSP bitstream can undergo different special processing, such as selective encryption of certain RBSPs. However, the actual size of a tile contains an integer number of CTUs, so special processing can only be applied to large areas of image content. For very small targets, it is not possible to flexibly apply special processing to the bitstream of small regions.

[0103] In response to the above findings, this embodiment divides a Tile bitstream into n RBSPs, writing all different content within a Tile into different RBSPs. Then, the different RBSPs are packaged into different NAL units for transmission. The n RBSP bitstreams obtained from the content of a Tile contain n bitstreams with different permissions, allowing for different processing of bitstreams with different permissions, such as different encryption permission processing.

[0104] This embodiment proposes an image decoding method and an image encoding method, and provides a bitstream processing method based on access control. During the encoding and decoding process, different permissions are assigned to different image regions, and the bitstreams based on different permissions are stored separately. When decoding a low-permission region, access to the high-permission bitstream is blocked. Based on this, when image security issues exist, access control can be implemented, efficiently and losslessly representing the various permission levels of the image. This allows high-permission users to view a large amount or even all of the image information, while low-permission users can only view a small amount. For example, the image can be divided into low-permission and high-permission regions; low-permission users can only view the low-permission region, while high-permission users can view both, thereby improving data security.

[0105] The image decoding method and the image encoding method are described in detail below with reference to several specific embodiments.

[0106] Example 1: This application proposes an image decoding method, see [link to example]. Figure 2 The diagram shown is a flowchart of the method, which is applied to the decoding end (also known as a video decoder). The method includes:

[0107] Step 201: When permission protection is enabled, acquire bitstreams of one or more permission levels corresponding to the target image. The target image is a patch-level image or a frame-level image, and the target image includes at least one CU. If the target image includes a CU of a first permission level and a CU of a second permission level, the bitstreams of multiple permission levels include the bitstreams of the first permission level and the bitstreams of the second permission level; wherein the first permission level is the lowest permission level, and the second permission level is a non-lowest permission level. The second permission level can be at least one permission level, that is, the non-lowest permission level can be at least one non-lowest permission level.

[0108] For example, if all CUs of the target image correspond to the same permission level, then the bitstream corresponding to one permission level of the target image can be obtained. In this embodiment, taking the example that all CUs of the target image correspond to multiple permission levels, the bitstreams corresponding to multiple permission levels of the target image can be obtained.

[0109] Step 202: Decode the CU partitioning information and CU permission information of the target image based on one or more permission levels of the bitstream. For example, decode the CU partitioning information and CU permission information of the target image based on the bitstream of the first permission level and / or the bitstream of the second permission level.

[0110] Step 203: Decode each CU of the target image based on CU partitioning information and CU permission information, that is, decode each CU of the target image one by one based on CU partitioning information and CU permission information.

[0111] In one possible implementation, decoding each CU of the target image based on CU partitioning information and CU permission information may include, but is not limited to: for the current CU to be decoded in the target image, determining the current permission level corresponding to the current CU based on the CU permission information. If it is determined based on the current permission level that the target permission level has access to the current CU, then decoding the high-dimensional reconstruction information corresponding to the current CU from the target bitstream based on the CU partitioning information, and determining the reconstructed image block corresponding to the current CU based on the high-dimensional reconstruction information. Here, the target permission level may be the permission level of the user accessing the decoding end, and the target bitstream may be a bitstream with a permission level less than or equal to the target permission level.

[0112] In one possible implementation, if it is determined based on the current permission level that the target permission level does not have the access permission of the current CU, then the decoding of the reconstruction information corresponding to the current CU from the target bitstream is skipped; the low-dimensional reconstruction information corresponding to the current CU is obtained, and the reconstructed image block corresponding to the current CU is determined based on the low-dimensional reconstruction information; wherein, the low-dimensional reconstruction information is a fixed reconstruction value, or, the low-dimensional reconstruction information is a reconstruction value obtained based on intra-frame prediction, or, the low-dimensional reconstruction information is a reconstruction value obtained based on inter-frame prediction.

[0113] For example, the above execution order is merely an example for ease of description. In practical applications, the execution order between steps can be changed, and there is no limitation on this execution order. Moreover, in other embodiments, the steps of the corresponding method are not necessarily executed in the order shown and described in this specification, and the method may include more or fewer steps than described in this specification. Furthermore, a single step described in this specification may be broken down into multiple steps in other embodiments; multiple steps described in this specification may also be combined into a single step in other embodiments.

[0114] This application proposes an image encoding method, see [link to relevant documentation] Figure 3 The diagram shown is a flowchart of the method, which is applied at the encoding end (also known as a video encoder). The method includes:

[0115] Step 301: When permission protection is enabled, acquire one or more bitstreams corresponding to the target image at different permission levels. The target image is a patch-level image or a frame-level image, and includes at least one CU. If the target image includes a CU of a first permission level and a CU of a second permission level, then the bitstreams of multiple permission levels may include the bitstreams of the first permission level and the bitstreams of the second permission level; wherein the first permission level can be the lowest permission level, and the second permission level can be a non-lowest permission level.

[0116] Step 302: Encode the CU partitioning information and CU permission information of the target image in a bitstream of one or more permission levels; wherein, the CU permission information is used to indicate the permission level corresponding to each CU in the target image.

[0117] Step 303: Based on CU partitioning information and CU permission information, encode each CU of the target image in the bitstream with multiple permission levels.

[0118] For example, for each CU in the target image, when encoding the CU, a candidate bitstream corresponding to the permission level of the CU is determined (i.e., the permission level of the candidate bitstream is the same as the permission level of the CU). Based on the CU partitioning information, the high-dimensional reconstruction information corresponding to the CU is encoded in the candidate bitstream, and the encoding of the high-dimensional reconstruction information corresponding to the CU in the bitstreams of other permission levels is skipped.

[0119] Step 304: Send the bitstreams of multiple permission levels corresponding to the target image to the decoding end.

[0120] For example, the above execution order is merely an example for ease of description. In practical applications, the execution order between steps can be changed, and there is no limitation on this execution order. Moreover, in other embodiments, the steps of the corresponding method are not necessarily executed in the order shown and described in this specification, and the method may include more or fewer steps than described in this specification. Furthermore, a single step described in this specification may be broken down into multiple steps in other embodiments; multiple steps described in this specification may also be combined into a single step in other embodiments.

[0121] As can be seen from the above technical solutions, in the embodiments of this application, when permission protection is enabled, various information permission protections for images and videos (such as license plates, partial areas of the human body, screen, text, etc.) can be implemented, improving data security. For example, when there are information security issues with images, permission protection can be implemented for the images, allowing users with high-level permissions to view a large amount or even all of the image information, while users with low-level permissions can only view a small amount of image information. For example, the image can be divided into low-permission areas and high-permission areas, where low-permission users can only view the low-permission areas, and high-permission users can view both the low-permission areas and the high-permission areas.

[0122] Example 2: To implement permission-based bitstream processing, the following method can be used:

[0123] First, when enabling access control for a target image, the decoder needs to acquire the bitstreams of multiple access levels corresponding to the target image. For example, if there are K access levels, where K is a positive integer greater than 1, then the decoder acquires the bitstreams of K access levels, such as the bitstream of access level 1, access level 2, ..., access level K. Assume access level 1 is the lowest access level (i.e., the first access level bitstream), access level K is the highest access level, and the bitstreams of access levels 2 through K are all second access level bitstreams. Corresponding to the decoding process, the encoder needs to acquire the bitstreams of multiple access levels corresponding to the target image and send them to the decoder. For example, if the target image includes a CU of the first access level and a CU of the second access level, the multiple access level bitstreams include the first access level bitstream and the second access level bitstream; where the first access level is the lowest access level, and the second access level is a non-lowest access level. For example, access level 1 is the first access level, and access levels 2 through K are the second access levels.

[0124] The target image can be a patch-level image or a frame-level image; there are no restrictions on the type of target image. For example, if the target image is a patch-level image, the bitstream format can be ABCABC, where the first group ABC represents the bitstream of multiple permission levels corresponding to the first patch-level image, and the second group ABC represents the bitstream of multiple permission levels corresponding to the second patch-level image. As another example, if the target image is a frame-level image, the bitstream format can be AAABBBCCC, where the first group AAA represents the bitstream of the first permission level corresponding to all frame-level images, the second group BBB represents the bitstream of the second permission level corresponding to all frame-level images, and the third group CCC represents the bitstream of the third permission level corresponding to all frame-level images. Of course, the above are just examples of bitstream formats; the bitstream format of a patch-level image can also be AAABBBCCC, and the bitstream format of a frame-level image can also be ABCABC; there are no restrictions on this.

[0125] Specifically, for the target image, the target image includes at least one CU, which can also be called an image block, or simply a block, i.e., a CU block. The current CU can also be called the current block (current image block).

[0126] Then, the decoding end needs to decode the CU partitioning information and CU permission information of the target image based on bitstreams with multiple permission levels. Corresponding to the decoding end's processing, the encoding end needs to encode the CU partitioning information and CU permission information of the target image in bitstreams with multiple permission levels. For example, the encoding end encodes the CU partitioning information and CU permission information of the target image in the bitstream with the lowest permission level (i.e., the first permission level), and the decoding end decodes the CU partitioning information and CU permission information of the target image from the bitstream with the lowest permission level (i.e., the first permission level). Of course, the above is just an example of decoding CU partitioning information and CU permission information; for the decoding method of this CU partitioning information and CU permission information, please refer to the following embodiments.

[0127] The CU partitioning information represents the position information of each CU within the target image. For each CU, the position information represents the starting position and length of the CU in the bitstream. Based on the position information of the CU, the reconstruction information of the CU can be parsed from the bitstream without any restrictions.

[0128] Here, CU permission information represents the permission information of each CU within the target image. For each CU, the permission information represents the permission level of that CU. For example, if there are K permission levels in total, the permission level of a CU may be permission level 1, permission level 2, ..., permission level K. Among them, the CU with permission level 1 is the first permission level CU, that is, the CU with the lowest permission level, and the CUs with permission levels 2, ..., permission level K are the second permission level CUs, that is, the CUs with non-lowest permission levels.

[0129] Regarding the permission level of each CU, the encoding end obtains the permission level of each CU without any restrictions on the method of obtaining it. The permission level of each CU in the target image is encoded in the bitstream, which is the CU permission information.

[0130] In addition to CU partitioning information and CU permission information, the decoding end can also decode the filtering information of each CU from the bitstream. This filtering information is used to filter the CU, and there are no restrictions on this filtering information.

[0131] Then, the decoding end performs decoding at the granularity of CUs. For the current CU (i.e., the CU to be decoded), the decoding end determines the current permission level corresponding to the current CU based on the CU permission information. For example, since the CU permission information represents the permission level of each CU in the target image, the decoding end can determine the current permission level corresponding to the current CU based on the CU permission information, such as permission level 1, permission level 2, etc.

[0132] Then, based on the current permission level corresponding to the current CU, the decoding end determines whether the target permission level has access to the current CU. For example, if the target permission level is greater than or equal to the current permission level, it is determined that the target permission level has access to the current CU. If the target permission level is less than the current permission level, it is determined that the target permission level does not have access to the current CU.

[0133] The target permission level refers to the permission level of the user accessing the decoding end. When permission protection is enabled, the decoding end can determine the target permission level. For example, if user A is currently viewing the image reconstructed by the decoding end, then user A's permission level is used as the target permission level. If user B is currently viewing the image reconstructed by the decoding end, then user B's permission level is used as the target permission level, and so on.

[0134] Then, if the target permission level has the access permission of the current CU, the decoding end decodes the high-dimensional reconstruction information corresponding to the current CU from the target bitstream based on the CU partitioning information, and determines the reconstructed image block corresponding to the current CU based on the high-dimensional reconstruction information. The target bitstream is a bitstream with a permission level less than or equal to the target permission level.

[0135] The decoding end can obtain the bitstream for each permission level. The bitstream of the lowest permission level can be an unencrypted bitstream (the bitstream of the lowest permission level can also be an encrypted bitstream, taking the unencrypted bitstream as an example). The bitstreams of non-lowest permission levels can be encrypted bitstreams. For example, the encoding end encrypts the bitstreams of non-lowest permission levels, but the encoding end does not encrypt the bitstreams of the lowest permission level.

[0136] When the access level of a bitstream is less than or equal to the target access level, the decoding end can parse the bitstream when a user with the target access level accesses it. For example, if the bitstream with the lowest access level is unencrypted, the decoding end can directly parse it. As another example, even if a bitstream with a non-lowest access level is encrypted, if the access level of that bitstream is less than or equal to the target access level, and a user with the target access level possesses decryption information, the decoding end can successfully decrypt the bitstream based on this information, and then parse the decrypted bitstream.

[0137] When the permission level of the bitstream is greater than that of the target permission level, the bitstream is encrypted. Although the user with the target permission level has the decryption information, the decoding end cannot successfully decrypt the bitstream based on the decryption information, resulting in the inability to parse the bitstream. This can be understood as the inability to obtain the bitstream.

[0138] In summary, a bitstream with a permission level less than or equal to the target permission level can be called a target bitstream. The decoding end can parse the target bitstream, which is a bitstream of at least one permission level. For example, if permission level 1 is the lowest permission level, and the target permission level is permission level 1, then the target bitstream is a bitstream of permission level 1. If the target permission level is permission level 2, then the target bitstream is a bitstream of permission level 1 and permission level 2. If the target permission level is permission level 3, then the target bitstream is a bitstream of permission level 1, permission level 2, and permission level 3, and so on.

[0139] Specifically, when the decoding end decodes the high-dimensional reconstruction information corresponding to the current CU from the target bitstream, it can select a candidate bitstream corresponding to the permission level of the current CU from the target bitstream. Based on the CU partitioning information, it decodes the high-dimensional reconstruction information corresponding to the current CU from the candidate bitstream, skipping the decoding of the high-dimensional reconstruction information corresponding to the current CU from the target bitstream of other permission levels. Corresponding to the decoding end's processing, for each CU in the target image, the encoding end determines the candidate bitstream corresponding to the permission level of that CU, encodes the high-dimensional reconstruction information corresponding to that CU in the candidate bitstream based on the CU partitioning information, skips encoding the high-dimensional reconstruction information corresponding to that CU in the bitstream of other permission levels, that is, does not encode the high-dimensional reconstruction information.

[0140] For example, if the current CU corresponds to permission level 1, the encoder only encodes the high-dimensional reconstruction information corresponding to the current CU in the bitstream of permission level 1 (i.e., the candidate bitstream), and does not encode the high-dimensional reconstruction information corresponding to the current CU in the bitstreams of other permission levels (such as permission level 2, permission level 3, etc.). Based on this, assuming the target permission level is permission level 2, the target bitstream is a bitstream of permission level 1 and a bitstream of permission level 2. The decoder decodes the high-dimensional reconstruction information corresponding to the current CU from the bitstream of permission level 1, and will not decode the high-dimensional reconstruction information corresponding to the current CU from the bitstream of permission level 2.

[0141] For example, if the current CU corresponds to permission level 2, the encoder only encodes the high-dimensional reconstruction information corresponding to the current CU in the bitstream of permission level 2 (i.e., the candidate bitstream), and does not encode the high-dimensional reconstruction information corresponding to the current CU in the bitstreams of other permission levels (such as permission level 1, permission level 3, etc.). Based on this, assuming the target permission level is permission level 2, the target bitstream is a bitstream of permission level 2, and the decoder decodes the high-dimensional reconstruction information corresponding to the current CU from the bitstream of permission level 2.

[0142] When the decoding end decodes the high-dimensional reconstruction information corresponding to the current CU, since the CU partitioning information includes the position information of the current CU, which represents the starting position and length of the current CU in the bitstream, the high-dimensional reconstruction information corresponding to the current CU can be parsed from the bitstream based on the CU partitioning information.

[0143] After obtaining the high-dimensional reconstruction information corresponding to the current CU, the reconstructed image patch corresponding to the current CU can be determined based on the high-dimensional reconstruction information. For example, high-dimensional reconstruction information can also be called open reconstruction information. High-dimensional reconstruction information includes the reconstructed values ​​of all pixels in the current CU, and the image content can be completely restored through high-dimensional reconstruction information. For example, for the current CU of areas such as human body parts, license plates, screens, and text, if the current CU is reconstructed based on high-dimensional reconstruction information, the real image of the human body parts, license plates, screens, and text can be viewed, thus obtaining the reconstructed image patch of the real image.

[0144] Then, if the target permission level does not have access to the current CU, the decoder skips decoding the reconstruction information corresponding to the current CU from the target bitstream. Furthermore, the decoder can obtain the low-dimensional reconstruction information corresponding to the current CU and determine the reconstructed image block corresponding to the current CU based on the low-dimensional reconstruction information. The low-dimensional reconstruction information can be a fixed reconstruction value, or it can be a reconstruction value obtained based on intra-frame prediction, or it can be a reconstruction value obtained based on inter-frame prediction.

[0145] If the target permission level does not have the access permission of the current CU, although the decoding end can also parse the target bitstream, the target bitstream will not contain the high-dimensional reconstruction information corresponding to the current CU. Therefore, the decoding end needs to skip decoding the reconstruction information corresponding to the current CU from the target bitstream.

[0146] For example, if the current CU corresponds to permission level 2, the encoder only encodes the high-dimensional reconstruction information corresponding to the current CU in the bitstream of permission level 2, and does not encode the high-dimensional reconstruction information corresponding to the current CU in the bitstreams of other permission levels (such as permission level 1, permission level 3, etc.). Based on this, assuming the target permission level is permission level 1 (permission level 1 does not have access to the current CU), the target bitstream is a bitstream of permission level 1. The bitstream of permission level 1 does not contain the high-dimensional reconstruction information corresponding to the current CU. Therefore, the decoder skips decoding the reconstruction information corresponding to the current CU from the target bitstream.

[0147] For example, if the current CU corresponds to the permission level of permission level 3, the encoder will only encode the high-dimensional reconstruction information corresponding to the current CU in the bitstream of permission level 3. If the target permission level is permission level 2, the target bitstream is the bitstream of permission level 1 and permission level 2. The target bitstream does not contain the high-dimensional reconstruction information corresponding to the current CU, so the decoding of the reconstruction information corresponding to the current CU from the target bitstream is skipped.

[0148] If the target permission level does not have access to the current CU, the decoding end can obtain the low-dimensional reconstruction information corresponding to the current CU. After obtaining the low-dimensional reconstruction information, the reconstructed image block corresponding to the current CU can be determined based on the low-dimensional reconstruction information. Low-dimensional reconstruction information can also be called hidden reconstruction information. It does not include the reconstruction values ​​of all pixels. It may include only the reconstruction values ​​of some pixels (the decoding end only parses these pixel reconstruction values, and the remaining pixels can use default values), or it may include only one pixel value (i.e., all pixels have this pixel value, such as 255, 0, etc.), or it may include several pixel values ​​(e.g., the average value of some pixels, such as 137, 245, etc.). In other words, the reconstruction values ​​of pixels are hidden in the low-dimensional reconstruction information, and the image content cannot be completely restored through it. For example, for the current CU containing areas such as human body parts, license plates, screens, and text, if the current CU is reconstructed based on low-dimensional reconstruction information, the true image of these areas cannot be viewed.

[0149] The low-dimensional reconstruction information can be fixed reconstruction values, for example, using fixed reconstruction values ​​as the reconstruction values ​​for each pixel within the current CU. Alternatively, the low-dimensional reconstruction information can be reconstruction values ​​obtained based on intra-frame prediction; for example, the intra-frame prediction mode can be used to obtain the reconstruction values ​​for each pixel within the current CU, and there are no restrictions on this intra-frame prediction process. Or, the low-dimensional reconstruction information can be reconstruction values ​​obtained based on inter-frame prediction; for example, the inter-frame prediction mode can be used to obtain the reconstruction values ​​for each pixel within the current CU, and there are no restrictions on this inter-frame prediction process. Of course, these are just a few examples, and no limitations are imposed.

[0150] In one possible implementation, for the access-protected encoding / decoding process, a multi-access-region encoding / decoding scheme is designed. This scheme must ensure that users with lower access levels at the decoding end cannot correctly decode bitstreams with higher access levels, while users with higher access levels can correctly decode bitstreams with higher access levels as well as bitstreams with access levels less than or equal to their own. To achieve this functionality, the processing at the encoding end may include:

[0151] Each region is labeled as a CU (Control Unit), and the permission level of all CUs is recorded. Transmission is based on patch-based bitstream packaging, meaning that region information with different permission levels within a patch is packaged into different bitstreams. For encoding within a patch, references between lower-permission-level regions and higher-permission-level regions are restricted. Furthermore, different permission levels are encoded using different entropy encoders to obtain bitstreams with different permission levels.

[0152] Taking two permission levels as an example, these two permission levels are high permission level and low permission level. For the patch image permission protection encoding process under high permission level and low permission level, please refer to [link / reference]. Figure 4A As shown.

[0153] The patch image is divided into multiple Control Unit (CU) blocks. Based on the high-privilege regions within the patch, the permission level corresponding to each CU block is determined. The permission level of a CU block can be either high or low. For example, if a CU block is located in a high-privilege region within the patch, its permission level can be high (or simply high permission); if a CU block is not located in a high-privilege region, its permission level can be low (or simply low permission).

[0154] For each CU block, if the corresponding permission level is high, then the CU block is subjected to prediction, transform quantization, and entropy encoding (using entropy encoder 1) to obtain a high-permission bitstream. For each CU block, if the corresponding permission level is low, then the CU block is subjected to constraint prediction, transform quantization, and entropy encoding (using entropy encoder 2) to obtain a low-permission bitstream.

[0155] In one possible implementation, for the decoding end, different users have different permission levels. The decoding end can only parse bitstreams within the permission level range; bitstreams outside the permission level range cannot be parsed. The decoding end obtains bitstreams with different permission levels. If the permission level of the bitstream makes parsing impossible, the bitstream parsing is skipped; otherwise, the bitstream is parsed. To achieve the above functionality, the processing procedure of the decoding end may include:

[0156] The sequence header and image header parameters are parsed from the lowest permission level bitstream. The sequence header parameters contain a flag indicating whether permission protection is enabled for the current sequence, and the image header parameters contain a flag indicating whether permission protection is enabled for the current image. Then, the bitstream is parsed patch by patch. If the sequence header or image header indicates that permission protection is disabled, the NAL unit packed into each patch bitstream contains the parameters for the entire patch. Otherwise, the NAL unit packed into each patch bitstream contains only the parameters for a specific permission level of the entire patch image. Based on the parameters parsed from each NAL unit, the reconstructed image is obtained. An example with two permission levels is provided: high permission level and low permission level. For the patch image permission protection decoding process under high and low permission levels, please refer to [link to relevant documentation]. Figure 4B and Figure 4C As shown.

[0157] See Figure 4B The diagram illustrates the patch image access control process for high-privilege users. Entropy decoding (using Entropy Decoder 2) is performed on the low-privilege bitstream to obtain CU partitioning and permissions (i.e., CU partitioning information and CU permission information). For the current CU, if it is determined to be a low-privilege CU based on the CU permission information, then entropy decoding, constraint prediction, inverse transform, and inverse quantization are performed on the low-privilege bitstream to obtain reconstruction information. A patch image is then generated based on this reconstruction information.

[0158] For the current CU, if it is determined that the current CU is of a high privilege level (referred to as high privilege) based on the CU permission information, then the high privilege bitstream is subjected to entropy decoding (such as entropy decoding using entropy decoder 1), prediction, inverse transformation and inverse quantization, etc., to obtain the reconstruction information, and a patch image is generated based on the reconstruction information.

[0159] See Figure 4CThe diagram illustrates the patch image permission protection process for low-privilege users. Entropy decoding (using Entropy Decoder 2) is performed on the low-privilege bitstream to obtain CU partitioning and permissions (i.e., CU partitioning information and CU permission information). For the current CU, if it is determined to be a low-privilege CU based on the CU permission information, then entropy decoding, constraint prediction, inverse transform, and inverse quantization operations are performed on the low-privilege bitstream to obtain reconstruction information. A patch image is then generated based on this reconstruction information.

[0160] For the current CU, if it is determined based on the CU permission information that the current CU has a high permission level (referred to as high permission), then the reconstruction information corresponding to the current CU is obtained. This reconstruction information can be a fixed reconstruction value, or a reconstruction value obtained based on intra-frame prediction, or a reconstruction value obtained based on inter-frame prediction. Figure 4C Taking fixed reconstruction values ​​as an example, the reconstruction information is obtained by filling in fixed values, and a patch image is generated based on the reconstruction information.

[0161] Example 3: In Examples 1 and 2, permission protection for the target image is enabled. Whether permission protection (also known as information protection) is enabled can be determined by passing flag bits at the sequence level, sequence-level extended data level, image level, image-level extended data level, slice level, tile level, and patch level. The flag bits indicate whether permission protection is enabled. If the flag bit is the first value, permission protection is enabled; if the flag bit is the second value, permission protection is not enabled. The syntax used to manage or control several images can be called sequence-level or sequence-level extended data; the syntax used to manage or control a single frame image can be called image-level or image-level extended data; the syntax used to manage or control parallel units (i.e., image regions) including several coding units can be called slice-level, tile-level, or patch-level.

[0162] For example, if the switch control information allows permission protection to be enabled, then permission protection is enabled. If the switch control information does not allow permission protection to be enabled, then permission protection is disabled. For instance, the switch control information may include, but is not limited to: sequence-level or sequence-level extended data switch control information; or image-level or image-level extended data switch control information; or slice-level switch control information; or tile-level switch control information; or patch-level switch control information.

[0163] For example, if the sequence-level switch control information allows permission protection to be enabled, then permission protection is enabled. If the sequence-level switch control information does not allow permission protection to be enabled, then permission protection is disabled. The sequence-level switch control information may include a sequence-level flag bit. If the sequence-level flag bit is the first value, permission protection is enabled; if the sequence-level flag bit is the second value, permission protection is disabled.

[0164] For example, if the switch control information for sequence-level extended data allows permission protection to be enabled, then permission protection is enabled. If the switch control information for sequence-level extended data does not allow permission protection to be enabled, then permission protection is disabled. The switch control information for sequence-level extended data may include a sequence-level extended data flag bit. If the sequence-level extended data flag bit is a first value, permission protection is enabled; if the sequence-level extended data flag bit is a second value, permission protection is disabled.

[0165] For example, if the image-level switch control information allows permission protection to be enabled, then permission protection is enabled. If the image-level switch control information does not allow permission protection to be enabled, then permission protection is disabled. The image-level switch control information may include an image-level flag bit. If the image-level flag bit is a first value, permission protection is enabled; if the image-level flag bit is a second value, permission protection is disabled.

[0166] For example, if the on / off control information for image-level extended data allows permission protection to be enabled, then permission protection is enabled. If the on / off control information for image-level extended data does not allow permission protection to be enabled, then permission protection is disabled. The on / off control information for image-level extended data may include an image-level extended data flag bit. If the image-level extended data flag bit is a first value, permission protection is enabled; if the image-level extended data flag bit is a second value, permission protection is disabled.

[0167] For example, if the slice-level switch control information allows permission protection to be enabled, then permission protection is enabled. If the slice-level switch control information does not allow permission protection to be enabled, then permission protection is disabled. The slice-level switch control information may include a slice-level flag bit. If the slice-level flag bit is the first value, permission protection is enabled; if the slice-level flag bit is the second value, permission protection is disabled.

[0168] For example, if the tile-level switch control information allows access protection to be enabled, then access protection is enabled. If the tile-level switch control information does not allow access protection to be enabled, then access protection is disabled. The tile-level switch control information may include a tile-level flag bit. If the tile-level flag bit is the first value, then access protection is enabled; if the tile-level flag bit is the second value, then access protection is disabled.

[0169] For example, if the patch-level switch control information allows permission protection to be enabled, then permission protection is enabled. If the patch-level switch control information does not allow permission protection to be enabled, then permission protection is disabled. The patch-level switch control information may include a patch-level flag bit. If the patch-level flag bit is the first value, permission protection is enabled; if the patch-level flag bit is the second value, permission protection is disabled.

[0170] For example, sequence-level / sequence-level extended data on / off control information allows permission protection to be enabled for the current sequence, that is, permission protection is enabled for all patch-level images or frame-level images (i.e., slice-level images) within the current sequence. Image-level / image-level extended data on / off control information allows permission protection to be enabled for the current image, that is, permission protection is enabled for all patch-level images or frame-level images within the current image. Slice-level on / off control information allows permission protection to be enabled for the current slice, that is, permission protection is enabled for the current frame-level image (i.e., slice-level image). Tile-level on / off control information allows permission protection to be enabled for the current tile, that is, permission protection is enabled for all patch-level images within the current tile. Patch-level on / off control information allows permission protection to be enabled for the current patch, that is, permission protection is enabled for the current patch-level image.

[0171] Example 4: In Examples 1 and 2, the encoding end can encode the CU partitioning information and CU permission information of the target image in the bitstream, and the decoding end can decode the CU partitioning information and CU permission information of the target image based on the bitstream. Based on this process, the position of the CU partitioning information and CU permission information in the bitstream can be designed, that is, the permission protection syntax elements can be designed. This process can include the following implementation methods:

[0172] Method 1: The CU partitioning information and CU permission information of the target image are located in the lowest permission level bitstream (i.e., the first permission level bitstream). For example, the encoding end encodes the CU partitioning information and CU permission information of the target image in the lowest permission level bitstream, and the decoding end decodes the CU partitioning information and CU permission information from the lowest permission level bitstream.

[0173] For example, the CU partitioning information (i.e., the partitioning information of each CU) and CU permission information (i.e., the permission information of each CU) can be placed only in the bitstream of the lowest permission level. Since users of all permission levels can parse the bitstream of the lowest permission level, the decoding end can decode the CU partitioning information and CU permission information from the bitstream of the lowest permission level, regardless of which target permission level the user adopts.

[0174] For decoding users with CU (Coding Unit) privileges (i.e., target privilege level greater than or equal to the current CU's current privilege level, possessing access to the current CU), the reconstruction information corresponding to the current CU can be obtained from the high-privilege bitstream (i.e., high-dimensional reconstruction information corresponding to the current CU is decoded from the target bitstream), and the reconstruction of the current CU is completed based on this reconstruction information. For decoding users with lower CU privileges (i.e., target privilege level less than the current CU's current privilege level, lacking access to the current CU), the parsing of the current CU is skipped by default, and the reconstruction of the current CU can be completed using the low-dimensional reconstruction information corresponding to the current CU. For example, a fixed value can be used to fill the current CU, or surrounding already reconstructed pixels can be used to fill the current CU. For surrounding already reconstructed pixels, a prediction mode (such as intra-frame prediction mode or inter-frame prediction mode) can be obtained, and the reference pixels in this prediction mode can be used to fill the current CU.

[0175] For example, decode CU permission information from the lowest permission level bitstream, determine the current CU permission level as permission level 2 based on the CU permission information, and the decoding user with permission level less than 2 (i.e. the target permission level is less than 2) skips the parsing of the current CU, while the decoding user with permission level greater than or equal to 2 (i.e. the target permission level is greater than or equal to 2) can parse the reconstruction information of the current CU from the bitstream with permission level 2.

[0176] Method 2: The CU partitioning information of the target image is located in the lowest permission level bitstream (i.e., the first permission level bitstream), and the CU permission information of the target image is located in a bitstream with a permission level less than or equal to that of the CU. That is, for each CU, the permission information is located in a bitstream with a permission level less than or equal to that CU. For example, the encoder encodes the CU partitioning information of the target image in the lowest permission level bitstream, and the decoder decodes the CU partitioning information from the lowest permission level bitstream. And / or, the encoder encodes the permission information of the first CU in a bitstream with a permission level less than or equal to the permission level of the first CU, and the decoder decodes the permission information of the first CU from a bitstream with a permission level less than or equal to the permission level of the first CU. Here, the first CU is any CU in the target image.

[0177] For example, CU partitioning information can be placed only in the bitstream of the lowest privilege level. Since users of all privilege levels can parse the bitstream of the lowest privilege level, the decoding end can decode the CU partitioning information from the bitstream of the lowest privilege level, regardless of which target privilege level the user adopts.

[0178] For example, the permission information of the CU can be placed in the bitstream with a permission level less than or equal to that of the CU. That is, in the bitstream with a permission level less than or equal to that of the CU, it is marked whether the reconstruction information of the current CU is in the current bitstream. When parsing the CU permissions, it is necessary to parse them in the order of permission levels.

[0179] For example, for each CU, the permission information of that CU can be placed in the bitstream at a permission level less than or equal to that CU. In this way, as long as the user's target permission level is greater than or equal to the permission level of that CU, the decoding end can decode the permission information of that CU from the bitstream.

[0180] For example, each CU in all CUs can be called the first CU, and the permission level A of the first CU can be determined. All permission levels less than or equal to permission level A can be called permission level A'. If permission level A is permission level 1, then permission level A' is permission level 1. If permission level A is permission level 2, then permission level A' is permission level 1 and permission level 2, and so on.

[0181] The encoding end can encode the permission information of the first CU in the bitstream of permission level A', and the decoding end can decode the permission information of the first CU from the bitstream of permission level A' (i.e., the bitstream with a permission level lower than the permission level of the first CU). For example, if the permission level of the first CU is permission level 2, the decoding end can decode the permission information of the first CU from the bitstream of permission level 1, and can decode the permission information of the first CU from the bitstream of permission level 2.

[0182] For example, when the decoding end decodes the permission information of the first CU from the bitstream with permission level A', if the permission level A' is less than the permission level of the first CU, that is, for the bitstream with permission level less than the permission level of the first CU, the permission information of the first CU in the bitstream is a first value, which indicates that the bitstream does not contain the high-dimensional reconstruction information corresponding to the first CU; or, if the permission level A' is equal to the permission level of the first CU, that is, for the bitstream with permission level equal to the permission level of the first CU, the permission information of the first CU in the bitstream is a second value, which indicates that the bitstream contains the high-dimensional reconstruction information corresponding to the first CU.

[0183] For example, if the permission level of the first CU is permission level 2, when the decoding end decodes the permission information of the first CU from the bitstream of permission level 1, the permission information of the first CU is a first value (such as 0). The first value indicates that the bitstream of permission level 1 does not contain the high-dimensional reconstruction information corresponding to the first CU. When the decoding end decodes the permission information of the first CU from the bitstream of permission level 2, the permission information of the first CU is a second value (such as 1). The second value indicates that the bitstream of permission level 2 contains the high-dimensional reconstruction information corresponding to the first CU.

[0184] For decoding users with CU (Coding Unit) privileges (i.e., target privilege level is greater than or equal to the current privilege level of the current CU), the reconstruction information corresponding to the current CU can be obtained from the bitstream of the CU privilege level (i.e., high-dimensional reconstruction information corresponding to the current CU is decoded from the target bitstream), and the reconstruction of the current CU is completed based on this reconstruction information. For decoding users with lower CU privileges (i.e., target privilege level is less than the current privilege level of the current CU), the parsing of the current CU is skipped by default, and the reconstruction of the current CU can be completed using the low-dimensional reconstruction information corresponding to the current CU. For example, a fixed value can be used to fill the current CU.

[0185] For example, if the current CU's permission level is 2 and the lowest permission level is 0, the encoder encodes the first value (e.g., 0) in the bitstreams of permission level 0 and permission level 1, respectively, indicating that the current bitstream does not contain the reconstruction information of the current CU. The encoder encodes the second value (e.g., 1) in the bitstream of permission level 2, indicating that the current bitstream contains the reconstruction information of the current CU, and the reconstruction information corresponding to the current CU can be parsed from the current bitstream. Once the decoder parses the bitstream of permission level 2, a decoding user with a target permission level greater than or equal to 2 can continue decoding the reconstruction information of the current CU from the bitstream of permission level 2.

[0186] Method 3: To increase the parallelism of bitstream parsing, i.e., to allow independent decoding of the bitstream at each permission level, CU partitioning information (i.e., the partitioning information of each CU) and CU permission information (i.e., the permission information of each CU) are placed in the bitstreams of all permission levels. CU permission is marked as whether the current bitstream contains information about the current CU. For example, the encoder encodes the CU partitioning information of the target image in the bitstream at each permission level (such as the first permission level and each second permission level). The decoder decodes the CU partitioning information from the bitstream at any permission level. For example, the decoder decodes the CU partitioning information from the bitstream at the first permission level, or from any bitstream at the second permission level.

[0187] For example, CU partitioning information and CU permission information can be placed in the bitstream of each permission level. This way, when the decoding end decodes the bitstream of any permission level, it can decode the CU partitioning information and CU permission information from that bitstream. For instance, when the decoding end decodes the bitstream of permission level 1, it can decode the CU partitioning information and CU permission information from the bitstream of permission level 1; when the decoding end decodes the bitstream of permission level 2, it can decode the CU partitioning information and CU permission information from the bitstream of permission level 2, and so on.

[0188] For example, if the permission information of the second CU of the target image is located in the bitstream of the first permission level and the second permission level, and the second CU is any CU, then: for the bitstream with a permission level lower than the permission level of the second CU, the permission information of the second CU in the bitstream is a first value, which is used to indicate that the bitstream does not contain the high-dimensional reconstruction information corresponding to the second CU; or, for the bitstream with a permission level greater than or equal to the permission level of the second CU, the permission information of the second CU in the bitstream is a second value, which is used to indicate that the bitstream contains the high-dimensional reconstruction information corresponding to the second CU.

[0189] For example, if the permission level of the second CU is permission level 2, and assuming there are bitstreams with permission level 1, permission level 2, and permission level 3, then the permission information of the second CU is located in the bitstreams with permission level 1, permission level 2, and permission level 3. Based on this, when the decoding end decodes the permission information of the second CU from the bitstream with permission level 1, the permission information of the second CU is a first value (e.g., 0), indicating that the bitstream with permission level 1 does not contain the high-dimensional reconstruction information corresponding to the second CU. When the decoding end decodes the permission information of the second CU from the bitstream with permission level 2, the permission information of the second CU is a second value (e.g., 1), indicating that the bitstream with permission level 2 contains the high-dimensional reconstruction information corresponding to the second CU. When the decoding end decodes the permission information of the second CU from the bitstream with permission level 3, the permission information of the second CU is a second value, indicating that the bitstream with permission level 3 contains the high-dimensional reconstruction information corresponding to the second CU.

[0190] For decoding users with CU (Coding Unit) privileges (i.e., target privilege level is greater than or equal to the current privilege level of the current CU), the reconstruction information corresponding to the current CU can be obtained from the bitstream of the CU privilege level (i.e., high-dimensional reconstruction information corresponding to the current CU is decoded from the target bitstream), and the reconstruction of the current CU is completed based on this reconstruction information. For decoding users with lower CU privileges (i.e., target privilege level is less than the current privilege level of the current CU), the parsing of the current CU is skipped by default, and the reconstruction of the current CU can be completed using the low-dimensional reconstruction information corresponding to the current CU. For example, a fixed value can be used to fill the current CU.

[0191] For example, if the current CU's permission level is 2 and the lowest permission level is 0, the encoder encodes a first value (e.g., 0) in the bitstreams of permission level 0 and permission level 1, respectively, indicating that the current bitstream does not contain the reconstruction information of the current CU. The encoder encodes a second value (e.g., 1) in the bitstreams of permission level 2 and permission level 3, indicating that the current bitstream contains the reconstruction information of the current CU, and the reconstruction information corresponding to the current CU can be parsed from the current bitstream. When the decoder parses the bitstream of permission level 2, a decoding user with a target permission level greater than or equal to 2 can continue to decode the reconstruction information of the current CU from the bitstream of permission level 2. When the decoder parses the bitstream of permission level 3, a decoding user with a target permission level greater than or equal to 3 can decode the bitstream of permission level 3. Although the bitstream of permission level 3 does not contain the reconstruction information of the current CU, it can still decode the reconstruction information of the current CU from the bitstream of permission level 2.

[0192] Method 4: To increase the parallelism of bitstream parsing, i.e., to allow independent decoding of the bitstream at each permission level, CU partitioning information (i.e., the partitioning information of each CU) and CU permission information (i.e., the permission information of each CU) are placed in the bitstreams of all permission levels. CU permission is marked as whether the current bitstream contains information about the current CU. For example, the encoder encodes the CU partitioning information of the target image in the bitstream at each permission level (such as the first permission level and each second permission level). The decoder decodes the CU partitioning information from the bitstream at any permission level. For example, the decoder decodes the CU partitioning information from the bitstream at the first permission level, or from any bitstream at the second permission level.

[0193] For example, CU partitioning information can be placed in the bitstream of each permission level, and CU permission information can be placed in the bitstream of each permission level. In this way, when the decoding end decodes the bitstream of any permission level, it can decode the CU partitioning information and CU permission information from the bitstream.

[0194] For example, if the permission information of the second CU of the target image is located in the bitstream of the first permission level and the second permission level, and the second CU is any CU, then: for bitstreams whose permission level is not equal to the permission level of the second CU (such as bitstreams whose permission level is greater than the permission level of the second CU, or bitstreams whose permission level is less than the permission level of the second CU), the permission information of the second CU in the bitstream is a first value, which indicates that the bitstream does not contain the high-dimensional reconstruction information corresponding to the second CU; for bitstreams whose permission level is equal to the permission level of the second CU, the permission information of the second CU in the bitstream is a second value, which indicates that the bitstream contains the high-dimensional reconstruction information corresponding to the second CU.

[0195] If the permission level of the second CU is permission level 2, and assuming there are bitstreams with permission level 1, permission level 2, and permission level 3, then the permission information of the second CU is located in the bitstreams with permission level 1, permission level 2, and permission level 3. Based on this, when the decoding end decodes the permission information of the second CU from the bitstream with permission level 1, the permission information of the second CU is a first value (e.g., 0), indicating that the bitstream with permission level 1 does not contain the high-dimensional reconstruction information corresponding to the second CU. When the decoding end decodes the permission information of the second CU from the bitstream with permission level 2, the permission information of the second CU is a second value (e.g., 1), indicating that the bitstream with permission level 2 contains the high-dimensional reconstruction information corresponding to the second CU. When the decoding end decodes the permission information of the second CU from the bitstream with permission level 3, the permission information of the second CU is a first value, indicating that the bitstream with permission level 3 does not contain the high-dimensional reconstruction information corresponding to the second CU.

[0196] For decoding users with CU (Coding Unit) privileges (i.e., target privilege level is greater than or equal to the current privilege level of the current CU), the reconstruction information corresponding to the current CU can be obtained from the bitstream of the CU privilege level (i.e., high-dimensional reconstruction information corresponding to the current CU is decoded from the target bitstream), and the reconstruction of the current CU is completed based on this reconstruction information. For decoding users with lower CU privileges (i.e., target privilege level is less than the current privilege level of the current CU), the parsing of the current CU is skipped by default, and the reconstruction of the current CU can be completed using the low-dimensional reconstruction information corresponding to the current CU. For example, a fixed value can be used to fill the current CU.

[0197] For example, if the current CU's permission level is 2 and the lowest permission level is 0, the encoder encodes the first value in the bitstreams of permission level 0 and permission level 1, respectively, indicating that the current bitstream does not contain the reconstruction information of the current CU. The encoder encodes the second value in the bitstream of permission level 2, indicating that the current bitstream contains the reconstruction information of the current CU, and the reconstruction information corresponding to the current CU can be parsed from the current bitstream. The encoder encodes the first value in the bitstream of permission level 3, indicating that the current bitstream does not contain the reconstruction information of the current CU. When the decoder parses the bitstream of permission level 2, since the bitstream of permission level 2 contains the second value, the decoding user with a target permission level greater than or equal to 2 can continue decoding the reconstruction information of the current CU from the bitstream of permission level 2. When the decoder parses the bitstream of permission level not equal to 2, since the bitstream contains the first value, this CU is skipped during reconstruction, and the decoding proceeds to the parsing of the next CU.

[0198] Example 5: To design permission-protected syntax elements, permission-protected syntax elements and region-of-interest (ROI) syntax elements are combined. Before obtaining the bitstreams of multiple permission levels corresponding to the target image, the permission-protected parameters and ROI parameters corresponding to the target image can be decoded from the high-level syntax corresponding to the target image. This high-level syntax can include, but is not limited to: sequence-level high-level syntax, sequence-level extended data high-level syntax, image-level high-level syntax, image-level extended data high-level syntax, slice-level high-level syntax, tile-level high-level syntax, and patch-level high-level syntax.

[0199] For ease of distinction, the high-level syntax used by the access control parameters can be referred to as the first syntax, and the high-level syntax used by the region of interest parameters can be referred to as the second syntax. The first syntax used by the access control parameters and the second syntax used by the region of interest parameters may differ, or they may be partially the same, or they may be completely identical.

[0200] In one possible implementation, the access control parameters may include, but are not limited to, at least one of the following: sequence header switch identifier, image header switch identifier, enhancement layer switch identifier, access control level classification information, access control level predictability identifier, probability model update identifier, and CU boundary filtering parameters. Region of interest parameters may include, but are not limited to, at least one of the following: sequence header switch identifier, image header switch identifier, enhancement layer switch identifier, region category classification information, region category predictability identifier, probability model update identifier, and CU boundary filtering parameters. Of course, the above are merely examples of parameters and are not intended to limit the scope of the implementation.

[0201] For example, the syntax elements of the Region of Interest (ROI) parameter may include: a sequence header switch identifier for the RIO, an image header switch identifier, an enhancement layer switch identifier, region category classification information (such as block-level / CU-level classification information), and an identifier indicating whether the region category of each CU block is predictable. In addition, the image header may include some extra syntax elements, such as whether the probability model updates the identifier when encoding and decoding the CU category identifier, and CU boundary filtering parameters (CU boundary filtering parameters are used to perform different filtering processes on the boundaries of different CU categories). Of course, the above are just a few examples of RIO parameters.

[0202] For example, the syntax elements of the access control parameters may include: sequence header switch identifier, image header switch identifier, enhancement layer switch identifier, access control level classification information (i.e., block-level / CU-level access control), and whether the access control level is predictable. Based on this, additional syntax elements can be designed for access control, such as access control level probability model updates (i.e., whether the probability model is updated), CU boundary filtering parameters (CU boundary filtering parameters are used to perform different filtering processes on CU boundaries for different access control levels), and mosaic information for different access control levels. There are no restrictions on these access control parameters.

[0203] To combine access protection with the syntax elements of regions of interest, the following approach can be used:

[0204] Method 1: The first syntax used for the access control parameters differs from the second syntax used for the region of interest (ROI) parameters. For example, the difference between the first and second syntaxes may include at least one of the following: The sequence header switch identifier of the access control parameters uses a different syntax than the sequence header switch identifier of the ROI parameters. The image header switch identifier of the access control parameters uses a different syntax than the image header switch identifier of the ROI parameters. The enhancement layer switch identifier of the access control parameters uses a different syntax than the enhancement layer switch identifier of the ROI parameters. The access control level classification information of the access control parameters uses a different syntax than the region category classification information of the ROI parameters. The access control level predictability identifier of the access control parameters uses a different syntax than the region category predictability identifier of the ROI parameters. The probability model update status identifier of the access control parameters uses a different syntax than the probability model update status identifier of the ROI parameters. The CU boundary filtering parameters of the access control parameters use a different syntax than the CU boundary filtering parameters of the ROI parameters.

[0205] For example, the first syntax used by the permission protection parameters and the second syntax used by the region of interest parameters are independent of each other. That is, the syntax elements used by the permission protection parameters and the region of interest parameters are completely independent. In this case, when the two tools (i.e. permission protection function and region of interest processing function) are started at the same time, when parsing the syntax elements of the CU, both the region of interest parameters and the permission protection parameters of the CU need to be parsed.

[0206] For example, when two tools are running simultaneously, during the parsing process, the parameters parsed second can refer to the parameters parsed first. For instance, if the Region of Interest (ROI) parameter is parsed first, followed by the Permission Protection parameter, the parsing result of the ROI parameter can be referenced when parsing the Permission Protection parameter. Similarly, if the Permission Protection parameter is parsed first, followed by the ROI parameter, the parsing result of the Permission Protection parameter can be referenced when parsing the ROI parameter. For example, if the CU's ROI category is 5 and the CU's permission level is 5, then the tool checks if the CU level equals the CU category. If they do, the CU level is directly set to the CU category; otherwise, the CU permissions are parsed, or the difference between the CU permissions and the category permissions is parsed.

[0207] Method 2: The first syntax used by the access control parameters is the same as the second syntax used by the region of interest (ROI) parameters. For example, the same first and second syntax may include, but is not limited to, at least one of the following: the sequence header switch identifier of the access control parameters uses a different syntax than the sequence header switch identifier of the ROI parameters, or the sequence header switch identifier of the access control parameters uses the same syntax. The image header switch identifier of the access control parameters uses a different syntax than the image header switch identifier of the ROI parameters, or the image header switch identifier of the access control parameters uses the same syntax. The enhancement layer switch identifier of the access control parameters uses a different syntax than the enhancement layer switch identifier of the ROI parameters, or the enhancement layer switch identifier of the access control parameters uses the same syntax. The access control level classification information of the access control parameters uses a different syntax than the region category classification information of the ROI parameters, or the access control level classification information of the access control parameters uses the same syntax. The syntax for the predictability of the permission level in the permission protection parameter and the predictability of the region category in the region of interest parameter can be different, or they can be the same. Similarly, the syntax for the update status of the probability model in the permission protection parameter and the update status of the probability model in the region of interest parameter can be different, or they can be the same. Finally, the syntax for the CU boundary filtering parameters in the permission protection parameter and the CU boundary filtering parameters in the region of interest parameter can be different, or they can be the same.

[0208] In one possible implementation, the first and second syntax portions being identical may include, but are not limited to, at least one of the following: the sequence header switch identifier of the permission protection parameter and the sequence header switch identifier of the region of interest parameter use different syntaxes; the image header switch identifier of the permission protection parameter and the image header switch identifier of the region of interest parameter use different syntaxes; the enhancement layer switch identifier of the permission protection parameter and the enhancement layer switch identifier of the region of interest parameter use the same syntax; the permission level classification information of the permission protection parameter and the region category classification information of the region of interest parameter use the same syntax; the permission level predictability identifier of the permission protection parameter and the region category predictability identifier of the region of interest parameter use the same syntax; the probability model update status identifier of the permission protection parameter and the probability model update status identifier of the region of interest parameter use the same syntax; the CU boundary filtering parameter of the permission protection parameter and the CU boundary filtering parameter of the region of interest parameter use the same syntax. Of course, the above are merely examples and are not intended to limit the scope.

[0209] For example, the first syntax used for permission protection parameters and the second syntax used for region of interest (ROI) parameters share a common but independent portion. The independent syntax elements can include sequence header switches and image header switches, while the remaining syntax elements can be shared. In this case, when both tools (i.e., permission protection function and ROI processing function) are running simultaneously, parsing the CU's syntax elements requires parsing both the CU's ROI parameters and permission protection parameters for the independent syntax elements. For the shared syntax elements, only one set of parameters needs to be parsed; based on this set of parameters, both the CU's ROI parameters and permission protection parameters can be obtained.

[0210] Method 3: The first syntax used for the permission protection parameters is exactly the same as the second syntax used for the region of interest (ROI) parameters. The first and second syntaxes being identical includes, but is not limited to, at least one of the following: the sequence header switch identifier of the permission protection parameters uses the same syntax as the sequence header switch identifier of the ROI parameters; the image header switch identifier of the permission protection parameters uses the same syntax as the image header switch identifier of the ROI parameters; the enhancement layer switch identifier of the permission protection parameters uses the same syntax as the enhancement layer switch identifier of the ROI parameters; the permission level classification information of the permission protection parameters uses the same syntax as the region category classification information of the ROI parameters; the permission level predictability identifier of the permission protection parameters uses the same syntax as the region category predictability identifier of the ROI parameters; the probability model update status identifier of the permission protection parameters uses the same syntax as the probability model update status identifier of the ROI parameters; and the CU boundary filtering parameters of the permission protection parameters use the same syntax as the CU boundary filtering parameters of the ROI parameters.

[0211] For example, the first syntax used for permission protection parameters is exactly the same as the second syntax used for region of interest parameters. When both tools (i.e., permission protection function and region of interest processing function) are enabled simultaneously, only one set of parameters needs to be parsed when parsing the syntax elements of the CU. Based on this set of parameters, both the region of interest parameters and the permission protection parameters of the CU can be obtained. Alternatively, the permission protection parameters and the region of interest parameters can be completely shared, meaning the two tools are strongly bound together, and both tools can be enabled and disabled together.

[0212] In methods 2 and 3, if the sequence header switch identifier of the permission protection parameter and the sequence header switch identifier of the region of interest parameter use the same syntax, when the syntax is a first value, it is determined that the sequence-level switch control information allows permission protection and region of interest processing to be enabled; when the syntax is a second value, it is determined that the sequence-level switch control information prohibits permission protection and region of interest processing to be enabled. The first and second values ​​can be configured arbitrarily, such as the first value being 1 and the second value being 0.

[0213] In methods 2 and 3, if the image header switch identifier of the permission protection parameter and the image header switch identifier of the region of interest parameter use the same syntax, when the syntax is a first value, it is determined that the image-level switch control information allows permission protection and region of interest processing to be enabled; when the syntax is a second value, it is determined that the image-level switch control information prohibits permission protection and region of interest processing to be enabled. The first and second values ​​can be configured arbitrarily, such as the first value being 1 and the second value being 0.

[0214] In methods 2 and 3, if the enhancement layer switch identifier for the access control parameter and the enhancement layer switch identifier for the region of interest parameter use the same syntax, when the syntax is the first value, it is determined that access control and region of interest processing are enabled in the enhancement layer; when the syntax is the second value, it is determined that access control and region of interest processing are disabled in the enhancement layer. For example, the first value can be 1, and the second value can be 0.

[0215] In methods 2 and 3, if the predictability flag for the permission level of the permission protection parameter and the predictability flag for the region category of the region of interest parameter use the same syntax, when the syntax is in the first value, it is determined that the permission level can be predicted based on the permission level of the preceding CU, and the region category can be predicted based on the region category of the preceding CU. When the syntax is in the second value, it is determined that the permission level and region category are unpredictable, that is, the permission level cannot be predicted based on the permission level of the preceding CU, and the region category cannot be predicted based on the region category of the preceding CU. For example, the first value can be 1, and the second value can be 0.

[0216] In methods 2 and 3, if the probability model update flag for the access control parameter and the probability model update flag for the region of interest parameter use the same syntax, then when the syntax has a first value, it determines whether to update the probability model in the access control process and the probability model in the region of interest processing process; when the syntax has a second value, it determines whether to not update the probability model in the access control process and the probability model in the region of interest processing process. For example, the first value can be 1, and the second value can be 0.

[0217] In methods 2 and 3, if the permission level classification information of the permission protection parameter and the region category classification information of the region of interest parameter use the same syntax, then the classification identifier is parsed from the syntax, and the permission level and region category are determined based on the classification identifier; or, the classification identifier is parsed from the syntax, the permission level is determined based on the classification identifier, and the region category is determined based on the permission level and the mapping relationship; or, the classification identifier is parsed from the syntax, the region category is determined based on the classification identifier, and the permission level is determined based on the region category and the mapping relationship; wherein, the mapping relationship represents the mapping relationship between the permission level and the region category.

[0218] For example, if block-level CU permission resolution and block-level (CU-level) classification share the same syntax element—that is, if the permission level classification information of the permission protection parameter and the region category classification information of the region of interest parameter use the same syntax—then, if the number of permission levels is the same as the number of region categories, the classification identifier is parsed from this syntax, and the permission level and region category are determined based on this classification identifier. For instance, if the classification identifier is 0, the permission level is the first permission level and the region category is the first region category; if the classification identifier is 1, the permission level is the second permission level and the region category is the second region category, and so on.

[0219] If the number of permission levels is different from the number of region categories, a mapping relationship can be obtained, which represents the mapping relationship between permission levels and region categories. For example, if the number of region categories is greater than the number of permission levels, the mapping relationship can represent mapping region categories to permission levels. For instance, if there are 8 region categories [0, 1, 2, 3, 4, 5, 6, 7], and different region categories represent different levels of interest in the region, and the number of permission levels is 2, then the mapping relationship is [0, 0, 0, 0, 1, 1, 1, 1], meaning that region categories 0, 1, 2, and 3 correspond to permission level 0, and region categories 4, 5, 6, and 7 correspond to permission level 1. If the number of permission levels is greater than the number of region categories, the mapping relationship can represent mapping permission levels to region categories. For example, if there are 8 permission levels [0, 1, 2, 3, 4, 5, 6, 7], and the number of region categories is 2, then the mapping relationship is [0, 0, 0, 0, 1, 1, 1, 1], which means that permission levels 0, 1, 2, and 3 correspond to region category 0, and permission levels 4, 5, 6, and 7 correspond to region category 1.

[0220] If the number of region categories exceeds the number of permission levels, the category identifier is parsed from the syntax, the region category is determined based on the category identifier, and the permission level is determined based on the region category and the mapping relationship. For example, if the category identifier is 0, the region category is the first region category 0, and querying the mapping relationship through region category 0 yields permission level 0. If the category identifier is 5, the region category is the fifth region category 4, and querying the mapping relationship through region category 4 yields permission level 1, and so on.

[0221] If the number of permission levels exceeds the number of region categories, the classification identifier is parsed from the syntax, the permission level is determined based on the classification identifier, and the region category is determined based on the permission level and the mapping relationship. For example, if the classification identifier is 0, the permission level is the first permission level 0, and the mapping relationship is queried through permission level 0 to obtain region category 0. If the classification identifier is 5, the permission level is the fifth permission level 4, and the mapping relationship is queried through permission level 4 to obtain region category 1, and so on.

[0222] Regarding this mapping relationship, it can be obtained by the decoding end from the current syntax, from other syntaxes, or pre-configured on the decoding end; the source of this mapping relationship is not restricted. For example, if the number of permission levels is not equal to the number of region categories, a syntax element can be added to the image header to mark the mapping relationship between permission levels and region categories. After parsing the region category (or permission level), the permission level (or region category) is obtained according to the mapping relationship.

[0223] In methods 2 and 3, if the CU boundary filtering parameters of the access control parameters and the CU boundary filtering parameters of the region of interest parameters use the same syntax, then candidate filtering parameters are parsed from this syntax, and the target filtering parameters for access control and the region of interest are determined based on these candidate filtering parameters; or, a first candidate filtering parameter is determined based on the access control level and the acquired first mapping relationship, and a second candidate filtering parameter is determined based on the region category and the acquired second mapping relationship. The target filtering parameters for access control are determined based on the first candidate filtering parameters, and the target filtering parameters for the region of interest are determined based on the second candidate filtering parameters; wherein, the first mapping relationship includes the mapping relationship between the access control level and the filtering parameters, and the second mapping relationship includes the mapping relationship between the region category and the filtering parameters.

[0224] For example, when filtering, using the same filtering process, the image header adds a syntax element representing the mapping relationship between the permission level and the region category filtering process. During filtering, two sets of filtering parameters may be derived, and one set may be selected to be used, or a weighted sum of the two parameters may be used as the final filtering parameters.

[0225] For example, the decoding end can directly parse candidate filtering parameters from this syntax, and it can also determine reference filtering parameters based on the current CU's image content; this process is not restricted. Based on the candidate filtering parameters and the reference filtering parameters, the target filtering parameters for access protection and the target filtering parameters for the region of interest can be determined. The access-protected region to be filtered is then filtered based on the access-protected target filtering parameters, and the region to be filtered is then filtered based on the region of interest's target filtering parameters.

[0226] For example, the decoding end can obtain the first and second mapping relationships. These relationships can be obtained from the current syntax, from other syntaxes, or pre-configured on the decoding end. The source of these first and second mapping relationships is not restricted. For instance, the image header adds a syntax element for mapping relationships, and the first and second mapping relationships are decoded from that syntax element.

[0227] The first mapping relationship can include the mapping relationship between permission levels and filtering parameters. After obtaining the permission level for permission protection, the decoding end can query the first mapping relationship through the permission level to obtain the first candidate filtering parameters for permission protection. The second mapping relationship can include the mapping relationship between region categories and filtering parameters. After obtaining the region category of the region of interest, the decoding end can query the second mapping relationship through the region category to obtain the second candidate filtering parameters for processing the region of interest.

[0228] The decoding end can also determine reference filtering parameters based on the current CU's image content. The decoding end can determine the target filtering parameters for access protection based on the first candidate filtering parameters and the reference filtering parameters; alternatively, the decoding end can perform operations (such as weighted operations) on the first and second candidate filtering parameters to obtain weighted candidate filtering parameters, and determine the target filtering parameters for access protection based on the weighted candidate filtering parameters and the reference filtering parameters. Then, the access-protected region to be filtered is filtered based on the target filtering parameters for access protection. Furthermore, the decoding end can determine the target filtering parameters for the region of interest based on the second candidate filtering parameters and the reference filtering parameters; alternatively, the decoding end can perform operations (such as weighted operations) on the first and second candidate filtering parameters to obtain weighted candidate filtering parameters, and determine the target filtering parameters for the region of interest based on the weighted candidate filtering parameters and the reference filtering parameters. Then, the region to be filtered for processing the region of interest is filtered based on the target filtering parameters for the region of interest.

[0229] In one possible implementation, for methods 1, 2, and 3, during the decoding process of the current CU, if the current CU belongs to both the region of interest (ROI) and the high-priority region (HPR), the reconstruction information of the current CU is parsed from the HPR bitstream. If the current CU belongs only to the ROI but not to the HPR, then during the reconstruction process, the current CU is restricted from referencing other CUs. If the current CU does not belong to the ROI but belongs to the HPR, and if the current CU chooses to skip parsing, then if the current CU is performing intra-frame prediction, the predicted pixels are obtained from the current frame based on the prediction mode in the HPR bitstream; if the current CU is performing inter-frame prediction, the predicted pixels are obtained from the reference frame using the prediction mode (such as the default prediction mode or a prediction mode obtained through some method), and these predicted pixels are used as the reconstructed pixels.

[0230] Example 6: In Examples 1 and 2, the encoding end sends multiple permission level bitstreams corresponding to the target image to the decoding end. The decoding end obtains the multiple permission level bitstreams corresponding to the target image. The packaging method for different permission level bitstreams can be based on patch-level packaging. For example, if there are K permission levels, where K is a positive integer greater than 1, the multiple permission level bitstreams corresponding to the target image (e.g., a patch-level image) include K permission level RBSP bitstreams corresponding to the target image. The K permission level RBSP bitstreams are located in K NAL units, and RBSP bitstreams of different permission levels are located in different NAL units. For example, when the frame-level permission protection switch is enabled, each patch-level image corresponds to at least two RBSP bitstream packets, and these at least two RBSP bitstream packets include one low-permission-level RBSP bitstream packet and at least one high-permission-level RBSP bitstream packet. For example, assuming there are K permission levels, specifically 3 permission levels, then each patch-level image corresponds to 3 permission level RBSP bitstream packets. These 3 permission level RBSP bitstream packets are located in 3 NAL units. For instance, the permission level 1 RBSP bitstream packet is located in NAL unit 1, the permission level 2 RBSP bitstream packet is located in NAL unit 2, and the permission level 3 RBSP bitstream packet is located in NAL unit 3. The information within the RBSP bitstream packets can be defined in at least one of the following ways:

[0231] In a high-privilege RBSP bitstream, there is partitioning information for each CU (such as position, shape, etc.).

[0232] Each RBSP stream contains only one stream with the same permissions from a single patch.

[0233] In a low-privilege RBSP bitstream, the partitioning information of each CU (such as position, shape, etc.) is included, while in a high-privilege RBSP bitstream, the partitioning information of each CU is not included. In this case, the low-privilege RBSP bitstream is parsed before parsing the high-privilege RBSP bitstream to obtain the partitioning information of each CU.

[0234] Only in the lowest-priority RBSP bitstream is patch header information included. The header information can include the CU partitioning information and filtering information of the entire patch, and there are no restrictions on the content of this header information.

[0235] The high-priority RBSP streams and low-priority RBSP streams obtained from the patch image are placed in different NAL units. When the low-priority RBSP streams and high-priority RBSP streams are interleaved for transmission, the high-priority RBSP streams can be transmitted first, or the low-priority RBSP streams can be transmitted first. Therefore, the interleaving order can be: Priority 0 NAL -> Priority 1 NAL -> Priority 2 NAL -> ... -> Priority n NAL, or the interleaving order can be: Priority n NAL -> Priority n-1 NAL -> ... -> Priority 1 NAL -> Priority 0 NAL, or any other order. Here, Priority 0 NAL represents the NAL unit containing the lowest-priority RBSP stream, Priority 1 NAL represents the NAL unit containing the RBSP stream with Priority 1, and so on, with Priority n NAL representing the NAL unit containing the highest-priority RBSP stream. Based on the above interleaving order, when the decoding end obtains the bitstreams of multiple permission levels corresponding to the target image, it can obtain the NAL units corresponding to the RBSP bitstreams of multiple permission levels in an interleaving order from high to low permission level; or, it can obtain the NAL units corresponding to the RBSP bitstreams of multiple permission levels in an interleaving order from low to high permission level; or, it can obtain the NAL units corresponding to the RBSP bitstreams of multiple permission levels in an interleaving order according to the preset interleaving order of permission levels.

[0236] For low-privilege RBSP streams, if the current CU's privilege level is high, parsing is skipped by default. For high-privilege RBSP streams, information for each CU is parsed. For example, if the decoding user is at a low privilege level (i.e., the target privilege level is low), default values ​​are used to fill in the data when reconstructing a high-privilege CU. If the decoding user is at a high privilege level, the corresponding high-privilege RBSP stream is parsed to obtain the reconstructed value when reconstructing a high-privilege CU. As another example, if the decoding user is at a low privilege level, intra-frame prediction uses a fixed prediction mode to obtain the reconstructed value when reconstructing a high-privilege CU, while inter-frame prediction uses the reconstructed value from the same position in the reference frame. If the decoding user is at a high privilege level, the corresponding high-privilege RBSP stream is parsed to obtain the reconstructed value when reconstructing a high-privilege CU.

[0237] For each permission level, the RBSP stream is encrypted using patch-level encryption, and / or, the RBSP stream is encrypted using CU-level encryption.

[0238] If the RBSP stream is encrypted using patch-level encryption and CU-level encryption, then: when the target permission level (i.e., the user's permission level at the decoding end) is lower than the current patch-level permission, parsing information in the RBSP stream is prohibited; that is, the entire patch cannot obtain high-dimensional reconstruction information. When the target permission level is greater than or equal to the current patch-level permission, parsing of CU information in the RBSP stream continues. When parsing CU information in the RBSP stream, because the RBSP stream is encrypted using CU-level encryption, if the target permission level is lower than the current CU-level permission, parsing of high-dimensional reconstruction information of the current CU is prohibited; that is, the current CU cannot obtain high-dimensional reconstruction information. When the target permission level is greater than or equal to the current CU-level permission, parsing of high-dimensional reconstruction information of the current CU continues; that is, the high-dimensional reconstruction information of the current CU can be parsed. In summary, after obtaining all parsable streams, CU partitioning information and CU permission information within a patch can be parsed from a certain permission-based stream, and high-dimensional reconstruction information can be parsed from the corresponding permission-based stream based on the CU partitioning information and CU permission information. For example, each RBSP stream's encapsulation packet contains two encryption methods: patch-level encryption and CU-level encryption. Each encryption method can contain multiple levels. If patch-level encryption is used, decoding endpoints with a permission level lower than the current patch's permission level cannot parse any information; even the lowest permission level information cannot be parsed. If the decoding endpoint user obtains patch-level permission, they continue to parse the CU information within each RBSP stream. Obtaining patch-level permission implies that the user has already acquired the permission level of the lowest permission level RBSP stream.

[0239] If the RBSP stream is encrypted using patch-level encryption but not CU-level encryption, then: if the target's permission level is lower than the current patch-level permission, parsing the information in the RBSP stream is prohibited; that is, the entire patch cannot obtain high-dimensional reconstruction information. If the target's permission level is greater than or equal to the current patch-level permission, parsing the CU information in the RBSP stream continues. When parsing the CU information in the RBSP stream, since the RBSP stream is not encrypted using CU-level encryption, the high-dimensional reconstruction information of the current CU can be directly parsed without comparing the relationship between the target's permission level and the current CU-level permission; that is, the high-dimensional reconstruction information of the current CU can be parsed.

[0240] If the RBSP stream is not encrypted at the patch level, but at the CU level, then: there is no need to compare the target permission level (i.e., the permission level of the decoding end user) with the current patch-level permission; the CU information in the RBSP stream can be directly parsed. When parsing the CU information in the RBSP stream, because the RBSP stream is encrypted at the CU level, if the target permission level is lower than the current CU level permission, parsing the high-dimensional reconstruction information of the current CU is prohibited; that is, the current CU cannot obtain high-dimensional reconstruction information. If the target permission level is greater than or equal to the current CU level permission, parsing the high-dimensional reconstruction information of the current CU continues, meaning the high-dimensional reconstruction information of the current CU can be parsed.

[0241] Example 7: In Examples 1 and 2, the encoding end sends multiple permission level bitstreams corresponding to the target image to the decoding end, and the decoding end obtains the multiple permission level bitstreams corresponding to the target image. Regarding the packaging method of different permission level bitstreams, it can also be combined with the knowledge base frame bitstream.

[0242] For example, if there are K permission levels, the bitstreams corresponding to multiple permission levels of a target image (such as a patch-level image) include RBSP bitstreams for the K permission levels of the target image. These RBSP bitstreams are located in K NAL units, with RBSP bitstreams for different permission levels located in different NAL units. Furthermore, if there are multiple target images, the knowledge base frame bitstream referenced by the i-th target image is interleaved with the main bitstream of the i-th target image. The main bitstream of the i-th target image includes the RBSP bitstreams for the K permission levels corresponding to the i-th target image, where the i-th target image can be any target image.

[0243] For example, if the knowledge base frame is transmitted separately as a knowledge bitstream, then the transmission of the high-priority RBSP bitstream and the low-priority RBSP bitstream as the main bitstream is independent of the knowledge bitstream. However, if the knowledge bitstream and the main bitstream are interleaved during transmission, an interleaving method can be specified. An example of an interleaving method is that the knowledge base frame bitstream referenced by the i-th target image is interleaved with the main bitstream of the i-th target image. For instance, the knowledge base frame bitstream referenced by the first target image is interleaved with the main bitstream of the first target image, the knowledge base frame bitstream referenced by the second target image is interleaved with the main bitstream of the second target image, and so on.

[0244] For example, the main bitstream of the i-th target image is interleaved with the bitstream of the j-th knowledge base frame, and the j-th knowledge base frame is the reference knowledge base frame for the i-th target image. Alternatively, one knowledge base frame bitstream is interleaved with the main bitstreams of N target images, where N is greater than or equal to 1. This one knowledge base frame serves as the reference knowledge base frame for the N target images, meaning the same knowledge base frame can be referenced by N target images. In summary, the main bitstream of the i-th target image is interleaved with the bitstream of the knowledge base frame referenced by the i-th target image. For instance, if the same knowledge base frame is referenced by one target image, the knowledge bitstream of the first patch is interleaved with the main bitstream of the first frame, the knowledge bitstream of the second patch is interleaved with the main bitstream of the second frame, and so on.

[0245] In one possible implementation, the following interleaving scheme can be used when combining with the knowledge base frame stream:

[0246] Interleaving scheme 1: The knowledge base frame bitstream referenced by the i-th target image is located after the last RBSP bitstream in the K permission levels of the RBSP bitstream corresponding to the i-th target image.

[0247] For example, each patch of the knowledge base image is packaged into an RBSP, which is processed to obtain a VCL NAL unit. The method of interleaving it with the NAL unit generated by each frame's main bitstream is as follows: current frame image header NAL -> permission 0 information NAL in the first patch of the current frame -> permission 1 information NAL in the first patch of the current frame -> ... (multiple permission information, multiple patches) -> knowledge base NAL (i.e., placed after each frame's NAL). In summary, the knowledge base frame bitstream is located after the RBSP bitstream of the last permission level.

[0248] For example, VCL NAL refers to the bitstream obtained by encoding the patch content. In other words, the NAL packaged with header information and the NAL packaged with extension information do not belong to VCL NAL.

[0249] Interleaving scheme 2: The knowledge base frame bitstream referenced by the i-th target image is located before the first RBSP bitstream in the K permission levels of the RBSP bitstream corresponding to the i-th target image.

[0250] For example, each patch of the knowledge base is packaged into an independent VCL NAL and placed before the VCL NAL of each frame. Alternatively, each patch of the knowledge base image is packaged into an RBSP, processed to obtain a VCL NAL unit, and interleaved with the NAL unit generated from each frame's main bitstream using the following method: current frame image header NAL -> knowledge base NAL -> permission 0 information NAL in the first patch of the current frame -> permission 1 information NAL in the first patch of the current frame -> ... (multiple permission information, multiple patches). In summary, the knowledge base frame bitstream is located before the RBSP bitstream of the first permission level.

[0251] Interleaving Scheme 3: If the i-th target image is not the last target image among all target images, then the knowledge base frame bitstream referenced by the i-th target image is located after the last RBSP bitstream among the K permission levels of the RBSP bitstream corresponding to the i-th target image. Alternatively, if the i-th target image is the last target image among all target images, then the knowledge base frame bitstream referenced by the i-th target image is located before the first RBSP bitstream among the K permission levels of the RBSP bitstream corresponding to the i-th target image.

[0252] For example, each patch of the knowledge base is packaged into an independent VCL NAL. When interleaving the VCL NAL generated by a patch other than the last one, it is placed after all the NALs of the corresponding image, such as after the last RBSP stream in the K permission levels of the target image (see interleaving scheme 1). When interleaving the VCL NAL generated by the last patch of the knowledge base, it is placed before the header NAL of the corresponding image, such as before the first RBSP stream in the K permission levels of the target image (see interleaving scheme 2). After decoding and obtaining the information of the last patch, a complete knowledge base frame can be obtained, and the current frame can be referenced from the knowledge base frame.

[0253] Interleaving scheme 4: Place the NAL generated by the knowledge base patch before or after the NAL position in the image header of each frame, or place the NAL generated by the knowledge base patch before or after a certain extended information NAL in each frame.

[0254] As can be seen from the above embodiments, when access protection is enabled, various information access restrictions for images and videos (such as license plates, partial human body areas, screens, and text) can be implemented, improving data security. For example, when an image presents an information security issue, access protection can be applied to the image, allowing users with high access levels to view a large amount or even all of the image information, while users with low access levels can only view a small amount of image information. For instance, the image can be divided into low-access and high-access areas, where low-access users can only view the low-access area, and high-access users can view both the low-access and high-access areas.

[0255] Example 8: An image decoding method is proposed in this application example, see [link to example]. Figure 5 The diagram shown is a flowchart of the method, which is applied to the decoding end (also known as a video decoder). The method includes:

[0256] Step 501: When permission protection is enabled, obtain the lowest permission level bitstream and at least one non-lowest permission level bitstream corresponding to the target image. The target image can be a patch-level image or a frame-level image.

[0257] Step 502: Decode the lowest-level permission bitstream to obtain the first reconstruction information corresponding to the target image, and determine the first reconstructed image corresponding to the target image based on the first reconstruction information.

[0258] Step 503: For each non-minimum privilege level bitstream, decode the non-minimum privilege level bitstream to obtain the second reconstruction information corresponding to the target image, and determine the second reconstruction image corresponding to the target image based on the first reconstruction information and the second reconstruction information corresponding to the target image.

[0259] In one possible implementation, when access protection is enabled, a first access level bitstream and a second access level bitstream corresponding to the target image can be obtained. The target image can be a patch-level image or a frame-level image, and the target image corresponds to at least two spatial domain images, which may include a base layer image and an enhancement layer image. The first access level bitstream is the lowest access level bitstream, and the second access level bitstream is a non-lowest access level bitstream. Then, the first access level bitstream is decoded to obtain first reconstruction information corresponding to the target image, and the base layer reconstructed image corresponding to the target image is determined based on the first reconstruction information. The second access level bitstream is decoded to obtain second reconstruction information corresponding to the target image, and the enhancement layer reconstructed image corresponding to the target image is determined based on the first and second reconstruction information. The resolution of the enhancement layer reconstructed image can be the same as the resolution of the base layer reconstructed image.

[0260] In one possible implementation, when access protection is enabled, a first access level bitstream and a second access level bitstream corresponding to the target image are obtained. The target image is a patch-level image or a frame-level image. The target image corresponds to at least two temporal images, including a k-th temporal image and a (k+i)-th temporal image, where i takes the value 1-M, M represents the total number of second access level bitstreams, the first access level bitstream is the lowest access level bitstream, and the second access level bitstream is a non-lowest access level bitstream. The first access level bitstream is decoded to obtain first reconstruction information corresponding to the target image, and the k-th temporal reconstructed image corresponding to the target image is determined based on the first reconstruction information. The ith second access level bitstream among the M second access level bitstreams is decoded to obtain second reconstruction information corresponding to the target image, and the (k+i)-th temporal image corresponding to the target image is determined based on the first and second reconstruction information.

[0261] For example, the above execution order is merely an example for ease of description. In practical applications, the execution order between steps can be changed, and there is no limitation on this execution order. Moreover, in other embodiments, the steps of the corresponding method are not necessarily executed in the order shown and described in this specification, and the method may include more or fewer steps than described in this specification. Furthermore, a single step described in this specification may be broken down into multiple steps in other embodiments; multiple steps described in this specification may also be combined into a single step in other embodiments.

[0262] This application proposes an image encoding method, see [link to relevant documentation] Figure 6 The diagram shown is a flowchart of the method, which is applied at the encoding end (also known as a video encoder). The method includes:

[0263] Step 601: When permission protection is enabled, obtain the lowest permission level bitstream and at least one non-lowest permission level bitstream corresponding to the target image. The target image can be a patch-level image or a frame-level image.

[0264] Step 602: Encode the first reconstruction information corresponding to the target image in the lowest privilege level bitstream; wherein, the first reconstruction information is used to determine the first reconstructed image corresponding to the target image.

[0265] Step 603: For each non-lowest privilege level bitstream, encode the second reconstruction information corresponding to the target image in the non-lowest privilege level bitstream; wherein, the first reconstruction information corresponding to the target image and the second reconstruction information corresponding to the target image are used to determine the second reconstructed image corresponding to the target image.

[0266] In one possible implementation, when permission protection is enabled, a first permission level bitstream and a second permission level bitstream corresponding to the target image can be obtained. The target image can be a patch-level image or a frame-level image, and the target image can correspond to at least two spatial domain images, which can include a base layer image and an enhancement layer image. The first permission level bitstream can be the lowest permission level bitstream, and the second permission level bitstream can be a non-lowest permission level bitstream. First reconstruction information corresponding to the target image is encoded in the first permission level bitstream, and this first reconstruction information is used to determine the base layer reconstructed image corresponding to the target image. Second reconstruction information corresponding to the target image is encoded in the second permission level bitstream, and this first and second reconstruction information are used to determine the enhancement layer reconstructed image corresponding to the target image. The resolution of the enhancement layer reconstructed image is the same as the resolution of the base layer reconstructed image.

[0267] In one possible implementation, when permission protection is enabled, a first permission level bitstream and a second permission level bitstream corresponding to the target image are acquired. The target image is a patch-level image or a frame-level image. The target image corresponds to at least two temporal images, including a k-th temporal image and a (k+i)-th temporal image, where i takes the value 1-M, and M represents the total number of second permission level bitstreams. The first permission level bitstream is the lowest permission level bitstream, and the second permission level bitstream is a non-lowest permission level bitstream. First reconstruction information corresponding to the target image is encoded in the first permission level bitstream. This first reconstruction information is used to determine the k-th temporal reconstructed image corresponding to the target image. Second reconstruction information corresponding to the target image is encoded in the i-th second permission level bitstream among the M second permission level bitstreams. The first and second reconstruction information are used to determine the (k+i)-th temporal image corresponding to the target image.

[0268] For example, the above execution order is merely an example for ease of description. In practical applications, the execution order between steps can be changed, and there is no limitation on this execution order. Moreover, in other embodiments, the steps of the corresponding method are not necessarily executed in the order shown and described in this specification, and the method may include more or fewer steps than described in this specification. Furthermore, a single step described in this specification may be broken down into multiple steps in other embodiments; multiple steps described in this specification may also be combined into a single step in other embodiments.

[0269] As can be seen from the above technical solutions, in this embodiment of the application, when permission protection is enabled, the bitstreams with different permissions can be divided into multiple frames for decoding, enabling various information permission protection for images and videos (such as license plates, partial areas of the human body, screen, text, etc.), thereby improving data security. For example, when there are information security issues with images, permission protection can be implemented for the images, allowing users with high permission levels to view a large amount or even all of the image information, while users with low permission levels can only view a small amount of image information.

[0270] Example 9: In Example 8, for the same target image (such as a patch-level image or a frame-level image), a frame-level permission protection bitstream distribution based on the spatial domain can be designed. When implementing permission protection, bitstreams with different permissions can be divided into multiple frames for decoding, and non-lowest permission bits can be treated as a spatial domain enhancement layer.

[0271] For example, the encoding end can obtain the lowest-level permission bitstream (i.e., the first-level permission bitstream) and at least one non-lowest-level permission bitstream (i.e., the second-level permission bitstream) corresponding to the target image, and send the lowest-level permission bitstream and at least one non-lowest-level permission bitstream to the decoding end. The decoding end obtains the lowest-level permission bitstream (i.e., the first-level permission bitstream) and at least one non-lowest-level permission bitstream (i.e., the second-level permission bitstream) corresponding to the target image. The target image can correspond to at least two frames of spatial domain images (i.e., at least two frames of spatial domain images generated based on the same target image), and the lowest-level permission bitstream and at least one non-lowest-level permission bitstream are bitstreams corresponding to the same target image; that is, each frame of spatial domain image corresponds to one bitstream. The at least two frames of spatial domain images can include a base layer image and an enhancement layer image.

[0272] For example, if the target image corresponds to at least two spatial domain images, that is, at least two different spatial domain images are generated based on the target image, the decoding end can decode the lowest-authority bitstream to obtain the first reconstruction information, and determine the first reconstructed image based on the first reconstruction information. The first reconstructed image can be a basic layer reconstructed image, that is, the decoding end decodes the lowest-authority bitstream to obtain the basic layer reconstructed image.

[0273] The decoding end can decode non-minimum privilege level bitstreams to obtain second reconstruction information. Based on the first and second reconstruction information, a second reconstructed image is determined. This second reconstructed image can be an enhancement layer reconstructed image. That is, the decoding end decodes non-minimum privilege level bitstreams to obtain enhancement layer reconstructed images. Specifically, when the target image corresponds to M non-minimum privilege level bitstreams, decoding each non-minimum privilege level bitstream yields enhancement layer reconstructed images, resulting in M ​​enhancement layer reconstructed images, where M is a positive integer.

[0274] For example, the resolution of the image reconstructed by the enhancement layer is the same as that of the image reconstructed by the base layer, and the CU partitioning information of the image reconstructed by the enhancement layer is the same as that of the image reconstructed by the base layer.

[0275] For example, each permissioned bitstream (such as the lowest permission level bitstream and at least one non-lowest permission level bitstream) includes CU partitioning information (i.e., the partitioning information of each CU) and CU permission information (i.e., the permission information of each CU), and each permissioned bitstream includes all image headers and patch header information.

[0276] For example, during prediction, the enhancement layer reconstruction process is allowed to reference the base layer, but the base layer reconstruction process is not allowed to reference the enhancement layer, while the division of the base layer and enhancement layer must be completely consistent. For instance, when decoding a lowest-priority bitstream, the base layer reconstructed image can only be determined based on the first reconstruction information, and cannot be determined by referring to the second reconstruction information; that is, the base layer reconstruction process is not allowed to reference the enhancement layer. When decoding a non-lowest-priority bitstream, the enhancement layer reconstructed image can be determined based on both the first and second reconstruction information; that is, the enhancement layer reconstruction process references the base layer.

[0277] For example, during reconstruction, the reconstructed image of the enhancement layer can be merged with the reconstructed image of the base layer, and the reconstructed content of the CU parsed by the enhancement layer can replace the corresponding positions of the reconstructed image of the base layer, and these positions can be marked. When the base layer is reconstructed in the next frame, the reconstructed image generated by the enhancement layer is not allowed to be referenced.

[0278] For example, the reconstructed image from the enhancement layer can be merged with the reconstructed image from the base layer to obtain a merged image. The target image can include at least one CU (Cubic Array). For each CU, if the reconstructed image from the enhancement layer includes the reconstructed image block corresponding to that CU, then that reconstructed image block replaces the corresponding image content of that CU in the base layer reconstructed image; if the reconstructed image from the enhancement layer does not include the reconstructed image block corresponding to that CU, then the corresponding image content of that CU in the base layer reconstructed image is retained. Based on this, the modified base layer reconstructed image is used as the merged image.

[0279] After obtaining the merged image, the first type of CU in the merged image can be labeled. The first type of CU is the CU whose image content comes from the image reconstructed by the enhancement layer. However, when generating the next base layer reconstructed image, it is prohibited to refer to the first type of CU in the merged image to generate the next base layer reconstructed image.

[0280] For example, during reconstruction, both the enhancement layer reconstructed image and the base layer reconstructed image can be retained simultaneously, i.e., two frames of images can be retained at the same time. When reconstructing the base layer in the next frame, the base layer reconstructed image of the previous frame can be referenced. When reconstructing the enhancement layer in the next frame, it is possible to reference the enhancement layer reconstructed image of the previous frame, the base layer reconstructed image of the current frame, and the base layer reconstructed image of the previous frame.

[0281] Example 10: In Example 8, for the same target image (such as a patch-level image or a frame-level image), a time-domain-based frame-level permission protection bitstream distribution can be designed. When implementing permission protection, bitstreams with different permissions can be divided into multiple frames for decoding, and non-lowest permission bits can be processed as time-domain images.

[0282] For example, the encoding end can obtain the lowest-level permission bitstream (i.e., the first-level permission bitstream) and at least one non-lowest-level permission bitstream (i.e., the second-level permission bitstream) corresponding to the target image, and send the lowest-level permission bitstream and at least one non-lowest-level permission bitstream to the decoding end. The decoding end obtains the lowest-level permission bitstream (i.e., the first-level permission bitstream) and at least one non-lowest-level permission bitstream (i.e., the second-level permission bitstream) corresponding to the target image. The target image can correspond to at least two temporal images (i.e., at least two temporal images generated based on the same target image), and the lowest-level permission bitstream and at least one non-lowest-level permission bitstream are bitstreams corresponding to the same target image; that is, each temporal image corresponds to one bitstream. The at least two temporal images may include the k-th temporal image and the (k+i)-th temporal image, where i takes the value 1-M, and M represents the total number of non-lowest-level permission bitstreams.

[0283] For example, if the target image corresponds to at least two temporal images, that is, at least two temporal images that are different in the temporal domain are generated based on the target image, the decoding end can decode the lowest privilege level bitstream to obtain the first reconstruction information, and determine the first reconstructed image based on the first reconstruction information. The first reconstructed image can be the k-th temporal reconstructed image, that is, the decoding end decodes the lowest privilege level bitstream to obtain the k-th temporal reconstructed image.

[0284] The decoding end decodes the i-th non-lowest privilege level bitstream among M non-lowest privilege level bitstreams to obtain second reconstruction information. Based on the first and second reconstruction information, a second reconstructed image is determined. This second reconstructed image can be the (k+i)-th frame temporal reconstructed image. That is, the decoding end decodes the i-th non-lowest privilege level bitstream to obtain the (k+i)-th frame temporal reconstructed image, where i is a positive integer. Specifically, when the target image corresponds to M non-lowest privilege level bitstreams, the first non-lowest privilege level bitstream is decoded to obtain the (k+1)-th frame temporal reconstructed image, the second non-lowest privilege level bitstream is decoded to obtain the (k+2)-th frame temporal reconstructed image, and so on, to obtain M frames of temporal reconstructed images of the M non-lowest privilege level bitstreams.

[0285] For example, the resolution of the temporal reconstructed image in frame (k+i) is the same as that in frame k, and the CU partitioning information of the temporal reconstructed image in frame (k+i) is the same as that in frame k. For instance, the resolutions of the temporal reconstructed images in frames k, (k+1), (k+2), ..., (k+M) can all be the same, and the CU partitioning information of the temporal reconstructed images in frames k, (k+1), (k+2), ..., (k+M) can all be the same.

[0286] For example, the lowest-authority bitstream is decoded to obtain the temporal reconstructed image of frame k, and the images decoded from non-lowest-authority bitstreams are used as the temporal reconstructed images of frames k+1, k+2, ..., k+M. Each authorization bitstream (such as the lowest-authority bitstream and at least one non-lowest-authority bitstream) includes CU partitioning information (i.e., the partitioning information of each CU), and each authorization bitstream includes all image headers and patch header information.

[0287] For example, during prediction, the reconstruction process of the temporal reconstructed image of frame (k+i) is allowed to refer to the temporal reconstructed image of frame k, but not the reconstruction process of the temporal reconstructed image of frame k is allowed to refer to the temporal reconstructed image of frame (k+i). For instance, when decoding a lowest-priority bitstream, the temporal reconstructed image of frame k can only be determined based on the first reconstruction information, and cannot be determined by referring to the temporal reconstructed image of frame (k+i). In other words, the reconstruction process of the temporal reconstructed image of frame k is not allowed to refer to the temporal reconstructed image of frame (k+i).

[0288] When decoding a non-lowest privilege level bitstream, the temporal reconstructed image of frame (k+i) can be determined based on the first reconstruction information and the second reconstruction information. That is, the reconstruction process of the temporal reconstructed image of frame (k+i) refers to the temporal reconstructed image of frame k. For example, the target image may include at least one CU. For each CU, the second reconstruction information corresponding to the CU can be obtained by decoding the i-th non-lowest privilege level bitstream (i.e., the second privilege level bitstream). Based on the first reconstruction information and the second reconstruction information corresponding to the CU, the reconstructed image block corresponding to the CU is determined. The temporal reconstructed image of frame (k+i) is generated based on the reconstructed image block corresponding to each CU.

[0289] For example, during prediction, the reconstruction process of the temporal reconstructed image of frame (k+1) is allowed to reference the temporal reconstructed image of frame k, but the temporal reconstructed image of frame k is not allowed to reference the temporal reconstructed image of frame (k+1) or frame (k-1). However, the temporal reconstructed image of frame k is allowed to reference the images of frames (k+M+1) or (kM), that is, it is allowed to reference the temporal reconstructed images corresponding to other target images. The reconstruction process of the temporal reconstructed image of frame (k+M) is allowed to reference the temporal reconstructed images of frames k, k+1, ..., k+M-1.

[0290] For example, during reconstruction, only the temporal reconstructed image of frame k is retained. For the temporal reconstructed images of frames k+1 to k+M, the reconstructed pixels marked as CUs with protected permissions directly replace the content at the corresponding position in frame k (requiring that the CU partitioning information of frame k and frame (k+i) are completely consistent). When predicting and referencing, it is stipulated that frame k cannot refer to the reconstructed pixels generated by frames k+1 to k+M. Therefore, the positions of these image information generated by frames k+1 to k+M are recorded and saved to restrict low permissions from referencing high permissions.

[0291] Example 11: In Example 8, for the same target image (such as a patch-level image or a frame-level image), a time-domain-based frame-level permission protection bitstream distribution can be designed. When implementing permission protection, bitstreams with different permissions can be divided into multiple frames for decoding, and non-lowest permission bits can be processed as time-domain images.

[0292] For example, the encoding end can obtain the lowest-level permission bitstream (i.e., the first-level permission bitstream) and at least one non-lowest-level permission bitstream (i.e., the second-level permission bitstream) corresponding to the target image, and send the lowest-level permission bitstream and at least one non-lowest-level permission bitstream to the decoding end. The decoding end obtains the lowest-level permission bitstream (i.e., the first-level permission bitstream) and at least one non-lowest-level permission bitstream (i.e., the second-level permission bitstream) corresponding to the target image. The target image can correspond to at least two temporal images (i.e., at least two temporal images generated based on the same target image), and the lowest-level permission bitstream and at least one non-lowest-level permission bitstream are bitstreams corresponding to the same target image; that is, each temporal image corresponds to one bitstream. The at least two temporal images may include the k-th temporal image and the (k+i)-th temporal image, where i takes the value 1-M, and M represents the total number of non-lowest-level permission bitstreams.

[0293] For example, if the target image corresponds to at least two temporal images, that is, at least two temporal images that are different in the temporal domain are generated based on the target image, the decoding end can decode the lowest privilege level bitstream to obtain the first reconstruction information, and determine the first reconstructed image based on the first reconstruction information. The first reconstructed image can be the k-th temporal reconstructed image, that is, the decoding end decodes the lowest privilege level bitstream to obtain the k-th temporal reconstructed image.

[0294] The decoding end decodes the i-th non-lowest privilege level bitstream among M non-lowest privilege level bitstreams to obtain second reconstruction information. Based on the first and second reconstruction information, a second reconstructed image is determined. This second reconstructed image can be the (k+i)-th frame temporal reconstructed image. That is, the decoding end decodes the i-th non-lowest privilege level bitstream to obtain the (k+i)-th frame temporal reconstructed image, where i is a positive integer. Specifically, when the target image corresponds to M non-lowest privilege level bitstreams, the first non-lowest privilege level bitstream is decoded to obtain the (k+1)-th frame temporal reconstructed image, the second non-lowest privilege level bitstream is decoded to obtain the (k+2)-th frame temporal reconstructed image, and so on, to obtain M frames of temporal reconstructed images of the M non-lowest privilege level bitstreams.

[0295] For example, the resolution of the temporal reconstructed image in frame (k+i) is the same as that in frame k, and the CU partitioning information of the temporal reconstructed image in frame (k+i) is the same as that in frame k. For instance, the resolutions of the temporal reconstructed images in frames k, (k+1), (k+2), ..., (k+M) can all be the same, and the CU partitioning information of the temporal reconstructed images in frames k, (k+1), (k+2), ..., (k+M) can all be the same.

[0296] For example, the k-th frame temporal reconstructed image is obtained by decoding the least privilege bitstream, and the images obtained by decoding non-least privilege bitstreams are used as the k+1, k+2, ..., k+M-th frame temporal reconstructed images.

[0297] For example, each permissioned bitstream (such as the lowest permission level bitstream and at least one non-lowest permission level bitstream) includes CU partitioning information (i.e., the partitioning information of each CU) and CU permission information (i.e., the permission information of each CU). Each permissioned bitstream includes all image headers and patch header information.

[0298] For example, during prediction, the reconstruction process of the temporal reconstructed image of frame (k+i) is allowed to refer to the temporal reconstructed image of frame k, but not the reconstruction process of the temporal reconstructed image of frame k is allowed to refer to the temporal reconstructed image of frame (k+i). For instance, when decoding a lowest-priority bitstream, the temporal reconstructed image of frame k can only be determined based on the first reconstruction information, and cannot be determined by referring to the temporal reconstructed image of frame (k+i). In other words, the reconstruction process of the temporal reconstructed image of frame k is not allowed to refer to the temporal reconstructed image of frame (k+i).

[0299] When decoding a non-minimum privilege level bitstream, the temporal reconstructed image of frame (k+i) can be determined based on the first and second reconstruction information. That is, the reconstruction process of the temporal reconstructed image of frame (k+i) refers to the temporal reconstructed image of frame k. For example, the target image includes a first type of CU and a second type of CU, where the privilege level of the first type of CU is higher than the minimum privilege level, and the privilege level of the second type of CU is the minimum privilege level. Decoding the i-th non-minimum privilege level bitstream (i.e., the second privilege level bitstream) yields the second reconstruction information corresponding to the first type of CU. Based on the first reconstruction information and the second reconstruction information corresponding to the first type of CU, the reconstructed image block corresponding to the first type of CU is determined. The reconstructed image block corresponding to the second type of CU is obtained from the first reconstructed image (i.e., not obtained by decoding the non-minimum privilege level bitstream). Based on this, the temporal reconstructed image of frame (k+i) can be generated based on the reconstructed image blocks corresponding to each first type of CU and each second type of CU.

[0300] For example, during reconstruction, only the temporal reconstructed image of frame k is retained. For the temporal reconstructed images of frames k+1 to k+M, the reconstructed pixels marked as CUs with protected permissions directly replace the content at the corresponding position in frame k (requiring that the CU partitioning information of frame k and frame (k+i) are completely consistent). When predicting and referencing, it is stipulated that frame k cannot refer to the reconstructed pixels generated by frames k+1 to k+M. Therefore, the positions of these image information generated by frames k+1 to k+M are recorded and saved to restrict low permissions from referencing high permissions.

[0301] For example, after generating the (k+i)th frame temporal reconstructed image based on the reconstructed image block corresponding to each first type CU and the reconstructed image block corresponding to each second type CU, the first type CU in the (k+i)th frame temporal reconstructed image is marked. When generating the next kth frame temporal reconstructed image, it is prohibited to refer to the first type CU in the (k+i)th frame temporal reconstructed image to generate the next kth frame temporal reconstructed image.

[0302] For example, the above embodiments can be implemented individually or in combination. For instance, each of embodiments 1-11 can be implemented individually, and at least two embodiments 1-11 can be implemented in combination. For example, in the above embodiments, the content at the encoding end can also be applied to the decoding end, that is, the decoding end can be processed in the same way, and the content at the decoding end can also be applied to the encoding end, that is, the encoding end can be processed in the same way. This will not be repeated here.

[0303] Based on the same application concept as the methods described above, this application also proposes an image decoding device. The device is applied at a decoding end and includes: a memory configured to store video data; and a decoder configured to implement the decoding methods described in embodiments 1-11 above, i.e., the processing flow at the decoding end. For example, in one possible implementation, the decoder is configured to:

[0304] When permission protection is enabled, the bitstreams of one or more permission levels corresponding to the target image are obtained. The target image is a patch-level image or a frame-level image, and the target image includes at least one CU. If the target image includes a CU of a first permission level and a CU of a second permission level, the bitstreams of the multiple permission levels include the bitstreams of the first permission level and the bitstreams of the second permission level. The first permission level is the lowest permission level, and the second permission level is not the lowest permission level.

[0305] Based on the bitstream of the one or more permission levels, decode the CU partitioning information and CU permission information of the target image, and decode each CU of the target image based on the CU partitioning information and CU permission information.

[0306] Alternatively, when permission protection is enabled, a first permission level bitstream and a second permission level bitstream corresponding to the target image are obtained. The target image is a patch-level image or a frame-level image, and the target image corresponds to at least two spatial domain images, which include a base layer image and an enhancement layer image. The first permission level bitstream is the lowest permission level bitstream, and the second permission level bitstream is a non-lowest permission level bitstream. The first permission level bitstream is decoded to obtain first reconstruction information corresponding to the target image, and the base layer reconstructed image corresponding to the target image is determined based on the first reconstruction information. The second permission level bitstream is decoded to obtain second reconstruction information corresponding to the target image, and the enhancement layer reconstructed image corresponding to the target image is determined based on the first and second reconstruction information. The resolution of the enhancement layer reconstructed image is the same as the resolution of the base layer reconstructed image.

[0307] Alternatively, when permission protection is enabled, a first permission level bitstream and a second permission level bitstream corresponding to the target image are obtained, wherein the target image is a patch-level image or a frame-level image; wherein the target image corresponds to at least two temporal images, the at least two temporal images include a k-th temporal image and a (k+i)-th temporal image, where i takes the value 1-M, M represents the total number of second permission level bitstreams, the first permission level bitstream is the lowest permission level bitstream, and the second permission level bitstream is a non-lowest permission level bitstream; the first permission level bitstream is decoded to obtain the first reconstruction information corresponding to the target image, and the k-th temporal reconstructed image corresponding to the target image is determined based on the first reconstruction information; the ith second permission level bitstream among the M second permission level bitstreams is decoded to obtain the second reconstruction information corresponding to the target image, and the (k+i)-th temporal image corresponding to the target image is determined based on the first reconstruction information and the second reconstruction information.

[0308] Based on the same application concept as the methods described above, this application also proposes an image encoding device applied at an encoding end. The device includes: a memory configured to store video data; and an encoder configured to implement the encoding methods described in embodiments 1-11 above, i.e., the processing flow at the encoding end. For example, in one possible implementation, the encoder is configured to:

[0309] When permission protection is enabled, the bitstreams of one or more permission levels corresponding to the target image are obtained. The target image is a patch-level image or a frame-level image, and the target image includes at least one CU. If the target image includes a CU of a first permission level and a CU of a second permission level, the bitstreams of the multiple permission levels include the bitstreams of the first permission level and the bitstreams of the second permission level. The first permission level is the lowest permission level, and the second permission level is not the lowest permission level.

[0310] The CU partitioning information and CU permission information of the target image are encoded in the bitstream of the one or more permission levels; the CU permission information is used to indicate the permission level corresponding to each CU in the target image;

[0311] Based on the CU partitioning information and the CU permission information, each CU of the target image is encoded in the bitstream of the multiple permission levels;

[0312] Send the bitstreams corresponding to multiple permission levels of the target image to the decoding end.

[0313] Alternatively, when permission protection is enabled, the bitstreams of multiple permission levels corresponding to the target image are obtained. The target image is a patch-level image or a frame-level image, and the target image includes at least one CU. If the target image includes a CU of a first permission level and a CU of a second permission level, the bitstreams of the multiple permission levels include the bitstreams of the first permission level and the bitstreams of the second permission level. The first permission level is the lowest permission level, and the second permission level is not the lowest permission level.

[0314] The CU partitioning information and CU permission information of the target image are encoded in the bitstream of the multiple permission levels; the CU permission information is used to indicate the permission level corresponding to each CU in the target image.

[0315] Based on the CU partitioning information and the CU permission information, each CU of the target image is encoded in the bitstream of the multiple permission levels;

[0316] Send the bitstreams corresponding to multiple permission levels of the target image to the decoding end.

[0317] Alternatively, when permission protection is enabled, a first permission level bitstream and a second permission level bitstream corresponding to the target image are obtained, wherein the target image is a patch-level image or a frame-level image; wherein the target image corresponds to at least two temporal images, the at least two temporal images include a k-th temporal image and a (k+i)-th temporal image, where i takes the value 1-M, M represents the total number of second permission level bitstreams, the first permission level bitstream is the lowest permission level bitstream, and the second permission level bitstream is a non-lowest permission level bitstream; the first reconstruction information corresponding to the target image is encoded in the first permission level bitstream, and the first reconstruction information is used to determine the k-th temporal reconstructed image corresponding to the target image;

[0318] The second reconstruction information corresponding to the target image is encoded in the i-th second permission level bitstream among the M second permission level bitstreams; wherein, the first reconstruction information and the second reconstruction information are used to determine the (k+i)-th frame temporal image corresponding to the target image.

[0319] Based on the same concept as the above method, the decoding device (also known as a video decoder) provided in this application embodiment, from a hardware perspective, its hardware architecture diagram can be found in [reference needed]. Figure 7A As shown, it includes: a processor 711 and a machine-readable storage medium 712, the machine-readable storage medium 712 storing machine-executable instructions (image decoding instructions) that can be executed by the processor 711; the processor 711 is used to execute the machine-executable instructions to implement the image decoding methods of embodiments 1-11 of this application.

[0320] Based on the same concept as the above method, the encoding end device (also known as a video encoder) provided in this application embodiment, from a hardware perspective, its hardware architecture diagram can be found in [reference needed]. Figure 7B As shown, it includes: a processor 721 and a machine-readable storage medium 722, the machine-readable storage medium 722 storing machine-executable instructions (image encoding instructions) that can be executed by the processor 721; the processor 721 is used to execute the machine-executable instructions to implement the image encoding methods of embodiments 1-11 of this application.

[0321] Based on the same application concept as the above methods, embodiments of this application also provide a machine-readable storage medium storing a plurality of computer instructions. When the computer instructions are executed by a processor, they can implement the methods disclosed in the above examples of this application, such as the image decoding method or image encoding method in the above embodiments.

[0322] Based on the same concept as the above method, this application also provides a computer application that, when executed by a processor, can implement the image decoding method or image encoding method disclosed in the above examples of this application.

[0323] Based on the same application concept as the above method, this application also proposes an image decoding device. The device is applied to a decoding end and includes: an acquisition module, used to acquire one or more permission level bitstreams corresponding to a target image when permission protection is enabled, wherein the target image is a patch-level image or a frame-level image, and the target image includes at least one CU; wherein, if the target image includes a CU of a first permission level and a CU of a second permission level, the bitstreams of the multiple permission levels include the bitstreams of the first permission level and the bitstreams of the second permission level; wherein, the first permission level is the lowest permission level, and the second permission level is a non-lowest permission level; and a decoding module, used to decode the CU partitioning information and CU permission information of the target image based on the bitstreams of the one or more permission levels, and to decode each CU of the target image based on the CU partitioning information and the CU permission information.

[0324] For example, when the decoding module decodes each CU of the target image based on CU partitioning information and CU permission information, it specifically performs the following: for the current CU to be decoded in the target image, it determines the current permission level corresponding to the current CU based on the CU permission information; if it is determined based on the current permission level that the target permission level has access to the current CU, it decodes the high-dimensional reconstruction information corresponding to the current CU from the target bitstream based on the CU partitioning information, and determines the reconstructed image block corresponding to the current CU based on the high-dimensional reconstruction information; wherein, the target permission level is the permission level of the user accessing the decoding end, and the target bitstream is a bitstream with a permission level less than or equal to the target permission level.

[0325] For example, when the decoding module decodes each CU of the target image based on CU partitioning information and CU permission information, it specifically performs the following: if it is determined based on the current permission level that the target permission level does not have access to the current CU, then it skips decoding the reconstruction information corresponding to the current CU from the target bitstream; it obtains the low-dimensional reconstruction information corresponding to the current CU, and determines the reconstructed image block corresponding to the current CU based on the low-dimensional reconstruction information; wherein, the low-dimensional reconstruction information is a fixed reconstruction value, or, the low-dimensional reconstruction information is a reconstruction value obtained based on intra-frame prediction, or, the low-dimensional reconstruction information is a reconstruction value obtained based on inter-frame prediction.

[0326] For example, when the decoding module decodes the CU partitioning information and CU permission information of the target image based on the bitstreams of the multiple permission levels, it is specifically used to: if the CU partitioning information and CU permission information of the target image are located in the bitstream of the first permission level, then decode the CU partitioning information and the CU permission information from the bitstream of the first permission level.

[0327] For example, when the decoding module decodes the CU partitioning information and CU permission information of the target image based on the multiple permission levels of the bitstream, it is specifically used to: if the CU partitioning information of the target image is located in the bitstream of the first permission level, then decode the CU partitioning information from the bitstream of the first permission level; and / or, if the permission information of the first CU of the target image is located in the bitstream of the permission level less than or equal to the permission level of the first CU, then decode the permission information of the first CU from the bitstream of the permission level less than or equal to the permission level of the first CU; wherein, the first CU is any CU.

[0328] For example, for a bitstream with a permission level lower than that of the first CU, the permission information of the first CU in the bitstream is a first value, which indicates that the bitstream does not contain high-dimensional reconstruction information corresponding to the first CU; or, for a bitstream with a permission level equal to that of the first CU, the permission information of the first CU in the bitstream is a second value, which indicates that the bitstream contains high-dimensional reconstruction information corresponding to the first CU.

[0329] For example, when the decoding module decodes the CU partitioning information and CU permission information of the target image based on the bitstreams of the multiple permission levels, it is specifically used to: if the CU partitioning information and CU permission information of the target image are located in the bitstreams of the first permission level and the second permission level, then decode the CU partitioning information and the CU permission information from the bitstream of the first permission level, or decode the CU partitioning information and the CU permission information from the bitstream of the second permission level.

[0330] For example, if the permission information of the second CU of the target image is located in the bitstream of the first permission level and the second permission level, and the second CU is any CU, then: for a bitstream with a permission level lower than the permission level of the second CU, the permission information of the second CU in the bitstream is a first value, which indicates that the bitstream does not contain high-dimensional reconstruction information corresponding to the second CU; or, for a bitstream with a permission level greater than or equal to the permission level of the second CU, the permission information of the second CU in the bitstream is a second value, which indicates that the bitstream contains high-dimensional reconstruction information corresponding to the second CU; or, for a bitstream with a permission level not equal to the permission level of the second CU, the permission information of the second CU in the bitstream is a first value, which indicates that the bitstream does not contain high-dimensional reconstruction information corresponding to the second CU; or, for a bitstream with a permission level equal to the permission level of the second CU, the permission information of the second CU in the bitstream is a second value, which indicates that the bitstream contains high-dimensional reconstruction information corresponding to the second CU.

[0331] For example, the decoding module is further configured to decode the permission protection parameters and the region of interest parameters corresponding to the target image from the high-level syntax corresponding to the target image; wherein the first syntax used by the permission protection parameters is different from the second syntax used by the region of interest parameters, or the first syntax is partially the same as the second syntax, or the first syntax is completely the same as the second syntax.

[0332] For example, if there are K permission levels, the bitstream of the target image corresponding to multiple permission levels includes the RBSP bitstream of the target image corresponding to the K permission levels. The RBSP bitstream of the K permission levels is located in K NAL units, and the RBSP bitstream of different permission levels is located in different NAL units.

[0333] For example, the decoding module is further configured to, for each permission level RBSP stream, encrypt the RBSP stream using a patch-level encryption method, and / or, encrypt the RBSP stream using a CU-level encryption method; if the RBSP stream is encrypted using a patch-level encryption method, then if the target permission level is less than the current patch-level permission, parsing the information in the RBSP stream is prohibited; if the target permission level is greater than or equal to the current patch-level permission, parsing the CU information in the RBSP stream continues; when parsing the CU information in the RBSP stream, if the RBSP stream is encrypted using a CU-level encryption method, then if the target permission level is less than the current CU-level permission, parsing the high-dimensional reconstruction information of the current CU is prohibited; if the target permission level is greater than or equal to the current CU-level permission, parsing the high-dimensional reconstruction information of the current CU continues.

[0334] For example, when the acquisition module acquires the bitstreams corresponding to multiple permission levels of the target image, it is specifically used to: sequentially acquire the NAL units corresponding to the RBSP bitstreams of multiple permission levels in an interleaving order from high to low permission levels; or, sequentially acquire the NAL units corresponding to the RBSP bitstreams of multiple permission levels in an interleaving order from low to high permission levels; or, sequentially acquire the NAL units corresponding to the RBSP bitstreams of multiple permission levels in a preset interleaving order of permission levels.

[0335] Based on the same application concept as the above method, this application also proposes an image encoding device. The device is applied at an encoding end and includes: an acquisition module, used to acquire one or more bitstreams of permission levels corresponding to a target image when permission protection is enabled, wherein the target image is a patch-level image or a frame-level image, and the target image includes at least one CU; wherein, if the target image includes CUs of a first permission level and CUs of a second permission level, the bitstreams of multiple permission levels include the bitstreams of the first permission level and the bitstreams of the second permission level; the first permission level is the lowest permission level, and the second permission level is a non-lowest permission level; an encoding module, used to encode CU partitioning information and CU permission information of the target image in the bitstreams of the one or more permission levels; the CU permission information is used to indicate the permission level corresponding to each CU in the target image; based on the CU partitioning information and the CU permission information, each CU of the target image is encoded in the bitstreams of multiple permission levels; and a sending module, used to send the bitstreams of multiple permission levels corresponding to the target image to a decoding end.

[0336] Based on the same application concept as the above method, this application also proposes an image decoding device. The device is applied at the decoding end and includes: an acquisition module, used to acquire a first permission level bitstream and a second permission level bitstream corresponding to a target image when permission protection is enabled. The target image is a patch-level image or a frame-level image, and the target image corresponds to at least two spatial domain images, which include a base layer image and an enhancement layer image; wherein the first permission level bitstream is the lowest permission level bitstream, and the second permission level bitstream is a non-lowest permission level bitstream; and a decoding module, used to decode the first permission level bitstream to obtain first reconstruction information corresponding to the target image, and determine the base layer reconstructed image corresponding to the target image based on the first reconstruction information; decode the second permission level bitstream to obtain second reconstruction information corresponding to the target image, and determine the enhancement layer reconstructed image corresponding to the target image based on the first reconstruction information and the second reconstruction information, wherein the resolution of the enhancement layer reconstructed image is the same as the resolution of the base layer reconstructed image.

[0337] For example, the decoding module is further configured to merge the enhancement layer reconstructed image with the base layer reconstructed image to obtain a merged image; wherein, the target image includes at least one CU, and for each CU, if the enhancement layer reconstructed image includes a reconstructed image block corresponding to the CU, then the image content corresponding to the CU in the base layer reconstructed image is replaced by the reconstructed image block; if the enhancement layer reconstructed image does not include the reconstructed image block corresponding to the CU, then the image content corresponding to the CU in the base layer reconstructed image is retained; wherein, the modified base layer reconstructed image is used as the merged image.

[0338] For example, the decoding module is further configured to mark a first type of CU in the merged image, wherein the first type of CU is a CU whose image content originates from the reconstructed image of the enhancement layer; wherein, when generating the reconstructed image of the next base layer, it is prohibited to refer to the first type of CU in the merged image to generate the reconstructed image of the next base layer.

[0339] Based on the same application concept as the above method, this application also proposes an image encoding device. The device is applied at the encoding end and includes: an acquisition module, used to acquire a first permission level bitstream and a second permission level bitstream corresponding to a target image when permission protection is enabled. The target image is a patch-level image or a frame-level image, and the target image corresponds to at least two spatial domain images, which include a base layer image and an enhancement layer image; wherein the first permission level bitstream is the lowest permission level bitstream, and the second permission level bitstream is a non-lowest permission level bitstream; and an encoding module, used to encode first reconstruction information corresponding to the target image in the first permission level bitstream, the first reconstruction information being used to determine the base layer reconstructed image corresponding to the target image; and to encode second reconstruction information corresponding to the target image in the second permission level bitstream, the first reconstruction information and the second reconstruction information being used to determine the enhancement layer reconstructed image corresponding to the target image; wherein the resolution of the enhancement layer reconstructed image is the same as the resolution of the base layer reconstructed image.

[0340] Based on the same application concept as the above method, this application also proposes an image decoding device. The device is applied to a decoding end and includes: an acquisition module, used to acquire a first permission level bitstream and a second permission level bitstream corresponding to a target image when permission protection is enabled, wherein the target image is a patch-level image or a frame-level image; wherein the target image corresponds to at least two temporal images, the at least two temporal images include a k-th temporal image and a (k+i)-th temporal image, i is 1-M, M represents the total number of second permission level bitstreams, the first permission level bitstream is the lowest permission level bitstream, and the second permission level bitstream is a non-lowest permission level bitstream; and a decoding module, used to decode the first permission level bitstream to obtain first reconstruction information corresponding to the target image, and determine the k-th temporal reconstructed image corresponding to the target image based on the first reconstruction information; decode the i-th second permission level bitstream among the M second permission level bitstreams to obtain second reconstruction information corresponding to the target image, and determine the (k+i)-th temporal image corresponding to the target image based on the first reconstruction information and the second reconstruction information.

[0341] For example, the target image includes at least one CU. For each CU, the decoding module is specifically used to decode the i-th second permission level bitstream to obtain the second reconstruction information corresponding to the CU, determine the reconstructed image block corresponding to the CU based on the first reconstruction information and the second reconstruction information corresponding to the CU, and generate the (k+i)-th frame temporal image based on the reconstructed image block corresponding to each CU.

[0342] For example, the target image includes a first type of CU and a second type of CU, where the permission level of the first type of CU is greater than the lowest permission level, and the permission level of the second type of CU is the lowest permission level. The decoding module is specifically used to decode the i-th second permission level bitstream to obtain the second reconstruction information corresponding to the first type of CU, determine the reconstructed image block corresponding to the first type of CU based on the first reconstruction information and the second reconstruction information corresponding to the first type of CU, obtain the reconstructed image block corresponding to the second type of CU from the first reconstructed image, and generate the (k+i)-th frame temporal image based on the reconstructed image block corresponding to each first type of CU and each reconstructed image block corresponding to each second type of CU.

[0343] For example, the decoding module generates the (k+i)th frame temporal image based on the reconstructed image block corresponding to each first type CU and the reconstructed image block corresponding to each second type CU. It is also used to: mark the first type CU in the (k+i)th frame temporal image, and prohibit referencing the first type CU in the (k+i)th frame temporal image when generating the next kth frame temporal image.

[0344] Based on the same application concept as the above method, this application also proposes an image encoding device. The device is applied at the encoding end and includes: an acquisition module, used to acquire a first permission level bitstream and a second permission level bitstream corresponding to a target image when permission protection is enabled, wherein the target image is a patch-level image or a frame-level image; wherein the target image corresponds to at least two temporal images, the at least two temporal images include a k-th temporal image and a (k+i)-th temporal image, i is 1-M, M represents the total number of second permission level bitstreams, the first permission level bitstream is the lowest permission level bitstream, and the second permission level bitstream is a non-lowest permission level bitstream; and an encoding module, used to encode first reconstruction information corresponding to the target image in the first permission level bitstream, the first reconstruction information being used to determine the k-th temporal reconstructed image corresponding to the target image; and to encode second reconstruction information corresponding to the target image in the i-th second permission level bitstream among the M second permission level bitstreams; wherein the first reconstruction information and the second reconstruction information are used to determine the (k+i)-th temporal image corresponding to the target image.

[0345] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. This application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Embodiments of this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0346] The above description is merely an embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of this application should be included within the scope of the claims of this application.

Claims

1. An image decoding method, characterized in that, Applied to the decoding end, the method includes: When permission protection is enabled, the bitstreams of one or more permission levels corresponding to the target image are obtained. The target image is a patch-level image or a frame-level image, and the target image includes at least one CU. If the target image includes a CU of a first permission level and a CU of a second permission level, the bitstreams of the multiple permission levels include the bitstreams of the first permission level and the bitstreams of the second permission level. The first permission level is the lowest permission level, and the second permission level is not the lowest permission level. The CU partitioning information and CU permission information of the target image are decoded based on the bitstream of one or more permission levels, and each CU of the target image is decoded based on the CU partitioning information and CU permission information; wherein, the CU permission information represents the permission information of each CU in the target image, and for each CU, the permission information of the CU represents the permission level of the CU. Specifically, for the current CU to be decoded in the target image, the current permission level corresponding to the current CU is determined based on the CU permission information; if the target permission level does not have the access permission for the current CU based on the current permission level, then decoding the reconstruction information corresponding to the current CU from the target bitstream is skipped; low-dimensional reconstruction information corresponding to the current CU is obtained, and the reconstructed image block corresponding to the current CU is determined based on the low-dimensional reconstruction information; wherein, the low-dimensional reconstruction information is a fixed reconstruction value; Wherein, the target permission level is the permission level of the user accessing the decoding end; The target bitstream is a bitstream with a permission level less than or equal to the target permission level.

2. The method according to claim 1, characterized in that, After determining the current permission level corresponding to the current CU based on the CU permission information, the method further includes: If it is determined based on the current permission level that the target permission level has the access permission of the current CU, then the high-dimensional reconstruction information corresponding to the current CU is decoded from the target bitstream based on the CU partitioning information, and the reconstructed image block corresponding to the current CU is determined based on the high-dimensional reconstruction information.

3. The method according to claim 1 or 2, characterized in that, The decoding of the CU partitioning information and CU permission information of the target image based on the bitstream of the multiple permission levels includes: If the CU partitioning information and CU permission information of the target image are located in the bitstream of the first permission level, then the CU partitioning information and the CU permission information are decoded from the bitstream of the first permission level.

4. The method according to claim 1 or 2, characterized in that, The decoding of the CU partitioning information and CU permission information of the target image based on the bitstream of the multiple permission levels includes: If the CU partitioning information of the target image is located in the bitstream of the first permission level, then the CU partitioning information is decoded from the bitstream of the first permission level; and / or, If the permission information of the first CU of the target image is located in a bitstream with a permission level less than or equal to the permission level of the first CU, then the permission information of the first CU is decoded from the bitstream with a permission level less than or equal to the permission level of the first CU; wherein, the first CU is any CU.

5. The method according to claim 4, characterized in that, For a bitstream with a permission level lower than that of the first CU, the permission information of the first CU in the bitstream is a first value, which indicates that the bitstream does not contain the high-dimensional reconstruction information corresponding to the first CU; or, for a bitstream with a permission level equal to that of the first CU, the permission information of the first CU in the bitstream is a second value, which indicates that the bitstream contains the high-dimensional reconstruction information corresponding to the first CU.

6. The method according to claim 1 or 2, characterized in that, The decoding of the CU partitioning information and CU permission information of the target image based on the bitstream of the multiple permission levels includes: If the CU partitioning information and CU permission information of the target image are located in the bitstream of the first permission level and the second permission level, then the CU partitioning information and the CU permission information are decoded from the bitstream of the first permission level, or the CU partitioning information and the CU permission information are decoded from the bitstream of the second permission level.

7. The method according to claim 6, characterized in that, If the permission information of the second CU of the target image is located in the bitstream of the first permission level and the second permission level, and the second CU is any CU, then: For a bitstream with a permission level lower than that of the second CU, the permission information of the second CU in the bitstream is a first value, which indicates that the bitstream does not contain the high-dimensional reconstruction information corresponding to the second CU; or, for a bitstream with a permission level greater than or equal to that of the second CU, the permission information of the second CU in the bitstream is a second value, which indicates that the bitstream contains the high-dimensional reconstruction information corresponding to the second CU. Alternatively, for a bitstream whose permission level is not equal to the permission level of the second CU, the permission information of the second CU in the bitstream is a first value, which indicates that the bitstream does not contain the high-dimensional reconstruction information corresponding to the second CU; or, for a bitstream whose permission level is equal to the permission level of the second CU, the permission information of the second CU in the bitstream is a second value, which indicates that the bitstream contains the high-dimensional reconstruction information corresponding to the second CU.

8. The method according to claim 1 or 2, characterized in that, Before obtaining the bitstreams of multiple permission levels corresponding to the target image, the method further includes: Decode the permission protection parameters and the region of interest parameters corresponding to the target image from the high-level syntax corresponding to the target image; wherein the first syntax used for the permission protection parameters is different from the second syntax used for the region of interest parameters, or the first syntax is partially the same as the second syntax, or the first syntax is completely the same as the second syntax.

9. The method according to claim 8, characterized in that, The permission protection parameters include at least one of the following: sequence header switch identifier, image header switch identifier, enhancement layer switch identifier, permission level classification information, permission level predictability identifier, probability model update identifier, and CU boundary filtering parameters; the region of interest parameters include at least one of the following: sequence header switch identifier, image header switch identifier, enhancement layer switch identifier, region category classification information, region category predictability identifier, probability model update identifier, and CU boundary filtering parameters.

10. The method according to claim 9, characterized in that, The first syntax differs from the second syntax in at least one of the following ways: the sequence header switch identifier of the permission protection parameter and the sequence header switch identifier of the region of interest parameter use different syntaxes; the image header switch identifier of the permission protection parameter and the image header switch identifier of the region of interest parameter use different syntaxes; the enhancement layer switch identifier of the permission protection parameter and the enhancement layer switch identifier of the region of interest parameter use different syntaxes; the permission level classification information of the permission protection parameter and the region category classification information of the region of interest parameter use different syntaxes; the permission level predictability identifier of the permission protection parameter and the region category predictability identifier of the region of interest parameter use different syntaxes; the probability model update status identifier of the permission protection parameter and the probability model update status identifier of the region of interest parameter use different syntaxes; the CU boundary filtering parameter of the permission protection parameter and the CU boundary filtering parameter of the region of interest parameter use different syntaxes. The first syntax and the second syntax are the same in at least one of the following: the sequence header switch identifier of the permission protection parameter and the sequence header switch identifier of the region of interest parameter use different syntax; the image header switch identifier of the permission protection parameter and the image header switch identifier of the region of interest parameter use different syntax; the enhancement layer switch identifier of the permission protection parameter and the enhancement layer switch identifier of the region of interest parameter use the same syntax; the permission level classification information of the permission protection parameter and the region category classification information of the region of interest parameter use the same syntax; the permission level predictability identifier of the permission protection parameter and the region category predictability identifier of the region of interest parameter use the same syntax; the probability model update status identifier of the permission protection parameter and the probability model update status identifier of the region of interest parameter use the same syntax; the CU boundary filtering parameter of the permission protection parameter and the CU boundary filtering parameter of the region of interest parameter use the same syntax. The first syntax is completely identical to the second syntax, including at least one of the following: the sequence header switch identifier of the permission protection parameter and the sequence header switch identifier of the region of interest parameter use the same syntax; the image header switch identifier of the permission protection parameter and the image header switch identifier of the region of interest parameter use the same syntax; the enhancement layer switch identifier of the permission protection parameter and the enhancement layer switch identifier of the region of interest parameter use the same syntax; the permission level classification information of the permission protection parameter and the region category classification information of the region of interest parameter use the same syntax; the permission level predictability identifier of the permission protection parameter and the region category predictability identifier of the region of interest parameter use the same syntax; the probability model update status identifier of the permission protection parameter and the probability model update status identifier of the region of interest parameter use the same syntax; the CU boundary filtering parameter of the permission protection parameter and the CU boundary filtering parameter of the region of interest parameter use the same syntax.

11. The method according to claim 10, characterized in that, If the sequence header switch identifier of the permission protection parameter and the sequence header switch identifier of the region of interest parameter use the same syntax, when the syntax is the first value, it is determined that the sequence-level switch control information allows permission protection to be enabled and allows region of interest processing to be enabled; when the syntax is the second value, it is determined that the sequence-level switch control information prohibits permission protection to be enabled and prohibits region of interest processing to be enabled. If the image header switch identifier of the permission protection parameter and the image header switch identifier of the region of interest parameter use the same syntax, when the syntax is the first value, it is determined that the image-level switch control information allows permission protection to be enabled and allows region of interest processing to be enabled; when the syntax is the second value, it is determined that the image-level switch control information prohibits permission protection to be enabled and prohibits region of interest processing to be enabled. If the enhancement layer switch identifier of the permission protection parameter and the enhancement layer switch identifier of the region of interest parameter use the same syntax, when the syntax is the first value, it is determined that the enhancement layer is enabled for permission protection and region of interest processing; when the syntax is the second value, it is determined that the enhancement layer is disabled for permission protection and region of interest processing. If the permission level classification information of the permission protection parameter and the region category classification information of the region of interest parameter use the same syntax, then the classification identifier is parsed from the syntax, and the permission level and region category are determined based on the classification identifier. Alternatively, the classification identifier can be parsed from the syntax, the permission level can be determined based on the classification identifier, and the region category can be determined based on the permission level and the obtained mapping relationship; Alternatively, the classification identifier can be parsed from the syntax, the region category can be determined based on the classification identifier, and the permission level can be determined based on the region category and the obtained mapping relationship; wherein, the mapping relationship represents the mapping relationship between the permission level and the region category; If the permission level prediction flag of the permission protection parameter and the region category prediction flag of the region of interest parameter use the same syntax, when the syntax is the first value, it is determined that the permission level can be predicted based on the permission level of the previous CU, and the region category can be predicted based on the region category of the previous CU. When the syntax is the second value, it is determined that the permission level and region category are unpredictable. If the probability model update flag for the permission protection parameter and the probability model update flag for the region of interest parameter use the same syntax, when the syntax is the first value, it is determined that the probability model in the permission protection process and the probability model in the region of interest processing process should be updated; when the syntax is the second value, it is determined that the probability model in the permission protection process and the probability model in the region of interest processing process should not be updated. If the CU boundary filtering parameters of the access control parameters and the CU boundary filtering parameters of the region of interest parameters use the same syntax, then candidate filtering parameters are parsed from this syntax, and the target filtering parameters for access control and the region of interest are determined based on these candidate filtering parameters; or, a first candidate filtering parameter is determined based on the access control level and the acquired first mapping relationship, and a second candidate filtering parameter is determined based on the region category and the acquired second mapping relationship. The target filtering parameters for access control are determined based on the first candidate filtering parameters, and the target filtering parameters for the region of interest are determined based on the second candidate filtering parameters; wherein, the first mapping relationship includes the mapping relationship between the access control level and the filtering parameters, and the second mapping relationship includes the mapping relationship between the region category and the filtering parameters.

12. The method according to claim 1 or 2, characterized in that, If there are K permission levels, where K is a positive integer greater than 1, the bitstream of the multiple permission levels corresponding to the target image includes the RBSP bitstream of the K permission levels corresponding to the target image. The RBSP bitstream of the K permission levels is located in K NAL units, wherein the RBSP bitstream of different permission levels is located in different NAL units.

13. The method according to claim 12, characterized in that, For each permission level of RBSP stream, the RBSP stream is encrypted using a patch-level encryption method, and / or, the RBSP stream is encrypted using a CU-level encryption method; If the RBSP stream is encrypted using patch-level encryption, then if the target permission level is lower than the current patch-level permission, parsing the information in the RBSP stream is prohibited. When the target permission level is greater than or equal to the current patch level permission, continue parsing the CU information in the RBSP stream; when parsing the CU information in the RBSP stream, if the RBSP stream is encrypted using a CU-level encryption method, then when the target permission level is less than the current CU level permission, the parsing of the high-dimensional reconstruction information of the current CU is prohibited; when the target permission level is greater than or equal to the current CU level permission, continue parsing the high-dimensional reconstruction information of the current CU.

14. The method according to claim 12, characterized in that, The process of obtaining the bitstreams corresponding to multiple permission levels of the target image includes: The NAL units corresponding to RBSP streams of multiple permission levels are obtained sequentially according to the interleaving order from high to low permission level; or, the NAL units corresponding to RBSP streams of multiple permission levels are obtained sequentially according to the interleaving order from low to high permission level; or, the NAL units corresponding to RBSP streams of multiple permission levels are obtained sequentially according to the preset interleaving order of permission levels.

15. The method according to claim 12, characterized in that, If there are multiple target images, the knowledge base frame bitstream referenced by the i-th target image is interleaved with the main bitstream of the i-th target image. The main bitstream of the i-th target image includes the RBSP bitstreams of the K permission levels corresponding to the i-th target image. The i-th target image can be any target image.

16. The method according to claim 15, characterized in that, The knowledge base frame bitstream referenced by the i-th target image is located after the last RBSP bitstream among the K permission levels of the RBSP bitstream corresponding to the i-th target image; or, The knowledge base frame bitstream referenced by the i-th target image is located before the first RBSP bitstream among the K permission levels of the RBSP bitstream corresponding to the i-th target image; or, If the i-th target image is not the last target image among all target images, then the knowledge base frame bitstream referenced by the i-th target image is located after the last RBSP bitstream among the K permission levels of the RBSP bitstream corresponding to the i-th target image; or, if the i-th target image is the last target image among all target images, then the knowledge base frame bitstream referenced by the i-th target image is located before the first RBSP bitstream among the K permission levels of the RBSP bitstream corresponding to the i-th target image.

17. An image decoding device, characterized in that, The device, applied at the decoding end, includes: The acquisition module is used to acquire bitstreams of one or more permission levels corresponding to a target image when permission protection is enabled. The target image is a patch-level image or a frame-level image, and the target image includes at least one CU. If the target image includes a CU of a first permission level and a CU of a second permission level, the bitstreams of the multiple permission levels include the bitstreams of the first permission level and the bitstreams of the second permission level. The first permission level is the lowest permission level, and the second permission level is a non-lowest permission level. A decoding module is configured to decode the CU partitioning information and CU permission information of the target image based on the bitstream of one or more permission levels, and to decode each CU of the target image based on the CU partitioning information and the CU permission information; wherein, the CU permission information represents the permission information of each CU in the target image, and for each CU, the permission information represents the permission level of the CU; wherein, when the decoding module decodes each CU of the target image based on the CU partitioning information and the CU permission information, it specifically performs the following: for the current CU to be decoded in the target image, it determines the current permission level corresponding to the current CU based on the CU permission information; if it is determined based on the current permission level that the target permission level does not have access to the current CU, it skips decoding the reconstruction information corresponding to the current CU from the target bitstream; it obtains the low-dimensional reconstruction information corresponding to the current CU, and determines the reconstructed image block corresponding to the current CU based on the low-dimensional reconstruction information; wherein, the low-dimensional reconstruction information is a fixed reconstruction value; and the target permission level is the permission level of the user accessing the decoding end; The target bitstream is a bitstream with a permission level less than or equal to the target permission level.

18. A decoding device, characterized in that, include: A processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; The processor is configured to execute machine-executable instructions to implement the method of any one of claims 1-16.

19. An electronic device, characterized in that, include: A processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; The processor is configured to execute machine-executable instructions to implement the method of any one of claims 1-16.

20. A machine-readable storage medium, characterized in that, The machine-readable storage medium stores a plurality of computer instructions, which, when executed by a processor, implement the method described in any one of claims 1-16.

Images