Image decoding and coding method and device, equipment and storage medium

Abstract

The invention belongs to the technical field of image processing, and discloses an image decoding and coding method and device, equipment and a storage medium. According to the method, when a privacy coding mode is in an open state, a unit permission level and a user permission level are obtained, the unit permission level is the permission level of a current processing unit, and the user permission level is the permission level of a decoding end user; determining a to-be-decoded code stream corresponding to the current processing unit according to the unit permission level and the user permission level; and decoding the code stream to be decoded to obtain a reconstructed image block corresponding to the current processing unit. When the privacy coding mode is in the open state, the corresponding to-be-decoded code stream is selected for decoding according to the unit permission level and the user permission level, so that even a low-permission user can decode a non-privacy code stream to obtain the reconstructed image corresponding to the privacy-removed image content corresponding to the privacy area, and the user experience is improved. And the situation that the low-permission user skips decoding of the processing unit corresponding to the privacy area is avoided.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular, to an image decoding and encoding method, apparatus, device, and storage medium. Background Art

[0002] The early research motivation of video privacy protection technology comes from the application requirement of realizing privacy protection for surveillance video sequences. By protecting the privacy areas (such as moving human bodies, faces, etc.) in the video that can directly or indirectly identify personal identities, while other areas remain visible to all users, a good balance between public safety and personal privacy is achieved.

[0003] Currently, the implementation of the privacy protection scheme is that for each frame of the image to be encoded, the encoding end generates two streams of codes. Among them, one stream of codes is the privacy code stream obtained by encoding the privacy area, and the other stream of codes is the non-privacy code stream obtained by encoding other information and the non-privacy area. The privacy code stream information includes all the information within the coding unit (CU) after the privacy area is divided by the image processing unit (CU). The non-privacy code stream information includes all other information.

[0004] In this privacy protection scheme, low-privilege decoding-end users cannot obtain the correct high-privilege code stream. Therefore, when low-privilege users decode, if they parse that the CU has a high privilege, they skip the parsing of this CU, and finally the visual effect of the images seen by low-privilege users is poor. Summary of the Invention

[0005] The main purpose of this application is to provide an image decoding and encoding method, apparatus, device, and storage medium, aiming to solve the technical problem that the visual effect of the images seen by low-privilege users in the privacy protection scheme is poor.

[0006] To achieve the above object, this application provides an image decoding method, and the image decoding method includes:

[0007] When the privacy encoding mode is in the enabled state, obtain the unit privilege level and the user privilege level, where the unit privilege level is the privilege level of the current processing unit, and the user privilege level is the privilege level of the decoding-end user;

[0008] Determine the code stream to be decoded corresponding to the current processing unit according to the unit privilege level and the user privilege level;

[0009] Decode the code stream to be decoded to obtain the reconstructed image block corresponding to the current processing unit.

[0010] In a possible implementation manner of this application, the determining the code stream to be decoded corresponding to the current processing unit according to the unit privilege level and the user privilege level includes:

[0011] If the unit permission level is not the lowest permission level and the user permission level is greater than or equal to the unit permission level, read the privacy code stream and non-privacy code stream corresponding to the current processing unit. The privacy code stream and the non-privacy code stream include at least one of the information for prediction, inverse quantization, or inverse transformation. The non-privacy code stream is the code stream corresponding to the lowest permission level, and the privacy code stream is the code stream corresponding to the unit permission level;

[0012] Use the privacy code stream as the code stream to be decoded corresponding to the current processing unit.

[0013] In a possible implementation manner of the present application, the step of reading the privacy code stream and non-privacy code stream corresponding to the current processing unit includes:

[0014] Read the privacy code stream corresponding to the current processing unit according to the first entropy decoder, and read the non-privacy code stream corresponding to the current processing unit according to the second entropy decoder. The first entropy decoder is the entropy decoder corresponding to the unit permission level, and the second entropy decoder is the entropy decoder corresponding to the lowest permission level.

[0015] In a possible implementation manner of the present application, the step of determining the code stream to be decoded corresponding to the current processing unit according to the unit permission level and the user permission level includes:

[0016] If the unit permission level is not the lowest permission level and the user permission level is less than the unit permission level, read the non-privacy code stream corresponding to the current processing unit. The non-privacy code stream includes at least one of the information for prediction, inverse quantization, or inverse transformation;

[0017] Use the non-privacy code stream as the code stream to be decoded corresponding to the current processing unit.

[0018] In a possible implementation manner of the present application, before obtaining the unit permission level and the user permission level when the privacy coding mode is in the enabled state, it further includes:

[0019] Extract the mode flag bit from the image code stream corresponding to the image to be decoded. The mode flag bit is a sequence-level syntax;

[0020] Determine whether the privacy coding mode is enabled according to the value of the mode flag bit.

[0021] In a possible implementation manner of the present application, after determining whether the privacy coding mode is enabled according to the value of the mode flag bit, it further includes:

[0022] When the privacy coding mode is in the disabled state, obtain the unit permission level and the user permission level;

[0023] If the user permission level is greater than or equal to the unit permission level, obtain the bitstream corresponding to the unit permission level;

[0024] Use the bitstream corresponding to the unit permission level as the bitstream to be decoded corresponding to the current processing unit;

[0025] Decode the bitstream to be decoded to obtain the reconstructed image block corresponding to the current processing unit.

[0026] In a possible implementation manner of the present application, after obtaining the unit permission level and the user permission level when the privacy coding mode is in the off state, it further includes:

[0027] If the user permission level is less than the unit permission level, construct the reconstructed image block corresponding to the current processing unit according to a fixed pixel value;

[0028] Or,

[0029] If the user permission level is less than the unit permission level, construct the reconstructed image block corresponding to the current processing unit according to the reconstructed image blocks corresponding to the already reconstructed units around the current processing unit.

[0030] In a possible implementation manner of the present application, determining the bitstream to be decoded corresponding to the current processing unit according to the unit permission level and the user permission level includes:

[0031] If the unit permission level is the lowest permission level, read the non-privacy bitstream corresponding to the current processing unit;

[0032] Use the non-privacy bitstream as the bitstream to be decoded corresponding to the current processing unit.

[0033] In a possible implementation manner of the present application, determining the bitstream to be decoded corresponding to the current processing unit according to the unit permission level and the user permission level includes:

[0034] Determine the bitstream to be parsed according to the unit permission level and the user permission level;

[0035] Obtain the surrounding processing units whose corresponding permission levels are less than or equal to the permission level of the bitstream to be parsed to obtain the referenceable units, where the surrounding processing units are the processing units adjacent to the current processing unit in the time domain or the spatial domain;

[0036] Refer to the decoding information corresponding to the referenceable units, read the bitstream to be parsed, and select the bitstream to be decoded corresponding to the current processing unit from the bitstream to be parsed, where the decoding information is the information that has been parsed and decoded during the image reconstruction of the referenceable units.

[0037] In a possible implementation manner of the present application, decoding the to-be-decoded code stream to obtain a reconstructed image block corresponding to the current processing unit includes:

[0038] If the custom mode is not enabled, decode the to-be-decoded code stream to obtain a reconstructed image block corresponding to the current processing unit.

[0039] In a possible implementation manner of the present application, decoding the to-be-decoded code stream to obtain a reconstructed image block corresponding to the current processing unit includes:

[0040] If the custom mode is enabled, obtain a reconstructed image block corresponding to the current processing unit in any of the following ways:

[0041] Decode the to-be-decoded code stream to obtain a decoded image block, and replace the decoded image block with an image block constructed by the pixel values specified in the custom mode as the reconstructed image block corresponding to the current processing unit;

[0042] Do not decode the to-be-decoded code stream, obtain the pixel values specified in the custom mode, and construct a reconstructed image block corresponding to the current processing unit according to the pixel values;

[0043] Decode the to-be-decoded code stream to obtain a reconstructed image block corresponding to the current processing unit.

[0044] In a possible implementation manner of the present application, the way to obtain a reconstructed image block corresponding to the current processing unit is specified according to the mode parameters corresponding to the custom mode.

[0045] In a possible implementation manner of the present application, before obtaining the unit permission level and the user permission level when the privacy encoding mode is in the enabled state, it further includes:

[0046] Extract the code stream corresponding to the lowest permission level from the image code stream corresponding to the to-be-decoded image;

[0047] Extract the unit division information from the code stream corresponding to the lowest permission level;

[0048] Perform unit division according to the unit division information to obtain a plurality of image processing units;

[0049] Traverse the plurality of image processing units to obtain the current processing unit;

[0050] After decoding the to-be-decoded code stream to obtain a reconstructed image block corresponding to the current processing unit, it further includes:

[0051] Cache the reconstructed image block corresponding to the current processing unit;

[0052] When the caching is completed, if the current processing unit is the last image processing unit among the multiple image processing units, the reconstructed image corresponding to the image to be decoded is constructed based on the reconstructed image blocks corresponding to the cached image processing units.

[0053] In a possible implementation manner of the present application, before caching the reconstructed image block corresponding to the current processing unit, it further includes:

[0054] Extract filtering pixels and reference pixels corresponding to the filtering pixels from the reconstructed image block, where the reference pixels are pixels referred to when filtering the filtering pixels or pixels at peripheral positions of the filtering pixels;

[0055] If the permission level of the filtering pixel is less than the permission level of any of the reference pixels, skip the filtering process of the filtering pixel and cache the reconstructed image block.

[0056] In a possible implementation manner of the present application, after extracting the filtering pixels and the reference pixels corresponding to the filtering pixels from the reconstructed image block, it further includes:

[0057] If the permission level of the filtering pixel is greater than or equal to the permission levels of all the reference pixels, filter the filtering pixel according to the reference pixels.

[0058] In a possible implementation manner of the present application, extracting the filtering pixels and the reference pixels corresponding to the filtering pixels from the reconstructed image block includes:

[0059] Extract filtering pixels from the reconstructed image block, and obtain target pixels according to the reference positions corresponding to the filtering pixels;

[0060] If the target pixels are available, use the target pixels as the reference pixels;

[0061] If the target pixels are not available, use the pixels at the positions closest to the target pixels in the filtering unit corresponding to the filtering pixels as the reference pixels, and use the permission levels of the pixels at the positions closest to the target pixels in the filtering unit corresponding to the filtering pixels as the permission levels of the reference pixels.

[0062] In a possible implementation manner of the present application, the current processing unit is an enhancement layer processing unit;

[0063] Decoding the bitstream to be decoded to obtain a reconstructed image block corresponding to the current processing unit includes:

[0064] Obtain the reference information corresponding to the current processing unit in the base layer, where the reference information includes mode reference information and pixel reference information;

[0065] If the permission level of the to-be-decoded bitstream is greater than the permission level of the reference information, then decode the to-be-decoded bitstream with reference to the reference information to obtain a reconstructed image block corresponding to the current processing unit.

[0066] In a possible implementation manner of the present application, the decoding the to-be-decoded bitstream to obtain a reconstructed image block corresponding to the current processing unit includes:

[0067] Obtain multiple sets of quantization parameters set according to the user permission level;

[0068] Update the multiple sets of quantization parameters;

[0069] After the update is completed, decode the to-be-decoded bitstream to obtain a reconstructed image block corresponding to the current processing unit.

[0070] In a possible implementation manner of the present application, each set of quantization parameters in the multiple sets of quantization parameters corresponds to a level of the permission level, and the multiple sets of quantization parameters are initialized according to the quantization information of the surrounding processing units;

[0071] Each set of quantization parameters in the multiple sets of quantization parameters satisfies the following conditions:

[0072] If the permission level of the surrounding processing unit is less than or equal to the permission level corresponding to the quantization parameter, then the current quantization parameter is equal to the quantization parameter of the surrounding processing unit;

[0073] If the permission level of the surrounding processing unit is greater than the permission level corresponding to the quantization parameter, then the quantization parameter is equal to the quantization parameter of the current slice level, the current coding tree unit or the current frame level.

[0074] In a possible implementation manner of the present application, the updating the multiple sets of quantization parameters includes:

[0075] If the unit permission level is not the lowest permission level, and the user permission level is greater than or equal to the unit permission level, then select a privacy quantization parameter from the multiple sets of quantization parameters according to the permission level of the privacy bitstream, and select a non-privacy quantization parameter from the multiple sets of quantization parameters according to the permission level of the non-privacy bitstream;

[0076] Update the privacy quantization parameter according to the quantization information of the current processing unit extracted from the privacy bitstream, and update the non-privacy quantization parameter according to the quantization information of the current processing unit extracted from the non-privacy bitstream.

[0077] In a possible implementation manner of the present application, the updating the multiple sets of quantization parameters includes:

[0078] If the unit permission level is the lowest permission level, or the user permission level is lower than the unit permission level, non-private quantization parameters are selected from the multiple sets of quantization parameters according to the permission level of the non-private bitstream;

[0079] Update the non-private quantization parameters according to the quantization information of the current processing unit extracted from the non-private bitstream.

[0080] In a possible implementation manner of the present application, the decoding the to-be-decoded bitstream to obtain a reconstructed image block corresponding to the current processing unit includes:

[0081] If it is detected that the sample reference condition is satisfied, obtain the reference sample corresponding to the current processing unit;

[0082] Perform intra prediction according to the reference sample to obtain a prediction result;

[0083] Decode according to the prediction result and the to-be-decoded bitstream to obtain a reconstructed image block corresponding to the current processing unit.

[0084] In a possible implementation manner of the present application, before the if it is detected that the sample reference condition is satisfied, obtain the reference sample corresponding to the current processing unit, further includes:

[0085] If there is at least one reference sample at the reference position corresponding to the current processing unit that belongs to the same slice level as the current processing unit, has completed image reconstruction, and the corresponding permission level is less than or equal to the permission level of the to-be-decoded bitstream, it is determined that the sample reference condition is satisfied.

[0086] In a possible implementation manner of the present application, the performing intra prediction according to the reference sample to obtain a prediction result includes:

[0087] Use the reference sample whose corresponding permission level is greater than the permission level of the to-be-decoded bitstream as the target sample;

[0088] Perform pixel substitution processing on the target sample in the reference samples, and the processing method of the pixel substitution processing is determined according to the position of the target sample relative to the current processing unit;

[0089] Perform intra prediction based on the reference samples after substitution processing to obtain a prediction result.

[0090] In addition, to achieve the above object, the present application also proposes an image encoding method, and the image encoding method includes:

[0091] When the privacy encoding mode is in the enabled state, obtain a privacy determination result, where the privacy determination result is used to indicate whether the current processing unit is located in a privacy area;

[0092] Encode the image corresponding to the current processing unit according to the privacy determination result to generate at least one bitstream corresponding to the current processing unit.

[0093] In a possible implementation manner of the present application, the encoding the image corresponding to the current processing unit according to the privacy determination result to generate at least one bitstream corresponding to the current processing unit includes:

[0094] If the privacy determination result is that it is located in the privacy area, encode the image corresponding to the current processing unit to generate a privacy bitstream and a non-privacy bitstream corresponding to the current processing unit;

[0095] If the privacy determination result is that it is not located in the privacy area, encode the image corresponding to the current processing unit to generate a non-privacy bitstream corresponding to the current processing unit.

[0096] In a possible implementation manner of the present application, different entropy encoders are used for encoding the privacy bitstream and the non-privacy bitstream;

[0097] The image encoding method further includes:

[0098] Obtain the permission level of the privacy area to obtain a privacy permission level;

[0099] Set the permission level of the current processing unit and the privacy bitstream to the privacy permission level, and set the permission level of the non-privacy bitstream to the lowest permission level.

[0100] In a possible implementation manner of the present application, the image encoding method further includes:

[0101] When the privacy encoding mode is in the disabled state, obtain a privacy determination result;

[0102] If the privacy determination result is that it is located in the privacy area, encode the image corresponding to the current processing unit to generate a privacy bitstream corresponding to the current processing unit;

[0103] If the privacy determination result is that it is not located in the privacy area, encode the image corresponding to the current processing unit to generate a non-privacy bitstream corresponding to the current processing unit.

[0104] In a possible implementation manner of the present application, before obtaining the privacy determination result when the privacy encoding mode is in the enabled state, further includes:

[0105] Perform unit division on the image to be encoded according to the unit division information to obtain multiple image processing units;

[0106] Traverse the multiple image processing units to obtain the current processing unit;

[0107] After encoding the image corresponding to the current processing unit according to the privacy determination result to generate at least one bitstream corresponding to the current processing unit, the method further includes:

[0108] If the current processing unit is the last processing unit among the multiple image processing units, construct an image bitstream according to at least one bitstream corresponding to each processing unit;

[0109] Send the image bitstream to the decoding end.

[0110] In a possible implementation manner of the present application, encoding the image corresponding to the current processing unit according to the privacy determination result to generate at least one bitstream corresponding to the current processing unit includes:

[0111] If it is detected that the sample reference condition is satisfied, obtain the reference sample corresponding to the current processing unit;

[0112] Perform intra prediction according to the reference sample to obtain a prediction result;

[0113] Encode the image corresponding to the current processing unit based on the prediction result and the privacy determination result to generate at least one bitstream corresponding to the current processing unit.

[0114] In addition, to achieve the above object, the present application further provides an image decoding device, and the image decoding device includes the following modules:

[0115] An acquisition module, configured to acquire the unit permission level and the user permission level when the privacy encoding mode is in an enabled state, where the unit permission level is the permission level of the current processing unit, and the user permission level is the permission level of the user at the decoding end;

[0116] An extraction module, configured to determine the bitstream to be decoded corresponding to the current processing unit according to the unit permission level and the user permission level;

[0117] A decoding module, configured to decode the bitstream to be decoded to obtain a reconstructed image block corresponding to the current processing unit.

[0118] In addition, to achieve the above object, the present application further provides an image encoding device, and the image encoding device includes the following modules:

[0119] An acquisition module, configured to acquire a privacy determination result when the privacy encoding mode is in an enabled state, where the privacy determination result is used to indicate whether the current processing unit is located in a privacy area;

[0120] An encoding module, configured to encode the image corresponding to the current processing unit according to the privacy determination result to generate at least one bitstream corresponding to the current processing unit.

[0121] In addition, to achieve the above object, the present application further provides a decoding device, including: a processor, a memory, and an image decoding program stored on the memory and executable on the processor. When the image decoding program is executed by the processor, the steps of the above-mentioned image decoding method are implemented.

[0122] In addition, to achieve the above object, the present application further provides an encoding device, including: a processor, a memory, and an image encoding program stored on the memory and executable on the processor. When the image encoding program is executed by the processor, the steps of the above-mentioned image encoding method are implemented.

[0123] In addition, to achieve the above object, the present application further provides an electronic device, including: a processor and a machine-readable storage medium, where a machine-executable instruction capable of being executed by the processor is stored on the machine-readable storage medium; the processor is configured to execute the machine-executable instruction to implement the above-mentioned method.

[0124] In addition, to achieve the above object, the present application further provides a storage medium, characterized in that an image decoding program and / or an image encoding program is stored on the storage medium. When the image decoding program is executed, the steps of the above-mentioned image decoding method are implemented, and when the image encoding program is executed, the steps of the above-mentioned image encoding method are implemented.

[0125] In addition, to achieve the above object, the present application further provides a computer application program product, including a computer application program, characterized in that when the computer application program is executed by a processor, the above-mentioned method is implemented.

[0126] In the embodiment of the present application, when the privacy coding mode is in the enabled state, the unit permission level and the user permission level are obtained. The unit permission level is the permission level of the current processing unit, and the user permission level is the permission level of the user at the decoding end. The code stream to be decoded corresponding to the current processing unit is determined according to the unit permission level and the user permission level. The code stream to be decoded is decoded to obtain the reconstructed image block corresponding to the current processing unit. Since when the privacy coding mode is in the enabled state, the corresponding code stream to be decoded is selected according to the unit permission level and the user permission level for decoding, it is ensured that even a low-privilege user can decode the non-privacy code stream and obtain the reconstructed image corresponding to the de-privatized image content in the privacy area, avoiding the low-privilege user skipping the decoding of the processing unit corresponding to the privacy area. Description of the Drawings

[0127] Figure 1 It is a schematic structural diagram of an electronic device in the hardware operating environment related to the solution of the embodiment of the present application;

[0128] Figure 2 It is a schematic diagram of a video encoding and decoding framework according to an embodiment of the present application;

[0129] Figure 3 It is a schematic diagram of the encoding process before improvement according to an embodiment of the present application;

[0130] Figure 4 It is a schematic diagram of the decoding process of a high-privilege user before improvement according to an embodiment of the present application;

[0131] Figure 5 It is a schematic diagram of the decoding process of a low-privilege user before improvement according to an embodiment of the present application;

[0132] Figure 6 It is a schematic diagram of the process of the first embodiment of the image decoding method of the present application;

[0133] Figure 7 It is a schematic diagram of the encoding process after improvement according to an embodiment of the present application;

[0134] Figure 8 It is a schematic diagram of the decoding process of a high-privilege user after improvement according to an embodiment of the present application;

[0135] Figure 9 It is a schematic diagram of the decoding process of a high-privilege user after improvement according to an embodiment of the present application;

[0136] Figure 10 It is a schematic diagram of the process of the second embodiment of the image decoding method of the present application;

[0137] Figure 11 It is a schematic diagram of the process of the third embodiment of the image decoding method of the present application;

[0138] Figure 12Schematic flowchart of the fourth embodiment of the image decoding method of the present application;

[0139] Figure 13 Schematic flowchart of the fifth embodiment of the image decoding method of the present application;

[0140] Figure 14 Schematic diagram of the reference sample position of an embodiment of the present application;

[0141] Figure 15 Schematic flowchart of the first embodiment of the image encoding method of the present application;

[0142] Figure 16 Block diagram of the structure of the first embodiment of the image decoding apparatus of the present application;

[0143] Figure 17 Block diagram of the structure of the first embodiment of the image encoding apparatus of the present application.

[0144] The realization, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. Detailed implementation manners

[0145] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0146] Refer to Figure 1 , Figure 1 Schematic diagram of the structure of an electronic device for the hardware operating environment involved in the solution of the embodiment of the present application.

[0147] As Figure 1 shown, the electronic device may include: a processor 1001, such as a Central Processing Unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display) and an input unit such as a keyboard (Keyboard). Optionally, the user interface 1003 may further include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a Wireless-Fidelity (WI-FI) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM), or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk memory. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0148] Those skilled in the art can understand that Figure 1 the structure shown in does not constitute a limitation on the electronic device, and it may include more or fewer components than shown in the figure, or combine some components, or have different component arrangements.

[0149] As Figure 1 shown, in the memory 1005 as a storage medium, an operating system, a network communication module, a user interface module, and an image decoding program and / or an image encoding program may be included.

[0150] In Figure 1 the electronic device shown, the network interface 1004 is mainly used for data communication with a network server; the user interface 1003 is mainly used for data interaction with a user; the processor 1001 and the memory 1005 in the electronic device of the present application may be arranged in the electronic device.

[0151] If the electronic device is applied at the decoding end for image decoding, then as a decoding device, at this time, the electronic device calls the image decoding program stored in the memory 1005 through the processor 1001, and executes the image decoding method provided by the embodiments of the present application;

[0152] If the electronic device is applied at the encoding end for image encoding, then as an encoding device, at this time, the electronic device calls the image encoding program stored in the memory 1005 through the processor 1001, and executes the image encoding method provided by the embodiments of the present application.

[0153] For the sake of easy understanding, now in combination with Figure 2 , 3 , 4, and 5 for illustration, but it does not limit the present solution.

[0154] The terms that may be involved in the image encoding and decoding process include:

[0155] Intra Prediction and Inter Prediction:

[0156] Intra prediction refers to using the reconstructed pixel values of the spatial adjacent blocks (in the same frame image as the current block) of the current block for predictive coding, while inter prediction refers to using the reconstructed pixel values of the temporal adjacent blocks (in different images from the current block) of the current block for predictive coding.

[0157] Prediction Signal: A prediction signal refers to the pixel value derived from the already encoded and decoded pixels. The residual is obtained by subtracting the original pixel from the prediction pixel, and then residual transform quantization and coefficient coding are performed. In particular, the prediction signal for inter-frame refers to the pixel value derived from the reference frame (reconstructed pixel frame) for the current block. Since pixel positions are discrete, interpolation operations are required to obtain the final prediction signal. The closer the prediction signal is to the original pixel, the smaller the residual energy obtained by subtracting the two, and the higher the coding compression performance.

[0158] As Figure 2 shown, the currently commonly used video encoding and decoding mainly involve modules such as prediction, transformation, quantization, entropy coding, and filtering during the entire image encoding and decoding process, where:

[0159] The prediction module includes intra-frame prediction and inter-frame prediction. Intra-frame prediction uses the reconstructed pixels around the current block for prediction to remove spatial redundancy; inter-frame prediction uses the reconstructed pixels on the temporal reference frame for prediction to remove temporal redundancy.

[0160] The transformation module linearly maps the residual information in the spatial domain to the transform domain (such as the frequency domain), aiming to make the energy more concentrated and remove the frequency domain correlation of the signal. The theoretical transformation matrix is invertible and does not cause signal loss.

[0161] The quantization module is a "many-to-one" mapping process, which is irreversible and causes signal loss; the advantage is that it can greatly reduce the value range of the signal, enabling the encoder to give a good approximation of the original signal with a small number of symbols, thereby improving the compression ratio.

[0162] The entropy coding module is a lossless coding method based on the principle of information entropy, which converts a series of element symbols (such as transform coefficients and mode information, etc.) used to represent the video sequence into a binary bitstream to remove the statistical redundancy of these video element symbols.

[0163] The filtering module enhances the reconstructed image, aiming to make the reconstructed image closer to the original image, while reducing the influence of block effect and ringing effect and improving the quality of the reconstructed image.

[0164] And the currently used privacy protection scheme is implemented as follows:

[0165] At the image encoding end, as Figure 3 shown, for each frame of the image to be encoded, the encoding end generates two bitstreams. One bitstream is the privacy bitstream obtained by encoding the privacy area, and the other bitstream is the non-privacy bitstream obtained by encoding other information and the non-privacy area. The privacy bitstream information includes all the information within the CU after CU partitioning of the privacy area. The non-privacy bitstream information includes all other information.

[0166] During the image decoding process, the sequence header, image header, patch header, filtering information, CU partitioning information, and CU permission flag bits are parsed from the non-private bitstream. Among them, the CU permission flag bit is a newly added flag bit, indicating that the current CU has low permission (non-private content) or high permission (private content). Finally, the information of the low-permission CU is parsed from the non-private bitstream, and the information of the high-permission CU is parsed from the private bitstream. Therefore, only high-permission users can parse the content of the high-permission CU (the image decoding process of high-permission users is as Figure 4 shown), and low-permission users need to skip the decoding process of the high-permission CU and use a fixed value for filling as the reconstruction value (the image decoding process of low-permission users is as Figure 5 shown).

[0167] An embodiment of the present application provides an image decoding method. Refer to Figure 6 , Figure 6 which is a schematic flowchart of the first embodiment of an image decoding method of the present application.

[0168] In this embodiment, the image decoding method includes the following steps:

[0169] Step S10: When the privacy encoding mode is in the enabled state, obtain the unit permission level and the user permission level.

[0170] It should be noted that the execution subject of this embodiment can be a decoding device. The decoding device can be the above-mentioned electronic device applied at the decoding end for image decoding. The decoding device can be an electronic device such as a personal computer or a server, or other devices that can implement the same or similar functions. This embodiment does not limit this. In this embodiment and the following embodiments, the image decoding method of the present application is described by taking the decoding device as an example. Among them, since when performing image encoding, the decoding end device will also perform image decoding when it is necessary to verify the image compression rate and adjust the image encoding effect. Under this premise, the execution subject of this embodiment can also be the decoding end device, and the decoding end device can be an electronic device applied at the encoding end for image encoding.

[0171] It should be noted that the unit permission level can be the permission level of the current processing unit, and the current processing unit can be the image processing unit currently performing image decoding processing. The user permission level can be the permission level of the decoding end user.

[0172] Since there may be multiple permission levels in the subsequent explanations, in order to avoid confusion, a unified explanation is given here:

[0173] The permission level of the processing unit can be the permission level of the image processing unit parsed from the lowest permission bitstream.

[0174] The user permission level can be input or specified by the decoding end, specified by the encoder, or obtained from other encrypted bitstreams. Each decoder has only one permission.

[0175] The bitstream permission level. The encoding end generates multiple bitstreams and sends them to the decoding end. Each processing unit determines the specific bitstream permission level to be parsed according to the user permission level and the current processing unit level.

[0176] The permission level of the reference pixels, and the permission is equal to the permission of the pixels actually participating in filtering.

[0177] The permission level of the peripheral units is equal to the permission level of the corresponding peripheral processing units of the current processing unit.

[0178] The permission level corresponding to the quantization parameter. The maximum value is the maximum permission corresponding to the current image, and the minimum value is the lowest permission level. The update of the quantization parameter is bound to the bitstream permission.

[0179] The private bitstream is a bitstream whose permission level is not the lowest permission level (i.e., the minimum value of the permission level, such as 0). If a certain image processing unit has a corresponding private bitstream, the permission level of the private bitstream is the same as the permission level of the image processing unit.

[0180] The non-private bitstream can be a bitstream with the lowest permission level, and its permission level is the minimum value among all permission levels (such as 0).

[0181] It should be noted that in order to improve the visual effect of the image seen by users with higher permissions, it is necessary to ensure that users with lower permissions do not skip the decoding of the processing units (CUs) corresponding to the private areas, and at the same time, it is necessary to ensure that privacy is not leaked. For this situation, this embodiment additionally sets a private encoding mode. If the private encoding mode is enabled, when the encoding end performs image encoding, it will encode the image content of the private area and write it into the private bitstream, and will perform privacy removal processing on the content of the private area (such as blurring, replacing with common content, etc.) to generate privacy-removed image content, and then write the privacy-removed image content corresponding to the private area into the non-private bitstream. At this time, both high-permission users and low-permission users can obtain the corresponding bitstreams for decoding without skipping the decoding of the image processing units corresponding to the private areas.

[0182] Therefore, if the decoding device detects that the private encoding mode is in the enabled state, it needs to determine which bitstream to decode and obtain the reconstructed image block corresponding to the current processing unit. In order to determine which specific bitstream to use for decoding, the unit permission level and the user permission level can be obtained at this time.

[0183] In a specific implementation, since low-privilege users cannot obtain the privacy bitstream, in order to ensure that all users can obtain the privilege level of the current processing unit, the encoding end can write information such as the division information of the image processing unit and the privilege level of each image processing unit into the bitstream with the lowest privilege level during encoding, that is, into the non-privacy bitstream.

[0184] Step S20: Determine the bitstream to be decoded corresponding to the current processing unit according to the unit privilege level and the user privilege level.

[0185] It should be noted that for the image processing unit corresponding to the privacy area, the unit privilege level is generally at a high level (that is, it indicates that the user privilege level needs to reach a certain level to decode the privacy bitstream corresponding to the current processing unit and obtain the image content of the privacy area). Therefore, according to the unit privilege level and the user privilege level, it can be determined whether the current processing unit is the image processing unit corresponding to the privacy area, and whether the current user at the decoding end is a high-privilege user or a low-privilege user, so as to determine the bitstream that the decoding-end user should decode for the current processing unit, that is, the bitstream to be decoded.

[0186] In a specific implementation, in order to reasonably determine the bitstream to be decoded, step S20 in this embodiment may include:

[0187] If the unit privilege level is not the lowest privilege level and the user privilege level is greater than or equal to the unit privilege level, read the privacy bitstream and the non-privacy bitstream corresponding to the current processing unit;

[0188] Use the privacy bitstream as the bitstream to be decoded corresponding to the current processing unit.

[0189] It should be noted that the privacy bitstream and the non-privacy bitstream may both include at least one type of information for prediction, inverse quantization, or inverse transformation. The non-privacy bitstream may be the bitstream corresponding to the lowest privilege level, the privilege level of the non-privacy bitstream may be the lowest privilege level, the privacy bitstream may be the bitstream corresponding to the unit privilege level, and the privilege level of the privacy bitstream may be the same as the unit privilege level of the current processing unit.

[0190] In actual use, if the unit privilege level is not the lowest privilege level, it means that the current processing unit is the image processing unit corresponding to the privacy area. If the user privilege level is greater than or equal to the unit privilege level, it means that the decoding-end user has the privilege to obtain the image content corresponding to the privacy area. Since the most basic information (such as the division information of the image processing unit, etc.) is stored in the non-privacy bitstream, at this time, in order to facilitate subsequent processing, the privacy bitstream and the non-privacy bitstream corresponding to the current processing unit can be read.

[0191] Among them, reading the privacy code stream and non-privacy code stream corresponding to the current processing unit can be achieved by parsing the code stream transmitted from the encoding end through an entropy decoder to read the privacy code stream and non-privacy code stream contained therein.

[0192] It can be understood that since the user at the decoding end has the permission to obtain the image content corresponding to the privacy area, the privacy code stream can be used as the code stream to be decoded at this time to ensure that the user at the decoding end can correctly obtain the image content of the privacy area.

[0193] In a specific implementation, in order to correctly read the privacy code stream and non-privacy code stream, the step of reading the privacy code stream and non-privacy code stream corresponding to the current processing unit described in this embodiment may include:

[0194] Read the privacy code stream corresponding to the current processing unit according to the first entropy decoder, and read the non-privacy code stream corresponding to the current processing unit according to the second entropy decoder.

[0195] It should be noted that when encoding the privacy code stream and non-privacy code stream, different entropy encoders can be used for encoding. Therefore, in order to ensure correct parsing, different entropy decoders are also required for decoding. Among them, the first entropy decoder is the entropy decoder corresponding to the unit permission level, and the second entropy decoder is the entropy decoder corresponding to the lowest permission level.

[0196] In a specific implementation, in order to reasonably determine the code stream to be decoded, step S20 described in this embodiment may include:

[0197] If the unit permission level is not the lowest permission level and the user permission level is less than the unit permission level, read the non-privacy code stream corresponding to the current processing unit, where the non-privacy code stream includes at least one of the information for prediction, inverse quantization, or inverse transformation;

[0198] Use the non-privacy code stream as the code stream to be decoded corresponding to the current processing unit.

[0199] It should be noted that if the unit permission level is not the lowest permission level, it means that the current processing unit is the image processing unit corresponding to the privacy area. If the user permission level is less than the unit permission level, it means that the user at the decoding end does not have the permission to obtain the image content corresponding to the privacy area. Therefore, at this time, there is no need to read the privacy code stream, and only the non-privacy code stream needs to be read. Among them, the privacy code stream can also be read using the above-mentioned second entropy encoder, that is, the entropy decoder corresponding to the lowest permission level reads the non-privacy code stream corresponding to the current processing unit.

[0200] In a specific implementation, in order to reasonably determine the code stream to be decoded, step S20 described in this embodiment may include:

[0201] If the unit permission level is the lowest permission level, read the non-privacy bitstream corresponding to the current processing unit;

[0202] Use the non-privacy bitstream as the bitstream to be decoded corresponding to the current processing unit.

[0203] It should be noted that if the unit permission level is the lowest permission level, it means that the current processing unit is not the image processing unit corresponding to the privacy area. At this time, there is no privacy bitstream corresponding to the current processing unit, only the non-privacy bitstream corresponding to the current processing unit. Regardless of the permission level of the decoding-end user, only the non-privacy bitstream can be decoded. Therefore, the non-privacy bitstream corresponding to the current processing unit can be directly read and used as the bitstream to be decoded corresponding to the current processing unit.

[0204] In a possible implementation manner of this embodiment, in some cases, when reading the bitstream, it may be necessary to refer to the decoding information of the surrounding reconstructed image processing units for reference. For a low-permission CU, if it refers to the information of a high-permission CU, it may cause incorrect decoding because a low-permission user cannot obtain high-permission content. To avoid this phenomenon, when parsing, it is necessary to limit the processing unit for reference. Then, step S20 described in this embodiment may include:

[0205] Determine the bitstream to be parsed according to the unit permission level and the user permission level;

[0206] Obtain the surrounding processing units whose corresponding permission levels are less than or equal to the permission level of the bitstream to be parsed to obtain the referenceable units;

[0207] Refer to the decoding information corresponding to the referenceable units, read the bitstream to be parsed, and select the bitstream to be decoded corresponding to the current processing unit from the bitstream to be parsed.

[0208] It should be noted that determining the bitstream to be parsed according to the unit permission level and the user permission level may be to determine the bitstream to be read according to the unit permission level and the user permission level. For example: if the unit permission level is not the lowest permission level and the user permission level is greater than or equal to the unit permission level, at this time, the privacy bitstream and the non-privacy bitstream need to be read, then the bitstream to be parsed includes the privacy bitstream and the non-privacy bitstream; if the unit permission level is not the lowest permission level and the user permission level is less than the unit permission level, at this time, the non-privacy bitstream is needed, then the bitstream to be parsed is the non-privacy bitstream; if the unit permission level is the lowest permission level, then the bitstream to be parsed is the non-privacy bitstream.

[0209] Among them, if there are multiple bitstreams to be parsed, at this time, the referenceable units can be obtained for each bitstream to be parsed respectively, and read according to the decoding information of the referenceable units.

[0210] It should be noted that the peripheral processing unit can be a processing unit adjacent to the current processing unit in the time domain or the spatial domain. The decoded information corresponding to the reference unit can be the information that has been parsed and decoded during the image reconstruction of the reference unit, such as prediction mode, quantization parameter, residual information, etc.

[0211] In actual use, when reading the code stream, it is still necessary to ensure that the reference information can be read. Therefore, it is necessary to avoid reading the reference information of a high-privilege processing unit when reading a code stream with a low privilege level. Therefore, the peripheral processing unit with a corresponding privilege level less than or equal to the privilege level of the code stream to be parsed can be used as the reference unit, and then the decoded information of the reference unit is referred to, and the code stream to be parsed is read.

[0212] Among them, if the code stream to be parsed is a privacy code stream, the privilege level of the code stream to be parsed is the same as the unit privilege level, and if the code stream to be parsed is a non-privacy code stream, the privilege level of the code stream to be parsed is the lowest privilege level.

[0213] It can be understood that since there may be multiple code streams to be parsed, after reading the code stream to be parsed, the code stream to be decoded can also be selected from them, and the selection method is the same as the method for determining the code stream to be decoded described above, which will not be elaborated here.

[0214] Step S30: Decode the code stream to be decoded to obtain the reconstructed image block corresponding to the current processing unit.

[0215] It can be understood that after determining the code stream to be decoded, the code stream to be decoded can be decoded for image reconstruction, so as to obtain the reconstructed image block corresponding to the current processing unit.

[0216] In a specific implementation, in order to improve the user experience and enable low-privilege users to select how to display the content of the privacy area by themselves, on the basis of the privacy masking mode, a custom mode is additionally added. If it is determined at this time that the current processing unit is the processing unit corresponding to the privacy area (that is, the unit privilege level is not the lowest privilege level), and the user is a low-privilege user (that is, the user privilege level is less than the unit privilege level), then the custom mode can be further detected to determine whether it is necessary to generate the reconstructed image block corresponding to the current processing unit in the custom mode. Then, step S30 in this embodiment may include:

[0217] If the custom mode is not enabled, decode the code stream to be decoded to obtain the reconstructed image block corresponding to the current processing unit.

[0218] It can be understood that if the custom mode is not enabled, it means that the low-privilege user does not use the custom method to limit how the content of the privacy area is displayed at this time. Therefore, the decoded bitstream can be decoded, and the image blocks obtained by image reconstruction can be used as the reconstructed image blocks corresponding to the current processing unit.

[0219] Among them, the custom mode is default closed. Whether the custom mode is enabled can be determined according to the value of the custom mode flag parameter of the relevant parameters in the decoding end. The user of the decoding end can modify the custom mode flag parameter to enable or disable the custom mode.

[0220] For example: when the value of the custom mode flag parameter is 0, it means the custom mode is closed; when the value of the custom mode flag parameter is 1, it means the custom mode is enabled. The default value of the custom mode flag parameter is 0.

[0221] If the custom mode is enabled, then the reconstructed image blocks of the current processing unit can be generated in any of the following ways:

[0222] Decode the bitstream to be decoded to obtain decoded image blocks, and replace the decoded image blocks with the image blocks constructed by the pixel values specified in the custom mode as the reconstructed image blocks corresponding to the current processing unit;

[0223] Do not decode the bitstream to be decoded, obtain the pixel values specified in the custom mode, and construct the reconstructed image blocks corresponding to the current processing unit according to the pixel values;

[0224] Decode the bitstream to be decoded to obtain the reconstructed image blocks corresponding to the current processing unit.

[0225] In actual use, the above ways of generating the reconstructed image blocks of the current processing unit can be specified by the mode parameters of the custom mode. For example: there is a specific mode parameter type in the custom mode, and the value of type is 1 - 3, and each value corresponds to one of the above ways of generating the reconstructed image blocks of the current processing unit. Of course, a second custom mode can also be additionally set on the custom mode, and it is specified according to the mode parameters corresponding to the second custom mode. This embodiment does not limit this.

[0226] It should be noted that the decoded image blocks can be the image blocks obtained by decoding the bitstream to be decoded and image reconstruction. The pixel values specified in the custom mode can be specified by the relevant parameters set in the decoding end.

[0227] It can be understood that when decoding the to-be-decoded bitstream to obtain a decoded image block, an image block constructed by pixel values specified in the custom mode is used to replace the decoded image block as the reconstructed image block corresponding to the current processing unit. At this time, the decoded image block obtained by decoding can be cached. If the user modifies the on / off state of the custom mode, then there is no need to re-decode at this time, and the decoded image block can be directly extracted from the cache for display.

[0228] If caching is not considered, in order to reduce the encoding processing flow, the reconstructed image block corresponding to the current processing unit can be directly constructed according to the pixel values specified in the custom mode.

[0229] Of course, the user can also choose to use the image block obtained by decoding the to-be-decoded bitstream as the reconstructed image block corresponding to the current processing unit.

[0230] In a specific implementation, when performing spatial layer coding, there is coding for the enhancement layer and the base layer. Among them, the processing unit of the base layer first parses and reconstructs, and then the processing unit of the enhancement layer can refer to the mode information and reconstructed pixels of the base layer processing unit for parsing and reconstruction. If the current processing unit is an enhancement layer processing unit, then step S30 of this embodiment may include:

[0231] Obtain the reference information corresponding to the current processing unit in the base layer;

[0232] If the permission level of the to-be-decoded bitstream is greater than the permission level of the reference information, then decode the to-be-decoded bitstream with reference to the reference information to obtain the reconstructed image block corresponding to the current processing unit.

[0233] It should be noted that the reference information may include mode reference information and pixel reference information. Of course, it may also include more reference information, and this embodiment does not limit this.

[0234] In actual use, when decoding a bitstream with a low permission level and referring to the reference information with a high permission level, it may cause the failure to obtain the reference information with a high permission level due to the difference in the user's permission level, and finally lead to decoding errors. To avoid this phenomenon, after obtaining the reference information corresponding to the current processing unit in the base layer, the permission level of the reference information can be compared with the permission level of the to-be-decoded bitstream first. Among them, the permission level of the reference information is the same as the permission level of the processing unit to which it belongs.

[0235] It can be understood that if the permission level of the bitstream to be decoded is greater than the permission level of the reference information, it means that when decoding the bitstream to be decoded at this time, the reference information with a permission level higher than that of the bitstream to be decoded is not referenced, and no decoding error will occur. Therefore, the reference information can be used as a reference to decode the bitstream to be decoded to obtain the reconstructed image block corresponding to the current processing unit.

[0236] In practical applications, in some cases, the enhancement process can also be skipped, and the pixels at the same position as the current processing unit in the base layer can be directly used as the reconstructed pixels of the current processing unit (at this time, the current processing unit is an enhancement layer processing unit). For example, if the privacy encoding mode is enabled, and the resolution of the enhancement layer is the same as that of the base layer and the block partitioning method is the same, when the user permission level is less than the permission level of the current processing unit (at this time, the current processing unit is an enhancement layer processing unit), the current processing unit skips parsing and directly uses the pixels at the same position as the current processing unit in the base layer as the reconstructed pixels of the current processing unit.

[0237] For the sake of easy understanding, now in combination with Figure 7 、 8 and 9 for illustration, but it does not limit this solution.

[0238] In the improved privacy protection scheme, the image encoding process is as Figure 7 shown. During the encoding process, if the current processing unit (current CU) is within the privacy area (that is, the current processing unit is the image processing unit corresponding to the privacy area), first encode the CU privacy bitstream. If the front-end masking privacy area mode (i.e., the privacy encoding mode) is enabled, then additionally encode the non-privacy bitstream corresponding to the current CU; if it is not within the privacy area, then only encode the non-privacy bitstream corresponding to the current CU.

[0239] In the improved privacy protection scheme, executed at the decoding end, the decoding process for high-privilege users is as Figure 8 shown, and the decoding process for low-privilege users is as Figure 9 shown. If the current CU is within the privacy area, then first detect whether the privacy encoding mode is enabled. If the privacy encoding mode is enabled, the low-privilege user will read and decode the non-privacy bitstream to obtain the reconstructed image block corresponding to the current processing unit, while the high-privilege user will read the privacy bitstream and the non-privacy bitstream, but only decode the privacy bitstream to obtain the reconstructed image block corresponding to the current processing unit; if the privacy encoding mode is not enabled, all users at the decoding end will read and decode the non-privacy bitstream.

[0240] In this embodiment, when the privacy coding mode is in the enabled state, the unit permission level and the user permission level are obtained. The unit permission level is the permission level of the current processing unit, and the user permission level is the permission level of the user at the decoding end. The bitstream to be decoded corresponding to the current processing unit is determined according to the unit permission level and the user permission level. The bitstream to be decoded is decoded to obtain the reconstructed image block corresponding to the current processing unit. Since when the privacy coding mode is in the enabled state, the corresponding bitstream to be decoded is selected according to the unit permission level and the user permission level for decoding, it is ensured that even a low-privilege user can decode the non-privacy bitstream and obtain the reconstructed image corresponding to the de-privatized image content in the privacy area, avoiding the decoding of the processing unit corresponding to the privacy area being skipped by the low-privilege user.

[0241] Reference Figure 10 , Figure 10 is a schematic flowchart of the second embodiment of an image decoding method according to the present application.

[0242] Based on the above first embodiment, before the step S10 of the image decoding method in this embodiment, it further includes:

[0243] Step S01: Extract the mode flag bit from the image bitstream corresponding to the image to be decoded.

[0244] It should be noted that the mode flag bit can be read from the image bitstream in sequence-level syntax. The image to be decoded can be the image currently being decoded for image decoding, and the image to be decoded can be a single-frame image or a multi-frame image. This embodiment does not limit this.

[0245] Step S02: Determine whether the privacy coding mode is enabled according to the value of the mode flag bit.

[0246] It should be noted that according to the value of the mode flag bit, it can be determined whether the privacy coding mode is enabled. Among them, the corresponding relationship between the value of the mode flag bit and whether the privacy coding mode is enabled can be set in advance by the management personnel of the decoding device. For example: when the value of the mode flag bit is 1, it means that the privacy coding mode is in the enabled state; when the value of the mode flag bit is 0, it means that the privacy coding mode is in the disabled state.

[0247] It should be noted that if the privacy coding mode is enabled, it means that the improved privacy protection scheme is adopted at this time, and the above step S10 and subsequent steps can be executed.

[0248] If the privacy coding mode is disabled, it means that the privacy protection scheme before improvement is continued to be used. After the step S02 in this embodiment, it may further include:

[0249] When the privacy coding mode is in the disabled state, the unit permission level and the user permission level are obtained;

[0250] If the unit permission level is not the lowest permission level and the user permission level is greater than or equal to the unit permission level, obtain the bitstream corresponding to the unit permission level.

[0251] Use the bitstream corresponding to the unit permission level as the bitstream to be decoded corresponding to the current processing unit.

[0252] Decode the bitstream to be decoded to obtain the reconstructed image block corresponding to the current processing unit.

[0253] It can be understood that if the privacy encoding mode is in the off state, it means that the image encoding end still uses the privacy protection scheme before improvement for encoding at this time. And if the unit permission level is not the lowest permission level and the user permission level is greater than or equal to the unit permission level at this time, it means that the current processing unit is the processing unit corresponding to the privacy area and the user has the permission to obtain the image content of the privacy area. Since the previous scheme skips the decoding of the processing unit corresponding to the privacy area for low-privilege users, there will be no relevant basic information for decoding the processing unit corresponding to the privacy area in the non-privacy bitstream. At this time, high-privilege users no longer need to obtain the non-privacy bitstream. Therefore, the bitstream corresponding to the current processing unit can be obtained, and the bitstream corresponding to the unit permission level is used as the bitstream to be decoded corresponding to the current processing unit, and then the bitstream to be decoded is decoded to obtain the reconstructed image block corresponding to the current processing unit.

[0254] In actual use, during the process of decoding the bitstream to be decoded, if the permission level of the already reconstructed units around the current processing unit is less than or equal to the permission level of the current processing unit, the reconstruction information of the already reconstructed units can be referred to for decoding the bitstream to be decoded; otherwise, it cannot be referred to.

[0255] For example: Assume that the current CU permission level is not equal to 0. But during parsing, the information within the current CU is parsed from the bitstream with a level of 0. During the parsing process, if it is necessary to refer to the CU information in the surrounding spatial or temporal domain and the CU permission level in the surrounding spatial or temporal domain is greater than the permission level of the bitstream to be decoded, this referred CU is marked as non-existent or unable to be referred to.

[0256] Assume that the current CU permission level is equal to 0 (the lowest permission). During parsing, the information within the current CU is parsed from the bitstream with a level of 0. During the parsing process, if it is necessary to refer to the CU information in the surrounding spatial or temporal domain and the CU permission level in the surrounding spatial or temporal domain is greater than the bitstream level, this referred CU is marked as non-existent or unable to be referred to.

[0257] If the current CU permission level is not equal to 0, when parsing, the current CU information is parsed from the bitstream with a non-0 permission level. During the parsing process, the CU information with a permission level less than or equal to the current CU permission level in the surrounding or time domain can be referred to.

[0258] In a possible implementation manner of this embodiment, after the step of obtaining the unit permission level and the user permission level when the privacy coding mode is in the off state, the following steps may further be included:

[0259] If the unit permission level is not the lowest permission level and the user permission level is less than the unit permission level, a reconstructed image block corresponding to the current processing unit is constructed according to a fixed pixel value;

[0260] Or,

[0261] If the unit permission level is not the lowest permission level and the user permission level is less than the unit permission level, a reconstructed image block corresponding to the current processing unit is constructed according to the reconstructed image blocks corresponding to the already reconstructed units around the current processing unit.

[0262] It should be noted that if the unit permission level is not the lowest permission level and the user permission level is less than the unit permission level, it means that the current processing unit is the processing unit corresponding to the privacy area at this time, and the user does not have the permission to obtain the image content of the privacy area. At this time, for this current processing unit, the decoding end needs to skip decoding. In order to ensure the integrity of the subsequent image, a fixed pixel value can be considered for filling at this time to construct a reconstructed image block corresponding to the current processing unit;

[0263] Certainly, it can also be tried to construct a reconstructed image block corresponding to the current processing unit according to the reconstructed image blocks corresponding to the already reconstructed units around the current processing unit. For example: one or more pixel values are extracted from the reconstructed image blocks corresponding to the already reconstructed units around the current processing unit for interpolation filling to construct a reconstructed image block corresponding to the current processing unit.

[0264] In this embodiment, the mode flag bit is extracted from the image bitstream corresponding to the image to be decoded, and the mode flag bit is a sequence-level syntax; whether the privacy coding mode is enabled is determined according to the value of the mode flag bit. Since the mode flag bit is stored in the image bitstream through the sequence-level syntax, after the decoding end obtains the image bitstream, it can quickly determine whether the privacy coding mode is enabled according to the sequence-level syntax.

[0265] Reference Figure 11 , Figure 11 is a schematic flowchart of the third embodiment of an image decoding method of the present application.

[0266] Based on the above first embodiment, before the step S10 of this embodiment's image decoding method, the following steps are further included:

[0267] Step S01': Extract the bitstream corresponding to the lowest privilege level from the image bitstream corresponding to the image to be decoded.

[0268] Step S02': Extract the unit division information from the bitstream corresponding to the lowest privilege level.

[0269] Step S03': Perform unit division according to the unit division information to obtain multiple image processing units.

[0270] Step S04': Traverse the multiple image processing units to obtain the current processing unit;

[0271] It should be noted that in order to ensure that all users can obtain the unit division information, during image encoding, the encoding end will encode and write the unit division information into the bitstream corresponding to the lowest privilege level (i.e., write it into the non-private bitstream). Then, in order to ensure the normal execution of image decoding, the decoding end can first extract the bitstream corresponding to the lowest privilege level from the image bitstream corresponding to the image to be decoded, then extract the unit division information from the bitstream corresponding to the lowest privilege level, and then perform unit division according to the unit division information to obtain multiple image processing units. After that, the multiple image processing units can be traversed, and the image processing unit obtained by traversing can be used as the current processing unit.

[0272] After the step S30, it may further include:

[0273] Cache the reconstructed image block corresponding to the current processing unit;

[0274] When the caching is completed, if the current processing unit is the last image processing unit among the multiple image processing units, construct the reconstructed image corresponding to the image to be decoded according to the cached reconstructed image blocks corresponding to each image processing unit.

[0275] It should be noted that after obtaining the reconstructed image block corresponding to the current processing unit, since it is not certain whether all the obtained multiple image processing units have been processed, that is, whether the image decoding of the image to be decoded has been completed, the reconstructed image block corresponding to the current processing unit can be cached first, and then it is detected whether the current processing unit is the last image processing unit among the obtained multiple image processing units.

[0276] In actual use, if the current processing unit is the last image processing unit among the multiple image processing units, it means that the image decoding of the image to be decoded has been completed. The reconstructed image blocks corresponding to each previously cached image processing unit can be read from the cache and processed such as combined and spliced to generate the reconstructed image corresponding to the image to be decoded.

[0277] If the current processing unit is not the last image processing unit among the multiple image processing units, it means that the image decoding of the image to be decoded has not been completed yet. At this time, the steps of S04' above can be returned to continue the traversal.

[0278] In a possible implementation manner of this embodiment, in order to ensure the display effect of the reconstructed image, before the step of caching the reconstructed image block corresponding to the current processing unit described in the embodiment, the following steps may further be included:

[0279] Extract filtering pixels and reference pixels corresponding to the filtering pixels from the reconstructed image block;

[0280] If the permission level of the filtering pixel is lower than the permission level of any of the reference pixels, skip the filtering process of the filtering pixel and cache the reconstructed image block.

[0281] It should be noted that since the image is obtained through image encoding and image decoding during image reconstruction, inevitably, certain noise will be generated, which affects the display effect of the image. At this time, filtering processing is required to suppress the generated noise to ensure a better display effect of the image. Extracting filtering pixels from the reconstructed image block can be taking some or all of the pixels in the reconstructed image block as filtering pixels. The reference pixel can be the pixel referred to when filtering the filtering pixel, or the pixel at the surrounding position of the filtering pixel. Among them, the pixels at the surrounding position of the filtering pixel can be the pixels corresponding to one or more samples above, below, left, right, upper left, lower left, upper right, and lower right of the filtering pixel. Among them, the permission levels of the filtering pixel and the reference pixel are the same as the permission level of the corresponding processing unit.

[0282] In practical applications, when performing filtering processing, it is also necessary to avoid filtering pixels with a reference permission level higher than the permission level of the filtering pixel. Therefore, the permission level of the filtering pixel can be compared with the permission levels of each reference pixel. If the permission level of the filtering pixel is lower than the permission level of any of the reference pixels, it means that there are pixels with a permission level higher than the permission level of the filtering pixel among the reference pixels used for filtering the filtering pixel at this time. At this time, it may cause a data reading failure due to the user's permission level, and finally an error may occur. To avoid this phenomenon, the filtering process of the filtering pixel can be skipped and the reconstructed image block can be directly cached.

[0283] In actual use, in order to ensure the comprehensiveness of the judgment, the permission level of the filtering pixel can first be compared with the permission levels of the pixels at its surrounding positions. When the permission level of the filtering pixel is higher than the permission levels of the pixels at the surrounding positions, then obtain the pixels referred to when filtering the filtering pixel, and compare the permission level of the filtering pixel with the permission levels of each pixel referred to when filtering.

[0284] Among them, if the permission level of the filtering pixel is lower than that of any pixel at its surrounding positions, or the permission level of the filtering pixel is lower than that of any pixel referred to during filtering, the filtering process for the filtering pixel can be skipped.

[0285] If the permission level of the filtering pixel is greater than or equal to the permission levels of all reference pixels, it means that there are no pixels with a permission level higher than that of the filtering pixel among the reference pixels used for filtering this filtering pixel at this time. At this time, the filtering process can proceed normally. Therefore, the filtering pixel can be filtered according to the reference pixels. Among them, the filtering process can be performed using common filtering algorithms, such as: mean filtering, median filtering, etc.

[0286] In a specific implementation, if the reference pixel is the pixel referred to during filtering of the filtering pixel, then the steps of extracting the filtering pixel and the reference pixel corresponding to the filtering pixel from the reconstructed image block described in this embodiment at this time may include:

[0287] Extract the filtering pixel from the reconstructed image block, and obtain the target pixel according to the reference position corresponding to the filtering pixel;

[0288] If the target pixel is available, use the target pixel as the reference pixel;

[0289] If the target pixel is not available, use the pixel at the position closest to the target pixel in the filtering unit corresponding to the filtering pixel as the reference pixel, and use the permission level of the pixel at the position closest to the target pixel in the filtering unit corresponding to the filtering pixel as the permission level of the reference pixel.

[0290] It should be noted that the reference position corresponding to the filtering pixel can be set in advance by the management personnel of the decoding device according to the used filtering algorithm. The filtering unit corresponding to the filtering pixel can be the filtering unit to which the filtering pixel belongs during the filtering process. The filtering unit can correspond to at least one processing unit. For example, during the filtering process, the filtering unit can include all the pixels in processing unit A and some pixels in processing unit B.

[0291] In actual use, if the target pixel is outside the image, outside the slice boundary when cross-slice filtering is not allowed, or outside the upper or lower boundary of the current processing unit, it can be determined that the target pixel is not available. Otherwise, it can be determined that the target pixel is available.

[0292] It can be understood that if the target pixel is available, it means that during the filtering process, the target pixel can be normally obtained. Therefore, the target pixel can be directly used as the reference pixel. If the target pixel is not available, it means that the target pixel cannot be directly obtained during the filtering process. In this case, substitution is required. At this time, the pixel with the smallest possible difference from it can be used for substitution. Therefore, the pixel at the closest position to the target pixel in the filtering unit corresponding to the filtering pixel can be used as the reference pixel. Since during the filtering process, the permission level of the reference pixel may be read according to the reference position, after substitution, in order to avoid incorrect reading of the permission level, the permission level of the pixel at the closest position to the target pixel in the filtering unit corresponding to the filtering pixel can also be used as the permission level of the reference pixel.

[0293] Among them, there may be multiple reference positions, and multiple reference pixels are obtained. For each reference pixel, it can be determined separately whether it is available and subsequent processing can be performed, without affecting each other.

[0294] In this embodiment, the code stream corresponding to the lowest permission level is extracted from the image code stream corresponding to the image to be decoded; the unit division information is extracted from the code stream corresponding to the lowest permission level; unit division is performed according to the unit division information to obtain multiple image processing units; and the multiple image processing units are traversed to obtain the current processing unit. Since the unit division information is extracted from the code stream corresponding to the lowest permission level, the image processing units can be divided according to the unit division information, ensuring that the image decoding method can be carried out reasonably and orderly.

[0295] Reference Figure 12 , Figure 12 is a schematic flowchart of the fourth embodiment of an image decoding method of the present application.

[0296] Based on the above first embodiment, step S30 of the image decoding method in this embodiment includes:

[0297] Step S301: Obtain multiple sets of quantization parameters set according to the user permission level.

[0298] It should be noted that the decoding end will set multiple sets of quantization parameters according to the user permission level.

[0299] Among them, the multiple sets of quantization parameters are initialized according to the quantization information of the surrounding processing units, and each set of quantization parameters in the set of quantization parameters set corresponds to a permission level.

[0300] For example, assume that the lowest privilege level is 0. If the user privilege level of the decoding end user is 1, then the decoding end will set 1 + 1 = 2 sets of quantization parameters at this time, one set corresponding to privilege level 0 and the other set corresponding to privilege level 1; if the user privilege level of the decoding end user is 2, then the decoding end will set 2 + 1 = 3 sets of quantization parameters at this time, and the 3 sets of quantization parameters correspond to privilege levels 0, 1, and 2 respectively.

[0301] In actual use, the above-mentioned multiple sets of quantization parameters all meet the following conditions:

[0302] If the privilege level of the peripheral processing unit is less than or equal to the privilege level corresponding to the quantization parameter, then the current quantization parameter is equal to the quantization parameter of the peripheral processing unit;

[0303] If the privilege level of the peripheral processing unit is greater than the privilege level corresponding to the quantization parameter, then the quantization parameter is equal to the quantization parameter of the current slice level, the current coding tree unit (LCU), or the current frame level.

[0304] Step S302: Update the multiple sets of quantization parameters.

[0305] It should be noted that updating the multiple sets of quantization parameters can be to update the multiple sets of quantization parameters according to the bitstream read previously.

[0306] In a possible implementation manner of this embodiment, in a fixed-size area, there may be multiple processing units. Then, step S302 described in this embodiment may include:

[0307] If the unit privilege level is not the lowest privilege level and the user privilege level is greater than or equal to the unit privilege level, then select the privacy quantization parameter from the multiple sets of quantization parameters according to the privilege level of the privacy bitstream, and select the non-privacy quantization parameter from the multiple sets of quantization parameters according to the privilege level of the non-privacy bitstream;

[0308] Update the privacy quantization parameter according to the quantization information of the current processing unit extracted from the privacy bitstream, and update the non-privacy quantization parameter according to the quantization information of the current processing unit extracted from the non-privacy bitstream.

[0309] It can be understood that if the unit permission level is not the lowest permission level and the user permission level is greater than or equal to the unit permission level, the decoding end will read the privacy code stream and non-privacy code stream corresponding to the current processing unit at this time. Then, the privacy quantization parameter can be selected from multiple sets of quantization parameters according to the permission level of the privacy code stream, and the non-privacy quantization parameter can be selected from multiple sets of quantization parameters according to the permission level of the non-privacy code stream. After that, the privacy quantization parameter is updated according to the quantization information of the current processing unit extracted from the privacy code stream, and the non-privacy quantization parameter is updated according to the quantization information of the current processing unit extracted from the non-privacy code stream. Among them, the current processing unit can also be divided into multiple sub-blocks, and the quantization information of the current processing unit can include information such as the quantization parameters of each divided sub-block.

[0310] For example: Suppose the user permission level of the decoding end user is 2. Then, the decoding end will set 2 + 1 = 3 sets of quantization parameters at this time. The 3 sets of quantization parameters correspond to the permission levels 0, 1, and 2 respectively. If the permission level of the privacy code stream is 1 and the permission level of the non-privacy code stream is 0 at this time, then the quantization parameter corresponding to the permission level 1 can be updated according to the quantization information of the current processing unit extracted from the privacy code stream, and the quantization parameter corresponding to the permission level 0 can be updated according to the quantization information of the current processing unit extracted from the non-privacy code stream.

[0311] Similarly, for low-permission users or low-permission current processing units, similar processing can also be performed. Then, step S302 described in this embodiment may include:

[0312] If the unit permission level is the lowest permission level, or the user permission level is less than the unit permission level, select the non-privacy quantization parameter from the multiple sets of quantization parameters according to the permission level of the non-privacy code stream;

[0313] Update the non-privacy quantization parameter according to the quantization information of the current processing unit extracted from the non-privacy code stream.

[0314] It should be noted that if the unit permission level is the lowest permission level, or the user permission level is less than the unit permission level, the decoding end will only read the non-privacy code stream at this time. Then, the non-privacy quantization parameter can be selected from multiple sets of quantization parameters according to the non-privacy code stream, and the non-privacy quantization parameter is updated according to the quantization information of the current processing unit extracted from the non-privacy code stream.

[0315] Step S303: After the update is completed, decode the code stream to be decoded to obtain the reconstructed image block corresponding to the current processing unit.

[0316] It can be understood that after the update is completed, when decoding the to-be-decoded bitstream thereafter, the corresponding target quantization parameter can be obtained from the updated multiple sets of quantization parameters according to the permission level of the to-be-decoded bitstream, and then inverse quantization and other processing are performed based on the target quantization parameter to decode the to-be-decoded bitstream and perform image reconstruction to obtain the reconstructed image block corresponding to the current processing unit.

[0317] In this embodiment, multiple sets of quantization parameters set according to the user permission level are obtained; the multiple sets of quantization parameters are updated; after the update is completed, the to-be-decoded bitstream is decoded to obtain the reconstructed image block corresponding to the current processing unit. Since multiple sets of quantization parameters are set according to the user permission level, and before decoding, the multiple sets of quantization parameters will be updated according to the obtained bitstream, ensuring that even if the current processing unit is further divided into sub-blocks downward, the quantization parameters will not be confused.

[0318] Reference Figure 13 , Figure 13 is a schematic flowchart of the fifth embodiment of an image decoding method of the present application.

[0319] Based on the above first embodiment, step S30 of the image decoding method in this embodiment includes:

[0320] Step S301': If it is detected that the sample reference condition is satisfied, obtain the reference sample corresponding to the current processing unit.

[0321] It should be noted that if the encoding end uses intra prediction, then intra prediction can also be performed when decoding the image at this time. However, if intra prediction is to be used, it is necessary to detect whether there is a reference sample. Among them, the reference sample can be a sample at the reference position corresponding to the current processing unit, and the reference position corresponding to the current processing unit can be determined according to the prediction mode of intra prediction.

[0322] In a specific implementation, in order to determine whether sample reference can be performed, before step S301' in this embodiment, it may further include:

[0323] If there is at least one reference sample at the reference position corresponding to the current processing unit that belongs to the same slice level as the current processing unit, has completed image reconstruction, and the corresponding permission level is less than or equal to the permission level of the to-be-decoded bitstream, it is determined that the sample reference condition is satisfied.

[0324] It can be understood that if there is at least one reference sample at the reference position corresponding to the current processing unit that belongs to the same slice level as the current processing unit, the image reconstruction has been completed, and the corresponding permission level is less than or equal to the permission level of the bitstream to be decoded, it means that there is at least one sample in the current processing unit that can be directly referenced for intra prediction. Therefore, it can be determined that the sample reference condition is satisfied.

[0325] Step S302': Perform intra prediction based on the referenceable samples to obtain a prediction result.

[0326] Step S303': Decode according to the prediction result and the bitstream to be decoded to obtain the reconstructed image block corresponding to the current processing unit.

[0327] It can be understood that after obtaining the referenceable samples, intra prediction can be performed based on the referenceable samples, and the bitstream to be decoded can be assisted in decoding according to the intra prediction result, so as to accelerate the decoding and image reconstruction speed, thereby ensuring that the reconstructed image block corresponding to the current processing unit can be quickly obtained.

[0328] In a specific implementation, to ensure that the intra prediction can be performed normally, step S302' in this embodiment may include:

[0329] Use the referenceable samples with corresponding permission levels greater than the permission level of the bitstream to be decoded as target samples;

[0330] Perform pixel substitution processing on the target samples among the referenceable samples, and the processing method of the pixel substitution processing is determined according to the position of the target samples relative to the current processing unit;

[0331] Perform intra prediction based on the referenceable samples after the substitution processing to obtain a prediction result.

[0332] It should be noted that in some cases, the referenceable samples may include some samples with corresponding permission levels greater than the permission level of the bitstream to be decoded. Due to the influence of the user permission level, the information of such samples may not be readable during decoding and cannot be directly referenced and needs to be processed. At this time, pixel substitution processing is performed, and then intra prediction is performed based on the referenceable samples after the substitution processing to obtain a prediction result, so as to ensure that the intra prediction will not cause execution errors due to information acquisition failure.

[0333] For ease of understanding, the substitution processing is now described in conjunction with Figure 14 This is only for illustrative purposes and does not limit the present solution. Figure 14 This is a schematic diagram of the reference sample position for this embodiment.

[0334] For directly above the current processing unit Figure 14 among r1 - r8 (or directly to the left Figure 14For the reference samples in c1-c8 in , if the permission level of the first part of the samples is less than or equal to the permission level of the current processing unit, and the permission level of the second part of the samples is greater than the permission level of the current processing unit, then use a fixed value to replace the pixel values of the second part of the reference samples.

[0335] For the upper right of the current processing unit Figure 14 in r9-r16 (or the lower left Figure 14 For the reference samples in c9-c16 in , if the permission level of the first part of the samples is less than or equal to the permission level of the current processing unit, and the permission level of the second part of the pixels is greater than the permission level of the current processing unit, and the upper right reference samples are arranged in the order from left to right, and the lower left reference samples are arranged in the order from top to bottom. Then the pixel r[i] of the second part of the upper right reference samples = r[i-1], and the pixel c[i] of the second part of the lower left reference samples = c[i-1].

[0336] It should be noted that the embodiments of the image decoding method provided in the above embodiments can be combined in any way to obtain new embodiments.

[0337] In this embodiment, if it is detected that the sample reference condition is satisfied, the reference samples corresponding to the current processing unit are obtained; intra-frame prediction is performed according to the reference samples to obtain a prediction result; decoding is performed according to the prediction result and the bitstream to be decoded to obtain a reconstructed image block corresponding to the current processing unit. Since reasonable reference samples are used for intra-frame prediction and the image decoding is assisted according to the intra-frame prediction result, the decoding efficiency of the image decoding is improved.

[0338] An embodiment of the present application provides an image encoding method, referring to Figure 15 , Figure 15 is a schematic flowchart of the first embodiment of an image encoding method of the present application.

[0339] In this embodiment, the image encoding method includes the following steps:

[0340] Step A10: When the privacy encoding mode is in the enabled state, obtain a privacy determination result, where the privacy determination result is used to indicate whether the current processing unit is located in a privacy area.

[0341] It should be noted that the execution subject of this embodiment may be a decoding end device, and the decoding end device may be an electronic device applied to the encoding end for image encoding. The electronic device may be an electronic device such as a personal computer or a server, or other devices that can implement the same or similar functions. This embodiment does not limit this. In this embodiment and the following embodiments, the decoding end device is used to illustrate the image encoding method of this embodiment.

[0342] It should be noted that if the privacy coding mode is in the enabled state, it means that the improved privacy protection mode is adopted at this time. At this time, additional processing methods need to be adopted for the image content corresponding to the privacy area. Therefore, the privacy determination result indicating whether the current processing unit is located in the privacy area can be obtained first.

[0343] Step A20: Encode the image corresponding to the current processing unit according to the privacy determination result to generate at least one bitstream corresponding to the current processing unit.

[0344] It can be understood that different encoding methods need to be adopted for the image content of the privacy area and the image content of the non-privacy area, and at least one bitstream corresponding to the current processing unit can be generated.

[0345] In a specific implementation, to ensure normal decoding during image decoding, step S20 in this embodiment may include:

[0346] If the privacy determination result is that it is located in the privacy area, encode the image corresponding to the current processing unit to generate the privacy bitstream and non-privacy bitstream corresponding to the current processing unit;

[0347] If the privacy determination result is that it is not located in the privacy area, encode the image corresponding to the current processing unit to generate the non-privacy bitstream corresponding to the current processing unit.

[0348] It can be understood that if the privacy determination result is that it is located in the privacy area, it means that the current processing unit is the image processing unit corresponding to the privacy area. At this time, to prevent the image encoding end from skipping the current during decoding, the image corresponding to the current processing unit needs to be encoded to generate the privacy bitstream and non-privacy bitstream corresponding to the current processing unit;

[0349] If the privacy determination result is not that it is located in the privacy area, it means that the current processing unit is not the image processing unit corresponding to the privacy area. At this time, the image corresponding to the current processing unit can be directly encoded to generate the non-privacy bitstream corresponding to the current processing unit.

[0350] Among them, different entropy encoders are used for encoding the privacy bitstream and non-privacy bitstream. The entropy encoder for encoding the privacy bitstream corresponds to the entropy decoder for reading the privacy bitstream at the decoding end, and the entropy encoder for encoding the non-privacy bitstream corresponds to the entropy decoder for reading the non-privacy bitstream at the decoding end.

[0351] In a possible implementation manner of this embodiment, to accelerate the processing speed of image encoding, step A20 in this embodiment may include:

[0352] If it is detected that the sample reference condition is satisfied, obtain the reference sample corresponding to the current processing unit;

[0353] Perform intra prediction according to the reference sample to obtain a prediction result;

[0354] Encode the image corresponding to the current processing unit based on the prediction result and the privacy determination result to generate at least one bitstream corresponding to the current processing unit.

[0355] It should be noted that the sample reference conditions, the method for obtaining the reference sample corresponding to the current processing unit, and the subsequent intra prediction method are all similar to those in the fifth embodiment of the above image decoding method. For details, please refer to the above description and will not be elaborated here.

[0356] In a possible implementation manner of this embodiment, to facilitate the decoding end to correctly perform permission control, after step A20 in this embodiment, the following steps may further be included:

[0357] Obtain the permission level of the privacy area to obtain a privacy permission level;

[0358] Set the permission levels of the current processing unit and the privacy bitstream to the privacy permission level, and set the permission level of the non-privacy bitstream to the lowest permission level.

[0359] It can be understood that the permission level of the privacy bitstream is the same as that of the current processing unit, and its permission level is set according to the permission level of the privacy area. Therefore, the permission levels of the current processing unit and the privacy bitstream can be set according to the permission level of the privacy area. When the decoding end performs decoding, the non-privacy bitstream is a bitstream that can be obtained by all users with permissions. Therefore, the permission level of the non-privacy bitstream can be set to the lowest permission level.

[0360] In a specific implementation, to ensure compatibility with the previous privacy protection mode, the image encoding method in this embodiment may further include:

[0361] When the privacy encoding mode is in the off state, obtain the privacy determination result;

[0362] If the privacy determination result is that the area is in the privacy area, encode the image corresponding to the current processing unit to generate a privacy bitstream corresponding to the current processing unit;

[0363] If the privacy determination result is that the area is not in the privacy area, encode the image corresponding to the current processing unit to generate a non-privacy bitstream corresponding to the current processing unit.

[0364] It should be noted that if the privacy encoding mode is in the off state, it means that the improved privacy protection mode is not adopted at this time. In this case, the processing flow of the privacy protection mode before the improvement needs to be followed. Therefore, according to the privacy determination result, the privacy bitstream or non-privacy bitstream corresponding to the current processing unit is generated.

[0365] In an implementable manner of this embodiment, before step A10 of this embodiment, the following may further be included:

[0366] Perform unit division on the image to be encoded according to the unit division information to obtain a plurality of image processing units;

[0367] Traverse the plurality of image processing units to obtain the current processing unit;

[0368] After step A20, the following may further be included:

[0369] If the current processing unit is the last processing unit among the plurality of image processing units, construct an image bitstream according to at least one bitstream corresponding to each processing unit;

[0370] Send the image bitstream to the decoding end.

[0371] It should be noted that the unit division information can be preferably set by the management personnel of the encoding device according to actual needs. The image to be encoded can be the image data that needs to be encoded currently. The image to be encoded can be a single-frame image or multiple-frame images. This embodiment does not limit this.

[0372] It can be understood that if the current processing unit is the last processing unit among the plurality of image processing units, it means that the image encoding has been completed at this time. Then, an image bitstream can be constructed according to at least one bitstream corresponding to each processing unit.

[0373] Among them, when constructing the image bitstream, the bitstreams obtained by each entropy encoder can be spliced, and then the bitstreams with different permissions can be spliced based on the frame level to obtain the complete image bitstream.

[0374] It should be noted that the decoding end device can also perform the reverse processing in the image decoding process of the decoding device. In this process, the image encoding method and the image decoding method can refer to each other. All the encoding methods corresponding to the embodiments of the image decoding method are applicable to this application.

[0375] Exemplarily, each of the above embodiments can be implemented alone or in combination. For example, each of Embodiments 1-6 can be implemented alone, and at least two of Embodiments 1-6 can be implemented in combination. Exemplarily, in each of the above embodiments, the content at the encoding end can also be adapted to the steps at the decoding end, that is, the implementation manner at the encoding end can be obtained according to the steps at the decoding end, which will not be repeated here.

[0376] In this embodiment, when the privacy encoding mode is in the enabled state, a privacy determination result is obtained, and the privacy determination result is used to indicate whether the current processing unit is located in a privacy area; according to the privacy determination result, the image corresponding to the current processing unit is encoded to generate at least one bitstream corresponding to the current processing unit. Since the privacy bitstream and the non-privacy bitstream are generated by encoding the image of the current processing unit when it is determined that the current processing unit is the image processing unit corresponding to the privacy area, it is ensured that when the decoding end performs decoding, a low-privilege user can also decode the non-privacy bitstream to obtain the de-privatized image content corresponding to the privacy area.

[0377] Based on the same application concept as the above method, an embodiment of the present application also proposes a storage medium, on which an image decoding program and / or an image encoding program are stored. When the image decoding program is executed by a processor, the steps of the image decoding method as described above are implemented, and when the image encoding program is executed by a processor, the steps of the image encoding method as described above are implemented.

[0378] Based on the same application concept as the above method, an embodiment of the present application also provides a computer application program, which, when executed by a processor, can implement the image decoding method or the image encoding method disclosed in the above examples of the present application.

[0379] Refer to Figure 16 , Figure 16 which is the structural block diagram of the first embodiment of the image decoding device of the present application.

[0380] As Figure 16 shown, the image decoding device proposed in the embodiment of the present application includes:

[0381] An acquisition module 10, configured to obtain a unit privilege level and a user privilege level when the privacy encoding mode is in the enabled state, where the unit privilege level is the privilege level of the current processing unit, and the user privilege level is the privilege level of the user at the decoding end;

[0382] An extraction module 20, configured to determine the bitstream to be decoded corresponding to the current processing unit according to the unit privilege level and the user privilege level;

[0383] The decoding module 30 is configured to decode the to-be-decoded bitstream to obtain a reconstructed image block corresponding to the current processing unit.

[0384] In this embodiment, when the privacy encoding mode is in the enabled state, the obtaining unit obtains the unit privilege level and the user privilege level. The unit privilege level is the privilege level of the current processing unit, and the user privilege level is the privilege level of the decoding-end user. The to-be-decoded bitstream corresponding to the current processing unit is determined according to the unit privilege level and the user privilege level. The to-be-decoded bitstream is decoded to obtain a reconstructed image block corresponding to the current processing unit. Since when the privacy encoding mode is in the enabled state, the corresponding to-be-decoded bitstream is selected for decoding according to the unit privilege level and the user privilege level, it is ensured that even a low-privilege user can decode the non-privacy bitstream and obtain the reconstructed image corresponding to the de-privatized image content in the privacy area, avoiding the low-privilege user skipping the decoding of the processing unit corresponding to the privacy area.

[0385] In a possible implementation manner of this embodiment, the extraction module 20 is further configured to, if the unit privilege level is not the lowest privilege level and the user privilege level is greater than or equal to the unit privilege level, read the privacy bitstream and the non-privacy bitstream corresponding to the current processing unit. The privacy bitstream and the non-privacy bitstream include at least one piece of information for prediction, inverse quantization, or inverse transformation. The non-privacy bitstream is the bitstream corresponding to the lowest privilege level, and the privacy bitstream is the bitstream corresponding to the unit privilege level. The privacy bitstream is used as the to-be-decoded bitstream corresponding to the current processing unit.

[0386] In a possible implementation manner of this embodiment, the extraction module 20 is further configured to read the privacy bitstream corresponding to the current processing unit according to the first entropy decoder, and read the non-privacy bitstream corresponding to the current processing unit according to the second entropy decoder. The first entropy decoder is the entropy decoder corresponding to the unit privilege level, and the second entropy decoder is the entropy decoder corresponding to the lowest privilege level.

[0387] In a possible implementation manner of this embodiment, the extraction module 20 is further configured to, if the unit privilege level is not the lowest privilege level and the user privilege level is less than the unit privilege level, read the non-privacy bitstream corresponding to the current processing unit. The non-privacy bitstream includes at least one piece of information for prediction, inverse quantization, or inverse transformation. The non-privacy bitstream is used as the to-be-decoded bitstream corresponding to the current processing unit.

[0388] In a possible implementation manner of this embodiment, the obtaining module 10 is further configured to extract a mode flag bit from the image bitstream corresponding to the to-be-decoded image. The mode flag bit is a sequence-level syntax. Whether the privacy encoding mode is enabled is determined according to the value of the mode flag bit.

[0389] In a possible implementation manner of this embodiment, the obtaining module 10 is further configured to, when the privacy encoding mode is in the off state, obtain the unit permission level and the user permission level; if the unit permission level is not the lowest permission level and the user permission level is greater than or equal to the unit permission level, obtain the bitstream corresponding to the unit permission level; use the bitstream corresponding to the unit permission level as the bitstream to be decoded corresponding to the current processing unit; decode the bitstream to be decoded to obtain the reconstructed image block corresponding to the current processing unit.

[0390] In a possible implementation manner of this embodiment, the obtaining module 10 is further configured to, if the unit permission level is not the lowest permission level and the user permission level is less than the unit permission level, construct the reconstructed image block corresponding to the current processing unit according to a fixed pixel value; or, if the unit permission level is not the lowest permission level and the user permission level is less than the unit permission level, construct the reconstructed image block corresponding to the current processing unit according to the reconstructed image blocks corresponding to the already reconstructed units around the current processing unit.

[0391] In a possible implementation manner of this embodiment, the extraction module 20 is further configured to, if the unit permission level is the lowest permission level, read the non-privacy bitstream corresponding to the current processing unit; use the non-privacy bitstream as the bitstream to be decoded corresponding to the current processing unit.

[0392] In a possible implementation manner of this embodiment, the extraction module 20 is further configured to determine the bitstream to be parsed according to the unit permission level and the user permission level; obtain the surrounding processing units whose corresponding permission levels are less than or equal to the permission level of the bitstream to be parsed to obtain the referenceable units, where the surrounding processing units are the processing units adjacent to the current processing unit in the time domain or the spatial domain; refer to the decoded information corresponding to the referenceable units, read the bitstream to be parsed, and select the bitstream to be decoded corresponding to the current processing unit from the bitstream to be parsed, where the decoded information is the information that has been parsed and decoded when performing image reconstruction on the referenceable units.

[0393] In a possible implementation manner of this embodiment, the decoding module 30 is further configured to, if the custom mode is not enabled, decode the bitstream to be decoded to obtain the reconstructed image block corresponding to the current processing unit.

[0394] In a possible implementation manner of this embodiment, the decoding module 30 is further configured to, if the custom mode is enabled, obtain the reconstructed image block corresponding to the current processing unit in any one of the following ways:

[0395] Decode the to-be-decoded bitstream to obtain a decoded image block, and replace the decoded image block with an image block constructed by pixel values specified in a custom mode as the reconstructed image block corresponding to the current processing unit;

[0396] Do not decode the to-be-decoded bitstream, and obtain pixel values specified in a custom mode, and construct a reconstructed image block corresponding to the current processing unit according to the pixel values;

[0397] Decode the to-be-decoded bitstream to obtain a reconstructed image block corresponding to the current processing unit.

[0398] In a possible implementation manner of this embodiment, the manner of obtaining the reconstructed image block corresponding to the current processing unit is specified according to the mode parameter corresponding to the custom mode.

[0399] In a possible implementation manner of this embodiment, the obtaining module 10 is further configured to extract the bitstream corresponding to the lowest permission level from the image bitstream corresponding to the to-be-decoded image; extract unit division information from the bitstream corresponding to the lowest permission level; perform unit division according to the unit division information to obtain a plurality of image processing units; traverse the plurality of image processing units to obtain the current processing unit;

[0400] The decoding module 30 is further configured to cache the reconstructed image block corresponding to the current processing unit; when the caching is completed, if the current processing unit is the last image processing unit among the plurality of image processing units, construct a reconstructed image corresponding to the to-be-decoded image according to the cached reconstructed image blocks corresponding to the respective image processing units.

[0401] In a possible implementation manner of this embodiment, the decoding module 30 is further configured to extract filtering pixels and reference pixels corresponding to the filtering pixels from the reconstructed image block, where the reference pixel is a pixel referred to when filtering the filtering pixel, or a pixel at a peripheral position of the filtering pixel; if the permission level of the filtering pixel is lower than the permission level of any of the reference pixels, skip the filtering process of the filtering pixel and cache the reconstructed image block.

[0402] In a possible implementation manner of this embodiment, the decoding module 30 is further configured to, if the permission level of the filtering pixel is greater than or equal to the permission levels of all the reference pixels, perform filtering processing on the filtering pixel according to the reference pixels.

[0403] In a possible implementation manner of this embodiment, the decoding module 30 is further configured to extract filtered pixels from the reconstructed image block, and obtain target pixels according to the reference positions corresponding to the filtered pixels; if the target pixels are available, use the target pixels as reference pixels; if the target pixels are not available, use the pixels at the position closest to the target pixels in the filtering unit corresponding to the filtered pixels as reference pixels, and use the permission level of the pixels at the position closest to the target pixels in the filtering unit corresponding to the filtered pixels as the permission level of the reference pixels.

[0404] In a possible implementation manner of this embodiment, the current processing unit is an enhancement layer processing unit;

[0405] The decoding module 30 is further configured to obtain the reference information corresponding to the current processing unit in the base layer, where the reference information includes mode reference information and pixel reference information; if the permission level of the to-be-decoded code stream is greater than the permission level of the reference information, decode the to-be-decoded code stream with reference to the reference information to obtain the reconstructed image block corresponding to the current processing unit.

[0406] In a possible implementation manner of this embodiment, the decoding module 30 is further configured to obtain multiple sets of quantization parameters set according to the user permission level; update the multiple sets of quantization parameters; after the update is completed, decode the to-be-decoded code stream to obtain the reconstructed image block corresponding to the current processing unit.

[0407] In a possible implementation manner of this embodiment, each set of quantization parameters in the multiple sets of quantization parameters corresponds to a permission level, and the multiple sets of quantization parameters are initialized according to the quantization information of the surrounding processing units;

[0408] Each set of quantization parameters in the multiple sets of quantization parameters satisfies the following conditions:

[0409] If the permission level of the surrounding processing unit is less than or equal to the permission level corresponding to the quantization parameter, the current quantization parameter is equal to the quantization parameter of the surrounding processing unit;

[0410] If the permission level of the surrounding processing unit is greater than the permission level corresponding to the quantization parameter, the quantization parameter is equal to the quantization parameter of the current slice level, the current coding tree unit, or the current frame level.

[0411] In a possible implementation of this embodiment, the decoding module 30 is further configured to, if the unit permission level is not the lowest permission level and the user permission level is greater than or equal to the unit permission level, select privacy quantization parameters from the multiple sets of quantization parameters according to the permission level of the privacy bitstream, and select non-privacy quantization parameters from the multiple sets of quantization parameters according to the permission level of the non-privacy bitstream; update the privacy quantization parameters according to the quantization information of the current processing unit extracted from the privacy bitstream, and update the non-privacy quantization parameters according to the quantization information of the current processing unit extracted from the non-privacy bitstream.

[0412] In a possible implementation of this embodiment, the decoding module 30 is further configured to, if the unit permission level is the lowest permission level or the user permission level is less than the unit permission level, select non-privacy quantization parameters from the multiple sets of quantization parameters according to the permission level of the non-privacy bitstream; update the non-privacy quantization parameters according to the quantization information of the current processing unit extracted from the non-privacy bitstream.

[0413] In a possible implementation of this embodiment, the decoding module 30 is further configured to, if it is detected that the sample reference condition is satisfied, obtain the reference sample corresponding to the current processing unit; perform intra prediction according to the reference sample to obtain a prediction result; and perform decoding according to the prediction result and the bitstream to be decoded to obtain the reconstructed image block corresponding to the current processing unit.

[0414] In a possible implementation of this embodiment, the decoding module 30 is further configured to determine that the sample reference condition is satisfied if there is at least one reference sample at the reference position corresponding to the current processing unit that belongs to the same slice level as the current processing unit, has completed image reconstruction, and the corresponding permission level is less than or equal to the permission level of the bitstream to be decoded.

[0415] In a possible implementation of this embodiment, the decoding module 30 is further configured to use the reference sample whose corresponding permission level is greater than the permission level of the bitstream to be decoded as the target sample; perform pixel substitution processing on the target sample in the reference sample, and the processing method of the pixel substitution processing is determined according to the position of the target sample relative to the current processing unit; and perform intra prediction based on the reference sample after the substitution processing to obtain a prediction result.

[0416] It should be understood that the above is only for illustration purposes and does not impose any limitation on the technical solution of the present application. In specific applications, the image decoding device provided in this embodiment may execute the image decoding method provided in any of the above embodiments, or may also execute the image decoding method provided in the embodiment formed by combining at least two of the above embodiments. Those skilled in the art can set it according to needs, and the present application does not make any restrictions on this.

[0417] It should be noted that the above-described work process is only illustrative and does not limit the protection scope of the present application. In actual applications, those skilled in the art can select some or all of them according to actual needs to achieve the purpose of the solution of this embodiment, and no restrictions are imposed here.

[0418] In addition, for the technical details not described in detail in this embodiment, reference can be made to the image decoding method provided in any embodiment of the present application, and details will not be elaborated here.

[0419] Refer to Figure 17 , Figure 17 which is the structural block diagram of the first embodiment of the image encoding device of the present application.

[0420] As Figure 17 shown, the image encoding device proposed in the embodiment of the present application includes:

[0421] An acquisition module 100, configured to acquire a privacy determination result when the privacy encoding mode is in an enabled state, where the privacy determination result is used to indicate whether the current processing unit is located in a privacy area;

[0422] An encoding module 200, configured to encode the image corresponding to the current processing unit according to the privacy determination result, and generate at least one bitstream corresponding to the current processing unit.

[0423] In this embodiment, when the privacy encoding mode is in an enabled state, a privacy determination result is acquired, where the privacy determination result is used to indicate whether the current processing unit is located in a privacy area; the image corresponding to the current processing unit is encoded according to the privacy determination result, and at least one bitstream corresponding to the current processing unit is generated. Since the image of the current processing unit is encoded to generate a privacy bitstream and a non-privacy bitstream when it is determined that the current processing unit is the image processing unit corresponding to the privacy area, it is ensured that when the decoding end performs decoding, a low-privilege user can also decode the non-privacy bitstream and obtain the de-privatized image content corresponding to the privacy area.

[0424] In a possible implementation of this embodiment, the encoding module 200 is further configured to, if the privacy determination result is that the region is a privacy region, encode the image corresponding to the current processing unit to generate a privacy bitstream and a non-privacy bitstream corresponding to the current processing unit; if the privacy determination result is that the region is not a privacy region, encode the image corresponding to the current processing unit to generate a non-privacy bitstream corresponding to the current processing unit.

[0425] In a possible implementation of this embodiment, different entropy encoders are used for encoding the privacy bitstream and the non-privacy bitstream;

[0426] The encoding module 200 is further configured to obtain the permission level of the privacy region to obtain a privacy permission level; set the permission levels of the current processing unit and the privacy bitstream to the privacy permission level, and set the permission level of the non-privacy bitstream to the lowest permission level.

[0427] In a possible implementation of this embodiment, the encoding module 200 is further configured to, when the privacy encoding mode is in the off state, obtain a privacy determination result; if the privacy determination result is that the region is a privacy region, encode the image corresponding to the current processing unit to generate a privacy bitstream corresponding to the current processing unit; if the privacy determination result is that the region is not a privacy region, encode the image corresponding to the current processing unit to generate a non-privacy bitstream corresponding to the current processing unit.

[0428] In a possible implementation of this embodiment, the acquisition module 100 is further configured to divide the image to be encoded according to the unit division information to obtain a plurality of image processing units; traverse the plurality of image processing units to obtain a current processing unit;

[0429] The encoding module 200 is further configured to, if the current processing unit is the last processing unit among the plurality of image processing units, construct an image bitstream according to at least one bitstream corresponding to each processing unit; send the image bitstream to the decoding end.

[0430] In a possible implementation of this embodiment, the encoding module 200 is further configured to, if it is detected that a sample reference condition is satisfied, obtain a reference sample corresponding to the current processing unit; perform intra prediction according to the reference sample to obtain a prediction result; encode the image corresponding to the current processing unit based on the prediction result and the privacy determination result to generate at least one bitstream corresponding to the current processing unit.

[0431] It should be understood that the above is only for illustrative purposes and does not constitute any limitation to the technical solution of this application. In specific applications, those skilled in the art can make settings according to needs, and this application does not make any restrictions in this regard.

[0432] It should be understood that the above is only for illustrative purposes and does not constitute any limitation to the technical solution of this application. In specific applications, those skilled in the art can make settings according to needs, and this application does not make any restrictions in this regard.

[0433] It should be noted that the above-described work process is only illustrative and does not constitute a limitation to the protection scope of this application. In actual applications, those skilled in the art can select some or all of them according to actual needs to achieve the purpose of the solution of this embodiment, and no restrictions are made here.

[0434] In addition, for the technical details not described in detail in this embodiment, reference can be made to the image coding method provided in any embodiment of this application, and details will not be elaborated here.

[0435] In addition, it should be noted that in this article, the term "including", "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such process, method, article or system. Without further limitations, an element defined by the statement "including a..." does not exclude the existence of another identical element in the process, method, article or system including that element.

[0436] The serial numbers of the embodiments of this application above are only for description and do not represent the superiority or inferiority of the embodiments.

[0437] Through the description of the above embodiments, those skilled in the art can clearly understand that the above embodiment methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as a read-only memory (ROM) / RAM, magnetic disk, optical disk), and includes several instructions to enable a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of this application.

[0438] The above are only the preferred embodiments of the present application, and do not limit the patent scope of the present application accordingly. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of the present application, or directly or indirectly applied to other related technical fields, shall similarly be included within the patent protection scope of the present application.

Claims

1. An image decoding method, characterized in that, The described image decoding method includes: When the privacy encoding mode is in the enabled state, obtain the unit permission level and the user permission level, where the unit permission level is the permission level of the current processing unit, and the user permission level is the permission level of the user at the decoding end; Determine the code stream to be decoded corresponding to the current processing unit according to the unit permission level and the user permission level; Decode the code stream to be decoded to obtain the reconstructed image block corresponding to the current processing unit.

2. The image decoding method according to claim 1, characterized in that, The determining the code stream to be decoded corresponding to the current processing unit according to the unit permission level and the user permission level includes: If the unit permission level is not the lowest permission level and the user permission level is greater than or equal to the unit permission level, read the privacy code stream and the non-privacy code stream corresponding to the current processing unit, where the privacy code stream and the non-privacy code stream include at least one piece of information for prediction, inverse quantization, or inverse transformation, the non-privacy code stream is the code stream corresponding to the lowest permission level, and the privacy code stream is the code stream corresponding to the unit permission level; Use the privacy code stream as the code stream to be decoded corresponding to the current processing unit.

3. The image decoding method according to claim 2, wherein The reading the privacy code stream and the non-privacy code stream corresponding to the current processing unit includes: Read the privacy code stream corresponding to the current processing unit according to the first entropy decoder, and read the non-privacy code stream corresponding to the current processing unit according to the second entropy decoder, where the first entropy decoder is the entropy decoder corresponding to the unit permission level, and the second entropy decoder is the entropy decoder corresponding to the lowest permission level.

4. The image decoding method according to claim 1 or 2, characterized in that, The determining the code stream to be decoded corresponding to the current processing unit according to the unit permission level and the user permission level includes: If the unit permission level is not the lowest permission level and the user permission level is less than the unit permission level, read the non-privacy code stream corresponding to the current processing unit, where the non-privacy code stream includes at least one piece of information for prediction, inverse quantization, or inverse transformation; Use the non-privacy code stream as the code stream to be decoded corresponding to the current processing unit.

5. The image decoding method according to claim 1, wherein Before the obtaining the unit permission level and the user permission level when the privacy encoding mode is in the enabled state, it further includes: Extract a mode flag from the image code stream corresponding to the image to be decoded, where the mode flag is a sequence-level syntax; Determine whether the privacy encoding mode is enabled according to the value of the mode flag.

6. The image decoding method according to claim 5, wherein After the determining whether the privacy encoding mode is enabled according to the value of the mode flag, it further includes: When the privacy encoding mode is in the disabled state, obtain the unit permission level and the user permission level; If the unit permission level is not the lowest permission level and the user permission level is greater than or equal to the unit permission level, obtain the code stream corresponding to the unit permission level; Use the code stream corresponding to the unit permission level as the code stream to be decoded corresponding to the current processing unit; Decode the code stream to be decoded to obtain the reconstructed image block corresponding to the current processing unit.

7. The image decoding method according to claim 6, wherein After the obtaining the unit permission level and the user permission level when the privacy encoding mode is in the disabled state, it further includes: If the unit permission level is not the lowest permission level and the user permission level is less than the unit permission level, a reconstructed image block corresponding to the current processing unit is constructed according to a fixed pixel value; Or, If the unit permission level is not the lowest permission level and the user permission level is less than the unit permission level, a reconstructed image block corresponding to the current processing unit is constructed according to the reconstructed image blocks corresponding to the already reconstructed units around the current processing unit.

8. The image decoding method according to claim 1 or 2, characterized in that The determining of the bitstream to be decoded corresponding to the current processing unit according to the unit permission level and the user permission level includes: If the unit permission level is the lowest permission level, a non-private bitstream corresponding to the current processing unit is read; The non-private bitstream is used as the bitstream to be decoded corresponding to the current processing unit.

9. The image decoding method according to claim 1, wherein, The determining of the bitstream to be decoded corresponding to the current processing unit according to the unit permission level and the user permission level includes: Determine the bitstream to be parsed according to the unit permission level and the user permission level; Obtain surrounding processing units whose corresponding permission levels are less than or equal to the permission level of the bitstream to be parsed to obtain referenceable units, where the surrounding processing units are processing units adjacent to the current processing unit in the time domain or the spatial domain; Refer to the decoded information corresponding to the referenceable units, read the bitstream to be parsed, and select the bitstream to be decoded corresponding to the current processing unit from the bitstream to be parsed, where the decoded information is the information that has been parsed and decoded during image reconstruction of the referenceable units.

10. The image decoding method according to claim 1, wherein The decoding of the bitstream to be decoded to obtain a reconstructed image block corresponding to the current processing unit includes: If the custom mode is not enabled, the bitstream to be decoded is decoded to obtain a reconstructed image block corresponding to the current processing unit.

11. The image decoding method according to claim 1, wherein, The decoding of the bitstream to be decoded to obtain a reconstructed image block corresponding to the current processing unit includes: If the custom mode is enabled, the reconstructed image block corresponding to the current processing unit is obtained in any of the following ways: Decode the bitstream to be decoded to obtain a decoded image block, and replace the decoded image block with an image block constructed by the pixel value specified in the custom mode as the reconstructed image block corresponding to the current processing unit; Do not decode the bitstream to be decoded, obtain the pixel value specified in the custom mode, and construct a reconstructed image block corresponding to the current processing unit according to the pixel value; Decode the bitstream to be decoded to obtain a reconstructed image block corresponding to the current processing unit.

12. The image decoding method according to claim 11, wherein Specify the way to obtain the reconstructed image block corresponding to the current processing unit according to the mode parameters corresponding to the custom mode.

13. The image decoding method according to claim 1, wherein Before obtaining the unit permission level and the user permission level when the privacy encoding mode is in the enabled state, it further includes: Extract the bitstream corresponding to the lowest permission level from the image bitstream corresponding to the image to be decoded; Extract unit division information from the bitstream corresponding to the lowest permission level; Perform unit division according to the unit division information to obtain a plurality of image processing units; Traverse the plurality of image processing units to obtain the current processing unit; After decoding the bitstream to be decoded to obtain a reconstructed image block corresponding to the current processing unit, it further includes: Cache the reconstructed image block corresponding to the current processing unit; When the caching is completed, if the current processing unit is the last image processing unit among the multiple image processing units, construct the reconstructed image corresponding to the image to be decoded according to the cached reconstructed image blocks corresponding to each image processing unit.

14. The image decoding method according to claim 13, wherein, Before caching the reconstructed image block corresponding to the current processing unit, it further includes: Extract filtering pixels and reference pixels corresponding to the filtering pixels from the reconstructed image block, where the reference pixels are the pixels referred to when filtering the filtering pixels or the pixels at the surrounding positions of the filtering pixels; If the permission level of the filtering pixel is less than the permission level of any of the reference pixels, skip the filtering process of the filtering pixel and cache the reconstructed image block.

15. The image decoding method according to claim 14, wherein After extracting the filtering pixels and the reference pixels corresponding to the filtering pixels from the reconstructed image block, it further includes: If the permission level of the filtering pixel is greater than or equal to the permission levels of all the reference pixels, perform filtering processing on the filtering pixel according to the reference pixels.

16. The image decoding method according to claim 14, wherein The extracting the filtering pixels and the reference pixels corresponding to the filtering pixels from the reconstructed image block includes: Extract filtering pixels from the reconstructed image block and obtain target pixels according to the reference positions corresponding to the filtering pixels; If the target pixels are available, use the target pixels as reference pixels; If the target pixels are not available, use the pixels at the nearest positions to the target pixels in the filtering unit corresponding to the filtering pixels as reference pixels, and use the permission level of the pixels at the nearest positions to the target pixels in the filtering unit corresponding to the filtering pixels as the permission level of the reference pixels.

17. The image decoding method according to claim 1, wherein The current processing unit is an enhancement layer processing unit; Decoding the bitstream to be decoded to obtain the reconstructed image block corresponding to the current processing unit includes: Obtain the reference information corresponding to the current processing unit in the base layer, where the reference information includes mode reference information and pixel reference information; If the permission level of the bitstream to be decoded is greater than the permission level of the reference information, decode the bitstream to be decoded with reference to the reference information to obtain the reconstructed image block corresponding to the current processing unit.

18. The image decoding method according to claim 1, wherein Decoding the bitstream to be decoded to obtain the reconstructed image block corresponding to the current processing unit includes: Obtain multiple sets of quantization parameters set according to the user permission level; Update the multiple sets of quantization parameters; After the update is completed, decode the bitstream to be decoded to obtain the reconstructed image block corresponding to the current processing unit.

19. The image decoding method according to claim 18, wherein Each set of quantization parameters in the multiple sets of quantization parameters corresponds to a level of permission level, and the multiple sets of quantization parameters are initialized according to the quantization information of the surrounding processing units; Each set of quantization parameters in the multiple sets of quantization parameters satisfies the following conditions: If the permission level of the surrounding processing unit is less than or equal to the permission level corresponding to the quantization parameter, the current quantization parameter is equal to the quantization parameter of the surrounding processing unit; If the privilege level of the peripheral processing unit is greater than the privilege level corresponding to the quantization parameter, the quantization parameter is equal to the quantization parameter of the current slice level, the current coding tree unit, or the current frame level.

20. The image decoding method according to claim 18, wherein The updating of the multiple sets of quantization parameters includes: If the unit privilege level is not the lowest privilege level, and the user privilege level is greater than or equal to the unit privilege level, select the privacy quantization parameter from the multiple sets of quantization parameters according to the privilege level of the privacy bitstream, and select the non-privacy quantization parameter from the multiple sets of quantization parameters according to the privilege level of the non-privacy bitstream; Update the privacy quantization parameter according to the quantization information of the current processing unit extracted from the privacy bitstream, and update the non-privacy quantization parameter according to the quantization information of the current processing unit extracted from the non-privacy bitstream.

21. The image decoding method according to claim 18, wherein The updating of the multiple sets of quantization parameters includes: If the unit privilege level is the lowest privilege level, or the user privilege level is less than the unit privilege level, select the non-privacy quantization parameter from the multiple sets of quantization parameters according to the privilege level of the non-privacy bitstream; Update the non-privacy quantization parameter according to the quantization information of the current processing unit extracted from the non-privacy bitstream.

22. The image decoding method according to claim 1, wherein The decoding of the bitstream to be decoded to obtain the reconstructed image block corresponding to the current processing unit includes: If it is detected that the sample reference condition is satisfied, obtain the reference sample corresponding to the current processing unit; Perform intra prediction according to the reference sample to obtain a prediction result; Decode according to the prediction result and the bitstream to be decoded to obtain the reconstructed image block corresponding to the current processing unit.

23. The image decoding method according to claim 22, wherein Before the step of if it is detected that the sample reference condition is satisfied, obtain the reference sample corresponding to the current processing unit, further includes: If there is at least one reference sample at the reference position corresponding to the current processing unit that belongs to the same slice level as the current processing unit, has completed image reconstruction, and the corresponding privilege level is less than or equal to the privilege level of the bitstream to be decoded, it is determined that the sample reference condition is satisfied.

24. The image decoding method according to claim 22, wherein The performing intra prediction according to the reference sample to obtain a prediction result includes: Use the reference sample whose corresponding privilege level is greater than the privilege level of the bitstream to be decoded as the target sample; Perform pixel substitution processing on the target sample in the reference samples, and the processing method of the pixel substitution processing is determined according to the position of the target sample relative to the current processing unit; Perform intra prediction based on the reference samples after substitution processing to obtain a prediction result.

25. An image encoding method, characterized in that, The image coding method includes: When the privacy coding mode is in the on state, obtain a privacy determination result, where the privacy determination result is used to indicate whether the current processing unit is located in a privacy area; Encode the image corresponding to the current processing unit according to the privacy determination result to generate at least one bitstream corresponding to the current processing unit.

26. The image encoding method according to claim 25, characterized in that, The encoding the image corresponding to the current processing unit according to the privacy determination result to generate at least one bitstream corresponding to the current processing unit includes: If the privacy determination result is that the area is a privacy area, encode the image corresponding to the current processing unit to generate a privacy bitstream and a non-privacy bitstream corresponding to the current processing unit; If the privacy determination result is that the area is not a privacy area, encode the image corresponding to the current processing unit to generate a non-privacy bitstream corresponding to the current processing unit.

27. The image encoding method according to claim 26, wherein The privacy bitstream and the non-privacy bitstream are encoded using different entropy encoders; The image encoding method further includes: Obtain the permission level of the privacy area to obtain a privacy permission level; Set the permission level of the current processing unit and the privacy bitstream to the privacy permission level, and set the permission level of the non-privacy bitstream to the lowest permission level.

28. The image encoding method according to claim 25, wherein, The image encoding method further includes: When the privacy encoding mode is in the off state, obtain the privacy determination result; If the privacy determination result is that the area is a privacy area, encode the image corresponding to the current processing unit to generate a privacy bitstream corresponding to the current processing unit; If the privacy determination result is that the area is not a privacy area, encode the image corresponding to the current processing unit to generate a non-privacy bitstream corresponding to the current processing unit.

29. The image encoding method according to claim 28, wherein Before obtaining the privacy determination result when the privacy encoding mode is in the on state, further includes: Divide the image to be encoded according to the unit division information to obtain a plurality of image processing units; Traverse the plurality of image processing units to obtain the current processing unit; After encoding the image corresponding to the current processing unit according to the privacy determination result to generate at least one bitstream corresponding to the current processing unit, further includes: If the current processing unit is the last processing unit among the plurality of image processing units, construct an image bitstream according to at least one bitstream corresponding to each processing unit; Send the image bitstream to the decoding end.

30. The image encoding method according to claim 25, wherein Encoding the image corresponding to the current processing unit according to the privacy determination result to generate at least one bitstream corresponding to the current processing unit includes: If it is detected that the sample reference condition is satisfied, obtain the reference sample corresponding to the current processing unit; Perform intra prediction according to the reference sample to obtain a prediction result; Encode the image corresponding to the current processing unit based on the prediction result and the privacy determination result to generate at least one bitstream corresponding to the current processing unit.

31. An image decoding device, characterized in that, The image decoding device includes the following modules: An acquisition module, configured to obtain a unit permission level and a user permission level when the privacy encoding mode is in the on state, where the unit permission level is the permission level of the current processing unit, and the user permission level is the permission level of the user at the decoding end; An extraction module, configured to determine the bitstream to be decoded corresponding to the current processing unit according to the unit permission level and the user permission level; A decoding module, configured to decode the bitstream to be decoded to obtain a reconstructed image block corresponding to the current processing unit.

32. An image encoding device, characterized in that, The image encoding device includes the following modules: An acquisition module, configured to acquire a privacy determination result when the privacy encoding mode is in an enabled state, where the privacy determination result is used to indicate whether the current processing unit is located in a privacy area; An encoding module, configured to encode an image corresponding to the current processing unit according to the privacy determination result, to generate at least one bitstream corresponding to the current processing unit.

33. A decoding device, characterized in that, The decoding device includes: a processor, a memory, and an image decoding program stored on the memory and executable on the processor. When the image decoding program is executed by the processor, the steps of the image decoding method according to any one of claims 1-24 are implemented.

34. A coding device, characterized in that, The encoding device includes: a processor, a memory, and an image encoding program stored on the memory and executable on the processor. When the image encoding program is executed by the processor, the steps of the image encoding method according to any one of claims 25-30 are implemented.

35. An electronic device, characterized in that, The electronic device includes: a processor and a machine-readable storage medium, where a machine-executable instruction capable of being executed by the processor is stored on the machine-readable storage medium; the processor is configured to execute the machine-executable instruction to implement the method according to any one of claims 1-30.

36. A storage medium, characterized in that, An image decoding program and / or an image encoding program is stored on the storage medium. When the image decoding program is executed, the steps of the image decoding method according to any one of claims 1-24 are implemented. When the image encoding program is executed, the steps of the image encoding method according to any one of claims 25-30 are implemented.

37. A computer application program product, including a computer application program, characterized in that, When the computer application program is executed by the processor, the method according to any one of claims 1-30 is implemented.

Images