Image decoding method and device, decoder and computer readable storage medium

Abstract

The embodiment of the invention provides an image decoding method and device, a decoder and a computer readable storage medium, relates to the technical field of image coding and decoding, and ensures that decoding results of users with different permission levels at a decoding end to a low permission level region are consistent under the condition that transmission code streams are not increased. The method comprises the following steps: determining a prediction mode of a to-be-processed image block and an authority level of the to-be-processed image block; determining a first reference image block of the to-be-processed image block according to the prediction mode; and if the permission level of the first reference image block does not meet the reference condition, prohibiting using the first reference image block to determine the prediction block of the to-be-processed image block or using a second reference image block allowed to be exported to determine the prediction block of the to-be-processed image block.

Description

[0001] This application is a divisional application. The application number of the original application is 202211422461.9, and the application date of the original application is November 14, 2022. The entire content of the original application can be incorporated into this application by reference. Technical Field

[0002] The embodiments of the present application relate to the field of image coding and decoding technology, and in particular, to an image decoding method, device, decoder, and computer-readable storage medium. Background Art

[0003] The surveillance industry has undergone tremendous changes in recent years. Digitalization and high-definition have significantly improved image clarity. Surveillance equipment is also becoming increasingly popular, with widespread use in transportation, schools, and other public places. However, in order to enable different users to access different areas, it is particularly important to establish a robust video transmission solution with permission-level protection.

[0004] Currently, a common approach is to define regions of interest (ROIs) and assign them high permission levels to restrict access to them by users with lower permission levels. The encoding and decoding process for video transmission involves references between regions with different permission levels. When a low-privilege region requires reference to information from a high-privilege region for encoding and decoding, ensuring consistent decoding results for users with different permission levels on the decoding end is a key issue in permission-based encoding and decoding technology.

[0005] The industry has devised various solutions to ensure consistent decoding results for low-privilege-level areas for users with different permission levels on the decoding end. One approach involves limiting the configuration of permission-level areas, setting the same permission level for all image blocks within a slice to avoid references between areas with different permission levels. However, this solution results in inaccurate permission-level area configuration, making it impossible to accurately implement access to different areas based on permission levels. Summary of the Invention

[0006] The present application provides an image decoding method, device, decoder and computer-readable storage medium, which achieve accurate access to different image areas based on different permission levels without increasing the transmission code stream, and ensure that users with different permission levels on the decoding end have consistent decoding results for low-privilege-level areas.

[0007] To achieve the above objectives, the present invention adopts the following technical solutions:

[0008] In a first aspect, an embodiment of the present application provides an image decoding method, the method comprising: determining a prediction mode of an image block to be processed and an authority level of the image block to be processed; determining a first reference image block of the image block to be processed according to the prediction mode; if the authority level of the first reference image block does not meet a reference condition, prohibiting the use of the first reference image block to determine the prediction block of the image block to be processed or using a second reference image block that is allowed to be derived to determine the prediction block of the image block to be processed; wherein the reference condition includes the authority level being lower than or equal to the authority level of the image block to be processed, or the authority level being different from the authority level of the image block to be processed, or the authority level being equal to the lowest authority level, or there being no pixel blocks with a higher authority level than the image block to be processed within a distance of s pixels above, below, left, and right, and s being greater than or equal to 1.

[0009] Through the solution provided by the present application, during the encoding / decoding process of an image configured with permission levels, when an image block (or pixel) in a low-privilege level area needs to refer to an image block (or pixel) with a high-privilege level, the reference is prohibited or the reference is allowed to be made to an alternative value of the image block (or pixel) with a high-privilege level that is allowed to be exported (and can be obtained by users of any permission level). The encoding / decoding end uses the same operation for reference, so that for the same low-privilege level area, there is no need to transmit a compensation value, which saves transmission code stream, and the decoding results of users with different permissions on the decoding end are consistent.

[0010] As a possible implementation manner, the second reference image block is an image block determined by image blocks whose permission levels meet a reference condition among image blocks adjacent to the first reference image block.

[0011] As another possible implementation, the second reference image block is a predicted pixel value obtained by decoding the first reference image block using the lowest user authority level at the decoding end.

[0012] As another possible implementation, the second reference image block is an image block with a default pixel value.

[0013] In this way, if the second reference image block that is allowed to be exported is used to determine the prediction block of the image block to be processed (that is, the image block whose predicted pixel value is supported by the authority level of the user at the decoding end and has been determined), according to the different user authority levels of the decoding end where the processing device is deployed, for the same image block to be processed, when the authority level of the first reference image block does not meet the reference condition, the determined second reference image block is consistent, so as to achieve consistent decoding results for the same low-authority level area by users with different authority levels.

[0014] As a possible implementation, the method may further include: if the image block to be processed is of the lowest authority level, updating the historical reference list according to the predicted reference information of the image block to be processed; if the image block to be processed is not of the lowest authority level, prohibiting the updating of the historical reference list according to the predicted reference information of the image block to be processed. The historical reference list is used to indicate the motion information of the predicted reference information of the processed image block. The predicted reference information of the image block is used to indicate the prediction process of the image block to be processed. The non-lowest authority level is one or more. In this way, the historical reference list will only be updated when the image block to be processed is of the lowest authority level to ensure the consistency of decoding of the low-authority level area by users of different authority levels.

[0015] As another possible implementation, the method may further include: if the image block to be processed has the lowest permission level, updating the historical reference list based on the prediction reference information of the image block to be processed; if the permission level of the image block to be processed is not the lowest permission level, updating the historical reference list based on alternative prediction reference information that is allowed to be derived. In this way, if the permission level of the image block to be processed is not the lowest permission level, the historical reference list is updated using alternative prediction reference information (the alternative prediction reference information that is allowed to be derived may refer to the prediction reference information of the image block that is supported by the permission level of the decoding end user and for which the predicted pixel value has been determined) to ensure consistency in decoding of low-privilege-level regions by users of different permission levels.

[0016] As another possible implementation, the method may further include: if the image block to be processed has the lowest permission level, updating the historical reference list based on the predicted reference information of the image block to be processed; if the permission level of the image block to be processed is not the lowest permission level, updating the temporary historical reference list corresponding to the permission level of the image block to be processed based on the predicted reference information of the image block to be processed. In this way, if the permission level of the image block to be processed is not the lowest permission level, the temporary historical reference list (not the original historical reference list) corresponding to the permission level of the image block to be processed can be updated based on the predicted reference information of the image block to be processed, and after processing the image block with the lowest permission, the original historical reference list is updated based on the predicted reference information of the image block with the lowest permission, so as to ensure consistency in decoding of low-privilege-level areas by users of different permission levels.

[0017] As another possible implementation, the method may further include: if the image block to be processed is of the lowest permission level and the historical reference list has not been updated by the predicted reference information of an image block not of the lowest permission level, updating the historical reference list based on the predicted reference information of the image block to be processed; if the permission level of the image block to be processed is of the lowest permission level and the historical reference list has not been updated by the predicted reference information of an image block not of the lowest permission level, reconstructing the historical reference list based on the predicted reference information of the image block to be processed; if the image block to be processed is of a non-lowest permission level, updating the historical reference list based on the predicted reference information of the image block to be processed. In this way, by configuring only one historical reference list, the historical reference list is also updated for image blocks not of the lowest permission level, but when processing an image block of the lowest permission level, the historical reference list is initialized, reconstructed, and updated to ensure consistency in decoding of low-privilege-level regions by users of different permission levels.

[0018] As another possible implementation, the method may further include: updating a historical reference list of the permission level region to which the image block to be processed belongs according to the predicted reference information of the image block to be processed; different historical reference lists are constructed for regions with different permission levels.

[0019] As another possible implementation, different historical reference lists are constructed for some permission level regions. The method may further include: if a historical reference list is configured for the permission level region where the image block to be processed is located, updating the historical reference list for the permission level region to which the image block to be processed belongs based on the predicted reference information of the image block to be processed. If a historical reference list is not configured for the permission level region where the image block to be processed is located, constructing or updating the historical reference list according to other possible implementations.

[0020] As another possible implementation, the method may further include: if the permission level of the image block to be processed is higher than the permission level of the previous region, updating the historical reference list according to the predicted reference information of the image block to be processed; if the permission level of the image block to be processed is lower than the permission level of the previous region, initializing the historical reference list according to the alternative predicted reference information.

[0021] In this way, the codec and decoder can use the same method to update the historical reference list based on the different methods described above. The encoder determines the reference method used for the image block to be processed and encodes it in the bitstream and sends it to the decoder. This ensures that both the codec and decoder use the same method to ensure consistent decoding of low-privilege-level regions by users of different privilege levels.

[0022] As a possible implementation manner, the replacement prediction reference information is the default prediction reference information.

[0023] As another possible implementation manner, the replacement prediction reference information is prediction reference information of a default image block.

[0024] As another possible implementation, the replacement prediction reference information is prediction reference information of an image block whose permission level meets the reference condition and precedes the image block to be processed in the coding and decoding order.

[0025] As another possible implementation, the replacement prediction reference information is prediction reference information derived from prediction reference information of an image block whose permission level meets the reference condition and precedes the image block to be processed in the coding and decoding order.

[0026] In this way, the historical reference list is constructed and updated based on the prediction reference information of the lowest-privilege image block. When processing non-lowest-privilege image blocks, the historical reference list is updated based on the alternative prediction reference information that can be derived to ensure consistency in decoding of low-privilege area by users of different privilege levels.

[0027] As a possible implementation method, the prediction reference information includes any one of the following information: position information, pattern information or frequency; the historical reference list includes any one of the following lists: historical motion information table, historical intra-frame copy information table, or historical point prediction information table.

[0028] As a possible implementation manner, the method may further include: if the permission level of the used adjacent spatial domain image block does not meet the reference condition, marking the adjacent spatial domain image block as non-existent.

[0029] As another possible implementation, if the permission level of the adjacent image block does not meet the reference condition, the spatial domain prediction information of the image block to be processed is obtained by parsing the bitstream.

[0030] As another possible implementation, if the permission level of the used adjacent spatial image blocks does not meet the reference condition, the spatial prediction information of the image block to be processed is derived according to the alternative spatial prediction information of the adjacent image blocks that are allowed to be derived.

[0031] In this way, by deriving spatial motion information, the motion information of the surrounding image blocks adjacent to the image block to be processed is used to determine the motion information of the current image block to be processed. If the authority level of the adjacent spatial image blocks used does not meet the reference conditions, the motion information of the current image block to be processed can be determined according to the above-mentioned different methods of deriving spatial motion information, and the same scheme can be used at both ends of the codec. The reference method used for the image block to be processed is decided by the encoding end, and the encoding is sent to the decoding end along with the bitstream so that the same scheme is used at both ends of the codec to ensure consistency in decoding of low-authority-level areas by users of different authority levels.

[0032] As a possible implementation method, the airspace prediction information is replaced with the default airspace prediction information.

[0033] As another possible implementation, the spatial prediction information is replaced by spatial prediction information derived from spatial prediction information of an image block whose permission level meets a reference condition and precedes the image block to be processed in the encoding and decoding order.

[0034] As a possible implementation, the method may further include: obtaining permission level configuration information for an image frame, the permission level configuration information being used to indicate a correspondence between a location region in the image frame and a permission level; an image block in the location region having one or more levels of permission; and using the permission level corresponding to the location of the image block in the image frame in the permission level configuration information as the permission level of the image block. In this way, the permission level corresponding to the location of the image block in the image frame in the permission level configuration information can be used as the permission level of the image block, thereby enabling the processing device to restrict operations based on different permission levels.

[0035] As a possible implementation manner, when intra-frame prediction filtering, inter-frame prediction filtering, or intra-frame reference pixel filtering is turned on, the first reference image block is the reference image block used during filtering.

[0036] As another possible implementation method, when the cross-component prediction technology is turned on, when the image block to be processed is an image block in the first channel, the first channel refers to the image blocks in the second channel and / or the third channel to generate a prediction block of the image block in the first channel, and the first reference image block is the image block in the second channel and / or the third channel.

[0037] In this way, the first reference image block can be an image block to be referenced when obtaining the prediction block of the image block to be processed, determined according to the prediction module of the image block to be processed. Under different prediction modules or in different scenarios, the content or position of the first reference image block may be different.

[0038] As a possible implementation manner, the method may further include: if the permission level of the first reference image block meets the reference condition, using the first reference image block to determine a prediction block for the image block to be processed.

[0039] As a possible implementation, the method may further include: if the permission level of the first reference image block does not meet the reference condition, determining the target reference mode of the image block to be processed; wherein the target reference mode includes: prohibiting the use of the first reference image block to determine the prediction block of the image block to be processed or using the second reference image block that is allowed to be derived to determine the prediction block of the image block to be processed; the decoding end determines the target reference mode according to the indication information in the code stream, and the encoding end decides to determine the target reference mode. In this way, when the permission level of the first reference image block does not meet the reference condition, different image blocks can adopt different processing methods in the above-mentioned determination of the prediction block of the image block to be processed to obtain the prediction block of the image block to be processed, and the same scheme can be adopted by both the encoding and decoding ends. The reference mode adopted by the image block to be processed is decided by the encoding end and encoded and sent to the decoding end along with the code stream, so that both the encoding and decoding ends adopt the same scheme to ensure consistency in decoding of low-privilege-level areas by users of different permission levels.

[0040] In a second aspect, an embodiment of the present application provides an image decoding device, which includes a determination module and a processing module.

[0041] The determination module is used to determine the prediction mode of the image block to be processed and the permission level of the image block to be processed.

[0042] The determination module is further configured to determine, according to the prediction mode, a first reference image block of the image block to be processed.

[0043] The processing module is configured to: if the permission level of the first reference image block does not meet a reference condition, prohibit the use of the first reference image block to determine the prediction block of the image block to be processed, or use a second reference image block that is allowed to be derived to determine the prediction block of the image block to be processed. The reference condition includes that the permission level is lower than or equal to the permission level of the image block to be processed, or the permission level is different from the permission level of the image block to be processed, or the permission level is equal to the minimum permission level, or there is no pixel block with a higher permission level than the image block to be processed within a distance of s pixels above, below, to the left, or to the right, where s is greater than or equal to 1.

[0044] As a possible implementation, the second reference image block is determined by an image block whose permission level satisfies a reference condition among neighboring image blocks of the first reference image block. Alternatively, the second reference image block is a predicted pixel value obtained by decoding the first reference image block using the lowest user permission level at the decoding end. Alternatively, the second reference image block is an image block with default pixel values.

[0045] As a possible implementation, the processing module is further configured to: if the image block to be processed is at the lowest permission level, update the historical reference list based on the predicted reference information of the image block to be processed. If the image block to be processed is at a non-minimum permission level, prohibit updating the historical reference list based on the predicted reference information of the image block to be processed. The historical reference list is used to indicate motion information of the predicted reference information of the processed image block. The predicted reference information of the image block is used to indicate the prediction process of the image block to be processed. There can be one or more non-minimum permission levels.

[0046] As another possible implementation, the processing module is further configured to: if the image block to be processed has a minimum permission level, update the historical reference list based on the prediction reference information of the image block to be processed; and if the permission level of the image block to be processed is not the minimum permission level, update the historical reference list based on alternative prediction reference information that is permitted to be derived.

[0047] As another possible implementation, the processing module is further configured to: if the image block to be processed has the lowest permission level, update the historical reference list based on the predicted reference information of the image block to be processed; and if the permission level of the image block to be processed is not the lowest permission level, update the temporary historical reference list corresponding to the permission level of the image block to be processed based on the predicted reference information of the image block to be processed.

[0048] As another possible implementation, the processing module is further configured to: if the image block to be processed is of the lowest permission level and the historical reference list has not been updated by the predicted reference information of the image block not of the lowest permission level, update the historical reference list based on the predicted reference information of the image block to be processed. If the permission level of the image block to be processed is of the lowest permission level and the historical reference list has not been updated by the predicted reference information of the image block not of the lowest permission level, reconstruct the historical reference list based on the predicted reference information of the image block to be processed. If the image block to be processed is of a non-lowest permission level, update the historical reference list based on the predicted reference information of the image block to be processed.

[0049] As another possible implementation, the processing module is further configured to update the historical reference list of the permission level region to which the image block to be processed belongs based on the prediction reference information of the image block to be processed. Different historical reference lists are constructed for regions of different permission levels.

[0050] As another possible implementation, the processing module is further configured to: if the permission level of the image block to be processed is higher than the permission level of the previous region, update the historical reference list based on the predicted reference information of the image block to be processed; and if the permission level of the image block to be processed is lower than the permission level of the previous region, initialize the historical reference list based on the replacement predicted reference information.

[0051] As a possible implementation, the replacement prediction reference information is the default prediction reference information. Alternatively, the replacement prediction reference information is the prediction reference information of the default image block. Alternatively, the replacement prediction reference information is the prediction reference information of an image block in the coding and decoding order that precedes the image block to be processed and whose permission level meets the reference conditions. Alternatively, the replacement prediction reference information is the prediction reference information derived from the prediction reference information of an image block in the coding and decoding order that precedes the image block to be processed and whose permission level meets the reference conditions.

[0052] As a possible implementation, the prediction reference information includes any one of the following information: position information, mode information, or frequency. The history reference list includes any one of the following lists: a history motion information table, a history intra copy information table, or a history point prediction information table.

[0053] As a possible implementation, the processing module is further configured to: if the permission level of the adjacent spatial image block in use does not meet the reference condition, mark the adjacent spatial image block as non-existent. Alternatively, the processing module is further configured to: if the permission level of the adjacent image block does not meet the reference condition, parse the codestream to obtain spatial prediction information for the image block to be processed. Alternatively, the processing module is further configured to: if the permission level of the adjacent spatial image block in use does not meet the reference condition, derive spatial prediction information for the image block to be processed according to alternative spatial prediction information of the adjacent image block that is allowed to be derived.

[0054] As a possible implementation, the replacement spatial prediction information is the default spatial prediction information. Alternatively, the replacement spatial prediction information is the spatial prediction information derived from the spatial prediction information of an image block whose permission level meets the reference condition and precedes the image block to be processed in the encoding and decoding order.

[0055] As one possible implementation, the determination module is specifically configured to obtain permission level configuration information for an image frame, where the permission level configuration information indicates a correspondence between a location region in the image frame and a permission level. An image block in the location region has one or more levels of permission. The permission level corresponding to the location of the image block in the image frame in the permission level configuration information is used as the permission level of the image block.

[0056] As a possible implementation, when intra-frame prediction filtering, inter-frame prediction filtering, or intra-frame reference pixel filtering is enabled, the first reference image block is the reference image block used during filtering. When cross-component prediction technology is enabled, when the image block to be processed is an image block in the first channel, the first channel refers to image blocks in the second channel and / or the third channel to generate a prediction block for the image block in the first channel, and the first reference image block is an image block in the second channel and / or the third channel.

[0057] As a possible implementation manner, the processing module is further configured to: if the permission level of the first reference image block meets the reference condition, use the first reference image block to determine a prediction block for the image block to be processed.

[0058] As a possible implementation, the processing module is further configured to: if the permission level of the first reference image block does not meet the reference condition, determine a target reference mode for the image block to be processed. The target reference mode includes: prohibiting the use of the first reference image block to determine the prediction block for the image block to be processed, or using a second reference image block that is permitted to be derived to determine the prediction block for the image block to be processed. The decoding end determines the target reference mode based on the indication information in the bitstream, and the encoding end makes the decision to determine the target reference mode.

[0059] In a third aspect, embodiments of the present application provide an encoder comprising a processor coupled to a memory. The memory is configured to store a computer program or instructions. The processor is configured to execute the computer program or instructions stored in the memory, so that the encoder performs the method described in any one of the first aspect and its possible implementations.

[0060] In a fourth aspect, embodiments of the present application provide a decoder comprising a processor coupled to a memory. The memory is configured to store a computer program or instructions. The processor is configured to execute the computer program or instructions stored in the memory, so that the decoder performs the method described in any one of the first aspect and its possible implementations.

[0061] In a fifth aspect, the present application provides a computer program product, comprising program code, which, when executed on a computer or processor, is used to execute the method described in the first aspect and any one of its possible implementations.

[0062] In a sixth aspect, the present application provides an electronic device comprising the encoder described in the third aspect, or the decoder described in the fourth aspect.

[0063] In a seventh aspect, the present application provides a computer-readable storage medium comprising program code, which, when executed by a computer device, is used to execute the method described in the first aspect and any one of its possible implementations.

[0064] It should be understood that the beneficial effects achieved by the technical solutions of the second to seventh aspects of the embodiments of the present application and the corresponding possible implementation methods can be referred to the technical effects of the first aspect and its corresponding possible implementation methods mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 A schematic diagram of the architecture of a coding and decoding system provided in an embodiment of the present application;

[0066] Figure 2A schematic block diagram of an encoder provided in an embodiment of the present application;

[0067] Figure 3 A schematic block diagram of a decoder provided in an embodiment of the present application;

[0068] Figure 4 A schematic diagram of a flow chart of an image encoding and decoding method provided in an embodiment of the present application;

[0069] Figure 5 A flowchart of another image encoding and decoding method provided in an embodiment of the present application;

[0070] Figure 6 A schematic diagram of an intra-frame prediction mode provided in an embodiment of the present application;

[0071] Figure 7 A schematic diagram of the positional relationship between a left / upper adjacent block and a current image block provided in an embodiment of the present application;

[0072] Figure 8 A schematic diagram of a peripheral block provided in an embodiment of the present application;

[0073] Figure 9 A schematic diagram of the structure of an image encoding and decoding device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0074] The term "and / or" in this article is merely a description of the association relationship between associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0075] In the description and claims of the embodiments of this application, the terms "first" and "second" are used to distinguish different objects, rather than to describe a specific order of objects. For example, the terms "first prediction mode" and "second prediction mode" are used to distinguish different prediction modes, rather than to describe a specific order of prediction modes.

[0076] In the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0077] In the description of the embodiments of the present application, unless otherwise specified, “a plurality of” means two or more than two. For example, a plurality of image blocks refers to two or more than two image blocks.

[0078] The image encoding and decoding method provided in the embodiment of the present application can be applied to the process of obtaining a prediction block during image encoding, and can also be applied to the process of obtaining a prediction block during image decoding.

[0079] First, the technical terms involved in the embodiments of this application are introduced:

[0080] Video sequence / image sequence: A complete image in a video is usually called a "frame", and a video composed of many frames in time sequence is called a video sequence (video sequence), or it can be called an image sequence.

[0081] Video coding technology: Video sequences contain a range of redundant information, including spatial redundancy, temporal redundancy, visual redundancy, information entropy redundancy, structural redundancy, knowledge redundancy, and importance redundancy. To minimize this redundant information and reduce the amount of data representing the video, video coding technology has been developed to reduce storage space and transmission bandwidth. Video coding technology is also known as video compression technology.

[0082] Permission level: This is information assigned to regions within an image to indicate the level of access rights. Users on the decoding end can only access information in regions with a permission level equal to or lower than their own.

[0083] Intra-frame prediction: refers to predictive coding using the reconstructed pixel values ​​of the spatially adjacent blocks of the current block (in the same frame image as the current image block).

[0084] Inter-frame prediction: refers to predictive coding using the reconstructed pixel values ​​of the time-domain neighboring blocks of the current block (in different images from the current image block).

[0085] Predicted pixel: refers to the pixel value derived from the pixel that has been encoded and decoded, that is, the pixel value in the predicted block of the image block. The difference between the original pixel and the predicted pixel can be used to obtain the residual, and then the residual transform quantization and coefficient encoding are performed. In particular, the predicted pixel of inter-frame prediction refers to the pixel value derived from the reference image block in the reference frame (reconstructed pixel frame) of the current image block. Since the pixel position is discrete, interpolation operation is required to obtain the final predicted pixel. The closer the predicted pixel is to the original pixel, the smaller the residual energy obtained by subtracting the two, and the higher the coding compression performance.

[0086] The present application provides an image encoding and decoding solution. During the encoding / decoding process of an image configured with permission levels, when an image block (or pixel) in a low-privilege level area needs to refer to an image block (or pixel) at a high-privilege level, the reference is prohibited or the reference is allowed to be made to an alternative value of an image block (or pixel) at a high-privilege level that is allowed to be exported (and can be obtained by users of any permission level). The encoding / decoding end uses the same operation for reference, so that the decoding results of users with different permissions at the decoding end can be consistent for the same low-privilege level area. Since the solution of the present application provides a solution for reference between areas with different permission levels, there is no need to force certain image blocks to be assigned to other permission levels when dividing image blocks, and accurate access to different image areas by permission level can be achieved. The present application uses alternative values ​​that are allowed to be exported for reference, which achieves consistency in decoding for users with different permission levels, eliminates the need to pass compensation values, and saves transmission code streams.

[0087] Next, the implementation environment and application scenarios of the embodiment of the present application are briefly introduced. The video encoding and decoding method provided by the present application can be applied to Figure 1 The video coding and decoding system shown.

[0088] See also Figure 1 , provides a schematic diagram of the architecture of the encoding and decoding system (also referred to as the encoding and decoding system) 10 used in the embodiment of the present application. Figure 1 As shown, the codec system 10 may include a source device 11 and a destination device 12. The source device 11 is used to encode images, and therefore, the source device 11 can be called a video encoding device. The destination device 12 is used to decode the encoded image data generated by the source device 11, and therefore, the destination device 12 can be called a video decoding device.

[0089] The source device 11 and the destination device 12 may include various devices, including, for example, desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, mobile phones, televisions, cameras, display devices, digital media players, video game consoles, in-vehicle computers, wireless communication devices, and the like.

[0090] In one example, Figure 1 The source device 11 and the destination device 12 may be two separate devices, or the source device 11 and the destination device 12 may be the same device, that is, the source device 11 or the corresponding function and the destination device 12 or the corresponding function may be integrated into the same device.

[0091] The source device 11 and the destination device 12 may communicate with each other. For example, the destination device 12 may receive encoded image data from the source device 11. In one example, one or more communication media may be provided between the source device 11 and the destination device 12, and the encoded image data may be transmitted via the one or more communication media. The one or more communication media may include routers, switches, base stations, or other devices that facilitate communication from the source device 11 to the destination device 12.

[0092] like Figure 1 As shown, source device 11 includes an encoder 112. In one example, source device 11 may also include an image preprocessor 111 and a communication interface 113. Image preprocessor 111 is configured to preprocess received images to be encoded. For example, the preprocessing performed by image preprocessor 111 may include retouch, color format conversion (e.g., from RGB to YUV), color grading, or denoising. Encoder 112 is configured to receive the preprocessed images and process them using a relevant prediction mode (such as the prediction mode described in various embodiments herein) to provide encoded image data. In some embodiments, encoder 112 may be configured to perform the image encoding process described in various embodiments below. Communication interface 113 may be configured to transmit the encoded image data to destination device 12 or any other device (e.g., a memory) for storage or direct reconstruction. The other device may be any device for decoding or storage. Communication interface 113 may also encapsulate the encoded image data into a suitable format before transmission.

[0093] In an example, the image preprocessor 111 , the encoder 112 , and the communication interface 113 may be hardware components in the source device 11 , or may be software programs in the source device 11 , which is not limited in the embodiment of the present application.

[0094] Continue as Figure 1 As shown, the destination device 12 includes a decoder 122. In one example, the destination device 12 may also include a communication interface 121 and an image post-processor 123. The communication interface 121 may be used to receive encoded image data from the source device 11 or any other source device, such as a storage device. The communication interface 121 may also decapsulate data transmitted by the communication interface 113 to obtain encoded image data. The decoder 122 is configured to receive the encoded image data and output decoded image data (also referred to as reconstructed image data or reconstructed image data). In some embodiments, the decoder 122 may be configured to perform the image decoding process described in the various embodiments described below.

[0095] The image post-processor 123 is configured to perform post-processing on the decoded image data to obtain post-processed image data. The post-processing performed by the image post-processor 123 may include color format conversion (e.g., from YUV format to RGB format), color grading, retouch or resampling, or any other processing. The image post-processor 123 may also be configured to transmit the post-processed image data to a display device for display.

[0096] Similarly, in one example, the communication interface 121 , decoder 122 , and image post-processor 123 may be hardware components in the destination device 12 , or may be software programs in the destination device 12 , which is not limited in the embodiment of the present application.

[0097] Below Figure 1 The structure of the encoder and decoder in .

[0098] See also Figure 2 , Figure 2 1 is a schematic block diagram showing an example of an encoder 20 for implementing an embodiment of the present application. Figure 2 In the embodiment, the encoder includes a prediction processing unit 201, a residual calculation unit 202, a transform processing unit 203, a quantization unit 204, an entropy coding unit 205, an inverse quantization unit (also called an inverse quantization unit) 206, an inverse transform unit (also called an inverse transform processing unit) 207, a reconstruction unit (or called a reconstruction unit) 208, and a filter unit 209. Optionally, the encoder 20 may further include a buffer and a decoded image buffer, wherein the buffer is used to cache the reconstructed image blocks output by the reconstruction unit 208, and the decoded image buffer is used to cache the filtered image blocks output by the filter unit 209.

[0099] The input of the encoder 20 is an image block of an image (which may be referred to as an image to be encoded), and the image block may also be referred to as a current image block, an image block to be encoded, or an image block to be processed.

[0100] The encoder 20 may further include a segmentation unit ( Figure 2 (not shown), the segmentation unit is configured to segment the image to be encoded into a plurality of image blocks. The encoder 20 is configured to encode the image block by block, for example, performing an encoding process on each image block. Exemplarily, the segmentation unit may segment the image based on location information indicating the location of the object of interest, such that each image block includes only image regions of one permission level.

[0101] The prediction processing unit 201 is configured to receive or obtain an image block (the current image block to be encoded of the current image to be encoded, also referred to as the current image block, which can be understood as the true value of the image block) and reconstructed image data, and predict the current image block based on relevant data in the reconstructed image data to obtain a prediction block for the current image block. Optionally, the prediction processing unit 201 may include an inter-frame prediction unit, an intra-frame prediction unit, and a mode selection unit. The mode selection unit is configured to select an intra-frame prediction mode or an inter-frame prediction mode. If the intra-frame prediction mode is selected, the intra-frame prediction unit performs the prediction process. If the inter-frame prediction mode is selected, the inter-frame prediction unit performs the prediction process.

[0102] The residual calculation unit 202 is used to calculate the residual between the real value of the image block and the predicted block of the image block to obtain a residual block, for example, by subtracting the pixel value of the predicted block from the pixel value of the image block pixel by pixel.

[0103] The transform processing unit 203 is used to perform a transform on the residual block, such as a discrete cosine transform (DCT) or a discrete sine transform (DST), to obtain a transform coefficient in the transform domain. The transform coefficient may also be referred to as a transform residual coefficient, which may represent the residual block in the transform domain.

[0104] The quantization unit 204 is configured to quantize the transform coefficients by applying scalar quantization or vector quantization to obtain quantized transform coefficients, which may also be referred to as quantized residual coefficients. The quantization process may reduce the bit depth associated with some or all of the transform coefficients. For example, an n-bit transform coefficient may be rounded down to an m-bit transform coefficient during quantization, where n is greater than m. The degree of quantization may be modified by adjusting a quantization parameter (QP). For example, for scalar quantization, different scales may be applied to achieve finer or coarser quantization. A smaller quantization step size corresponds to finer quantization, while a larger quantization step size corresponds to coarser quantization. The appropriate quantization step size may be indicated by a quantization parameter (QP).

[0105] The entropy coding unit 205 is used to encode the quantized residual coefficients or transform coefficients, and then arrange the coefficients into units to be decoded in a scanning manner and a grouping manner, and output the encoded image data (i.e., the units to be decoded) in the form of a coded bit stream. The coded bit stream can then be transmitted to a decoder, or stored and subsequently transmitted to a decoder or used for retrieval. The entropy coding unit 205 can also be used to encode other syntax elements of the current image block, such as encoding the prediction mode into the bit stream, encoding the reference mode indication information into the bit stream, etc. Entropy coding algorithms include but are not limited to variable length coding (VLC) algorithm, context adaptive VLC (CAVLC) algorithm, arithmetic coding algorithm, context adaptive binary arithmetic coding (CABAC) algorithm, syntax-based context-adaptive binary arithmetic coding (SBAC) algorithm, and probability interval partitioning entropy (PIPE) algorithm.

[0106] The inverse quantization unit 206 is configured to inversely quantize the quantized coefficients to obtain inversely quantized coefficients. The inverse quantization is a reverse application of the quantization unit 204, for example, applying an inverse quantization scheme of the quantization scheme applied by the quantization unit 204 based on or using the same quantization step size as the quantization unit 204. The inversely quantized coefficients may also be referred to as inversely quantized residual coefficients.

[0107] The inverse transform unit 207 is used to perform an inverse transform on the inverse quantized coefficients. It should be understood that the inverse transform is the reverse application of the transform processing unit 203. For example, the inverse transform may include an inverse discrete cosine transform (DCT) or an inverse discrete sine transform (DST) to obtain an inverse transform block in the pixel domain (or sample domain). The inverse transform block may also be called an inverse transform dequantized block or an inverse transform residual block.

[0108] The reconstruction unit 208 is used to add the inverse transformed block (i.e., the inverse transformed residual block) to the prediction block to obtain a reconstructed block in the sample domain. The reconstruction unit 208 can be a summer, for example, adding the sample values ​​(i.e., pixel values) of the residual block to the sample values ​​of the prediction block. The reconstructed block output by the reconstruction unit 208 can be subsequently used to predict other image blocks, for example, in intra-frame prediction mode.

[0109] The filter unit 209 (or simply "filter") is used to filter the reconstructed block to obtain a filtered block, thereby smoothly performing pixel conversion or improving image quality. The filter unit can be a loop filter unit, which is intended to represent one or more loop filters, such as a deblocking filter, a sample-adaptive offset (SAO) filter, or other filters, such as a bilateral filter, an adaptive loop filter (ALF), or a sharpening or smoothing filter, or a collaborative filter. Optionally, the filtered block output by the filter unit 209 can be subsequently used to predict other image blocks, for example, in an inter-frame prediction mode.

[0110] Specifically, in the embodiment of the present application, the encoder 20 is used to implement the image encoding and decoding method described in the embodiments below.

[0111] See also Figure 3 , Figure 3 A schematic block diagram of an example of a decoder 30 for implementing an embodiment of the present application is shown. The decoder 30 is configured to receive encoded image data (i.e., an encoded bitstream, e.g., an encoded bitstream including image blocks and associated syntax elements) encoded by the encoder 20 to obtain a decoded image. The decoder 30 includes an entropy decoding unit 301, an inverse quantization unit 302, an inverse transform unit 303, a prediction processing unit 304, a reconstruction unit 305, and a filter unit 306. In some examples, the decoder 30 may perform operations substantially similar to those described in the preceding text. Figure 2 The decoding pass is the inverse of the encoding pass described by the encoder 20. In one example, the decoder 30 may further include a buffer and a decoded image buffer, wherein the buffer is used to cache the reconstructed image blocks output by the reconstruction unit 305, and the decoded image buffer is used to cache the filtered image blocks output by the filter unit 306.

[0112] The entropy decoding unit 301 is configured to perform entropy decoding on the encoded image data to obtain quantized coefficients and / or decoded coding parameters (e.g., the decoded parameters may include any one or all of inter-frame prediction parameters, intra-frame prediction parameters, filter parameters, and / or other syntax elements). The entropy decoding unit 301 is further configured to forward the decoded coding parameters to the prediction processing unit 304, so that the prediction processing unit performs a prediction process based on the coding parameters to obtain a prediction block for the current image block.

[0113] The function of the inverse quantization unit 302 may be the same as that of the inverse quantization unit 206 of the encoder 20 , for inverse quantizing (ie, inverse quantizing) the quantized coefficients decoded by the entropy decoding unit 301 .

[0114] The function of the inverse transform unit 303 may be the same as that of the inverse transform unit 207 of the encoder 20, and the function of the reconstruction unit 305 (e.g., a summer) may be the same as that of the reconstruction unit 208 of the encoder 20, and is configured to perform an inverse transform (e.g., an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process) on the above-mentioned inverse quantized coefficients to obtain an inverse transform block (also referred to as an inverse transform residual block), which is the residual block of the current image block in the pixel domain.

[0115] The prediction processing unit 304 is configured to receive or obtain coded image data (e.g., a coded bitstream of the current image block) and reconstructed image data. The prediction processing unit 301 may also receive or obtain prediction-related parameters and / or information about the selected prediction mode (i.e., decoded coding parameters) from, for example, the entropy decoding unit 302, and predict the current image block based on the relevant data in the reconstructed image data and the decoded coding parameters to obtain a prediction block for the current image block. Optionally, the prediction processing unit 304 may include an inter-frame prediction unit, an intra-frame prediction unit, and a mode selection unit. The mode selection unit is configured to select either an intra-frame prediction mode or an inter-frame prediction mode. If the intra-frame prediction mode is selected, the intra-frame prediction unit performs the prediction process. If the inter-frame prediction mode is selected, the inter-frame prediction unit performs the prediction process.

[0116] The reconstruction unit 305 is configured to add the inverse transform block (ie, the inverse transform residual block) to the prediction block to obtain a reconstructed block in the sample domain, for example, by adding sample values ​​of the inverse transform residual block to sample values ​​of the prediction block.

[0117] The filter unit 306 is configured to filter the reconstructed block to obtain a filtered block, which is the decoded image block.

[0118] Specifically, in the embodiment of the present application, the prediction processing unit 201 / the prediction processing unit 304 is used to implement the image encoding and decoding method described in the embodiments below.

[0119] It should be understood that, in the encoder 20 and decoder 30 of the embodiment of the present application, the processing result of a certain link can also be output to the next link after further processing.

[0120] In one aspect, embodiments of the present application provide an image encoding and decoding method, performed by a processing device. The processing device may be the aforementioned encoder 20, to perform the encoding process. Alternatively, the processing device may be the aforementioned decoder 30, to perform the decoding process. Furthermore, the processing device may be the aforementioned prediction processing unit 201 or prediction processing unit 304, to perform the prediction process.

[0121] Among them, the processing object in the embodiment of the present application is an image block. The image block segmentation method and the image block size can be implemented with reference to the video compression coding standard, and the embodiment of the present application is not limited to this.

[0122] As mentioned above, to ensure accurate access to different image regions based on permission levels, users on the decoding end can only access image regions with permission levels equal to or lower than their own. When performing block segmentation, the encoding end selects an appropriate block segmentation scheme so that each segmented image block falls within a region of only one permission level. If the currently segmented image block contains regions of multiple permission levels, the current image block is segmented again until the segmented image block falls within a region of only one permission level.

[0123] Among them, an image area of ​​a permission level can be a coding unit CU, a maximum coding unit LCU, an image slice (such as slice, tile, patch) or a frame level, etc., which is not limited in the embodiment of the present application. An image block can be an LCU, CU or prediction unit PU or others.

[0124] The processing device processes each image block in the image frame in a specific order. In the following embodiments, the image block currently being processed is referred to as the pending image block. The processing device performs the same processing on each pending image block. The following embodiments only describe the processing of one pending image block, and the subsequent processing is not repeated.

[0125] It should be noted that when the processing device is deployed at the decoding end, different users have different permission levels. The processing device can correctly decode image blocks with permission levels lower than or equal to the user's permission and obtain their true pixel values. For image blocks with permission levels higher than the user's permission, the processing device may not decode them, or may decode them to obtain incorrect pixel value records.

[0126] For example, when a user with a low authority level has an image block to be processed that is currently an image block with a high authority level, decoding can be skipped directly, or the erroneous pixel value of the image block to be processed can be obtained through decoding as the predicted pixel value of the image block with a high authority level. Of course, the predicted pixel value is not the real pixel value.

[0127] like Figure 4 As shown, the image encoding and decoding method may include:

[0128] S401: The processing device determines a prediction mode of an image block to be processed and an authority level of the image block to be processed.

[0129] The prediction mode of the image block to be processed includes inter-frame prediction and / or intra-frame prediction. The specific contents of inter-frame prediction and intra-frame prediction can be referred to the video compression coding standard, which will not be described in detail in the embodiment of this application.

[0130] Exemplarily, the processing device is deployed at the encoding end (ie, the aforementioned source device 11 or encoder 112), and can determine the prediction mode of the image block to be processed by decision-making. The embodiment of the present application does not limit the content of the decision.

[0131] Exemplarily, the processing device is deployed at the decoding end (i.e., the aforementioned destination device 12 or decoder 122), and can obtain indication information indicating the prediction mode of the image block to be processed from the syntax elements decoded in the code stream, and then obtain the prediction mode of the image block to be processed.

[0132] Specifically, the permission level of the image block depends on the location area of ​​the image block in the image frame, and the location area in the image frame is configured with different permission levels according to needs. The method provided in the embodiment of the present application can also include a process of determining the permission levels of different areas in the image frame.

[0133] Exemplarily, the process of determining the permission levels of different areas in an image frame may include: obtaining permission level configuration information of the image frame, where the permission level configuration information is used to indicate a correspondence between position areas and permission levels in the image frame.

[0134] The permission level configuration information can be configured and input by the user. The encoder / decoder can simply obtain the permission level configuration information input by the user. Alternatively, the encoder can obtain the permission level configuration information input by the user, encode it, and send it to the decoder, which then parses the bitstream to obtain the permission level configuration information. Alternatively, the encoder can obtain the permission level configuration information input by the user, then determine the permission level for each image block, encode the permission level of the image block, and send it to the decoder, which then parses the bitstream to obtain the permission level of the image block.

[0135] For example, the permission level configuration information may be as shown in Table 1.

[0136] Table 1

[0137]

[0138]

[0139] Furthermore, after determining the permission levels of different regions in the image frame, the processing device may use the permission level corresponding to the location of the image block in the image frame in the permission level configuration information as the permission level of the image block. Specifically, the processing device may determine the location of the image block based on the coordinates of the image block, and then determine the permission level of the image block.

[0140] For example, in S401, the processing device determines the permission level of the image block to be processed, and may use the permission level corresponding to the position of the image block to be processed in the image frame in the permission level configuration information as the permission level of the image block to be processed.

[0141] For example, the processing device is deployed at the decoding end. In S401 , the processing device determines the permission level of the image block to be processed and can obtain the permission level of the image block to be processed by parsing the bitstream.

[0142] S402: The processing device determines a first reference image block for the image block to be processed according to a prediction mode of the image block to be processed.

[0143] The processing device may determine a reference image block for the image block to be processed based on the provisions of the video compression coding standard, the prediction module of the image block to be processed, and the position of the image block to be processed. This embodiment of the present application refers to this as the first reference image block. This embodiment of the present application does not limit the process of determining the first reference image block for the image block to be processed.

[0144] In a possible implementation, during intra-frame prediction, the first reference image block is an image block in an image frame where the image block to be processed is located.

[0145] In another possible implementation, during inter-frame prediction, the first reference image block is an image block in a reference image frame of the image block to be processed.

[0146] For example, during inter-frame prediction, the video compression coding standard specifies a search area for image blocks, which includes a reference frame of the image block and image blocks allowed for reference in the reference frame. The processing device in S402 can select the optimal reference image block, i.e., the first reference image block, within the search area based on the prediction module of the image block to be processed.

[0147] In another possible implementation, when intra-frame prediction filtering, inter-frame prediction filtering, or intra-frame reference pixel filtering is enabled, the first reference image block may be a reference image block used during filtering.

[0148] In another possible implementation, when the cross-component prediction technology is turned on, when the image block to be processed is an image block in the first channel, the first channel refers to the image blocks in the second channel and / or the third channel to generate a prediction block of the image block in the first channel, and the first reference image block is the image block in the second channel and / or the third channel.

[0149] It should be noted that the first reference image block is an image block to be referenced when obtaining the prediction block of the image block to be processed, determined according to the prediction module of the image block to be processed. Under different prediction modules or in different scenarios, the content or position of the first reference image block may be different. The embodiment of the present application does not specifically limit the content or position of the first reference image block, nor does it specifically limit the process of determining the first reference image block.

[0150] Exemplarily, the first reference image block is an image block for which a prediction block has been determined and which precedes the image block to be processed in the coding and decoding order.

[0151] S403: If the permission level of the first reference image block does not meet the reference condition, prohibit using the first reference image block to determine the prediction block of the image block to be processed or prohibit using the second reference image block that is allowed to be derived to determine the prediction block of the image block to be processed.

[0152] Illustratively, the reference conditions may include: a permission level lower than or equal to the permission level of the image block to be processed; or a permission level different from the permission level of the image block to be processed; or a permission level equal to the minimum permission level; or, there are no pixel blocks with a higher permission level than the image block to be processed within a distance of s pixels above, below, to the left, or to the right, where s is greater than or equal to 1. The embodiments of the present application do not limit the content of the reference conditions.

[0153] For example, if the current image block is n×m, then there is no pixel block with a higher authority level than the image block to be processed within (n+s)×(m+s), and the reference condition is met.

[0154] The following describes in detail two solutions: prohibiting the use of the first reference image block to determine the prediction block of the image block to be processed; and using the second reference image block to determine the prediction block of the image block to be processed.

[0155] The first solution is to prohibit the use of the first reference image block to determine the prediction block of the image block to be processed.

[0156] Specifically, the first solution is equivalent to limiting the reference to image blocks that do not meet the reference conditions. In the first solution, the processing method for non-existent reference image blocks specified in the video compression coding standard can be used to process the image blocks to be processed, which will not be repeated here.

[0157] Furthermore, in the first solution, the first reference image block may be marked as unavailable or not allowed to be referenced.

[0158] The second solution is to use a second reference image block that can be derived to determine a prediction block for the image block to be processed.

[0159] The second reference image block allowed to be derived is an image block whose predicted pixel value has been determined and whose permission level supports it. For decoding end users with different permission levels, the second reference image block corresponding to the same first reference image block is the same.

[0160] Exemplarily, the second reference image block may include but is not limited to any of the following:

[0161] Case 1: The second reference image block is an image block determined by an image block whose permission level meets a reference condition among adjacent image blocks of the first reference image block.

[0162] In a possible implementation, the second reference image block may be an image block whose permission level satisfies a reference condition among image blocks adjacent to the first reference image block.

[0163] Among them, the adjacent image blocks can be surrounding image blocks, and their specific content can be configured according to actual needs, which is not limited in this embodiment of the present application.

[0164] Exemplarily, the adjacent image block may be a left image block, an upper image block, an upper-left image block or others.

[0165] In another possible implementation, the second reference image block may be an image block obtained by calculation of an image block whose permission level satisfies a reference condition among image blocks adjacent to the first reference image block.

[0166] The calculation may be to superimpose default pixel values ​​or other calculations, and the embodiment of the present application does not limit the content of the calculation.

[0167] Case 2: The second reference image block is a predicted pixel value obtained by decoding the first reference image block using the lowest user authority level at the decoding end.

[0168] In case 2, the predicted pixel value obtained by decoding the first reference image block using the lowest user authority level at the decoding end is not the true pixel value of the first reference image block, that is, the aforementioned decoding error value.

[0169] Case 3: The second reference image block is an image block with a default pixel value.

[0170] Among them, the default value can be determined according to actual needs, and the embodiments of the present application are not limited to this.

[0171] It should be noted that for the same image block to be processed, when the authority level of the first reference image block does not meet the reference condition, the second reference image block determined by the decoding end of the deployed processing device with different user authority levels is consistent, so as to achieve consistent decoding results for the same low-authority level area for users with different authority levels.

[0172] Furthermore, exemplarily, when the processing device is deployed at the encoding end, a second reference image block that is allowed to be derived can be used to determine a temporary prediction block of the image block to be processed; if the residual between the temporary prediction block and the image block to be processed is less than a threshold, the temporary prediction block is used as the prediction block of the image to be processed; if the residual between the temporary prediction block and the image block to be processed is greater than the threshold, it is determined that the reference image block of the image block to be processed is unavailable, and the above-mentioned first scheme is used to determine the prediction block of the image block to be processed.

[0173] Specifically, in the second scheme, the processing device determines the prediction block of the image block to be processed, and the processing device predicts the current image block to be processed based on the second reference image block (the relevant data in the reconstructed image data) to obtain the prediction block of the current image block to be processed. This application does not elaborate on the specific process.

[0174] The solution provided by this application provides a reference solution between regions of different permission levels. Therefore, when dividing image blocks, certain image blocks are not forced into different permission levels, allowing accurate access to different image regions by permission level. This application uses a permitted derivative substitute value for reference, achieving consistent decoding of low-privilege-level regions by users of different permission levels, eliminating the need to pass compensation values ​​and saving transmission bitstream.

[0175] Further, such as Figure 5 As shown, after S402, the method provided in the embodiment of the present application may further include S404.

[0176] S404: If the permission level of the first reference image block meets the reference condition, use the first reference image block to determine a prediction block for the image block to be processed.

[0177] Specifically, in S404, the processing device determines the prediction block of the image block to be processed, and the processing device predicts the current image block to be processed based on the first reference image block (the relevant data in the reconstructed image data) to obtain the prediction block of the current image block to be processed. This application does not elaborate on the specific process.

[0178] In a possible implementation, any one of the solutions in S403 may be selected according to actual needs to obtain the prediction block of the image block to be processed, and both ends of the encoding and decoding may adopt the same solution.

[0179] In another possible implementation, when the permission level of the first reference image block does not meet the reference condition, different image blocks can adopt different processing methods in S403 to obtain the prediction block of the image block to be processed, and the codec can use the same method. The reference method used by the image block to be processed is decided by the encoder and encoded and sent to the decoder along with the bitstream, so that the encoder and decoder can use the same method. Figure 5As shown, the method provided in the embodiment of the present application may further include S405.

[0180] S405: If the permission level of the first reference image block does not meet the reference condition, determine a target reference mode for the image block to be processed.

[0181] The target reference mode includes: prohibiting the use of the first reference image block to determine the prediction block of the image block to be processed, or using the second reference image block that is allowed to be derived to determine the prediction block of the image block to be processed.

[0182] Specifically, the decoding end determines the target reference mode according to the indication information in the bit stream, and the encoding end makes a decision to determine the target reference mode.

[0183] After S405 , S403 is executed according to the target reference method.

[0184] Furthermore, after S403, the method provided in the embodiment of the present application may also record a historical reference list, which is used to indicate motion information or frequency information of prediction reference information of the processed image block. The historical reference list is used as a reference for subsequent inter-frame prediction of the image block.

[0185] The historical reference list may include any one of the following lists: a historical motion information table, a historical intra-frame copy information table, or a historical point prediction information table.

[0186] like Figure 5 As shown, the method provided in the embodiment of the present application may further include S406.

[0187] S406: Update the historical reference list.

[0188] Specifically, the specific implementation of the update history reference list provided in the embodiments of the present application may include but is not limited to any one of the following solutions.

[0189] Solution 1: Update the historical reference list only based on the prediction reference information of the image blocks of the lowest permission level, and do not update the historical reference list based on the prediction reference information of the image blocks of non-lowest permission levels.

[0190] The non-minimum permission level is one or more.

[0191] The prediction reference information of the image block is used to indicate the prediction process of the image block to be processed.

[0192] Exemplarily, the prediction reference information may include any one of the following information: location information, pattern information, or frequency. Of course, the content of the prediction reference information may be configured according to actual needs, and the present embodiment of the application is not limited thereto.

[0193] In Solution 1, after S403, S406 can be specifically implemented as follows: if the image block to be processed is of the lowest permission level, updating the historical reference list based on the predicted reference information of the image block to be processed. If the image block to be processed is not of the lowest permission level, prohibiting updating the historical reference list based on the predicted reference information of the image block to be processed.

[0194] Here, prohibiting the updating of the historical reference list according to the prediction reference information of the image block to be processed can be understood as: keeping the historical reference list unchanged.

[0195] Solution 2: Build and update the historical reference list based on the predicted reference information of the image block with the lowest permission level; after processing the image block with non-lowest permission, update the historical reference list as the historical reference list within the permission based on the predicted reference information of the image block with non-lowest permission; after processing the image block with the lowest permission, update the original historical reference list based on the predicted reference information of the image block with the lowest permission.

[0196] In solution 2, a temporary historical reference list is configured for each non-minimum permission region, and the historical reference list is used for inter-frame prediction of image blocks in the permission level region. It should be understood that in solution 2, multiple historical reference lists are recorded.

[0197] In Solution 2, after S403, S406 can be specifically implemented as follows: if the image block to be processed has the lowest permission level, updating the historical reference list based on the predicted reference information of the image block to be processed. If the permission level of the image block to be processed is not the lowest permission level, updating the temporary historical reference list corresponding to the permission level of the image block to be processed based on the predicted reference information of the image block to be processed.

[0198] Solution 3: Build and update the historical reference list based on the predicted reference information of the image block with the lowest permission level. After processing the image block with a non-lowest permission level, update the historical reference list based on the predicted reference information of the image block with the non-lowest permission level. After processing the image block with the lowest permission level, initialize the historical reference list and rebuild the historical reference list based on the predicted reference information of the image block with the lowest permission level.

[0199] In solution 3, only one historical reference list is configured, and the historical reference list is also updated for image blocks that are not of the lowest authority. However, when processing an image block of the lowest authority, the historical reference list is initialized and then rebuilt and updated.

[0200] In Solution 3, after the aforementioned S403, S406 can be specifically implemented as follows: if the image block to be processed is of the lowest authority level and the historical reference list has not been updated by the predicted reference information of the image block not of the lowest authority level, the historical reference list is updated according to the predicted reference information of the image block to be processed. If the authority level of the image block to be processed is of the lowest authority level and the historical reference list is updated by the predicted reference information of the image block not of the lowest authority level, the historical reference list is reconstructed according to the predicted reference information of the image block to be processed; if the image block to be processed is of a non-lowest authority level, the historical reference list is updated according to the predicted reference information of the image block to be processed.

[0201] Solution 4: Build different historical reference lists for different authority areas.

[0202] In solution 4, a historical reference list is configured for each permission level region, and the historical reference list is used for inter-frame prediction of image blocks within the permission level region. The image block to be processed is used to update the historical reference list corresponding to its permission level.

[0203] In solution 4, after the aforementioned S403, S406 may be specifically implemented as: updating the historical reference list of the permission level area to which the image block to be processed belongs according to the prediction reference information of the image block to be processed.

[0204] Solution 5: Different historical reference lists are constructed for some permission level areas.

[0205] In solution 5, a historical reference list is configured for each of the partial permission level regions, and the historical reference list is used for inter-frame prediction of image blocks in the permission level region. The image block to be processed is used to update the historical reference list corresponding to its permission level.

[0206] In Solution 5, after S403, S406 can be specifically implemented as follows: if a historical reference list is configured for the permission level region where the image block to be processed is located, the historical reference list of the permission level region to which the image block to be processed belongs is updated based on the predicted reference information of the image block to be processed. If a historical reference list is not configured for the permission level region where the image block to be processed is located, a historical reference list is constructed or updated according to other solutions.

[0207] Solution 6: Build and update the historical reference list based on the prediction reference information of the least privileged image block. After processing the image block that is not the least privileged, update the historical reference list based on the alternative prediction reference information that is allowed to be derived.

[0208] The alternative prediction reference information allowed to be derived may refer to the prediction reference information of an image block that is supported by the authority level of the user at the decoding end and for which the predicted pixel value has been determined.

[0209] In a possible implementation, the replacement prediction reference information may be default prediction reference information.

[0210] The default prediction reference information may be pre-configured fixed information, or may be dynamically generated fixed information, which is not limited in the embodiment of the present application.

[0211] In another possible implementation manner, the replacement prediction reference information may be prediction reference information of a default image block.

[0212] The default image block may be a pre-specified image block, and the embodiment of the present application does not limit the specific position of the default image block.

[0213] Exemplarily, the default image block may be an image block with the lowest permission level.

[0214] In another possible implementation, the replacement prediction reference information is prediction reference information of an image block whose permission level meets the reference condition and precedes the image block to be processed in the coding and decoding order.

[0215] In another possible implementation, the replacement prediction reference information is prediction reference information derived from prediction reference information of an image block whose permission level meets the reference condition and precedes the image block to be processed in the coding and decoding order.

[0216] In Scheme 6, after the aforementioned S403, S406 can be specifically implemented as follows: if the image block to be processed is of the lowest authority level, the historical reference list is updated according to the predicted reference information of the image block to be processed; if the authority level of the image block to be processed is not the lowest authority level, the historical reference list is updated according to the alternative predicted reference information allowed to be exported.

[0217] Solution 7: If the permission of the image block to be processed is higher than that of the previous region, the historical reference list is updated based on the prediction reference information of the image block to be processed. If the permission of the image block to be processed is lower than that of the previous region, the historical reference list is initialized based on the alternative prediction reference information that is allowed to be derived.

[0218] It should be noted that the updating history reference list in S406 provides a reference basis for updating the history reference list in the embodiment of the present application. The specific updating process can refer to the video compression coding standard.

[0219] For example, the historical intra-frame copy information table and the historical reference list may be updated with reference to section 9.5.6.7 of the video compression coding standard AVS3. The process may be as follows:

[0220] In Example 1, the reference list is a historical motion information list used for inter-frame prediction and is composed of motion information of prediction units. After decoding of the current prediction unit, if one of the following conditions is met, the operation of updating the historical motion information list is not performed.

[0221] Condition 1: The current prediction unit is an affine prediction unit or an angle-weighted prediction unit.

[0222] Condition 2: The coding unit subtype of the current coding unit is 'P_Skip_Mvap', 'P_Direct_Mvap', 'B_Skip_Mvap', or 'B_Direct_Mvap'.

[0223] Condition three: The coding unit subtype of the current coding unit is 'P_Skip_Etmvp', 'P_Direct_Etmvp', 'B_Skip_Etmvp', or 'B_Direct_Etmvp'.

[0224] Condition 4: The coding unit subtype of the current coding unit is 'P_Skip_SbTemporal', 'P_Direct_SbTemporal', 'B_Skip_SbTemporal', or 'B_Direct_SbTemporal'.

[0225] Condition 5: When NumOfHmvpCand is greater than 0, the historical motion information table HmvpCandList is updated according to the motion information, BgcFlag and BgcIndex of the current prediction block; otherwise, the operation defined in this clause is not performed.

[0226] Specifically, the method for updating the historical motion information table is as follows:

[0227] a. Initialize hmvpIndex to 0.

[0228] b. If CntHmvp is equal to 0, the motion information, BgcFlag, and BgcIndex in HmvpCandList[CntHmvp] are equal to the motion information, BgcFlag, and BgcIndex of the current prediction unit, respectively, and CntHmvp is increased by 1.

[0229] c. Otherwise, determine whether the motion information of the current prediction block is the same as HmvpCandList[hmvpIndex].

[0230] If the motion information is the same, execute step d); otherwise, add 1 to hmvpIndex; if hmvpIndex is less than CntHmvp, execute step c); otherwise, execute step d).

[0231] d. If hmvpIndex is less than CntHmvp, then:

[0232] i. From hmvpIndex to CntHmvp-1, let HmvpCandList[i] equal to HmvpCandList[i+1]; the motion information, BgcFlag, and BgcIndex in HmvpCandList[CntHmvp-1] are equal to the motion information, BgcFlag, and BgcIndex of the current prediction unit respectively.

[0233] e. If hmvpIndex is equal to CntHmvp and CntHmvp is equal to NumOfHmvpCand, then:

[0234] i ranges from 0 to CntHmvp-1, and HmvpCandList[i] is equal to HmvpCandList[i+1];

[0235] The motion information, BgcFlag, and BgcIndex in HmvpCandList[CntHmvp-1] are equal to the motion information, BgcFlag, and BgcIndex of the current prediction unit, respectively.

[0236] f. If hmvpIndex is equal to CntHmvp and CntHmvp is less than NumOfHmvpCand, the motion information, BgcFlag, and BgcIndex in HmvpCandList[CntHmvp] are equal to the motion information, BgcFlag, and BgcIndex of the current prediction unit, respectively, and CntHmvp is increased by 1.

[0237] Example 2: The reference list is a historical intra copy information list, which is used for block copy intra prediction or string copy intra prediction and consists of intra copy information.

[0238] The historical intra-frame copy information in the historical intra-frame copy information table is recorded as IntraHmvpCandList[X], including the displacement vector intraMvCandX, position (xCandX, yCandX), size sizeCandX and repetition number cntCandX.

[0239] Let intraCur be the intra copy information of the current block, including the displacement vector intraMvCur, position (xCur, yCur), size sizeCur and number of repetitions cntCur. The method for updating the historical intra copy information table can be:

[0240] a) Initialize both X and cntCur to 0.

[0241] b) If CntIntraHmvp is equal to 0, then IntraHmvpCandList[CntIntraHmvp] is the intra prediction motion information of the current prediction unit, CntIntraHmvp is increased by 1, and the update process of this item is ended.

[0242] c) Otherwise, determine whether the intra prediction motion information of the current prediction block is the same as IntraHmvpCandList[X] according to whether intraMvCur and intraMvCandX are equal.

[0243] 1) If intraMvCur and intraMvCandX are the same, go to step d); otherwise, increase X by 1.

[0244] 2) If X is less than CntIntraHmvp, go to step c); otherwise, go to step e).

[0245] d) cntCur is equal to the value of cntCandX plus 1. If sizeCur is less than sizeCandX, then xCur, yCur, and sizeCur are equal to xCandx, yCandx, and sizeCandX, respectively.

[0246] e) If X is less than CntIntraHmvp, then:

[0247] 1) if i is from X to CntIntraHmvp-1, let IntraHmvpCandList[i] be equal to IntraHmvpCandList[i+1];

[0248] 2) IntraHmvpCandList[CntIntraHmvp-1] is equal to the intra prediction motion information of the current prediction unit.

[0249] f) Otherwise, if X is equal to CntIntraHmvp and CntIntraHmvp is equal to NumOfIntraHmvpCand, then:

[0250] 1) i ranges from 0 to CntIntraHmvp-1, and IntraHmvpCandList[i] is equal to IntraHmvpCandList[i+1];

[0251] 2) IntraHmvpCandList[CntIntraHmvp-1] is equal to the intra prediction motion information of the current prediction unit.

[0252] Otherwise, if X is equal to CntIntraHmvp and CntIntraHmvp is less than NumOfIntraHmvpCand, then IntraHmvpCandList[CntIntraHmvp] is equal to the intra prediction motion information of the current prediction unit, and CntIntraHmvp is increased by 1.

[0253] Example 3: The reference list is a historical point prediction information list, which is used in a non-normal string mode and is composed of point prediction information lists of prediction units.

[0254] After completing the decoding of the current coding unit, if the current coding unit adopts the string copy intra-frame prediction non-ordinary string sub-mode and (IscNumOfNewPv+IscNumofReusedPv) is not equal to 0, the historical point prediction information table PrevPpInfoList is updated according to the point prediction information table PpInfoList of the current coding unit, and the lists PrevFopYonly, PrevEvsDpbReactivatedYonly, PrevCompLumaFreqOccurPos and PrevEvsDpbIndex are updated; otherwise, the operations defined in this article are not performed.

[0255] Let PrevPvBufSize be equal to the total number of point vectors in the historical point prediction information table, PvBufSize be equal to the total number of point vectors in the point prediction information table of the current coding unit, tmpPvBuf[i], tmpFlag[i], tmpEvsDpbIndex[i], tmpEvsDpbReactivatedYonly[i] and tmpCompLumaFreqOccurPos[i] (i=0-27) are temporary buffers for point prediction information. The steps for updating the historical point prediction information table in the string copy prediction mode can be:

[0256] tmpIndex=0;

[0257] for(k=0;k <PrevPvBufSize;k++){

[0258] if(PrevPpInfoList[k][0]!=-1&&PrevPpInfoList[k][1]!=-1){

[0259] tmpPvBuf[tmpIndex][0]=PrevPpInfoList[k][0]

[0260] tmpPvBuf[tmpIndex][1]=PrevPpInfoList[k][1]

[0261] tmpFlag[tmpIndex]=PrevFopYonly[k]

[0262] tmpEvsDpbIndex[tmpIndex]=PrevEvsDpbIndex[k]

[0263] tmpEvsDpbReactivatedOnly[tmpIndex]=PrevEvsDpbReactivatedOnly[k]

[0264] tmpCompLumaFreqOccurPos[tmpIndex]=PrevCompLumaFreqOccurPos[k]

[0265] tmpIndex++

[0266] }

[0267] }

[0268] PrevPvBufSize=Min(28,PvNum+tmpIndex)

[0269] for(k=0;k <PrevPvBufSize;k++){

[0270] if(k <PvBufSize){

[0271] PrevPpInfoList[k][0]=PpInfoList[k][0]

[0272] PrevPpInfoList[k][1]=PpInfoList[k][1]

[0273] PrevFopYonly[k]=FopYonly[k]

[0274] PrevEvsDpbIndex[k]=EvsDpbIndex[k]

[0275] PrevCompLumaFreqOccurPos[k]=CompLumaFreqOccurPos[k]

[0276] PrevEvsDpbReactivatedYonly[k]=EvsDpbReactivatedYonly[k]

[0277] }

[0278] else{

[0279] PrevPpInfoList[k][0]=tmpPvBuf[k-PvNum][0]

[0280] PrevPpInfoList[k][1]=tmpPvBuf[k-PvNum][1]

[0281] PrevFopYonly[k]=tmpFlag[k-PvNum]

[0282] PrevEvsDpbIndex[k]=tmpEvsDpbIndex[k-PvNum]

[0283] PrevCompLumaFreqOccurPos[k]=tmpCompLumaFreqOccurPos[k-PvNum]

[0284] PrevEvsDpbReactivatedYonly[k]=tmpEvsDpbReactivatedYonly[k-PvNum]

[0285] }

[0286] }

[0287] The point vectors used in the current coding unit are stored in the point prediction information table PpInfoList. The point vectors in PpInfoList consist of two parts: one part comes from the historical point prediction information table PrevPpInfoList

[28] [2], and the number of point vectors in the historical point prediction information table should not be greater than 28; the other part is the new point vectors in the current coding unit.

[0288] If the current coding unit uses a normal string sub-mode, the sum of the number of matching strings, the number of incomplete matching strings containing at least one matching sample, the number of unmatched samples, and IscPartNumSplit should be less than or equal to one-quarter of the number of samples in the current coding unit; otherwise, if the current coding unit uses a non-normal string sub-mode, the sum of the number of equivalent strings, the number of unit basis vector strings, the number of unmatched samples, and IscPartNumSplit should be less than or equal to one-quarter of the number of samples in the current coding unit.

[0289] Furthermore, after S403, the method provided in the embodiment of the present application derives spatial motion information. Deriving spatial motion information refers to using the motion information of the surrounding image blocks adjacent to the image block to be processed to determine the motion information of the current image block to be processed. In intra-frame prediction and inter-frame prediction P-frames and B-frames, by deriving spatial motion information, the motion information of the current image block to be processed is determined using the motion information of the surrounding image blocks adjacent to the image block to be processed.

[0290] like Figure 5 As shown, the method provided in the embodiment of the present application may further include S407.

[0291] S407: Export airspace movement information.

[0292] Specifically, the specific implementation of deriving spatial motion information provided in the embodiments of the present application may include but is not limited to any one of the following schemes.

[0293] Solution a: If the permission level of the adjacent spatial image block does not meet the reference condition, the adjacent spatial image block is marked as non-existent. That is, the image block to be processed does not have an adjacent spatial image block from which spatial motion information can be derived.

[0294] Solution b: If the permission level of the adjacent image block does not meet the reference condition, the spatial domain prediction information of the image block to be processed is obtained from the code stream.

[0295] Solution c: If the permission level of the adjacent spatial domain image block used does not meet the reference condition, the spatial domain prediction information of the image block to be processed is derived according to the alternative spatial domain prediction information of the adjacent image block that is allowed to be derived.

[0296] The alternative spatial prediction information of the adjacent image blocks that is allowed to be derived may refer to the spatial prediction information of the image blocks that are supported by the authority level of the decoding end user and for which the predicted pixel values ​​have been determined.

[0297] In one possible implementation, the replacement airspace prediction information is the default airspace prediction information.

[0298] Among them, the default airspace prediction information can be pre-configured fixed information, or can be dynamically generated fixed information, which is not limited in the embodiments of the present application.

[0299] In another possible implementation, the default spatial domain prediction information may be spatial domain prediction information of a default image block.

[0300] In a possible implementation, the replacement spatial prediction information is spatial prediction information derived from spatial prediction information of an image block whose permission level meets a reference condition and precedes the image block to be processed in the encoding and decoding order.

[0301] It should be noted that, regarding the export of spatial prediction information in S407, the embodiment of the present application provides a reference basis for exporting the spatial prediction information. For the specific export process, reference can be made to the video compression coding standard.

[0302] For example, the spatial motion information derivation method in section 9.5.7.8.3 of the video compression coding standard AVS3 may be referred to to derive the spatial motion information. The process may be as follows:

[0303] Example 1: P-image spatial motion information derivation method. The motion information, BgcFlag, and BgcIndex derivation methods are as follows:

[0304] a) If the adjacent luminance prediction blocks F, G, C, A, B, and D of the luminance prediction block of the current prediction unit are ( Figure 8 ), if the number of prediction blocks with the prediction reference mode being 'PRED_List0' is greater than or equal to 1, the adjacent luminance prediction blocks are scanned in the order of F, G, C, A, B, and D to obtain the first scanned prediction block with the prediction reference mode being 'PRED_List0', and the L0 motion vector and L0 reference index of the spatial motion information storage unit of the prediction block are used as the L0 motion vector mvE0 and L0 reference index refIndexL0 of the current prediction unit, respectively.

[0305] b) Otherwise, the L0 motion vector mvE0 of the current prediction unit is a zero vector, and the value of the L0 reference index refIndexL0 of the current prediction unit is equal to 0;

[0306] c) The value of interPredRefMode is equal to 'PRED_List0', the value of refIndexL1 is equal to -1, mvE1 is the zero vector, and the values ​​of BgcFlag and BgcIndex are both set to 0.

[0307] Example 2: Method 1 for deriving B-picture spatial motion information. Method 1 for deriving motion information, BgcFlag, and BgcIndex is as follows:

[0308] a) If the number of prediction blocks with a prediction reference mode of 'PRED_List01' among the adjacent luminance prediction blocks F, G, C, A, B, and D of the luminance prediction block of the current prediction unit is greater than or equal to 1, the adjacent luminance prediction blocks are scanned in the order of F, G, C, A, B, and D to obtain the first scanned prediction block with a prediction reference mode of 'PRED_List01', and the L0 motion vector and L1 motion vector of the spatial motion information storage unit of the prediction block are used as the L0 motion vector mvE0 and L1 motion vector mvE1 of the current prediction unit, respectively, and the L0 motion vector and L1 motion vector ... The L0 reference index and L1 reference index of the spatial motion information storage unit are used as the L0 reference index refIndexL0 and L1 reference index refIndexL1 of the current prediction unit, respectively. At the same time, the BgcFlag and BgcIndex of the spatial motion information storage unit are recorded as BgcFlagX and BgcIndexX. The values ​​of BgcFlag and BgcIndex of the current prediction unit are (BgcFlagX&&!InterPcFlag) and (BgcIndexX&&!InterPcFlag), respectively.

[0309] b) Otherwise, if the number of prediction blocks with a prediction reference mode of 'PRED_List0' among the six adjacent luminance prediction blocks F, G, C, A, B, and D of the luminance prediction block of the current prediction unit is greater than or equal to 1, and the number of prediction blocks with a prediction reference mode of 'PRED_List1' is greater than or equal to 1, then the adjacent luminance prediction blocks are scanned in the order of F, G, C, A, B, and D to obtain the first scanned prediction block with a prediction reference mode of 'PRED_List0' and the first scanned prediction block with a prediction reference mode of 'PRED_List1', and The L0 motion vector and L0 motion index of the spatial motion information storage unit of the prediction block with the prediction reference mode of 'PRED_List0' are used as the L0 motion vector mvE0 and L0 motion index refIndexL0 of the current prediction unit; the L1 motion vector and L1 reference index of the spatial motion information storage unit of the prediction block with the prediction reference mode of 'PRED_List1' are used as the L1 motion vector mvE1 and L1 reference index refIndexL1 of the current prediction unit, and the BgcFlag and BgcIndex of the current prediction unit are both set to 0;

[0310] c) Otherwise, the L0 motion vector mvE0 and the L1 motion vector mvE1 of the current prediction unit are both zero vectors, and the values ​​of the L0 reference index refIndexL0 and the L1 reference index refIndexL1 of the current prediction unit are both equal to 0, and the BgcFlag and BgcIndex of the current prediction unit are both set to 0;

[0311] d) The value of interPredRefMode is equal to 'PRED_List01'.

[0312] Example 3, B-picture spatial motion information derivation method 2, motion information, BgcFlag and BgcIndex derivation method 2 is as follows:

[0313] a) If the number of prediction blocks with a prediction reference mode of 'PRED_List1' among the adjacent luma prediction blocks F, G, C, A, B, and D of the luma prediction block of the current prediction unit is greater than or equal to 1, then the adjacent luma prediction blocks are scanned in the order of F, G, C, A, B, and D to obtain the first scanned prediction block with a prediction reference mode of 'PRED_List1', and the L1 motion vector and L1 reference index of the spatial motion information storage unit of the prediction block are used as the L1 motion vector mvE1 and L1 reference index refIndexL1 of the current prediction unit;

[0314] b) Otherwise, if the number of prediction blocks whose prediction reference mode is 'PRED_List01' among the six adjacent luma prediction blocks F, G, C, A, B, and D of the luma prediction block of the current prediction unit is greater than or equal to 1, then the adjacent luma prediction blocks are scanned in the order of D, B, A, C, G, and F to obtain the first scanned prediction block whose prediction reference mode is 'PRED_List01', and the L1 motion vector and L1 reference index of the spatial motion information storage unit of the prediction block are used as the L1 motion vector mvE1 and L1 reference index refIndexL1 of the current prediction unit;

[0315] c) Otherwise, the L1 motion vector mvE1 of the current prediction unit is a zero vector, and the value of the L1 reference index refIndexL1 of the current prediction unit is equal to 0;

[0316] d) The value of interPredRefMode is equal to 'PRED_List1', the value of refIndexL0 is equal to -1, mvE0 is the zero vector, and the values ​​of BgcFlag and BgcIndex are both set to 0.

[0317] Example 4: Method 3 for deriving spatial motion information of B-pictures. The method for deriving motion information, BgcFlag, and BgcIndex is as follows:

[0318] a) If the number of prediction blocks with a prediction reference mode of 'PRED_List0' among the adjacent luma prediction blocks F, G, C, A, B, and D of the luma prediction block of the current prediction unit is greater than or equal to 1, then the adjacent luma prediction blocks are scanned in the order of F, G, C, A, B, and D to obtain the first scanned prediction block with a prediction reference mode of 'PRED_List0', and the L0 motion vector and L0 reference index of the spatial motion information storage unit of the prediction block are used as the L0 motion vector mvE0 and L0 reference index refIndexL0 of the current prediction unit;

[0319] b) Otherwise, if the number of prediction blocks with a prediction reference mode of 'PRED_List01' among the six adjacent luma prediction blocks F, G, C, A, B, and D of the luma prediction block of the current prediction unit is greater than or equal to 1, then the adjacent luma prediction blocks are scanned in the order of D, B, A, C, G, and F to obtain the first scanned prediction block with a prediction reference mode of 'PRED_List01', and the L0 motion vector and L0 reference index of the spatial motion information storage unit of the prediction block are used as the L0 motion vector mvE0 and L0 reference index refIndexL0 of the current prediction unit;

[0320] c) Otherwise, the L0 motion vector mvE0 of the current prediction unit is a zero vector, and the value of the L0 reference index refIndexL0 of the current prediction unit is equal to 0;

[0321] d) The value of interPredRefMode is equal to 'PRED_List0', the value of refIndexL1 is equal to -1, mvE1 is the zero vector, and the values ​​of BgcFlag and BgcIndex are both set to 0.

[0322] The solution provided in this application is described below with examples.

[0323] Example 1:

[0324] In intra-frame prediction, the Audio Video Coding Standard 2 (AVS2) has 33 intra-frame prediction modes, including 30 angle modes and 3 special modes, using 2 most probable modes (MPM) encoding, and the remaining modes using 5-bit fixed-length encoding. To support more precise angle prediction, the Audio Video Coding Standard 3 (AVS3) expands the angle prediction modes to 62. Figure 6The figure shows the intra-frame prediction modes, where the solid lines represent the existing angular prediction modes and the dashed lines represent the newly added angular modes. The existing angular mode numbers remain unchanged, while the newly added angular modes are numbered from 34 to 65. When the permission level of the corresponding reference pixel in a prediction direction is higher, when other areas with lower permission levels refer to that direction for prediction, the reference is prohibited or an alternative value that is allowed to be derived is used as the reference for prediction.

[0325] Example 2:

[0326] The MPM list is derived using spatial information, and the current block prediction mode can be derived according to the MPM list. The MPM list stores the prediction mode of the left / upper adjacent block. Figure 7 This table shows the positional relationship between the left / top neighboring blocks and the current image block. If the left / top neighboring blocks do not exist or are equal, one or two fixed patterns will be used to fill the list. If the permission level of the neighboring blocks of the current image block is higher than or not equal to the permission level of the current image block, when constructing the spatial information export, it is prohibited to derive the prediction module of the current image block based on the prediction mode of the left / top neighboring blocks, or, alternatively, derive the prediction module of the current image block based on the prediction module of the alternative image block of the left / top neighboring blocks.

[0327] Example 3:

[0328] In the intra reference pixel filtering (IRPF) scenario, the design concept of intra reference pixel filtering in the second stage of AVS3 is as follows: based on the area of ​​the block where the current pixel is located, a set of two candidate filters is determined. When the area is less than or equal to Thd_Area = 64 luminance pixels, the first set of candidate filters is selected; otherwise, the second set of candidate filters is selected. Furthermore, based on the position P (P ≥ 0) of the current pixel in the current block, if P is less than row N or column N (counting starts from row 0, column 0), the first candidate filter in the determined candidate filter set is selected; otherwise, the second candidate filter is selected.

[0329] The first filter group is {f3, f2}, and the second filter group is {f2, f1}. The luminance component N = 1 (i.e., the first filter in the group is selected for the first row and first column of the pixel, and the second filter is selected for the remaining pixels). The chrominance component N = 2 (i.e., the first filter in the group is selected for the first and second rows and the first and second columns of the pixel, and the second filter is selected for the remaining pixels). The above f1, f2, f3, and f4 are four different filter groups.

[0330] When some pixels in the intra-frame reference pixels have an authority level higher than or not equal to the authority level of the current image block prediction block, it is prohibited to refer to or use the allowed derived alternative value as a reference for prediction.

[0331] Example 4:

[0332] In the intra prediction filter (IPF) scenario, spatial correlation can be effectively enhanced by using prediction filtering, thereby improving the accuracy of intra prediction. IPF uses reference pixels in the URB to filter the intra prediction block. There are three types of filters in IPF, namely horizontal 2-tap filter (1), vertical 2-tap filter (2) and 3-tap filter (3) that filters in both horizontal and vertical directions. When the reference pixel authority level corresponding to different filters during filtering is higher than or not equal to the current image block authority, reference is prohibited or reference is allowed to be derived as an alternative value.

[0333] Example 5:

[0334] The solution of the present application is applied in a two-step cross-component prediction code (TSCPM) and an extended prediction from multiple cross-components (EPMC) mode.

[0335] TSCPM is an inter-component prediction technique that removes inter-component redundancy by exploring the linear relationships between different components. TSCPM is performed in two steps: first, a co-located luma block is used to generate a temporary prediction block of the same size using parameters α and β. Second, downsampling is performed to obtain predicted values ​​for the chrominance components.

[0336] First, we divide the availability of adjacent block pixels into three cases to obtain four available pixel pairs. We calculate α and β through the four available pixel pairs. After obtaining α and β, we reconstruct the pixels through luminance according to the linear relationship between luminance and chrominance to obtain the chrominance prediction value. Therefore, when obtaining available pixels, we need to consider the permission level of the pixel. When the permission level of the pixel is higher than or equal to the permission level of the current block, the pixel can be marked as unavailable or replaced by other pixels. When marked as unavailable, the value of the pixel is as follows:

[0337] When selecting four pairs of available pixels, the availability of the upper and left pixels needs to be considered, which can be divided into the following three cases:

[0338] Case A: If the pixels immediately above and immediately to the left of the current block are both “available”, two pixel pairs are selected from the upper side and two pixel pairs are selected from the left side.

[0339] Case B: If only the upper side of the current block is available, the four pixel pairs are all selected from the upper side, and the selected position widths are: 0 / 4, 1 / 4, 2 / 4, 3 / 4.

[0340] Case C: If only the left pixel is available in the current block, the four pixel pairs are all selected from the left side, and the selected positions are: 0 / 4, 1 / 4, 2 / 4, 3 / 4 of the height.

[0341] For the above situation A, enhanced TSCPM proposes two enhanced modes, namely TSCPM_T and TSCPM_L:

[0342] One mode: 4 pixel pairs only come from the top side (TSCPM_T).

[0343] Another mode: 4 pixel pairs only come from the left side (TSCPM_L).

[0344] Example 6:

[0345] The history-based motion vector prediction (HMVP) technology copies 8 motion information candidates from the previous coding block into a first-in first-out (FIFO) queue, which is continuously updated in a first-in first-out manner. If the motion candidate in the FIFO is the same as the motion information just encoded, the duplicate candidate will be removed first, and the motion information of the current coding unit will be added to the end of the FIFO. If the motion information of the current coding unit is different from the motion information of any candidate in the FIFO, the first candidate in the FIFO will be removed, and the latest motion information will be added to the end of the FIFO to ensure that the 8 latest motion candidates are always retained in the FIFO. If the permission level of the current coding unit (the image block to be processed) is not the lowest permission level, the available candidates in the candidate list will not be used to update the list, or a different candidate list will be constructed for each permission level.

[0346] Example 7:

[0347] The candidate motion vector list for angular weighted prediction (AWP) is constructed using the motion vectors of surrounding blocks (spatial neighbors). Reference weights are set for locations around the current block (whole-pixel and sub-pixel positions). Angular prediction is then used to determine the weight for each pixel. The resulting weights are then used to weight the two different inter-frame prediction values.

[0348] The reference weight configuration is a distribution function of the reference weight values ​​obtained based on the reference weight index value. A non-strictly monotonically increasing function is assigned using the 8-point position of the reference weight effective length as the reference point. The reference weight effective length is calculated from the predicted angle and the current block size. The angle is divided into 4 partitions, and the formula for deriving the pixel-by-pixel weight varies slightly depending on the area where the angle is located. The block size of the current block is M×N, where M is the width, N is the height, X is log2 (the absolute value of the slope of the weight prediction angle), and Y is the weight prediction position.

[0349] 1) Taking angle 0 and angle 1 in angle partition 0 as an example, the derivation process is as follows:

[0350] Calculate the reference weight valid length ValidLenth, ValidLenth = (N + (M>>X)) <<1.

[0351] Set the reference weight value ReferenceWeights[x], where the value range of x is 0 to ValidLength-1.

[0352] FirstPos=(ValidLength>>1)-6+Y*((ValidLength-1)>>3)

[0353] ReferenceWeights[x]=Clip3(0,8,x-FirstPos)

[0354] Derive pixel-by-pixel weights SampleWeight[x][y].

[0355] SampleWeight[x][y]=ReferenceWeights[(y<<1)+((x<<1)>>X)]

[0356] 2) Chroma weight is derived as:

[0357] The chroma weight is derived by directly taking the upper left corner position of the corresponding 2×2 brightness weight. The block size of the current block is M×N, where M is the width and N is the height. The value range of x is 0 to (M / 2-1); the value range of y is 0 to (N / 2-1).

[0358] SampleWeightChroma[x][y]=SampleWeight[x>>1][y>>1].

[0359] The motion vector storage scheme of the angle weighted mode is related to the angle partition. The block size of the current block is M×N, where M is the width, N is the height, X is log2 (the absolute value of the weighted prediction angle slope), and Y is the weighted prediction position. Taking angle 0 and angle 1 in angle partition 0 as an example, the motion vector storage scheme is:

[0360] Calculate the reference weight effective length ValidLenth, ValidLenth = (N + (M>>X)) << 1

[0361] For each 4×4 block of the current block, record its center position as (x, y).

[0362] FirstPos=(ValidLength>>1)-6+Y*((ValidLength-1)>>3).

[0363] If (y<<1)+((x<<1)>>X) is greater than or equal to FirstPos, the first motion information is stored; otherwise, the second motion information is stored.

[0364] Angle weighted prediction requires two unidirectional motion vectors for weighted prediction. The unidirectional motion vector is selected from the unidirectional candidate motion vector list according to the index in the bitstream. The candidate motion vector list is constructed by the motion vectors of the surrounding blocks. The surrounding blocks used are as follows: Figure 8 As shown in Figure 1, each block is given a sequence number in advance. The specific process of constructing the candidate motion vector list is as follows:

[0365] Add the blocks to the list in the order of T, F, G, C, A, B, and D and check for duplicates (worst case: 18 one-way duplicate checks). When the permission level of the neighboring block is higher than or not equal to the permission level of the current image block, the neighboring block information is marked as unavailable when constructing the candidate motion vector of the current block, or the default value information is used to replace the neighboring block information to construct the list. Then, when constructing the list, follow the following scheme:

[0366] If the motion information of the T time domain block L0 is available, the motion information of L0 is used for scaling to obtain the motion information; if the motion information of the time domain block L0 is not available, the motion information of the time domain block L1 is used for scaling to obtain the motion information.

[0367] For spatial blocks, if the current block's motion information is unidirectional, the unidirectional motion information is directly retrieved and checked for duplicates. If the current block's motion information is bidirectional, the information is trimmed to unidirectional motion information based on parity and checked for duplicates. If the list is not full, the scaling scheme is applied up to four times based on the first motion information in the unidirectional and bidirectional lists.

[0368] It should be noted that the above examples are only used to illustrate the solution provided by this application and do not constitute a limitation on the application scenarios of the solution of this application.

[0369] Figure 9 Schematic diagram of the structure of an image coding and decoding device 90 provided in an embodiment of the present application. The image coding and decoding device 90 includes: a determination module 901 and a processing module 902.

[0370] The determination module 901 is used to determine the prediction mode of the image block to be processed and the authority level of the image block to be processed. Figure 4 or Figure 5 The process of S401.

[0371] As a possible implementation, the determination module 901 may also be used to determine the first reference image block of the image block to be processed according to the prediction mode of the image block to be processed. Figure 4 or Figure 5 The process of S402.

[0372] As a possible implementation, the determination module 901 may also be used to determine a target reference mode for the image block to be processed if the permission level of the first reference image block does not meet the reference condition. The target reference mode includes: prohibiting the use of the first reference image block to determine the prediction block of the image block to be processed, or using the second reference image block that is allowed to be derived to determine the prediction block of the image block to be processed. Exemplarily, the determination module 901 may be used to support the image encoding and decoding apparatus 90 to perform Figure 5 The process of S405.

[0373] The processing module 902 is used to determine if the permission level of the first reference image block does not meet the reference condition, prohibit the use of the first reference image block to determine the prediction block of the image block to be processed, or use the second reference image block that is allowed to be derived to determine the prediction block of the image block to be processed. Exemplarily, the processing module 902 can be used to support the image encoding and decoding device 90 to perform Figure 4 or Figure 5 The process of S403.

[0374] As a possible implementation, the processing module 902 may also be configured to determine a prediction block of the image block to be processed using the first reference image block if the permission level of the first reference image block meets the reference condition. Figure 5 The process of S404.

[0375] Furthermore, the processing module 902 may also be used to update the historical reference list. Figure 5 The process of S406.

[0376] As a possible implementation method, the processing module 902 is also used to: if the image block to be processed is of the lowest authority level, update the historical reference list according to the predicted reference information of the image block to be processed; if the image block to be processed is not of the lowest authority level, prohibit updating the historical reference list according to the predicted reference information of the image block to be processed; the historical reference list is used to indicate the motion information of the predicted reference information of the processed image block; the predicted reference information of the image block is used to indicate the prediction process of the image block to be processed; the non-lowest authority level is one or more.

[0377] As another possible implementation method, the processing module 902 is also used to: if the image block to be processed is of the lowest authority level, update the historical reference list according to the predicted reference information of the image block to be processed; if the authority level of the image block to be processed is not the lowest authority level, update the historical reference list according to the alternative predicted reference information allowed to be exported.

[0378] As another possible implementation method, the processing module 902 is also used to: if the image block to be processed is of the lowest authority level, update the historical reference list according to the predicted reference information of the image block to be processed; if the authority level of the image block to be processed is not the lowest authority level, update the temporary historical reference list corresponding to the authority level of the image block to be processed according to the predicted reference information of the image block to be processed.

[0379] As another possible implementation method, the processing module 902 is also used to: if the image block to be processed is of the lowest authority level and the historical reference list has not been updated by the predicted reference information of the image block not of the lowest authority level, update the historical reference list according to the predicted reference information of the image block to be processed; if the authority level of the image block to be processed is of the lowest authority level and the historical reference list is updated by the predicted reference information of the image block not of the lowest authority level, reconstruct the historical reference list according to the predicted reference information of the image block to be processed; if the image block to be processed is not of the lowest authority level, update the historical reference list according to the predicted reference information of the image block to be processed.

[0380] As another possible implementation, the processing module 902 is further configured to update the historical reference list of the permission level region to which the image block to be processed belongs according to the predicted reference information of the image block to be processed; different historical reference lists are constructed for regions with different permission levels.

[0381] As another possible implementation method, the processing module 902 is also used to: if the permission level of the image block to be processed is higher than the permission level of the previous area, update the historical reference list according to the predicted reference information of the image block to be processed; if the permission level of the image block to be processed is lower than the permission level of the previous area, initialize the historical reference list according to the alternative predicted reference information.

[0382] Furthermore, the processing module 902 can also be used to derive spatial motion information. For example, the processing module 902 can be used to support the image encoding and decoding device 90 to perform Figure 5 The process of S407.

[0383] As a possible implementation manner, the processing module 902 is further configured to: if the permission level of the used adjacent spatial domain image block does not meet the reference condition, mark the adjacent spatial domain image block as non-existent.

[0384] As a possible implementation manner, the processing module 902 is further configured to: if the permission level of the adjacent image block does not meet the reference condition, parse the code stream to obtain spatial domain prediction information of the image block to be processed.

[0385] As a possible implementation method, the processing module 902 is also used to: if the permission level of the adjacent spatial domain image block used does not meet the reference condition, export the spatial domain prediction information of the image block to be processed according to the alternative spatial domain prediction information of the adjacent image block allowed to be exported.

[0386] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented using a software program, all or part of the embodiments may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions in accordance with the embodiments of the present application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (eg, a floppy disk, a magnetic disk, a magnetic tape), an optical medium (eg, a digital video disc (DVD)), or a semiconductor medium (eg, a solid state drive (SSD)).

[0387] Through the description of the above embodiments, those skilled in the art will clearly understand that for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working processes of the above-described systems, devices, and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0388] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0389] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0390] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0391] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as flash memory, mobile hard disk, read-only memory, random access memory, magnetic disk or optical disk.

[0392] The above is only a specific embodiment of the present application, but the scope of protection of this application is not limited to this. Any changes or substitutions within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. An image decoding method, characterized in that: The method comprises: Determining a prediction mode of an image block to be processed and an authority level of the image block to be processed; Determining a first reference image block for the image block to be processed according to the prediction mode; If the permission level of the first reference image block does not meet a reference condition, prohibiting the use of the first reference image block to determine the prediction block of the image block to be processed or prohibiting the use of the second reference image block that is allowed to be derived to determine the prediction block of the image block to be processed; wherein the reference condition includes a permission level that is lower than the permission level of the image block to be processed; The second reference image block includes an image block determined by an image block whose permission level satisfies the reference condition among image blocks adjacent to the first reference image block, or an image block with a default pixel value; If the image block to be processed is of a non-minimum authority level, it is prohibited to update the historical reference list based on the prediction reference information of the image block to be processed; the historical reference list is used to indicate the motion information of the prediction reference information of the processed image block; the prediction reference information of the image block to be processed is used to indicate the prediction process of the image block to be processed; the non-minimum authority level is one or more.

2. The method according to claim 1, characterized in that The method further comprises: If the image block to be processed is at the lowest authority level, the historical reference list is updated according to the prediction reference information of the image block to be processed.

3. The method according to claim 1, characterized in that The prediction reference information includes any one of the following information: location information, pattern information or frequency; The historical reference list includes any one of the following lists: a historical motion information table, a historical intra-frame copy information table, or a historical point prediction information table.

4. The method according to claim 1, wherein The method further comprises: If the permission level of the used adjacent spatial domain image block does not meet the reference condition, mark the adjacent spatial domain image block as not existing; or, If the permission level of the used adjacent spatial image block does not meet the reference condition, the spatial prediction information of the image block to be processed is derived according to the alternative spatial prediction information of the adjacent image block that is allowed to be derived.

5. The method according to claim 4, characterized in that The alternative airspace prediction information is the default airspace prediction information.

6. The method according to claim 1, characterized in that The method further comprises: If the permission level of the first reference image block meets the reference condition, the first reference image block is used to determine a prediction block for the image block to be processed.

7. The method according to claim 1, characterized in that The method further includes: if the permission level of the first reference image block does not meet the reference condition, determining a target reference mode for the image block to be processed; wherein the target reference mode includes: prohibiting the use of the first reference image block to determine the prediction block of the image block to be processed or using a second reference image block that is allowed to be derived to determine the prediction block of the image block to be processed.

8. An image decoding device, characterized in that: The device comprises: a determination module, configured to determine a prediction mode of an image block to be processed and a permission level of the image block to be processed; The determining module is further configured to determine a first reference image block for the image block to be processed according to the prediction mode; a processing module, configured to prohibit use of the first reference image block to determine a prediction block for the image block to be processed, or to use a second reference image block that is allowed to be derived to determine a prediction block for the image block to be processed, if the permission level of the first reference image block does not meet a reference condition; wherein the reference condition includes a permission level being lower than the permission level of the image block to be processed; The second reference image block includes an image block determined by an image block whose permission level satisfies the reference condition among image blocks adjacent to the first reference image block, or an image block with a default pixel value; The processing module is further configured to prohibit updating a historical reference list based on the prediction reference information of the image block to be processed if the image block to be processed is not of the lowest authority level; the historical reference list is used to indicate motion information of the prediction reference information of the processed image block; the prediction reference information of the image block to be processed is used to indicate a prediction process of the image block to be processed; and the non-lowest authority level is one or more.

9. A decoder, characterized in that: comprising a processor coupled to a memory; Memory, used to store computer programs or instructions; A processor, configured to execute the computer program or instructions stored in the memory, so that the decoder performs the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The program code comprises a program code which, when executed by a computer device, executes the method according to any one of claims 1 to 7.

Images