Adaptive loop filter classification method in video coding and decoding and related device

Abstract

The embodiment of the invention provides a self-adaptive loop filtering classification method in video coding and decoding and a related device, and relates to the technical field of video coding and decoding, multimedia, cloud technology, artificial intelligence and the like. The method comprises the steps of obtaining classification information corresponding to a target image, and then determining a target category of an adaptive loop filter corresponding to the target image based on the classification information. The classification information includes at least one of a target range, intermediate information, or a subset of categories. Wherein the target image corresponds to an image sequence, a frame of image, and a strip image, a rectangular image or a block image in the frame of image in the video. According to the embodiment of the invention, the determination accuracy of the category of the ALF is improved, so that the technical effect of improving the video quality is achieved.

Description

Technical Field

[0001] The present application relates to the field of video coding and decoding technology, and may involve the fields of multimedia, cloud technology, artificial intelligence, etc. Specifically, it relates to an adaptive loop filter classification method and related devices in video coding and decoding. Background Art

[0002] Various video codec technologies may be used to compress video data, so that one or more video codec standards may be executed to perform encoding and decoding of video data. Exemplary, video codec standards may include, but are not limited to, Versatile Video Codec (VVC), Joint Exploration Test Model (JEM), High Efficiency Video Codec (H.265 / HEVC), Advanced Video Codec (H.264 / AVC), Moving Picture Experts Group (MPEG) codec, etc. Video codecs may generally perform prediction methods (e.g., inter-frame prediction, intra-frame prediction, etc.), which utilize redundancy present in a sequence of image frames of a video. For example, by performing video codecs, redundancy present in image frames of a video may be reduced or removed from video data.

[0003] In the related technologies of video coding and decoding, the video coding and decoding process generally requires the use of adaptive loop filter (ALF). ALF can adaptively determine the filter coefficient according to different video contents, thereby reducing the mean square error (MSE) between the reconstructed component and the original component, thereby improving the video quality.

[0004] However, it is found in practice that the current video quality needs to be further improved. Summary of the invention

[0005] The embodiments of the present application provide an adaptive loop filter classification method and related devices in video encoding and decoding, which can improve the accuracy of determining the ALF category and thus improve the video quality.

[0006] On the one hand, an embodiment of the present application provides an adaptive loop filtering classification method in video coding and decoding, characterized in that it includes:

[0007] Obtaining classification information corresponding to the target image, the classification information including at least one of a target range, intermediate information, or a category subset; wherein the target image corresponds to an image sequence in a video, a frame of image, a strip image in a frame of image, a rectangular image, or a block image;

[0008] Based on the classification information, determining a target category of an adaptive loop filter corresponding to the target image;

[0009] The target range includes at least one of a first dynamic range or a second dynamic range, the first dynamic range is a range determined based on pixel information of the target image, and the second dynamic range is a range determined based on a preset dynamic range, and at least one preset dynamic range is smaller than a maximum range that can be represented by a signal bit width of the target image;

[0010] Intermediate information includes information generated by the target image during the encoding and decoding process;

[0011] The category subset includes at least two subcategories. The at least two subcategories are categories of adaptive loop filters corresponding to the target image determined by respective classifiers in the classifier group. Each subcategory corresponds to a classifier.

[0012] In a possible implementation, a minimum value of the first dynamic range is determined based on a minimum signal value corresponding to the target image, and a maximum value of the first dynamic range is determined based on a maximum signal value corresponding to the target image.

[0013] In one possible implementation, the target image includes at least one of an original uncompressed signal, a reconstructed signal, a residual signal, or a predicted signal;

[0014] The minimum signal value and the maximum signal value are determined based on at least one of an original uncompressed signal, a reconstructed signal, a residual signal, or a predicted signal.

[0015] In a possible implementation, the second dynamic range is selected from N preset dynamic ranges, wherein M preset dynamic ranges among the N preset dynamic ranges are maximum ranges, and NM preset dynamic ranges among the N preset dynamic ranges are smaller than the maximum range;

[0016] Wherein, M is a natural number less than 2, and N≥2.

[0017] In a possible implementation, the method further includes at least one of the following:

[0018] Using the target category as the adaptive loop filter category corresponding to the first image, where the first image is a sub-image in the target image;

[0019] Using the target category as the adaptive loop filter category corresponding to the second image, and the target image is a sub-image of the second image;

[0020] The target category is used as the adaptive loop filter category corresponding to the third image. The third image and the target image are located in the same image, and the third image and the target image have the same size.

[0021] In a possible implementation, the intermediate information includes at least one of the following:

[0022] a reconstructed signal before being filtered by a deblocking filter, wherein the reconstructed signal includes a reconstructed signal after being filtered by a fixed filter or a reconstructed signal after not being filtered by a fixed filter;

[0023] The signal to be processed is filtered by a fixed filter, and the signal to be processed includes a brightness signal.

[0024] In a possible implementation, the classification information includes a category subset; the target category is determined from L preset categories based on at least two subcategories, where L is the product of the number of categories corresponding to each classifier in the classifier group.

[0025] In a possible implementation, the subcategory is represented by a category index, and a subcategory is one category among multiple categories corresponding to a classifier;

[0026] Based on the classification information, the target category of the adaptive loop filter corresponding to the target image is determined, including:

[0027] Based on the individual category indices in the category subset, the category index classIdx of the target category is determined by the following expression:

[0028] classIdx=(n 1 *n 2 *…*n k-1 )*classIdx 0 +(n 2 *…*n k-1 )*classIdx 1 +…+classIdx k-1

[0029] Where k represents the number of classifiers included in the classifier group, n k-1 Indicates the number of categories corresponding to the k-th classifier, classIdx k-1 Represents the category index corresponding to the k-th classifier in the category subset.

[0030] In a possible implementation, applied to an encoding end, the method further includes:

[0031] determining whether to use an adaptive loop filter;

[0032] If it is determined to use an adaptive loop filter, a classification method is executed, and a bitstream corresponding to the target image is sent to a decoding end, where the bitstream includes decoding indication information, and the decoding indication information includes at least one of first indication information, second indication information, third indication information, fourth indication information, fifth indication information, or sixth indication information;

[0033] The first indication information indicates whether to use an adaptive loop filter;

[0034] The second indication information indicates the classification information used by the classification method;

[0035] The third indication information indicates a target level for executing the classification method, the target level includes at least one of an image sequence level, an image level, or a sub-image level, the image sequence level indicates that the classification method is executed in units of at least two frames of images, the image level indicates that the classification method is executed in units of one frame of image, the sub-image level indicates that the classification method is executed in units of sub-images in one frame of image, and the sub-image level includes at least one of a stripe level, a rectangular region level, or a block level;

[0036] The fourth indication information indicates a target sub-image in a frame of image for executing the classification method, where the target sub-image is at least one of the sub-images divided from the frame of image;

[0037] The fifth indication information indicates the size of the sub-image in a frame of image on which the classification method is executed;

[0038] The sixth indication information indicates the number of images in the image sequence for which the classification method is executed.

[0039] In a possible implementation, applied to a decoding end, the method further includes:

[0040] receiving a code stream, wherein the code stream includes decoding indication information;

[0041] If it is determined based on the decoding indication information that an adaptive loop filter is used, a classification method is performed;

[0042] The decoding indication information includes at least one of the first indication information, the second indication information, the third indication information, the fourth indication information, the fifth indication information or the sixth indication information;

[0043] The first indication information indicates whether to use an adaptive loop filter;

[0044] The second indication information indicates the classification information used by the classification method;

[0045] The third indication information indicates a target level for executing the classification method, the target level includes at least one of an image sequence level, an image level, or a sub-image level, the image sequence level indicates that the classification method is executed in units of at least two frames of images, the image level indicates that the classification method is executed in units of one frame of image, the sub-image level indicates that the classification method is executed in units of sub-images in one frame of image, and the sub-image level includes at least one of a stripe level, a rectangular region level, or a block level;

[0046] The fourth indication information indicates a target sub-image in a frame of image for executing the classification method, where the target sub-image is at least one of the sub-images divided from the frame of image;

[0047] The fifth indication information indicates the size of the sub-image in a frame of image on which the classification method is executed;

[0048] The sixth indication information indicates the number of images in the image sequence for which the classification method is executed.

[0049] On the other hand, an embodiment of the present application provides an adaptive loop filtering classification device in video encoding and decoding, including:

[0050] An information acquisition module, used to acquire classification information corresponding to a target image, the classification information including at least one of a target range, intermediate information, or a category subset; wherein the target image corresponds to an image sequence in a video, a frame of an image, a strip image in a frame of an image, a rectangular image, or a block image;

[0051] A classification module, used for determining a target category of an adaptive loop filter corresponding to a target image based on the classification information;

[0052] The target range includes at least one of a first dynamic range or a second dynamic range, the first dynamic range is a range determined based on pixel information of the target image, and the second dynamic range is a range determined based on a preset dynamic range, and at least one preset dynamic range is smaller than a maximum range that can be represented by a signal bit width of the target image;

[0053] Intermediate information includes information generated by the target image during the encoding and decoding process;

[0054] The category subset includes at least two subcategories. The at least two subcategories are categories of adaptive loop filters corresponding to the target image determined by respective classifiers in the classifier group. Each subcategory corresponds to a classifier.

[0055] On the other hand, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of any of the above methods.

[0056] On the other hand, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of any of the above methods are implemented.

[0057] On the other hand, an embodiment of the present application provides a computer program product, including a computer program, which implements the steps of any of the above methods when the computer program is executed by a processor.

[0058] In the related art, the adaptive loop filter classification in video coding and decoding generally determines the ALF category based on the maximum dynamic range that can be represented by the signal bit width of the video image; or the intermediate information of the video image in the coding and decoding process is not used when determining the ALF category, or the ALF category is determined based on a single classifier.

[0059] However, in the scheme of determining the ALF category based on the maximum dynamic range that can be represented by the signal bit width of the video image, the video image signal is generally distributed in a limited dynamic range, and determining the ALF category based on the maximum dynamic range will result in low accuracy in determining the ALF category, which in turn leads to the problem of low video quality. Secondly, a variety of different types of intermediate information will be generated during the encoding and decoding process, and these intermediate information can be used as a reference to determine the ALF category to a certain extent, but the related technology does not use the intermediate information, which will also lead to low accuracy in determining the ALF category, which will lead to the problem of low video quality. In addition, the ALF category corresponding to a single classifier is limited, which in turn leads to the determined ALF category being limited to a certain extent, which will also lead to low accuracy in determining the ALF category, which will lead to the problem of low video quality.

[0060] The beneficial effects of the technical solution provided in the embodiment of the present application are: by obtaining classification information corresponding to the target image, the classification information includes at least one of the target range, intermediate information or category subset; based on the classification information, the target category of the adaptive loop filter corresponding to the target image is determined, which can improve the accuracy of determining the ALF category and thereby improve the video quality.

[0061] For the target range, the target range includes at least one of the first dynamic range or the second dynamic range, and the first dynamic range is a range determined based on the pixel information of the target image, and the second dynamic range is a range determined based on the preset dynamic range, and at least one preset dynamic range is smaller than the maximum range that can be represented by the signal bit width of the target image; that is, the target range used to determine the adaptive loop filter category of the target image may be smaller than the maximum range, so the target range is more matched with the target image relative to the maximum range, thereby improving the accuracy of determining the ALF category, thereby improving the video quality. In addition, for the intermediate information, the intermediate information includes the information generated by the target image during the encoding and decoding process, that is, the information used to determine the adaptive loop filter category also includes the information generated by the target image during the encoding and decoding process, so more valuable information can be used to determine the category of the adaptive loop filter, thereby improving the accuracy of determining the category of the ALF, thereby improving the video quality. In addition, for the category subset, the category subset includes at least two subcategories, and the at least two subcategories are the categories of the adaptive loop filter corresponding to the target image determined by each classifier in the classifier group, and each subcategory corresponds to a classifier. That is to say, the category results output by at least two classifiers in the classifier group can be used to comprehensively determine the category of the adaptive loop filter corresponding to the target image, thereby improving the accuracy of determining the ALF category and thereby improving the video quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0063] Figure 1 A schematic diagram of a system architecture for implementing encoding and decoding provided in an embodiment of the present application;

[0064] Figure 2 A schematic diagram of a flow chart of an adaptive loop filtering classification method in video encoding and decoding provided in an embodiment of the present application;

[0065] Figure 3 A schematic diagram of filtering an image provided in an embodiment of the present application;

[0066] Figure 4 A schematic diagram of a different type of diamond filter provided in an embodiment of the present application;

[0067] Figure 5A schematic diagram of tap coefficients corresponding to a plurality of samples of an input filter provided in an embodiment of the present application;

[0068] Figure 6 A schematic diagram of comparison between a target image and a first image provided in an embodiment of the present application;

[0069] Figure 7 A schematic diagram of comparison between a target image and a second image provided in an embodiment of the present application;

[0070] Figure 8 A schematic diagram of comparison between a target image and a third image provided in an embodiment of the present application;

[0071] Fig. 9 A schematic diagram of a flow chart of another adaptive loop filtering classification method in video encoding and decoding provided in an embodiment of the present application;

[0072] Fig.10 A schematic diagram of the structure of an adaptive loop filtering classification device provided in an embodiment of the present application;

[0073] Fig.11 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0074] The embodiments of the present application are described below in conjunction with the drawings in the present application. It should be understood that the implementation methods described below in conjunction with the drawings are exemplary descriptions for explaining the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions of the embodiments of the present application.

[0075] It will be understood by those skilled in the art that, unless specifically stated, the singular forms "one", "an", and "the" used herein may also include plural forms. It will be further understood that the terms "including" and "comprising" used in the embodiments of the present application refer to that the corresponding features can be implemented as the presented features, information, data, steps, operations, components and / or components, but do not exclude the implementation as other features, information, data, steps, operations, components, components and / or combinations thereof supported by the technical field. It will be understood that when we refer to a component as being "connected" or "coupled" to another component, the component may be directly connected or coupled to the other component, or may refer to the connection relationship between the component and the other component through an intermediate component. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein indicates at least one of the items defined by the term, for example, "A and / or B" indicates implementation as "A", or implementation as "A", or implementation as "A and B". "Multiple" refers to "more than two".

[0076] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0077] The following describes several exemplary embodiments to illustrate the technical solutions of the embodiments of the present application and the technical effects produced by the technical solutions of the present application. It should be noted that the following embodiments can refer to, draw on or combine with each other, and the same terms, similar features and similar implementation steps in different embodiments will not be described repeatedly.

[0078] First, several terms involved in the embodiments of the present application are introduced and explained:

[0079] ALF: Adaptive Loop Filtering

[0080] APS: Adaptation Parameter Set

[0081] BIF: Bilateral filter, bilateral filter

[0082] CC-ALF: Cross-Component Adaptive Loop Filtering, cross-component adaptive loop filtering

[0083] CCSAO: Cross-Component Sample Adaptive Offset, cross-component sample adaptive compensation

[0084] CTB: Coding Tree Block, coding tree block

[0085] CTU: Coding Tree Unit, coding tree unit

[0086] DF: Deblocking Filter, deblocking filter

[0087] ECM: Enhanced Compression Model

[0088] PPS: Picture Parameter Set, image parameter set

[0089] RDO: Rate-Distortion Optimization

[0090] SAO: Sample Adaptive Offset, sample adaptive compensation

[0091] SPS: Sequence Parameter Set, sequence parameter set

[0092] VTM: VVC test model, general video coding test model

[0093] VVC: Versatile Video Coding

[0094] Secondly, the system framework of the embodiment of the present application is illustrated.

[0095] See also Figure 1 , Figure 1 A schematic diagram of a system architecture for implementing encoding and decoding provided in an embodiment of the present application. Figure 1 As shown, system 10 includes an encoding end 12, which can provide encoded video data to a decoding end 14, and the decoding end 14 can decode the encoded video data. The encoding end 12 and the decoding end 14 can include any of a variety of devices, including a desktop computer, a notebook computer, a tablet computer, a set-top box, an intelligent terminal (such as a so-called "smart" phone), a television, a camera, a display device, a digital media player, a video game console, or at least one of a video streaming device. In some cases, the encoding end 12 and the decoding end 14 are equipped for wireless communication. Therefore, the encoding end 12 and the decoding end 14 can be wireless communication devices. The technology described in the embodiments of the present disclosure can be applied to wireless and / or wired applications. The encoding end 12 is an exemplary video encoding device, that is, a device for encoding video data. The decoding end 14 is an exemplary video decoding device, that is, a device for decoding video data.

[0096] Figure 1 The system 10 shown is only an example. The technology for encoding, decoding and processing video data can be performed by any digital video encoding and / or decoding device. In some examples, the technology can be performed by a video encoder / decoder, which is generally referred to as a "codec (CODEC)". The encoding end 12 and the decoding end 14 are examples of such codec devices, wherein the encoding end 12 generates encoded video data to send to the decoding end 14. In some examples, the encoding end 12 and the decoding end 14 operate in a substantially symmetrical manner, so that each of the encoding end 12 and the decoding end 14 includes a video encoding and decoding component. Therefore, the system 10 can support one-way or two-way video transmission between the encoding end 12 and the decoding end 14, such as for video streaming, video playback, video broadcasting or video telephony.

[0097] exist Figure 1In the example of , the encoding end 12 includes a video source 121, a video encoder 122, and an output interface 123. The decoding end 14 includes an input interface 141, a video decoder 142, and a display device 143. In other examples, the encoding end 12 and the decoding end 14 include other components or arrangements. For example, the encoding end 12 can receive video data from an external video source (such as an external camera). Similarly, the decoding end 14 can be connected to an external display device instead of including an integrated display device.

[0098] Video source 121 is a video data source. Video data may include a series of pictures. Video source 121 may include a video capture device (such as a camera), a video archive containing previously captured video and / or a video feed interface for receiving video data from a video content provider. In some examples, video source 121 generates video data based on computer graphics, or a combination of live video, archived video and computer-generated video. In each case, the captured, pre-captured or computer-generated video may be encoded by video encoder 122.

[0099] In some examples, the video encoder 122 and the video decoder 142 encode and decode the video data according to one or more video codec standards or specifications. For example, the video encoder 122 and the video decoder 142 can encode and decode the video data according to ITU-T H.261, ISO / IEC MPEG-1 Visual, ITU-T H.262 or ISO / IEC MPEG-2 Visual, ITU-T H.263, ISO / IEC MPEG-4 Visual and ITU-T H.264 (also known as ISO / IEC MPEG-4 AVC) (including its scalable video codec (SVC) and multi-view video codec (MVC) extensions or another video codec standard or specification). In some examples, the video encoder 122 and the video decoder 142 encode and decode the video data according to the High Efficiency Video Codec (HEVC) standard (referred to as ITU-T H.265, its range and screen content codec extension, its 3D video codec extension (3D-HEVC), its multi-view extension (MV-HEVC) or its scalable extension (SHVC)).

[0100] The technical solution of the embodiments of the present application can be used in advanced video codecs (such as extensions of HEVC or next-generation video codec standards (for example, H.266 / VVC)).

[0101] In the field of video coding and decoding, filtering (e.g., ALF) is usually applied to enhance the quality of the reconstructed and / or decoded video signal. In an embodiment of the present application, the reconstructed video data block may refer to a video data block reconstructed in the reconstruction loop of the video encoder 122, or may refer to a video data block decoded by the video decoder 142. In some examples, the filter may be applied as a post filter, wherein the filtered frame is not used to predict future frames, or the filter may be applied as a loop filter, wherein the filtered frame is used to predict future frames. For example, the filter may be designed by minimizing the error between the original signal and the reconstructed / decoded filtered signal. Similar to transform coefficients.

[0102] In the related art, the adaptive loop filter classification in video coding and decoding generally determines the ALF category based on the maximum dynamic range that can be represented by the signal bit width of the video image; or the intermediate information of the video image in the coding and decoding process is not used when determining the ALF category, or the ALF category is determined based on a single classifier.

[0103] However, the result of adaptive loop filtering classification has a certain degree of influence on the video quality. Therefore, in order to further improve the video quality, we can start from improving the accuracy of adaptive loop filtering classification.

[0104] See also Figure 2 , Figure 2 The following is a flow chart of an adaptive loop filter classification method in a video codec provided in an embodiment of the present application. The classification method of this embodiment can be applied to an electronic device. The electronic device can be used as an encoding end, or as a decoding end, or as an encoding end and a decoding end respectively, which is not limited here. Figure 2 The classification methods shown include:

[0105] S210: Obtain classification information corresponding to the target image, where the classification information includes at least one of a target range, intermediate information, or a category subset.

[0106] Among them, the target image corresponds to an image sequence, a frame image, a strip image, a rectangular image or a block image in a frame image in a video. In this embodiment, the image sequence includes at least two frames of images. Optionally, the image sequence may include at least two consecutive frames of images. In this embodiment, the classification method may be performed with the target image as a unit. Specifically, if the target image of this embodiment corresponds to the image sequence, it can be understood that the adaptive loop filtering classification is performed with the image sequence as a unit, and the target category obtained is also the target category corresponding to the image sequence. Similarly, if the target image of this embodiment corresponds to a frame image, it can be understood that the adaptive loop filtering classification is performed with the frame image as a unit, and the target category obtained is also the target category corresponding to the frame image. In this embodiment, according to needs, the classification method can be selected to perform on the image sequence, a frame image, a strip image, a rectangular image or a block image in a video, so as to obtain the target category corresponding to the image sequence, a frame image, a strip image, a rectangular image or a block image in a video.

[0107] The target range includes at least one of a first dynamic range or a second dynamic range. The first dynamic range is a range determined based on pixel information of the target image. The second dynamic range is a range determined based on a preset dynamic range, and at least one preset dynamic range is smaller than the maximum range that can be represented by the signal bit width of the target image.

[0108] In this embodiment, the range determined based on the pixel information of the target image may be less than or equal to the maximum range that can be represented by the signal bit width of the target image, which is determined according to the actual calculation results and is not limited here. The target range can be understood as a subset of the maximum range. Specifically, the maximum range can be 0-(2 信号位宽 -1). For example, taking the signal bit width as 10 bits as an example, the maximum range may be 0-1023. For another example, taking the signal bit width as 8 bits as an example, the maximum range may be 0-511. Specifically, the second dynamic range may be a partial or full range of the total range that can be formed by all preset dynamic ranges.

[0109] For the classification information including a target range, the target range includes at least one of a first dynamic range or a second dynamic range, and the first dynamic range is a range determined based on pixel information of the target image, and the second dynamic range is a range determined based on a preset dynamic range, and at least one preset dynamic range is smaller than the maximum range that can be represented by the signal bit width of the target image; that is, the target range used to determine the adaptive loop filter category of the target image may be smaller than the maximum range, so the target range is more matched with the target image relative to the maximum range, thereby improving the accuracy of determining the ALF category, and then improving the video quality.

[0110] The intermediate information includes information generated by the target image during the encoding and decoding process.

[0111] In this embodiment, the target image will generate some intermediate information during the encoding and decoding process. The intermediate information has certain value for determining the target category. Therefore, the intermediate information can be used to determine the target category of the adaptive loop filter corresponding to the target image.

[0112] Specifically, the intermediate information may be information that is not used in the current solution for determining the adaptive loop filter.

[0113] It is understandable that, with the iteration of technology, the information used to determine the adaptive loop filter will change accordingly. This embodiment does not limit the specific intermediate information.

[0114] For the classification information including the intermediate information, the intermediate information includes the information generated by the target image in the encoding and decoding process, that is, the information used to determine the category of the adaptive loop filter also includes the information generated by the target image in the encoding and decoding process. Then, more valuable information can be used to determine the category of the adaptive loop filter, thereby improving the accuracy of determining the category of the ALF, thereby improving the video quality.

[0115] The category subset includes at least two subcategories, and the at least two subcategories are categories of adaptive loop filters corresponding to the target image determined by respective classifiers in the classifier group, and each subcategory corresponds to a classifier.

[0116] In this embodiment, the classifier group includes at least two different classifiers, and each classifier in the classifier group can determine the subcategory corresponding to the target image from the corresponding multiple categories. Therefore, when the target image is classified using the classifier group, the subcategory corresponding to each classifier in the classifier group can be obtained, and then the target category can be determined based on the subcategory corresponding to each classifier.

[0117] For the classification information including a category subset, the category subset includes at least two subcategories, and the at least two subcategories are categories of the adaptive loop filter corresponding to the target image determined by each classifier in the classifier group, and each subcategory corresponds to a classifier. That is to say, the category results output by at least two classifiers in the classifier group can be used to comprehensively determine the category of the adaptive loop filter corresponding to the target image, thereby improving the accuracy of determining the category of the ALF, thereby improving the video quality.

[0118] S220: Determine a target category of the adaptive loop filter corresponding to the target image based on the classification information.

[0119] In this embodiment, the classification information includes at least one of a target range, intermediate information, or a category subset, that is, this embodiment can determine the target category of the adaptive loop filter corresponding to the target image based on at least one of the target range, intermediate information, or a category subset.

[0120] The technical solution of this embodiment obtains classification information corresponding to the target image, and determines the target category of the adaptive loop filter corresponding to the target image based on the classification information, and the classification information includes at least one of a target range, intermediate information or a category subset, and the target range includes at least one of a first dynamic range or a second dynamic range, the first dynamic range is a range determined based on pixel information of the target image, and the second dynamic range is a range determined based on a preset dynamic range, and at least one preset dynamic range is smaller than the maximum range that can be represented by the signal bit width of the target image; the intermediate information includes information generated by the target image during the encoding and decoding process; the category subset includes at least two subcategories, and the at least two subcategories are categories of the adaptive loop filter corresponding to the target image respectively determined by each classifier in the classifier group, and each subcategory corresponds to a classifier, which can improve the accuracy of determining the category of the ALF, thereby improving the video quality.

[0121] The following embodiments are based on any of the above embodiments and take VCC as an example to further illustrate how to classify and filter in video encoding and decoding in the related art.

[0122] As a newly adopted loop filter in VVC, ALF and CCALF are a kind of Wiener filter, which adaptively determines the filter coefficient according to different video contents, thereby reducing the mean square error (MSE) between the reconstructed component and the original component. Among them, the input of ALF is the reconstructed pixel value before ALF processing, and the output is the enhanced reconstructed luminance image and reconstructed chrominance image. As an adaptive filter, the Wiener filter can generate different filter coefficients for video content with different characteristics. Therefore, ALF needs to classify the video content first and use the corresponding filter for each category of video content. In the design of VVC, each 4x4 block is divided into one of 25 categories according to its own directionality and activity. The corresponding filter coefficient is calculated for each type of video content.

[0123] For the luminance component, in addition to 4x4 block-level adaptation, VVC also supports CTU-level ALF adaptive switching. Each CTU can use the filter group generated by the current strip image, or the filter group generated by the coded slice or a set of fixed filters from the 16 offline trained fixed filter groups. Inside the CTU, each 4x4 block selects a filter of the corresponding category from the filter group for filtering according to its own category. The filter coefficients and the corresponding clipping index are transmitted to the decoding end by ALF_APS. An ALF_APS can contain a luminance filter group (including up to 25 filters) and up to 8 chrominance filters.

[0124] CCALF uses the luminance component to correct the chrominance component. The processing flow of ALF and CCALF is as follows: Figure 3 As shown in the figure. CCALF uses the luminance component as input and outputs the correction value of the chrominance component. The two chrominance components can independently control whether to use the corresponding correction value. The correction value and the output of the chrominance ALF together constitute the final chrominance component.

[0125] In VVC, ALF uses Figure 4 Different types of diamond filters are shown. The luminance component uses a 7x7 diamond filter, and the chrominance component uses a 5x5 diamond filter.

[0126] For the luminance component, ALF will adaptively use different filters at the sub-block level (4x4), that is, each 4x4 pixel block needs to be divided into one of 25 categories. For the chrominance component, ALF does not need to classify the pixels at the sub-block level, and all chrominance pixels in a CTU use the same filter. The classification index C of the luminance component pixel block is composed of the directional feature (Directionality) D of the block and the quantized activity feature (Activity) The formula is as follows:

[0127]

[0128] To calculate D and First, you need to calculate the horizontal, vertical, diagonal, and anti-diagonal gradient values ​​of each pixel in the 4x4 pixel block:

[0129] H k,l =|2R(k,l)-R(k-1,l)-R(k+1,l)| (2-2)

[0130] V k,l =|2R(k,l)-R(k,l-1)-R(k,l+1)| (2-3)

[0131] D0 k,l=|2R(k,l)-R(k-1,l-1)-R(k+1,l+1)| (2-4)

[0132] D1 k,l =|2R(k,l)-R(k-1,l+1)-R(k+1,l-1)| (2-5)

[0133] Based on the pixel gradient, the horizontal, vertical, diagonal and anti-diagonal gradients of each 4x4 block are calculated as follows:

[0134]

[0135]

[0136] Among them, i and j represent the coordinates of the upper left corner pixel of the 4x4 pixel block, and R(k,l) represents the reconstructed pixel value at the (k,l) position before ALF filtering.

[0137] After obtaining the gradient value of the pixel block, the maximum and minimum values ​​of the horizontal and vertical gradient values ​​are:

[0138]

[0139] The maximum and minimum values ​​of the diagonal and anti-diagonal gradient values ​​are:

[0140]

[0141] The directional feature D is derived by comparing the maximum and minimum values ​​of the gradient values ​​in the four directions obtained by equations (2-8)-(2-9):

[0142] Step 1: If If both are true, then D is set to 0.

[0143] Step 2: If Then go to Step 3, otherwise go to Step 4.

[0144] Step 3: If Then D is set to 2, otherwise D is set to 1.

[0145] Step 4: If Then D is set to 4, otherwise D is set to 3.

[0146] The activity characteristic A is calculated by the following formula:

[0147]

[0148] Activity feature A will be quantized to the range of [0-4] as the quantized activity feature

[0149] Before filtering each 4x4 luminance block, the filter coefficients and corresponding limit values ​​are geometrically transformed according to the gradient value of the current block according to the rules in Table 1-1, including no transformation, diagonal transformation, vertical flip, and rotation transformation. Applying geometric transformation to the filter coefficients is equivalent to applying geometric transformation to the pixel values ​​and then filtering without changing the coefficients. The purpose of geometric transformation is to align the directionality of the content of different blocks as much as possible, thereby reducing the number of classifications required for ALF and allowing different pixels to share the same filter coefficients. Using geometric transformation can increase the actual classification from 25 to 100 without increasing the number of ALF filters, thereby improving its adaptability.

[0150] Table 1-1 Geometric transformation based on pixel block gradient values

[0151] Gradient Value Several transformations <![CDATA[g d1 <g d0 And g h <g v ]]> constant <![CDATA[g d1 <g d0 And g v ≤g h ]]> Diagonal Transformation <![CDATA[g d0 ≤g d1 And g h <g v ]]> Flip vertically <![CDATA[g d0 ≤g d1 And g v ≤g h ]]> Rotation Transformation

[0152] For the ALF filtering process of VCC, at the decoding end, if the ALF flag at the CTU level is true, each pixel R(i,j) in the current CTU will be filtered. The filtering process and output are as follows:

[0153]

[0154] Where f(k,l) represents the filter tap coefficient, K(x,y) is the limit function, and c(k,l) is the parameter related to the limit operation. The range of k and l is to Where L is the length of the filter. The specific definition of the clipping function is K(x,y)=min(y,max(-y,x)). The clipping operation adds a nonlinear function to the ALF, which can reduce the impact of pixels with large surrounding differences on the current pixel.

[0155] In ECM-8.0, the downsampling operation and virtual boundary restrictions when calculating gradients during ALF classification are removed. At the same time, the basic unit of ALF classification operation is changed from 4x4 sub-blocks to 2x2 sub-blocks. The shapes of the luminance and chrominance filters are also changed accordingly.

[0156] For the brightness component, ECM-8.0 uses three different classifiers (C 0 , C 1 and C 2 ) and three different sets of filters (F 0 , F 1 and F 2 ). The filter bank F 0 and F1 Contains fixed filters whose coefficients are based on the classifier C 0 With C 1 Offline training generation. 2 Contains the filter coefficients generated by the content to be encoded, which need to be written into the bitstream and transmitted to the decoding end.

[0157] In ECM-8.0, each 2x2 sub-block is divided into two groups according to its directional feature D i With activity characteristics A i Generate the corresponding category index C i , as shown below:

[0158]

[0159] Among them, i represents the classifier index, M D,i Represents the activity feature D used by the corresponding classifier i The total number of .

[0160] Similar to the calculation process in VVC, the horizontal, vertical, diagonal, and anti-diagonal gradients of each pixel are generated using the 1-D Laplacian operator. 0 , the gradient of its sub-block is generated by adding the pixel gradient values ​​of all positions in the 4x4 area covering the target 2x2 sub-block. 1 With C 2 , the sub-block gradient is generated by adding the pixel gradient values ​​of all positions in the 12x12 area covering the target 2x2 sub-block. The horizontal, vertical, diagonal, and anti-diagonal sub-block gradients are specified as The directional characteristic D i Obtained by comparing the following two values ​​to a set of thresholds:

[0161]

[0162] Directional characteristics D 2 Use the same thresholds of 2 and 4.5 as for VVC. 0 and D 1 , first calculate the horizontal / vertical edge strength Diagonal edge strength Use threshold Th = [1.25, 1.5, 2, 3, 4.5, 8]. When the edge strength Set to 0; otherwise, is satisfied The maximum integer of . When the edge strength Set to 0; otherwise To satisfy The maximum integer of . That is, when the horizontal / vertical edge is strong, the directional feature D i Generated by Table 1-2(a). Otherwise, the directional feature D i Generated from Table 1-2(b).

[0163] Table 1-2 Edge strength With D i The mapping relationship

[0164]

[0165] Activity characteristics It is the result of accumulating the horizontal and vertical gradients of the sub-block level A i The quantized value range is 0 to n. n is set to 4; for and n is set to 15. One ALF_APS can transmit up to 4 sets of luminance component filters, each filter set contains up to 25 filters.

[0166] In ECM-8.0, the ALF classification process uses a new classifier. For a set of filters transmitted to the decoder, a flag is used to indicate whether to use the original classifier or the new classifier. The new classifier cannot use geometric transformations. When the new classifier is used, the pixels at all positions in a 2x2 sub-block are added and then classified based on the sum of the pixel values.

[0167] class index =(sum×25)>>(sample bitdepth+2) (2-13)

[0168] When filtering, the ALF first uses two fixed filters F using a 13x13 diamond filter. 0 With F 1 Two intermediate values ​​R will be generated for the current pixel to be filtered 0 (x,y) and R 1 (x,y). Then, the filter F generated online 2 Will be applied in R 0 (x,y), R 1 (x, y) is combined with the surrounding pixels to generate the filtered pixel, as shown in the following formula:

[0169]

[0170] Among them, f i,j Represents the difference between the surrounding pixels after the clipping operation and the current pixel R(x,y), g i Represents R after limiting operation i-20The difference between (x,y) and the current pixel. Filter coefficient c i , i=0,…21, need to be transmitted to the decoding end.

[0171] In ECM-8.0, the luminance filter generated by online training contains four types of inputs: spatially adjacent samples, reconstructed samples before deblocking, extended samples generated by fixed filters, and residual components. Figure 5 As shown, the tap coefficients corresponding to various samples include spatial tap coefficients, fixed filter output based tap coefficients, tap coefficients of reconstructed samples before deblocking (recbefore DB based taps), residual component tap coefficients (prediction based taps), and residual component tap coefficients filtered by fixed filter (prediction filtered by fixed filter based taps). Among them, #0-#19 represent spatial adjacent samples, #20-#25, #28-#29 represent samples generated by fixed filter filtering, #26, #27, #30 represent reconstructed samples before deblocking, and #31, #32 represent residual components. According to various types of input, the filtering process is as follows:

[0172]

[0173] Among them, f i,j Represents the difference between the limited spatial surrounding pixel samples and the current sample R(x,y), g i Represents the difference between the pixel sample generated by filtering with a fixed filter and the current sample R(x,y), h i,j Represents the difference between the reconstructed sample before the deblocking effect after clipping and the current sample R(x,y). i Represents the residual component after limiting, rFiltered i Represents the residual component generated by filtering with a fixed filter after clipping. The residual component and the reconstructed component use the same fixed filter.

[0174] In ALF_APS, a flag is used to indicate whether to use only the residual component or the residual component and the residual component filtered by the fixed filter at the same time. In JVET-AD0219, a new classifier based on the luminance residual component is proposed as the third classifier of ALF. For any 2x2 luminance sub-block, the sum of the absolute values ​​of the residual components of all positions in the 8x8 area covering the current 2x2 sub-block is first calculated, and then the following formula is used for classification based on the cumulative sum:

[0175] classIdx=sum>>(sample bit depth-4) (2-16)

[0176] The final class classIdx has a value range of 0 to 24. In ALF_APS, a classifier is required for each filter bank transmission.

[0177] In a possible implementation, the intermediate information includes at least one of the following:

[0178] a reconstructed signal before being filtered by a deblocking filter, wherein the reconstructed signal includes a reconstructed signal after being filtered by a fixed filter or a reconstructed signal after not being filtered by a fixed filter;

[0179] The signal to be processed is filtered by a fixed filter, and the signal to be processed includes a brightness signal.

[0180] The deblocking filter and the fixed filter may refer to the description of the above embodiments, which will not be elaborated here.

[0181] Optionally, the reconstructed signal before being filtered by the deblocking filter may include a reconstructed signal that has not been filtered by a fixed filter and has not been filtered by the deblocking filter, and may also include a reconstructed component before being filtered by the fixed filter and having not been filtered by the deblocking filter.

[0182] The following embodiments further illustrate how to determine the first dynamic range based on pixel information of the target image based on any of the above embodiments.

[0183] In a possible implementation manner, determining the first dynamic range based on pixel information of the target image includes:

[0184] Determine the minimum signal value and the maximum signal value corresponding to the image based on the pixel information of the target image;

[0185] A range minimum value of the first dynamic range is determined based on the minimum signal value, and a range maximum value of the first dynamic range is determined based on the maximum signal value.

[0186] In this embodiment, the pixel information may refer to the pixel information of each pixel in the target image. The target image includes at least one of the original uncompressed signal, the reconstructed signal, the residual signal or the predicted signal, and it can also be understood that the pixel information includes at least one of the original uncompressed signal, the reconstructed signal, the residual signal or the predicted signal.

[0187] The original uncompressed signal refers to the signal of the target image without compression coding. The reconstructed signal can be obtained by reconstructing the signal obtained after encoding and decoding the original uncompressed signal. The residual signal can refer to the signal obtained by processing the original uncompressed signal using the residual coding and decoding technology. The predicted signal can refer to the signal obtained by processing the original uncompressed signal using the predictive coding and decoding technology.

[0188] In this embodiment, the minimum signal value and the maximum signal value are determined based on at least one of an original uncompressed signal, a reconstructed signal, a residual signal, or a predicted signal.

[0189] Specifically, taking the case where the target image includes one signal as an example, the minimum signal value and the maximum signal value in the target image can be determined for the signal value corresponding to each pixel in the target image. Taking the case where the target image includes at least two signals as an example, the signal value corresponding to each pixel can be calculated after normalizing the at least two signals, thereby determining the first dynamic range.

[0190] Optionally, the range minimum value is not less than the minimum signal value, and the range maximum value is not greater than the maximum signal value.

[0191] It can be understood that in this embodiment, illustratively, if the target image corresponds to an image sequence, what is determined is the signal value corresponding to each pixel in the image sequence, and then the minimum signal value and the maximum signal value in the image sequence are determined to determine the target range corresponding to the image sequence. Similarly, if the target image corresponds to a frame of image, what is determined is the minimum signal value and the maximum signal value in the frame of image. Similarly, strip images, rectangular images, or block images can all refer to the description of image sequences and a frame of image, and will not be repeated here.

[0192] The technical solution of this embodiment determines the minimum signal value and the maximum signal value corresponding to the image based on the pixel information of the target image, and determines the minimum value of the first dynamic range based on the minimum signal value and determines the maximum value of the first dynamic range based on the maximum signal value. Therefore, it is possible to perform classification in combination with the actual image content, thereby improving the accuracy of the determined filter category.

[0193] The following embodiments illustrate the second dynamic range based on any of the above embodiments.

[0194] In a possible implementation, the number of preset dynamic ranges is N, where N is an integer not less than 1, that is, the number of preset dynamic ranges can be one, can be more than two, can be set as needed, and is not limited here. Specifically, when N≥2, any two of the N preset dynamic ranges can have different values. Optionally, when N≥2, M preset dynamic ranges can be the maximum range, NM preset dynamic ranges can be less than the minimum range, and M can be a natural number less than 2. Optionally, if N≥3, any two of the NM preset dynamic ranges do not have the same value.

[0195] It should be noted that if N is 1, that is, when there is one preset dynamic range, the one preset dynamic range is smaller than the maximum range. If N ≥ 2, that is, at least one preset dynamic range among the N or more preset dynamic ranges is smaller than the maximum range, that is, some or all of the N or more preset dynamic ranges are smaller than the maximum range.

[0196] In a possible implementation manner, if the preset dynamic range is one, the preset dynamic range may be used as the second preset dynamic range.

[0197] In this embodiment, by presetting a preset dynamic range, the target category can be directly determined based on the preset dynamic range, which can improve the efficiency of determining the target category and further improve the encoding efficiency.

[0198] In another possible implementation, optionally, if N≥2, that is, when the preset dynamic range is more than two, the second dynamic range can be selected from N preset dynamic ranges, wherein M preset dynamic ranges among the N preset dynamic ranges are the maximum range, and NM preset dynamic ranges among the N preset dynamic ranges are smaller than the maximum range; wherein M is a natural number less than 2.

[0199] Exemplarily, the N preset dynamic ranges are as follows:

[0200] [l 0 ,h 0 ],[l 1 ,h 1 ],…,[l n-1 ,h n-1 ], each interval is different from each other, and at most one dynamic range is the maximum dynamic range that can be represented by the signal bit width, and the other dynamic ranges are sub-intervals of the maximum dynamic range that can be represented by the signal bit width, then one of the N preset dynamic ranges can be selected as the second dynamic range, that is, one of the following N preset dynamic ranges:

[0201] [l 0 ,h 0],[l 1 ,h 1 ],…,[l n-1 ,h n-1 ].

[0202] In this embodiment, by presetting at least two preset dynamic ranges, different preset dynamic ranges can be selected according to different situations, thereby improving the flexibility and accuracy of the selected preset dynamic range, thereby improving the accuracy of determining the target category, and thus improving the video quality.

[0203] The following embodiments further illustrate the situation where the target range includes at least one of the first dynamic range or the second dynamic range based on any of the above embodiments.

[0204] In a possible implementation, when the target range includes a first dynamic range and a second dynamic range, the target range may be a union of the first dynamic range and the second dynamic range, or an intersection of the first dynamic range and the second dynamic range. The selection may be based on actual needs and is not limited herein.

[0205] It can be understood that, by combining the first dynamic range determined by the pixel information and the second dynamic range determined based on the preset dynamic range, the determined target range takes into account more dimensions, and the target range is more accurate, further improving the accuracy of ALF classification.

[0206] The following embodiments, based on any of the above embodiments, further explain how to determine the target category based on the target range, how to determine the target category based on the intermediate information, how to determine the target category based on the category subset, and how to determine the target category based on the target range, the intermediate information and the category subset.

[0207] First, further explanation is given on determining target categories based on target range.

[0208] In a possible implementation, the target category may be determined by referring to the following formula 2-17.

[0209]

[0210] Among them, input represents the input information used for classification, range min ,range max They represent the minimum and maximum values ​​of the true dynamic range respectively, and total_number represents the total number of categories of the corresponding classifier.

[0211] Optionally, the input information may be determined based on the signal value of each pixel in the target image, for example, the sum of the signal values ​​of each pixel in the target image.

[0212] Secondly, the determination of the target category based on the intermediate information is further explained.

[0213] In a possible implementation, the target category may be determined by referring to the above formula 2-17.

[0214] In this embodiment, the input information used for classification includes intermediate information. min ,range max They may represent the minimum value and the maximum value of the real dynamic range, or they may represent the minimum value and the maximum value of the maximum range, which is not limited here.

[0215] In addition, further description is given by determining the target category based on the category subset.

[0216] In a possible implementation, referring to the above formulas 2-17, the subcategories corresponding to each classifier in the classifier group are determined respectively, and then the target category is determined from L preset categories based on at least two subcategories.

[0217] Wherein, L is the product of the number of categories corresponding to each classifier in the classifier group. In this embodiment, after combining at least two classifiers, the number of categories corresponding to the classifier group increases exponentially. Compared with a single classifier, this embodiment can select a target category from L more detailed preset categories, thereby improving the accuracy of the determined target category and improving the video quality.

[0218] For example, assuming that there are currently a first classifier and a second classifier, the number of categories corresponding to the first classifier is a, and the number corresponding to the second classifier is b, then in the related art, the ALF classification can be determined by using the first classifier or the second classifier, that is, the determined ALF classification is one of a+b. In this embodiment, through the classifier group, the target classification can be selected from a*b=L preset classifications.

[0219] Specifically, each of the L preset categories in this embodiment is related to one of the categories corresponding to each classifier in the classifier group, and the number of preset categories corresponding to the classifier group obtained by the permutation and combination is the product of the number of categories corresponding to each classifier in the classifier group.

[0220] Optionally, the target filter coefficient corresponding to each preset classification in the L preset classifications may be related to the filter coefficient corresponding to one of the categories of each classifier. For example, the target filter coefficient corresponding to the fifth preset classification in the L preset classifications is respectively related to the filter coefficient corresponding to the first preset category in the first classifier and the filter coefficient corresponding to the second preset category in the second classifier. Exemplarily, it may be the mean of the filter coefficient corresponding to the first preset category and the filter coefficient corresponding to the second preset category in the second classifier, which is not limited here.

[0221] In a possible implementation, the subcategory is represented by a category index, and a subcategory is one category among multiple categories corresponding to a classifier;

[0222] Based on the classification information, the target category of the adaptive loop filter corresponding to the target image is determined, including:

[0223] Based on the individual category indices in the category subset, the category index classIdx of the target category is determined by the following expression:

[0224] classIdx=(n 1 *n 2 *…*n k-1 )*classIdx 0 +(n 2 *…*n k-1 )*classIdx 1

[0225] +…+classIdx k-1 (2-18)

[0226] Where k represents the number of classifiers included in the classifier group, n k-1 Indicates the number of categories corresponding to the k-th classifier, classIdx k-1 Represents the category index corresponding to the k-th classifier in the category subset.

[0227] In this embodiment, the category indexes in the L preset categories can be defined in the following manner:

[0228] You can first select one of the k classifiers as the current classifier, and keep the category index corresponding to each of the remaining k-1 classifiers the same, and then define the category index in the L preset categories according to the change of the category index corresponding to the current classifier. Then, replace the current classifier, and continue to define the category index in the L preset categories until any of the k classifiers is polled as the current classifier.

[0229] For example, take k classifiers including the first classifier and the second classifier as an example, assuming that the first classifier corresponds to 3 categories and the second classifier corresponds to 2 categories, then the classifier group corresponds to 6 categories. At this time, the category index definition of the 6 categories can be as follows:

[0230] The 1st subclass index of the first classifier and the 1st subclass index of the second classifier define the 1st class index of the classifier group, the 1st subclass index of the first classifier and the 2nd subclass index of the second classifier define the 2nd class index of the classifier group, the 2nd subclass index of the first classifier and the 1st subclass index of the second classifier define the 3rd class index of the classifier group, the 2nd subclass index of the first classifier and the 2nd subclass index of the second classifier define the 4th class index of the classifier group, the 3rd class index of the first classifier and the 1st subclass index of the second classifier define the 5th class index of the classifier group, and the 3rd class index of the first classifier and the 2nd subclass index of the second classifier define the 6th class index of the classifier group.

[0231] At this point, the final category index of the combined classifier can be:

[0232] classIdx=25*classIdx 0 +classIdx 1

[0233] Among them, x 0 and x 1 They respectively represent the subcategory index corresponding to the first classifier and the subcategory index corresponding to the second classifier.

[0234] The technical solution of this embodiment can quickly determine the category index of the target category in L preset categories through the category index corresponding to each classifier in the k classifiers, which can improve the efficiency of determining the ALF category and further improve the compression efficiency.

[0235] In addition, it is further explained by determining the target category based on the target range, intermediate information and category subsets.

[0236] In a possible implementation, for each classifier in the classifier group, the subcategory corresponding to each classifier can be determined by the above formula, and then the subcategory corresponding to each classifier is used to determine the target category.

[0237] Specifically, assuming that the classification information includes two of the target range, intermediate information, or category subsets, the classification information not included can refer to the description of the prior art and will not be described in detail here.

[0238] It can be understood that, generally speaking, the more specific information the classification information includes, the more accurate the corresponding ALF classification is.

[0239] It should be noted that the classification method of the above embodiment can be executed by default, that is, the encoder and the decoder will execute the classification method of the embodiment of the present application by default. In addition, the classification information used to execute the classification method can also be preset, for example, it is specified which information in the target range, intermediate information or category subset is used for classification.

[0240] In a possible implementation, the method of the embodiment of the present application further includes at least one of the following:

[0241] Using the target category as the adaptive loop filter category corresponding to the first image, where the first image is a sub-image in the target image;

[0242] Using the target category as the adaptive loop filter category corresponding to the second image, and the target image is a sub-image of the second image;

[0243] The target category is used as the adaptive loop filter category corresponding to the third image. The third image and the target image are located in the same image, and the third image and the target image have the same size.

[0244] In this embodiment, after determining the target category corresponding to the target image, the target category may be used as the ALF category corresponding to the first image of the sub-image in the target image, or the target category may be used as the ALF category corresponding to the second image including the target image, or the target category may be used as the ALF category corresponding to the third image located in the same image as the target image.

[0245] It is understandable that at least one of the above situations can be selected for setting as needed, and no specific limitation is made here.

[0246] The technical solution of this embodiment, after calculating the target category corresponding to the target image, uses the target category as the adaptive loop filter category corresponding to the first image, the first image is a sub-image in the target image; and / or uses the target category as the adaptive loop filter category corresponding to the second image, the target image is a sub-image of the second image; and / or uses the target category as the adaptive loop filter category corresponding to the third image, the third image and the target image are located in the same image, and the third image and the target image have the same size, so as to improve the efficiency of determining the ALF category, thereby improving the encoding and decoding efficiency.

[0247] See also Figure 6-Figure 8 . Figure 6 A schematic diagram of comparison between a target image and a first image provided in an embodiment of the present application. Figure 7 A schematic diagram of comparison between a target image and a second image provided in an embodiment of the present application. Figure 8A schematic diagram of comparison between a target image and a third image provided in an embodiment of the present application.

[0248] like Figure 6 The ALF category corresponding to the first image shown may be the same as the target category. Figure 7 The ALF category corresponding to the second image shown may be the same as the target category. Figure 8 The ALF category corresponding to the third image shown may be the same as the target category.

[0249] The technical solution of this embodiment, after determining the target category corresponding to the target image, uses the target category as the ALF category of other images related to the target image. In this way, the ALF category can be determined for other images related to the target image without executing the classification method, which can improve the efficiency of determining the ALF classification and further improve the efficiency of encoding and decoding.

[0250] It should be noted that in some example situations, there are some situations where it is not necessary to use an adaptive loop filter, or it is not necessary to use the classification method of the embodiment of the present application for classification. If classification is still performed according to the classification method of the embodiment of the present application at this time, the flexibility of executing the classification method is not high, and both the decoding end and the encoding end will perform some unnecessary processing.

[0251] Therefore, the following embodiments further illustrate how to improve the flexibility of executing the classification method based on any of the above embodiments.

[0252] See also Fig. 9 , Fig. 9 The following is a flow chart of another adaptive loop filter classification method in video coding and decoding provided by an embodiment of the present application. Fig. 9 The methods shown include:

[0253] S910: The encoding end determines whether to use an adaptive loop filter.

[0254] How to determine whether to use the adaptive loop filter can be set as needed and is not limited here. In this embodiment, if the adaptive loop filter is used, it means that the ALF classification method needs to be executed, so step S920 can be executed.

[0255] S920: The encoding end obtains classification information corresponding to the target image.

[0256] It should be noted that the classification information in this embodiment may be the information used for ALF classification in related technologies, such as the maximum range, etc., or the classification information mentioned in the embodiments of the present application, which is not limited here.

[0257] If the classification information includes at least one of the target range, intermediate information or category subset, the classification information may refer to the description of any of the above embodiments, which will not be described in detail here.

[0258] S930: The encoder determines a target category of an adaptive loop filter corresponding to the target image based on the classification information.

[0259] S940: The encoder sends a code stream corresponding to the target image to the decoder, where the code stream includes decoding indication information.

[0260] The code stream may be obtained after the encoding end encodes the target image. The decoding indication information may indicate whether the decoding end executes the classification method and how to execute the classification method, etc., which are not limited here.

[0261] S950: The decoding end determines whether to use an adaptive loop filter based on the decoding indication information.

[0262] In this embodiment, if the decoding end determines to use the adaptive loop filter based on the decoding indication information, step S960 is executed.

[0263] S960, the decoding end obtains classification information corresponding to the target image;

[0264] In this embodiment, generally speaking, the classification information obtained by the decoding end and the information obtained by the encoding end are the same.

[0265] S970: The decoding end determines a target category of an adaptive loop filter corresponding to the target image based on the classification information.

[0266] It can be understood that the classification method implemented in this embodiment can be the classification method mentioned in the embodiment of the present application, or it can be the classification method in the related technology, which is not limited here.

[0267] The technical solution of this embodiment determines whether an adaptive loop filter needs to be used by the encoding end, obtains classification information corresponding to the target image when it is determined to use the adaptive loop filter, and sends a code stream including decoding indication information to the decoding end. After receiving the code stream, the decoding end can know whether to use the adaptive loop filter based on the decoding indication information, and execute the classification method when it is determined that the adaptive loop filter needs to be used, and can choose to turn on or off the execution of the classification method as needed, which can improve the flexibility of ALF classification.

[0268] In a possible implementation manner, the decoding indication information includes at least one of first indication information, second indication information, third indication information, fourth indication information, fifth indication information, or sixth indication information.

[0269] The first indication information may indicate whether to use an adaptive loop filter. In this embodiment, the encoding end may determine whether to use an adaptive loop filter, and then send the first indication information to the decoding end according to the determination result of whether to use the adaptive loop filter, and the decoding end may also synchronously use the adaptive loop filter or not use the adaptive loop filter, and then determine whether to execute the ALF classification method.

[0270] In this embodiment, the flexibility of whether to execute the ALF classification method can be improved by using the first indication information.

[0271] The second indication information may indicate the classification information used by the classification method. In this embodiment, the encoding end may select one or more of the classification information as needed, and then inform the decoding end through the second indication information, so that the decoding end may also use the classification information used by the encoding end when executing the classification method.

[0272] It should be noted that the second indication information can implicitly inform the decoding end of the classification information used, for example, implicitly inform the decoding end whether to use the target range, intermediate information or category subset, and the decoding end can calculate the specific target range, intermediate information and category subset according to the above embodiment. In addition, the second indication information can explicitly inform the decoding end of the specific target range, intermediate information or category subset used, for example, explicitly inform the decoding end that the target range used is 64-940, and the decoding end can omit the calculation process of the classification information and directly use the explicit classification information for ALF classification.

[0273] In this embodiment, the encoding and decoding efficiency can be improved by explicitly notifying the decoding end of the classification information used.

[0274] The third indication information indicates a target level for executing the classification method. The target level includes at least one of an image sequence level, an image level, or a sub-image level. The image sequence level indicates that the classification method is executed in units of at least two frames of images. The image level indicates that the classification method is executed in units of one frame of image. The sub-image level indicates that the classification method is executed in units of sub-images in one frame of image. The sub-image level includes at least one of a stripe level, a rectangular region level, or a block level.

[0275] In this embodiment, the encoder can determine the target level at which the classification method needs to be executed as needed, such as executing the classification method at the image sequence level, executing the classification method at the image level, executing the classification method at the strip level, executing the classification method at the rectangular region level, or executing the classification method at the block level, etc. Then, the encoder can inform the decoder through the third indication information, and the decoder can also know the target level at which the encoder executes the classification method, so that the encoder and the decoder can execute the classification method at the same target level.

[0276] It can be understood that, in this embodiment, the level corresponding to the specific classification information indicated by the second indication information may be the same as or different from the level at which the decoding end executes the classification method indicated by the third indication information. For example, the third indication information may indicate that the decoding end executes the classification method at the sub-image level, while the second indication information may indicate the specific target range corresponding to the sub-image level, or the second indication information may indicate the specific target range corresponding to the image level, which is not limited here.

[0277] In this embodiment, the third indication information indicates the target level for executing the classification method, so as to improve the flexibility of the level applicable to the adaptive loop filtering classification.

[0278] The fourth indication information indicates a target sub-image in a frame image for executing the classification method, and the target sub-image is at least one of the sub-images divided from the frame image. In this embodiment, a frame image can be divided into multiple sub-images.

[0279] In some example situations, the important content of the video is generally located in the middle of the video image, so a partial sub-image of a frame of the image can be selected to perform the classification method.

[0280] In this embodiment, the encoding end can determine the target sub-image that needs to be adaptively loop filtered in each frame of image, and perform the classification method on the target sub-image. Then, the encoding end can inform the encoding end through the fourth indication information, and the decoding end can also know the target sub-image, so as to perform the classification method on the target sub-image. Optionally, the target sub-image of this embodiment can reflect the main content of a frame of image.

[0281] In this embodiment, the fourth indication information indicates the target sub-image in a frame image for executing the classification method, and some sub-images in a frame image can be selected to execute the classification method, thereby taking into account both video quality and encoding and decoding efficiency.

[0282] The fifth indication information indicates the size of a sub-image in a frame of image for executing the classification method.

[0283] In some cases of the examples, too large or too small a size will have a certain impact on the filtering effect, and thus have a certain impact on the encoding and decoding efficiency and video quality of the video codec. Therefore, sub-images of different sizes can be selected for adaptive loop filtering according to actual conditions.

[0284] In this embodiment, the encoding end can determine the target size of the sub-image for adaptive loop filtering as needed, and then perform the classification method for the sub-image of the target size. Then, the encoding end can inform the encoding end through the fifth indication information, and the decoding end can also know the target size, so that the decoding end can also perform the classification method for the sub-image of the target size. Optionally, the sub-image of the target size can reflect the main content of a frame of image. Optionally, the size can include shape and size, etc.

[0285] In this embodiment, the size of the sub-image for executing the classification method in a frame of image is indicated by the fifth indication information, and the size of the sub-image for executing the classification method can be selected, thereby taking into account both video quality and encoding and decoding efficiency.

[0286] The sixth indication information indicates the number of images in the image sequence for executing the classification method.

[0287] In some cases of the examples, in a video, generally, the content difference between some consecutive image frames is small, and the content difference between some consecutive image frames is large. Therefore, at least two frames in the image sequence for performing the classification method may have a certain impact on the video quality. Therefore, the number of images in the image sequence for performing the classification method can be selected according to actual conditions.

[0288] In this embodiment, the encoding end can determine at least two frames of images for executing the classification method as needed, and form an image sequence with the at least two frames of images, and then execute the classification method for the image sequence. Then, the decoding end is informed through the sixth indication information, and the decoding end can also execute the classification method for the image sequence with the same image sequence. Optionally, the image sequence can include at least two frames of images with relatively small content differences.

[0289] Exemplarily, the encoder regards the first frame image, the second frame image, and the third frame image as a group of image sequences, and the encoder may send sixth indication information indicating that the number of images is 3, and the decoder may start counting from the first frame image of the code stream until the third frame image. Then, the encoder regards the fourth frame image and the fifth frame image as a group of image sequences, and the encoder may send sixth indication information indicating that the number of images is 2, and the decoder may start counting from the fourth frame image of the code stream until the fifth frame image.

[0290] In this embodiment, the video quality can be improved by indicating the number of images in the image sequence for executing the classification method through the sixth indication information.

[0291] It is understandable that if the decoding end receives one of the second indication information, the third indication information, the fourth indication information, the fifth indication information and the sixth indication information, it means that the video encoding and decoding needs to use an adaptive loop filter.

[0292] For ease of understanding, the following embodiments are respectively directed to a target range, an intermediate range, and a classification subset, and are described in conjunction with a technical solution for decoding indication information.

[0293] First, a technical solution for combining a target range with decoding indication information is described. In the following examples, the target range may also be referred to as a true dynamic range.

[0294] This embodiment proposes to use the real dynamic range in ALF classification. The so-called real dynamic range means that the dynamic range of the classified content is not necessarily the same as the maximum range that can be represented by the signal bit width. For example: for a signal with a 10-bit bit width, the maximum range that can be represented is 0-1023, and the real dynamic range of a certain content is a subset of the maximum range.

[0295] In embodiment 1.1, the real dynamic range is generated by analyzing the content to be classified, which may be from an image sequence, a frame of an image, a slice, a tile or a block.

[0296] In embodiment 1.1.1, the real dynamic range is generated by the content of the entire video sequence. By analyzing the entire image sequence, the minimum and maximum values ​​of the sequence signal values ​​are found as the real dynamic range of the image sequence when performing ALF classification.

[0297] In embodiment 1.1.2, the real dynamic range is generated by the content of a frame. By analyzing the content of a frame of image, the minimum and maximum values ​​of the image signal are found as the real dynamic range of the frame content when performing ALF classification.

[0298] In embodiment 1.1.3, the real dynamic range is generated by the content of a slice. By analyzing the image content of a slice, the minimum and maximum values ​​of the signal values ​​in the slice are found as the real dynamic range of the slice content when performing ALF classification.

[0299] In embodiment 1.1.4, the real dynamic range is generated by the content of a tile. By analyzing the image content of a tile, the minimum and maximum values ​​of the signal value in the tile are found as the real dynamic range of the tile content when performing ALF classification.

[0300] In embodiment 1.1.5, the real dynamic range is generated by the content of a block. By analyzing the image content in a block, the minimum and maximum values ​​of the signal value in the block are found as the real dynamic range of the block content when ALF classification is performed. Here, the size of the block can be 256x256, 128x128, 64x64, 32x32, etc.

[0301] In embodiment 1.2, the real dynamic range used for ALF classification is an interval predefined at both the encoding end and the decoding end, and the interval is a sub-interval of the maximum dynamic range that can be represented by the signal bit width.

[0302] In Example 1.2.1, the real dynamic range may use the dynamic range specified by ITU-R BT.2020, that is, for content represented by a 10-bit bit width, the effective dynamic range is [64,940].

[0303] Example 1.3, whether to use the true dynamic range when ALF classification needs to be transmitted at the HLS (i.e. VPS, SPS, PPS, APS, Picture Header, Slice Header) or block level (i.e. 256x256, 128x128, 64x64, 32x32, etc.).

[0304] In embodiment 1.4, the real dynamic range used in ALF classification is transmitted to the decoding end along with the ALF filter bank. When the filter bank is selected for ALF filtering, the corresponding real dynamic range is used for classification.

[0305] Example 1.5, the actual dynamic range used in ALF classification needs to be transmitted at HLS (i.e. VPS, SPS, PPS, APS, Picture Header, Slice Header) or block level (i.e. 256x256, 128x128, 64x64, 32x32, etc.).

[0306] In Example 1.6, when multiple levels of switches are used to determine whether to use the real dynamic range in ALF classification, the switches of each level need to be explicitly transmitted to the decoding end.

[0307] In Example 1.6.1, when whether to use the real dynamic range in ALF classification can be determined at the block level of different sizes (for example, it can be determined at the block level of four different sizes: 256x256, 128x128, 64x64, and 32x32), the optimal block-level switch size needs to be transmitted to the decoder.

[0308] In embodiment 1.6.2, when whether to use the real dynamic range in ALF classification can be determined at both the non-block level (sequence level, frame level, slice level, tile level, etc.) and the block level (256x256, 128x128, 64x64, 32x32, etc.), it is necessary to first transmit a flag indicating whether it is a block-level switch (or a non-block-level switch). If the flag is true, then the optimal block-level switch size (or the optimal non-block-level switch level) is further transmitted. Otherwise, it is a non-block-level switch (or a block-level switch), and the optimal non-block-level switch level (or the optimal block-level switch size) is further transmitted.

[0309] In embodiment 1.7, when the real dynamic range used in ALF classification can be generated by contents of multiple levels, whether the contents of each level are used to generate the real dynamic range needs to be explicitly transmitted to the decoding end.

[0310] Example 1.7.1, when the real dynamic range used in ALF classification can be generated by block-level contents of different sizes (for example, it can be generated by block-level contents of four different sizes: 256x256, 128x128, 64x64, and 32x32), the optimal block-level size needs to be transmitted to the decoding end.

[0311] Embodiment 1.7.2, when the real dynamic range used in ALF classification can be generated at both non-block level (sequence level, frame level, slice level, tile level, etc.) and block level (256x256, 128x128, 64x64, 32x32, etc.), it is necessary to first transmit a flag indicating whether to use block-level content generation (or non-block-level content generation), and if the flag is true, further transmit the best block-level size (or best non-block-level level). Otherwise, it is generated using non-block-level content (or generated using block-level content), and further transmit the best non-block-level level (or best block-level size).

[0312] Embodiment 1.8: When multiple real dynamic ranges predefined at both the encoding end and the decoding end can be used for ALF classification, the best real dynamic range needs to be transmitted to the decoding end.

[0313] Example 1.8.1, when n predefined real dynamic ranges are used in ALF classification, the dynamic ranges are [l 0 ,h 0 ],[l 1 ,h 1 ],…,[l n-1 ,h n-1], each interval is different from each other, and at most one dynamic range is the maximum dynamic range that the signal bit width can represent, and the remaining dynamic ranges are sub-intervals of the maximum dynamic range that the signal bit width can represent. The encoder selects the best real dynamic range through comparison and transmits the best real dynamic range to the decoder.

[0314] Example 1.9, when using the true dynamic range for ALF classification, the final category is generated according to the following formula:

[0315]

[0316] Among them, input represents the input information used for classification, range min ,range max They represent the minimum and maximum values ​​of the true dynamic range respectively, and total_number represents the total number of categories of the corresponding classifier.

[0317] In Example 1.9.1, a single sample is classified using a true dynamic range classifier. The classifier classifies each sample and generates a corresponding category. At this time, the information of the single sample is used as the input of the classifier.

[0318] Embodiment 1.9.2, using a true dynamic range classifier to classify sub-blocks. The classifier classifies each N×N (such as 2×2, 4×4, etc.) sub-block, and all samples (pixels) in the sub-block use the same category. At this time, the information of an M×M (where M≥N) area containing the N×N sub-block is used as the input of the classifier, and the true dynamic range needs to be modified to the true dynamic range corresponding to the M×M.

[0319] Secondly, a technical solution for combining intermediate information with decoding indication information is described.

[0320] In this embodiment, during ALF classification, intermediate information generated by other coding and decoding processes is used as input of the classifier for classification. The so-called intermediate information generated by the coding and decoding process refers to other information except the information currently used by the ALF classifier.

[0321] In embodiment 2.1, the reconstructed component before the deblocking filter is used as the input of the classifier for classification.

[0322] Embodiment 2.2 uses the reconstructed component before the deblocking filter filtered by the fixed filter as the input of the classifier for classification. The method first needs to use the fixed filter to filter the reconstructed component before the deblocking filter, and then use the filtered output for classification.

[0323] Embodiment 2.3, using the current signal to be processed filtered by a fixed filter as the input of the classifier for classification. This method first needs to use a fixed filter to filter the current signal to be processed, and then use the filtered output for classification.

[0324] Embodiment 2.4, whether to use the intermediate information generated by other encoding and decoding processes for classification during ALF classification needs to be transmitted at HLS (i.e. VPS, SPS, PPS, APS, Picture Header, Slice Header) or block level (i.e. 256x256, 128x128, 64x64, 32x32, etc.).

[0325] It can be understood that the classifier proposed in this embodiment that uses intermediate information generated by other encoding and decoding processes for classification can be applied to a single sample or to the block level (for example, 2x2, 4x4, etc.), and all samples in the block use the same category.

[0326] The classification using the intermediate information generated by other encoding and decoding processes proposed in this embodiment is applicable to any classification method, and only needs to use the corresponding intermediate information as the input of the classifier.

[0327] In addition, a technical solution for combining category subsets with decoding indication information is described.

[0328] In this embodiment, it is proposed to use a combined classifier combining multiple individual classifiers for classification during ALF classification. The combined classifier can be any classifier, such as the gradient-based classifier used in 2.1.1 and 2.1.2, the 2x2 block-based classifier, the residual component-based classifier, etc.

[0329] Assume that there are k separate classifiers (classifier 0 ,classifier 1 ,…,classifier k-1 ) are combined into a classifier, and the number of categories corresponding to each individual classifier is n 0 ,n 1 ,…,n k-1 The final category index of the combined classifier is generated as follows:

[0330] classIdx=(n 1 *n 2 *…*n k-1 )*classIdx 0 +(n 2 *…*n k-1 )*classIdx 1 +…+classIdxk-1

[0331] Among them, classIdx 0 ,classIdx 1 ,…,classIdx k-1 The category index corresponding to each individual classifier.

[0332] Example 3.1, ALF classification is performed using a classifier that is a combination of a gradient-based classifier and a sample-based classifier. In this example, there are two separate classifiers, each with n number of categories. 0 =25,n 1 =25, then the final category index of the combined classifier is:

[0333] classIdx=25*classIdx 0 +classIdx 1

[0334] Whether to use a combined classifier when ALF classification needs to be transmitted at the HLS (i.e. VPS, SPS, PPS, APS, PictureHeader, Slice Header) or block level (i.e. 256x256, 128x128, 64x64, 32x32, etc.).

[0335] The combined classifier proposed in the embodiment of the present application can be applied to a single sample or to a block level (eg, 2x2, 4x4, etc.), where all samples in the block use the same category.

[0336] See also Fig.10 , Fig.10 Schematic diagram of the structure of an adaptive loop filtering classification device 100 provided in an embodiment of the present application. Fig.10 The adaptive loop filtering classification device 100 shown includes an information acquisition module 1010 and a classification module 1020, wherein:

[0337] The information acquisition module 1010 is used to acquire classification information corresponding to the target image, where the classification information includes at least one of a target range, intermediate information, or a category subset; wherein the target image corresponds to an image sequence, a frame of image, a strip image in a frame of image, a rectangular image, or a block image in a video;

[0338] A classification module 1020, for determining a target category of an adaptive loop filter corresponding to a target image based on the classification information;

[0339] The target range includes at least one of a first dynamic range or a second dynamic range, the first dynamic range is a range determined based on pixel information of the target image, and the second dynamic range is a range determined based on a preset dynamic range, and at least one preset dynamic range is smaller than a maximum range that can be represented by a signal bit width of the target image;

[0340] Intermediate information includes information generated by the target image during the encoding and decoding process;

[0341] The category subset includes at least two subcategories. The at least two subcategories are categories of adaptive loop filters corresponding to the target image determined by respective classifiers in the classifier group. Each subcategory corresponds to a classifier.

[0342] In a possible implementation, a minimum value of the first dynamic range is determined based on a minimum signal value corresponding to the target image, and a maximum value of the first dynamic range is determined based on a maximum signal value corresponding to the target image.

[0343] In one possible implementation, the target image includes at least one of an original uncompressed signal, a reconstructed signal, a residual signal, or a predicted signal; the minimum signal value and the maximum signal value are determined based on at least one of the original uncompressed signal, the reconstructed signal, the residual signal, or the predicted signal.

[0344] In a possible implementation, the second dynamic range is selected from N preset dynamic ranges, wherein M preset dynamic ranges among the N preset dynamic ranges are maximum ranges, and NM preset dynamic ranges among the N preset dynamic ranges are smaller than the maximum range;

[0345] Wherein, M is a natural number less than 2, and N≥2.

[0346] In a possible implementation, the classification module 1020 is further used for at least one of the following:

[0347] Using the target category as the adaptive loop filter category corresponding to the first image, where the first image is a sub-image in the target image;

[0348] Using the target category as the adaptive loop filter category corresponding to the second image, and the target image is a sub-image of the second image;

[0349] The target category is used as the adaptive loop filter category corresponding to the third image. The third image and the target image are located in the same image, and the third image and the target image have the same size.

[0350] In a possible implementation, the intermediate information includes at least one of the following:

[0351] a reconstructed signal before being filtered by a deblocking filter, wherein the reconstructed signal includes a reconstructed signal after being filtered by a fixed filter or a reconstructed signal after not being filtered by a fixed filter;

[0352] The signal to be processed is filtered by a fixed filter, and the signal to be processed includes a brightness signal.

[0353] In a possible implementation, the classification information includes a category subset; the target category is determined from L preset categories based on at least two subcategories, where L is the product of the number of categories corresponding to each classifier in the classifier group.

[0354] In a possible implementation, the subcategory is represented by a category index, and a subcategory is one category among multiple categories corresponding to a classifier;

[0355] Based on the classification information, the target category of the adaptive loop filter corresponding to the target image is determined, including:

[0356] Based on the individual category indices in the category subset, the category index classIdx of the target category is determined by the following expression:

[0357] classIdx=(n 1 *n 2 *…*n k-1 )*classIdx 0 +(n 2 *…*n k-1 )*classIdx 1 +…+classIdx k-1

[0358] Where k represents the number of classifiers included in the classifier group, n k-1 Indicates the number of categories corresponding to the k-th classifier, classIdx k-1 Represents the category index corresponding to the k-th classifier in the category subset.

[0359] In a possible implementation, the adaptive loop filtering device 100 is applied to an encoding end, and the adaptive loop filtering device 100 is further used to determine whether to use an adaptive loop filter;

[0360] If it is determined to use an adaptive loop filter, a classification method is executed, and a bitstream corresponding to the target image is sent to a decoding end, where the bitstream includes decoding indication information, and the decoding indication information includes at least one of first indication information, second indication information, third indication information, fourth indication information, fifth indication information, or sixth indication information;

[0361] The first indication information indicates whether to use an adaptive loop filter;

[0362] The second indication information indicates the classification information used by the classification method;

[0363] The third indication information indicates a target level for executing the classification method, the target level includes at least one of an image sequence level, an image level, or a sub-image level, the image sequence level indicates that the classification method is executed in units of at least two frames of images, the image level indicates that the classification method is executed in units of one frame of image, the sub-image level indicates that the classification method is executed in units of sub-images in one frame of image, and the sub-image level includes at least one of a stripe level, a rectangular region level, or a block level;

[0364] The fourth indication information indicates a target sub-image in a frame of image for executing the classification method, where the target sub-image is at least one of the sub-images divided from the frame of image;

[0365] The fifth indication information indicates the size of the sub-image in a frame of image on which the classification method is executed;

[0366] The sixth indication information indicates the number of images in the image sequence for which the classification method is executed.

[0367] In a possible implementation, the adaptive loop filtering device 100 is applied to a decoding end, and the adaptive loop filtering device 100 is further used to receive a bit stream, wherein the bit stream includes decoding indication information;

[0368] If it is determined based on the decoding indication information that an adaptive loop filter is used, a classification method is performed;

[0369] The decoding indication information includes at least one of the first indication information, the second indication information, the third indication information, the fourth indication information, the fifth indication information or the sixth indication information;

[0370] The first indication information indicates whether to use an adaptive loop filter;

[0371] The second indication information indicates the classification information used by the classification method;

[0372] The third indication information indicates a target level for executing the classification method, the target level includes at least one of an image sequence level, an image level, or a sub-image level, the image sequence level indicates that the classification method is executed in units of at least two frames of images, the image level indicates that the classification method is executed in units of one frame of image, the sub-image level indicates that the classification method is executed in units of sub-images in one frame of image, and the sub-image level includes at least one of a stripe level, a rectangular region level, or a block level;

[0373] The fourth indication information indicates a target sub-image in a frame of image for executing the classification method, where the target sub-image is at least one of the sub-images divided from the frame of image;

[0374] The fifth indication information indicates the size of the sub-image in a frame of image on which the classification method is executed;

[0375] The sixth indication information indicates the number of images in the image sequence for which the classification method is executed.

[0376] The adaptive loop filtering classification device 100 of this embodiment can execute the steps of adaptive loop filtering classification shown in the aforementioned embodiments of the present application. The implementation principles are similar and will not be repeated here.

[0377] The beneficial effects of the adaptive loop filter classification device 100 in video coding and decoding can be found in the description of the adaptive loop filter classification method in video coding and decoding in the foregoing text, which is also applicable to the effect description of the electronic device, computer-readable storage medium, and computer program product below.

[0378] An embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory, and the processor executes the above computer program to implement the steps of the method of any of the above embodiments.

[0379] In an alternative embodiment, an electronic device is provided, such as Fig.11 As shown, Fig.11 The electronic device 1100 shown includes: a processor 1101 and a memory 1103. The processor 1101 and the memory 1103 are connected, such as through a bus 1102. Optionally, the electronic device 1100 may also include a transceiver 1104, which may be used for data interaction between the electronic device and other electronic devices, such as data transmission and / or data reception. It should be noted that in actual applications, the transceiver 1104 is not limited to one, and the structure of the electronic device 1100 does not constitute a limitation on the embodiments of the present application.

[0380] Processor 1101 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 1101 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0381] The bus 1102 may include a path to transmit information between the above components. The bus 1102 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 1102 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.11 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0382] The memory 1103 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium that can be used to carry or store computer programs and can be read by a computer, without limitation herein.

[0383] The memory 1103 is used to store the computer program for executing the embodiment of the present application, and the execution is controlled by the processor 1101. The processor 1101 is used to execute the computer program stored in the memory 1103 to implement the steps shown in the above method embodiment.

[0384] An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps and corresponding contents of the aforementioned method embodiment can be implemented.

[0385] The embodiment of the present application also provides a computer program product, including a computer program, which can implement the steps and corresponding contents of the aforementioned method embodiment when executed by a processor.

[0386] It is to be understood that, although each operation step is indicated by arrows in the flowchart of the embodiment of the present application, the implementation order of these steps is not limited to the order indicated by the arrows. Unless clearly stated herein, in some implementation scenarios of the embodiment of the present application, the implementation steps in each flowchart can be performed in other orders according to demand. In addition, some or all of the steps in each flowchart may include multiple sub-steps or multiple stages based on actual implementation scenarios. Some or all of these sub-steps or stages may be executed at the same time, and each sub-step or stage in these sub-steps or stages may also be executed at different times respectively. In different scenarios of execution time, the execution order of these sub-steps or stages may be flexibly configured according to demand, and the embodiment of the present application does not limit this.

[0387] The terms "first", "second", etc. in the claims, specification and drawings of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or electronic device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or electronic devices. Mentioning "embodiment" in this article means that the specific features, structures or characteristics described in conjunction with the embodiment may be included in at least one embodiment of the present application. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments. The term "and / or" used in the present application specification and the attached claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0388] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Professional and technical personnel may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0389] The above disclosure is only the preferred embodiment of the present application, which cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. An adaptive loop filter classification method in video coding and decoding, characterized in that: include: Acquire classification information corresponding to a target image, the classification information including at least one of a target range, intermediate information, or a category subset; wherein the target image corresponds to an image sequence in a video, a frame of image, a strip image in a frame of image, a rectangular image, or a block image; Based on the classification information, determining a target category of an adaptive loop filter corresponding to the target image; The target range includes at least one of a first dynamic range and a second dynamic range, the first dynamic range is a range determined based on pixel information of the target image, the second dynamic range is a range determined based on a preset dynamic range, and at least one of the preset dynamic ranges is smaller than a maximum range that can be represented by a signal bit width of the target image; The intermediate information includes information generated by the target image during the encoding and decoding process; The category subset includes at least two subcategories, and the at least two subcategories are categories of adaptive loop filters corresponding to the target image determined by respective classifiers in the classifier group, and each subcategory corresponds to a classifier.

2. The method according to claim 1, characterized in that The minimum value of the first dynamic range is determined based on the minimum signal value corresponding to the target image, and the maximum value of the first dynamic range is determined based on the maximum signal value corresponding to the target image.

3. The method according to claim 2, characterized in that The target image includes at least one of an original uncompressed signal, a reconstructed signal, a residual signal or a predicted signal; The minimum signal value and the maximum signal value are determined based on at least one of the original uncompressed signal, the reconstructed signal, the residual signal, or the predicted signal.

4. The method according to claim 1, characterized in that: The second dynamic range is selected from N preset dynamic ranges, wherein M preset dynamic ranges of the N preset dynamic ranges are the maximum range, and NM preset dynamic ranges of the N preset dynamic ranges are smaller than the maximum range; Wherein, M is a natural number less than 2, and N≥2.

5. The method according to claim 1, characterized in that: The method further comprises at least one of the following: Using the target category as an adaptive loop filter category corresponding to a first image, where the first image is a sub-image in the target image; Using the target category as an adaptive loop filter category corresponding to a second image, wherein the target image is a sub-image of the second image; The target category is used as an adaptive loop filter category corresponding to a third image, the third image and the target image are located in the same image, and the third image and the target image have the same size.

6. The method according to claim 1, characterized in that The intermediate information includes at least one of the following: a reconstructed signal before being filtered by a deblocking filter, wherein the reconstructed signal includes a reconstructed signal after being filtered by a fixed filter or a reconstructed signal after not being filtered by a fixed filter; The signal to be processed is filtered by a fixed filter, wherein the signal to be processed includes a brightness signal.

7. The method according to claim 1, characterized in that The classification information includes the category subset; the target category is determined from L preset categories based on the at least two subcategories, where L is the product of the number of categories corresponding to each classifier in the classifier group.

8. The method according to claim 7, characterized in that The subcategory is represented by a category index, and a subcategory is one of the multiple categories corresponding to a classifier; The step of determining a target category of an adaptive loop filter corresponding to the target image based on the classification information includes: Based on the category indexes in the category subset, the category index classIdx of the target category is determined by the following expression: classIdx=(n1*n2*…*n k-1 )*classIdx0+(n2*…*n k-1 )*classIdx1+…+classIdx k-1 Where k represents the number of classifiers included in the classifier group, n k-1 Indicates the number of categories corresponding to the kth classifier, classIdx k-1 Represents the classification index corresponding to the kth classifier in the category subset.

9. The method according to any one of claims 1 to 8, characterized in that: Applied to the encoding end, the method further includes: determining whether to use an adaptive loop filter; If it is determined to use an adaptive loop filter, a classification method is executed, and a bitstream corresponding to the target image is sent to a decoding end, wherein the bitstream includes decoding indication information, and the decoding indication information includes at least one of first indication information, second indication information, third indication information, fourth indication information, fifth indication information, or sixth indication information; The first indication information indicates whether to use an adaptive loop filter; The second indication information indicates classification information used by the classification method; The third indication information indicates a target level for executing the classification method, the target level includes at least one of an image sequence level, an image level, or a sub-image level, the image sequence level indicates that the classification method is executed in units of at least two frames of images, the image level indicates that the classification method is executed in units of one frame of image, the sub-image level indicates that the classification method is executed in units of sub-images in one frame of image, and the sub-image level includes at least one of a stripe level, a rectangular region level, or a block level; The fourth indication information indicates a target sub-image in a frame of image for which a classification method is to be executed, and the target sub-image is at least one of the sub-images divided from the frame of image; The fifth indication information indicates the size of a sub-image in a frame of image on which the classification method is executed; The sixth indication information indicates the number of images in the image sequence for executing the classification method.

10. The method according to any one of claims 1 to 8, characterized in that: Applied to a decoding end, the method further includes: receiving a code stream, wherein the code stream includes decoding indication information; If it is determined to use an adaptive loop filter based on the decoding indication information, executing a classification method; The decoding indication information includes at least one of first indication information, second indication information, third indication information, fourth indication information, fifth indication information or sixth indication information; The first indication information indicates whether to use an adaptive loop filter; The second indication information indicates classification information used by the classification method; The third indication information indicates a target level for executing the classification method, the target level includes at least one of an image sequence level, an image level, or a sub-image level, the image sequence level indicates that the classification method is executed in units of at least two frames of images, the image level indicates that the classification method is executed in units of one frame of image, the sub-image level indicates that the classification method is executed in units of sub-images in one frame of image, and the sub-image level includes at least one of a stripe level, a rectangular region level, or a block level; The fourth indication information indicates a target sub-image in a frame of image for which a classification method is to be executed, and the target sub-image is at least one of the sub-images divided from the frame of image; The fifth indication information indicates the size of a sub-image in a frame of image on which the classification method is executed; The sixth indication information indicates the number of images in the image sequence for executing the classification method.

11. An adaptive loop filter classification device in video coding and decoding, characterized in that: include: An information acquisition module, used to acquire classification information corresponding to a target image, wherein the classification information includes at least one of a target range, intermediate information, or a category subset; wherein the target image corresponds to an image sequence in a video, a frame of image, a strip image in a frame of image, a rectangular image, or a block image; A classification module, used to determine a target category of an adaptive loop filter corresponding to the target image based on the classification information; The target range includes at least one of a first dynamic range and a second dynamic range, the first dynamic range is a range determined based on pixel information of the target image, the second dynamic range is a range determined based on a preset dynamic range, and at least one of the preset dynamic ranges is smaller than a maximum range that can be represented by a signal bit width of the target image; The intermediate information includes information generated by the target image during the encoding and decoding process; The category subset includes at least two subcategories, and the at least two subcategories are categories of adaptive loop filters corresponding to the target image determined by respective classifiers in the classifier group, and each subcategory corresponds to a classifier.

12. An electronic device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 10.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

14. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

Images