Method, apparatus, storage medium, and electronic device for video processing

Abstract

An embodiment of the present invention provides a method, an apparatus, a storage medium and an electronic device for video processing, the method including: determining a neural network loop filtering enable flag of a reconstructed video unit, setting an adaptive loop filtering enable flag for the reconstructed video unit based on the neural network loop filtering enable flag, and reporting by signaling at least one of filter information related to neural network loop filtering of the reconstructed video unit and filter information related to adaptive loop filtering of the reconstructed video unit.

Description

Technical Field

[0001] This disclosure claims priority based on Chinese Application No. CN202111679995.5, filed on December 31, 2021 (invention name: Method, Apparatus, Storage Medium, and Electronic Device for Video Processing), and all the contents of this application are incorporated herein by reference in their entirety. Embodiments of the present invention relate to the field of communications, and more specifically, to a method, apparatus, storage medium, and electronic device for video processing.

Background Art

[0002] It has become a trend to use machine learning technologies represented by Neural Network (abbreviated as NN) in image / video coding. Neural Network In-loop Filter (abbreviated as NNF) can significantly improve the quality of the reconstructed video by establishing a non-linear mapping relationship from the distortion region to the original / lossless region.

[0003] Due to the offline nature of NNF network training, generally, an Adaptive Loop Filter (abbreviated as ALF) filter is still used after NNF to further improve the performance of the encoded image. NNF can often achieve optimal performance for the luminance component of the video. However, when the ALF filtering module makes a decision and selection at the Slice level, if it is judged only based on the rate-distortion optimization performance of the luminance component, the ALF filtering module may be turned off. On the other hand, the NNF of the chroma component is often not optimal, and it is necessary to further perform chroma ALF and Cross-component Adaptive Loop Filter (abbreviated as CCALF) to achieve better performance of the chroma component.

[0004] However, in the current ALF filtering technology, directly using the ALF module after NNF filtering will cause the following problems.

[0005] The luminance component at the slice level achieves optimal performance after NNF, and if the ALF filtering cannot further improve in terms of the luminance component, the current slice-level luminance ALF is set to the off state. However, once the ALF is turned off according to the luminance performance, both the chroma ALF and CCALF are forced to be off, and the chroma ALF filtering operation will not be performed. If the chroma component after NNF filtering is not subjected to ALF filtering, more deviation between values and the original values will be introduced, and the image quality may not be optimized.

[0006] Regarding the problem in the related art that the image quality after processing is relatively poor, no effective solution has been proposed yet.

Summary of the Invention

Problems to be Solved by the Invention

[0007] Embodiments of the present invention provide a method, apparatus, storage medium, and electronic device for video processing that at least solve the problem in the related art that the image quality after processing is relatively poor.

Means for Solving the Problems

[0008] According to one embodiment of the present invention, a method for video processing is provided. The method includes determining a neural network loop filtering enabling flag for a reconstructed video unit, setting an adaptive loop filtering enabling flag for the reconstructed video unit based on the neural network loop filtering enabling flag, and signaling at least one of filter information related to neural network loop filtering of the reconstructed video unit and filter information related to adaptive loop filtering of the reconstructed video unit.

[0009] In one alternative embodiment, the filter information related to the adaptive loop filtering includes an adaptive loop filtering enabling flag and an adaptive parameter set referred to by the adaptive loop filtering.

[0010] In one alternative embodiment, the adaptive loop filtering enabling flag includes a luminance adaptive loop filtering enabling flag and a chroma adaptive loop filtering enabling flag.

[0011] In one alternative embodiment, the chroma adaptive loop filtering enabling flag includes at least one of a first chroma adaptive loop filtering enabling flag, a second chroma adaptive loop filtering enabling flag, a first chroma component - to - component adaptive loop filtering enabling flag, and a second chroma component - to - component adaptive loop filtering enabling flag.

[0012] In one alternative embodiment, the adaptive loop filtering enabling flag is signaled in at least one syntax element of a sequence parameter set SPS, a picture parameter set PPS, a picture header PH, a slice header SH, and an encoding / decoding tree unit CTU for generating encoded video data of the reconstructed video unit.

[0013] In one alternative embodiment, the method further includes setting a luma adaptive loop filtering enable flag and a chroma adaptive loop filtering enable flag respectively for the reconstructed video unit when the neural network loop filtering enable flag instructs to perform neural network loop filtering on the reconstructed video unit at sequence level or picture level or slice level.

[0014] In one alternative embodiment, the method further includes setting a luma adaptive loop filtering enable flag and a chroma adaptive loop filtering enable flag respectively for the reconstructed video unit when the neural network loop filtering enable flag instructs to perform neural network loop filtering on the reconstructed video unit at sequence level and the adaptive loop filtering enable flag instructs not to perform luma adaptive loop filtering on the reconstructed video unit at picture level or slice level.

[0015] In one alternative embodiment, the method further includes determining whether to perform chroma adaptive loop filtering on the reconstructed video unit based on the chroma adaptive loop filtering enable flag.

[0016] In one alternative embodiment, setting a luma adaptive loop filtering enable flag and a chroma adaptive loop filtering enable flag respectively for the reconstructed video unit includes setting a first flag related to an adaptive loop filter of a luma component for the reconstructed video unit.

[0017] In one alternative embodiment, setting the luma adaptive loop filtering enable flag and the chroma adaptive loop filtering enable flag respectively for the reconstructed video unit includes setting a second flag related to the adaptive loop filter for the chroma component for the reconstructed video unit.

[0018] In one alternative embodiment, when the value of the adaptive loop filtering enable flag is a first value, the adaptive loop filtering enable flag is used to instruct to perform adaptive loop filtering on the reconstructed video unit, and when the value of the adaptive loop filtering enable flag is a second value, the adaptive loop filtering enable flag is used to instruct not to perform adaptive loop filtering on the reconstructed video unit, and the first value is different from the second value.

[0019] In one alternative embodiment, performing adaptive loop filtering on the reconstructed video unit includes at least one of the operations of performing luma adaptive loop filtering on the luma component of the reconstructed video unit, performing first chroma adaptive loop filtering on the first chroma component of the reconstructed video unit, performing second chroma adaptive loop filtering on the second chroma component of the reconstructed video unit, performing first inter-chroma component adaptive loop filtering on the first chroma component of the reconstructed video unit, and performing first inter-chroma component adaptive loop filtering on the second chroma component of the reconstructed video unit.

[0020] In one alternative embodiment, when the neural network loop filtering enabling flag instructs to perform neural network loop filtering on the reconstructed video unit at the sequence level or the picture level or the slice level, a determination to perform luminance adaptive loop filtering on the luminance component of the reconstructed video unit, a determination to perform first chroma adaptive loop filtering on the first chroma component of the reconstructed video unit, a determination to perform second chroma adaptive loop filtering on the second chroma component of the reconstructed video unit, a determination to perform first inter-chroma adaptive loop filtering on the first chroma component of the reconstructed video unit, and a determination to perform first inter-chroma adaptive loop filtering on the second chroma component of the reconstructed video unit. Based on at least one of these determinations, an adaptive loop filtering enabling flag is set for the reconstructed video unit.

[0021] In one alternative embodiment, the filter information related to the neural network loop filtering includes a neural network loop filtering enabling flag and an adaptive parameter set referred to by the neural network loop filtering.

[0022] In one alternative embodiment, the neural network loop filtering enabling flag includes at least one of a luminance neural network loop filtering enabling flag, a first chroma neural network loop filtering enabling flag, and a second chroma neural network loop filtering enabling flag.

[0023] In one alternative embodiment, the neural network loop filtering enabling flag is signaled in at least one syntax element of a sequence parameter set SPS, a picture parameter set PPS, a picture header PH, a slice header SH, and an encoding / decoding tree unit CTU of the encoded video data of the reconstructed video unit.

[0024] In one alternative embodiment, the reconstructed video unit corresponds to at least one of a video image, a video clip, a video pattern block, a slice, an encoding / decoding tree unit CTU, and an encoding / decoding unit CU.

[0025] According to another embodiment of the present invention, a method for video processing is further provided. The method includes receiving at least one of filter information related to neural network loop filtering of a reconstructed video unit and filter information related to adaptive loop filtering of the reconstructed video unit, determining a neural network loop filtering enabling flag of the reconstructed video unit and an adaptive loop filtering enabling flag of the reconstructed video unit, and performing adaptive loop filtering on the reconstructed video unit based on the adaptive loop filtering enabling flag.

[0026] In one alternative embodiment, the filter information related to the adaptive loop filtering includes an adaptive loop filtering enabling flag and an adaptive parameter set referenced by the adaptive loop filtering.

[0027] In one alternative embodiment, the adaptive loop filtering enabling flag includes a luminance adaptive loop filtering enabling flag and a chroma adaptive loop filtering enabling flag.

[0028] In one alternative embodiment, the chroma adaptive loop filtering enabling flag includes at least one of a first chroma adaptive loop filtering enabling flag, a second chroma adaptive loop filtering enabling flag, a first inter-chroma component adaptive loop filtering enabling flag, and a second inter-chroma component adaptive loop filtering enabling flag.

[0029] In one alternative embodiment, the adaptive loop filtering enabling flag is determined in at least one syntax element of a sequence parameter set SPS, a picture parameter set PPS, a picture header PH, a slice header SH, and an encoding / decoding tree unit CTU for generating encoded video data of the reconstructed video unit.

[0030] In one alternative embodiment, the method further includes determining a luminance adaptive loop filtering enabling flag and a chroma adaptive loop filtering enabling flag of the reconstructed video unit when the neural network loop filtering enabling flag instructs to perform neural network loop filtering on the reconstructed video unit at the sequence level or picture level or slice level.

[0031] In one alternative embodiment, the method further includes determining a luminance adaptive loop filtering enabling flag of the reconstructed video unit at the picture level or slice level when the neural network loop filtering enabling flag instructs to perform neural network loop filtering on the reconstructed video unit at the sequence level, and determining a chroma adaptive loop filtering enabling flag of the reconstructed video unit when the luminance adaptive loop filtering enabling flag instructs not to perform luminance adaptive loop filtering on the reconstructed video unit.

[0032] In one alternative embodiment, the method further includes determining whether to perform chroma adaptive loop filtering on the reconstructed video unit based on the chroma adaptive loop filtering enable flag.

[0033] In one alternative embodiment, determining the luminance adaptive loop filtering enable flag of the reconstructed video unit includes determining the value of a first flag related to the adaptive loop filter of the luminance component of the reconstructed video unit.

[0034] In one alternative embodiment, determining the chroma adaptive loop filtering enable flag of the reconstructed video unit includes determining the value of a second flag related to the adaptive loop filter of the chroma component of the reconstructed video unit.

[0035] In one alternative embodiment, when the value of the adaptive loop filtering enable flag is a first value, the adaptive loop filtering enable flag is used to indicate to perform adaptive loop filtering on the reconstructed video unit, and when the value of the adaptive loop filtering enable flag is a second value, the adaptive loop filtering enable flag is used to indicate not to perform adaptive loop filtering on the reconstructed video unit, and the first value is different from the second value.

[0036] In one alternative embodiment, performing adaptive loop filtering on the reconstructed video unit includes performing luminance adaptive loop filtering on the luminance component of the reconstructed video unit, performing first chroma adaptive loop filtering on the first chroma component of the reconstructed video unit, performing second chroma adaptive loop filtering on the second chroma component of the reconstructed video unit, performing first inter-chroma adaptive loop filtering on the first chroma component of the reconstructed video unit, and performing first inter-chroma adaptive loop filtering on the second chroma component of the reconstructed video unit, including at least one of the operations.

[0037] In one alternative embodiment, the filter information related to the neural network loop filtering includes a neural network loop filtering enable flag and an adaptive parameter set referred to by the neural network loop filtering.

[0038] In one alternative embodiment, the neural network loop filtering enable flag includes at least one of a luminance neural network loop filtering enable flag, a first chroma neural network loop filtering enable flag, and a second chroma neural network loop filtering enable flag.

[0039] In one alternative embodiment, the neural network loop filtering enable flag is determined in at least one syntax element of the sequence parameter set SPS, the picture parameter set PPS, the picture header PH, the slice header SH, and the coding / decoding tree unit CTU for generating the encoded video data of the reconstructed video unit.

[0040] In one alternative embodiment, the reconstructed video unit corresponds to at least one of a video image, a video clip, a video pattern block, a slice, a coding / decoding tree unit CTU, and a coding / decoding unit CU.

[0041] According to another embodiment of the present invention, there is further provided an apparatus for video processing. The apparatus includes a determination module for determining a neural network loop filtering enabling flag of the reconstructed video unit, a setting module for setting an adaptive loop filtering enabling flag for the reconstructed video unit based on the neural network loop filtering enabling flag, and a notification module for notifying at least one of filter information related to neural network loop filtering of the reconstructed video unit and filter information related to adaptive loop filtering of the reconstructed video unit by signaling.

[0042] In one alternative embodiment, the filter information related to the adaptive loop filtering includes an adaptive loop filtering enabling flag and an adaptive parameter set referred to by the adaptive loop filtering.

[0043] In one alternative embodiment, the notification module includes a notification unit for notifying the adaptive loop filtering enabling flag by signaling in at least one syntax element of a sequence parameter set SPS, a picture parameter set PPS, a picture header PH, a slice header SH, and a coding / decoding tree unit CTU of the encoded video data for generating the reconstructed video unit.

[0044] According to another embodiment of the present invention, there is further provided an apparatus for video processing. The apparatus includes a receiving module for receiving at least one of filter information related to neural network loop filtering of a reconstructed video unit and filter information related to adaptive loop filtering of the reconstructed video unit, an identifying module for identifying an adaptive loop filtering enabling flag of the reconstructed video unit when a neural network loop filtering enabling flag instructs to perform neural network loop filtering on the reconstructed video unit, and a filtering module for performing adaptive loop filtering on the reconstructed video unit based on the adaptive loop filtering enabling flag.

[0045] In one alternative embodiment, the filter information related to the adaptive loop filtering includes an adaptive loop filtering enabling flag and an adaptive parameter set APS referred to by the adaptive loop filtering.

[0046] In one alternative embodiment, the identifying module includes an identifying unit for determining the adaptive loop filtering enabling flag in at least one syntax element of a sequence parameter set SPS, a picture parameter set PPS, a picture header PH, a slice header SH, and an encoding / decoding tree unit CTU of an encoded bitstream for generating the encoded video data of the reconstructed video unit.

[0047] According to still another embodiment of the present invention, there is further provided a computer-readable storage medium. A computer program is stored in the computer-readable storage medium. The computer program is configured to execute the steps in the embodiment of any one of the above methods when executed.

[0048] According to another embodiment of the present invention, an electronic device is further provided. The electronic device includes a memory and a processor, and a computer program is stored in the memory. The processor is configured to execute the computer program to perform the steps in the embodiment of any one of the above methods.

Brief Description of the Drawings

[0049]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Modes for Carrying Out the Invention

[0050] Hereinafter, with reference to the drawings, embodiments of the present invention will be described in detail while combining the embodiments.

[0051] It should be noted that terms such as "first" and "second" in the specification, claims, and the above drawings of the present invention are used to distinguish similar objects and are not necessarily for describing a specific order or sequence.

[0052] First, the related technology of the present invention will be described.

[0053] The next-generation video coding standard H.266 / VVC (Versatile Video Coding, also known as multi-functional video coding, an alias for H.266) adopts a block-based hybrid coding framework. Distortion effects such as block effects, ringing effects, color deviation, and image blurring still exist in the compressed video that adopts the H.266 / VVC standard. To reduce the impact of such distortion on video quality, H.266 / VVC adopts in-loop filtering techniques including luminance mapping with chroma scaling (abbreviated as LMCS), deblocking filtering (abbreviated as DBF), sample adaptive offset (abbreviated as SAO), and adaptive loop filtering (abbreviated as ALF). LMCS improves compression efficiency by reassigning codewords to information within the dynamic range. DBF is used to reduce block effects. SAO is used to improve ringing effects. ALF can reduce decoding errors. The H.266 / VVC in-loop filtering flow is shown in Figure 1. Figure 1 is an in-loop filtering flowchart in the H.266 / VVC standard in the related art. Following the release of H.266 / VVC, video coding based on NN has become the focus of next-generation video coding research. Among the in-loop filtering modules, the NN filtering module may be added to in-loop filtering as a new module or may replace the original DBF and SAO modules. This can not only save transmission overhead but also improve image quality. Generally, in a codec, specifically as shown in Figure 2, even after filtering based on NN, the ALF filter is used to further improve image performance. In Figure 2, the "slice-level ALF switch" is set, and this switch is mainly used to control whether to turn on the luminance ALF of the current slice.Here, FIG. 2 is a flowchart of NN filtering and ALF filtering in the related art.

[0054] According to one embodiment of the present invention, the operation steps on the encoding side are as follows.

[0055] (1) When the NN filtering of the Sequence parameter set (abbreviated as SPS) layer is turned on, NN filtering is performed on the reconstructed image after LMCS according to the following steps a to c. When it is turned off, filtering operations are performed according to the conventional filtering flow (DBF, SAO).

[0056] a. Perform NN filtering in advance. b. Perform switch determination at the slice / block (the block may refer to the CTU block or the NN filtering block) level of NN filtering by Rate Distortion Optimization (abbreviated as RDO), c. Obtain the reconstructed image after NN filtering.

[0057] (2) Perform ALF filtering operations.

[0058] a. Classify the luminance component, and the chroma does not require classification. b. Calculate a new APS filtering set for each of luminance and chroma. c. Perform switch determination and filtering set selection at the Slice / CTU level for luminance. d. Perform the ALF switch determination and filtering set selection for the current slice / CTU. First, perform the ALF determination for the luminance. If the luminance ALF is turned on after the determination, turn on the ALF for the current slice, and continue the switch determination and filtering set selection at the chroma slice / CTU level. If the luminance ALF is turned off after the determination, turn off the ALF for the current slice, the chroma ALF will be in the off state by default, and the switch determination and filtering set selection at the chroma slice / CTU level will not be performed.

[0059] e. Obtain the reconstructed image after ALF filtering.

[0060] (3) Perform the CCALF filtering operation. If the ALF is on for the current slice, perform the CCALF filtering operation according to the following steps a - c, and output the final reconstructed frame. If the ALF is off for the current slice, CCALF will be in the off state by default, do not perform the CCALF operation, and directly output the result of the previous step as the final reconstructed frame.

[0061] a. Calculate the CCALF filtering set for the chroma. b. Perform the switch determination and filtering set selection at the slice / CTU level. c. Obtain the reconstructed image after CCALF filtering.

[0062] According to one embodiment of the present invention, the operation steps of the decoder - side filtering are as follows.

[0063] (1) If the NN filtering in the SPS layer is on, perform the NN filtering on the reconstructed image after LMCS according to the following steps a - d. If the NN filtering in the SPS layer is off, perform the filtering operation according to the conventional filtering flow (DBF, SAO).

[0064] a. Obtain the slice / CTU-level switch determination of NN filtering from the code stream, b. According to the switch of the slice, if it is on, continue to obtain the switch of the CTU and proceed to step c. If it is off, do not perform the NN filtering operation and proceed to step d. c. Sequentially obtain the switch information of each CTU from the code stream. If the current CTU switch information is on, perform the NN operation. If it is off, directly output the reconstructed picture. d. Obtain the reconstructed image after NN filtering.

[0065] (2) When the ALF filtering of the SPS layer is on, perform the ALF filtering process on the reconstructed image in (1) according to the following steps a~d. When the ALF filtering of the SPS layer is off, directly output the reconstructed image in (1).

[0066] a. Classification: Classify the luminance component, and there is no need to classify the chroma.

[0067] b. Obtain the ALF switch determination of the current slice from the code stream. First, obtain the ALF switch information of the current slice from the code stream. If it is on, the luminance supports filtering using ALF, and continue to obtain the slice-level ALF switch information of the first chroma component and the second chroma component respectively. If it is on, the corresponding chroma component supports filtering using ALF. When the ALF switch information of the current slice is off, this slice does not support luminance ALF, chroma ALF, and CCALF.

[0068] c. When the current slice supports the use of ALF, sequentially obtain the ALF switch determination of each component in each CTU from the code stream. If the switch is on, perform the ALF filtering operation on the component corresponding to the CTU.

[0069] (3) Perform the ALF filtering operation to obtain the reconstructed image.

[0070] a. Obtain the CCALF switch determination for the current slice / CTU from the code stream. If the ALF of the current slice is on, continue to obtain the slice-level CC ALF switch information for the first chroma component and the second chroma component respectively. If it is on, the corresponding chroma component supports filtering using CCALF. When the ALF of the current slice is off, CCALF is defaulted to the off state and the information of CCALF is no longer obtained.

[0071] b. If the current slice supports the use of CC ALF, sequentially obtain the CC ALF switch determination for each component in each CTU from the code stream. If the switch is on, perform the CC ALF filtering operation on the component corresponding to the CTU.

[0072] c. Perform CCALF filtering to obtain the reconstructed image.

[0073] As can be seen from the above content, the luminance ALF after NN filtering is in the off state. However, when the luminance ALF is off, the chroma ALF and CCALF are also forced to be off, so the image quality after processing may be relatively low.

[0074] Regarding the above problems in the related art, in the embodiments of the present disclosure, it is proposed that in the ALF module after NN filtering, the luminance ALF and the chroma ALF are separated, and even when the luminance ALF is off, the chroma ALF and CCALF are performed. Since NN filtering significantly enhances the luminance, in many cases, the luminance ALF after NN filtering is in the off state. However, when the luminance ALF is off, the chroma ALF and CCALF are also forced to be off. Therefore, by separating the luminance and the chroma, the needs for chroma filtering can be better met, and the chroma performance can be further improved.

[0075] Hereinafter, in conjunction with the embodiments, it will be described how the present invention solves the above problems in the related art.

[0076] Examples of the method provided in the embodiments of the present application may be executed on a mobile terminal, a computer terminal, or a similar computing device. Taking the execution on a mobile terminal as an example, FIG. 3 is a hardware configuration block diagram of the mobile terminal used in the video processing method according to the embodiments of the present invention. As shown in FIG. 3, the mobile terminal may include one or more (only one is shown in FIG. 3) processors 302 (the processor 302 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), and a memory 304 for storing data. Here, the mobile terminal may further include a transmission device 306 for communication functions and an input / output device 308. As can be understood by those skilled in the art, the configuration shown in FIG. 3 is only schematic and does not limit the configuration of the mobile terminal. For example, the mobile terminal may further include more or fewer components than those shown in FIG. 3, or may have a configuration different from that shown in FIG. 3.

[0077] The memory 304 may be used to store computer programs, such as software programs and modules of application software, for example, computer programs corresponding to the methods for video processing in the embodiments of the present invention. The processor 302 executes the computer programs stored in the memory 304 to execute various functional applications and data processing, that is, to implement the above method. The memory 304 may include a high-speed random access memory, or may include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid memories. In some examples, the memory 304 may further include a memory installed remotely with respect to the processor 302. These remote memories may be connected to the mobile terminal via a network. Examples of the above network include, but are not limited to, the Internet, enterprise intranets, local area networks, mobile communication networks, and combinations thereof.

[0078] The transmission device 306 is used to receive or transmit data via a network. Specific examples of the above network may include a wireless network provided by a communication vendor of a mobile terminal. In one example, the transmission device 306 includes a network interface controller (abbreviated as NIC) that can communicate with the Internet by connecting to other network devices via a base station. In one example, the transmission device 306 may be a radio frequency (RF) module configured to communicate with the Internet in a wireless manner.

[0079] In this embodiment, a method for video processing is provided. FIG. 4 is a flowchart of a method for video processing according to an embodiment of the present invention. As shown in FIG. 4, this flow includes the following steps.

[0080] In step S402, determine the neural network loop filtering activation flag of the reconstructed video unit, In step S404, based on the neural network loop filtering activation flag, set the adaptive loop filtering activation flag for the reconstructed video unit, In step S406, notify at least one of the filter information related to the neural network loop filtering of the reconstructed video unit and the filter information related to the adaptive loop filtering of the reconstructed video unit by signaling.

[0081] Here, the one that executes the above operations may be a device on the video encoding side, such as a processor, a controller, or a device with similar processing capabilities.

[0082] In the above embodiment, the determined neural network loop filtering activation flag may determine the specific state of the neural network loop filtering activation flag or determine the specific value of the neural network loop filtering activation flag. Here, different states or different values correspond to different meanings. For example, when it is determined that the state of the neural network loop filtering activation flag is on or the value is 1 (of course, it may be other values, for example, the value is true, etc.), it indicates that it is necessary to execute neural network loop filtering. When it is determined that the state of the neural network loop filtering activation flag is off or the value is 0 (of course, it may be other values, for example, the value is false, etc.), it indicates that it is not necessary to execute neural network loop filtering. Similarly, setting the adaptive loop filtering activation flag actually means setting the state of the adaptive loop filtering activation flag or setting the specific value of the adaptive loop filtering activation flag, and different states or different values of the adaptive loop filtering activation flag also correspond to different meanings, which will not be further explained here.

[0083] Note that the operation of notifying the filter information related to the neural network loop filtering of the reconstructed video unit and / or the filter information related to the adaptive loop filtering of the reconstructed video unit is not necessarily in an inevitable sequential relationship with the operation of setting the adaptive loop filtering activation flag. It is possible to first set the adaptive loop filtering activation flag and then notify the filter information related to the neural network loop filtering and / or the filter information related to the adaptive loop filtering. Or, it is possible to first notify the filter information related to the neural network loop filtering and / or the filter information related to the adaptive loop filtering and then set the adaptive loop filtering activation flag. Of course, it is also possible to execute the operation of setting the adaptive loop filtering activation flag and the operation of notifying the filter information related to the neural network loop filtering and / or the filter information related to the adaptive loop filtering simultaneously.

[0084] In this embodiment, regardless of whether the neural network loop filtering and the adaptive loop filtering are activated, the operation of notifying the filter information related to the neural network loop filtering and / or the filter information related to the adaptive loop filtering may be executed. It should be noted that when the filter information related to the neural network loop filtering and / or the filter information related to the adaptive loop filtering is related to the operation results of S402 and S404, it is necessary to execute S406 after the execution of S402 and S404 is completed. The situation on the decoding side is similar.

[0085] According to the above embodiments, an adaptive loop filtering enable flag can be set for the reconstructed video unit based on the neural network loop filtering enable flag of the reconstructed video unit. Therefore, adaptive loop filtering can be executed based on the setting result, and by directly turning off the adaptive loop filtering after performing neural network loop filtering, further luminance and / or chroma processing cannot be performed on the image, and the problem that the image quality cannot be optimized is avoided, achieving the effect of improving the image quality.

[0086] In one alternative embodiment, the filter information related to the adaptive loop filtering includes an adaptive loop filtering enable flag and an adaptive parameter set referred to by the adaptive loop filtering. In this embodiment, the adaptive loop filtering enable flag may be determined by S404. The adaptive parameter set referred to by the above adaptive loop filtering may include an APS related to chroma and / or an adaptive parameter set APS related to luminance. For the specific calculation method and selection policy of the APS related to chroma and / or the APS related to luminance, refer to the specific description in the subsequent specific embodiments.

[0087] In one alternative embodiment, the adaptive loop filtering enable flag includes a luminance adaptive loop filtering enable flag and a chroma adaptive loop filtering enable flag. In this embodiment, the luminance adaptive loop filtering enable flag and the chroma adaptive loop filtering enable flag may be set respectively based on the neural network loop filtering enable flag. That is, the activation of the luminance adaptive loop filtering has no inevitable relationship with the activation of the chroma adaptive loop filtering. By setting luminance and chroma separately, the needs for chroma filtering can be better met, and the chroma performance can be further improved.

[0088] In one alternative embodiment, the chroma adaptive loop filtering enabling flag includes at least one of a first chroma adaptive loop filtering enabling flag, a second chroma adaptive loop filtering enabling flag, a first inter-chroma component adaptive loop filtering enabling flag, and a second inter-chroma component adaptive loop filtering enabling flag. In this embodiment, the chroma adaptive filtering enabling flag may be further subdivided into adaptive loop filtering enabling flags for various types of chroma. Here, the first chroma adaptive loop filtering enabling flag may refer to an adaptive loop filtering enabling flag for the blue difference Cb chroma component. The second chroma adaptive loop filtering enabling flag may refer to an adaptive loop filtering enabling flag for the red difference Cr chroma component. The first inter-chroma component adaptive loop filtering enabling flag may refer to a Cb CCALF enabling flag. The second inter-chroma component adaptive loop filtering enabling flag may refer to a Cr CCALF enabling flag.

[0089] In one alternative embodiment, the adaptive loop filtering enabling flag is signaled in at least one syntax element of a sequence parameter set SPS, a picture parameter set PPS, a picture header PH, a slice header SH, and an encoding / decoding tree unit CTU for generating the encoded video data of the reconstructed video unit. Specific descriptions of each syntax element will be given below.

[0090] According to one embodiment of the present invention, the neural network loop filtering enabling flag and the adaptive loop filtering enabling flag are signaled at the SPS level of the encoded bitstream.

[0091] Table 1 is a syntax table at the SPS level of the encoded bitstream. Specifically, it is shown as follows (it should be noted that in each subsequent table, the values of each identifier are only for illustrative purposes and may take other types of values in actual applications. For example, "true" and "false" can replace "1" and "0", or other types of characters or symbols can replace "1" and "0", etc.).

[0092]

Table 1

[0093] The semantic descriptions of the related syntax fields in Table 1 are as follows. Here,[[]]END]] sps_nn_enabled_flag: The NNF filtering enabling flag at the SPS level is used to indicate whether to turn on the NNF filtering at the SPS level. A value of 1 indicates turning it on, and a value of 0 indicates not turning it on.

[0094] sps_alf_enabled_flag: The ALF filtering enabling flag at the SPS level is used to indicate whether to turn on the ALF filtering at the SPS level. A value of 1 indicates turning it on, and a value of 0 indicates not turning it on.

[0095] sps_ccalf_enabled_flag: The ALF filtering enabling flag at the SPS level is used to indicate whether to turn on the CC-ALF filtering at the SPS level. A value of 1 indicates turning it on, and a value of 0 indicates not turning it on.

[0096] According to one embodiment of the present invention, at the PPS level of the encoded bitstream, the positions of the filter information related to the neural network loop filtering and the positions of the filter information related to the adaptive loop filtering are signaled.

[0097] Table 2 is the syntax table at the PPS level of the encoded bitstream. Specifically, it is shown as follows.

[0098]

Table 2

[0099] The semantic description of the related syntax fields in Table 2 is as follows. Here, pps_nnf_info_in_ph_flag: Indicates whether the NNF filtering information appears at the PH level. If it is equal to 1, it means that the NNF filtering information is at the PH level. If it is equal to 0, it means that the NNF filtering information is at the SH level. The decoder can determine the NNF activation flag and the corresponding filter information for each component by analyzing the related syntax elements at the PH level or the SH level.

[0100] pps_alf_info_in_ph_flag: Indicates whether the ALF filtering information appears at the PH level. If it is equal to 1, it means that the ALF filtering information is at the PH level. If it is equal to 0, it means that the ALF filtering information is at the SH level. The decoder can determine the ALF activation flag and the corresponding filter information for each component by analyzing the related syntax elements at the PH level or the SH level.

[0101] According to one embodiment of the present invention, the filter information related to neural network loop filtering and the filter information related to adaptive loop filtering are signaled at the PH level of the encoded bitstream.

[0102] Table 3 is the syntax table at the PH level of the encoded bitstream. Specifically, it is shown as follows.

[0103]

Table 3

[0104] The semantic descriptions of the related syntax fields in Table 3 are as follows.

[0105] Here, when pps_alf_info_in_ph_flag is equal to 1, it indicates that the ALF filtering information appears at the PH level. The decoder analyzes the alf_split_structure( ) syntax configuration or the alf_structure( ) syntax configuration at the PH level to determine the ALF activation flag of each component and the corresponding filter information.

[0106] According to one embodiment of the present invention, filter information related to neural network loop filtering and filter information related to adaptive loop filtering are signaled at the SH level of the encoded bitstream.

[0107] Table 4 is a syntax table at the SH level of the encoded bitstream. Specifically, it is shown as follows.

[0108] [Table 4]

[0109] The semantic descriptions of the related syntax fields in Table 4 are as follows. Here, when pps_alf_info_in_ph_flag is equal to 0, it indicates that the ALF filtering information appears at the SH level. The decoder analyzes the alf_split_structure( ) syntax configuration or the alf_structure( ) syntax configuration at the SH level to determine the ALF activation flag of each component and the corresponding filter information.

[0110] The nn_structure( ) syntax composition included in the PH level or SH level (i.e., nn_structure( ) in Table 3 and nn_structure( ) in Table 4) is specifically as shown in Table 5.

[0111]

Table 5

[0112] Here, The neural network loop filtering enabling flag in the nn_structure( ) syntax composition includes at least one of the luminance component neural network loop filtering enabling flag (nnf_luma_enabled_flag), the first chroma component neural network loop filtering enabling flag (nnf_cb_enabled_flag), and the second chroma component neural network loop filtering enabling flag (nnf_cr_enabled_flag). nn_luma_enabled_flag: Identifies whether the current image / slice luminance component is using NN. A value of 1 indicates using NN, and a value of 0 indicates not using it.

[0113] nn_cb_enabled_flag: Identifies whether the current image / slice luminance component is using NN. A value of 1 indicates using NN, and a value of 0 indicates not using it.

[0114] nn_cr_enabled_flag: Identifies whether the current image / slice luminance component is using NN. A value of 1 indicates using NN, and a value of 0 indicates not using it.

[0115] According to one embodiment of the present invention, when the NNF filtering enabling flag (sps_nn_enabled_flag) set at the SPS level of the encoded bitstream instructs to perform NNF filtering at the SPS level, the alf_split_structure( ) syntax configuration for setting the luma adaptive loop filtering enabling flag and the chroma adaptive loop filtering enabling flag at the PH level or SH level of the encoded bitstream is included at the PH level or SH level of the encoded bitstream, respectively.

[0116] The adaptive loop filtering enabling flags in the alf_split_structure( ) syntax configuration are the luma component adaptive loop filtering enabling flag (alf_luma_enabled_flag), the first chroma adaptive loop filtering enabling flag (alf_cb_enabled_flag), the second chroma adaptive loop filtering enabling flag (alf_cr_enabled_flag), the first chroma component - to - component adaptive loop filtering enabling flag (alf_cc_cb_enabled_flag), and include at least one of the second chroma component - to - component adaptive loop filtering enabling flag (alf_cc_cr_enabled_flag).

[0117] The alf_split_structure( ) syntax configuration included at the PH level or SH level is specifically as shown in Table 6.

[0118]

Table 6

[0119] The semantic explanations of the related syntax fields in Table 6 are as follows. Here, alf_luma_enabled_flag: The ALF filtering enable flag for the luminance component at the PH level or SH level is used to indicate whether to turn on the luminance component ALF filtering at the PH level or SH level. A value of 1 indicates turning on, and a value of 0 indicates not turning on.

[0120] num_alf_aps_ids_luma: Identifies the number of subsets for which the current picture / slice luminance ALF uses APS filtering. alf_aps_id_luma[ i ]: Identifies the ID number of the i-th luminance APS filtering subset.

[0121] alf_cb_enabled_flag: The ALF filtering enable flag for the first chroma (i.e., Cb blue difference) component at the PH level or SH level is used to indicate whether to turn on the ALF filtering for the first chroma (i.e., Cb blue difference) component at the PH level or SH level. A value of 1 indicates turning on, and a value of 0 indicates not turning on.

[0122] alf_cr_enabled_flag: The ALF filtering enable flag for the second chroma (i.e., Cr red difference) component at the PH level or SH level is used to indicate whether to turn on the ALF filtering for the first chroma (i.e., Cr red difference) component at the PH level or SH level. A value of 1 indicates turning on, and a value of 0 indicates not turning on.

[0123] alf_aps_id_chroma: Identifies the ID for which the current picture / slice chroma ALF uses the APS filter.

[0124] alf_cc_cb_enabled_flag: The ALF filtering enable flag between the first chroma components at the PH level or SH level (i.e., with respect to the luminance for the Cb blue difference) is used to indicate whether to turn on the ALF filtering between the first chroma components at the PH level or SH level (i.e., with respect to the luminance for the Cb blue difference). A value of 1 indicates to turn it on, and a value of 0 indicates not to turn it on.

[0125] alf_cc_cb_aps_id: Identifies the number of the CC-ALF filter used by the current image / slice chroma Cb component.

[0126] alf_cc_cr_enabled_flag: The ALF filtering enable flag between the second chroma components at the PH level or SH level (i.e., with respect to the luminance for the Cr red difference) is used to indicate whether to turn on the ALF filtering between the second chroma components at the PH level or SH level (i.e., with respect to the luminance for the Cr red difference). A value of 1 indicates to turn it on, and a value of 0 indicates not to turn it on.

[0127] alf_cc_cr_aps_id: Identifies the number of the CC-ALF filter used by the current image / slice chroma Cr component.

[0128] According to one embodiment of the present invention, when the NNF filtering enable flag (sps_nn_enabled_flag) set at the SPS level of the encoded bitstream indicates not to perform NNF filtering at the SPS level, the PH level or SH level of the encoded bitstream includes the picture_header_structure( ) and slice_header( ) syntax structures defined in the H.266 / VVC standard.

[0129] According to one embodiment of the present invention, filter information related to neural network loop filtering and filter information related to adaptive loop filtering are notified by signaling at the CTU level of the encoded bitstream.

[0130] Table 7 is a syntax table at the CTU level and is specifically shown as follows.

[0131] [Table 7]

[0132] The semantic explanations of the related syntax fields in Table 7 are as follows.

[0133] nn_ctb_flag

[0000] [ CtbAddrX ][ CtbAddrY ]: Identifies the CTB luma neural network loop filtering enabling flag. nn_ctb_flag

[0001] [ CtbAddrX ][ CtbAddrY ]: Identifies the CTB first chroma neural network loop filtering enabling flag. nn_ctb_flag

[0002] [ CtbAddrX ][ CtbAddrY ]: Identifies the CTB second chroma neural network loop filtering enabling flag. However, sps_nn_enabled_flag corresponds to the NNF filtering enabling flag at the SPS level shown in Table 1 and is used to indicate whether to turn on NNF filtering at the SPS level. Note that a value of 1 indicates turning on, and a value of 0 indicates not turning on.

[0134] According to one embodiment of the present invention, when the NNF filtering enabling flag (sps_nn_enabled_flag) set at the SPS level of the encoded bitstream instructs to perform NNF filtering at the SPS level, at the CTU level of the encoded bitstream, an alf_split_ctb_structure( ) syntax structure for setting the luma adaptive loop filtering enabling flag and the chroma adaptive loop filtering enabling flag at the CTB level of the encoded bitstream is included.

[0135] The adaptive loop filtering enabling flag in the alf_split_ctb_structure( ) syntax structure is the luma component adaptive loop filtering enabling flag (alf_luma_enabled_flag), and the first chroma adaptive loop filtering enabling flag (alf_cb_enabled_flag), and includes at least one of the second chroma adaptive loop filtering enabling flag (alf_cr_enabled_flag). The alf_split_ctb_structure( ) syntax structure included at the CTU level is specifically as shown in Table 8.

[0136]

Table 8

[0137] The semantic description of the related syntax fields in Table 8 is as follows. Here, alf_luma_enabled_flag: The luma component ALF filtering enabling flag at the CTB level is used to indicate whether to turn on the luma component ALF filtering at the CTB level. A value of 1 indicates turning on, and a value of 0 indicates not turning on. Here, the purpose of introducing the luma main switch is realized by introducing luma-related elements. alf_cb_enabled_flag: The first chroma component ALF filtering enable flag at the CTB level is used to indicate whether to turn on the first chroma component ALF filtering at the CTB level. A value of 1 indicates turning it on, and a value of 0 indicates not turning it on. alf_cr_enabled_flag: The second chroma component ALF filtering enable flag at the CTB level is used to indicate whether to turn on the second chroma component ALF filtering at the CTB level. A value of 1 indicates turning it on, and a value of 0 indicates not turning it on.

[0138] In the above alf_split_ctb_structure( ) syntax structure, alf_ctb_flag

[0000] [ CtbAddrX ][ CtbAddrY ]: Identifies whether this CTB uses luma ALF filtering, alf_ctb_flag

[0001] [ CtbAddrX ][ CtbAddrY ]: Identifies whether this CTB uses the first chroma ALF filtering, alf_ctb_flag

[0002] [ CtbAddrX ][ CtbAddrY ]: Identifies whether this CTB uses the second chroma ALF filtering.

[0139] According to one embodiment of the present invention, when the NNF filtering enable flag (sps_nn_enabled_flag) set at the SPS level of the encoded bitstream indicates not to perform NNF filtering at the SPS level, the CTU level of the encoded bitstream includes the coding_tree_unit( ) syntax configuration defined in the H.266 / VVC standard.

[0140] In one alternative embodiment, when the neural network loop filtering enabling flag instructs to perform neural network loop filtering on the reconstructed video unit at the sequence level, image level, or slice level, a luminance adaptive loop filtering enabling flag and a chroma adaptive loop filtering enabling flag are respectively set for the reconstructed video unit. In this embodiment, setting the luminance adaptive loop filtering enabling flag and the chroma adaptive loop filtering enabling flag actually means setting the states of the adaptive loop filtering enabling flag for luminance and the adaptive loop filtering enabling flag for chroma. When the NN filtering of the SPS layer is on, the following operations a - c can be performed on the reconstructed image after LMCS for NN filtering. When the NN filtering of the SPS layer is off, the filtering operation is performed according to the conventional filtering flow (DBF, SAO).

[0141] a. Perform NN filtering in advance, b. Perform switch determination at the slice e / block level (the block may refer to the CTU block or the NN filtering block) of the NN filtering by RDO, c. Reconstruct the output after NN filtering.

[0142] In one alternative embodiment, the method further includes setting a luma adaptive loop filtering enable flag and a chroma adaptive loop filtering enable flag for the reconstructed video unit when the neural network loop filtering enable flag instructs to perform neural network loop filtering on the reconstructed video unit at the sequence level and the adaptive loop filtering enable flag instructs not to perform luma adaptive loop filtering on the reconstructed video unit at the picture level or slice level. In this embodiment, setting the luma adaptive loop filtering enable flag and the chroma adaptive loop filtering enable flag actually means setting the states of the adaptive loop filtering enable flag for luma and the adaptive loop filtering enable flag for chroma. In this embodiment, when it is determined that it is necessary to perform neural network loop filtering and it is determined that it is not necessary to perform luma adaptive loop filtering, one chroma adaptive loop filtering enable flag update operation may be introduced. That is, in this case, the luma adaptive loop filtering enable flag and the chroma adaptive loop filtering enable flag for the reconstructed video unit may be set respectively. That is, when the NNF filtering enable flag at the current SPS level is on and the ALF filtering enable flag at the current PH level / SH level is off, the update operation of the chroma adaptive loop filtering enable flag is triggered.

[0143] The following describes specifically how to set the chroma adaptive loop filtering enable flag. Specifically, reference can be made to the update operation shown in FIG. 7. In this embodiment, what is actually mainly included in the update is the transmission of the chroma ALF flag. This embodiment mainly changes the code stream transmission method, and the encoding end is the same as that in the previous embodiment. However, during transmission, the transmission parameters are adaptively adjusted based on NN. The overall flow after adding the update operation can be referred to FIG. 6. Here, the specific flow of the update operation can be referred to FIG. 7. In FIGS. 6 and 7, it is possible to determine whether to turn on each switch based on the value of the field corresponding to each switch. For example, when the field value corresponding to the switch is 1, it can be confirmed to turn on the switch, and when the field value corresponding to the switch is 0, it can be confirmed to turn off the switch. The operations of the switches related to the following other drawings are all similar, that is, when the field value corresponding to the switch is 1, it is confirmed to turn on the switch, and when the field value corresponding to the switch is 0, it is confirmed to turn off the switch. This will not be further described below.

[0144] After adding the update operation, the syntax semantics at the PH level, SH level, and CTU level are adjusted accordingly.

[0145] According to one embodiment of the present invention, the NNF filtering activation flag (sps_nn_enabled_flag) set at the SPS level of the encoded bitstream instructs to perform NNF filtering at the SPS level. However, when the ALF filtering activation flag (alf_enabled_flag) at the PH level or SH level of the encoded bitstream instructs not to perform ALF filtering at the PH level or SH level, at the PH level or SH level of the encoded bitstream, the chroma adaptive loop filtering activation flag is updated at the PH level or SH level of the encoded bitstream, that is, the alf_update_structure( ) syntax structure for setting the luminance adaptive loop filtering activation flag and the chroma adaptive loop filtering activation flag respectively is included.

[0146] According to one embodiment of the present invention, filter information related to neural network loop filtering and filter information related to adaptive loop filtering are signaled at the PH level of the encoded bitstream. Table 9 is a syntax structure table of picture_header_structure( ) included at the PH level after adding the update operation, and is specifically shown as follows.

[0147]

Table 9

[0148] Here, the above alf_update_structure( ) is an element newly added to the PH level syntax structure and is used to update the activation flag of chroma ALF filtering. When the conventional adaptive filtering activation flag (alf_enabled_flag) in the PH level syntax structure is equal to 0, it indicates that it is necessary to execute the update operation of the chroma adaptive loop filtering activation flag.

[0149] According to one embodiment of the present invention, filter information related to neural network loop filtering and filter information related to adaptive loop filtering are notified by signaling at the SH level of the encoded bitstream. Table 10 is a slice_header( ) syntax structure table included at the SH level after adding an update operation, and is specifically shown as follows.

[0150]

Table 10

[0151] Here, the above alf_update_structure( ) is an element newly added to the SH level syntax structure and is used to update the enable flag of chroma ALF filtering. When the adaptive filtering enable flag (alf_enabled_flag) in the SH level syntax structure is equal to 0, it indicates that it is necessary to execute an update operation of the chroma adaptive loop filtering enable flag.

[0152] The alf_update_structure( ) syntax structure included in the PH level or the SH level is specifically as shown in Table 11.

[0153]

Table 11

[0154] According to one embodiment of the present invention, the NNF filtering enabling flag (sps_nn_enabled_flag) set at the SPS level of the encoded bitstream instructs to perform NNF filtering at the SPS level. However, when the ALF filtering enabling flag (alf_enabled_flag) at the PH level or SH level of the encoded bitstream instructs not to perform ALF filtering at the PH level or SH level, at the CTU level of the encoded bitstream, the chroma adaptive loop filtering enabling flag is updated at the CTU level of the encoded bitstream, that is, an alf_update_structure( ) syntax structure for setting the luma adaptive loop filtering enabling flag and the chroma adaptive loop filtering enabling flag respectively is included.

[0155] Table 12 is a coding_tree_unit( ) syntax structure table included at the CTU level after adding an update operation, and is specifically shown as follows.

[0156]

Table 12

[0157] Here, the above alf_update_ctb_structure( ) is an element newly added to the CTU level syntax structure and is used to update the enabling flag of chroma ALF filtering. When the adaptive filtering enabling flag (alf_enabled_flag) is equal to 0 in the PH level or SH level syntax structure, it indicates that it is necessary to perform the update operation of the chroma adaptive loop filtering enabling flag.

[0158] According to one embodiment of the present disclosure, filter information related to adaptive loop filtering may be notified by signaling at the CTB level of the encoded bitstream, that is, the notification of filtering information related to adaptive loop filtering may be realized by the related structure at the CTB level.

[0159] Here, the alf_update_ctb_structure( ) element included in the CTB level is the switch information at the CTB level for instructing the transmission of chroma ALF. The semantic meaning of this element is specifically as shown in Table 13.

[0160]

Table 13

[0161] Here, alf_ctb_flag[1] is used to indicate the activation flag of the first chroma adaptive loop filtering of this CTB, alf_ctb_flag[2] is used to indicate the activation flag of the second chroma adaptive loop filtering of this CTB, alf_ctb_filter_alt_idx[0] is used to indicate the filter ID used by this first chroma adaptive network loop filtering, alf_ctb_filter_alt_idx[1] is used to indicate the filter ID used by this second chroma adaptive network loop filtering.

[0162] In one alternative embodiment, the method further includes determining whether to perform chroma adaptive loop filtering on the reconstructed video unit based on the chroma adaptive loop filtering enable flag. In this embodiment, the operations corresponding to different chroma adaptive loop filtering enable flags are different. When the value of the chroma adaptive loop filtering enable flag is 1, it can indicate that chroma adaptive loop filtering needs to be performed on the reconstructed video unit. When the value of the chroma adaptive loop filtering enable flag is 0, it can indicate that there is no need to perform chroma adaptive loop filtering on the reconstructed video unit. Of course, setting the values to 1 and 0 is only for illustrative purposes.

[0163] In one alternative embodiment, setting the luma adaptive loop filtering enable flag and the chroma adaptive loop filtering enable flag respectively for the reconstructed video unit includes setting a first flag related to the adaptive loop filter for the luma component for the reconstructed video unit. In the original VVC standard, the enabling of luma ALF depends on the ALF master switch. When the master switch is on, luma ALF is always used. When the master switch is off, luma ALF is turned off and chroma is also turned off. Since this is not reasonable, a new luma ALF switch can be added to alf_structure( ) or a single main switch can be added for chroma. This embodiment mainly changes the code stream transmission method. The encoding side is the same as the previous embodiment. However, during transmission, the transmission parameters are adaptively adjusted based on NN. In this embodiment, the modified alf_structure( ) and alf_ctb_structure( ) are mainly described. In this embodiment, an element for representing the luma main switch may be added to alf_structure( ), or an element for representing the luma main switch at the CTB level may be added to alf_ctb_structure( ). For the transmission flow of the picture / slice layer after adding the luma switch in this embodiment, reference may be made to FIG. 9.

[0164] The following respectively describes the modified alf_structure( ) and the modified alf_ctb_structure( ).

[0165] For the syntax semantics of the modified alf_structure( ), reference may be made to Table 14.

[0166] [Table 14]

[0167] Here, the alf_luma_enabled_flag is a newly added luminance main switch and is used to indicate whether to turn on the luminance ALF. Other elements in the above alf_structure( ) are consistent with the elements in the alf_structure() syntax structure defined in the conventional H.266 / VVC.

[0168] For the syntax semantics of the modified alf_ctb_structure( ), refer to Table 15.

[0169]

Table 15

[0170] Here, the alf_luma_enabled_flag is a newly added CTB-level luminance main switch.

[0171] It should be noted that here, the alf_enabled_flag may or may not be transmitted. When not transmitted, the alf_enabled_flag takes values in the following manner.

[0172] alf_enabled_flag = alf_luma_enabled_flag || alf_cb_enabled_flag || alf_cr_enabled_flag || alf_cc_cb_enabled_flag || alf_cc_cr_enabled_flag.

[0173] In one alternative embodiment, setting the luma adaptive loop filtering enable flag and the chroma adaptive loop filtering enable flag respectively for the reconstructed video unit includes setting a second flag related to the adaptive loop filter for the chroma component for the reconstructed video unit. In this embodiment, mainly the addition of the chroma main switch will be described. In this embodiment, an element for representing the chroma main switch may be added to alf_structure( ), or an element for representing the chroma main switch at the CTB level may be added to alf_ctb_structure( ). Here, for the transmission flow of the picture / slice layer after adding the chroma main switch, refer to FIG. 10. Note that after adding the chroma main switch, alf_enabled_flag acts only on the luma. The syntax semantics are as shown in Tables 18 to 19.

[0174] Hereinafter, the modified alf_structure( ) and the modified alf_ctb_structure( ) will be described respectively.

[0175] The syntax semantics of the modified alf_structure( ) are as shown in Table 16.

[0176]

Table 16

[0177] Here, alf_chroma_enabled_flag is a newly added chroma main switch and is used to indicate whether to turn on the chroma ALF. Other elements in the above alf_structure( ) are consistent with the elements in the alf_structure() syntax structure defined in the conventional H.266 / VVC.

[0178] The syntax semantics of the modified alf_ctb_structure( ) are as shown in Table 17.

[0179]

Table 17

[0180] Here, the alf_chroma_enabled_flag is a newly added CTB-level chroma main switch.

[0181] It should be noted that here, the alf_enabled_flag may or may not be transmitted. When not transmitted, the alf_enabled_flag takes a value in the following manner.

[0182] alf_enabled_flag = alf_luma_enabled_flag || alf_cb_enabled_flag || alf_cr_enabled_flag || alf_cc_cb_enabled_flag || alf_cc_cr_enabled_flag.

[0183] In an alternative embodiment, when the value of the adaptive loop filtering enabling flag is a first value, the adaptive loop filtering enabling flag is used to instruct to perform adaptive loop filtering on the reconstructed video unit. When the value of the adaptive loop filtering enabling flag is a second value, the adaptive loop filtering enabling flag is used to instruct not to perform adaptive loop filtering on the reconstructed video unit. Here, the first value is different from the second value. In this embodiment, the above first value may be "1" or "true" or other values. The above second value may be "0" or "false" or other values.

[0184] In one alternative embodiment, performing adaptive loop filtering on the reconstructed video unit includes at least one of the following operations: performing luminance adaptive loop filtering on the luminance component of the reconstructed video unit; performing first chroma adaptive loop filtering on the first chroma component of the reconstructed video unit; performing second chroma adaptive loop filtering on the second chroma component of the reconstructed video unit; performing first inter-chroma component adaptive loop filtering on the first chroma component of the reconstructed video unit; and performing first inter-chroma component adaptive loop filtering on the second chroma component of the reconstructed video unit. In this embodiment, the luminance adaptive loop filtering is Luma ALF, the first chroma adaptive loop filtering is Cb ALF, the second chroma adaptive loop filtering is Cr ALF, the first inter-chroma component adaptive loop filtering is Cb CCALF, and the second inter-chroma component adaptive loop filtering is Cr CCALF. Here, the first chroma component is the blue difference (Cb) chroma component, and the second chroma component is the red difference (Cr) chroma component.

[0185] In one alternative embodiment, when the neural network loop filtering enabling flag instructs to perform neural network loop filtering on the reconstructed video unit at the sequence level, image level, or slice level, a determination to perform luminance adaptive loop filtering on the luminance component of the reconstructed video unit, a determination to perform first chroma adaptive loop filtering on the first chroma component of the reconstructed video unit, a determination to perform second chroma adaptive loop filtering on the second chroma component of the reconstructed video unit, a determination to perform first inter-chroma component adaptive loop filtering on the first chroma component of the reconstructed video unit, and a determination to perform first inter-chroma component adaptive loop filtering on the second chroma component of the reconstructed video unit. The adaptive loop filtering enabling flag for the reconstructed video unit is set based on at least one of the determinations. In this embodiment, if the ALF-SPLIT on condition is satisfied, it is necessary to perform filtering determinations on the luminance component and chroma components respectively.

[0186] In one alternative embodiment, the filter information related to the neural network loop filtering includes a neural network loop filtering enabling flag and an adaptive parameter set referred to by the neural network loop filtering. In this embodiment, the operations performed with different states or different values of the neural network loop filtering enabling flag are different.

[0187] In one alternative embodiment, the neural network loop filtering enabling flag includes at least one of a luminance neural network loop filtering enabling flag, a first chroma neural network loop filtering enabling flag, and a second chroma neural network loop filtering enabling flag. In this embodiment, the luminance neural network loop filtering enabling flag, the first chroma neural network loop filtering enabling flag, and the second chroma neural network loop filtering enabling flag may be determined respectively, and then corresponding operations may be executed respectively based on the luminance neural network loop filtering enabling flag, the first chroma neural network loop filtering enabling flag, and the second chroma neural network loop filtering enabling flag. In this embodiment, the syntax structure for indicating the neural network loop filtering enabling flag may be as shown in Table 18.

[0188]

Table 18

[0189] Here, nn_luma_enabled_flag identifies the above-mentioned luminance neural network loop filtering enabling flag, nn_cb_enabled_flag identifies the above-mentioned first chroma neural network loop filtering enabling flag, nn_cr_enabled_flag identifies the above-mentioned second chroma neural network loop filtering enabling flag.

[0190] In one alternative embodiment, a neural network loop filtering enabling flag is signaled by signaling in at least one syntax element among a sequence parameter set SPS, a picture parameter set PPS, a picture header PH, a slice header SH, and a coding / decoding tree unit CTU for generating encoded video data of the reconstructed video unit. Since the description of each syntax element has already been specifically described in the foregoing embodiments, the description thereof is omitted here.

[0191] In one alternative embodiment, the reconstructed video unit corresponds to at least one of a video image, a video clip, a video pattern block, a slice, a coding / decoding tree unit CTU, and a coding / decoding unit CU.

[0192] The operations in the above embodiments are all descriptions of operations on the encoding side.

[0193] Hereinafter, the flow on the encoding side will be described in general while combining specific embodiments. FIG. 8 is a flowchart of an ALF-SPLIT operation based on NN filtering on the encoding side according to this specific embodiment, and specifically includes the following steps.

[0194] Step 1: NN filtering When the NN filtering of the SPS layer is on, perform operation NN filtering on the reconstructed image after LMCS according to the following a to c. When the NN filtering of the SPS layer is off, perform operation filtering according to the conventional filtering flow (DBF, SAO). Specifically, it includes the following sub-steps a to c.

[0195] a. Perform NN filtering in advance, b. Perform slice / block-level (the block may refer to a CTU block or an NN filtering block) switch determination for NN filtering by RDO, c. Reconstruct the output after NN filtering.

[0196] Below, each sub-step will be specifically described.

[0197] Regarding performing the NN filtering operation in advance in Step 1-a, After obtaining the reconstructed image of LMCS, first perform NN filtering.

[0198] First, construct each data that needs to be input into the network, such as reconstructed image samples, QP information, CU splitting information, deblocking filtering information, prediction samples, etc., and input this information into the network module to obtain the samples after passing through the NN process.

[0199] Regarding the slice / block-level NN filtering switch determination in Step 1-b, When the NN filtering of the channel corresponding to the slice is in the on state, calculate and compare the cost before NN filtering and the cost after NN filtering for each block among them to obtain the switch state of each block, and finally obtain the cost when the slice turns on NN filtering, When the slice does not use NN filtering, obtain the cost when the slice does not adopt NN filtering, Compare the cost before NN filtering and the cost after NN filtering for each channel of the current slice respectively to determine whether to turn on the slice-level NN filtering. If the slice turns on NN filtering, determine whether to turn on the NN filtering at each block level.

[0200] Regarding reconstructing the output after NN filtering in Step 1-c, Based on the slice / block level switch determination in Step 1-b, obtain the reconstructed image after NN filtering.

[0201] Step 2: Adaptive Loop Filtering (ALF) Perform adaptive loop filtering processing on the reconstructed image according to the following steps, specifically including the following sub-steps.

[0202] a. Classify the luminance component, and chroma does not require classification. b. Calculate a new APS filter set for luminance and chroma respectively. c. Perform ALF slice / ctu level filtering determination.

[0203] If the ALF-SPLIT on condition is satisfied, perform slice / CTU level switch determination and filter set selection for luminance and chroma respectively. If the ALF-SPLIT on condition is not satisfied, first perform slice / CTU level switch determination and filter set selection for luminance. If the luminance ALF is on after selection, set the ALF of the current slice to on, and continue with slice / CTU level switch determination and filter set selection for chroma. If the luminance ALF is off after determination, set the ALF of the current slice to off. By default, chroma ALF is off, and slice / CTU level switch determination and filter set selection for chroma are no longer performed.

[0204] d. Reconstruct the output after luminance and chroma ALF.

[0205] The following explains each sub-step.

[0206] In Step 2-a: Classification Luminance ALF Classification: Based mainly on the content characteristics of 4x4 pixel blocks, the filter type is selected as the classification result. The method steps for determining the filter class of 4x4 pixel blocks are as follows.

[0207] (1) Calculation of Laplacian gradient For each 4x4 luminance block, calculate the one-dimensional Laplacian gradient of each pixel in the 8x8 pixel block centered on this block in the horizontal 0°, vertical 90°, 135°, and 45° directions. To reduce the computational complexity, calculate the gradients in the horizontal 0°, vertical 90°, 135°, and 45° directions only for some pixel points where both the abscissa and ordinate are even or both are not even, and obtain g h , g v .

[0208] (2) Calculation of directionality coefficient D The directionality coefficient D represents the gradient direction. First, calculate the ratio of the maximum value to the minimum value of the gradients in the horizontal and vertical directions and the ratio of the maximum value to the minimum value of the gradients in the diagonal directions based on g h , g v , and then compare with the threshold values t1 = 2 and t2 = 4.5 to obtain the directionality coefficient.

[0209] (3) Calculation of activity coefficient A The activity coefficient A represents the strength of the gradient. It is obtained by means of table search based on g h , g v .

[0210] (4) Classification result The classification of the 4x4 small block is filtIdx = 5 * D + A.

[0211] Chrominance does not need to be classified.

[0212] Regarding step 2-b: Calculation of a new APS filtering set Luminance calculation APS filtering set method: The basic idea of constructing the APS filter is to divide the slice into 4x4 blocks, calculate the directionality and activity of each 4x4 block based on the gradient, and further obtain the filter class of each block. For each filter class, by treating all 4x4 blocks that adopt this type of filter as a whole and using the Wiener equation, the coefficients of this type of filter are obtained.

[0213] To improve the algorithm performance and reduce the computational complexity, it can be optimized as follows.

[0214] (1) Since the ALF cannot improve the compression performance for each CTU, the rate-distortion optimization policy is used to determine the CTUs that do not adopt the ALF, and then the CTUs that do not adopt the ALF are excluded and the filter coefficients are recalculated. This optimization policy may be repeated multiple times until it is optimal.

[0215] (2) The APS filtering coefficients need to be encoded and transmitted. By integrating and reducing 25 types of filters, the compression efficiency can be improved. After obtaining 25 sets of filtering coefficients, if the reduction in compression performance is limited when one set of filter coefficients is adopted for adjacent filter classes, the adjacent filter classes are merged into one class. This optimization policy may be repeated multiple times until it is optimal.

[0216] Chrominance calculation APS filtering set method: For the chrominance component, the slice is divided into 8 parts, and each region contains consecutive integer numbers of CTUs, and the number of CTUs in each part is approximately the same. A set of filter coefficients is calculated for each region, and a total of 8 sets of filters are obtained. Then, similar to the luminance component, the integrated filter group can be optimized by the method of integrating the chrominance component regions, thus saving encoded bits.

[0217] Regarding Step 2-c: Slice / CTU-level Switch Determination and Filtering Set Selection Luminance Switch Determination and Filtering Set Selection: For luminance, it is possible to select whether to turn on the ALF operation at the slice and CTU switch levels.

[0218] For luminance ALF, the filtering coefficient set includes 16 fixed subsets and up to 7 APS subsets, and each subset contains 25 filter classes. For each CTU that selects luminance ALF, the selected filtering subset is determined by the subset index of encoding, and the selected filter class is determined by the content characteristics of the pixel block. Specifically, which filtering set to select is selected based on RDO.

[0219] Chrominance Switch Determination and Filtering Set Selection: Similarly, for chrominance, it is possible to select whether to turn on the ALF operation at the slice and CTU switch levels.

[0220] Chrominance ALF uses only the APS filter subset. When the APS layer encodes and transmits the chrominance APS subset, it shares one ID number with the luminance information APS subset and also maintains up to 7 APS subsets in the same way. Each slice uses only one APS subset, and the Cb component and the Cr component share one APS subset.

[0221] Each of the chrominance components Cb and Cr also includes up to 8 filters in each APS subset. Each CTU selects and uses one of them. The Cb component and the Cr component may use filters with different indexes. Specifically, which filter to use is selected based on RDO.

[0222] Note that, for the transmission flow of the image (picture) / slice layer after adding the luminance and chroma switches, reference can be made to FIG. 5. In FIG. 5, the Luma ALF switch is mainly used to control whether to turn on the luminance ALF. That is, when the field value for instructing this Luma ALF switch is 1, it represents on, and when the value is 0, it represents off. The Cb ALF switch is mainly used to control whether to turn on the first chroma component Cb ALF. That is, when the field value for instructing this Cb ALF switch is 1, it represents on, and when the value is 0, it represents off. The Cr ALF switch is mainly used to control whether to turn on the second chroma component Cr ALF. That is, when the field value for instructing this Cr ALF switch is 1, it represents on, and when the value is 0, it represents off. The Cb CC ALF switch is mainly used to control whether to turn on the ALF between the first chroma components. That is, when the field value for instructing this Cb CC ALF switch is 1, it represents on, and when the value is 0, it represents off. The Cr CC ALF switch is mainly used to control whether to turn on the ALF between the second chroma components. That is, when the field value for instructing this Cr CC ALF switch is 1, it represents on, and when the value is 0, it represents off.

[0223] Regarding Step 2-d: ALF filtering operation Luminance filtering operation: Based on the above steps, one 7x7 filtering template can be determined for each 4x4 luminance block. And filtering processing is performed on each pixel in the 4x4 luminance block.

[0224] Chroma filtering operation: Based on the above steps, one 5x5 filtering template can be determined for each chroma CTU. And filtering processing is performed on each pixel in the chroma block in the CTU.

[0225] Integrate luminance and chroma to obtain the reconstructed frame after ALF.

[0226] Step 3: Component - Adaptive Loop Filtering (CCALF) Generally, the luminance component of a video contains more fine - grained texture, while the chroma component is relatively flat. Also, the human eye is more sensitive to luminance information, and video coding tries to retain as much detail as possible. By performing ALF on the luminance information and compensating for the details of the chroma component, the compression performance of the chroma component can be improved. Therefore, in H.266 / VVC, component - adaptive ALF (CC - ALF) is introduced, which uses the reconstructed luminance values before ALF filtering to perform CC - ALF and makes supplementary and corrective operations on the chroma values. The ALF filtering framework including CC - ALF is as shown in Figure 11.

[0227] The specific steps on the CCALF encoding side are as follows.

[0228] When the NN filtering in the SPS layer is on, directly perform the following CCALF steps and output the final reconstructed frame. When the NN filtering in the SPS layer is off, it is necessary to determine whether the luminance ALF of the current slice is on. If it is on, perform the following CCALF steps. If the luminance ALF of the current slice is off, CCALF is in the default off state, does not perform the CCALF operation, and directly outputs the result of the previous step as the final reconstructed frame. Specifically, it includes the following sub - steps.

[0229] a. Calculate the CCALF filtering set for chroma. b. Make slice / CTU - level switch decisions and select the filtering set. c. Reconstruct the output after CCALF.

[0230] The following explains each sub - step.

[0231] Regarding Step 3-a: Calculation of the CCALF Filtering Set Divide the slice into four parts. Each region contains a continuous integer number of CTUs, and the number of CTUs in each part is approximately the same. Calculate a set of filter coefficients for each region to obtain a total of four sets of filters. Then, similar to chroma ALF, the integrated filter group can be optimized by integrating the chroma component regions, thus saving encoding bits.

[0232] Regarding Step 3-b: Slice / CTU Level Switch Decision and Filtering Set Selection The CCALF operation can select whether to turn on at the slice and CTU switch levels.

[0233] CC-ALF only uses the APS filter subset. When the APS layer encodes and transmits the APS subset of CC-ALF, it maintains a maximum of seven APS subsets. Each slice uses only one APS subset, and the Cb component and Cr component may use different APS subsets.

[0234] For the chroma components Cb and Cr, each APS subset of CC-ALF also contains a maximum of four filters. Each CTU selects and uses one of them. The Cb component and Cr component may use filters with different indexes. Specifically, which filter to use is selected based on RDO.

[0235] Regarding Step 3-c: CCALF Filtering Operation Based on the above steps, a 3x4 filtering template can be determined for each chroma CTU. Then, filtering processing is performed on each pixel in the chroma block of the CTU.

[0236] It should be noted that the filtering coefficients mentioned in the above specific embodiments may be transmitted in the APS layer. When aps_params_type is ALF_APS, the alf_data( ) syntax structure is analyzed. The alf_data( ) contains the filtering set parameters of ALF. The decoding side can obtain the filtering set parameters corresponding to ALF by analyzing alf_data( ) in the APS layer. The syntax semantics are as shown in Table 19.

[0237]

Table 19

[0238] Here, the syntax semantics of alf_data( ) are as shown in Table 20.

[0239]

Table 20

[0240] Here, alf_luma_filter_signal_flag: Identifies whether it contains the luminance ALF filtering coefficient. A value of 1 indicates that it contains, and a value of 0 indicates that it does not contain.

[0241] alf_chroma_filter_signal_flag: Identifies whether it contains the chroma ALF filtering coefficient. A value of 1 indicates that it contains, and a value of 0 indicates that it does not contain.

[0242] alf_cc_cb_filter_signal_flag: Identifies whether it contains the Cb component CC-ALF filtering coefficient. A value of 1 indicates that it contains, and a value of 0 indicates that it does not contain.

[0243] alf_cc_cr_filter_signal_flag: Identifies whether it contains the Cr component CC-ALF filtering coefficient. A value of 1 indicates inclusion, and a value of 0 indicates non-inclusion.

[0244] alf_luma_clip_flag: Identifies whether it contains the luminance clamp threshold. A value of 1 indicates inclusion, and a value of 0 indicates non-inclusion. alf_luma_num_filters_signalled_minus1: Identifies the number of filters included in the luminance ALF subset. This value is incremented by 1.

[0245] alf_luma_coeff_delta_idx[ filtIdx ]: Since multiple filter classes may use the same set of filters, it identifies the serial number of the filter used for the filteridx-th classification of luminance ALF.

[0246] alf_luma_coeff_abs[ sfIdx ][j]: Identifies the absolute value of the j-th filtering coefficient of the filtIdx-th filter of luminance ALF.

[0247] alf_luma_coeff_sign[ sfldx ][j ]: Identifies the sign of the j-th filtering coefficient of the filtldx-th filter of luminance ALF.

[0248] alf_luma_clip_idx[ sfldx ][j ]: Identifies the clamp threshold corresponding to the j-th filtering coefficient of the filtldx-th filter of luminance ALF.

[0249] alf_chroma_clip_flag: Identifies whether it contains the chroma clamp threshold. A value of 1 indicates inclusion, and a value of 0 indicates non-inclusion. alf_chroma_num_alt_filters_minus1: Identifies the number of filters included in the chroma ALF subset. This value is incremented by 1.

[0250] alf_chroma_coeff_abs[altldx][j] identifies the absolute value of the j-th filtering coefficient of the altIdx-th filter of chroma ALF.

[0251] alf_chroma_coeff_sign[altldx][j] identifies the sign of the j-th filtering coefficient of the altIdx-th filter of chroma ALF.

[0252] alf_chroma_clip_idx[altIldx][j] identifies the clamp threshold corresponding to the j-th filtering coefficient of the altldx-th filter of chroma ALF.

[0253] alf_cc_cb_filters_signalled_minus1 identifies the number of filters included in the CC-ALF subset of the chroma Cb component. It is the value obtained by adding 1 to this value.

[0254] alf_cc_cb_mapped_coeff_abs[k][j] identifies the absolute value of the j-th filtering coefficient of the i-th filter of chroma Cb component of CC-ALF.

[0255] alf_cc_cb_coef_sign[k][j] identifies the sign of the j-th filtering coefficient of the i-th filter of chroma Cb component of CC-ALF.

[0256] alf_cc_cr_filters_signalled_minus1 identifies the number of filters included in the CC-ALF subset of the chroma Cr component. It is the value obtained by adding 1 to this value.

[0257] alf_cc_cr_mapped_coeff_abs[k][j] identifies the absolute value of the j-th filtering coefficient of the i-th filter of chroma Cr component of CC-ALF.

[0258] alf_cc_or_coeff_sign[k][j] identifies the sign of the j-th filtering coefficient of the i-th filter of the CC-ALF for the Cr component of chroma

[0259] In this embodiment, a method for video processing is further provided. FIG. 12 is a second flowchart of a method for video processing according to an embodiment of the present invention. As shown in FIG. 12, this flow includes the following steps.

[0260] In step S1202, at least one of the filter information related to the neural network loop filtering of the reconstructed video unit and the filter information related to the adaptive loop filtering of the reconstructed video unit is received. In step S1204, the neural network loop filtering activation flag of the reconstructed video unit and the adaptive loop filtering activation flag of the reconstructed video unit are determined. In step S1206, adaptive loop filtering is performed on the reconstructed video unit based on the adaptive loop filtering activation flag.

[0261] Here, the one that executes the above operation may be a device on the video decoding side, such as a processor, a controller, or a device having a similar processing ability.

[0262] According to the above embodiments, an adaptive loop filtering enable flag can be set for the reconstructed video unit based on the neural network loop filtering enable flag of the reconstructed video unit. Thus, the adaptive loop filtering enable flag of the reconstructed video unit can be identified based on the neural network loop filtering enable flag, and further, adaptive loop filtering can be executed based on the identification result. By directly turning off the adaptive loop filtering after performing the neural network loop filtering, it is possible to avoid the problem that further luminance and / or chroma processing cannot be performed on the image and the image quality cannot reach optimization, and achieve the effect of improving the image quality.

[0263] In one alternative embodiment, the filter information related to the adaptive loop filtering includes an adaptive loop filtering enable flag and an adaptive parameter set referenced by the adaptive loop filtering.

[0264] In one alternative embodiment, the adaptive loop filtering enable flag includes a luminance adaptive loop filtering enable flag and a chroma adaptive loop filtering enable flag.

[0265] In one alternative embodiment, the chroma adaptive loop filtering enable flag includes at least one of a first chroma adaptive loop filtering enable flag, a second chroma adaptive loop filtering enable flag, a first chroma component - to - component adaptive loop filtering enable flag, and a second chroma component - to - component adaptive loop filtering enable flag.

[0266] In one alternative embodiment, the adaptive loop filtering enabling flag is determined in at least one syntax element of the sequence parameter set SPS, the picture parameter set PPS, the picture header PH, the slice header SH, and the coding / decoding tree unit CTU for generating the encoded video data of the reconstructed video unit.

[0267] In one alternative embodiment, the method further includes determining the luminance adaptive loop filtering enabling flag and the chrominance adaptive loop filtering enabling flag of the reconstructed video unit when the neural network loop filtering enabling flag instructs to perform neural network loop filtering on the reconstructed video unit at the sequence level or the picture level or the slice level.

[0268] In one alternative embodiment, the method further includes determining the luminance adaptive loop filtering enabling flag of the reconstructed video unit at the picture level or the slice level when the neural network loop filtering enabling flag instructs to perform neural network loop filtering on the reconstructed video unit at the sequence level, and determining the chrominance adaptive loop filtering enabling flag of the reconstructed video unit when the luminance adaptive loop filtering enabling flag instructs not to perform luminance adaptive loop filtering on the reconstructed video unit.

[0269] In one alternative embodiment, the method further includes determining whether to perform chrominance adaptive loop filtering on the reconstructed video unit based on the chrominance adaptive loop filtering enabling flag.

[0270] In one alternative embodiment, determining the luma adaptive loop filtering enable flag of the reconstructed video unit includes determining the value of a first flag related to the adaptive loop filter of the luma component of the reconstructed video unit.

[0271] In one alternative embodiment, determining the chroma adaptive loop filtering enable flag of the reconstructed video unit includes determining the value of a second flag related to the adaptive loop filter of the chroma component of the reconstructed video unit.

[0272] In one alternative embodiment, when the value of the adaptive loop filtering enable flag is a first value, the adaptive loop filtering enable flag is used to instruct to perform adaptive loop filtering on the reconstructed video unit, and when the value of the adaptive loop filtering enable flag is a second value, the adaptive loop filtering enable flag is used to instruct not to perform adaptive loop filtering on the reconstructed video unit, where the first value is different from the second value.

[0273] In one alternative embodiment, performing adaptive loop filtering on the reconstructed video unit includes performing at least one of the following operations: performing luma adaptive loop filtering on the luma component of the reconstructed video unit, performing first chroma adaptive loop filtering on the first chroma component of the reconstructed video unit, performing second chroma adaptive loop filtering on the second chroma component of the reconstructed video unit, performing first inter-chroma component adaptive loop filtering on the first chroma component of the reconstructed video unit, and performing first inter-chroma component adaptive loop filtering on the second chroma component of the reconstructed video unit.

[0274] In one alternative embodiment, the filter information related to the neural network loop filtering includes a neural network loop filtering activation flag and an adaptive parameter set referred to by the neural network loop filtering.

[0275] In this embodiment, the neural network filtering adaptive parameter set APS syntax element is specifically as shown in Table 21.

[0276] [Table 21]

[0277] The filter value NNF of the adaptive parameter set type (aps_params_type) field indicates the NN filtering adaptive parameter set.

[0278] Here, the filtering parameter information included in nnf_data mainly includes the absolute value of the filtering coefficient and the symbol of the filtering coefficient.

[0279] As one alternative embodiment, the syntax elements related to the luminance component NN filtering in nnf_data are specifically as shown in Table 22.

[0280] [Table 22]

[0281] As one alternative embodiment, the syntax elements related to the chroma component NN filtering in nnf_data are specifically as shown in Table 23.

[0282] [Table 23]

[0283] In one alternative embodiment, the neural network loop filtering enable flag includes at least one of a luminance neural network loop filtering enable flag, a first chroma neural network loop filtering enable flag, and a second chroma neural network loop filtering enable flag.

[0284] In one alternative embodiment, a neural network loop filtering enable flag included in filter information related to the neural network loop filtering is determined in at least one syntax element of a sequence parameter set SPS, a picture parameter set PPS, a picture header PH, a slice header SH, and a coding / decoding tree unit CTU for generating encoded video data of the reconstructed video unit.

[0285] In one alternative embodiment, the reconstructed video unit corresponds to at least one of a video image, a video clip, a video pattern block, a slice, a coding / decoding tree unit CTU, and a coding / decoding unit CU.

[0286] The operations in the above embodiments are all descriptions of operations on the decoding side.

[0287] Hereinafter, the flow on the decoding side will be described in general in conjunction with specific embodiments, specifically including the following steps.

[0288] Figure 13 is an ALF-SPLIT operation flowchart based on NN filtering on the decoding side according to a specific embodiment of the present invention. When the main ALF_SPLIT operation is used in the decoding-side filtering operation, as shown in Figure 13, the selection of the slice / CTU-level switch and filtering set of the luminance and chroma ALF, and CCALF of the current slice is obtained by analyzing the code stream. The overall flow is as follows.

[0289] ALF-SPLIT on condition: NN filtering switch on in the SPS layer (1) After obtaining the reconstructed image of LMCS, if the NN filtering switch in the SPS layer is on, perform NN filtering according to the following steps. If the NN filtering in the SPS layer is off, perform filtering according to the conventional filtering flow.

[0290] a. Obtain the slice / block-level switch decision for filtering from the code stream.

[0291] b. Reconstruct the output after NN filtering based on the switch information.

[0292] (2) Perform ALF processing on the reconstructed image after NN to obtain the final reconstructed image.

[0293] a. Classification. Classify the luminance component, and chroma does not need to be classified.

[0294] b. Obtain the ALF switch decision of the current slice / CTU from the code stream.

[0295] If the ALF-SPLIT on condition is satisfied, obtain the slice / CTU-level switch information of luminance and chroma and the corresponding filtering set from the code stream respectively. If the ALF-SPLIT on condition is not satisfied, obtain the switch information of the current slice from the code stream. If it is on, set the luminance ALF to on, continue to obtain the slice-level switch information of chroma, and then obtain the ctu switch information and the corresponding filtering set based on the switch status of luminance and chroma. If the ALF of the current slice is off, set the luminance ALF to off, stop obtaining the slice switch information of chroma, and assume that the chroma ALF is off by default.

[0296] c. Perform chroma ALF filtering operation on the luminance.

[0297] (3) Perform CCALF processing on the reconstructed image after NN and output the final reconstructed frame.

[0298] a. Obtain the CCALF switch determination of the current slice / CTU from the code stream.

[0299] If the ALF-SPLIT on condition is satisfied, directly obtain the slice / CTU-level switch information of the current slice CCALF and the corresponding filtering set from the code stream. If the ALF-SPLIT on condition is not satisfied, it is necessary to determine whether the current slice luminance ALF is on. If it is on, continue to obtain the slice / CTU-level switch information of CCALF and the corresponding filtering set from the code stream. If the ALF of the current slice is in the off state, CCALF is off by default, and the slice / CTU-level switch determination and filtering set selection of chroma are not performed.

[0300] b. Perform CCALF filtering.

[0301] According to one specific embodiment of the present invention, a solution for changing the on condition of ALF-SPLIT is further provided. In the above solution (specifically, referring to the solution for generally explaining the flow on the encoding side as described above), ALF-SPLIT is just one implementation method. The main idea of the above solution is to separate the luminance and chroma ALFs after NN filtering. Considering that a large number of ALFs turn off after NN, because the luminance performance is greatly improved by NN, so that a large number of luminance ALFs turn off, the chroma ALF and CCALF are forcibly turned off. When the current image / slice does not perform NN filtering, if the luminance ALF is generally on, there is no need to perform ALF-SPLIT.

[0302] In the above solution, when the following two situations occur, there is no need to perform the ALF-SPLIT operation.

[0303] When NN makes a filtering decision and the slice layer decides to turn off NN filtering.

[0304] When the NN filtering switch of the current slice is on, but the NN filtering switches of many CTUs among them are off.

[0305] Based on the above two points, this specific implementation solution extracts the switch of ALF-SPLIT from the NN filtering module, increases the adaptability of ALF-SPLIT to NN filtering, and proposes the following alternative implementation solutions.

[0306] The solution in this specific embodiment mainly changes the encoding side, and the decoding side is the same as that in the above embodiment.

[0307] Figure 14 is a flowchart of the ALF-SPLIT operation based on NN filtering on the encoding side according to this specific embodiment. The overall flow at the encoding end is specifically as follows.

[0308] (1) Perform NN filtering on the reconstructed image after LMCS.

[0309] a. NN filtering operation. It is the same as step 1-a on the encoding side in the foregoing embodiment. b. Determine the NN filtering switch at the slice / block level of NN and generate the switch decision of ALF-SPLIT. The determination of the NN filtering switch at the slice / block level of NN is consistent with step 1-b on the encoding side in the foregoing embodiment. There are two generation methods for the switch decision of ALF-SPLIT. Then, write the ALF-SPLIT switch decision into the code stream.

[0310] 1. Based on the NN slice switch, determine whether to turn on ALF-SPLIT. If the NN filtering of the current slice is on, set it so that the ALF-SPLIT on condition is satisfied. If the NN filtering of the current slice is off, set it so that the ALF-SPLIT on condition is not satisfied.

[0311] 2. Based on the occupancy rate of the NN CTU switch, determine whether to turn on ALF-SPLIT. If the current slice is on, count the proportion of the CTUs that are on in the total number of CTUs. If it is greater than the threshold (assuming the threshold is 50% here), set the flag of ALF-SPLIT to on; otherwise, set the flag of ALF-SPLIT to off. If the NN filtering of the current slice is off, the flag of ALF-SPLIT is off.

[0312] c. Reconstruct the output after NN filtering. It is the same as step 1-c on the encoding side in the foregoing embodiment.

[0313] (2) Perform ALF filtering operation. Obtain whether the ALF-SPLIT on condition is satisfied. Other steps are consistent with step 2 on the encoding side in the foregoing embodiment.

[0314] (3) Perform the CCALF filtering operation. Obtain whether the ALF-SPLIT on condition is satisfied. This is consistent with step 3 on the encoding side in the foregoing embodiments.

[0315] Hereinafter, the syntax semantics related to this specific embodiment will be described.

[0316] For the transmission flowchart of the image / slice layer after adding the ALF-SPLIT switch, reference can be made to FIG. 15.

[0317] In this specific embodiment, the syntax semantics mainly change the ALF-SPLIT switch conditions of the PH layer, SH layer, and CTU layer, and the remaining layers cannot be changed. In nn_structure( ), the switch information of ALF-SPLIT is transmitted.

[0318] For the syntax semantics of nn_structure( ) in this specific embodiment, specifically refer to Table 24.

[0319]

Table 24

[0320] Here, this Table 24 is actually Table 18 with the alf_split_enabled_flag element added. This new element is used to indicate whether to turn on the ALF-SPLIT technology. A value of 1 indicates turning on, and a value of 0 indicates not turning on. If it does not exist, it is processed as equal to 0.

[0321] (This syntax element is generated based on the information in the NN. For example, as mentioned above, when the CTU occupancy rate for turning on using NN filtering is greater than the threshold, it is turned on.)

[0322] In this specific embodiment, filter information related to neural network loop filtering and filter information related to adaptive loop filtering are notified by signaling at the PH level of the encoded bitstream. For the syntax semantics in the PH layer related to this specific embodiment, specifically refer to Table 25.

[0323]

Table 25

[0324] Here, this Table 25 is actually an adjustment of the execution prerequisite of alf_split_structure( ) on the basis of Table 3. That is, when it is determined that alf_split_enabled_flag is equal to 1, the ALF-SPLIT operation is executed.

[0325] In this specific embodiment, filter information related to neural network loop filtering and filter information related to adaptive loop filtering are notified by signaling at the SH level of the encoded bitstream. For the syntax semantics in the SH layer related to this specific embodiment, specifically refer to Table 26.

[0326]

Table 26

[0327] Here, this Table 26 is actually an adjustment of the execution prerequisite of alf_split_structure( ) on top of Table 4. That is, when it is determined that alf_split_enabled_flag is equal to 1, the ALF-SPLIT operation is executed.

[0328] For the syntax semantics in the CTU layer, specifically refer to Table 27.

[0329]

Table 27

[0330] According to one specific embodiment of the present invention, a method for realizing ALF-SPLIT without changing the code stream is further provided. This specific embodiment only relates to the modification on the encoding side and does not change the transmission of the code stream.

[0331] The steps on the encoding side are as follows.

[0332] (1) NN operation filtering. It is consistent with step 1 of the foregoing solution (specifically, reference can be made to the solution for generally describing the foregoing flow on the encoding side).

[0333] (2) Perform ALF filtering operation.

[0334] a. Classify the luminance components. There is no need to classify the chroma. b. Calculate a new APS filtering set for luminance and chroma respectively. c. Perform ALF slice / ctu level filtering determination.

[0335] Then, perform slice / CTU level switch determination and filtering set selection for luminance and chroma respectively. If at least one of luminance and chroma is on after determination, set the ALF switch of the current slice to on. If both ALFs of luminance and chroma are off after determination, set the ALF switch of the current slice to off.

[0336] d. Reconstruct the output after luminance and chroma ALF.

[0337] (3) Perform CCALF filtering operation.

[0338] a. Calculate a CCALF filtering set for chroma. b. Perform CCALF slice / CTU level switch determination and filtering set selection.

[0339] After the decision, if the current slice turns on CCALF, update it to set the ALF switch of the current slice to on. If CCALF is turned off after the decision, keep the previous ALF switch unchanged.

[0340] c. Reconstruct the output after CCALF.

[0341] It should be noted that in this case, once chroma ALF and CCALF are performed, the luminance ALF transmits information.

[0342] As can be clearly understood by those skilled in the art from the description of the above embodiments, the method of the above embodiments can be realized in the form of software and the necessary general-purpose hardware platform. Of course, it may also be realized by hardware, but in many cases, the former is a more preferred embodiment. Based on such an understanding, the technical solution of the present invention may also be embodied in the form of a software product in essence or in the part that contributes to the prior art. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions for causing a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the embodiments of the present invention.

[0343] In this embodiment, a device for video processing is further provided. This device is used to realize the above embodiments and preferred embodiments, and has already been described, so it will not be described further. As used below, the term "module" can realize a combination of software and / or hardware with a predetermined function. The device described in the following embodiments is preferably realized by software, but it is also possible to be realized by hardware or a combination of software and hardware as assumed.

[0344] FIG. 16 is a block diagram of a device for video processing according to an embodiment of the present invention. As shown in FIG. 16, this device A first determination module 162 for determining a neural network loop filtering activation flag of the reconstructed video unit; A setting module 164 for setting an adaptive loop filtering activation flag for the reconstructed video unit based on the neural network loop filtering activation flag; A notification module 166 for notifying, by signaling, at least one of filter information related to neural network loop filtering of the reconstructed video unit and filter information related to adaptive loop filtering of the reconstructed video unit; Comprising.

[0345] In one alternative embodiment, the filter information related to the adaptive loop filtering includes an adaptive loop filtering activation flag and an adaptive parameter set referenced by the adaptive loop filtering.

[0346] In one alternative embodiment, the adaptive loop filtering activation flag includes a luminance adaptive loop filtering activation flag and a chroma adaptive loop filtering activation flag.

[0347] In one alternative embodiment, the chroma adaptive loop filtering activation flag includes at least one of a first chroma adaptive loop filtering activation flag, a second chroma adaptive loop filtering activation flag, a first inter-chroma component adaptive loop filtering activation flag, and a second inter-chroma component adaptive loop filtering activation flag.

[0348] In one alternative embodiment, the notification module 166 is used to notify the adaptive loop filtering enabling flag by signaling in at least one syntax element of a sequence parameter set SPS, a picture parameter set PPS, a picture header PH, a slice header SH, and an encoding / decoding tree unit CTU of an encoded bitstream for generating the encoded video data of the reconstructed video unit.

[0349] In one alternative embodiment, the apparatus is further used to set a luma adaptive loop filtering enabling flag and a chroma adaptive loop filtering enabling flag respectively for the reconstructed video unit when the neural network loop filtering enabling flag instructs to perform neural network loop filtering on the reconstructed video unit at sequence level or picture level or slice level.

[0350] In one alternative embodiment, the apparatus is further used to set a luma adaptive loop filtering enabling flag and a chroma adaptive loop filtering enabling flag respectively for the reconstructed video unit when the neural network loop filtering enabling flag instructs to perform neural network loop filtering on the reconstructed video unit at sequence level and the adaptive loop filtering enabling flag instructs not to perform luma adaptive loop filtering on the reconstructed video unit at picture level or slice level.

[0351] In one alternative embodiment, the apparatus is further used to determine whether to perform chroma adaptive loop filtering on the reconstructed video unit based on the chroma adaptive loop filtering enabling flag.

[0352] In one alternative embodiment, the apparatus is used to set a luminance adaptive loop filtering enabling flag and a chroma adaptive loop filtering enabling flag respectively for the reconstructed video unit in a manner of setting a first flag related to an adaptive loop filter for the luminance component for the reconstructed video unit.

[0353] In one alternative embodiment, the apparatus is used to set a luminance adaptive loop filtering enabling flag and a chroma adaptive loop filtering enabling flag respectively for the reconstructed video unit in a manner of setting a second flag related to an adaptive loop filter for the chroma component for the reconstructed video unit.

[0354] In one alternative embodiment, when the value of the adaptive loop filtering enabling flag is a first value, the adaptive loop filtering enabling flag is used to instruct to perform adaptive loop filtering on the reconstructed video unit; when the value of the adaptive loop filtering enabling flag is a second value, the adaptive loop filtering enabling flag is used to instruct not to perform adaptive loop filtering on the reconstructed video unit, and the first value is different from the second value.

[0355] In one alternative embodiment, the apparatus is used to perform adaptive loop filtering on the reconstructed video unit in at least one of the following ways: performing luminance adaptive loop filtering on the luminance component of the reconstructed video unit; performing first chroma adaptive loop filtering on the first chroma component of the reconstructed video unit; performing second chroma adaptive loop filtering on the second chroma component of the reconstructed video unit; performing first inter-chroma adaptive loop filtering on the first chroma component of the reconstructed video unit; and performing first inter-chroma adaptive loop filtering on the second chroma component of the reconstructed video unit.

[0356] In one alternative embodiment, when the neural network loop filtering enabling flag instructs to perform neural network loop filtering on the reconstructed video unit at sequence level or picture level or slice level, the apparatus is used to set the neural network loop filtering enabling flag for the reconstructed video unit based on at least one of the following decisions: the decision to perform luminance adaptive loop filtering on the luminance component of the reconstructed video unit; the decision to perform first chroma adaptive loop filtering on the first chroma component of the reconstructed video unit; the decision to perform second chroma adaptive loop filtering on the second chroma component of the reconstructed video unit; the decision to perform first inter-chroma adaptive loop filtering on the first chroma component of the reconstructed video unit; and the decision to perform first inter-chroma adaptive loop filtering on the second chroma component of the reconstructed video unit.

[0357] In one alternative embodiment, the filter information related to the neural network loop filtering includes a neural network loop filtering enable flag and an adaptive parameter set referred to by the neural network loop filtering.

[0358] In one alternative embodiment, the neural network loop filtering enable flag includes at least one of a luminance neural network loop filtering enable flag, a first chroma neural network loop filtering enable flag, and a second chroma neural network loop filtering enable flag.

[0359] In one alternative embodiment, the notification module 166 is used to notify the neural network loop filtering enable flag by signaling in at least one syntax element of a sequence parameter set SPS, a picture parameter set PPS, a picture header PH, a slice header SH, and an encoding / decoding tree unit CTU of the encoded bitstream for generating the encoded video data of the reconstructed video unit.

[0360] In one alternative embodiment, the reconstructed video unit corresponds to at least one of a video image, a video clip, a video pattern block, a slice, an encoding / decoding tree unit CTU, and an encoding / decoding unit CU.

[0361] FIG. 17 is a configuration block diagram of an apparatus for another video process according to an embodiment of the present invention. As shown in FIG. 17, this apparatus includes a receiving module 172 for receiving at least one of the filter information related to the neural network loop filtering of the reconstructed video unit and the filter information related to the adaptive loop filtering of the reconstructed video unit, A second determination module 174 for determining a neural network loop filtering activation flag of the reconstructed video unit and an adaptive loop filtering activation flag of the reconstructed video unit, and A filtering module 176 for performing adaptive loop filtering on the reconstructed video unit based on the adaptive loop filtering activation flag, and Comprises.

[0362] In one alternative embodiment, the filter information related to the adaptive loop filtering includes an adaptive loop filtering activation flag and an adaptive parameter set referred to by the adaptive loop filtering.

[0363] In one alternative embodiment, the adaptive loop filtering activation flag includes a luminance adaptive loop filtering activation flag and a chroma adaptive loop filtering activation flag.

[0364] In one alternative embodiment, the chroma adaptive loop filtering activation flag includes at least one of a first chroma adaptive loop filtering activation flag, a second chroma adaptive loop filtering activation flag, a first inter-chroma component adaptive loop filtering activation flag, and a second inter-chroma component adaptive loop filtering activation flag.

[0365] In one alternative embodiment, the second determination module 174 is used to determine the adaptive loop filtering activation flag in at least one syntax element of a sequence parameter set SPS, a picture parameter set PPS, a picture header PH, a slice header SH, and an encoding / decoding tree unit CTU of an encoding bitstream for generating the encoded video data of the reconstructed video unit.

[0366] In one alternative embodiment, the apparatus is further used to determine the luma adaptive loop filtering enable flag and the chroma adaptive loop filtering enable flag of the reconstructed video unit when the neural network loop filtering enable flag instructs to perform neural network loop filtering on the reconstructed video unit at the sequence level or the picture level or the slice level.

[0367] In one alternative embodiment, the apparatus is further used to determine the luma adaptive loop filtering enable flag of the reconstructed video unit at the picture level or the slice level when the neural network loop filtering enable flag instructs to perform neural network loop filtering on the reconstructed video unit at the sequence level, and to determine the chroma adaptive loop filtering enable flag of the reconstructed video unit when the luma adaptive loop filtering enable flag instructs not to perform luma adaptive loop filtering on the reconstructed video unit.

[0368] In one alternative embodiment, the apparatus is further used to determine whether to perform chroma adaptive loop filtering on the reconstructed video unit based on the chroma adaptive loop filtering enable flag.

[0369] In one alternative embodiment, the apparatus is used to determine the luma adaptive loop filtering enable flag of the reconstructed video unit in a manner of determining the value of a first flag related to the adaptive loop filter of the luma component of the reconstructed video unit.

[0370] In one alternative embodiment, the apparatus is used to determine a chroma adaptive loop filtering enabling flag of the reconstructed video unit in a manner of determining a value of a second flag related to an adaptive loop filter of a chroma component of the reconstructed video unit.

[0371] In one alternative embodiment, when the value of the adaptive loop filtering enabling flag is a first value, the adaptive loop filtering enabling flag is used to instruct to perform adaptive loop filtering on the reconstructed video unit, and when the value of the adaptive loop filtering enabling flag is a second value, the adaptive loop filtering enabling flag is used to instruct not to perform adaptive loop filtering on the reconstructed video unit, and the first value is different from the second value.

[0372] In one alternative embodiment, the apparatus is used to perform adaptive loop filtering on the reconstructed video unit in at least one of the following manners: performing luminance adaptive loop filtering on a luminance component of the reconstructed video unit, performing first chroma adaptive loop filtering on a first chroma component of the reconstructed video unit, performing second chroma adaptive loop filtering on a second chroma component of the reconstructed video unit, performing first inter-chroma component adaptive loop filtering on the first chroma component of the reconstructed video unit, and performing first inter-chroma component adaptive loop filtering on the second chroma component of the reconstructed video unit.

[0373] In one alternative embodiment, the filter information related to the neural network loop filtering includes a neural network loop filtering enabling flag and an adaptive parameter set referred to by the neural network loop filtering.

[0374] In one alternative embodiment, the neural network loop filtering enabling flag includes at least one of a luminance neural network loop filtering enabling flag, a first chroma neural network loop filtering enabling flag, and a second chroma neural network loop filtering enabling flag.

[0375] In one alternative embodiment, the second determination module 174 is used to determine the neural network loop filtering enabling flag in at least one syntax element of a sequence parameter set SPS, a picture parameter set PPS, a picture header PH, a slice header SH, and a coding / decoding tree unit CTU of an encoded bitstream for generating the encoded video data of the reconstructed video unit.

[0376] In one alternative embodiment, the reconstructed video unit corresponds to at least one of a video image, a video clip, a video pattern block, a slice, a coding / decoding tree unit CTU, and a coding / decoding unit CU.

[0377] It should be noted that each of the above modules may be implemented by software or hardware. In the latter case, it may be implemented in the following manner, but not limited thereto, that is, all of the above modules are located in the same processor, or each of the above modules is located in different processors in any combination form.

[0378] The embodiments of the present invention further provide a computer-readable storage medium. A computer program is stored in this computer-readable storage medium. This computer program is configured to execute the steps in the embodiments of any one of the above methods when executed.

[0379] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB memory, a read-only memory (abbreviated as ROM), a random access memory (abbreviated as RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0380] Embodiments of the present invention further provide an electronic device. The electronic device includes a memory and a processor. A computer program is stored in the memory. The processor is configured to execute the computer program to perform the steps in any one of the above method embodiments.

[0381] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device. The transmission device is connected to the processor, and the input / output device is connected to the processor.

[0382] For the specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary embodiments, so this embodiment will not be further described here.

[0383] Obviously, it should be understood by those skilled in the art that each module or each step of the present invention described above may be implemented by a general-purpose computing device. They may be concentrated on a single computing device, or may be distributed over a network consisting of a plurality of computing devices. They may be implemented by executable program codes of a computing device. Therefore, they may be stored in a storage device and executed by a computing device. Also, in some cases, the steps shown or described in an order different from this may be executed, or they may be implemented as respective integrated circuit modules, or a plurality of them or steps may be implemented as a single integrated circuit module. Thus, this application is not limited to any specific combination of hardware and software.

[0384] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, various changes and modifications are possible to the present invention. Any modifications, equivalent replacements, improvements, etc. made within the principles of the present invention should all be included within the protection scope of the present invention.

Claims

1. Determining a neural network loop filtering enabling flag for a reconstructed video unit; Based on the neural network loop filtering enabling flag, setting an adaptive loop filtering enabling flag for the reconstructed video unit; Notifying, by signaling, at least one of filter information related to neural network loop filtering of the reconstructed video unit and filter information related to adaptive loop filtering of the reconstructed video unit; comprising; The filter information related to the adaptive loop filtering includes the adaptive loop filtering enabling flag and an adaptive parameter set referred to by the adaptive loop filtering, or The adaptive loop filtering enabling flag includes a luma adaptive loop filtering enabling flag and a chroma adaptive loop filtering enabling flag, or The filter information related to the neural network loop filtering includes the neural network loop filtering enabling flag and an adaptive parameter set referred to by the neural network loop filtering; A method for video processing.

2. The chroma adaptive loop filtering enabling flag includes at least one of a first chroma adaptive loop filtering enabling flag, a second chroma adaptive loop filtering enabling flag, a first chroma component - to - component adaptive loop filtering enabling flag, and a second chroma component - to - component adaptive loop filtering enabling flag, or The neural network loop filtering enabling flag includes at least one of a luma neural network loop filtering enabling flag, a first chroma neural network loop filtering enabling flag, and a second chroma neural network loop filtering enabling flag; The method according to claim 1.

3. The method according to claim 1.

3. In at least one syntax element among a sequence parameter set SPS, a picture parameter set PPS, a picture header PH, a slice header SH, and an encoding / decoding tree unit CTU for generating encoded video data of the reconstructed video unit, signaling is used to notify the adaptive loop filtering enabling flag or the neural network loop filtering enabling flag. The method according to claim 1.

4. When the neural network loop filtering enabling flag instructs to perform neural network loop filtering on the reconstructed video unit at the sequence level or picture level or slice level, setting a luma adaptive loop filtering enabling flag and a chroma adaptive loop filtering enabling flag respectively for the reconstructed video unit, or When the neural network loop filtering enabling flag instructs to perform neural network loop filtering on the reconstructed video unit at the sequence level, and the adaptive loop filtering enabling flag instructs not to perform luma adaptive loop filtering on the reconstructed video unit at the picture level or slice level, setting a luma adaptive loop filtering enabling flag and a chroma adaptive loop filtering enabling flag respectively for the reconstructed video unit, or Determining whether to perform chroma adaptive loop filtering on the reconstructed video unit based on the chroma adaptive loop filtering enabling flag. The method according to claim 1, further comprising.

5. Setting a luma adaptive loop filtering enabling flag and a chroma adaptive loop filtering enabling flag respectively for the reconstructed video unit includes Setting a first flag related to the adaptive loop filter of the luma component for the reconstructed video unit, or Setting a second flag related to the adaptive loop filter of the chroma component for the reconstructed video unit, and includes. The method according to claim 4.

6. When the value of the adaptation loop filtering activation flag is the first value, the adaptation loop filtering activation flag is used to instruct to perform adaptation loop filtering on the reconstructed video unit. When the value of the adaptation loop filtering activation flag is the second value, the adaptation loop filtering activation flag is used to instruct not to perform adaptation loop filtering on the reconstructed video unit. The first value is different from the second value. Performing adaptation loop filtering on the reconstructed video unit includes performing luminance adaptation loop filtering on the luminance component of the reconstructed video unit. Performing first chroma adaptation loop filtering on the first chroma component of the reconstructed video unit. Performing second chroma adaptation loop filtering on the second chroma component of the reconstructed video unit. Performing first inter-chroma adaptation loop filtering on the first chroma component of the reconstructed video unit. Performing first inter-chroma adaptation loop filtering on the second chroma component of the reconstructed video unit. includes at least one of the operations. The method according to claim 1.

7. When the neural network loop filtering activation flag instructs to perform neural network loop filtering on the reconstructed video unit at sequence level or picture level or slice level, a determination to perform luminance adaptation loop filtering on the luminance component of the reconstructed video unit. a determination to perform first chroma adaptation loop filtering on the first chroma component of the reconstructed video unit. a determination to perform second chroma adaptation loop filtering on the second chroma component of the reconstructed video unit. a determination to perform first inter-chroma adaptation loop filtering on the first chroma component of the reconstructed video unit. a determination to perform first inter-chroma adaptation loop filtering on the second chroma component of the reconstructed video unit Based on at least one of the determinations, set an adaptive loop filtering enabling flag for the reconstructed video unit. The method according to claim 1. **Claim 8** Receiving at least one of filter information related to neural network loop filtering of a reconstructed video unit and filter information related to adaptive loop filtering of the reconstructed video unit; Determining a neural network loop filtering enabling flag of the reconstructed video unit and an adaptive loop filtering enabling flag of the reconstructed video unit; Based on the adaptive loop filtering enabling flag, performing adaptive loop filtering on the reconstructed video unit; comprising The filter information related to the adaptive loop filtering includes the adaptive loop filtering enabling flag and an adaptive parameter set referred to by the adaptive loop filtering, or The adaptive loop filtering enabling flag includes a luminance adaptive loop filtering enabling flag and a chroma adaptive loop filtering enabling flag, or The filter information related to the neural network loop filtering includes a neural network loop filtering enabling flag and an adaptive parameter set referred to by the neural network loop filtering. A method for video processing. **Claim 9** The chroma adaptive loop filtering enabling flag is a first chroma adaptive loop filtering enabling flag, a second chroma adaptive loop filtering enabling flag, a first chroma component - to - component adaptive loop filtering enabling flag, a second chroma component - to - component adaptive loop filtering enabling flag including at least one of them, or The neural network loop filtering enabling flag is a luminance neural network loop filtering enabling flag, a first chroma neural network loop filtering enabling flag, a second chroma neural network loop filtering enabling flag including at least one of them. The method according to claim 8. **Claim 10** Determine the adaptive loop filtering enabling flag or the neural network loop filtering enabling flag in at least one syntax element among a sequence parameter set SPS, a picture parameter set PPS, a picture header PH, a slice header SH, and an encoding / decoding tree unit CTU for generating encoded video data of the reconstructed video unit. The method according to claim 8. **Claim 11** When the neural network loop filtering enabling flag instructs to perform neural network loop filtering on the reconstructed video unit at the sequence level or picture level or slice level, determining the luminance adaptive loop filtering enabling flag and the chrominance adaptive loop filtering enabling flag of the reconstructed video unit, or When the neural network loop filtering enabling flag instructs to perform neural network loop filtering on the reconstructed video unit at the sequence level, determining the luminance adaptive loop filtering enabling flag of the reconstructed video unit at the picture level or slice level, or When the luminance adaptive loop filtering enabling flag instructs not to perform luminance adaptive loop filtering on the reconstructed video unit, determining the chrominance adaptive loop filtering enabling flag of the reconstructed video unit, or Further including determining whether to perform chrominance adaptive loop filtering on the reconstructed video unit based on the chrominance adaptive loop filtering enabling flag. The method according to claim 8. **Claim 12** An apparatus for video processing, comprising means for implementing the method according to any one of claims 1 to 7 or the method according to any one of claims 8 to 11. **Claim 13** A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1 to 7 or the steps of the method according to any one of claims 8 to 11. Computer-readable storage medium.

Images