Loop filtering method and apparatus, and storage medium and program product

Abstract

The present application belongs to the technical field of video encoding and decoding. Disclosed are a loop filtering method and apparatus, and a storage medium and a program product, which can solve the problem of poor filtering performance. The loop filtering method in the embodiments of the present application comprises: performing pixel padding at a first boundary of an image unit on the basis of a first padding mode, wherein the first padding mode is determined on the basis of at least one of the following: a projection format corresponding to the image unit, and a rate-distortion decision result for pixel padding at a reference image boundary; performing loop filtering on the pixel-padded image unit on the basis of a first filter; and writing filtering parameters and syntax elements of loop filtering into a bit stream, wherein the syntax elements are used for indicating the first padding mode. The method can be applied to video encoding and decoding scenarios.

Description

Loop filtering method and device, storage medium and program product TECHNICAL FIELD

[0001] The present application belongs to the technical field of video coding, and particularly relates to a loop filtering method and device, a storage medium and a program product. BACKGROUND

[0002] In the Versatile Video Coding (VVC) technology, when reconstructing an image, loop filtering can be used to filter the reconstructed image to improve the subjective and objective quality of the reconstructed image.

[0003] In the related art, when filtering the reconstructed image by using the loop filtering, if the distance between the pixel to be filtered and the boundary is less than or equal to a certain distance, the filtering template will exceed the image boundary, and in this case, pixel padding is needed for the image boundary. The currently used pixel padding method is to copy a row or a column of pixels at the image boundary to the boundary.

[0004] However, according to the above pixel padding method, the texture topology at the image boundary after filtering will change, thereby reducing the filtering effect at the image boundary and further reducing the filtering performance.

[0005] SUMMARY

[0006] The embodiments of the present application provide a loop filtering method and device, a storage medium and a program product, which can avoid the change of the texture topology at the image boundary after filtering, thereby improving the filtering performance.

[0007] In a first aspect, a loop filtering method is provided, which can be applied to an encoder. The method comprises: performing pixel padding at a first boundary of an image unit based on a first padding method, the first padding method being determined based on at least one of the following: a projection format corresponding to the image unit, a rate-distortion decision result of pixel padding at a reference image boundary; performing loop filtering on the image unit after the pixel padding based on a first filter; and writing a filtering parameter of the loop filtering and a syntax element into a bitstream, the syntax element being used to indicate the first padding method.

[0008] In a possible implementation, the performing pixel padding at the first boundary of the image unit based on the first padding method comprises: when the distance from a pixel to be filtered to the first boundary of the image unit is less than M, performing pixel padding at the first boundary based on the first padding method, the pixel to be filtered being a pixel in the image unit, the value of M being determined according to the first filter, and M being a positive integer.

[0009] In a possible implementation, based on the first padding mode, the pixel padding is performed at the first boundary of the image unit, including: based on the first padding mode, first pixel information corresponding to the first padding mode is padded to the first boundary of the image unit.

[0010] In a possible implementation, the first filter is an adaptive loop filter (ALF), and the first pixel information includes at least one of: luminance information after sample adaptive offset (SAO) processing, chroma information after SAO processing, luminance information before deblocking filtering, chroma information before deblocking filtering, luminance residual information, luminance information after a luminance fixed filter, chroma information after a chroma fixed filter, or luminance information after a luminance Gaussian filter.

[0011] In a possible implementation, the first filter is a cross-component adaptive loop filter (CCALF), and the first pixel information includes at least one of: luminance information after SAO processing, chroma information after SAO processing, luminance information before deblocking filtering, chroma information before deblocking filtering, luminance residual information, luminance information after a luminance fixed filter, chroma information after a chroma fixed filter, or luminance information after a luminance Gaussian filter.

[0012] In a possible implementation, the image unit is an image, and the first boundary of the image unit is a virtual boundary of the image.

[0013] In a possible implementation, the image unit is a coding tree unit (CTU), and the first boundary of the image unit is a virtual boundary of the CTU.

[0014] In a possible implementation, the image unit is a CTU, and based on the first padding mode, the pixel padding is performed at the first boundary of the image unit, including: if there is no adjacent CTU at the first boundary of the CTU, the pixel padding is performed at the boundary of the CTU based on the first padding mode.

[0015] In a possible implementation, based on the first padding mode, the first pixel information is padded to the first boundary of the image unit, including: based on the first padding mode, information of N rows of pixels or N columns of pixels in the first boundary is padded to the first boundary, the N rows of pixels are the first row of pixels to the Nth row of pixels in the first boundary, the first row of pixels is a pixel adjacent to the first boundary, the N columns of pixels are the first column of pixels to the Nth column of pixels in the first boundary, the first column of pixels is a pixel adjacent to the first boundary, and N is a positive integer.

[0016] In a possible implementation, the first padding manner is a mirror padding manner; and the filling, according to the first padding manner, of the information of the N rows or N columns of pixels within the first boundary to the first boundary includes: filling, according to the mirror padding manner, the information of the N rows or N columns of pixels within the first boundary to the first boundary as a symmetric axis.

[0017] In a possible implementation, the first boundary of the image unit is outside the first boundary, and the region adjacent to the first boundary.

[0018] The loop filtering method provided by the embodiments of the present application can adaptively determine a padding manner based on a projection format corresponding to the image unit or a rate-distortion decision result of pixel padding at a boundary of a reference image in the process of loop filtering the image unit. Since the processing manners at the image boundary of the image block of different projection formats are different, the padding manner of directly copying one row or one column of pixels of the boundary cannot well adapt to the image boundary characteristics under different projection formats, and the corresponding padding manner is determined based on the projection format corresponding to the image unit, so that a suitable padding strategy can be adaptively selected according to the image characteristics of different projection formats, the distortion or artifact of the filtered image is avoided, and thus the filtering effect on the boundary region of the image block is improved. In addition, the rate-distortion decision result of pixel padding at the boundary of the reference image is introduced, the image quality change after encoding under different padding manners is evaluated, and the padding manner with the best filtering effect while ensuring the encoding efficiency can be selected. In this way, the padding manner is adaptively determined based on the projection format of the image unit or the rate-distortion decision result in the present application, the change of the texture topology at the boundary of the filtered image is avoided, and thus the filtering performance is improved.

[0019] In a second aspect, a loop filtering method is provided, which includes: parsing a code stream to obtain a syntax element, the syntax element being used to indicate a first pixel padding manner adopted when loop filtering is performed; determining a first boundary of a reconstructed image unit that is exceeded by a filtering template, the filtering template being used to filter the image unit; performing pixel padding at the first boundary according to the first padding manner; and performing loop filtering on the image unit after the pixel padding according to a first filter.

[0020] In a possible implementation, the pixel padding at the first boundary of the image unit according to the first padding manner includes: when a distance from a pixel to be filtered to the first boundary of the image unit is less than M, performing pixel padding at the first boundary according to the first padding manner, the pixel to be filtered being a pixel in the image unit, the value of M being determined according to the first filter, and M being a positive integer.

[0021] In a possible implementation, based on the first padding mode, the pixel padding is performed at the first boundary of the image unit, including: based on the first padding mode, first pixel information corresponding to the first padding mode is padded to the first boundary of the image unit.

[0022] In a possible implementation, the first filter is an adaptive loop filter (ALF), and the first pixel information includes at least one of: luminance information after sample adaptive offset (SAO) processing, chroma information after SAO processing, luminance information before deblocking filtering, chroma information before deblocking filtering, luminance residual information, luminance information after a luminance fixed filter, chroma information after a chroma fixed filter, or luminance information after a luminance Gaussian filter.

[0023] In a possible implementation, the first filter is a cross-component adaptive loop filter (CCALF), and the first pixel information includes at least one of: luminance information after SAO processing, chroma information after SAO processing, luminance information before deblocking filtering, chroma information before deblocking filtering, luminance residual information, luminance information after a luminance fixed filter, chroma information after a chroma fixed filter, or luminance information after a luminance Gaussian filter.

[0024] In a possible implementation, the image unit is an image, and the first boundary of the image unit is a virtual boundary of the image.

[0025] In a possible implementation, the image unit is a coding tree unit (CTU), and the first boundary of the image unit is a virtual boundary of the CTU.

[0026] In a possible implementation, the image unit is a CTU, and based on the first padding mode, the pixel padding is performed at the first boundary of the image unit, including: if there is no adjacent CTU at the first boundary of the CTU, the pixel padding is performed at the boundary of the CTU based on the first padding mode.

[0027] In a possible implementation, based on the first padding mode, the first pixel information is padded to the first boundary of the image unit, including: based on the first padding mode, information of N rows of pixels or N columns of pixels in the first boundary is padded to the first boundary; the N rows of pixels are the first row of pixels to the Nth row of pixels in the first boundary, the first row of pixels is a pixel adjacent to the first boundary, the N columns of pixels are the first column of pixels to the Nth column of pixels in the first boundary, the first column of pixels is a pixel adjacent to the first boundary, and N is a positive integer.

[0028] In a possible implementation, the first padding manner is a mirror padding manner; and the information of the N rows or N columns of pixels within the first boundary is padded to the first boundary based on the first padding manner, including: based on the mirror padding manner, mirroring the information of the N rows or N columns of pixels within the first boundary to the first boundary as a symmetric axis.

[0029] In a possible implementation, the region adjacent to the first boundary is outside the first boundary of the image unit.

[0030] The loop filtering method provided by the embodiments of the present application can effectively improve the filtering effect of the image boundary region by parsing the filtering parameters and the padding manner information from the code stream and using the corresponding pixel padding technology in the filtering process, and can avoid the distortion or artifact problem of the filtered image, thereby improving the filtering effect of the image boundary region.

[0031] In a third aspect, a loop filtering device is provided, which can be applied to an encoder, and the device includes: a filtering module and an encoding module. The filtering module is configured to perform pixel padding at a first boundary of an image unit based on a first padding manner, and the first padding manner is determined based on at least one of the following: a projection format corresponding to the image unit, and a rate-distortion decision result of pixel padding at a reference image boundary. The filtering module is further configured to perform loop filtering on the image unit after the pixel padding based on a first filter. The encoding module is configured to write filtering parameters and syntax elements of the loop filtering into a code stream, and the syntax elements are used to indicate the first padding manner.

[0032] In a possible implementation, the filtering module is specifically configured to perform pixel padding at the first boundary based on the first padding manner when a distance from a pixel to be filtered to the first boundary of the image unit is less than M, the pixel to be filtered is a pixel in the image unit, the value of M is determined according to the first filter, and M is a positive integer.

[0033] In a possible implementation, the filtering module is specifically configured to perform pixel padding at the first boundary of the image unit based on the first padding manner, and the first pixel information is information of a pixel corresponding to the first padding manner within the first boundary.

[0034] In a possible implementation, the filtering module is specifically configured to perform loop filtering on the pixel to be filtered in the image unit after the pixel padding based on the first filter.

[0035] In a possible implementation, the first filter is an adaptive loop filter (ALF) or a cross-component adaptive loop filter (CCALF); and the first pixel information comprises at least one of the following: luminance information after sample adaptive offset (SAO) processing; chroma information after SAO processing; luminance information before de-blocking filtering; chroma information before de-blocking filtering; luminance residual information; luminance information after a luminance fixed filter; chroma information after a chroma fixed filter; or luminance information after a luminance Gaussian filter.

[0036] In a possible implementation, the first filter is a cross-component adaptive loop filter (CCALF); and the first pixel information comprises at least one of the following: luminance information after sample adaptive offset (SAO) processing; chroma information after SAO processing; luminance information before de-blocking filtering; chroma information before de-blocking filtering; luminance residual information; luminance information after a luminance fixed filter; chroma information after a chroma fixed filter; or luminance information after a luminance Gaussian filter.

[0037] In a possible implementation, the image unit is an image, and the first boundary of the image unit is a virtual boundary of the image.

[0038] In a possible implementation, the image unit is a coding tree unit (CTU), and the first boundary of the image unit is a virtual boundary of the CTU.

[0039] In a possible implementation, the image unit is a CTU; and the filter module is specifically configured to, if there is no adjacent CTU at the first boundary of the CTU, perform pixel padding at the boundary of the CTU based on a first padding manner.

[0040] In a possible implementation, the filter module is specifically configured to fill, based on the first padding manner, information of N rows of pixels or N columns of pixels in the first boundary to the first boundary; the N rows of pixels are the 1st row of pixels to the Nth row of pixels in the first boundary, and the 1st row of pixels is a pixel adjacent to the first boundary; the N columns of pixels are the 1st column of pixels to the Nth column of pixels in the first boundary, and the 1st column of pixels is a pixel adjacent to the first boundary; and N is a positive integer.

[0041] In a possible implementation, the first padding manner is a mirror padding manner; and the filling, based on the first padding manner, of the information of the N rows of pixels or the N columns of pixels in the first boundary to the first boundary comprises: mirroring, based on the mirror padding manner, the information of the N rows of pixels or the N columns of pixels in the first boundary to the first boundary with the first boundary as a symmetry axis.

[0042] In a possible implementation, the first boundary of the image unit is a region adjacent to the first boundary and outside the first boundary.

[0043] The loop filtering device provided by the embodiments of the present application can adaptively determine the filling mode based on the projection format corresponding to the image unit or the rate-distortion decision result of pixel filling at the boundary of the reference image in the process of loop filtering the image unit. Since the processing modes at the image boundary of the image block of different projection formats are different, the filling mode of directly copying one row or one column of pixels at the boundary cannot well adapt to the image boundary characteristics under different projection formats, and the corresponding filling mode is determined based on the projection format corresponding to the image unit, so that a suitable filling strategy can be adaptively selected according to the image characteristics of different projection formats, the distortion or artifact of the filtered image is avoided, and thus the filtering effect on the boundary region of the image block is improved. In addition, the rate-distortion decision result of pixel filling at the boundary of the reference image is introduced, the image quality change after encoding under different filling modes is evaluated, and the filling mode with the best filtering effect while ensuring the encoding efficiency can be selected. In this way, the filling mode is adaptively determined based on the projection format of the image unit or the rate-distortion decision result in the present application, the change of the texture topology at the boundary of the filtered image is avoided, and thus the filtering performance is improved.

[0044] In a fourth aspect, the embodiments of the present application provide a loop filtering device, which can be applied to a decoder, and the device comprises a code stream analysis module and a filtering module, wherein: the code stream analysis module is configured to analyze a code stream to obtain a syntax element, the syntax element being used to indicate a first filling mode adopted when loop filtering is performed; the filtering module is configured to perform pixel filling at a first boundary of an image unit based on the first filling mode; and the filtering module is further configured to perform loop filtering on the image unit after the pixel filling based on a first filter.

[0045] In a possible implementation, the filtering module is specifically configured to perform pixel filling at the first boundary based on the first filling mode when a distance from a pixel to be filtered to the first boundary of the image unit is less than M, the pixel to be filtered being a pixel in the image unit, the value of M being determined according to the first filter, and M being a positive integer.

[0046] In a possible implementation, the filtering module is specifically configured to fill first pixel information corresponding to the first filling mode into the first boundary of the image unit based on the first filling mode, the first pixel information being information of the pixel corresponding to the first filling mode in the first boundary.

[0047] In a possible implementation, the filtering module is specifically configured to perform loop filtering on the pixel to be filtered in the image unit after the pixel filling based on the first filter.

[0048] In a possible implementation, the first filter is an adaptive loop filter (ALF) or a cross-component adaptive loop filter (CCALF); and the first pixel information comprises at least one of: luminance information after sample adaptive offset (SAO) processing; chroma information after SAO processing; luminance information before de-blocking filtering; chroma information before de-blocking filtering; luminance residual information; luminance information after a luminance fixed filter; chroma information after a chroma fixed filter; or luminance information after a luminance Gaussian filter.

[0049] In a possible implementation, the first filter is a cross-component adaptive loop filter (CCALF); and the first pixel information comprises at least one of: luminance information after sample adaptive offset (SAO) processing; chroma information after SAO processing; luminance information before de-blocking filtering; chroma information before de-blocking filtering; luminance residual information; luminance information after a luminance fixed filter; chroma information after a chroma fixed filter; or luminance information after a luminance Gaussian filter.

[0050] In a possible implementation, the image unit is an image, and the first boundary of the image unit is a virtual boundary of the image.

[0051] In a possible implementation, the image unit is a coding tree unit (CTU), and the first boundary of the image unit is a virtual boundary of the CTU.

[0052] In a possible implementation, the image unit is a CTU; and the filter module is specifically configured to, if there is no adjacent CTU at the first boundary of the CTU, perform pixel padding at the boundary of the CTU based on a first padding manner.

[0053] In a possible implementation, the filter module is specifically configured to fill, based on the first padding manner, information of N rows of pixels or N columns of pixels in the first boundary to the first boundary; the N rows of pixels are the 1st row of pixels to the Nth row of pixels in the first boundary, and the 1st row of pixels is a pixel adjacent to the first boundary; the N columns of pixels are the 1st column of pixels to the Nth column of pixels in the first boundary, and the 1st column of pixels is a pixel adjacent to the first boundary; and N is a positive integer.

[0054] In a possible implementation, the first padding manner is a mirror padding manner; and the filling, based on the first padding manner, of the information of the N rows of pixels or the N columns of pixels in the first boundary to the first boundary comprises: mirroring, based on the mirror padding manner, the information of the N rows of pixels or the N columns of pixels in the first boundary to the first boundary with the first boundary as a symmetry axis.

[0055] In a possible implementation, the first boundary of the image unit is a region adjacent to the first boundary and outside the first boundary.

[0056] The loop filtering device provided by the embodiments of the present application can effectively improve the filtering effect of the image boundary region, avoid the distortion or artifact problem of the filtered image, and thus improve the filtering effect of the image boundary region by analyzing the filtering parameters and the padding mode information from the code stream and using the corresponding pixel padding technology in the filtering process.

[0057] The various possible implementation manners described above can be executed independently, or executed in combination without contradiction, and the specific execution can be determined according to actual use requirements, which is not limited in the embodiments of the present application.

[0058] In a fifth aspect, the embodiments of the present application provide a chip, which includes a processor and a communication interface, the communication interface is coupled with the processor, and the processor is used to run a program or an instruction to realize the loop filtering method according to the first aspect or the second aspect.

[0059] In a sixth aspect, the embodiments of the present application provide a computer readable storage medium, which stores a computer program, and the computer program is executed by at least one processor to realize the loop filtering method according to the first aspect or the loop filtering method according to the second aspect.

[0060] In a seventh aspect, the embodiments of the present application provide a computer program product including computer program instructions, and the computer program instructions make a computer execute the method according to the first aspect or the loop filtering method according to the second aspect.

[0061] The loop filtering method provided in the embodiments of the present application determines a boundary of a reconstructed image unit which is exceeded by a filtering template used for filtering the image unit, and performs pixel padding at the first boundary based on a first padding mode determined based on at least one of the following: a projection format corresponding to the image unit, and a rate-distortion decision result of pixel padding at the boundary of the reference image. By this method, when the filtering template exceeds the first boundary of the image unit, the padding mode can be adaptively determined based on the projection format corresponding to the image unit or the rate-distortion decision result of pixel padding at the boundary of the reference image. Since the processing manners at the boundaries of the image blocks of different projection formats are different, the padding mode of directly copying one row or one column of pixels at the boundary cannot well adapt to the boundary characteristics of the image under different projection formats, and the corresponding padding mode is determined based on the projection format corresponding to the image unit, so that a suitable padding strategy can be adaptively selected according to the image characteristics of different projection formats, thereby avoiding distortion or artifacts of the filtered image, and improving the filtering effect on the boundary region of the image block. In addition, the rate-distortion decision result of pixel padding at the boundary of the reference image is introduced, so that the image quality change after encoding under different padding modes can be evaluated, and the padding mode with the best filtering effect while ensuring the encoding efficiency can be selected. In this way, the padding mode is adaptively determined based on the projection format of the image unit or the rate-distortion decision result in the present application, so that the change of texture topology at the boundary of the filtered image is avoided, thereby improving the filtering performance. BRIEF DESCRIPTION OF DRAWINGS

[0062] FIG. 1 is a schematic diagram of a process of video data encoding and video data decoding provided in the embodiments of the present application;

[0063] FIG. 2 is a schematic diagram of an architecture of a codec system provided in the embodiments of the present application;

[0064] FIG. 3 is a schematic block diagram of a possible structure of an encoder to which the loop filtering method provided in the embodiments of the present application is applied;

[0065] FIG. 4 is a schematic block diagram of another possible structure of an encoder to which the loop filtering method provided in the embodiments of the present application is applied;

[0066] FIG. 5 is a schematic block diagram of a possible structure of a decoder to which the loop filtering method provided in the embodiments of the present application is applied;

[0067] FIG. 6 is a flowchart of one of the loop filtering methods provided in the embodiments of the present application;

[0068] FIG. 7A is a schematic diagram of a luminance filtering template of ALF provided in the embodiments of the present application;

[0069] FIG. 7B is a schematic diagram of a chrominance filtering template of ALF provided in the embodiments of the present application;

[0070] FIG. 8 is a schematic diagram of a filter template of CCALF according to an embodiment of the present application;

[0071] FIG. 9A is a schematic diagram of image boundary padding according to an embodiment of the present application;

[0072] FIG. 9B is a schematic diagram of image boundary padding according to an embodiment of the present application;

[0073] FIG. 9C is a schematic diagram of image boundary padding according to an embodiment of the present application;

[0074] FIG. 10 is a schematic diagram of CTU boundary padding according to an embodiment of the present application;

[0075] FIG. 11A is a schematic diagram of CTU according to an embodiment of the present application;

[0076] FIG. 11B is a schematic diagram of CTU boundary padding according to an embodiment of the present application;

[0077] FIG. 11C is a schematic diagram of CTU boundary padding according to an embodiment of the present application;

[0078] FIG. 11D is a schematic diagram of CTU boundary padding according to an embodiment of the present application;

[0079] FIG. 12A is a schematic diagram of ALF and CCALF pixel padding according to an embodiment of the present application;

[0080] FIG. 12B is a schematic diagram of ALF and CCALF pixel padding according to an embodiment of the present application;

[0081] FIG. 12C is a schematic diagram of ALF and CCALF pixel padding according to an embodiment of the present application;

[0082] FIG. 12D is a schematic diagram of ALF and CCALF pixel padding according to an embodiment of the present application;

[0083] FIG. 13 is a schematic diagram of a loop filtering method according to an embodiment of the present application;

[0084] FIG. 14 is a schematic diagram of a loop filtering apparatus according to an embodiment of the present application;

[0085] FIG. 15 is a schematic diagram of a loop filtering apparatus according to an embodiment of the present application;

[0086] FIG. 16 is a schematic diagram of a loop filtering apparatus according to an embodiment of the present application;

[0087] FIG. 17 is a schematic diagram of a loop filtering apparatus according to an embodiment of the present application. DETAILED DESCRIPTION

[0088] The technical solutions in the embodiments of the present application will be clearly described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art belong to the scope of protection of the present application.

[0089] In the description of the embodiments of the present application, "first", "second", and the like are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than that illustrated or described herein, and the objects distinguished by "first", "second" are generally of a kind and do not limit the number of objects, for example, the first object can be one or more. In addition, "or" in the present application means at least one of the connected objects. For example, "A or B" covers three schemes, namely, scheme one: including A and not including B; scheme two: including B and not including A; scheme three: including A and including B. The character " / " generally represents that the objects before and after are in an "or" relationship.

[0090] In the description of the embodiments of the present application, "indication" can be a direct indication (or explicit indication) or an indirect indication (or implicit indication). Among them, the direct indication can be understood as that the sender explicitly informs the receiver of specific information, operations to be performed or requested results and the like in the sent indication; the indirect indication can be understood as that the receiver determines the corresponding information according to the indication sent by the sender, or judges and determines the operations to be performed or the requested results according to the judgment result.

[0091] In the description of the embodiments of the present application, "at least one (term)", "at least one of" and the like mean any one, any two or a combination of two or more of the objects contained therein. For example, at least one (term) of a, b and c can mean "a", "b", "c", "a and b", "a and c", "b and c", and "a, b and c", wherein a, b and c can be single or multiple. Similarly, "at least two (terms)" means two or more, and the meaning expressed is similar to "at least one (term)".

[0092] In the description of the embodiments of the present application, the meaning of "multiple" is two or more. For example, multiple prediction units refer to two or more multiple prediction units, and multiple coding blocks refer to two or more coding blocks. Among them, at least two and multiple have similar meanings, and in some embodiments, the two can also be used interchangeably.

[0093] In the description of the embodiments of the present application, "comprise", "contain" or any other variant thereof is intended to encompass a non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or further includes elements inherent to such a process, method, article or device. Without more limitations, the element defined by the statement "comprises a" does not exclude the presence of another identical element in the process, method, article or device including the element.

[0094] Firstly, the technical terms related to the embodiments of the present application are introduced:

[0095] 1. Digital video compression technology

[0096] Digital video compression technology mainly compresses huge digital video data for transmission and storage. With the rapid growth of Internet video and the increasing demand for video clarity, although existing digital video compression standards can save a lot of video data, better digital video compression technology is still needed to reduce the bandwidth and traffic pressure of digital video transmission.

[0097] In the process of digital video encoding, the encoder reads unequal pixels of the original video sequence of different color formats, including luminance component and chrominance component, that is, the encoder reads a black and white or color image. Then it is divided into blocks, and the block data is handed over to the encoder for encoding. Nowadays, the encoder is usually a hybrid framework encoding mode, which generally includes intra-frame and inter-frame prediction, transformation and quantization, inverse transformation and inverse quantization, loop filtering and entropy encoding, etc.

[0098] Among them, intra-frame prediction only refers to the information of the same frame image, predicts the pixel information in the current divided block, and is used to eliminate spatial redundancy; inter-frame prediction can refer to the image information of different frames, use motion estimation to search for the motion vector information that best matches the current divided block, and is used to eliminate temporal redundancy; transformation converts the predicted image block to the frequency domain, redistributes the energy, and combines quantization to remove information that is not sensitive to the human eye, which is used to eliminate visual redundancy; loop filtering uses the statistical information of the image to filter the image to improve the subjective and objective quality of the image; entropy coding can eliminate character redundancy according to the current context model and the probability information of the binary code stream.

[0099] 2. Video encoding technology

[0100] There are a series of redundant information in a video sequence, such as spatial redundancy, temporal redundancy, visual redundancy, information entropy redundancy, structural redundancy, knowledge redundancy, importance redundancy, etc. In order to remove the redundant information in the video sequence as much as possible and reduce the amount of data representing the video, video coding technology is proposed to achieve the effect of reducing storage space and saving transmission bandwidth. Video coding technology is also called video compression technology.

[0101] In the international general range, the video compression coding standards include, for example: the Advanced Video Coding (AVC) in the MPEG-2 and the 10th part of the MPEG-4 standard formulated by the Motion Picture Experts Group (MPEG), the H.263, H.264 and H.265 (also known as High Efficiency Video Coding standard, HEVC) formulated by the International Telecommunication Union-Telecommunication Standardization Sector (ITU-T), and the H.266 (also known as Versatile Video Coding, VVC) formulated by the Joint Video Experts Team (JVET) composed of the Video Coding Experts Group (VCEG) under the MPEG and the ITU-T. Among them, H.266 / VVC as a new generation of video coding standard, not only can help users store more high-definition videos on the device, thereby reducing the data traffic on the network, but also can support high resolution, high dynamic range, screen content coding and other functions in the main10 profile. Compared with the last generation standard H.265 / HEVC, the H.266 / VVC standard further improves the compression performance, which can support reducing 50% of the data size for users while keeping the video subjective quality unchanged.

[0102] It should be noted that in the coding algorithm based on the hybrid coding architecture, the above compression coding methods can be mixedly used.

[0103] 3、Video decoding technology

[0104] Video decoding technology is a component of video coding technology, corresponding to video encoding technology, and involves the process of restoring compressed video data into original video signals for playback. Specifically, video encoding technology compresses original video data into smaller files through specific compression algorithms for storage and transmission; and video decoding technology is the inverse operation of this process, i.e., decompressing compressed video data into original video signals for playback on a display device. This technology is of great significance to the network television, digital cinema, distance education, video conference, and other industries, as it can significantly reduce the bandwidth and storage space required for storage and transmission while ensuring video quality.

[0105] Generally, in the encoding process of video data, taking a video frame (i.e., a frame of video image) of video data as an example, a video encoding device takes a coding unit (CU, corresponding to a coding block) as the encoding unit, and completes the encoding of all CUs included in a coding tree unit (CTU) according to a certain encoding rule (including prediction, transformation, quantization, and entropy encoding, etc.), and further completes the encoding of multiple CTUs included in an image to obtain the encoding information corresponding to the image. Correspondingly, in the decoding process of video data, a decoding device completes the reconstruction of multiple CUs included in a CTU according to a decoding rule corresponding to the encoding process (including prediction, inverse quantization, inverse transformation, etc.), and further completes the reconstruction of multiple CTUs of an image to obtain the reconstructed image.

[0106] CTU and CU are nodes on a coding tree, and CTU is the root node. CTU is divided to obtain a certain number of nodes, and further, part or all of the nodes obtained by one division of CTU can be further divided into smaller nodes, and so on, until all nodes do not need to be divided any more. The node that does not need to be divided any more is called a leaf node, i.e., the leaf node is the end node of the coding tree that does not need to be divided any more. In H.266 / VVC, a leaf node corresponds to a CU.

[0107] It should be noted that in the encoding or decoding process of the CU, whether the CU is divided into multiple regions to obtain multiple regions, and each region is transformed or inverse transformed, so as to obtain the transformation result or inverse transformation result of the CU according to the transformation result or inverse transformation result of the multiple regions, can also be considered.

[0108] In the field of encoding and decoding of video data, the method for partitioning a CU to obtain multiple regions includes quad-tree partitioning, binary-tree partitioning, ternary-tree partitioning, and the like. When partitioning a CU, any one or more of quad-tree partitioning, ternary-tree partitioning, and binary-tree partitioning can be selected. For example, in an embodiment, a partitioning method of cascading quad-tree and binary-tree can be used, that is, a CU is first partitioned by quad-tree to obtain four regions, and if one or more of the four regions can be further partitioned, the region that needs to be further partitioned is partitioned into smaller regions by using the binary-tree partitioning method. The specific partitioning method can be determined according to actual use requirements, and the embodiments of the present application do not make specific limitations.

[0109] Next, as shown in FIG. 1, taking an image block (which can be a CU) in a video frame of video data as an example, a brief process of video data encoding and video data decoding is described. In the encoding process, the image block is referred to as a to-be-encoded block, and in the decoding process, the image block is referred to as a to-be-decoded block. The process of encoding the to-be-encoded block by a video encoding device includes prediction, transformation, quantization, entropy encoding, and the like. After the video encoding device obtains a prediction block of the current to-be-encoded block by prediction, the video encoding device obtains a residual block based on the prediction block (the residual block is the difference between the original block and the prediction block), then the video encoding device can perform transformation on the residual block according to a transformation pair (the transformation pair includes a horizontal transformation kernel and a vertical transformation kernel), and then performs quantization on the transformed residual block. Finally, after completing entropy encoding (that is, encoding of syntax elements and the like), a bitstream is formed, thereby completing the encoding of the to-be-encoded block. In the above encoding process, the transformation kernel can be various types of transformation kernels based on discrete cosine transform (DCT) (for example, DCT-II, DCT-VIII, and the like), or can be various types of transformation kernels based on discrete sine transform (DST) (for example, DST-VII, and the like). In addition, the video encoding process also includes corresponding encoding of syntax elements (for example, partition flags, partition modes), and the encoded syntax elements and the encoded image block constitute the encoding information (that is, the bitstream). That is, the encoding information of the video data includes the encoded image data and the encoded syntax elements. The process of decoding the to-be-decoded block by a video decoding device includes entropy decoding, prediction, inverse transformation, inverse quantization, and the like. The video decoding process can be understood as the reverse process of the video encoding process. The video decoding device reconstructs the image block (that is, the to-be-decoded block) based on the syntax elements associated with the image block and the encoded image block, that is, the video decoding device obtains the reconstructed image block of the image block based on the syntax elements associated with the image block and the encoded image block, thereby completing the decoding of the to-be-decoded block.

[0110] 4. Adaptive loop filtering

[0111] Adaptive loop filtering is a technique used in video encoding to restore and enhance the quality of reconstructed images. It is usually placed in the later stage of the codec loop, such as after deblocking filtering (DF) and sample adaptive offset (SAO). The main purpose of adaptive loop filtering is to filter out the distortion introduced during the encoding process, particularly Gaussian white noise, by adaptively adjusting the filter coefficients, thereby optimizing the visual effect of the image.

[0112] The principle of adaptive loop filtering is based on the general understanding of image filtering, which can be represented as a convolution operation between the image and a convolution kernel. In video encoding standards such as H.266 / VVC, ALF uses a specific shape of filter, where a 7x7 filter is used for the luminance component and a 5x5 filter is used for the chrominance component. To reduce the code words for transmitting filter coefficients and to reduce the computational complexity of filtering, ALF adopts a diamond-shaped and center-symmetric filter coefficient matrix, only the pixels within the diamond will participate in filtering, and the center-symmetric pixels use the same filter coefficient.

[0113] The workflow of ALF usually includes the following steps:

[0114] (1) Classification: Each image frame (slice) or coding tree unit (CTU) is divided into smaller blocks (such as 4x4 blocks), and these blocks are classified according to their gradient information.

[0115] (2) Filter coefficient calculation: For each category, calculate the optimal filter coefficients. These coefficients are determined according to the statistical characteristics (such as the covariance matrix) of the pixels within the category.

[0116] (3) Filtering operation: Use the calculated filter coefficients to perform filtering operations on the pixel blocks of the corresponding category to restore and enhance the image quality.

[0117] (4) Rate-distortion optimization (RDO): In order to optimize the filtering effect while ensuring the encoding efficiency, a rate-distortion optimization process is usually performed. This includes merging categories with fewer occurrences to reduce the code rate, and evaluating the impact of different filtering schemes on the encoding performance.

[0118] In loop filter (ILF) mainly includes luma mapping and chroma scaling, deblocking filter, sample offset compensation, adaptive loop filtering (ALF) and cross-component adaptive loop filter (CCALF). ALF is a filter designed to minimize the mean square error between the reconstructed image and the original image. CCALF is a filter designed to minimize the mean square error between the chroma reconstructed image and the original image using luma information.

[0119] In the VVC encoder, ALF filters the luma reconstructed image and the chroma reconstructed image respectively. The filtering implementation process of CCALF in VVC is explained below.

[0120] In the VVC encoder, CCALF classifies CTUs, and the classification process uses a rate-distortion optimization function (RDO) to make decisions. Each pixel in the CTU belongs to the same category. Specifically, when the cost calculated by using a filter of a certain category for the CTU is the smallest, the category of the CTU is the category of the filter. Of course, if the cost of not filtering is smaller than the cost of using the filter, the CTU is not filtered. At this time, the CTU has no category. Whether the CTU is filtered and the corresponding category information are written into the code stream and transmitted to the decoding end.

[0121] After determining the category of each CTU, the covariance matrix and the error vector of all pixels in the same category are accumulated respectively. Specifically, for each pixel, its covariance matrix A is a 7x7 square matrix. Let each element of matrix A be a i,j , i, j ∈ [0, 1, 2, …, 5, 6], and for a i,j , the value is: a i,j = R(i) * R(j)

[0122] The value of R(i) is: R(i) = recY(i) - recY(x)

[0123] Where recY(*) represents the reconstructed value of the luma pixel output by the previous module. x corresponds to the position of x in the filter shape, and i is the pixel at the non-x position.

[0124] For the error vector B, it is a 7x1 vector. Let each element of vector B be b i , i ∈ [0, 1, 2, …, 5, 6], and for b i , the value is: b i = R(i) * E

[0125] R(i) has the same meaning as above, and for E, we have: E = orgC-recC

[0126] where recC is the reconstructed pixel value of the pixel to be filtered, and orgC represents the original pixel value of the pixel to be filtered.

[0127] After obtaining the covariance matrix A and the error vector B of each pixel, the covariance matrix and the error vector of the same category of pixels in a frame of image are summed up respectively. The Wiener-Hopf equation Ac = B is constructed, where A is the sum of the covariance matrices of the same category of pixels, and B is the sum of the error vectors of the same category of pixels. The coefficient c of this category needs to be solved, where c is a 7x1 vector.

[0128] It should be noted that the filter coefficients solved by the equation are all floating-point types. Since the value range of floating-point type numbers is almost not limited, a lot of bits are needed to encode the coefficients. In order to solve this problem, the VVC encoder performs integerization on the filter coefficients. First, the filter coefficients are scaled: c' i = scale * c i

[0129] where c i is the filter coefficient solved by the Wiener-Hopf equation, scale is a scale factor, which is set to a fixed value 7 in ECM. c' i is the scaled value. After obtaining c' i , look up table is performed on c' i . By comparison, the number closest to c' i in {-64, -32, -16, -8, -4, -2, -1, 0, 1, 2, 4, 8, 16, 32, 64} is selected as the integerized coefficient c" i .

[0130] After obtaining the integerized filter coefficient c", the next step is to filter the pixels. The filtering process is as follows:

[0131] where recC' is the filtered coefficient. recC, c" i , recY(i), recY(x), resiY(i), scale, etc. have been described above, and will not be repeated here. After filtering the pixels that need to be filtered, they are written into the reconstructed image. For the pixels that do not need to be filtered, they are directly written into the reconstructed image.

[0132] In addition, related syntax elements need to be written into the bitstream. For example, the current frame image CCALF switch, whether each CTU is filtered, and the corresponding category information, etc. If a new filter is needed, the number of filters and the coefficients of each filter need to be written into the APS. Finally, the APS ID used by the frame image is written into the slice header or picture header information.

[0133] At the VVC decoding end, after receiving the bitstream, the current frame image CCALF switch, whether each CTU is filtered, and the corresponding category information, etc. are determined. In addition, the APS ID is parsed, and the corresponding APS is selected to determine the coefficients of the filter. The pixels that need to be filtered are filtered and written into the reconstructed image. For pixels that do not need to be filtered, they are directly written into the reconstructed image.

[0134] The system architecture to which the embodiments of the present application are applied is described below.

[0135] FIG. 2 shows a schematic diagram of the architecture of a coding system 10 to which the embodiments of the present application are applied. As shown in FIG. 2, the coding system 10 can include a source device 110 and a destination device 120. The source device 110 is configured to encode images, and thus, the source device 110 can be referred to as a loop filter apparatus (device) or a video encoding apparatus (device). The destination device 120 is configured to decode the encoded image data generated by the source device 110, and thus, the destination device 120 can be referred to as an image decoding apparatus (device) or a video decoding apparatus (device).

[0136] The specific forms of the source device 110 and the destination device 120 can be various apparatuses or devices, and the embodiments of the present application do not limit the specific forms of the source device 110 and the destination device 120. For example, the source device 110 and the destination device 120 can be a desktop computer, a mobile computing device, a notebook (e.g., a laptop) computer, a tablet computer, a set-top box, a telephone handset such as a so-called “smart” phone, a television, a camera, a display device, a digital media player, a video gaming console, an in-vehicle computer, or other similar devices.

[0137] Optionally, the source device 110 and the destination device 120 shown in FIG. 2 can be two separate devices. Alternatively, the source device 110 and the destination device 120 can be one device, i.e., the source device 110 or the corresponding functions and the destination device 120 or the corresponding functions can be integrated on the same device.

[0138] Optionally, source device 110 and destination device 120 can communicate with each other. For example, destination device 120 can receive encoded image data from source device 110. In one example, source device 110 and destination device 120 can include one or more communication devices that can be used to transmit encoded image data from source device 110 to destination device 120. The one or more communication devices can include routers, switches, base stations, or any other possible devices that facilitate communication from source device 110 to destination device 120, as can be desired in particular implementations, and need not be limited to the examples given here. In some embodiments, source device 110 and destination device 120 can be configured to communicate via a wired or wireless network, such as a wireless network.

[0139] As shown in FIG. 2, source device 110 can include an encoder 112. Optionally, source device 110 can also include an image pre-processor 111 and a communication interface 113. Image pre-processor 111 can be configured to perform pre-processing on received images to be encoded, such as any possible processing including cropping, color format conversion (e.g., from RGB to YUV), color adjustment, or noise reduction. Encoder 112 can be configured to receive images pre-processed by image pre-processor 111, and process the pre-processed images using a relevant prediction mode to output encoded image data. In some embodiments, encoder 112 can be configured to perform the encoding processes described in various embodiments below. Communication interface 113 can be configured to transmit encoded image data output by encoder 112 to destination device 120 or any other device (e.g., a storage device) for storage or direct reconstruction. Other devices can be any device configured to decode or store. Of course, in actual implementations, communication interface 113 can also encapsulate encoded image data output by encoder 112 into a suitable format before transmission.

[0140] Optionally, image pre-processor 111, encoder 112, and communication interface 113 can be hardware components in source device 110, software programs in source device 110, or a combination of hardware components and software programs in source device 110, as can be desired in particular implementations, and need not be limited to the examples given here.

[0141] The destination device 120 can include a decoder 122. Optionally, the destination device 120 can also include a communication interface 121 and an image post-processor 123. The communication interface 121 can be configured to receive the encoded image data from the source device 110 or any other source device, such as a storage device, etc. The communication interface 121 can also de-encapsulate the data transmitted by the communication interface 113 to obtain the encoded image data. The decoder 122 can be configured to receive the encoded image data and output decoded image data (also referred to as reconstructed image data or reconstructed image data). In some embodiments, the decoder 122 can be configured to perform the decoding process described in various embodiments described below. The image post-processor 123 can be configured to perform post-processing on the decoded image data to obtain post-processed image data. The post-processing performed by the image post-processor 123 can include any possible processing, such as color format conversion (e.g., from YUV format to RGB format), toning, retouching or resampling, etc. The image post-processor 123 can also be configured to transmit the post-processed image data to a display device for display.

[0142] Optionally, the communication interface 121, the decoder 122 and the image post-processor 123 can be hardware components in the destination device 120, software programs in the destination device 120, or a combination of hardware components and software programs in the destination device 120. The specific implementation can be determined according to actual use requirements, which is not limited in the embodiments of the present application.

[0143] In order to more clearly understand the encoder 112 and the decoder 122 shown in FIG. 3, the structure of the encoder 112 and the decoder 122 shown in FIG. 3 will be briefly introduced as follows.

[0144] Referring to FIG. 3, FIG. 3 shows a possible structural schematic block diagram of the encoder 112 for implementing the loop filtering method according to an embodiment of the present application.

[0145] As shown in FIG. 3, the encoder 112 can include an intra prediction unit 201, a residual calculation unit 202, a residual transformation unit 203, a quantization unit 204, an encoding unit 205, a dequantization unit (also referred to as inverse quantization unit) 206, a residual inverse transformation unit 207, a reconstruction unit (also referred to as reconstruction unit) 208 and a filter unit 209. Optionally, the encoder 112 can also include a buffer and a decoded image buffer. The buffer can be configured to buffer the reconstructed image block (or referred to as reconstructed image block) output by the reconstruction unit 208. The decoded image buffer can be configured to buffer the filtered image block output by the filter unit 209.

[0146] For the encoder architecture diagram shown in FIG. 3, the filter unit 209 can be a loop filter, also known as an in loop filter, which can include a de blocking filter (DBF), a sample adaptive offset (SAO) filter, an adaptive loop filter (ALF), a cross-component adaptive loop filter (CCALF), etc. Among them, the de blocking filter is used to realize de blocking filtering. In the next generation video coding standard H.266 / Versatile Video Coding (VVC), for all coding block boundaries in the original image, first, the boundary strength is determined according to the coding parameters on both sides of the boundary, and whether to perform de blocking filtering is determined according to the calculated block boundary texture value. Finally, the pixel information on both sides of the coding block boundary is modified according to the boundary strength and the filtering decision. In VVC, after the de blocking filtering is performed, in order to reduce the quantization distortion of high frequency alternating coefficients, the SAO technology, i.e. the sample adaptive offset filter, is introduced; further, from the pixel domain, negative values are added to the pixels at the wave peaks and positive values are added to the pixels at the wave troughs for compensation processing. In VVC, after the de blocking filtering and the sample adaptive offset filtering are performed, the adaptive loop filter is further used for filtering processing; for the adaptive loop filter, it is calculated according to the pixel values of the original image and the pixel values of the distorted image to obtain the optimal filter in the mean square sense.

[0147] In one example, the input of the encoder 112 is an image block of an image to be encoded (i.e. a block to be encoded or a coding unit). In another example, the input of the encoder 112 is an image to be encoded, and the encoder 20 can include a partition unit which can be used to partition the image to be encoded into a plurality of image blocks. The encoder 112 can complete the encoding of the image to be encoded by encoding the plurality of image blocks block by block. For example, the encoder 112 can perform the encoding process on each image block respectively to complete the encoding of the image to be encoded.

[0148] Referring to FIG. 4, FIG. 4 shows another possible structural schematic block diagram of the encoder 112 for implementing the loop filtering method according to an embodiment of the present application.

[0149] As shown in FIG. 4, the encoder 112 can include an intra prediction unit 301, a residual calculation unit 302, a transform / quantization unit 303, an entropy coding unit 304, an inverse quantization / inverse transform unit 305, a reconstruction unit 306, a loop filter unit 307, a decoded picture buffer 308, and a motion estimation / compensation unit 309. Among them, the intra prediction unit 301 is configured to predict a current block to generate a prediction block; the residual calculation unit 202 is configured to calculate the difference between the original image block and the prediction block generated by the intra prediction unit to obtain a residual block; the transform / quantization unit 303 is configured to transformally encode the residual block, and quantize the transform coefficients to map the continuous transform coefficients to a limited number of discrete values; the entropy coding unit 304 is configured to further encode the quantized data, such as using entropy coding (such as Huffman coding, arithmetic coding or CABAC, etc.) to reduce the redundant information in the data, and the encoded data (bitstream) will be sent to the decoder or storage medium; the inverse quantization / inverse transform unit 305 is configured to inverse quantize the quantized data to recover the transform coefficients, and inverse transform the transform coefficients to recover the residual block; the reconstruction unit 306 is configured to add the prediction block generated by the intra prediction unit and the residual block recovered by the inverse transform unit to reconstruct the original image block; the loop filter unit 307 is configured to filter the reconstructed image block to improve the image quality and reduce visual artifacts such as blocking effect; the decoded picture buffer 308 is configured to buffer the filtered image block output by the loop filter unit 307; and the motion estimation / compensation unit 309 is configured to obtain the filtered image block and use the image block as a reference block for subsequent motion estimation or motion compensation.

[0150] For the encoder architecture diagram described in FIG. 4, the loop filter unit 307 can include a de-blocking filter (DBF), a sample adaptive offset (SAO) filter, and an adaptive loop filter (ALF), etc.

[0151] Specifically, in the embodiments of the present application, the filter unit 209 or the loop filter unit 307 can be configured to implement the loop filtering method described in the following embodiments.

[0152] The loop filtering method of the embodiments of the present application will be described below taking the loop filter unit 307 performing the loop filtering method described in the following embodiments as an example.

[0153] In one example, a loop filtering method implemented by the loop filter unit 307 can include the following steps:

[0154] Step 11: The loop filter unit 307 performs pixel padding at the first boundary of the image unit based on the first padding mode.

[0155] The first padding mode is determined based on at least one of the following: a projection format corresponding to the image unit, and a rate-distortion decision result of pixel padding at the boundary of the reference image.

[0156] Step 12: The loop filter unit 307 performs loop filtering on the image unit after pixel padding based on the first filter.

[0157] Optionally, in the embodiments of the present application, the step 11 can be implemented by the following step 11a.

[0158] Step 11a: When the distance between the pixel to be filtered and the first boundary of the image unit is less than M, the loop filter unit 307 performs pixel padding at the first boundary based on the first padding mode.

[0159] The pixel to be filtered is a pixel in the image unit, the value of M is determined according to the first filter, and M is a positive integer.

[0160] Optionally, in the embodiments of the present application, the step 11 can be implemented by the following step 11b.

[0161] Step 11b: The loop filter unit 307 fills the first pixel information to the first boundary based on the first padding mode.

[0162] The first pixel information is information of a pixel corresponding to the first padding mode within the first boundary.

[0163] Optionally, in the embodiments of the present application, the first filter can be ALF; and the first pixel information can include at least one of the following:

[0164] luminance information after SAO processing;

[0165] chroma information after SAO processing;

[0166] luminance information before deblocking filtering;

[0167] chroma information before deblocking filtering;

[0168] luminance residual information;

[0169] luminance information after luminance fixed filter processing;

[0170] chroma information after chroma fixed filter processing;

[0171] luminance information after luminance Gaussian filter processing.

[0172] Optionally, in the embodiment of the present application, the first filter can be a CCALF; and the first pixel information can include at least one of the following:

[0173] luminance information after SAO processing;

[0174] chroma information after SAO processing;

[0175] luminance information before de-blocking filtering;

[0176] chroma information before de-blocking filtering;

[0177] luminance residual information;

[0178] luminance information after luminance fixed filter processing;

[0179] chroma information after chroma fixed filter processing;

[0180] luminance information after luminance Gaussian filter processing.

[0181] Optionally, in the embodiment of the present application, the image unit is an image, and the first boundary of the image unit is a virtual boundary of the image.

[0182] Optionally, in the embodiment of the present application, the image unit is a coding tree unit (CTU), and the first boundary of the image unit is a virtual boundary of the CTU.

[0183] Optionally, in the embodiment of the present application, the step 11 can be implemented by the following step 11c.

[0184] Step 11c: If there is no adjacent CTU at the first boundary, the loop filtering unit 307 performs pixel padding at the boundary of the CTU based on a first padding mode.

[0185] Optionally, in the embodiment of the present application, the first boundary is a boundary of the CTU; and the step 11b can be implemented by the following step 11b1.

[0186] Step 11b1: The loop filtering unit 307 fills information of N rows of pixels or N columns of pixels in the first boundary to the first boundary based on a first padding mode; the N rows of pixels are the 1st row of pixels to the Nth row of pixels in the first boundary, the 1st row of pixels is a pixel in a row adjacent to the first boundary, the N columns of pixels are the 1st column of pixels to the Nth column of pixels in the first boundary, the 1st column of pixels is a pixel in a column adjacent to the first boundary, and N is a positive integer.

[0187] Optionally, in the embodiments of the present application, the first padding manner is a mirror padding manner; and the step 11b1 can be implemented by step 13, for example.

[0188] Step 13: The loop filter unit 307 mirrors the information of N rows or N columns of pixels within the first boundary to the first boundary based on the mirror padding manner, with the first boundary as the symmetry axis.

[0189] Optionally, in the embodiments of the present application, the first boundary of the image unit is a region outside the first boundary and adjacent to the first boundary.

[0190] Referring to FIG. 5, a possible structural schematic block diagram of the decoder 122 for implementing the loop filtering method of the embodiments of the present application is shown.

[0191] The decoder 122 can be configured to receive image data (i.e., an encoded bitstream, for example, including an encoded bitstream of image blocks and syntax elements associated therewith) encoded by the encoder 112, for example, to obtain decoded image blocks.

[0192] As shown in FIG. 5, the decoder 122 can include a bitstream parsing unit 401, an inverse quantization unit 402, a residual inverse transform unit 404, a prediction processing unit 404, a reconstruction unit 405, and a filter unit 406. In some examples, the decoder 122 can perform a decoding process generally reciprocal to the encoding process described for the encoder 112. Optionally, the decoder 122 can also include a buffer and a filtered image buffer. The buffer can be configured to buffer reconstructed image blocks output by the reconstruction unit 405, and the filtered image buffer can be configured to buffer filtered image blocks output by the filter unit 406.

[0193] The bitstream parsing unit 401 can be configured to perform decoding on the encoded bitstream to obtain quantized residual coefficients (or quantized residual values) and / or decoding parameters (e.g., the decoding parameters can include any one or more of inter prediction parameters, intra prediction parameters, filter parameters, and / or other syntax elements performed on the encoding side). The bitstream parsing unit 401 can also be configured to forward the decoding parameters to the prediction processing unit 304, so that the prediction processing unit 304 performs a prediction process according to the decoding parameters.

[0194] The inverse quantization unit 402 can have the same function as the inverse quantization unit 206 of the encoder 112, and is configured to inverse quantize (i.e., dequantize) the quantized residual coefficients decoded by the bitstream parsing unit 401.

[0195] The residual inverse transform unit 403 can have the same function as the residual inverse transform unit 207 of the encoder 112, for performing inverse transform (e.g., inverse DCT, inverse integer transform, or conceptually similar inverse transform process) on the aforementioned dequantized residual coefficients to obtain reconstructed residual values. The block obtained after the inverse transform is the residual block of the reconstructed to-be-decoded block in the pixel domain.

[0196] The reconstruction unit 405 (e.g., a summer) can have the same function as the reconstruction unit 208 of the encoder 112.

[0197] The prediction processing unit 404 is configured to receive or obtain the encoded image data (e.g., the encoded bitstream of the current image block) and the reconstructed image data. The prediction processing unit 404 can also receive or obtain the related parameters of the prediction mode and / or information about the selected prediction mode (i.e., the aforementioned decoding parameters) from, for example, the bitstream parsing unit 302, and predict the current image block based on the related data in the reconstructed image data and the decoding parameters to obtain the prediction block of the current image block.

[0198] The reconstruction unit 405 can be configured to add the reconstructed residual block to the prediction block to obtain the reconstructed block of the to-be-decoded image in the sample domain, for example, by adding the residual values in the reconstructed residual block to the prediction values in the prediction block.

[0199] The filter unit 406 can be configured to filter the reconstructed block to obtain a filtered block, which is the decoded image block.

[0200] Specifically, in the embodiments of the present disclosure, the filter unit 306 can be configured to implement the loop filtering method described in the following embodiments.

[0201] It can be understood that in the encoder 112 and the decoder 122 provided in the embodiments of the present disclosure, the processing result of a certain link can also be further processed and then output to the next link. For example, after the prediction, transform, or filtering link, the processing result of the corresponding link can be further subjected to a clip or shift operation.

[0202] In one example, a decoding method implemented by the decoder 122 can include the following steps:

[0203] Step 21: The filter unit 406 parses the bitstream to obtain a syntax element.

[0204] The aforementioned syntax element is used to indicate a first padding mode used when performing loop filtering.

[0205] Step 22: The filter unit 406 performs pixel padding at the first boundary of the image unit based on the first padding mode.

[0206] Step 23: The filter unit 406 performs loop filtering on the image unit after pixel filling based on the first filter.

[0207] Optionally, in an embodiment of the present application, the above step 22 can be implemented through the following step 22a.

[0208] Step 22a: When the distance from the pixel to be filtered to the first boundary of the image unit is less than M, perform pixel filling at the first boundary based on the first filling manner.

[0209] Wherein, the pixel to be filtered is a pixel in the image unit, the value of M is determined according to the first filter, and M is a positive integer.

[0210] Optionally, in an embodiment of the present application, the above step 22 can be implemented through the following step 22b.

[0211] Step 22b: The filter unit 406 fills the first pixel information to the first boundary of the image unit based on the first filling manner.

[0212] Wherein, the first pixel information is the information of the pixel corresponding to the first filling manner within the first boundary.

[0213] Optionally, in an embodiment of the present application, the above step 23 can be implemented through the following step 23a.

[0214] Step 23a: The filter unit 406 performs loop filtering on the pixel to be filtered in the image unit after pixel filling based on the first filter.

[0215] Wherein, the pixel to be filtered is a pixel in the pixel corresponding to the first filling manner within the first boundary.

[0216] Optionally, in an embodiment of the present application, the first filter can be ALF; and exemplarily, the above first pixel information can include at least one of the following:

[0217] Luminance information after SAO processing;

[0218] Chrominance information after SAO processing;

[0219] Luminance information before deblocking filtering;

[0220] Chrominance information before deblocking filtering;

[0221] Luminance residual information;

[0222] Luminance information after luminance fixed filter processing;

[0223] Chrominance information after chrominance fixed filter processing;

[0224] luma information processed by a luma fixed filter.

[0225] Optionally, in the embodiments of the present application, the first filter can be CCALF; and the first pixel information can include at least one of the following:

[0226] luma information processed by SAO;

[0227] chroma information processed by SAO;

[0228] luma information before the de-blocking filter;

[0229] chroma information before the de-blocking filter;

[0230] luma residual information;

[0231] luma information processed by a luma fixed filter;

[0232] chroma information processed by a chroma fixed filter;

[0233] luma information processed by a luma Gaussian filter.

[0234] Optionally, in the embodiments of the present application, the image unit is an image, and the first boundary of the image unit can be a virtual boundary of the image.

[0235] Optionally, in the embodiments of the present application, the image unit is a CTU, and the first boundary of the image unit can be a virtual boundary of the CTU.

[0236] Optionally, in the embodiments of the present application, the image unit is a CTU; and the step 22 can be implemented by step 22c.

[0237] Step 22c: if there is no adjacent CTU at the first boundary of the CTU, the filter unit 406 performs pixel padding at the boundary of the CTU based on a first padding mode.

[0238] Optionally, in the embodiments of the present application, the step 22b can be implemented by step 22b1.

[0239] Step 22b1: the filter unit 406 fills the information of N rows of pixels or N columns of pixels within the first boundary to the first boundary based on the first padding mode; the N rows of pixels are the 1st row of pixels to the Nth row of pixels within the first boundary.

[0240] wherein the 1st row of pixels is a pixel of a row adjacent to the first boundary, the N columns of pixels are the 1st column of pixels to the Nth column of pixels within the first boundary, the 1st column of pixels is a pixel of a column adjacent to the first boundary, and N is a positive integer.

[0241] Optionally, in the embodiment of the present application, the first padding mode is a mirror padding mode; and the step 22b1 can be implemented by the following step 24.

[0242] Step 24: The loop filter unit 406 performs mirror padding on the N rows or N columns of pixels within the first boundary to the first boundary based on the mirror padding mode and taking the first boundary as the symmetry axis.

[0243] Optionally, in the embodiment of the present application, the first boundary of the image unit is a region outside the first boundary and adjacent to the first boundary.

[0244] At present, in the filtering process, a loop filter (such as ALF or CCALF) filters the reconstructed information output by the previous stage. When filtering the pixels of the image boundary, if the distance from the pixel to be filtered to the boundary of the image unit is less than a preset value M, that is, the filtering template exceeds the boundary of the image, it is necessary to perform pixel padding on the boundary of the image. In the related art, a row or a column of pixel values of the image boundary is usually repeatedly copied to the boundary. However, according to the above pixel padding mode, when the number of rows or columns of pixels repeatedly padded in the region outside the boundary is large, the texture topology of the image boundary after filtering changes, thereby reducing the filtering effect at the image boundary and further reducing the filtering performance.

[0245] The loop filtering method of the embodiment of the present application can adaptively determine the padding mode based on the projection format corresponding to the image unit or the rate-distortion decision result of the pixel padding at the boundary of the reference image, and perform pixel padding on the boundary of the image unit based on the padding mode. Since the boundary characteristics of images of different projection formats are usually different, the padding mode of directly copying a row or a column of pixels of the boundary cannot well adapt to the image boundary characteristics under different projection formats, and determining the corresponding padding mode based on the projection format corresponding to the image unit can adaptively select a suitable padding strategy according to the image characteristics of different projection formats, thereby avoiding the distortion or artifacts of the image after filtering, and improving the filtering effect on the boundary region of the image block. In addition, the rate-distortion decision result of the pixel padding at the boundary of the reference image is introduced, and the image quality change after encoding under different padding modes is evaluated, so that the padding mode with the best filtering effect while ensuring the encoding efficiency can be selected. In this way, the projection format of the image unit or the rate-distortion decision result is used to adaptively determine the padding mode in the present application, which can avoid the change of the texture topology at the boundary of the image after filtering, improve the quality of the image after filtering, and thus improve the filtering performance.

[0246] The loop filtering method provided by the embodiment of the present application will be described exemplarily in combination with the accompanying drawings.

[0247] FIG. 6 is a flow diagram of a loop filtering method according to an embodiment of the present disclosure. The loop filtering method can be applied to an encoder. As shown in FIG. 6, the loop filtering method can include the following steps 501-503.

[0248] Step 501: performing pixel padding at a first boundary of a picture unit based on a first padding mode.

[0249] The first padding mode is determined based on at least one of a projection format corresponding to the picture unit and a rate-distortion decision result of pixel padding at a boundary of a reference picture.

[0250] In some embodiments of the present disclosure, the picture unit includes, but is not limited to, at least one of a picture unit containing a virtual boundary, a picture unit corresponding to image information output by a second filter, and a picture unit before deblocking filtering.

[0251] For example, the second filter includes, but is not limited to, at least one of an SAO, a luma fixed filter, a luma Gaussian filter, a luma new filter, a chroma fixed filter, a chroma new filter, and the like.

[0252] In some embodiments of the present disclosure, the image information of the picture unit can include at least one of chroma information, luma information, and residual information.

[0253] It should be noted that the chroma information can also be referred to as chroma component or chroma data, the luma information can also be referred to as luma component or luma data, and the residual information can be referred to as residual data. In the fields of video coding, image processing, and color science, the chroma information and the luma information are two important aspects of describing the characteristics of an image. In YUV, HSI, and other color spaces, the chroma component (e.g., U, V, or H, S) and the luma component (e.g., Y or I) are separated, which makes it possible to process the chroma information and the luma information independently.

[0254] In some embodiments of the present disclosure, the chroma information can be reconstructed chroma information, the luma information can be reconstructed luma information, and the residual information can be difference information between a reconstructed image and an original image.

[0255] In some embodiments of the present disclosure, the picture unit can be a reconstructed image, or an image block in the reconstructed image. Further, the picture unit can be a luma reconstructed image or a chroma reconstructed image, or the picture unit can be an image block in the luma reconstructed image or an image block in the chroma reconstructed image.

[0256] It should be noted that the luma reconstructed image can be an image containing reconstructed luma information, and the chroma reconstructed image can be an image containing reconstructed chroma information.

[0257] It should be noted that reconstruction can also be referred to as reconstruction.

[0258] It should be noted that the reconstructed image refers to the image obtained after a series of operations such as encoding, quantization, inverse quantization, etc. in the process of video encoding and decoding. The size and shape of the image are determined by filters (such as filtering templates), which are usually consistent with the original image. As part of the video frame, the quality of the reconstructed image will affect the subjective effect of the entire video. By filtering the reconstructed image through post-processing techniques such as loop filtering, errors and distortions in the encoding process can be reduced, improving the quality of the reconstructed image and thus improving the quality of the entire video.

[0259] The reconstructed chrominance information, reconstructed luminance information and residual information involved in the embodiments of the present application are explained and described below.

[0260] The reconstructed chrominance information refers to the information describing the color properties of an image obtained after a series of processes such as encoding, compression, transmission and decoding of the image or video through corresponding reconstruction algorithms. Chrominance information mainly focuses on the color components in the image, including hue and saturation, which together determine the types and shades of colors in the image. Since chrominance information usually contains less detail and texture information, it is more susceptible to compression algorithms than luminance information during video encoding, resulting in some distortion in the reconstructed chrominance information. By filtering the reconstructed chrominance information through ALF or CCALF loop filters, the quality of the reconstructed image can be improved.

[0261] The reconstructed luminance information refers to the information describing the brightness of an image obtained after a series of processes such as encoding, compression, transmission and decoding of the image or video through corresponding reconstruction algorithms. Luminance information mainly focuses on the gray level and brightness contrast in the image, and the quality of the reconstructed luminance information directly affects the visual effect of the image. High-quality luminance information can present a clearer and more natural image effect, while low-quality luminance information may result in image blurring, insufficient contrast and other problems. By filtering the reconstructed luminance information through ALF or CCALF loop filters, the reconstruction quality of the information can be optimized, thereby improving the quality of the reconstructed image.

[0262] In the context of video encoding or image compression, residual information usually refers to the difference between the original image or video frame and the predicted image or video frame. However, due to the introduction of inverse transform and quantization steps, there will still be some differences between the reconstructed image and the original image. This difference is usually referred to as reconstruction error or reconstruction distortion. In order to describe this difference, the difference between the reconstructed image and the original image is referred to as the reconstructed residual information.

[0263] In some embodiments of the present application, the first boundary refers to a set of pixels at the outermost layer of the image unit, which can be a row of pixels or a column of pixels at the outermost side, and the set of pixels marks the edge of the image.

[0264] It can be understood that the boundary of an image represents a demarcation line between a region (e.g., a part of the image or the entire image) and the surrounding non-region part. In video coding, an image is regarded as a two-dimensional array composed of pixels, and the boundary is the outermost layer of the array. For the entire image, the boundary usually refers to the first row, the first column, the last row and the last column of pixels of the image.

[0265] In some embodiments of the present application, the first boundary described above can include at least one of an upper boundary, a lower boundary, a left boundary and a right boundary.

[0266] In some possible implementations, the image unit described above is an image. Illustratively, the first boundary described above can include at least one of a boundary of the image and a virtual boundary of the image.

[0267] In some embodiments of the present application, the image described above can be an image frame, a slice, a block, etc.

[0268] In some embodiments of the present application, the image described above can contain the reconstructed chroma information, the reconstructed luma information or the luma residual information described above.

[0269] In some embodiments of the present application, the boundary of the image refers to the actual edge of the image in the physical space or the pixel grid, i.e., the natural limit of the image data when stored or transmitted, which is usually determined by the width and height of the image.

[0270] It should be noted that the boundary of the image can also be referred to as the physical boundary of the image.

[0271] In some embodiments of the present application, the virtual boundary of the image refers to a boundary set inside the image.

[0272] It should be noted that the virtual boundary does not directly refer to the physical boundary line actually drawn or set inside the image, but refers to the boundary logically or computationally assumed for a specific purpose when processing the image.

[0273] It should be noted that in video coding and image processing, the virtual boundary is often used to solve edge effects and reduce artifacts.

[0274] The video frame of the 360 video projection format can include multiple faces, and when the faces are placed into the coding frame structure through translation, rotation, flipping, scaling, padding, and the like, discontinuous boundaries can be caused inside the coding image frame. If the six faces of the CMP are placed in a two-row three-column manner: the right face, the front face, and the left face are placed in the first row, and the lower face, the back face, and the upper face are placed in the second row, the content on both sides of the boundary between the first row and the second row is discontinuous. In this case, if in-loop filtering, the pixels on the other side are referenced when filtering on both sides near the boundary, a seam phenomenon occurs when the 360 video is normally displayed. In order to avoid the seam phenomenon, a virtual boundary is usually defined inside the image, and the position information of the virtual boundary can be encoded in the sequence parameter set (SPS) or the picture header (PH) to avoid the above seam phenomenon.

[0275] In some possible implementation manners, the image unit can be a coding tree unit (CTU). For example, the first boundary includes at least one of a boundary of the CTU or a virtual boundary of the CTU.

[0276] In some embodiments of the present application, the CTU can include the reconstructed chroma information, the reconstructed luma information, or the luma residual information.

[0277] In some embodiments of the present application, the boundary of the CTU refers to the actual boundary of the CTU in the image or frame, that is, the CTU boundary. Further, the boundary of the CTU can be determined by the size and position of the CTU.

[0278] It should be noted that the boundary of the CTU can also be referred to as the physical boundary of the CTU.

[0279] In some embodiments of the present application, the virtual boundary of the CTU refers to the boundary set inside the CTU. Further, the setting of the virtual boundary of the CTU depends on the actual processing requirement.

[0280] In the context of video coding, the CTU is a basic processing unit in the coding process, and the CTU can be composed of multiple smaller coding units (such as CU, PU, TU, and the like) for performing prediction, transformation, quantization, coding, and the like.

[0281] In some embodiments of the present application, the first padding mode includes, but is not limited to, any one of the following: mirror padding, replicate padding, zero padding, and circular padding.

[0282] The filling modes involved in the embodiments of the present application are explained and described as follows.

[0283] Mirror padding: Mirror padding is to mirror the pixel values within the image boundary to the outside of the boundary. This method can maintain the continuity of pixel values at the boundary and reduce the unnaturalness after filtering. Mirror padding is suitable for scenarios that require maintaining the continuity of pixel values at the image boundary, such as edge detection, image enhancement, and the like.

[0284] Zero padding: Zero padding is to directly fill zero-value pixels outside the image boundary. This method is simple and direct, but may produce obvious black edges or color discontinuity at the image boundary after filtering. Zero padding is suitable for scenarios that have low requirements for boundary processing or the filtering template is small and the boundary effect is not obvious.

[0285] Copy padding: Copy padding is to directly copy the pixel values on the image boundary to the outside of the boundary. This method can also maintain the continuity of pixel values at the boundary, but compared with mirror padding, it is easy to produce a blur effect after filtering. Copy padding is suitable for scenarios that have not very strict requirements for boundary processing and want to maintain a certain continuity.

[0286] Periodic padding: Periodic padding (also known as cyclic padding) is to regard the image as a periodic two-dimensional array, and when the filtering template exceeds the image boundary, the pixel values are read from the other side of the image again. This method can simulate an infinite image and reduce the boundary effect. Periodic padding is suitable for scenarios that need to simulate an infinite image or a periodic image, such as texture synthesis, image denoising, and the like.

[0287] It should be noted that when the reconstructed image information is filtered by a filter or a filtering template, the pixel padding mode is an important consideration factor, which determines the processing manner of the filtering template on the image boundary or edge pixel, and then can affect the filtering effect on the image. In addition, different padding modes can have different effects on the filtering result, and the filtering efficiency under different padding modes is usually different, so in actual application, a suitable padding mode can be selected to improve the filtering effect while ensuring the filtering efficiency.

[0288] In the embodiments of the present application, the appropriate padding mode is adaptively determined through the projection format of the image unit or the rate-distortion decision result, which can better adapt to the image content, avoid the change of the texture topology at the image boundary after filtering, and thus improve the filtering performance.

[0289] In some embodiments of the present application, the projection format corresponding to the image unit refers to the projection format of the original video or the original video frame corresponding to the image unit.

[0290] It should be noted that the projection format mentioned above refers to the projection format of projecting the video frame of the 360 video from the spherical surface to the 2D plane.

[0291] It can be understood that the reconstructed image (such as the chroma image) is obtained by a series of processing (such as coding, filtering, etc.) on the original video frame (including chroma component and luminance component). The current filtered image block is the result of filtering processing on the data of the corresponding position in the original video frame. Therefore, in the reconstructed chroma image, the current filtered image maintains the same spatial position relationship as the corresponding image in the original video frame.

[0292] In some embodiments of the present application, the projection format corresponding to the image unit includes but is not limited to any one of the following: equirectangular projection (ERP), adjusted equal-area projection (AEP), cubemap projection (CMP), adjusted cubemap projection (ACP), equi-angular cubemap (EAC), hybrid equi-angular cubemap (HEC), generalized cubemap projection (GCMP), segmented sphere projection (SSP), rotated sphere projection (RSP), fisheye projection, octahedron projection (OHP), icosahedron projection (ISP), truncated square pyramid projection (TSP), equatorial cylindrical projection (ECP).

[0293] It should be noted that the above-mentioned 14 kinds of 360 video 2D plane projection formats are currently specified, and these projection formats are used to map the spherical content of 360-degree video (also known as panoramic video or VR video) to a two-dimensional plane, so as to facilitate subsequent coding, storage and transmission.

[0294] It should be noted that the specific implementation and parameters of the projection format above can be different in different application scenarios, and in actual application, a suitable projection format can be selected according to specific needs and scenarios.

[0295] In some embodiments of the present application, there can be a mapping relationship between a projection format and a padding mode. Further, different projection formats can correspond to the same or different padding modes.

[0296] Exemplarily, the equirectangular projection can have a mapping relationship with the mirror padding mode.

[0297] It should be noted that the mapping relationship between the projection format and the padding mode can be set according to actual needs, and the embodiments of the present application do not limit this.

[0298] In some embodiments of the present application, when performing pixel padding at the first boundary of the image unit in the filtering process, the projection format corresponding to the image unit can be obtained, and the padding mode corresponding to the projection format corresponding to the image unit is determined as the first padding mode to be used.

[0299] In some embodiments of the present application, the rate-distortion decision result of the pixel padding at the boundary of the reference image refers to the selection result of the best padding mode obtained by evaluating the filtering effect under different padding modes.

[0300] In some embodiments of the present application, the reference image above can be another reconstructed image.

[0301] It should be noted that the rate-distortion decision can calculate the difference (such as mean square error MSE) between the multiple filtered images obtained by filtering after applying different padding modes to fill the boundary pixels of the reference image and the original image, and determine the best padding mode with the smallest difference.

[0302] The process of determining the first padding mode according to the rate-distortion decision result of the pixel padding at the boundary of the reference image is described below.

[0303] Exemplarily, the reference image is filtered I times, in one filtering, pixel padding is performed in a repeated padding mode, and the reference after pixel padding is filtered to obtain a filtered image P1; in another filtering, pixel padding is performed in a mirror padding mode, and the reference image after pixel padding is filtered to obtain a filtered image P2; after the reference image is padded with the padding mode I and the reference after padding is filtered to obtain an image PI, the image P1 to the image PI are compared with the original image corresponding to the reference image respectively, a target image with the smallest difference from the original image is determined through rate distortion technology, and a pixel padding mode corresponding to the target image is determined as the best padding mode, i.e. the first padding mode.

[0304] It should be noted that I can be an integer greater than 1, and the value of I can be determined based on the number of padding modes to be determined actually, which is not limited in the embodiments of the present application.

[0305] In some embodiments of the present application, in the process of filtering the image unit, at least one row of pixel information or at least one column of pixel information can be padded in the to-be-padded region outside the first boundary of the image unit and directly adjacent to the first boundary based on the first padding mode.

[0306] It should be noted that the explanation of the process of pixel padding can be referred to in the following, which is not described herein.

[0307] Step 502: performing loop filtering on the image unit after padding pixels based on the first filter.

[0308] In some embodiments of the present application, the first filter can be a loop filter.

[0309] Exemplarily, the first filter includes but is not limited to at least one of ALF and CCALF.

[0310] It should be noted that ALF and CCALF are an important post-processing technology in video encoding, which is used to further improve the quality of decoded video and reduce block effect and other compression artifacts. ALF filters the luminance reconstructed image and the chrominance reconstructed image respectively, uses different sizes of filter templates for luminance components and chrominance components, and CCALF filters the chrominance reconstructed image.

[0311] In some embodiments of the present application, the encoder can perform loop filtering on the boundary region after padding pixels in the image unit based on the first filter.

[0312] In some embodiments of the present application, the boundary region can be a region containing to-be-filtered pixels at the first boundary.

[0313] In some embodiments of the present application, the pixel information required to be used when performing loop filtering on the pixels at the boundary based on the first filter is included in the boundary region.

[0314] It should be noted that performing loop filtering on the boundary region after pixel padding refers to performing loop filtering on the boundary region after pixel padding, and the input information of the loop filtering can include the pixel information of the boundary region after pixel padding, and the output information of the loop filtering can be the filtering result of the pixels in the first boundary.

[0315] In some embodiments of the present application, the first filter can filter the image unit using a filter template. The filter template can be a filter template used when filtering the reconstructed image unit by the first filter.

[0316] In some embodiments of the present application, the filter template includes but is not limited to at least one of the following: a chroma filter template, a luminance filter template, a template for filtering residual information (such as a residual filter template).

[0317] The filter templates of ALF and CCALF will be explained in detail below.

[0318] In VVC (Versatile Video Coding) or similar video coding standards, ALF defines two main shapes of filter templates, namely luminance filter template and chroma filter template, which are used for filtering of luminance components and chroma components respectively. Among them, the luminance filter template usually uses a 7x7 diamond template, which contains more pixel points and can capture the changes of luminance components more carefully, thereby better filtering processing; the chroma filter template usually uses a 5x5 diamond template, compared with the luminance component, the chroma component usually contains less high frequency information, so a smaller filter template can meet the demand, at the same time, it can also reduce the calculation complexity; the filter template of CCALF usually uses a 4x3 diamond template.

[0319] FIG. 7A is a schematic diagram of a luminance filter template of ALF, as shown in FIG. 7A, the luminance filter template can be a 7x7 diamond template, wherein "x" located at the center position of the template represents the position of the to-be-filtered chroma pixel, and the numbers filled in the filter template represent the adjacent luminance pixels.

[0320] FIG. 7B is a schematic diagram of a chroma filter template of ALF, as shown in FIG. 7B, the chroma filter template can be a 5x5 diamond template, wherein "x" located at the center position of the template represents the position of the to-be-filtered chroma pixel, and the numbers filled in the filter template represent the adjacent luminance pixels.

[0321] FIG. 8 is a schematic diagram of a CCALF filter template, as shown in FIG. 8, the chroma filter template can be a 4x3 diamond-shaped template, where "x" at the center of the template represents the position of the luma pixel corresponding to the chroma pixel to be filtered, and the numbers filled in the filter template represent the neighboring luma pixels.

[0322] Further, the numbers filled in the filter template (including luma filter and chroma filter) generally represent the weight coefficients of the filter template, which determine the degree of influence of surrounding pixels on the current pixel in the filtering process. Specifically, each number in the filter template is multiplied by the pixel value at the corresponding position in the template coverage area, and then the products are added to obtain a result that is used to replace the pixel value at the center of the template or for further processing. The weight coefficients determine the characteristics of the filter, such as smoothing degree, edge preservation ability, etc.

[0323] It should be noted that the size of the template determines the number of pixel points involved in the filtering. For the luma component, a 7x7 template can provide more refined filtering effect; while for the chroma component, a 5x5 template can meet the filtering requirements while keeping low computational complexity. Each pixel point in the filter template corresponds to a filter coefficient, which determines the degree of contribution of the pixel point to the filtering result. The selection of filter coefficients is obtained through an optimization algorithm, aiming to optimize the quality of the filtered image. In addition, different filter algorithms and application scenarios may use filter templates of different shapes, sizes and weight coefficients. For example, in luma filtering, different sizes of square, rectangular or diamond-shaped templates may be used, and the weight coefficients in the template may also be adjusted according to the specific algorithm requirements, therefore, by selecting appropriate filter coefficients and template shapes, the filtered image can be made closer to the original image.

[0324] It should be noted that the shape of the filter template, the selection of filter coefficients, etc. can be determined according to actual needs, and the embodiments of the present application do not limit this.

[0325] In some embodiments of the present application, in the loop filtering process, the pixels in the image unit are filtered by sliding the filter template on the image unit. Illustratively, filtering generally includes steps such as calculating boundary strength, selecting filter parameters, applying filter kernel, etc., and finally obtaining a filtered image block.

[0326] It can be understood that based on the first filter, the loop filtering is performed on the reconstructed image unit, which can adjust the pixel values of the image unit, reduce errors and distortions in the encoding process, and thus improve the overall quality of the image unit, including detail preservation, edge sharpening, etc. By selecting appropriate filter coefficients and template shapes, the filtered image can be made closer to the original image.

[0327] It should be noted that in video coding and image processing, when loop filtering is performed on reconstructed information, the area currently requiring filtering of the filtering template or the area currently covered by the filtering template can be referred to as a filtering area. The size and shape of the filtering area can be determined by the filtering template. The size of the filtering template can be adjusted as needed to adapt to different image content and coding requirements.

[0328] In some embodiments of the present application, in the process of loop filtering the image unit, the pixels (or pixel groups) at the first boundary can be filtered based on the above-mentioned filtering template to adjust the pixel values of the pixels to be filtered within the first boundary, so as to improve the overall quality of the image, and thus improve the quality of the reconstructed image. Specifically, when the pixels at the first boundary are loop filtered, the center of the filtering template to which the first filter is applied is coincided with the current pixel position requiring processing, the coefficients on the filtering template are operated with the pixels corresponding to the filtering template, and the operation result is assigned to the pixel at the position corresponding to the center of the filtering template, so as to complete the filtering of the pixel.

[0329] It should be noted that loop filtering utilizes the principle of feedback loop to achieve the effect of filtering by continuously feeding back and adjusting the signal. Specifically, the loop filter mainly consists of a feedback path and a feedforward path. In the feedback path, the output signal is sampled and returned to the input end through the feedback loop, and is added to the input signal. This feedback structure can make the filter have a certain gain or attenuation effect on signals of a certain frequency, thereby achieving the function of filtering.

[0330] The loop filtering method provided by the embodiments of the present application can adaptively determine the filling mode based on the projection format corresponding to the image unit or the rate-distortion decision result of the pixel filling at the boundary of the reference image in the process of loop filtering the image unit. Since the image boundary characteristics of image blocks of different projection formats are usually different, the filling mode of directly copying one row or one column of pixels of the boundary cannot well adapt to the image boundary characteristics under different projection formats, and determining the corresponding filling mode based on the projection format corresponding to the image unit can adaptively select a suitable filling strategy according to the image characteristics of different projection formats, thereby avoiding distortion or artifacts of the filtered image, and thus improving the filtering effect on the boundary area of the image block. In addition, the rate-distortion decision result of the pixel filling at the boundary of the reference image is introduced, and the image quality change after coding under different filling modes is evaluated, so that the filling mode with the best filtering effect while ensuring coding efficiency can be selected. In this way, the filling mode is adaptively determined based on the projection format of the image unit or the rate-distortion decision result in the present solution, which not only avoids the change of the texture topology at the boundary of the filtered image, thereby improving the filtering performance.

[0331] Step 503: write the filtering parameters and syntax elements of the loop filtering into the bitstream.

[0332] The syntax element is used to indicate the first padding mode.

[0333] In some embodiments of the present application, the filtering parameters include, but are not limited to, at least one of the following: filtering strength, filtering type, etc.

[0334] Exemplarily, the filtering type includes, but is not limited to, deblocking filtering, sample adaptive offset, adaptive loop filtering, etc.

[0335] It should be noted that, in order to indicate the padding mode used in the encoding process, a syntax element (such as a flag) can be introduced, which is written into the bitstream and transmitted to the decoding end in the encoding process.

[0336] Exemplarily, the syntax element is represented as follows:

[0337] Exemplarily, if the syntax element sh_alf_enabled_flag takes the value of 1, it indicates that the mirror padding is adopted; if the syntax element takes the value of 0, it indicates that the simple padding (i.e. repeated padding) is adopted.

[0338] In some embodiments of the present application, the filtering parameters and syntax elements of the loop filtering can be entropy coded to obtain a binary bitstream before performing the loop filtering, or in the process of performing the loop filtering, or after performing the loop filtering.

[0339] It should be noted that, in addition to the indication of the padding mode, other related syntax elements can also be coded into the bitstream so that the decoder can correctly reconstruct the video frame.

[0340] In some embodiments of the present application, the bitstream can also contain information for reconstructing the image, including but not limited to image data, coding mode, etc. For example, in the encoding process, other syntax elements related to loop filtering can be written into the bitstream, such as image data, current frame image CCALF switch state, whether each CTU is filtered and the category information of the filter used, etc.

[0341] Exemplarily, the current frame image CCALF switch state is used to indicate whether the current frame can be applied to the CCALF for loop filtering.

[0342] It should be noted that the above entropy encoding process generally follows the hierarchy of encoding standards and the data stream organization manner, and all determined filtering parameters and related syntax elements are integrated into the video code stream, for example, in HEVC (H.265), the syntax elements can be organized into sequence parameter set (SPS), picture parameter set (PPS) or video parameter set (VPS), and transmitted in the code stream in a specific order and format.

[0343] In some embodiments of the present application, the obtained code stream after encoding is transmitted to the decoding end (i.e. the decoder), so that the decoding end can parse the filtering parameters and syntax element information from the code stream after receiving and decoding the code stream, thereby being able to perform pixel padding at the boundary according to the indicated padding mode when performing loop filtering on the image unit after reconstructing the video frame, so as to improve the quality of the video frame.

[0344] Optionally, in the embodiments of the present application, the above step 501 can be implemented through the following step 501a.

[0345] Step 501a: when the distance from the pixel to be filtered to the first boundary of the image unit is less than M, performing pixel padding at the first boundary based on the first padding mode.

[0346] Wherein, the pixel to be filtered is a pixel in the image unit, the value of M is determined according to the first filter, and M is a positive integer.

[0347] In some embodiments of the present application, M can be a preset value. For example, the value of M can be 2, 4, etc. It should be noted that the specific value of M can be determined according to the shape of the first filter or actual needs, for example, M can be related to the radius of the filtering template of the first filter, and the embodiments of the present application do not limit this.

[0348] In some embodiments of the present application, in the process of filtering the first boundary of the image unit through the first filter, it can be judged whether the distance from the current pixel to be filtered to the first boundary is less than or equal to the preset value M, if the distance is less than or equal to the preset value M, pixel padding is performed outside the first boundary, or if the distance is greater than the preset value M, the pixel to be filtered is directly filtered without pixel padding outside the first boundary.

[0349] Exemplarily, taking the first filter as ALF, the image unit as SAO outputted luminance reconstructed image, the first padding mode as mirror padding, and the first boundary as the upper boundary of the luminance reconstructed image as an example, when filtering the upper boundary region of the chroma reconstructed image by ALF, if the distance from the to-be-filtered pixel inside the upper boundary to the upper boundary is less than M (for example, 4), the mirror padding mode is adopted, and the information of at least one row of continuous pixels inside the upper boundary and adjacent to the upper boundary is padded to the to-be-padded region outside the upper boundary and adjacent to the upper boundary.

[0350] Exemplarily, taking the first filter as CCALF, the image unit as residual image, the first padding mode as mirror padding, and the first boundary as the left boundary of the luminance reconstructed image as an example, when filtering the left boundary region of the residual image by CCALF, if the distance from the to-be-filtered pixel inside the left boundary to the left boundary is less than M (for example, 2), the mirror padding mode is adopted, and the information of at least one row of continuous pixels inside the left boundary and adjacent to the left boundary is padded to the to-be-padded region outside the left boundary and adjacent to the upper boundary.

[0351] In some embodiments of the present application, when the distance from the to-be-filtered pixel to the first boundary of the image unit is less than M, it is indicated that the current filter template exceeds or exceeds the first boundary.

[0352] In some embodiments of the present application, the first boundary of the reconstructed image unit exceeded by the filter template means that, when the first filter is used to filter the pixels in the image unit, the range covered exceeds the boundary of the image unit itself.

[0353] Exemplarily, the following takes the image unit as image and the first boundary as the boundary of the image as an example, and the boundary padding process of the image is exemplarily described in combination with the drawings.

[0354] In an example, as shown in FIG. 9A, when the pixels at the upper boundary, the lower boundary, the left boundary and the right boundary of the image 10a2 are filtered by the 4x3 diamond filter template 10a1 respectively, the filter template 10a1 exceeds one row of pixels of the upper boundary, exceeds two rows of pixels of the lower boundary, and exceeds one column of pixels of the left boundary and the right boundary respectively. At this time, at least one row of pixels needs to be padded at the upper boundary and the lower boundary respectively, and at least one column of pixels needs to be padded at the left boundary and the right boundary respectively. The region of the padding shape in FIG. 9A represents the padding region where the pixel padding is performed at the physical boundary.

[0355] In another example, as shown in Figure 9B, when the pixels at the upper, lower, left, and right boundaries of image 10b2 are filtered using a 4×3 diamond-shaped filter template 10b1, the filter template 10b1 extends beyond the upper and lower boundaries by two rows of pixels, and beyond the left and right boundaries by two columns of pixels, respectively. At this point, it is necessary to fill at least two rows of pixels at the upper and lower boundaries, and at least two columns of pixels at the left and right boundaries, respectively. The filled shape in Figure 9B represents the filling area where pixels are filled at these physical boundaries.

[0356] In another example, as shown in Figure 9C, when the pixels at the upper, lower, left, and right boundaries of image 10c2 are filtered using a 4×3 diamond-shaped filter template 10c1, the filter template 10c1 extends beyond the upper boundary by three rows of pixels, the lower boundary by one row of pixels, the left boundary by one column of pixels, and the right boundary by two columns of pixels. In this case, at least three rows of pixels need to be filled at the upper boundary, at least one row at the lower boundary, at least one column at the left boundary, and at least two columns at the right boundary. The filled shape in Figure 9C represents the filling area where pixels are filled at these physical boundaries.

[0357] It should be noted that in Figure 9C, the upper half of the third row of pixels, the first row of pixels, and the second row of pixels in the filter template 10c1 exceed the upper boundary and can be considered as exceeding three rows of pixels.

[0358] For example, the image unit is CTU, and the first boundary includes at least one of the image boundary and the virtual boundary. The boundary filling process of the image is illustrated in conjunction with the accompanying drawings.

[0359] In one example, as shown in Figure 10, when filtering pixels at the upper, lower, left, and right boundaries of CTU 11a2 using a 4×3 diamond-shaped filter template 11a1, the filter template extends beyond the upper boundary by one row of pixels, the lower boundary by two rows of pixels, and the left and right boundaries by one column of pixels. Therefore, it is necessary to fill at least one row of pixels at the upper boundary, at least two rows of pixel values ​​at the lower boundary, and at least one column of pixel values ​​at the left and right boundaries. The filled area in Figure 10 represents the filling region for pixel filling at these physical boundaries.

[0360] In another example, as shown in Figure 11A, the CTU 12a1 includes a virtual boundary of the CTU boundary, which is represented by the dashed line in Figure 11A. This virtual boundary divides the CTU 11b1 into two parts, which are denoted as CTU1 and CTU2, respectively.

[0361] In the process of boundary filtering of the CTU 11b1, as shown in FIG. 11B, when the filtering template exceeds the boundaries (the upper boundary, the lower boundary, the left boundary or the right boundary) of the CTU 12a1, respectively, pixel padding can be performed at the upper boundary, the lower boundary, the left boundary and the right boundary of the CTU 12a1. In FIG. 11B, the areas filled with shapes represent the areas where pixels are padded at the boundaries.

[0362] In the process of filtering at the virtual boundary of the CTU1 and the virtual boundary of the CTU2, as shown in FIG. 11C and FIG. 11D, for the CTU1, the boundaries include the upper boundary, the lower boundary, the left boundary and the right boundary, wherein the upper boundary, the lower boundary and the left boundary are CTU boundaries, and the right boundary is a virtual boundary. After the filtering template exceeds the CTU boundaries and the virtual boundary, at least one row or at least one column of pixel values are padded at the corresponding boundaries, respectively. The CTU1 after pixel padding is shown in FIG. 11C. For the CTU2, the boundaries include the upper boundary, the lower boundary, the left boundary and the right boundary, wherein the upper boundary, the lower boundary and the right boundary are CTU boundaries, and the left boundary is a virtual boundary. After the filtering template exceeds the CTU boundaries and the virtual boundary, at least one row or at least one column of pixel values are padded at the corresponding boundaries, respectively. The CTU2 after pixel padding is shown in FIG. 11D.

[0363] It should be noted that the areas filled with shapes in FIG. 11C and FIG. 11D represent the areas where pixels are padded.

[0364] It should be noted that the coding tree unit is a processing unit of High Efficiency Video Coding (HEVC). The processing unit is similar to the Macroblock in Advanced Video Coding (AVC). The size of the coding tree unit can be from 16x16 to 64x64 to achieve better compression effect.

[0365] Optionally, in the embodiments of the present application, the step 501 can be implemented by the following step 501a.

[0366] Step 501a: filling the first pixel information to the first boundary of the image unit based on the first padding mode.

[0367] The first pixel information is the information of the pixels corresponding to the first padding mode in the first boundary.

[0368] In some embodiments of the present application, the first pixel information can include but is not limited to pixel values.

[0369] Optionally, in the embodiments of the present application, the first filter is a filter template of ALF or CCALF; and the first pixel information may, for example, include but is not limited to at least one of the following: luminance information after SAO processing, chroma information after SAO processing, luminance information before deblocking filtering, chroma information before deblocking filtering, luminance residual information, luminance information after luminance fixed filter processing, chroma information after chroma fixed filter processing, and luminance information after luminance Gaussian filter processing.

[0370] In a conventional video coding exploration platform ECM, ALF requires various new input information. For example, when ALF filters luminance pixels, it needs to input luminance information output by SAO, luminance information output before DBF, luminance residual information, etc. Further, the ALF luminance filter is divided into a luminance fixed filter, a luminance Gaussian filter and a luminance new filter. In addition to the above input information, the output of the luminance fixed filter and the output of the luminance Gaussian filter are also input information of the luminance new filter. When ALF filters chroma pixels, it needs to input chroma information output by SAO, chroma information output before DBF, etc. Further, the ALF chroma filter is divided into a chroma fixed filter and a chroma new filter. In addition to the above input information, the output of the chroma fixed filter is also input information of the chroma new filter. CCALF filters chroma information, and needs to input luminance information output by SAO, luminance residual information, etc.

[0371] In the embodiments of the present application, the additional input information of all filters of ALF and CCALF can be selected from mirror padding, simple padding or other padding modes, or the additional input information of all filters of ALF and CCALF can be adaptively selected from mirror padding, simple padding or other padding modes, so that the additional input information of ALF and CCALF can be finely processed, and the continuity and consistency of the boundary region in the filtering process can be ensured. By this method, the filtering effect is improved, and the coding quality of the video is significantly improved, which provides strong support for the further development of video coding technology.

[0372] In some embodiments of the present application, the first pixel information may, for example, include at least one row or at least one column of pixels determined from the image unit according to the first padding mode.

[0373] In some embodiments of the present application, the number of rows or columns of pixels of the first pixel information (i.e. N below) can be determined based on the size of the first filter template, and further, the number of rows or columns of pixels of the first pixel information can be determined according to the number of rows or columns of pixels of the filter template that exceed the first boundary of the image unit.

[0374] In some embodiments of the present application, the number of pixel rows or the number of pixel columns of the first pixel information can be a preset maximum number of pixel rows or a preset maximum number of pixel columns; or the number of pixel rows or the number of pixel columns of the first pixel information can be determined according to at least one of a filter shape of the first filter and a distance from the to-be-filtered pixel to the first boundary.

[0375] For example, the number of pixel rows or the number of pixel columns of the first pixel information can be greater than or equal to a radius of a filter template used by the first filter.

[0376] In some embodiments of the present application, the number of pixel rows or the number of pixel columns of the first pixel information can be determined according to a size of the filter template.

[0377] For example, the number of pixel rows of the first pixel information can be greater than or equal to the number of pixel rows of the filter template beyond the first boundary of the image unit, or the number of pixel columns of the first pixel information can be greater than or equal to the number of pixel columns of the filter template beyond the first boundary of the image unit.

[0378] For example, when the number of pixel rows of the filter template beyond the first boundary of the image unit is 2 rows, the first pixel information can be 2 rows; or when the number of pixel rows of the filter template beyond the first boundary of the image unit is 2 columns, the first pixel information can be 2 columns.

[0379] It should be noted that when pixel padding is performed at the first boundary, the number of pixel rows to be padded can be no less than the number of rows of the filter template beyond the first boundary, so as to ensure that the filter template can cover the pixel information without exceeding the first boundary.

[0380] It should be noted that the number of pixel rows and the number of pixel columns of the first pixel information can be determined according to actual needs, and the embodiments of the present application do not limit this.

[0381] It can be understood that the number of pixel rows or the number of pixel columns of the first pixel information is the same as the number of pixel rows and the number of pixel columns of the pixel information to be padded outside the first boundary.

[0382] In some embodiments of the present application, when the first padding mode is mirror padding, the pixel information to be padded outside the first boundary is mirror symmetric with the pixel position of the first pixel information located inside the first boundary.

[0383] Exemplarily, taking the lower boundary of the image unit as an example, as shown in FIG. 12A, the lower boundary B1 of the image unit includes four rows of pixel data, and in the loop filtering process of the image unit, the distance from the pixel to be filtered to the lower boundary is less than M, that is, when the filtering template exceeds the lower boundary B1 of the image unit, the pixel values of the four rows of pixels are copied and mirror mapped to the pixel positions to be filled at the lower boundary. Further, after filling the four rows of pixels at the lower boundary, the entire filtering template is located on the valid information (including the original pixel information and the filled pixel information), and then the filtering operation can be performed on the image boundary region currently processed based on the filtering template.

[0384] It should be noted that for each pixel to be filled at the lower boundary, there is a mirror-symmetrical pixel inside the image unit. That is, the mirror position is calculated based on the image lower boundary, that is, for each row (i.e., each row to be filled) exceeding the lower boundary, the mirror row inside the image can be a corresponding row with equal distance from top to bottom. In FIG. 12A, the pixels above the lower boundary B1 are original pixels, and the pixels below B1 are filled pixels.

[0385] Exemplarily, taking the right boundary of the chroma reconstructed image as an example, as shown in FIG. 12B, assuming that the right boundary of the image unit includes four columns of pixel data, in the loop filtering process of the image unit, the distance from the pixel to be filtered to the right boundary is less than M, that is, when the filtering template exceeds the right boundary R1 of the image unit, the pixel values of the four columns of pixels are copied and mirror mapped to the pixel positions to be filled at the right boundary. Further, after filling the four columns of pixels at the right boundary, the entire filtering template is located on the valid information (including the original pixel information and the filled pixel information), and then the filtering operation can be performed on the image boundary region currently processed based on the filtering template.

[0386] It should be noted that the mirror position should be calculated based on the image right boundary, that is, for each column (i.e., each column to be filled) exceeding the right boundary, the mirror column inside the image can be a corresponding column with equal distance from right to left. In FIG. 12B, the pixels to the right of the right boundary R1 are filled pixels, and the pixels to the left of R1 are original pixels.

[0387] In some embodiments of the present application, when the first filling mode is the repetition filling, for each pixel to be filled outside the first boundary, the pixel value thereof is set to be the same as that of the boundary pixel (i.e., the pixel located on the same row or column and located inside the image) closest thereto.

[0388] Exemplarily, taking the lower boundary of the image unit as the first boundary, and the repeated padding as the first padding mode, in the loop filtering process of the image unit, when the filtering template exceeds the lower boundary B1 of the image unit, the pixel values of a row of pixels at the lower boundary are repeatedly copied to the pixel positions of the four rows of pixels to be padded, until four rows of pixels are padded at the lower boundary. Further, after four rows of pixels are padded at the lower boundary, the entire filtering template is located on the valid information (including the original pixel information and the padded pixel information), and then the filtering operation can be performed on the image boundary region currently processed based on the filtering template, as shown in FIG. 12C.

[0389] It should be noted that after the pixel padding is performed on the lower boundary by using the repeated padding mode, the pixel information of each row of pixels after padding is the same as the pixel information of a row of pixels at the lower boundary before padding.

[0390] Exemplarily, taking the lower right boundary of the chroma reconstructed image as the first boundary, and the repeated padding as the first padding mode, in the loop filtering process of the image unit, when the filtering template exceeds the right boundary R1 of the image unit, the pixel values of a column of pixels at the right boundary R1 can be repeatedly copied to the pixel positions of the four columns of pixels to be padded, until four columns of pixels are padded at the right boundary, as shown in FIG. 12D.

[0391] It should be noted that after the pixel padding is performed on the right boundary by using the repeated padding mode, the pixel information of each column of pixels after padding is the same as the pixel information of a column of pixels at the right boundary before padding. The pixels to the right of the right boundary R1 in FIG. 12D are padded pixels, and the pixels to the left of R1 are original pixels.

[0392] It should be noted that for the padding process after the filtering template exceeds the upper boundary of the image, the padding process after the filtering template exceeds the lower boundary of the image can be referred to, and for the padding process after the filtering template exceeds the left boundary of the image, the padding process after the filtering template exceeds the right boundary of the image can be referred to, which will not be described herein again.

[0393] In the embodiments of the present application, the information of the corresponding pixels is padded to the outside of the first boundary of the image unit based on the adaptively determined first padding mode (such as repeated padding, mirror padding, zero padding, etc.), so as to be applicable to different image processing scenes and requirements. For example, the mirror padding performs better in the scene of needing to maintain edge texture or structure information, and the zero padding can be more suitable for certain specific mathematical operation or algorithm requirements. Therefore, the pixel padding outside the boundary according to the adaptive padding mode can flexibly adapt to different image processing tasks.

[0394] Optionally, in the embodiments of the present application, the step 501a can be implemented through the following step 501a1.

[0395] Step 501a1: filling the information of N rows of pixels or N columns of pixels within the first boundary to the first boundary based on a first filling manner.

[0396] wherein the N rows of pixels are the 1st row of pixels to the Nth row of pixels within the first boundary, the 1st row of pixels is the pixel in the row adjacent to the first boundary, the N columns of pixels are the 1st column of pixels to the Nth column of pixels within the first boundary, the 1st column of pixels is the pixel in the column adjacent to the first boundary, and N is a positive integer.

[0397] In some embodiments of the present application, the N rows of pixels can be the continuous N rows of pixels of the first boundary, and the N columns of pixels can be the continuous N columns of pixels of the first boundary.

[0398] In some embodiments of the present application, in the process of loop filtering the image unit, the number N of rows or columns of pixels filled at the first boundary can be determined according to the filter shape and the distance from the pixel to be filtered to the first boundary, for example, according to the radius of the filter template of the filter and the distance from the pixel to be filtered to the first boundary. Then, the N rows of pixels or N columns of pixels within the first boundary are filled to the outside of the first boundary based on the first filling manner.

[0399] It should be noted that the first pixel information is the N rows of pixels or N columns of pixels.

[0400] In some embodiments of the present application, after determining the number N of rows or columns of pixels filled at the first boundary, a pixel area is created at the boundary of the image unit, and the pixel information of the N rows or N columns of pixels within the first boundary is copied into the pixel area according to the first filling manner.

[0401] It should be noted that the pixel area created at the boundary of the image unit is actually a “virtual” pixel area, and the pixels to be filled in the area are “virtual” pixels.

[0402] Optionally, in the embodiments of the present application, the first filling manner is a mirror filling manner; and exemplarily, the step 501a1 can be implemented through the following step 501a2.

[0403] Step 501a2: filling the information of N rows of pixels or N columns of pixels within the first boundary to the first boundary based on the mirror filling manner, taking the first boundary as the axis of symmetry.

[0404] In some embodiments of the present application, the first boundary of the image unit can be: a region adjacent to the first boundary outside the first boundary of the image unit.

[0405] It should be noted that the above-mentioned symmetry axis refers to the mirror symmetry axis.

[0406] Exemplarily, when mirror filling is performed with the first boundary as the symmetry axis, a coordinate system can be established in the image unit, the origin (O) of the coordinate system is located at the top-left corner pixel of the image unit, the x-axis extends to the right, the x-axis coordinate represents the horizontal coordinate of the pixel relative to the top-left corner pixel, the y-axis extends downward, the y-axis coordinate represents the vertical coordinate of the pixel relative to the top-left corner pixel, the position of each pixel is determined by its horizontal coordinate x and vertical coordinate y, and for each pixel (x, y), the corresponding pixel value P of the pixel can be determined according to the coordinates of the pixel.

[0407] Exemplarily, when pixel filling is performed based on the mirror filling mode, the coordinates of the pixel symmetrical to each pixel to be filled inside and outside the first boundary about the symmetry axis can be calculated based on the above-mentioned coordinate system, and then the pixel value of the pixel at the coordinates is respectively assigned to the pixel to be filled symmetrical to the pixel about the symmetry axis.

[0408] The calculation process of determining the pixel value of the pixel to be filled in the mirror filling mode is explained and described below.

[0409] It is assumed that the value (such as the pixel value) corresponding to the image is P, and the position information of each pixel is (x, y), x represents the horizontal coordinate of the pixel relative to the top-left corner of the image, and y represents the vertical coordinate of the pixel relative to the top-left corner pixel of the image.

[0410] For the upper boundary pixel of the image, y = 0, and for the pixel to be filled, there is: PadingInfo(x i ,-y j ) = P(x i ,y j )

[0411] Where x i is the horizontal coordinate of the pixel to be filled, y j is the vertical coordinate of the pixel to be filled, x i ∈ [0, picWidth-1], y j ∈ [0, paddingSize-1]. picWidth represents the width of the image, and paddingSize is the number of rows to be filled in the image.

[0412] For the left boundary pixel of the image, x = 0, and for the pixel to be filled, there is: PadingInfo(-x i ,y j)=P(x i ,y j )

[0413] Where x i Let y be the x-coordinate of the pixel to be filled. j x is the ordinate of the pixel to be filled. i ∈[0,paddingSize-1],y j ∈[0, picHeight-1]. picHeight represents the height of the image, and paddingSize is the number of columns that the image needs to be padded with.

[0414] For the right boundary pixels of the image, x = picWidth, then for the pixels that need to be padded, we have: PadingInfo(x i ,y j ) = P(picWidth-x i ,y j )

[0415] Where x i Let y be the x-coordinate of the pixel to be filled. j x is the ordinate of the pixel to be filled. i ∈[0,paddingSize-1],y j ∈[0, picHeight-1]. picHeight represents the height of the image, paddingSize is the number of columns the image needs to be padded with, and picWidth represents the width of the image.

[0416] For the lower boundary pixels of the image, y = picHeight, then for the pixels that need to be padded, we have: PadingInfo(x i ,y j )=P(x i ,picHeight-y j )

[0417] Where x i Let y be the x-coordinate of the pixel to be filled. j x is the ordinate of the pixel to be filled. i ∈[0, picWidth-1], y j ∈[0,paddingSize-1]. picWidth represents the width of the image, and paddingSize is the number of rows the image needs to be padded with. picHeight represents the height of the image.

[0418] For example, taking four rows of pixels as an example, when performing loop filtering on the image unit based on the filtering template, if the filtering template exceeds the lower boundary of the image unit, taking the lower boundary as the axis of symmetry, four rows of pixels are selected from the lower boundary of the image unit upwards as the mirror source, and then the pixel values in the mirror source are copied into the region outside the first boundary and adjacent to the first boundary (i.e., the "virtual" pixel region to be filled) pixel by pixel. Specifically, the pixel value of the first row of pixels counted from the lower boundary upwards is copied into the first row of pixels counted from the upper boundary downwards, the pixel value of the second row of pixels counted from the lower boundary upwards is copied into the second row of pixels counted from the upper boundary downwards, the pixel value of the third row of pixels counted from the lower boundary upwards is copied into the third row of pixels counted from the upper boundary downwards, and finally the pixel value of the fourth row of pixels counted from the lower boundary upwards is copied into the fourth row of pixels counted from the upper boundary downwards.

[0419] It should be noted that the two pixels inside the image unit and outside the image unit (i.e., the region to be filled) that are mirror symmetric about the first boundary can be referred to as a pair of mirror pixels, and the coordinates of the pair of mirror pixels are symmetric about the axis of symmetry. In addition, the filling method for other boundaries is similar to that of the lower boundary, which will not be described here.

[0420] It can be understood that the above-mentioned "four rows of pixels to be filled" actually refer to the "virtual" pixel region covered by the filtering template exceeding the lower boundary, and do not actually exist in the original image unit, so these pixels need to be "filled" by mirror symmetry.

[0421] In the embodiments of the present application, the N rows or N columns of pixels inside the first boundary are mirror filled to the outside of the first boundary of the image unit by taking the first boundary as the axis of symmetry. Since the mirror filling is based on the symmetry relationship of the pixels inside the boundary to generate the pixels outside the boundary, the generated pixel values are more natural in vision, reducing the artifacts or discontinuity phenomenon caused by filling, thereby ensuring the data continuity of the image unit at the boundary and improving the filtering effect on the image unit.

[0422] Optionally, in the embodiments of the present application, the step 502 can be implemented by the following step 502a.

[0423] Step 503a: performing loop filtering on the to-be-filtered pixels in the image unit after pixel filling based on the first filter.

[0424] The to-be-filtered pixels can include the first pixel information corresponding to the pixels.

[0425] In some embodiments of the present application, in the process of loop filtering the image unit, the pixel to be filtered can be loop filtered based on the first filter and the pixel information required to be used in the boundary region after pixel padding.

[0426] In some embodiments of the present application, in the process of loop filtering the image unit, the coefficient of the filter template can be operated with the pixel information corresponding to the filter template, and the operation result can be assigned to the pixel (i.e. the pixel to be filtered) corresponding to the center of the filter template, so as to complete the filtering of the pixel.

[0427] Optionally, in the embodiments of the present application, the first boundary is the physical boundary of the CTU where the image unit is located; and exemplarily, the step 501 can be implemented through the following step 501b.

[0428] The step 501b: if there is no adjacent CTU at the first boundary of the CTU, pixels are padded at the physical boundary of the CTU based on the first padding mode.

[0429] In some embodiments of the present application, the adjacent CTU refers to the CTU located outside the first boundary of the CTU and adjacent to the CTU.

[0430] Exemplarily, the adjacent CTU can be at least one of the following: the CTU located above the CTU and adjacent to the CTU, the CTU located below the CTU and adjacent to the CTU, the CTU located to the left of the CTU and adjacent to the CTU, and the CTU located to the right of the CTU and adjacent to the CTU.

[0431] In some embodiments of the present application, if the distance of the pixel to be padded to the first boundary is less than M, i.e. the filter template exceeds the first boundary of the CTU, and if there is no adjacent CTU at the first boundary of the CTU, i.e. the CTU has no adjacent CTU sharing the first boundary, at least one row or at least one column of pixels is padded outside the boundary of the CTU based on the first padding mode.

[0432] It should be noted that the adjacent CTU can also be referred to as the neighbor CTU.

[0433] It can be understood that the absence of the adjacent CTU at the first boundary of the CTU indicates that the boundary of the CTU coincides with the boundary of the image.

[0434] Exemplarily, in combination with the above FIG. 10, taking the CTU in the luminance reconstructed image before deblocking filtering as the CTU, and taking the mirror padding as the first padding mode, for example, when the boundary pixels of the CTU are filtered, the filter template exceeds the upper boundary of the CTU, and if there is no adjacent CTU at the upper boundary of the CTU, N rows (e.g. two rows) of pixels are padded at the upper boundary of the CTU according to the mirror padding mode.

[0435] In the embodiments of the present application, for the CTU without adjacent CTU, i.e. the CTU at the edge of the image, when the filter template exceeds the boundary of the CTU, pixel padding can be performed on the boundary using the pixel value within the boundary, so as to improve the filtering effect.

[0436] Optionally, in the embodiments of the present application, the first boundary is the boundary of the CTU where the image unit is located; after the step 501, the loop filtering method provided by the embodiments of the present application can further include the following step 504:

[0437] Step 504: if the CTU has adjacent CTU, performing loop filtering on the pixel to be filtered based on the first filter.

[0438] For example, when filtering the boundary pixel of the CTU, if the filter template exceeds the upper boundary of the CTU and the CTU has adjacent CTU, pixel padding is not needed at the upper boundary of the CTU, but the pixel information of the adjacent CTU is used for filtering.

[0439] In the embodiments of the present application, when the adjacent CTU of the CTU currently being filtered exists, the pixel information of the adjacent CTU is directly used, unnecessary pixel padding and repeated calculation are avoided, and the coding efficiency is improved. In addition, by using the pixel information of the adjacent CTU as the pixel information to be filtered of the CTU currently being filtered, the continuity of the boundary pixel of the CTU and the pixel of the adjacent CTU can be maintained, which helps to reduce the image quality degradation caused by the filter template exceeding the boundary, such as edge blur or artifacts, and thus improves the filtering effect.

[0440] In the following, another loop filtering method provided by the embodiments of the present application is exemplarily described with reference to the accompanying drawings.

[0441] FIG. 13 is a flowchart of a loop filtering method provided by the embodiments of the present application, which is applied to a decoder, as shown in FIG. 13, the loop filtering method can include the following steps 601 to 603:

[0442] Step 601: parsing the code stream to obtain a syntax element.

[0443] The syntax element is used to indicate a first pixel padding mode used when performing loop filtering.

[0444] In some embodiments of the present application, in the decoding process, after receiving and parsing the code stream, the filtering parameters and the syntax element are parsed from the code stream, and the filtering parameters and the syntax element are used to perform loop filtering after reconstructing the video frame.

[0445] In some embodiments of the present application, the bitstream further comprises information for reconstructing the image, including but not limited to image data, coding mode, filtering parameter, etc.

[0446] Exemplarily, after reconstructing the video frame, in the process of loop filtering the image unit, pixel padding processing can be performed according to the parsed padding mode and other information, so as to reduce the block effect and artifacts introduced in the coding process and improve the video quality.

[0447] Specifically, the syntax element sh_alf_enabled_flag is parsed, and if the syntax element sh_alf_enabled_flag takes the value 1, it indicates that the current slice uses the ALF or CCALF filter for filtering. If the value is 0, it indicates that the current slice does not use the ALF or CCALF filter for filtering. If the syntax element sh_alf_enabled_flag takes the value 1, the syntax element sh_alf_padding_mode is parsed. If the syntax element sh_alf_padding_mode takes the value 0, simple padding (padding one row or one column of pixels at the boundary) is directly used, and if the syntax element sh_alf_padding_mode takes the value 1, mirror padding is used for padding.

[0448] Step 602: Based on the first padding mode, pixel padding is performed at the first boundary of the image unit.

[0449] Step 603: Based on the first filter, loop filtering is performed on the image unit after the pixel padding.

[0450] The loop filtering method provided by the embodiments of the present application effectively improves the filtering effect of the image block, especially the boundary region of the image block, by parsing the filtering parameter and padding mode information from the bitstream and using the corresponding pixel padding technology in the filtering process, avoids the distortion or artifact problem of the filtered image, and thus improves the filtering effect.

[0451] Optionally, in the embodiments of the present application, the above step 601 can be implemented by the following step 601a.

[0452] Step 601a: When the distance of the pixel to be filtered to the first boundary of the image unit is less than M, pixel padding is performed at the first boundary based on the first padding mode.

[0453] Wherein, the pixel to be filtered is a pixel in the image unit, the value of M is determined according to the first filter, and M is a positive integer.

[0454] Optionally, in the embodiments of the present application, the above step 601 can be implemented by the following step 601b.

[0455] Step 601b: Based on the first padding mode, the first pixel information is padded to the first boundary.

[0456] The first pixel information is information of pixels in the first boundary corresponding to the first padding mode.

[0457] It should be noted that the implementation steps of the pixel padding of the decoder are the same as the implementation steps of the pixel padding of the encoder.

[0458] Optionally, in the embodiment of the present application, the first filter is ALF or CCALF; and the first pixel information includes at least one of the following: luminance information after SAO processing; chroma information after SAO processing; luminance information before deblocking filtering; chroma information before deblocking filtering; luminance residual information; luminance information after luminance fixed filter processing; chroma information after chroma fixed filter processing; and luminance information after luminance Gaussian filter processing.

[0459] Optionally, in the embodiment of the present application, the image unit is an image, and the first boundary of the image unit is a virtual boundary of the image.

[0460] Optionally, in the embodiment of the present application, the image unit is a CTU, and the first boundary of the image unit is a virtual boundary of the CTU.

[0461] Optionally, in the embodiment of the present application, the image unit is a CTU; and the step 602 can be implemented by the following step 602a.

[0462] The step 602a includes: if there is no adjacent CTU at the first boundary of the CTU, performing pixel padding at the boundary of the CTU based on the first padding mode.

[0463] Optionally, in the embodiment of the present application, the first boundary is a physical boundary of a CTU in which the image unit is located; and after the step 601, the loop filtering method provided by the embodiment of the present application can further include the following step 605.

[0464] The step 605 includes: if there is an adjacent CTU of the CTU, performing loop filtering on a CTU region in which the filter template is located based on the first filter.

[0465] Optionally, in the embodiment of the present application, the step 601b can be implemented by the following step 601b1.

[0466] The step 601b1 includes: filling information of N rows of pixels or N columns of pixels in the first boundary to the first boundary based on the first padding mode.

[0467] The N rows of pixels are the 1st row of pixels to the Nth row of pixels within the first boundary, the 1st row of pixels is a pixel adjacent to the first boundary, the N columns of pixels are the 1st column of pixels to the Nth column of pixels within the first boundary, the 1st column of pixels is a pixel adjacent to the first boundary, and N is a positive integer.

[0468] Optionally, in the embodiment of the present application, the step 601b1 can be implemented through a step 601b2.

[0469] Step 601b2: based on a mirror filling mode, mirroring and filling information of N rows of pixels or N columns of pixels within the first boundary to a position outside the first boundary and adjacent to the first boundary, with the first boundary as a symmetric axis.

[0470] Optionally, in the embodiment of the present application, the first boundary of the image unit is a region outside the first boundary and adjacent to the first boundary.

[0471] It should be noted that the explanation of this embodiment can refer to the description of the corresponding method embodiment of FIG. 6, which will not be repeated here.

[0472] The loop filtering method provided by the embodiment of the present application will be exemplarily described through specific embodiments.

[0473] Exemplarily, the first filter is ALF, and the image unit is an image, which can be at least one of the following: a luma reconstructed image output by SAO, a luma reconstructed image before deblocking filtering, an image of luma residual information, a luma reconstructed image output by a luma fixed filter, and a luma reconstructed image output by a luma Gaussian filter, and the first boundary is a lower boundary of the image. When the encoder filters the image through ALF, if the distance from a to-be-filtered pixel in the image to the lower boundary of the image is less than a preset value M, the neighborhood of the to-be-filtered pixel is expanded based on a mirror filling mode. Specifically, N continuous rows of pixels inside the lower boundary of the image and close to the lower boundary are selected as a data source for mirror filling, and the lower boundary of the image is used as a symmetric axis. Starting from the 1st row of pixels inside the lower boundary of the image from bottom to top, information of the 1st row of pixels (i.e., the last row of pixels of the image) is filled to the 1st row of to-be-filled pixels outside the lower boundary from top to bottom, information of the 2nd row of pixels (i.e., the second last row of pixels of the image) is filled to the 2nd row of to-be-filled pixels outside the lower boundary from top to bottom, and so on, until information of the Nth row of pixels inside the lower boundary of the image from bottom to top is filled to the Nth row of to-be-filled pixels outside the lower boundary from top to bottom. Then, after the mirror filling is completed, the to-be-filtered pixel is subjected to loop filtering through an ALF filter by applying an ALF filtering algorithm to further optimize the image quality, and the filtering parameters of the loop filtering and a syntax element indicating the mirror filling are encoded to obtain a bitstream for transmission to a decoder.

[0474] Further, the decoder receives the bitstream from the encoder, parses the bitstream, and when the distance between the to-be-filtered pixel and the boundary of the image is less than M during the filtering of the image, performs pixel padding according to the mirror padding mode indicated by the parsed syntax element, and after the ALF filtering of the image is completed, the decoder outputs the filtered image or image block for subsequent video processing or display.

[0475] It should be noted that the pixel padding step of the decoder is the same as the pixel padding step of the encoder, which will not be described here.

[0476] Further, the first filter is ALF, and the image unit is an image, which can include at least one of the following: a chroma reconstructed image output by SAO, a chroma reconstructed image output before deblocking filtering, and a chroma reconstructed image output by a fixed chroma filter, and the first boundary is the right boundary of the image. When the image is filtered by ALF, if the distance between the to-be-filtered pixel and the right boundary of the image is less than M, the information of the first column of pixels to the Nth column of pixels near the right boundary of the image is mirrored and padded to the region outside the right boundary and adjacent to the right boundary based on the mirror padding mode, with the right boundary as the axis of symmetry. Specifically, the information of the first column of pixels from right to left (i.e., the last column of pixels of the image) in the right boundary is padded to the first column of to-be-padded pixels from left to right outside the right boundary, the information of the second column of pixels from right to left (i.e., the second last column of pixels of the image) in the right boundary is padded to the second column of to-be-padded pixels from left to right outside the right boundary, and so on, until the information of the Nth column of pixels from right to left in the right boundary is padded to the Nth column of to-be-padded pixels from left to right outside the right boundary. Then, after the mirror padding is completed, the ALF filtering algorithm is applied to the to-be-filtered pixel by the ALF filter to further optimize the image quality.

[0477] It should be noted that the process after the filtering processing can refer to the description of the above examples, which will not be described here.

[0478] In another example, the first filter is CCALF, the image unit is a picture, and the picture can include at least one of the following: a luma reconstructed picture output by SAO, a picture containing luma residual information, and an upper boundary of the picture. When filtering the picture by CCALF, if a distance from a pixel to be filtered in the picture to the upper boundary of the picture is less than M, the information of the first row of pixels to the Nth row of pixels in the upper boundary of the picture that are close to the upper boundary is mirrored and filled to a region adjacent to the upper boundary outside the upper boundary based on a mirroring and filling manner with the upper boundary as a symmetry axis. Specifically, the information of the first row of pixels from top to bottom in the upper boundary is filled to the first row of pixels to be filled from bottom to top outside the upper boundary, the information of the second row of pixels from top to bottom in the upper boundary is filled to the second row of pixels to be filled from bottom to top outside the upper boundary, and so on until the information of the Nth row of pixels from top to bottom in the upper boundary is filled to the Nth row of pixels to be filled from bottom to top outside the upper boundary. Then, after the mirroring and filling, the pixel to be filtered is filtered by an ALF filter using an ALF filtering algorithm to further optimize the decoded picture quality.

[0479] It should be noted that the process after the filtering can refer to the description of the above examples, which will not be repeated here.

[0480] In another example, the first filter is ALF, the image unit is a picture, and the picture can be at least one of the following: a luma reconstructed picture output by SAO, a luma reconstructed picture before deblocking filtering, a picture of luma residual information, a luma reconstructed picture output by a luma fixed filter, and a luma reconstructed picture output by a luma Gaussian filter, and the first boundary is a virtual boundary of the picture. When filtering the picture by ALF, if a pixel to be filtered in the picture is below the virtual boundary and a distance from the pixel to be filtered to the virtual boundary is less than M, the information of the first row of pixels to the Nth row of pixels in the virtual boundary that are close to the virtual boundary is mirrored and filled to a region adjacent to the virtual boundary outside the virtual boundary based on a mirroring and filling manner with the virtual boundary as a symmetry axis. Specifically, the information of the first row of pixels from top to bottom in the virtual boundary is filled to the first row of pixels to be filled from bottom to top outside the virtual boundary, the information of the second row of pixels from top to bottom in the virtual boundary is filled to the second row of pixels to be filled from bottom to top outside the virtual boundary, and so on until the information of the Nth row of pixels from top to bottom in the virtual boundary is filled to the Nth row of pixels to be filled from bottom to top outside the virtual boundary.

[0481] It should be noted that the process after the filtering can refer to the description of the above examples, which will not be repeated here.

[0482] In another example, the first filter is ALF, and the image unit is a CTU. The CTU includes at least one of: a CTU in a luma reconstructed picture output by SAO, a CTU in a luma reconstructed picture before deblocking filtering, a CTU containing luma residual information, a CTU in a luma reconstructed picture output by a luma fixed filter, and a virtual boundary of the CTU. When filtering the CTU by ALF, a pixel to be filtered in the CTU is located at a left side of the virtual boundary, and a distance from the pixel to be filtered to the virtual boundary is less than a preset value M. In response to the distance being less than the preset value M, a neighborhood of the pixel to be filtered is expanded based on a mirror padding manner. Specifically, N continuous columns of pixels inside the virtual boundary and close to the virtual boundary are taken as a mirror source. A first column of pixels inside the virtual boundary is taken as a first mirror source. A first column of pixels outside the virtual boundary is taken as a first pixel to be padded. A second column of pixels inside the virtual boundary is taken as a second mirror source. A second column of pixels outside the virtual boundary is taken as a second pixel to be padded. A third column of pixels inside the virtual boundary is taken as a third mirror source. A third column of pixels outside the virtual boundary is taken as a third pixel to be padded. A fourth column of pixels inside the virtual boundary is taken as a fourth mirror source. A fourth column of pixels outside the virtual boundary is taken as a fourth pixel to be padded. A fifth column of pixels inside the virtual boundary is taken as a fifth mirror source. A fifth column of pixels outside the virtual boundary is taken as a fifth pixel to be padded. A sixth column of pixels inside the virtual boundary is taken as a sixth mirror source. A sixth column of pixels outside the virtual boundary is taken as a sixth pixel to be padded. A seventh column of pixels inside the virtual boundary is taken as a seventh mirror source. A seventh column of pixels outside the virtual boundary is taken as a seventh pixel to be padded. An eighth column of pixels inside the virtual boundary is taken as an eighth mirror source. An eighth column of pixels outside the virtual boundary is taken as an eighth pixel to be padded. A ninth column of pixels inside the virtual boundary is taken as a ninth mirror source. A ninth column of pixels outside the virtual boundary is taken as a ninth pixel to be padded. A tenth column of pixels inside the virtual boundary is taken as a tenth mirror source. A tenth column of pixels outside the virtual boundary is taken as a tenth pixel to be padded. An eleventh column of pixels inside the virtual boundary is taken as an eleventh mirror source. An eleventh column of pixels outside the virtual boundary is taken as an eleventh pixel to be padded. A twelfth column of pixels inside the virtual boundary is taken as a twelfth mirror source. A twelfth column of pixels outside the virtual boundary is taken as a twelfth pixel to be padded.

[0483] It should be noted that a process after the filtering process can refer to the description of the above examples, and will not be described here.

[0484] In another example, the first filter is ALF, and the image unit is a CTU. The CTU includes at least one of: a CTU in a luma reconstructed picture output by SAO, a CTU in a luma reconstructed picture before deblocking filtering, a CTU containing luma residual information, a CTU in a luma reconstructed picture output by a luma fixed filter, and a virtual boundary of the CTU. When filtering the CTU by ALF, a pixel to be filtered in the CTU is located at a left side of the virtual boundary, and a distance from the pixel to be filtered to the virtual boundary is less than a preset value M. In response to the distance being less than the preset value M, a neighborhood of the pixel to be filtered is expanded based on a mirror padding manner. Specifically, N continuous columns of pixels inside the virtual boundary and close to the virtual boundary are taken as a mirror source. A first column of pixels inside the virtual boundary is taken as a first mirror source. A first column of pixels outside the virtual boundary is taken as a first pixel to be padded. A second column of pixels inside the virtual boundary is taken as a second mirror source. A second column of pixels outside the virtual boundary is taken as a second pixel to be padded. A third column of pixels inside the virtual boundary is taken as a third mirror source. A third column of pixels outside the virtual boundary is taken as a third pixel to be padded. A fourth column of pixels inside the virtual boundary is taken as a fourth mirror source. A fourth column of pixels outside the virtual boundary is taken as a fourth pixel to be padded. A fifth column of pixels inside the virtual boundary is taken as a fifth mirror source. A fifth column of pixels outside the virtual boundary is taken as a fifth pixel to be padded. A sixth column of pixels inside the virtual boundary is taken as a sixth mirror source. A sixth column of pixels outside the virtual boundary is taken as a sixth pixel to be padded. A seventh column of pixels inside the virtual boundary is taken as a seventh mirror source. A seventh column of pixels outside the virtual boundary is taken as a seventh pixel to be padded. An eighth column of pixels inside the virtual boundary is taken as an eighth mirror source. An eighth column of pixels outside the virtual boundary is taken as an eighth pixel to be padded. A ninth column of pixels inside the virtual boundary is taken as a ninth mirror source. A ninth column of pixels outside the virtual boundary is taken as a ninth pixel to be padded. A tenth column of pixels inside the virtual boundary is taken as a tenth mirror source. A tenth column of pixels outside the virtual boundary is taken as a tenth pixel to be padded. An eleventh column of pixels inside the virtual boundary is taken as an eleventh mirror source. An eleventh column of pixels outside the virtual boundary is taken as an eleventh pixel to be padded. A twelfth column of pixels inside the virtual boundary is taken as a twelfth mirror source. A twelfth column of pixels outside the virtual boundary is taken as a twelfth pixel to be padded.

[0485] It should be noted that a process after the filtering process can refer to the description of the above examples, and will not be described here.

[0486] Exemplarily, the first filter is the CCALF, and the image unit is a CTU. The CTU includes at least one of the following: a CTU in a luma reconstructed image output by the SAO, a CTU in a luma reconstructed image before the deblocking filtering, a CTU containing luma residual information, a CTU in a luma reconstructed image output by the luma fixed filter, and a virtual boundary of the CTU. When the CCALF is used to filter the CTU, if a pixel to be filtered in the CTU is located on the right side of the virtual boundary and the horizontal distance from the pixel to the virtual boundary is less than a preset value M, the neighborhood of the pixel to be filtered is expanded based on a mirror filling mode. Specifically, taking the virtual boundary as a symmetric axis, the information of the continuous N columns of pixels inside the virtual boundary and close to the virtual boundary is filled to the region outside the virtual boundary and adjacent to the virtual boundary in a mirror symmetric manner. The specific filling process is as follows: starting from the first column of pixels inside the virtual boundary from left to right, the information of the first row of pixels is copied and filled to the first column of pixel positions to be filled outside the virtual boundary from right to left, then the information of the second column of pixels inside the virtual boundary from left to right is filled to the second column of pixel positions to be filled outside the virtual boundary from right to left, and so on, until the information of the Nth column of pixels inside the virtual boundary from left to right is filled to the Nth column of pixel positions to be filled outside the virtual boundary from right to left.

[0487] In the embodiments of the present application, mirror filling is performed on the additional input information of the front-stage filter of the ALF or the CCALF, such as the luma information and the chroma information output before the deblocking filtering, the luma residual information, the output of the luma fixed filter, the output of the chroma fixed filter, and the output of the luma Gaussian filter. According to the shape of the filter, while filling, the texture topology at the boundary is ensured to be as unchanged as possible, so as to improve the filtering effect and improve the quality of the encoded image.

[0488] It should be noted that the encoding end in any of the embodiments of the present application can be the encoder 112 in FIG. 1 or FIG. 3, or the source device 110 in FIG. 1. The decoding end in any of the embodiments of the present application can be the decoder 122 in FIG. 1 or FIG. 5, or the destination device 120 in FIG. 1, which is not limited in the embodiments of the present application.

[0489] Correspondingly, the embodiment of the present application provides a loop filtering device, which can be any loop filtering device in a call process. According to the above method example, the loop filtering device can be divided into functional modules. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The integrated module can be realized in the form of hardware or in the form of a software functional module. It should be noted that the division of the modules in the embodiment of the present application is illustrative, and is only a logical functional division. In actual implementation, another division mode can be used.

[0490] In the case of dividing each functional module according to each function, FIG. 14 shows a possible structural schematic diagram of the loop filtering device 700 involved in the above embodiment. As shown in FIG. 14, the loop filtering device 700 includes a filtering module 701 and an encoding module 702. The filtering module 701 is configured to perform pixel padding at a first boundary of an image unit based on a first padding mode, the first padding mode being determined based on at least one of a projection format corresponding to the image unit and a rate-distortion decision result of pixel padding at a reference image boundary. The filtering module 701 is further configured to perform loop filtering on the image unit after the pixel padding based on a first filter. The encoding module 702 is configured to write a filtering parameter of the loop filtering and a syntax element into a bitstream, the syntax element being used to indicate the first padding mode.

[0491] In a possible implementation, the filtering module is specifically configured to perform pixel padding at the first boundary of the image unit based on the first padding mode when a distance of a pixel to be filtered to the first boundary of the image unit is less than M, the pixel to be filtered being a pixel in the image unit, a value of M being determined according to the first filter, and M being a positive integer.

[0492] In a possible implementation, the filtering module is specifically configured to fill first pixel information corresponding to the first padding mode into the first boundary of the image unit based on the first padding mode, the first pixel information being information of a pixel within the first boundary.

[0493] In a possible implementation, the filtering module is specifically configured to perform loop filtering on the pixel to be filtered in the image unit after the pixel padding based on the first filter.

[0494] In a possible implementation, the first filter is an adaptive loop filter ALF or a cross-component adaptive loop filter CCALF; and the first pixel information comprises at least one of the following: luminance information after sample adaptive offset filter SAO processing; chroma information after SAO processing; luminance information before deblocking filtering; chroma information before deblocking filtering; luminance residual information; luminance information after luminance fixed filter processing; chroma information after chroma fixed filter processing; and luminance information after luminance Gaussian filter processing.

[0495] In a possible implementation, the first filter is a cross-component adaptive loop filter CCALF; and the first pixel information comprises at least one of the following: luminance information after sample adaptive offset filter SAO processing; chroma information after SAO processing; luminance information before deblocking filtering; chroma information before deblocking filtering; luminance residual information; luminance information after luminance fixed filter processing; chroma information after chroma fixed filter processing; and luminance information after luminance Gaussian filter processing.

[0496] In a possible implementation, the image unit is an image, and the first boundary of the image unit is a virtual boundary of the image.

[0497] In a possible implementation, the image unit is a coding tree unit CTU, and the first boundary of the image unit is a virtual boundary of the CTU.

[0498] In a possible implementation, the image unit is a CTU; and the filter module is specifically configured to, if there is no adjacent CTU at the first boundary of the CTU, perform pixel padding at the boundary of the CTU based on a first padding manner.

[0499] In a possible implementation, the filter module is specifically configured to fill, based on the first padding manner, information of N rows of pixels or N columns of pixels in the first boundary to the first boundary; the N rows of pixels are the first row of pixels to the Nth row of pixels in the first boundary, and the first row of pixels is a pixel adjacent to the first boundary; the N columns of pixels are the first column of pixels to the Nth column of pixels in the first boundary, and the first column of pixels is a pixel adjacent to the first boundary; and N is a positive integer.

[0500] In a possible implementation, the first padding manner is a mirror padding manner; and the filling, based on the first padding manner, of the information of the N rows of pixels or the N columns of pixels in the first boundary to the first boundary comprises: mirroring, based on the mirror padding manner, the information of the N rows of pixels or the N columns of pixels in the first boundary to the first boundary with the first boundary as a symmetry axis.

[0501] In a possible implementation, the first boundary of the image unit is a region adjacent to the first boundary and outside the first boundary.

[0502] The loop filtering device provided by the embodiments of the present application can adaptively determine the filling mode based on the projection format corresponding to the image unit or the rate-distortion decision result of pixel filling at the boundary of the reference image in the process of loop filtering the image unit. Since the processing modes at the image boundary of the image block of different projection formats are different, the filling mode of directly copying one row or one column of pixels at the boundary cannot well adapt to the image boundary characteristics under different projection formats, and the corresponding filling mode is determined based on the projection format corresponding to the image unit, so that a suitable filling strategy can be adaptively selected according to the image characteristics of different projection formats, and the distortion or artifact of the filtered image is avoided, thereby improving the filtering effect on the boundary region of the image block. In addition, the rate-distortion decision result of pixel filling at the boundary of the reference image is introduced, and the image quality change after encoding under different filling modes is evaluated, so that the filling mode with the best filtering effect while ensuring the encoding efficiency can be selected. In this way, the filling mode is adaptively determined based on the projection format of the image unit or the rate-distortion decision result in the present application, so that the change of the texture topology at the boundary of the filtered image is avoided, thereby improving the filtering performance. Each module of the loop filtering device described above can also be used to perform other actions in the method embodiments described above, and all related contents of each step involved in the method embodiments described above can be cited to the function description of the corresponding function module, which will not be described here.

[0503] The loop filtering device described above can be a device used to perform loop filtering related functions in the video encoding device shown in FIG. 1, or a device used to perform loop filtering related functions in the encoder 112 shown in FIG. 3; or the filter unit 209 shown in FIG. 3 or a device in the filter unit 209; or the loop filtering unit 307 shown in FIG. 4 or a device in the loop filtering unit 307.

[0504] In the case of using an integrated unit, FIG. 15 shows another possible structural schematic diagram of the loop filtering device involved in the embodiments described above. As shown in FIG. 15, the loop filtering device provided by the embodiments of the present application can include a processing module 801, a communication module 802 and a storage module 803. The processing module 801 can be used to control and manage the actions of the loop filtering device, for example, the processing module 801 can be used to support the loop filtering device to perform steps 501 to 503 and step 504 and the like in the method embodiments described above, and / or other processes of the technologies described herein. The communication module 802 can be used to support the communication of the loop filtering device with other network entities, for example, the communication with the reconstruction unit 208, the storage module 803 is used to store the program code and data of the loop filtering device, for example, to store the filtered image or the bitstream.

[0505] The processing module 801 can be a processor, for example, the processor can be the encoder 122 in FIG. 3. The communication module 802 can be a transceiver, transceiver circuit or communication interface, for example, the communication interface 121 in FIG. 3, and the storage module 803 can be a memory.

[0506] For more details of the modules of the loop filtering apparatus to implement the above functions, please refer to the descriptions in the foregoing method embodiments, which will not be repeated here.

[0507] The modules of the loop filtering apparatus can also be used to perform other actions in the foregoing method embodiments, and all related contents of the steps involved in the foregoing method embodiments can be referred to the descriptions of the corresponding functional modules, which will not be repeated here.

[0508] The loop filtering apparatus shown in FIG. 15 can be the apparatus in the video encoding device in FIG. 1 for performing loop filtering related functions, or the apparatus in the encoder 112 in FIG. 3 for performing loop filtering related functions.

[0509] FIG. 16 shows another possible structural schematic diagram of the loop filtering apparatus involved in the foregoing embodiments. As shown in FIG. 16, the loop filtering apparatus 900 includes a bitstream parsing module 901 and a filtering module 902; the bitstream parsing module 901 is configured to parse a bitstream to obtain a syntax element, the syntax element being used to indicate a first pixel padding mode adopted when performing loop filtering; the filtering module 902 is configured to perform pixel padding at a first boundary of an image unit based on the first padding mode; and the filtering module 902 is further configured to perform loop filtering on the image unit after the pixel padding based on a first filter.

[0510] In a possible implementation, the filtering module is specifically configured to perform pixel padding at the first boundary based on the first padding mode when a distance from a to-be-filtered pixel to the first boundary of the image unit is less than M, the to-be-filtered pixel being a pixel in the image unit, a value of M being determined according to the first filter, and M being a positive integer.

[0511] In a possible implementation, the filtering module is specifically configured to fill first pixel information corresponding to the first padding mode to the first boundary of the image unit based on the first padding mode, the first pixel information being information of a pixel within the first boundary.

[0512] In a possible implementation, the filtering module is specifically configured to perform loop filtering on the to-be-filtered pixel in the image unit after the pixel padding based on the first filter.

[0513] In a possible implementation, the first filter is an adaptive loop filter ALF or a cross-component adaptive loop filter CCALF; and the first pixel information comprises at least one of the following: luminance information after sample adaptive offset filter SAO processing; chroma information after SAO processing; luminance information before deblocking filtering; chroma information before deblocking filtering; luminance residual information; luminance information after luminance fixed filter processing; chroma information after chroma fixed filter processing; and luminance information after luminance Gaussian filter processing.

[0514] In a possible implementation, the first filter is a cross-component adaptive loop filter CCALF; and the first pixel information comprises at least one of the following: luminance information after sample adaptive offset filter SAO processing; chroma information after SAO processing; luminance information before deblocking filtering; chroma information before deblocking filtering; luminance residual information; luminance information after luminance fixed filter processing; chroma information after chroma fixed filter processing; and luminance information after luminance Gaussian filter processing.

[0515] In a possible implementation, the image unit is an image, and the first boundary of the image unit is a virtual boundary of the image.

[0516] In a possible implementation, the image unit is a coding tree unit CTU, and the first boundary of the image unit is a virtual boundary of the CTU.

[0517] In a possible implementation, the image unit is a CTU; and the filter module is specifically configured to, if there is no adjacent CTU at the first boundary of the CTU, perform pixel padding at the boundary of the CTU based on a first padding manner.

[0518] In a possible implementation, the filter module is specifically configured to fill, based on the first padding manner, information of N rows of pixels or N columns of pixels in the first boundary to the first boundary; the N rows of pixels are the first row of pixels to the Nth row of pixels in the first boundary, the first row of pixels being a pixel adjacent to the first boundary; the N columns of pixels are the first column of pixels to the Nth column of pixels in the first boundary, the first column of pixels being a pixel adjacent to the first boundary; and N is a positive integer.

[0519] In a possible implementation, the first padding manner is a mirror padding manner; and the filling, based on the first padding manner, of the information of the N rows of pixels or the N columns of pixels in the first boundary to the first boundary comprises: mirroring, based on the mirror padding manner, the information of the N rows of pixels or the N columns of pixels in the first boundary to the first boundary with the first boundary as a symmetry axis.

[0520] In a possible implementation, the first boundary of the image unit is a region adjacent to the first boundary and outside the first boundary.

[0521] The loop filtering device provided by the embodiments of the present application can effectively improve the filtering effect of the image boundary region, avoid the distortion or artifact problem of the filtered image, and thus improve the filtering effect of the image boundary region, by parsing the filtering parameters and the padding mode information from the code stream and using the corresponding pixel padding technology in the filtering process.

[0522] The modules of the loop filtering device described above can also be used to perform other actions in the method embodiments described above, and all related content of each step involved in the method embodiments described above can be referred to the function description of the corresponding function module, which will not be repeated here.

[0523] In the case of using an integrated unit, FIG. 17 shows another possible structural schematic diagram of the loop filtering device involved in the embodiments described above. As shown in FIG. 17, the loop filtering device provided by the embodiments of the present application can include a processing module 1801, a communication module 1802 and a storage module 1803. The processing module 1801 can be used to control and manage the actions of the image decoding device, for example, the processing module 1801 can be used to support the loop filtering device to perform steps 601 to 604 and the like in the method embodiments described above, and / or other processes of the technologies described herein. The communication module 1802 can be used to support the communication of the image decoding device with other network entities, for example, the communication with the loop filtering device. The storage module 1803 is used to store the program code and data of the loop filtering device, for example, to store the decoded image or the code stream obtained by the loop.

[0524] The processing module 1801 can be a processor, for example, the processor can be the decoder 122 in FIG. 3. The communication module 1802 can be a transceiver, transceiver circuit or communication interface, for example, the communication interface 121 in FIG. 3, and the storage module 1803 can be a memory.

[0525] The loop filtering device shown in FIG. 17 can be the device used to perform the loop filtering related functions in the decoder 122 in FIG. 3 described above.

[0526] For more details of the functions of the modules included in the image decoding device described above, please refer to the description in the method embodiments described above, which will not be repeated here.

[0527] It should be noted that the scope of the methods and devices in the embodiments of the present application is not limited to the order of the functions shown or discussed, and can also include performing the functions in a substantially simultaneous manner or in a reverse order, for example, the described method can be performed in an order different from the described order, and various steps can also be added, omitted or combined. In addition, the features described with reference to certain examples can be combined in other examples.

[0528] In the above embodiments, all or part can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, the computer instructions produce the processes or functions in the embodiments of the present application in whole or in part. The computer can be a general purpose computer, a special purpose computer, a computer network or other programmable apparatus. The computer instructions can be stored in a computer readable storage medium or transferred from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be transferred from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer readable storage medium can be any available medium accessible by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available media can be magnetic media (such as floppy disk, magnetic disk, magnetic tape), optical media (such as digital video disc (DVD)), or semiconductor media (such as solid state drive (SSD)) and the like.

[0529] Through the description of the above embodiments, those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above division of functional modules is taken as an example, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0530] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules or units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed each other can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0531] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one place, or distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0532] In addition, the functional units in various embodiments of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0533] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the method described in various embodiments of the present application. The aforementioned storage medium includes: a flash memory, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk, and various media that can store program codes.

[0534] The embodiment of the present application also provides a readable storage medium, which stores a program or instructions, and the program or instructions are executed by a processor to realize various processes of the loop filtering method embodiment and achieve the same technical effects. To avoid repetition, details are not repeated here. The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer readable memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0535] The embodiment of the present application further provides a chip, which includes a processor and a communication interface, the communication interface is coupled with the processor, and the processor is used to run a program or instructions to realize various processes of the loop filtering method embodiment and achieve the same technical effects. To avoid repetition, details are not repeated here.

[0536] It should be understood that the chip mentioned in the embodiment of the present application can also be referred to as a system level chip, a system chip, a chip system or a system on chip, etc.

[0537] The embodiment of the present application provides a computer program product, the program product is stored in a storage medium, the program product is executed by at least one processor to realize the various processes of the loop filtering method embodiment and can achieve the same technical effects, and here is not repeated for avoiding repetition.

[0538] It should be noted that, in this document, the terms "comprising", "including", or any other variant thereof are intended to cover a non-exclusive inclusion, so that processes, methods, articles, or apparatuses that comprise a list of elements not only include those elements, but also include other elements that are not expressly listed, or other elements inherent in such processes, methods, articles, or apparatuses. Without more limitations, the element defined by the sentence "comprising a" does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element. In addition, it should be pointed out that the scope of the method and apparatus in the embodiments of the present application is not limited to the order of performing the functions shown or discussed, and can also include performing the functions in a substantially simultaneous manner or in a reverse order, for example, the described method can be performed in an order different from the described order, and various steps can also be added, omitted, or combined. In addition, the features described with reference to certain examples can be combined in other examples.

[0539] From the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment method can be realized by means of software and a necessary general hardware platform, of course, it can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a plurality of instructions for making a terminal (which can be a mobile phone, computer, server, or network equipment, etc.) execute the method described in each embodiment of the present application.

[0540] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, any change or replacement within the technical scope disclosed in the present application should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A loop filtering method applied to an encoder, characterized in that, The method comprises: performing pixel padding at a first boundary of an image unit based on a first padding mode, the first padding mode being determined based on at least one of a projection format corresponding to the image unit and a rate-distortion decision result of pixel padding at a reference image boundary; performing loop filtering on the image unit after pixel padding based on a first filter; writing, into a bitstream, a filter parameter of the loop filtering and a syntax element used for indicating the first padding mode.

2. The method of claim 1, wherein, The performing pixel padding at the first boundary of the image unit based on the first padding mode comprises: when a distance from a pixel to be filtered to the first boundary of the image unit is less than M, performing pixel padding at the first boundary based on the first padding mode, the pixel to be filtered being a pixel in the image unit, a value of M being determined according to the first filter, and M being a positive integer.

3. The method of claim 2, wherein, The performing pixel padding at the first boundary of the image unit based on the first padding mode comprises: filling, based on the first padding mode, first pixel information corresponding to the first padding mode into the first boundary of the image unit, the first pixel information being information of a pixel in the first boundary.

4. The method according to any one of claims 1 to 3, characterized in that, The first filter is an adaptive loop filter (ALF) or a cross-component adaptive loop filter (CCALF).

5. The method of claim 4, wherein, The first pixel information comprises at least one of: luminance information after SAO processing; chroma information after the SAO processing; luminance information before de-blocking filtering; chroma information before the de-blocking filtering; luminance residual information; luminance information after luminance fixed filter processing; chroma information after chroma fixed filter processing; luminance information after luminance Gaussian filter processing.

6. The method of claim 1, wherein, The image unit is an image, and the first boundary of the image unit is a virtual boundary of the image.

7. The method of claim 1, wherein, The image unit is a coding tree unit (CTU), and the first boundary of the image unit is a virtual boundary of the CTU.

8. The method of claim 1, wherein, The image unit is a CTU. The performing pixel padding at the first boundary of the image unit based on the first padding mode comprises: if there is no adjacent CTU at the first boundary of the CTU, performing pixel padding at the boundary of the CTU based on the first padding mode.

9. The method of claim 3, wherein, The filling, based on the first padding mode, first pixel information corresponding to the first padding mode into the first boundary of the image unit comprises: filling, based on the first padding mode, information of N rows of pixels or N columns of pixels in the first boundary into the first boundary, the N rows of pixels being a first row of pixels to an Nth row of pixels in the first boundary, the first row of pixels being a pixel in a row adjacent to the first boundary, the N columns of pixels being a first column of pixels to an Nth column of pixels in the first boundary, the first column of pixels being a pixel in a column adjacent to the first boundary, and N being a positive integer.

10. The method of claim 9, wherein, The first padding mode is a mirror padding mode, and the filling, based on the first padding mode, information of N rows of pixels or N columns of pixels in the first boundary into the first boundary comprises: Fill the information of N rows of pixels or N columns of pixels in the first boundary with the first boundary as a symmetric axis based on the mirror filling manner.

11. The method according to claim 9 or 10, characterized in that, The first boundary of the image unit is adjacent to the first boundary.

12. A loop filtering method applied to a decoding end, characterized in that, The method comprises: Parsing a bitstream to obtain a syntax element, the syntax element being used to indicate a first filling manner adopted when performing loop filtering; Performing pixel filling at a first boundary of an image unit based on the first filling manner; Performing loop filtering on the image unit after pixel filling based on a first filter.

13. The method of claim 12, wherein, The pixel filling at the first boundary of the image unit based on the first filling manner comprises: When a distance from a pixel to be filtered to the first boundary of the image unit is less than M, performing pixel filling at the first boundary based on the first filling manner, the pixel to be filtered being a pixel in the image unit, the value of M being determined according to the first filter, and M being a positive integer.

14. The method of claim 13, wherein, The pixel filling at the first boundary of the image unit based on the first filling manner comprises: Filling first pixel information corresponding to the first filling manner to the first boundary of the image unit based on the first filling manner, the first pixel information being information of a pixel in the first boundary.

15. The method according to any one of claims 12 to 14, characterized in that, The first filter is an adaptive loop filter (ALF) or a cross-component adaptive loop filter (CCALF).

16. The method of claim 15, wherein, The first pixel information comprises at least one of: luminance information after SAO processing; chroma information after SAO processing; luminance information before deblocking filtering; chroma information before deblocking filtering; luminance residual information; luminance information after luminance fixed filter processing; chroma information after chroma fixed filter processing; luminance information after luminance Gaussian filter processing.

17. The method of claim 12, wherein, The image unit is an image, and the first boundary of the image unit is a virtual boundary of the image.

18. The method of claim 12, wherein, The image unit is a coding tree unit (CTU), and the first boundary of the image unit is a virtual boundary of the CTU.

19. The method of claim 12, wherein, The image unit is a CTU. The pixel filling at the first boundary based on the first filling manner comprises: If there is no adjacent CTU at the first boundary of the CTU, performing pixel filling at the boundary of the CTU based on the first filling manner.

20. The method of claim 14, wherein, The filling of the first pixel information to the first boundary based on the first filling manner comprises: Filling information of N rows of pixels or N columns of pixels in the first boundary to the first boundary based on the first filling manner, the N rows of pixels being a first row of pixels to an Nth row of pixels in the first boundary, the first row of pixels being a pixel in a row adjacent to the first boundary, the N columns of pixels being a first column of pixels to an Nth column of pixels in the first boundary, the first column of pixels being a pixel in a column adjacent to the first boundary, and N being a positive integer.

21. The method of claim 20, wherein, The first filling manner is a mirror filling manner, and the filling of the information of N rows or N columns of pixels in the first boundary to the first boundary based on the first filling manner comprises: The first boundary is taken as a symmetric axis, and information of N rows or N columns of pixels inside the first boundary is filled to the first boundary based on the mirror filling manner.

22. The method of claim 20 or 21, wherein, The first boundary of the image unit is adjacent to the first boundary.

23. A loop filtering apparatus applied to an encoder, characterized in that, The device comprises a filtering module and an encoding module, wherein: The filtering module is configured to perform pixel filling at the first boundary of the image unit based on a first filling manner, and the first filling manner is determined based on at least one of the following: a projection format corresponding to the image unit, and a rate-distortion decision result of pixel filling at a reference image boundary. The filtering module is further configured to perform loop filtering on the image unit after pixel filling based on a first filter. The encoding module is configured to write a filtering parameter and a syntax element of the loop filtering into a bitstream, and the syntax element is used to indicate the first filling manner.

24. The apparatus of claim 23, wherein, The filtering module is specifically configured to perform pixel filling at the first boundary based on the first filling manner when a distance between a pixel to be filtered and the first boundary of the image unit is less than M, the pixel to be filtered is a pixel in the image unit, a value of M is determined according to the first filter, and M is a positive integer.

25. The apparatus of claim 24, wherein, The filtering module is specifically configured to fill first pixel information to the first boundary of the image unit based on a first filling manner, and the first pixel information is information of a pixel inside the first boundary and corresponding to the first filling manner.

26. The apparatus of any one of claims 23-25, wherein, The first filter is an adaptive loop filter (ALF) or a cross-component adaptive loop filter (CCALF).

27. The apparatus of claim 26, wherein, The first pixel information comprises at least one of the following: luminance information after sample adaptive offset (SAO) processing; chroma information after the SAO processing; luminance information before de-blocking filtering; chroma information before de-blocking filtering; luminance residual information; luminance information after luminance fixed filter processing; chroma information after chroma fixed filter processing; luminance information after luminance Gaussian filter processing.

28. The apparatus of claim 23, wherein, The image unit is an image, and the first boundary of the image unit is a virtual boundary of the image.

29. The apparatus of claim 23, wherein, The image unit is a coding tree unit (CTU), and the first boundary of the image unit is a virtual boundary of the CTU.

30. The apparatus of claim 23, wherein, The image unit is a CTU, and the filtering module is specifically configured to perform pixel filling at a boundary of the CTU based on the first filling manner if there is no adjacent CTU at the first boundary of the CTU.

31. The apparatus of claim 25, wherein, The filtering module is specifically configured to fill information of N rows or N columns of pixels inside the first boundary to the first boundary based on a first filling manner, the N rows of pixels are a first row of pixels to an Nth row of pixels inside the first boundary, the first row of pixels is a pixel of a row adjacent to the first boundary, the N columns of pixels are a first column of pixels to an Nth column of pixels inside the first boundary, the first column of pixels is a pixel of a column adjacent to the first boundary, and N is a positive integer.

32. The apparatus of claim 31, wherein, The first padding manner is a mirror padding manner; and the information of N rows or N columns of pixels in the first boundary is filled to the first boundary based on the first padding manner, including: The information of N rows or N columns of pixels in the first boundary is mirrored and filled to the first boundary based on the mirror padding manner, with the first boundary as a symmetry axis.

33. The apparatus of claim 31 or 32, wherein, The first boundary of the image unit is a region outside the first boundary and adjacent to the first boundary.

34. A loop filtering apparatus applied to a decoding end, characterized in that, The apparatus includes a bitstream parsing module and a filtering module, wherein: The bitstream parsing module is configured to parse a bitstream to obtain a syntax element, which is used to indicate a first padding manner used when performing loop filtering. The filtering module is configured to perform pixel padding at a first boundary of an image unit based on the first padding manner. The filtering module is further configured to perform loop filtering on the image unit after pixel padding based on a first filter.

35. The apparatus of claim 34, wherein, The filtering module is specifically configured to perform pixel padding at the first boundary based on the first padding manner when a distance from a pixel to be filtered to the first boundary of the image unit is less than M, the pixel to be filtered being a pixel in the image unit, M being determined according to the first filter, and M being a positive integer.

36. The device of claim 35, wherein, The filtering module is specifically configured to fill first pixel information to the first boundary of the image unit based on the first padding manner.

37. The apparatus of any one of claims 34-36, wherein, The first filter is an adaptive loop filter (ALF) or a cross-component adaptive loop filter (CCALF).

38. The device of claim 37, wherein, The first pixel information includes at least one of the following: luminance information processed by a sample adaptive offset (SAO) filter; chroma information processed by the SAO filter; luminance information before de-blocking filtering; chroma information before de-blocking filtering; luminance residual information; luminance information processed by a luminance fixed filter; chroma information processed by a chroma fixed filter; luminance information processed by a luminance Gaussian filter.

39. The device of claim 34, wherein, The image unit is an image, and the first boundary of the image unit is a virtual boundary of the image.

40. The apparatus of claim 34, wherein, The image unit is a coding tree unit (CTU), and the first boundary of the image unit is a virtual boundary of the CTU.

41. The apparatus of claim 34, wherein, The image unit is a CTU; and the filtering module is specifically configured to perform pixel padding at a boundary of the CTU based on a first padding manner if there is no adjacent CTU at the first boundary of the CTU.

42. The device of claim 34, wherein, The filtering module is specifically configured to fill information of N rows or N columns of pixels in the first boundary to the first boundary based on the first padding manner; the N rows of pixels are a first row of pixels to an Nth row of pixels in the first boundary, the first row of pixels being a pixel in a row adjacent to the first boundary; the N columns of pixels are a first column of pixels to an Nth column of pixels in the first boundary, the first column of pixels being a pixel in a column adjacent to the first boundary; and N is a positive integer.

43. The device of claim 42, wherein, The first padding mode is a mirror padding mode; and the filter module is specifically configured to mirror pad information of N rows of pixels or N columns of pixels within the first boundary to the first boundary based on the mirror padding mode and taking the first boundary as a symmetric axis.

44. The device of claim 42 or 43, wherein, The first boundary of the image unit is a region outside the first boundary and adjacent to the first boundary.

45. A computer-readable storage medium, comprising: The computer readable storage medium stores a computer program, and the computer program is executed by at least one processor to implement the loop filtering method in any one of claims 1 to 11, or implement the loop filtering method in any one of claims 12 to 22.

46. A computer program product, characterised in that, The computer program instructions cause a computer to execute the loop filtering method in any one of claims 1 to 11, or execute the loop filtering method in any one of claims 12 to 22.

Images