Video decoding method and device, computer system and storage medium

Abstract

A method, an apparatus, a non-transitory computer-readable medium, and a computer system for encoding or decoding video data are provided. The method may include: receiving video data including a chroma component and a luminance component; parsing, deriving, or selecting a preset number of chroma components in a frame and a preset number of luma components in the frame; and decoding the video data. The method may further include performing a first constrained directional enhancement filter (CDEF) process that filters the chroma components based on a preset number of the chroma components in the frame, performing a second CDEF process that filters the luma components based on a preset number of the luma components in the frame, the first CDEF processing and the second CDEF processing are independent of each other and are respectively executed.

Description

[0001] Priority information

[0002] This application claims the priority of U.S. Provisional Application No. 63 / 040,856, filed June 18, 2020, and U.S. Application No. 17 / 091,759, filed November 6, 2020, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This disclosure generally relates to the field of data processing, and more specifically to video encoding and / or decoding (e.g., by encoders, decoders, or codecs (decoders and encoders)). Background Technology

[0004] AOMedia Video 1 (AV1) is an open video coding format designed for video transmission over the Internet. AOMedia Video 1 was developed as a successor to codec extensions in related technologies, for example.

[0005] AOMedia 1 typically uses Constrained Directional Enhancement Filters (CDEF). Existing CDEFs have three limitations: one is that the luma and chroma components are forced to share a preset at the image level; another limitation is that the luma and chroma components are forced to pick the same preset index at the block level; and yet another limitation is that when deriving the filter strength for the chroma component, the luma block size is used to determine the input for the chroma component. These limitations may restrict the performance of CDEFs. Summary of the Invention

[0006] Implementations relate to methods, apparatus, systems, and computer-readable media for encoding and / or decoding video data. According to one aspect, a method for encoding and / or decoding video data is provided. The method may include: receiving video data comprising chroma components and luma components; parsing, deriving, or selecting a preset number of filters for the chroma components and a preset number of filters for the luma components in a frame; and decoding the video data, wherein the method includes: performing a first constrained directional enhancement filter (CDEF) process to filter the chroma components based on the preset number of filters for the chroma components in the frame, and performing a second CDEF process to filter the luma components based on the preset number of filters for the luma components in the frame, the first CDEF process and the second CDEF process being performed independently of each other and separately.

[0007] The filter is preset to include primary and secondary intensities.

[0008] The method may include performing first and second constrained directional enhancement filter (CDEF) processes that independently filter the luma and chroma components when the luma and chroma components have different partitions or semi-decoupled partitions; and obtaining the output of an independent CDEF process that includes filtered reconstructed samples of the luma / chroma components, wherein the input of the first / second CDEF process is the reconstructed samples of the luma / chroma components, and the intermediate output of the first and second CDEF processes includes derived filter presets and per-block preset indices.

[0009] At the image level, the number of preset filters derived from the luminance component differs from the number of preset filters derived from the chrominance component.

[0010] The preset number of filters at the image level can be one of 1, 2, 4 or 8.

[0011] The preset number of filters derived and selected for the luminance component in a frame is 2, while the preset number of filters derived and selected for the chrominance component in a frame is 1.

[0012] The number of presets for the luminance component is N, where N is a positive integer, while the number of presets for the chrominance component is fixed at 1. The number of preset filters for the chrominance component is exported as 1 in the decoder without signaling notification.

[0013] The selected preset index for the current luma block is different from that for the current chroma block, and the input to the independent CDEF process is the luma / chroma reconstruction sample of the current block, as well as the filter presets derived and selected at the frame level. The output of this process is an index indicating which filter preset to select for the current block.

[0014] The method may further include: when the preset number of filters for the luminance component at the frame level corresponds to 8 presets and the preset number of filters for the chrominance component corresponds to 4 presets, selecting the preset index of luminance block A as 7 and the preset index of chrominance block B as 1, wherein luminance block A and chrominance block B are located in the same position or partially in the same position.

[0015] The method may also include: when deriving the CDEF filter intensity of the chroma components, determining the input reconstructed sample based on the current chroma coding block size.

[0016] The method may also include: when the current chroma block has a specific size, the input is the chroma reconstruction sample value of the current block with a specific size.

[0017] The method may also include: when independent partitioning or semi-decoupled partitioning is applied to the luma block and the chroma block, the luma block and the chroma block still share the same preset index, and only one of the sizes of the luma block or the chroma block is used in the preset index export / signaling processing.

[0018] The method may further include performing CDEF filtering on the luma component and the chroma component respectively when the luma component and the chroma component have the same coding block size.

[0019] Image-level presets can be signaled separately for the luminance and chrominance components in the advanced parameter set, slice header, image header, or SEI message.

[0020] The preset filter for the luminance component can be notified first using signaling, and then the preset filter for the chrominance component can be notified using signaling.

[0021] Signaling is used to notify the block-level preset index for the luminance and chrominance components respectively.

[0022] First, signal the preset index of the luminance component, and then signal the preset index of the chrominance component.

[0023] A computer system for decoding video data, the computer system comprising: one or more computer-readable non-transitory storage media configured to store computer program code; and one or more computer processors configured to access the computer program code and operate according to instructions of the computer program code, the computer program code comprising: receiving code configured to cause the one or more computer processors to receive video data including chroma components and luma components; parsing, exporting, or selecting code configured to cause the one or more computer processors to parse, export, or select a preset number of filters for the chroma components in a frame and a preset number of filters for the luma components in that frame; and decoding code configured to cause the one or more computer processors to decode the video data, wherein the method comprises: performing a first constrained directional enhancement filter (CDEF) process to filter the chroma components based on the preset number of filters for the chroma components in the frame, and performing a second CDEF process to filter the luma components based on the preset number of filters for the luma components in the frame, the first CDEF process and the second CDEF process being independent of each other and executed separately.

[0024] A non-transitory computer-readable medium storing a computer program for decoding video data, the computer program being configured to cause one or more computer processors to: receive video data including chroma and luma components; parse, export, or select code configured to cause one or more computer processors to parse, export, or select a preset number of filters for the chroma components and a preset number of filters for the luma components in a frame; and decode code configured to cause one or more computer processors to decode the video data, wherein the method includes: performing a first constrained directional enhancement filter (CDEF) process to filter the chroma components based on a preset number of filters for the chroma components in the frame, and performing a second CDEF process to filter the luma components based on a preset number of filters for the luma components in the frame, the first CDEF process and the second CDEF process being independent of each other and executed separately.

[0025] This application proposes an Independent Constrained Directional Enhancement Filter (SCDEF) that performs CDEF processing on the luma and chroma components separately. Compared to conventional CDEF, SCDEF allows the luma and chroma components to be filtered independently. More specifically, the luma and chroma components can have different numbers of presets at the image level; furthermore, the luma and chroma components can select different preset indices at the block level, thereby enhancing the performance of video data filtering and resulting in more accurate and efficient video decoding. Attached Figure Description

[0026] These and other objects, features, and advantages will become apparent from the following detailed description of illustrative embodiments, which should be read in conjunction with the accompanying drawings. The various features in the drawings are not to scale, as the illustrations are intended to be clear to those skilled in the art in conjunction with the detailed description. In the drawings:

[0027] Figure 1 A networked computer environment according to at least one embodiment is illustrated;

[0028] Figure 2 The shape of the adaptive loop filter (ALF) is shown;

[0029] Figures 3A-3D The subsampling locations of the diagonal gradient are shown;

[0030] Figure 4 The modified block classification at the virtual boundary is shown;

[0031] Figure 5 The ALF filter used for modifying the luminance component at the virtual boundary is shown;

[0032] Figure 6 This shows the position of the chroma sample relative to the luminance sample;

[0033] Figure 7 An example of directional search for an 8×8 block is shown;

[0034] Figure 8 An example of directional search for an 8×8 block is shown;

[0035] Figure 9 An example of the coding tree structure (luminance and chrominance) is shown;

[0036] Figure 10 Independently constrained directional enhancement filter (SCDEF) is shown.

[0037] Figure 11 This is an operation flowchart illustrating the steps performed by a program that encodes video data according to at least one embodiment;

[0038] Figure 12 It is based on at least one embodiment. Figure 1 A block diagram depicting the internal and external components of a computer and server;

[0039] Figure 13 It includes, according to at least one embodiment. Figure 1 A block diagram illustrating a cloud computing environment for a computer system; and

[0040] Figure 14 It is based on at least one embodiment. Figure 13 A block diagram illustrating the functional layers of an illustrative cloud computing environment. Detailed Implementation

[0041] Detailed embodiments of the claimed structures and methods are disclosed herein; however, it is to be understood that the disclosed embodiments are merely illustrative of the claimed structures and methods, which can be implemented in various forms. These structures and methods may be implemented in many different forms and should not be construed as limited to the exemplary embodiments set forth herein. Rather, these exemplary embodiments are provided so that this disclosure will be thorough and complete and will fully convey the scope to those skilled in the art. Details of well-known features and techniques may be omitted in the specification to avoid unnecessarily obscuring the presented embodiments.

[0042] Implementation methods generally relate to the field of data processing, and more specifically to video encoding and / or decoding. The exemplary embodiments described below provide a system, method, and computer program for encoding and / or decoding video data, particularly.

[0043] As previously described, AOMedia Video 1 (AV1) is an open video coding format designed for video transmission over the Internet. AOMedia Video 1 was developed by the Alliance for Open Media (AOMedia) as the successor to VP9. The Alliance for Open Media is an alliance formed in 2015 that includes semiconductor companies, video-on-demand providers, video content producers, software development companies, and web browser vendors.

[0044] This document describes aspects with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer-readable media according to various embodiments. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0045] Now refer to Figure 1 This diagram illustrates a networked computer environment for a video encoding system 100 (hereinafter referred to as the "System") for encoding and / or decoding video data according to an embodiment. It should be understood that... Figure 1 This is merely an illustration of one implementation and does not imply any limitations regarding the environments in which different implementations can be implemented. Many modifications to the depicted environment are possible based on design and implementation requirements.

[0046] System 100 may include computer 102 and server computer 114. Computer 102 may communicate with server computer 114 via communication network 110 (hereinafter referred to as "network"). Computer 102 may include processor 104 and software program 108 stored on data storage device 106, and is capable of interfaceing with a user and communicating with server computer 114. (Referring to the following...) Figure 12 As discussed, computer 102 may include internal component 800A and external component 900A, and server computer 114 may include internal component 800B and external component 900B. Computer 102 may be, for example, a mobile device, telephone, personal digital assistant, netbook, laptop computer, tablet computer, desktop computer, or any type of computing device capable of running programs, accessing networks, and accessing databases.

[0047] Server computer 114 can also operate in cloud computing service models, such as Software as a Service (SaaS), Platform as a Service (PaaS), or Infrastructure as a Service (IaaS), as described below. Figure 13and Figure 14 The server computer 114 can also be located in a cloud computing deployment model, such as a private cloud, community cloud, public cloud, or hybrid cloud.

[0048] A server computer 114, which can be used to encode video data, is capable of running a video encoding or decoding program 116 (hereinafter referred to as the "program") that can interact with a database 112. The video encoding or decoding program method is described in more detail below with reference to FIG3. In one embodiment, computer 102 may operate as an input device including a user interface, while program 116 may primarily run on server computer 114. In an alternative embodiment, program 116 may primarily run on one or more computers 102, while server computer 114 may be used to process and store the data used by program 116. It should be noted that program 116 may be a standalone program or may be integrated into a larger video encoding program. Video encoding or decoding program 116 may correspond to an encoder, decoder, or codec (both encoder and decoder).

[0049] However, it should be noted that in some cases, the processing of program 116 can be shared between computer 102 and server computer 114 at any ratio. In another embodiment, program 116 can operate on more than one computer, server computer, or some combination of computers and server computers, for example, multiple computers 102 communicating with a single server computer 114 via network 110. In another embodiment, for example, program 116 can operate on multiple server computers 114 communicating with multiple client computers via network 110. Alternatively, the program can operate on a web server communicating with a server and multiple client computers via a network.

[0050] Network 110 may include wired connections, wireless connections, fiber optic connections, or some combination thereof. Typically, network 110 can be any combination of connections and protocols that support communication between computer 102 and server computer 114. Network 110 may include various types of networks, such as local area networks (LANs), wide area networks (WANs) such as the Internet, telecommunications networks such as public switched telephone networks (PSTNs), wireless networks, public switched networks, satellite networks, cellular networks (e.g., fifth-generation (5G), long-term evolution (LTE), third-generation (3G), code division multiple access (CDMA), etc.), public land mobile networks (PLMNs), metropolitan area networks (MANs), private networks, self-organizing networks, intranets, fiber optic-based networks, etc., and / or combinations of these or other types of networks.

[0051] Figure 1 The number and arrangement of devices and networks shown are provided as examples. In reality, additional devices and / or networks, fewer devices and / or networks, different devices and / or networks, or networks may exist. Figure 1 The devices and / or networks shown are arranged differently. Furthermore, Figure 1 The two or more devices shown can be implemented within a single device, or Figure 1 The single device shown can be implemented as multiple distributed devices. Alternatively or alternatively, a group of devices in system 100 (e.g., one or more devices) can perform one or more functions described as being performed by another group of devices in system 100.

[0052] 1 Adaptive loop filter (ALF)

[0053] In Universal Video Coding (VVC) (Draft 8), an Adaptive Loop Filter (ALF) with block-based filter adaptation is applied. For the luminance component, one of 25 filters is selected for each 4×4 block based on the direction and activity of the local gradient.

[0054] 1.1 Filter shape

[0055] In VVC (Draft 8), two diamond filter shapes can be used (such as...). Figure 2 (As shown). A 7×7 rhombus is suitable for the luminance component, and a 5×5 rhombus is suitable for the chrominance component.

[0056] 1.2 Block classification

[0057] For the luminance component, each 4×4 block is classified into one of 25 classes. The classification index C is based on its orientation D and the quantization value of the activity. Export, as follows:

[0058] (Equation 1)

[0059] To calculate D and First, use a one-dimensional Laplacian to calculate the gradients in the horizontal, vertical, and two diagonal directions:

[0060] (Equation 2)

[0061] (Equation 3)

[0062] (Equation 4)

[0063] (Equation 5)

[0064] Here, indices i and j refer to the coordinates of the top-left sample within the 4×4 block, and Representing coordinates Reconstructed samples at the location.

[0065] To reduce the complexity of block classification, a one-dimensional Laplace calculation based on subsampling is applied. For example... Figures 3A-3D As shown, gradient calculations in all directions use the same subsampling location (e.g., subsampling Laplacian calculations in all directions). For example, Figure 3A The subsampling locations of the vertical gradient are shown. Figure 3B The subsampling locations of the horizontal gradient are shown, and Figure 3C and Figure 3D The subsampled portion of the diagonal gradient is shown.

[0066] Then the maximum and minimum values ​​of the gradient D in the horizontal and vertical directions are set as follows:

[0067] (Equation 6)

[0068] The maximum and minimum values ​​of the gradients in the two diagonal directions are set as follows:

[0069] (Equation 7)

[0070] To derive the value of directionality D, these values ​​are compared to each other, and two thresholds are used. and :

[0071] Step 1. If and If both are true, then D is set to 0.

[0072] Step 2. If If yes, continue from step 3; otherwise, continue from step 4.

[0073] Step 3. If If the condition is met, then D is set to 2; otherwise, D is set to 1.

[0074] Step 4. If If the value is 4, then D is set to 4; otherwise, D is set to 3.

[0075] Activity value A is calculated as:

[0076] (Equation 8)

[0077] A is further quantized to the range of 0 to 4, inclusive, and the quantized value is represented as .

[0078] For the chromaticity components in the image, no classification method was applied; that is, a single set of ALF coefficients was applied to each chromaticity component.

[0079] 1.3 Geometric transform of filter coefficients and clipping values

[0080] Before filtering each 4×4 luminance block, the filter coefficients are adjusted based on the gradient value calculated for that block. and the corresponding filter limiting value Apply geometric transformations such as rotation or diagonal and vertical flipping. This is equivalent to applying these transformations to samples within the filter's support region. The idea is to make different blocks applying ALF more similar by aligning their orientations.

[0081] Three geometric transformations are introduced: diagonal transformation, vertical flip, and rotation.

[0082] diagonal: (Equation 9)

[0083] Vertical Flip: (Equation 10)

[0084] Rotation: (Equation 11)

[0085] Where K is the size of the filter, 0≤k,l≤ These are coefficient coordinates, such that the position (0,0) is in the upper left corner, and the position ( , (In the bottom right corner.) Based on the gradient value calculated for this block, adjust the filter coefficients. and amplitude limit Apply the transformation. Table 1 below summarizes the relationship between the transformation and the four gradients in the four directions.

[0086] Table 1 - Mapping of gradients and transformations computed for a block

[0087]

[0088] 1.4 Filter parameter signaling

[0089] In VVC (Draft 8), ALF filter parameters are signaled in the Adaptive Parameter Set (APS). Within an APS, up to 25 sets of luma filter coefficients and limiting indexes, and up to 8 sets of chroma filter coefficients and limiting indexes, can be signaled. To reduce bit overhead, filter coefficients from different classifications of the luma component can be merged. In the slice header, the index of the APS used for the current slice is signaled. ALF signaling in VVC is based on CTU (Draft 8).

[0090] The limiting index based on APS decoding allows for the determination of limiting values ​​using tables of luminance and chrominance limiting values. These limiting values ​​depend on the internal bit depth. More precisely, the table of limiting values ​​is obtained using the following formula:

[0091] (Equation 12)

[0092] When B equals the internal bit depth It is a predefined constant value equal to 2.35, and N equals 4, which is the number of allowable limit values ​​in VVC (Draft 8).

[0093] Table 2 shows the output of Equation 12.

[0094] Table 2 - Specifications of Alfclip depend on bitDepth and clipIdx

[0095]

[0096] In the slice header, up to seven APS indices can be signaled to specify the luma filter bank for the current slice. Further control over the filtering process can be achieved at the Coding Tree Block (CTB) level. A signaling flag indicating whether an ALF should be applied to the luma CTB is always used. The luma CTB can select a filter bank from 16 fixed filter banks and filter banks from the APS. The filter bank index is signaled for the luma CTB to indicate which filter bank to apply. The 16 fixed filter banks are predefined and hard-coded in both the encoder and decoder.

[0097] For chroma components, the APS index of the chroma filter bank used for the current slice is indicated by signaling in the slice header. At the CTB level, if there is more than one chroma filter bank in the APS, the filter index is indicated by signaling for each chroma CTB.

[0098] Filter coefficients can be quantized using a norm equal to 128. To limit multiplication complexity, bitstream consistency is applied, ensuring that coefficient values ​​at non-center positions are within the range of -27 to 27-1, inclusive. Center position coefficients are not signaled in the bitstream, and are assumed to be equal to 128.

[0099] In VVC (Draft 8), the syntax and semantics of the clipping index and clipping value are defined as follows: `alf_luma_clip_idx[sfIdx][j]` specifies the clipping index used before multiplying the clipping value by the j-th coefficient of the luminance filter notified by signaling, indicated by `sfIdx`. The requirement for bitstream consistency is that the value of `alf_luma_clip_idx[sfIdx][j]` with `sfIdx=0..alf_luma_num_filters_signalled_minus1` and `j=0..11` should be in the range of 0 to 3, inclusive.

[0100] The luminance filter limiting value AlfclipL[adaptation_parameter_set_id][filtIdx][j] with the element AlfClipL[adaptation_parameter_set_id][filtIdx][j] (where filtIdx=0..NumAlfFilters-1 and j=0..11) is derived as specified in Table 2 based on the bitDepth set equal to BitDepthY and the clipIdx set equal to alf_luma_clip_idx[alf_luma_coeff_delta_idx[filtIdx]][j].

[0101] `alf_chroma_clip_idx[altIdx][j]` specifies the clipping index used before multiplying the clipping value by the j-th coefficient of the alternative chroma filter with index `altIdx`. The requirement for bitstream consistency is that the value of `alf_chroma_clip_idx[altIdx][j]` with `altIdx=0..alf_chroma_num_alt_filters_minus1` and `j=0..5` should be in the range of 0 to 3, inclusive.

[0102] The chroma filter limiting value AlfClipC[adaptation_parameter_set_id][altIdx][j] with the element AlfClipC[adaptation_parameter_set_id][altIdx][j] (where altIdx=0..alf_chroma_num_alt_filters_minus1, j=0..5) is derived as specified in Table 2 based on the bitDepth set equal to BitDepthC and the clipIdx set equal to alf_chroma_clip_idx[altIdx][j].

[0103] 1.5 Filtering process

[0104] On the decoder side, when ALF is enabled for CTB, for each sample within the CU Filtering is performed to obtain the sample values ​​shown below. ,

[0105] (Equation 13)

[0106] in, Indicates the filter coefficients used for decoding. It is a limiting function. This represents the decoding limiting parameters. Variables k and l in... and The value varies between these values, where L represents the filter length. (Limiting function) It corresponds to the function By incorporating the limiting function, this loop filtering method becomes a nonlinear process, known as nonlinear ALF. The selected limiting value is encoded in the "alf_data" syntax element using the Golomb coding scheme corresponding to the index of the limiting value in Table 2. This coding scheme is the same as the coding scheme for the filter index.

[0107] 1.6 Virtual boundary filtering process for row buffer reduction

[0108] To reduce the row buffer requirements of ALF, modified block classification and filtering were used for samples near the horizontal CTU boundaries. For this purpose, such as... Figure 4 As shown, by shifting the horizontal CTU boundary by “N” samples, the virtual boundary can be defined as a line, where N equals 4 for the luminance component and N equals 2 for the chrominance component.

[0109] The modified block classification is applied to, for example Figure 4 The brightness components are shown. For the one-dimensional Laplacian gradient calculation of the 4×4 block above the virtual boundary, only samples above the virtual boundary are used. Similarly, for the one-dimensional Laplacian gradient calculation of the 4×4 block below the virtual boundary, only samples below the virtual boundary are used. By taking into account the reduced number of samples used in the one-dimensional Laplacian gradient calculation, the quantization of the activity value A is scaled accordingly.

[0110] For filtering, symmetrical padding at virtual boundaries is applied to both the luminance and chrominance components. For example... Figure 5 As shown in "ALF filtering for luminance component modification at virtual boundaries", when the sample being filtered is below the virtual boundary, the adjacent sample above the virtual boundary is filled. Simultaneously, the corresponding sample on the other side is also symmetrically filled.

[0111] 1.7 Largest coding unit (LCU)-aligned picture quadtree partitioning

[0112] To improve coding efficiency, an adaptive loop filter based on a sync image quadtree of coding units was proposed in JCTVC-C143[3]. The luminance image is divided into several multi-level quadtree partitions, and the boundary of each partition is aligned with the boundary of the maximum coding unit (LCU). Each partition has its own filtering process, and is therefore called a filtering unit (FU).

[0113] The two-pass coding stream is described below. In the first pass, the quadtree segmentation pattern and optimal filter for each function element (FU) are determined. During the decision-making process, the filtering distortion is estimated by FFDE. The reconstructed image is filtered based on the determined quadtree segmentation patterns of all FUs and the selected filter. In the second pass, CU-synchronized ALF on / off control is performed. Based on the ALF on / off result, the first filtered image is partially recovered from the reconstructed image.

[0114] A top-down segmentation strategy is employed, dividing the image into multi-level quadtree partitions using a rate-distortion metric. Each partition is called a filtering unit (LU). The segmentation process aligns the quadtree partitions with the LCU boundaries. The encoding order of the FUs follows the z-scan order. For example, an image can be segmented into 10 FUs, with the encoding order being FU0, FU1, FU2, FU3, FU4, FU5, FU6, FU7, FU8, and FU9.

[0115] To indicate the quadtree segmentation pattern of an image, segmentation flags can be encoded and sent in z-order.

[0116] Filters for each element-time (FU) can be selected from two filter banks based on a rate-distortion criterion. The first bank can have newly derived 1 / 2 symmetric square and diamond filters for the current FU. The second bank can come from a time-delay filter buffer; the time-delay filter buffer stores filters previously derived for FUs of previous images. The filter with the lowest rate-distortion cost from these two banks can be selected for the current FU. Similarly, if the current FU is not the lowest FU and can be further segmented into 4 sub-FUs, the rate-distortion cost of the 4 sub-FUs is calculated. By recursively comparing the rate-distortion costs in the segmented and unsegmented cases, the quadtree segmentation pattern of the image can be determined.

[0117] In JCTVC-C143, the maximum quadtree splitting level is 2, meaning the maximum number of functional units (FUs) is 16. During the quadtree splitting decision, the correlation values ​​used to derive the Wiener coefficients of the 16 FUs at the bottom quadtree level (minimum FUs) can be reused. The remaining FUs can then have their Wiener filters derived from the correlations of the 16 FUs at the bottom quadtree level. Therefore, only one framebuffer access is needed to derive the filter coefficients for all FUs.

[0118] After determining the quadtree segmentation pattern, CU-synchronized ALF on / off control is implemented to further reduce filtering distortion. By comparing filtering distortion and non-filtering distortion, leaf CUs can explicitly switch ALF on / off in their local regions. Coding efficiency can be further improved by redesigning the filter coefficients based on the ALF on / off result. However, the redesign process requires additional frame buffer accesses. In the proposed CS-PQALF encoder design, there is no redesign process after the CU-synchronized ALF on / off decision, minimizing the number of frame buffer accesses.

[0119] 2. Cross-component adaptive loop filter

[0120] Cross-component adaptive loop filter (CC-ALF) uses luminance sample values ​​to refine each chrominance component.

[0121] CC-ALF operates by applying a linear diamond filter to the luminance channel for each chromaticity component. The filter coefficients are sent in the APS, factored. Scaling and rounding are performed for fixed-point representation. Filter application is controlled by a variable block size and is notified by signaling via a context-coded flag received for each sample block. At the slice level, the block size and CC-ALF enable flag are received for each chroma component. The following block sizes (in chroma samples) are supported in the contribution: 16×16, 32×32, and 64×64.

[0122] The syntax changes of CC-ALF are described in Table 3 below.

[0123] Table 3

[0124]

[0125] The semantic description of CC-ALF related syntax is as follows:

[0126] The value of alf_ctb_cross_component_cb_idc[xCtb>>CtbLog2SizeY][yCtb>>CtbLog2SizeY] being equal to 0 indicates that the cross component Cb filter was not applied to the Cb color component sample block at the luminance location (xCtb, yCtb).

[0127] alf_cross_component_cb_idc[xCtb>>CtbLog2SizeY][yCtb>>CtbLog2SizeY] is not equal to 0, indicating that the alf_cross_component_cb_idc[xCtb>>CtbLog2SizeY][yCtb>>CtbLog2SizeY] cross component Cb filter is applied to the Cb color component sample block at the luminance position (xCtb, yCtb).

[0128] `alf_ctb_cross_component_cr_idc[xCtb>>CtbLog2SizeY][yCtb>>CtbLog2SizeY]` equal to 0 indicates that the cross-component Cr filter was not applied to the Cr color component sample block at the luminance position (xCtb, yCtb). `alf_cross_component_cr_idc[xCtb>>CtbLog2SizeY][yCtb>>CtbLog2SizeY]` not equal to 0 indicates that the Cr color component sample block at the luminance position (xCtb, yCtb) was applied to the `alf_cross_component_cr_idc[xCtb>>CtbLog2SizeY][yCtb>>CtbLog2SizeY]`.

[0129] 3 Chroma sampling format

[0130] This application Figure 6 The section on "Position of Chroma Samples Relative to Luminance Samples" shows the relative position of the top-left chroma sample when chroma_format_idc equals 1 (4:2:0 chroma format) and chroma_sample_loc_type_top_field or chroma_sample_loc_type_bottom_field equals a variable ChromaLocType value. The region represented by the top-left 4:2:0 chroma sample (depicted as a large red square with a large red dot at its center) is shown relative to the region represented by the top-left luminance sample (depicted as a small black square with a small black dot at its center). Regions represented by adjacent luminance samples are depicted as small gray squares with a small gray dot at their center.

[0131] 4 Constrained directional enhancement filter

[0132] The primary goal of the In-Loop Constrained Directional Enhancement Filter (CDEF) is to filter out coded artifacts while preserving image details. In HEVC, the Sample Adaptive Offset (SAO) algorithm achieves a similar goal by defining signal offsets for pixels of different categories. Unlike SAO, CDEF is a nonlinear spatial filter. The filter's design is constrained to be easily vectorized (i.e., achievable via SIMD operations), which is not the case for other nonlinear filters such as median filters and bilateral filters.

[0133] The CDEF design is based on the following observations: The amount of ringing artifacts in the encoded image is often roughly proportional to the quantization step size. The amount of detail is a property of the input image, but the smallest detail preserved in the quantized image is also often proportional to the quantization step size. For a given quantization step size, the amplitude of ringing is typically smaller than the amplitude of detail.

[0134] The CDEF works by identifying the orientation of each block, then performing adaptive filtering along that orientation, and a smaller degree of adaptive filtering along an orientation rotated 45 degrees from the identified orientation. The filter strength is explicitly communicated via signaling, allowing for a high degree of control over ambiguity. An effective encoder search is designed for the filter strength. CDEF is based on two previously proposed in-loop filters, and a combined filter is used in the emerging AV1 codec.

[0135] 4.1 Direction search

[0136] The orientation search operates on the reconstructed pixels immediately after the deblocking filter. Since these pixels are available for the decoder, orientation signaling is not required. The search operates on 8×8 blocks, small enough to adequately handle non-linear edges, yet large enough to reliably estimate orientation when applied to a quantized image. Having a constant orientation over an 8×8 region also simplifies the vectorization of the filter. For each block, we determine the orientation that best matches the pattern in the block by minimizing the sum of squared differences (SSD) between the quantized block and the nearest perfectly oriented block. A perfectly oriented block is a block in which all pixels along a line in one direction have the same value. Figure 7 This is an example of a direction search for an 8x8 block. In this case, a 45-degree direction is chosen (as shown in the box around column 12) because a 45-degree direction minimizes the error.

[0137] 4.2 Non-linear low-pass directional filter

[0138] The primary reason for identifying the direction is to align the filter taps along that direction to reduce ringing while preserving directional edges or patterns. However, directional filtering alone is sometimes insufficient to adequately reduce ringing. It is also desirable to use filter taps for pixels not along the main direction. To mitigate the risk of blurring, these additional taps are handled more appropriately. For this purpose, CDEF defines main taps and auxiliary taps. A complete two-dimensional CDEF filter is represented as follows:

[0139] (Equation 14)

[0140] Where D is the damping parameter, and These are the strengths of the main tap and the auxiliary tap, respectively. Round off when the value is far from zero. It is the filter weight, and It is a constraint function that operates on the difference between the filtered pixel and each of its neighboring pixels. For small differences, This makes the filter's characteristics similar to a linear filter. When the difference is large, This actually ignores the filter taps.

[0141] 5. Loop restoration in AV1

[0142] A set of in-loop recovery schemes is proposed for post-coding deblocking of video to generally denoise and improve edge quality beyond conventional deblocking operations. These schemes are switchable within a frame, using appropriately sized tiles. The specific schemes described are based on separable symmetric Wiener filters and dual self-guided filters with subspace projection. Because content statistics can vary significantly within a frame, these tools are integrated within a switchable framework, where different tools can be triggered in different regions of the frame.

[0143] 5.1 Separable symmetric wiener filter

[0144] One promising restoration tool has been demonstrated in the literature: the Wiener filter. Each pixel in the degraded frame can be reconstructed from its surrounding pixels. The non-causal filtering version of the pixels within the window, where For integers r that are odd. If the taps of the two-dimensional filter are in column vector form The element vector F represents the filter parameters. Direct LMMSE optimization then leads to the filter parameters being... Given, among which It is the autocovariance of x. Samples around pixels The column vectorized version in the window, and yes With the scalar source sample to be estimated The cross-correlation. The encoder can estimate H and M based on the implementation in the deblocked frame and the source, and send the resulting filter F to the decoder. However, this not only affects the transmission... Each tap incurs a considerable bitrate cost, and non-separable filtering makes decoding overly complex. Therefore, several additional constraints are imposed on the properties of F. First, F is constrained to be separable, such that filtering can be implemented as separable horizontal and vertical w-tap convolutions. Second, each of the horizontal and vertical filters is constrained to be symmetric. Third, it is assumed that the sum of the horizontal and vertical filter coefficients is 1.

[0145] 5.2 Double self-guided filtering with subspace projection

[0146] Guided filtering is one of the recent paradigms of image filtering, in which a local linear model is used:

[0147] (Equation 15)

[0148] Used based on unfiltered samples Calculate the filter output F and G are determined based on statistical information from the degraded and guiding images near the filtered pixels. If the guiding image is identical to the degraded image, the resulting self-guided filter has the effect of preserving smooth edges. The specific form of our proposed self-guided filter depends on two parameters: radius r and noise parameter e, as listed below:

[0149] 1. Obtain the surrounding area of ​​each pixel Mean value of pixels in the window and variance This can be effectively achieved using box filtering based on integral imaging.

[0150] 2. Perform calculations for each pixel:

[0151] 3. Calculate the F and G values ​​for each pixel as the average of the f and g values ​​in a 3×3 window around the pixel for use.

[0152] The filtering is controlled by r and e, where a larger r means a larger spatial variance, and a larger e means a larger range variance.

[0153] exist Figure 8 The principle of subspace projection is illustrated schematically. Even with inexpensive recovery... There is no single element close to the source Y. A proper multiplier {α,β} can bring them closer to the source, as long as they are moved slightly in the right direction. Figure 8 The subspace projection is shown using inexpensive recovery to produce a final recovery that is closer to the source.

[0154] 6. Semi-decoupled partitioning

[0155] Semi-decoupled partitioning (SDP), semi-splittered tree (SST), or flexible block partitioning is used for chroma components. In this method, luma blocks and chroma blocks within a superblock (SB) can have the same or different block partitions, depending on the luma coding block size or the luma tree depth. Specifically, when the luma block region size is greater than a threshold T1 or the luma block coding tree partition depth is less than or equal to a threshold T2, the chroma block uses the same coding tree structure as the luma. Otherwise, when the block region size is less than or equal to T1 or the luma partition depth is greater than T2, the corresponding chroma block can have a different coding block partition than the luma component; this is called flexible block partitioning for the chroma component. T1 is a positive integer, such as 128 or 256. T2 is a positive integer, such as 1 or 2.

[0156] An improved semi-decoupled partitioning (SDP) scheme is proposed, in which the luma and chroma components can share a portion of the tree structure from the root node of the superblock, and the condition under which luma and chroma begin to separate the tree partitions depends on the luma partitioning information. For example, Figure 9 An example of the encoding tree structure for the luminance and chrominance components is shown.

[0157] In Constrained Directed Enhancement Filters (CDEF), the luma and chroma components are constrained to share a preset at the image level. Furthermore, they are also constrained to have the same preset index at the block level. Finally, when deriving the filter strength for the chroma components, the luma block size is used to determine the input for the chroma components. These constraints may limit the coding efficiency of CDEF.

[0158] In traditional CDEF, a preset contains the primary / secondary intensities for luma and chroma. At the picture level, signaling indicates the number of allowed / available presets. At the code block level, signaling indicates the index to indicate which preset to select for the current block. CDEF code block sizes include 128×128, 128×64, 64×64, and 64×128. Traditional CDEF has three limitations: one is that luma and chroma components are forced to share presets at the picture level; another is that luma and chroma components are forced to pick the same preset index at the block level; and yet another is that the luma block size is used to determine the input to the chroma component when deriving the filter intensity of the chroma component. These limitations can collectively restrict CDEF performance, especially when the luma and chroma components have different partitioning schemes, such as those in semi-decoupled partitioning (SDP).

[0159] This document proposes an Independent Constrained Directional Enhancement Filter (SCDEF) that performs CDEF processing on the luma and chroma components separately. Compared to traditional CDEF, SCDEF allows the luma and chroma components to be filtered independently of each other. More specifically, the luma and chroma components can have different numbers of presets at the image level; furthermore, the luma and chroma components can select different preset indices at the block level; and when deriving the filtering intensity of the chroma component, the chroma block size is used to determine the input of the chroma component.

[0160] like Figure 12 As shown, a method is proposed that when the luminance and chrominance components have different partitions or semi-decoupled partitions, CDEF filtering is performed separately for the luminance and chrominance components. The input to the CDEF filtering is the reconstructed samples of the luminance / chrominance components. The intermediate outputs of this process include, but are not limited to, the derived filter presets and per-block preset indices mentioned in the proposed method. The final output of this process is the filtered reconstructed samples of the luminance / chrominance components.

[0161] In one implementation, the number of derived presets for the luminance and chrominance components may differ at the image level. The input to the CDEF filtering process is reconstructed samples from the luminance / chrominance components. The output of this process is the derived preset at the image level. Examples of the number of presets at the image level include, but are not limited to, 1, 2, 4, and 8.

[0162] In one example, the number of presets derived and selected for the current luminance component in a frame is 2, and the number of presets derived and selected for the current chrominance component in that frame is 1.

[0163] In another example, the number of derived and selected presets for the luminance component is N, where N is a positive integer, such as 1, 2, 4, or 8, while the number of presets for the chrominance component is fixed at 1. It is not necessary to signal the number of presets for the chrominance component in the bitstream, and the number of presets for the chrominance component is derived as 1 in the decoder.

[0164] For example, Figure 10 Independently constrained directional enhancement filter (SCDEF) is shown.

[0165] In one implementation, the selected preset indices for the current luma block and chroma block can be different from each other. The input to the process is the luma / chroma reconstruction sample of the current block, and the presets derived and selected at the frame level. The output of the process is an index indicating which preset to select for the current block.

[0166] In one example, when the luma component has 8 presets and the chroma component has 4 presets at the frame level, the selected preset index for luma block A is 7, and the selected preset index for chroma block B is 1. Luma block A and chroma block B are located in the same position or partially in the same position.

[0167] In one implementation, when deriving the CDEF filter intensity of the chroma component, the input reconstructed sample is determined by the current chroma coding block size.

[0168] In one example, when the current chroma block size is 32×64, the input is the chroma reconstruction sample value of the current 32×64 block.

[0169] In one implementation, when separate or semi-decoupled partitioning is applied to the luma and chroma blocks, the luma and chroma blocks still share the same preset index, and the luma (or chroma) block size is only used in the preset index export / signaling processing.

[0170] In some implementations, when the luminance and chrominance components have the same coding block size, CDEF filtering is performed on the luminance and chrominance components respectively.

[0171] In some implementations, SCDEF signaling is performed on the luminance component and the chrominance component respectively.

[0172] In one implementation, image-level presets are notified separately for the luminance and chrominance components using signaling. These presets can be notified via signaling in advanced parameter sets (DPS, VPS, SPS, PPS, APS), slice headers, image headers, and SEI messages.

[0173] In one example, the luminance preset is notified first using signaling, and then the chrominance preset is notified using signaling.

[0174] In one implementation, signaling is used to notify the block-level preset index for the luminance component and the chrominance component respectively.

[0175] In one example, the preset index of the luminance component is first notified by signaling, and then the preset index of the chrominance component is notified by signaling.

[0176] Now refer to Figure 11 The diagram illustrates the steps of a method 300 for decoding video data. However, those skilled in the art will understand that the encoding process is based on... Figure 11 How it works. In some implementations, one or more processing blocks of Figure 3 can be processed by computer 102 ( Figure 1 ) and server computer 114 ( Figure 1 ) Execution. In some implementations, one or more processing blocks of Figure 3 may be executed by another device or a group of devices that are separate from or include computer 102 and server computer 114.

[0177] At 302, method 300 includes receiving video data including chroma and luminance components.

[0178] At 304, method 300 includes resolving, exporting, or selecting a preset number of chroma components and a preset number of luminance components in a frame.

[0179] At 306, method 300 includes encoding and / or decoding the video data.

[0180] Operation 306 can be based on a preset number of chroma components and a preset number of luminance components in a frame.

[0181] The method may further include performing independent constrained directional enhancement filter (CDEF) processing that filters the luminance and chrominance components independently based on a preset number of chrominance components in a frame and a preset number of luminance components in the same frame.

[0182] Understandable. Figure 11 This is merely an illustration of one implementation and does not imply any limitations on how different implementations can be achieved. Many modifications to the depicted environment are possible based on design and implementation requirements.

[0183] Figure 12 According to the illustrative implementation method Figure 1 Block diagram 400 depicts the internal and external components of the computer. It should be understood that... Figure 12This is merely an illustration of an implementation and does not imply any limitations regarding the environments in which different implementations can be achieved. Many modifications to the depicted environment are possible based on design and implementation requirements.

[0184] Computer 102 ( Figure 1 ) and server computer 114 ( Figure 1 ) can be included in Figure 12 The internal components 800A, 800B and the corresponding groups of external components 900A, 900B are shown in the figure. Each group of internal components 800 includes one or more processors 820 on one or more buses 826, one or more computer-readable RAMs 822 and one or more computer-readable ROMs 824, one or more operating systems 828 and one or more computer-readable tangible storage devices 830.

[0185] Processor 820 is implemented in hardware, firmware, or a combination of hardware and software. Processor 820 is a central processing unit (CPU), graphics processing unit (GPU), accelerated processing unit (APU), microprocessor, microcontroller, digital signal processor (DSP), field-programmable gate array (FPGA), application-specific integrated circuit (ASIC), or other types of processing units. In some implementations, processor 820 includes one or more processors that can be programmed to perform functions. Bus 826 includes components that allow communication between internal components 800A and 800B.

[0186] One or more operating systems 828, software programs 108 ( Figure 1 ) and server computer 114 ( Figure 1 The video encoding program 116 on ) Figure 1 The data are stored on one or more of the respective computer-readable tangible storage devices 830 for execution by one or more of the respective processors 820 via one or more of the respective RAMs 822 (which typically include cache memory). Figure 12In the embodiments shown, each of the computer-readable tangible storage devices 830 is a disk storage device of an internal hard disk drive. Alternatively, each of the computer-readable tangible storage devices 830 is a semiconductor storage device, such as ROM 824, EPROM, flash memory, optical disc, magneto-optical disc, solid-state disk, compact disc (CD), digital versatile disc (DVD), floppy disk, cartridge, magnetic tape, and / or another type of non-transitory computer-readable tangible storage device that can store computer programs and digital information.

[0187] Each set of internal components 800A, 800B also includes an R / W drive or interface 832 for reading from and writing to one or more portable computer-readable tangible storage devices 936, such as CD-ROM, DVD, Memory Stick, magnetic tape, disk, optical disc, or semiconductor storage devices. Software programs, such as software program 108 ( Figure 1 ) and video encoding program 116 ( Figure 1 It can be stored on one or more of the corresponding portable computer-readable tangible storage devices 936, read from and loaded into the corresponding hard disk drive 830 via the corresponding R / W drive or interface 832.

[0188] Each set of internal components 800A, 800B also includes a network adapter or interface 836, such as a TCP / IP adapter card; a wireless Wi-Fi interface card; or a 3G, 4G, or 5G wireless interface card or other wired or wireless communication links. Software program 108 ( Figure 1 ) and server computer 114 ( Figure 1 The video encoding program 116 on ) Figure 1 ) can be downloaded from an external computer to computer 102 via a network (e.g., the Internet, a local area network, or other, a wide area network) and a corresponding network adapter or interface 836. Figure 1 The network includes a network adapter or interface 836 and a server computer 114. From the network adapter or interface 836, software program 108 and video encoding program 116 on the server computer 114 are loaded into the corresponding hard disk drive 830. The network may include copper wire, fiber optic, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers.

[0189] Each group of external components 900A and 900B may include a computer monitor 920, a keyboard 930, and a computer mouse 934. External components 900A and 900B may also include a touchscreen, a virtual keyboard, a touchpad, a pointing device, and other human-machine interface devices. Each group of internal components 800A and 800B also includes a device driver 840 for interfacing with the computer monitor 920, keyboard 930, and computer mouse 934. Device driver 840, R / W driver or interface 832, and network adapter or interface 836 include hardware and software (stored in storage device 830 and / or ROM 824).

[0190] It is understood in advance that although this disclosure includes a detailed description of cloud computing, the implementation of the teachings described herein is not limited to cloud computing environments. Rather, some implementations can be implemented in conjunction with any other type of computing environment now known or developed hereafter.

[0191] Cloud computing is a service delivery model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, storage, storage devices, applications, virtual machines, and services) that can be rapidly provisioned and deployed with minimal management effort or interaction with service providers. This cloud model may include at least five features, at least three service models, and at least four deployment models.

[0192] The features are as follows:

[0193] On-demand self-service: Cloud consumers can unilaterally and automatically supply computing power, such as server time and network storage, as needed, without requiring human interaction with the service provider.

[0194] Extensive network access: Capabilities are available via the network and are accessed through standard mechanisms that facilitate use by heterogeneous thin-client or thick-client platforms, such as mobile phones, laptops, and PDAs.

[0195] Resource pooling: A provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, where different physical and virtual resources are dynamically allocated and reallocated based on demand. There is a sense of location independence because consumers typically do not control or know the exact location of the resources provided, but can specify the location at a higher level of abstraction (e.g., country, state, or data center).

[0196] Rapid elasticity: Capacity can be supplied quickly and elastically (in some cases automatically) to rapidly scale out horizontally, and capacity can be rapidly deployed to rapidly scale in vertically. For consumers, the capacity available for supply often appears unlimited and can be purchased at any time and in any quantity.

[0197] Measurement services: Cloud systems automatically control and optimize resource usage by leveraging metering capabilities at some level of abstraction appropriate to service types (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be managed, controlled, and reported, providing transparency for both service providers and consumers.

[0198] The service model is as follows:

[0199] Software as a Service (SaaS): This provides consumers with the ability to use the provider's applications running on cloud infrastructure. Applications can be accessed from various client devices through a thin client interface such as a web browser (e.g., web-based email). Consumers do not manage or control the underlying cloud infrastructure, including the network, servers, operating system, storage devices, or even individual application capabilities, with possible exceptions such as limited user-specific application configuration settings.

[0200] Platform as a Service (PaaS): This provides consumers with the ability to deploy consumer-created or acquired applications, built using programming languages ​​and tools supported by the provider, onto cloud infrastructure. Consumers do not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, or storage devices, but they have control over the deployed applications and the configuration of any application hosting environments.

[0201] Infrastructure as a Service (IaaS): This provides consumers with the capability to supply processing, storage, networking, and other basic computing resources that enable them to deploy and run arbitrary software, which may include operating systems and applications. Consumers do not manage or control the underlying cloud infrastructure, but they do have control over the operating system, storage, deployed applications, and possibly limited control over the selection of networking components (e.g., host firewalls).

[0202] The deployment model is as follows:

[0203] Private cloud: Cloud infrastructure operated solely by an organization. It can be managed by the organization or a third party and can exist on-site or off-site.

[0204] Community cloud: A cloud infrastructure shared by several organizations and supporting a specific community with shared concerns (e.g., tasks, security requirements, policies, and compliance considerations). It can be managed by an organization or a third party and can exist on-premises or off-premises.

[0205] Public cloud: Cloud infrastructure available to the general public or large industry groups and owned by the organization that sells cloud services.

[0206] Hybrid cloud: A cloud infrastructure is a combination of two or more clouds (private, community, or public) that remain a single entity but are bound together by standardized or proprietary technologies that enable data and application portability (e.g., cloud bursts for load balancing between clouds).

[0207] Cloud computing environments are service-oriented, focusing on statelessness, loose coupling, modularity, and semantic interoperability. At the heart of cloud computing is the infrastructure of a network of interconnected nodes.

[0208] Reference Figure 13 The illustration depicts a cloud computing environment 500. As shown, the cloud computing environment 500 includes one or more cloud computing nodes 10, with local computing devices used by cloud consumers, such as personal digital assistants (PDAs) or cellular phones 54A, desktop computers 54B, laptop computers 54C, and / or automotive computer systems 54N, capable of communicating with the one or more cloud computing nodes 10. The cloud computing nodes 10 can communicate with each other. They can be physically or virtually grouped (not shown) in one or more networks, such as private clouds, community clouds, public clouds, or hybrid clouds, or combinations thereof, as described above. This allows the cloud computing environment 600 to provide infrastructure, platform, and / or software as a service, without requiring cloud consumers to maintain resources on their local computing devices. It should be understood that... Figure 13 The types of computing devices 54A to 54N shown are intended to be illustrative only, and cloud computing node 10 and cloud computing environment 500 can communicate with any type of computerized device via any type of network and / or network-addressable connection (e.g., using a web browser).

[0209] Reference Figure 14 This demonstrates the 500-fold cloud computing environment ( Figure 13 This provides a set of functional abstraction layers 600. It should be understood beforehand that... Figure 6 The components, layers, and functions shown are intended to be illustrative only, and the implementation is not limited thereto. As depicted, the following layers and corresponding functions are provided:

[0210] The hardware and software layer 60 includes hardware components and software components. Examples of hardware components include: a mainframe 61; a server 62 based on a RISC (Reduced Instruction Set Computer) architecture; a server 63; a blade server 64; a storage device 65; and a network and network components 66. In some implementations, software components include network application server software 67 and database software 68.

[0211] The virtualization layer 70 provides an abstraction layer from which the following examples of virtual entities can be provided: virtual server 71; virtual storage device 72; virtual network including virtual private network 73; virtual application and operating system 74; and virtual client 75.

[0212] In one example, management layer 80 may provide the following functionalities: Resource Provisioning 81 provides dynamic procurement of computing resources and other resources used to perform tasks within the cloud computing environment. Metering and Pricing 82 provides cost queries for utilizing resources within the cloud computing environment, as well as billing or invoicing for the consumption of these resources. In one example, these resources may include application software licenses. Security provides authentication for cloud consumers and tasks, and protection for data and other resources. User Portal 83 provides access to the cloud computing environment for consumers and system administrators. Service Level Management 84 provides cloud resource allocation and management to meet the required service level. Service Level Agreement (SLA) Planning and Fulfillment 85 provides pre-scheduling and procurement of cloud resources, anticipating future demand for those resources according to the SLA.

[0213] Workload tier 90 provides examples of functionalities that can be leveraged in a cloud computing environment. Examples of workloads and functionalities that can be provided from this tier include: mapping and navigation 91; software development and lifecycle management 92; virtual classroom delivery 93; data analytics and processing 94; transaction processing 95; and video encoding / decoding 96.

[0214] Some implementations may involve systems, methods, and / or computer-readable media at any possible level of technical detail integration. A computer-readable medium may include a computer-readable non-transitory storage medium (or multiple media) having computer-readable program instructions on it for causing a processor to perform operations.

[0215] Computer-readable storage media can be tangible devices that can hold and store instructions used by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. A non-exhaustive list of further specific examples of computer-readable storage media includes the following: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compactdisc read-only memory (CD-ROM), digital versatile disk (DVD), memory sticks, floppy disks, mechanical encoding devices such as punch cards or raised structures in recesses on which instructions are recorded, and any suitable combination of the foregoing. As used herein, computer-readable storage media should not be construed as being transient signals, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses passing through fiber optic cables), or electrical signals transmitted through leads.

[0216] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a suitable computing / processing device, or downloaded via a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network) to an external computer or external storage device. This network may include copper cables, optical fibers, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to a computer-readable storage medium within the suitable computing / processing device.

[0217] Computer-readable program code / instructions used to perform operations can be assembly instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, integrated circuit configuration data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and procedural programming languages ​​such as "C" or similar programming languages. The computer-readable program instructions can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of execution entirely on a remote computer or server, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., through the use of the Internet provided by an Internet service provider). In some implementations, electronic circuits, including, for example, programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), can be personalized by executing computer-readable program instructions using state information of computer-readable program instructions in order to perform aspects or operations.

[0218] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / actions specified in one or more blocks of a flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that can specifically direct a computer, programmable data processing apparatus, and / or other device to function, such that the computer-readable storage medium storing the instructions includes an article of writing comprising instructions for implementing aspects of the functions / actions specified in one or more blocks of a flowchart and / or block diagram.

[0219] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer-implemented process, such that the instructions, which execute on the computer, other programmable apparatus or other device, implement the function / action specified in one or more boxes of a flowchart and / or block diagram.

[0220] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer-readable media according to various embodiments. In this regard, each box in the flowchart or block diagram may represent a module, segment, or portion of instructions comprising one or more executable instructions for implementing a specified logical function. Methods, computer systems, and computer-readable media may include additional boxes, fewer boxes, different boxes, or boxes arranged differently compared to those depicted in the figures. In some alternative implementations, the functions indicated in the boxes may not occur in the order shown in the figures. For example, two boxes shown consecutively may actually be executed simultaneously or substantially simultaneously, or the boxes may sometimes be executed in reverse order depending on the functions involved. It will also be noted that each box in the block diagrams and / or flowcharts, as well as combinations of boxes in the block diagrams and / or flowcharts, can be implemented by a system based on dedicated hardware that performs the specified function or action, or by implementing a combination of dedicated hardware and computer instructions.

[0221] It will be apparent that the systems and / or methods described herein can be implemented in various forms of hardware, firmware, or combinations of hardware and software. The actual dedicated control hardware or software code used to implement these systems and / or methods does not limit the implementation. Therefore, the operation and behavior of the systems and / or methods are described herein without reference to specific software code—it should be understood that software and hardware can be designed to implement the systems and / or methods based on the descriptions herein.

[0222] Unless explicitly stated otherwise, no element, action, or instruction used herein should be construed as critical or necessary. Furthermore, as used herein, the articles “a” and “an” are intended to include one or more items and may be used interchangeably with “one or more.” Additionally, as used herein, the term “group” is intended to include one or more items (e.g., related items, unrelated items, combinations of related and unrelated items, etc.) and may be used interchangeably with “one or more.” The term “a” or similar language is used where only one item is intended. Moreover, as used herein, the terms “having,” “possessing,” “containing,” etc., are intended to be open-ended terms. Furthermore, unless explicitly stated otherwise, the phrase “based on” is intended to mean “at least partially based on.”

[0223] Descriptions of various aspects and embodiments have been presented for illustrative purposes; however, these descriptions are not intended to be exhaustive or limited to the disclosed embodiments. Although combinations of features are recited in the claims and / or disclosed in the specification, these combinations are not intended to limit the disclosure of possible implementations. In fact, many of these features can be combined in ways not specifically recited in the claims and / or not specifically disclosed in the specification. Although each dependent claim listed below may directly refer to only one claim, the disclosure of possible implementations includes every dependent claim combined with every other claim in the group of claims. Many modifications and variations will be apparent to those skilled in the art without departing from the scope of the described embodiments. The terminology used herein has been chosen to best explain the principles of the embodiments, their practical application, or technical improvements relative to technologies found in the market, or to enable those skilled in the art to understand the embodiments disclosed herein.

[0224] The acronyms used throughout the public content include the following:

[0225] HEVC: High Efficiency Video Coding

[0226] HDR: High Dynamic Range

[0227] SDR: Standard Dynamic Range

[0228] VVC: Versatile Video Coding

[0229] JVET: Joint Video Exploration Team

[0230] MPM: Most Probable Mode

[0231] WAIP: Wide-Angle Intra Prediction

[0232] CU: Coding Unit

[0233] CTB: Coding Tree Block

[0234] PU: Prediction Unit

[0235] TU: Transform Unit

[0236] CTU: Coding Tree Unit

[0237] PDPC: Position Dependent Prediction Combination

[0238] ISP: Intra Sub-Partitions

[0239] SPS: Sequence Parameter Setting

[0240] PPS: Picture Parameter Set

[0241] APS: Adaptive Parameter Set

[0242] VPS: Video Parameter Set

[0243] DPS: Decoding Parameter Set

[0244] ALF: Adaptive Loop Filter

[0245] SAO: Sample Adaptive Offset

[0246] CC-ALF: Cross-Component Adaptive Loop Filter

[0247] CDEF: Constrained Directional Enhancement Filter

[0248] LR: Loop Restoration Filter

[0249] AV1: AOMedia Video 1

[0250] AV2: AOMedia Video 2

[0251] SDP: Semi-Decoupled Partitioning

[0252] SEI: Supplementary Enhancement Information.

Claims

1. A video decoding method, characterized in that, The method includes: Receive a video bitstream comprising multiple frames, the video bitstream including a current frame having chroma and luminance components; and In the case where the semi-decoupled partitioning SDP mode is enabled in the current frame: One or more parameters processed by the first constrained directional enhancement filter (CDEF) are parsed from the video bitstream; The first CDEF processing is applied to the chroma components of the current frame according to one or more parameters; Parse one or more second parameters of the second CDEF process from the video bitstream; The second CDEF processing is applied to the luminance component of the current frame according to one or more second parameters, wherein the second CDEF processing is separate from the first CDEF processing; and The current frame is reconstructed based on the first CDEF process and the second CDEF process.

2. The method according to claim 1, characterized in that, The first CDEF process and the second CDEF process are selected based on the different divisions of the chromaticity component and the luminance component.

3. The method according to claim 1, characterized in that, The input to the first CDEF process includes one or more reconstructed samples of the chromaticity component, and the input to the second CDEF process includes one or more reconstructed samples of the luminance component.

4. The method according to claim 3, characterized in that, Also includes: The filtering intensity of the second CDEF process is derived based on the magnitude of the luminance component.

5. The method according to claim 1, characterized in that, The first CDEF process and the second CDEF process are selected based on the magnitude of the luminance component or the magnitude of the chrominance component.

6. The method according to claim 1, characterized in that, The output of the first CDEF processing is a set of filtered chroma samples, and the output of the second CDEF processing is a set of filtered luminance samples.

7. The method according to claim 1, characterized in that, Also includes: Before applying the first CDEF processing and the second CDEF processing, deblocking processing or loop recovery processing is applied to the current frame.

8. The method according to claim 1, characterized in that, The luminance component and the chrominance component have the same block size.

9. A video encoding method, characterized in that, The method includes: Receive video data comprising multiple frames, the video data including the current frame having chroma and luminance components; and In the case where the semi-decoupled partitioning SDP mode is enabled in the current frame: A first constrained directional enhancement filter (CDEF) is selected for the chroma components of the current frame for processing; A second CDEF process is selected for the luminance component of the current frame, wherein the second CDEF process is different from the first CDEF process; Send one or more parameters processed by the first CDEF in the video bitstream; and One or more second parameters processed by the second CDEF processing are transmitted in the video bitstream.

10. The method according to claim 9, characterized in that, The first CDEF process and the second CDEF process are selected based on the different divisions of the chromaticity component and the luminance component.

11. The method according to claim 9, characterized in that, The input to the first CDEF process includes one or more samples of the chromaticity component, and the input to the second CDEF process includes one or more samples of the luminance component.

12. The method according to claim 9, characterized in that, Also includes: The filtering intensity of the second CDEF process is derived based on the magnitude of the luminance component.

13. The method according to claim 9, characterized in that, The first CDEF process and the second CDEF process are selected based on the magnitude of the luminance component or the magnitude of the chrominance component.

14. The method according to claim 9, characterized in that, The luminance component and the chrominance component have the same block size.

15. The method according to claim 9, characterized in that, The output of the first CDEF processing is a set of filtered chroma samples, and the output of the second CDEF processing is a set of filtered luminance samples.

16. A method for storing video bitstreams, characterized in that, include: The video bitstream is generated by performing the video encoding method as described in any one of claims 9-15; as well as Store the video bitstream.

17. A non-transitory computer-readable medium, characterized in that, The non-transitory computer-readable medium stores a computer program, characterized in that the computer program is configured to cause one or more processors to perform the method as described in any one of claims 1-15.

18. A video decoding apparatus, characterized in that, The apparatus includes a memory and one or more processors, the processors being configured to perform the method as described in any one of claims 1-8.

19. A video encoding apparatus, characterized in that, The apparatus includes a memory and one or more processors, the processors being configured to perform the method as described in any one of claims 9-15.

20. An electronic device, characterized in that, The electronic device includes: A processing circuit system configured to perform the method as described in any one of claims 1 to 15.

Images