Data processing method, apparatus, computer program and computer device

Abstract

An embodiment of the present application provides a data processing method, an apparatus, and a device. The method includes the steps of: identifying classification mode information corresponding to a block to be coded in video data, the classification mode information including an extended co-located luminance reconstructed pixel corresponding to a color component pixel in the block to be coded and a target classification mode, the extended co-located luminance reconstructed pixel belonging to a target region centered on a realistic co-located luminance reconstructed pixel of the color component pixel; identifying a boundary category corresponding to the color component pixel according to the extended co-located luminance reconstructed pixel and the target classification mode; compensating the reconstructed pixel of the color component pixel based on the boundary category, obtaining a compensated reconstructed pixel, and performing an encoding process on the block to be coded based on the compensated reconstructed pixel. According to the embodiment of the present application, it is possible to improve the category accuracy of the boundary compensation corresponding to the color component pixel, and further improve the overall coding performance of the video data.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of priority to Chinese Patent Application No. 2022108492507, entitled "Data Processing Method, Apparatus and Device," filed with the State Intellectual Property Office of the People's Republic of China on July 19, 2022, the entire contents of which are incorporated herein by reference.

[0002] Technical Field The present application relates to the technical field of data processing, and in particular to video encoding technology. [Background technology]

[0003] In order to transmit video data, it is usually necessary to encode and compress the video data, which may be called a video code stream, and the video code stream is transmitted to a user terminal via a wired or wireless network, decoded, and then viewed. The general video encoding process may include processes such as block division, prediction, transformation, quantization, and encoding. In the video encoding stage, after the entire frame image is reconstructed, the pixel values ​​in the reconstructed image are filtered or compensated to adjust the pixel values ​​in the reconstructed image, thereby further improving the image quality. However, in the existing video encoding compensation method, the position of the same-position luminance component used in classification is fixed and invariant, and as a result, the category accuracy is too low, which may further affect the overall encoding performance of the video data. Summary of the Invention [Means for solving the problem]

[0004] The embodiments of the present application provide a data processing method, apparatus and device, which can improve the category accuracy of boundary compensation corresponding to color component pixels, and further improve the overall coding performance of video data.

[0005] According to one aspect of an embodiment of the present application, there is provided a data processing method performed by a computing device, comprising the steps of: identifying classification mode information corresponding to a block to be coded in the video data, the classification mode information including an extended co-located luminance reconstruction pixel corresponding to a color component pixel in the block to be coded and a target classification mode, the extended co-located luminance reconstruction pixel belonging to a target region centered on a realistic co-located luminance reconstruction pixel of the color component pixel; Identifying a boundary category corresponding to a color component pixel according to the expanded co-located luminance reconstruction pixel and a target classification mode; The present invention provides a data processing method including a step of compensating reconstructed pixels of color component pixels based on a boundary category, obtaining compensated reconstructed pixels, and performing an encoding process on a block to be encoded based on the compensated reconstructed pixels.

[0006] According to one aspect of an embodiment of the present application, there is provided a data processing method performed by a computing device, comprising the steps of: performing a decoding process on a block to be decoded in the video data to obtain classification mode information corresponding to the block to be decoded, the classification mode information including an extended co-located luminance reconstructed pixel corresponding to a color component pixel in the block to be decoded and a target classification mode, the extended co-located luminance reconstructed pixel belonging to a target area centered on a realistic co-located luminance reconstructed pixel of the color component pixel; Identifying a boundary category corresponding to a color component pixel according to the expanded co-located luminance reconstruction pixel and a target classification mode; reconstructing color component pixels in the block to be decoded according to compensation values ​​corresponding to the boundary categories.

[0007] According to one aspect of an embodiment of the present application, the following modules: a classification mode information identification module configured to identify classification mode information corresponding to a block to be coded in the video data, the classification mode information including an extended co-located luminance reconstruction pixel corresponding to a color component pixel in the block to be coded and a target classification mode, the extended co-located luminance reconstruction pixel belonging to a target region centered on a realistic co-located luminance reconstruction pixel of the color component pixel; a first category identification module configured to identify a boundary category corresponding to a color component pixel according to the expanded co-located luminance reconstruction pixel and a target classification mode; and an encoding processing module configured to compensate reconstructed pixels of color component pixels based on a boundary category, obtain compensated reconstructed pixels, and perform encoding processing on a block to be encoded based on the compensated reconstructed pixels.

[0008] According to one aspect of an embodiment of the present application, the following modules: a decoding processing module configured to perform a decoding process on a block to be decoded in video data and obtain classification mode information corresponding to the block to be decoded, the classification mode information including an extended co-located luminance reconstruction pixel corresponding to a color component pixel in the block to be decoded and a target classification mode, the extended co-located luminance reconstruction pixel belonging to a target region centered on a realistic co-located luminance reconstruction pixel of the color component pixel; a second category identification module configured to identify a boundary category corresponding to the color component pixel according to the expanded co-located luminance reconstruction pixel and the target classification mode; a pixel reconstruction module configured to reconstruct color component pixels in the block to be decoded according to a compensation value corresponding to a boundary category.

[0009] According to one aspect of an embodiment of the present application, there is provided a computing device including a memory for storing a computer program, and a processor connected to the memory, the processor causing, when the computer program is invoked, to execute a method according to the aspect of the embodiment of the present application.

[0010] According to one aspect of an embodiment of the present application, there is provided a computer-readable storage medium having stored thereon a computer program suitable, when loaded and executed by a processor, to cause a computing device having a processor to perform a method according to said one aspect of an embodiment of the present application.

[0011] According to one aspect of an embodiment of the present application, there is provided a computer program product or computer program comprising computer instructions stored on a computer readable storage medium, the computer instructions being read by a processor of a computing device from the computer readable storage medium and causing the processor to execute the computer instructions, thereby causing the computing device to perform a method according to said aspect.

[0012] In an embodiment of the present application, when performing an encoding process on video data, classification mode information corresponding to a block to be encoded in the video data can be identified, and the classification mode information includes an extended same-position luminance reconstructed pixel corresponding to a color component pixel in the block to be encoded and a target classification mode, and the extended same-position luminance reconstructed pixel belongs to a target area centered on a realistic same-position luminance reconstructed pixel of the color component pixel. Then, a boundary category corresponding to the color component pixel can be identified according to the extended same-position luminance reconstructed pixel and the target classification mode. The reconstructed pixel of the color component pixel is compensated based on the boundary category, the compensated reconstructed pixel is obtained, and an encoding process is performed on the block to be encoded based on the compensated reconstructed pixel. In this way, in the encoding stage of the video data, the extended same-position luminance reconstructed pixel can be introduced to perform category classification, so that it is possible to improve the category accuracy of the boundary compensation corresponding to the color component pixel and further improve the overall encoding performance of the video data. [Brief description of the drawings]

[0013] [Figure 1] FIG. 2 is a schematic diagram of a structure of an encoding frame according to an embodiment of the present application; [Diagram 2] 4 is a flow chart of loop filtering according to an embodiment of the present application. [Diagram 3] 1 is a basic flow chart of cross-component sample adaptive offset compensation according to an embodiment of the present application. [Figure 4] FIG. 13 is a schematic diagram illustrating the location of band category classification samples in cross-component sample adaptive offset compensation according to an embodiment of the present application. [Diagram 5] FIG. 13 is a schematic diagram of a border category classification mode in cross-component sample adaptive offset compensation according to an embodiment of the present application; [Figure 6] FIG. 2 is a frame schematic diagram of cross-component adaptive loop filtering according to an embodiment of the present application; [Figure 7] FIG. 2 is a schematic diagram of a cross-component adaptive loop filtering filter geometry according to an embodiment of the present application; [Figure 8] 1 is a schematic diagram illustrating the configuration of a data processing system according to an embodiment of the present application. [Figure 9] 1 is a flowchart of a data processing method according to an embodiment of the present application. [Figure 10] FIG. 2 is a schematic diagram of a selection region of extended co-located luminance reconstruction pixels according to an embodiment of the present application; [Figure 11] FIG. 2 is a schematic diagram of an advanced classification mode according to an embodiment of the present application; [Figure 12] FIG. 1 is a schematic diagram of the mode coverage of the extended co-located luminance reconstruction pixels according to an embodiment of the present application; [Figure 13] FIG. 1 is a schematic diagram of the locations of extended co-located luminance reconstruction pixels and corresponding classification modes according to an embodiment of the present application; [Figure 14] FIG. 1 is a schematic diagram for filling unavailable areas of extended co-located luminance reconstruction pixels with coverage according to an embodiment of the present application; [Figure 15] 4 is a flowchart of another data processing method according to an embodiment of the present application. [Figure 16] 1 is a schematic diagram illustrating a configuration of a data processing device according to an embodiment of the present application. [Figure 17] FIG. 2 is a schematic diagram illustrating the configuration of another data processing device according to an embodiment of the present application. [Figure 18] FIG. 1 is a schematic diagram of a configuration of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, the technical methods according to the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application, but it is clear that the embodiments described herein are only some embodiments of the present application, and are not all embodiments. Based on the embodiments of the present application, other embodiments that a person skilled in the art can obtain without labor equivalent to an inventive step shall all be included in the technical scope of the present application.

[0015] In order to more clearly understand the technical approaches according to the embodiments of the present application, here we first introduce some basic concepts referred to in the embodiments of the present application.

[0016] Video coding: Video coding means encoding pixels in an image frame of video data to obtain a coded bitstream (also called video codestream), and may also refer to converting a file in one video format to another video format file by a certain compression technique. An embodiment of the present application is a technical method proposed based on Enhanced Compression Model (ECM) used in existing video coding techniques, in which ECM improves the loop filtering part of Versatile Video Coding (VVC), i.e., in addition to the existing loop filter in VVC, multiple types of loop filters are further introduced. The following describes the coding frame using ECM and the loop filtering process of ECM.

[0017] Referring to Fig. 1, Fig. 1 is a schematic diagram of a coded frame according to an embodiment of the present application. The coded frame shown in Fig. 1 is a block-based hybrid coded frame, in which one image frame (e.g., image frame 10a) in video data is divided into a plurality of blocks (e.g., coding tree blocks (CTBs) or coding tree units (CTUs)), and coding processes such as prediction, transformation, quantization, loop filtering, and entropy coding are performed on a block-by-block basis, and each block has a corresponding coding parameter.

[0018] Prediction means that the current signal (i.e., the object to be coded, such as an image frame, block, or pixel to be coded) is not directly coded, but the current signal is predicted using one or more previous signals, and coding is performed based on the difference between the actual value and the predicted value to achieve the purpose of coding. Here, prediction may include intra prediction and inter prediction.

[0019] Intra prediction refers to predicting pixel values ​​within a current block to be coded based on neighboring pixels that have already been coded, so as to achieve the purpose of effectively removing spatial redundancy in video data.

[0020] Inter prediction means predicting pixels of a current image frame to be coded based on pixels of adjacent coded image frames by utilizing the temporal correlation of video data in order to achieve the purpose of removing temporal redundancy of video data. Here, the inter prediction process mainly involves motion estimation and motion compensation. Motion estimation means finding a reference block that best matches a current block to be coded from a reference image frame (coded image frame), and using a motion vector to represent the positional relationship between the best-matched reference block and the current block to be coded (a block of the image frame to be coded that needs to be coded, i.e., a block that has not yet been coded). Motion compensation means coding and transmitting a difference value between the best-matched reference block and the current block to be coded.

[0021] The transformation means orthogonally transforming the image frame to be coded in the image data to remove correlation between spatial pixels. Through the orthogonal transformation, the energy previously distributed in each pixel is concentrated in a small number of low-frequency bands of the frequency band, which represents most of the information of the image. Such characteristics of the frequency coefficients are advantageous to adopt a quantization method based on the human HVS (Human Visual System). Here, the transformation method may include, but is not limited to, the Karhunen-Loeve Transform (KL Transform), the Discrete Cosine Transform (DCT), and the Discrete Wavelet Transform (DWT).

[0022] Quantization refers to reducing the precision of the representation of video data. Quantization reduces the amount of data that needs to be encoded. Quantization is a lossy compression technique. There are two types of quantization: vector quantization and scalar quantization. Vector quantization is the joint quantization of a data set, while scalar quantization is the independent quantization of each input data.

[0023] Loop filtering can remove or reduce various types of coding artifacts that occur during the process of block-by-block coding, such as block noise, ringing effects, image content texture, or excessive boundary smoothing caused by discontinuity of block boundaries. Specifically, quantization of the image frame to be coded after transformation is a lossy process that may result in information loss of video data, and since there is a certain error between the block (sometimes called a reconstructed block) restored by inverse quantization (InvQuantization) and inverse transform (InvTransform) and the original block, the last restored image frame (reconstructed image frame) may be blocked, but the blocked image frame may have a significant impact on the prediction of the subsequent image frame, so the blocking phenomenon can be removed using a loop filtering method.

[0024] The Decoded Picture Buffer is used to store all the reconstructed image frames in the encoding process.

[0025] Entropy coding refers to removing statistically redundant information that still exists after prediction and transformation by compressing the bit rate using the information entropy of the source. Entropy coding can increase the video compression ratio, and since the entropy coding belongs to lossless compression, the original video data can be reconstructed without distortion at the decoding side using the video data compressed by entropy coding. Commonly used entropy coding methods may include, but are not limited to, variable length coding and content-based binary arithmetic coding (CABAC).

[0026] Here, variable length coding can express the difference value (i.e., the difference value between the reconstructed pixel value obtained by loop filtering and the original pixel value in the block to be coded, the original pixel value being any pixel value in the block to be coded) or coefficient that needs to be coded with code words of different lengths. The code word length needs to be designed according to the occurrence probability of the symbol, and for example, a short code word is assigned to a residual or coefficient with a high occurrence probability, and a long code word is assigned to one with a low occurrence probability. Exponential Golomb coding (exp-Golomb) and arithmetic coding are commonly used.

[0027] CABAC may include several steps: (1) Binarization: CABAC performs coding using binary operators, meaning that only two numbers (1 or 0) are coded. Non-binary numeric symbols such as transform coefficients and motion vectors are binarized, or converted into binary codewords, before arithmetic coding. This process is similar to converting a single number into a variable-length codeword, but this binary codeword is further coded through an arithmetic coder before being transmitted. (2) Context model selection: The context model is a probabilistic model that is selected based on the statistics of recently coded data symbols. This model stores the probability that each "bin" is a 1 or a 0. (3) Arithmetic coding: The arithmetic coder performs coding for each “bin” based on a selected probability model. (4) Probability Update: The selected context model is updated based on the actual encoded value. For example, if the value of “bin” is 1, the frequency counter of 1 is incremented.

[0028] In order to more clearly understand the loop filtering process in the coding frame shown in FIG. 1 above, the loop filtering process will now be described in detail with reference to FIG.

[0029] Referring to FIG. 2, FIG. 2 is a flow chart of loop filtering according to an embodiment of the present application. As shown in FIG. 2, the loop filtering process may include a number of modules, such as luma mapping with chroma scaling (LMCS), deblocking filter (DBF), bilateral filter (BIF), bilateral filter on chroma (BIF-Chroma), sample adaptive offset compensation (SAO), cross-component sample adaptive offset compensation (CCSAO), adaptive loop filtering (ALF), and cross-component adaptive loop filtering (CC-ALF). Here, the bilateral filter, chroma component bilateral filter, sample adaptive offset compensation, and cross-component sample adaptive offset compensation operate in parallel, and the compensation values ​​(correction values) generated thereby can be added to the deblocking filtered reconstructed pixels at the same time. Specifically, as follows:

[0030] (1) Luminance Mapping with Chroma Scaling (LMCS): LMCS is not targeted at a specific type of coding distortion, but rather aims to improve coding efficiency by setting sample values ​​to more suitable intervals.

[0031] (2) Deblocking Filtering (DBF): DBF is used to reduce distortion due to the coding process, specifically to reduce discontinuities at block-to-block boundaries due to block-based operations.

[0032] (3) Bilateral Filtering (BIF): BIF is a nonlinear filtering method, which is a smooth processing method that combines the spatial proximity of image frames and the similarity of pixel values, and is a method that achieves the purpose of preserving the boundaries between blocks and removing noise by considering the spatial information and gray level similarity of the reconstructed pixels that need to be filtered. Here, the reconstructed pixels in the embodiment of the present application refer to the result generated by reconstructing pixels in a block to be encoded in the encoding stage, and the reconstructed pixels may include luminance reconstructed pixels (e.g., Y component pixels) and chroma reconstructed pixels (e.g., U component pixels and V component pixels), where Y, U, and V refer to three color components in the YUV color space. Of course, the embodiment of the present application may adopt other color spaces such as YCbCr color space (Y is the luminance component, Cb is the blue chrominance component, and Cr is the red chrominance component) in addition to the YUV color space, but the color space actually used in the embodiment of the present application is not limited.

[0033] (4) Chroma Component Bilateral Filtering (BIF-Chroma): The main difference between BIF-Chroma and the BIF described above is that BIF performs bilateral filtering on all three color components of a reconstructed pixel, whereas BIF-Chroma performs bilateral filtering only on the chroma reconstructed pixel (i.e., the reconstructed value on the chroma component of the reconstructed pixel).

[0034] (5) Sample Adaptive Offset Compensation (SAO): SAO is a technique for adaptively compensating each pixel sample with a compensation value to reduce the difference between the pixel and the original pixel in the block to be coded by the quantization operation. Specifically, the input reconstructed pixel is classified into different categories, a corresponding compensation value (offset) is generated for each category, and the compensation value is added to the reconstructed pixel belonging to the corresponding category, thereby reducing the difference between the reconstructed pixel and the original pixel. However, when performing SAO classification, classification is performed using only the reconstructed value of the color component currently being processed. For example, when a reconstructed value on the chroma component of the reconstructed pixel (chroma reconstructed pixel) is input, SAO classifies only the input chroma reconstructed pixel.

[0035] (6) Cross-Component Sample Adaptive Offset Compensation (CCSAO): Similar to SAO, CCSAO classifies input reconstructed pixels into different categories, generates corresponding compensation values ​​for each category, and adds the compensation values ​​to reconstructed pixels belonging to the corresponding category, thereby achieving the purpose of reducing the difference between the reconstructed pixels and the original pixels. Note that CCSAO can classify any color component to be processed using the reconstructed values ​​of all three color components of the reconstructed pixel. For example, when a chroma reconstructed pixel is input to CCSAO, it can classify using all the chroma reconstructed pixels and luminance reconstructed pixels of the same pixel.

[0036] Now, refer to Fig. 3, which is a basic flow chart of cross-component sample adaptive offset compensation according to an embodiment of the present application. As shown in Fig. 3, a reconstructed pixel that has undergone deblocking filtering (DBF) may include pixel values ​​on three color components, and here, taking three color components in a YUV color space as an example, the pixel values ​​on the three color components may include a luminance reconstructed pixel (Y component pixel) and a chroma reconstructed pixel (U component pixel, V component pixel).

[0037] Here, when the Y component pixel on which DBF has been performed is input to the SAO, sample adaptive offset compensation can be performed on the Y component pixel on which DBF has been performed, thereby obtaining a compensation value 1 corresponding to the Y component pixel. At the same time, the Y component pixel on which DBF has been performed can be input to the CCSAO, and cross-component sample adaptive offset compensation can be performed on the Y component pixel on which DBF has been performed, thereby obtaining a compensation value 2 corresponding to the Y component pixel. Furthermore, the compensation value 1 output from the SAO and the compensation value 2 output from the CCSAO can be added to the Y component pixel on which DBF has been performed, thereby obtaining a compensated Y component pixel.

[0038] For ease of understanding, the embodiments of the present application will be described using only SAO and CCSAO as examples, but in actual applications, the compensation values ​​by BIF and BIF-Chroma can also be added to the compensated Y component pixel, that is, the deblocking filtered Y component pixel can be input to BIF and BIF-Chroma sequentially to obtain the corresponding compensation values. Similarly, the U component pixel and V component pixel that have been subjected to DBF can both perform the same operation as the Y component pixel described above to obtain the compensated U component pixel and the compensated V component pixel, which will not be described further here.

[0039] CCSAO can be understood as having two types of offset compensation: band offset compensation (BO) and boundary offset compensation (EO). The details are as follows:

[0040] (1) BO type of CCSAO: For the BO type, the CCSAO can directly classify based on the pixel value of the reconstructed pixel. Specifically, for any luminance or chrominance reconstructed pixel, classification can be performed using the corresponding reconstructed pixels of the three color components {co-located Y pixel, co-located U pixel, co-located V pixel}, where the co-located Y pixel, co-located U pixel, and co-located V pixel are understood to be the reconstructed pixels on the three color components at the position where the input reconstructed pixel is located. The reconstructed pixels for the above three classifications are first divided into their respective band categories {bandY, bandU,bandV}, and then a common category index is generated as the BO category of the currently input reconstructed pixel based on the band categories of the three color components.

[0041] A compensation value may be generated for each BO category, and the compensation value may be added to the input reconstructed pixel. Such a process of processing by the CCSAO BO may be expressed by the following formula (1).

number

[0042] For the same-position reconstructed pixels of the three color components for classification, the same-position chroma reconstructed pixels (U col ,V col) are realistic co-located chrominance reconstructed pixels (also called co-located chrominance components), and the co-located luminance reconstructed pixels (Y col ) can be selected from a region of size 3*3 centered on a realistic co-located luminance reconstruction pixel as shown in FIG.

[0043] Fig. 4 is a schematic diagram of the positions of band category classification samples in cross-component sample adaptive offset compensation according to an embodiment of the present application. Region 20d shown in Fig. 4 represents the realistic co-located luma reconstructed pixel at position 4 (i.e., the realistic co-located Y pixel), region 20a represents a target region with a size of 3*3 centered on the realistic co-located luma reconstructed pixel, and region 20b and region 20c both represent the co-located chroma reconstructed pixel at position 4, in which region 20b represents the co-located U pixel and region 20c represents the co-located V pixel.

[0044] For the CCSAO BO type, the corresponding parameters, i.e., the position of the co-located luminance reconstructed pixel, the total number of band categories corresponding to the three color components, and the compensation value for each BO category, need to be determined by a rate-distortion optimization (RDO) process and transferred to the decoding side. Here, rate-distortion optimization is performed for any possible co-located luminance reconstructed pixel positions in the area 20a shown in Fig. 4 to obtain the rate-distortion loss for each possible co-located luminance reconstructed pixel position, and the position of the co-located luminance reconstructed pixel with the smallest rate-distortion loss can be set as the position of the co-located luminance reconstructed pixel to be finally transferred to the decoding side.

[0045] (2) CSAO EO type: Similar to the sample adaptive offset compensation (SAO) process, the CCSAO can also use a boundary-based classification method. The existing CCSAO can support four different boundary offset compensation (EO) classification modes. Referring to FIG. 5, FIG. 5 is a schematic diagram of the classification modes of boundary categories in the cross-component sample adaptive offset compensation according to an embodiment of the present application. As shown in FIG. 5, the four different boundary offset compensation (EO) classification modes in the CCSAO are a horizontal classification mode shown in region 30a, a vertical classification mode shown in region 30b, a diagonal classification mode shown in region 30c, and an anti-diagonal classification mode shown in region 30d, respectively. For each EO classification mode, for either the input luma or chroma reconstruction pixel, it is necessary to first calculate the difference value between the corresponding same-position luma reconstruction pixel (position c shown in FIG. 5) and the adjacent pixels (positions a and b shown in FIG. 5), and compare the difference value with a predefined threshold (Th) to obtain the EO category of the currently processed reconstruction pixel. It should be understood that each classification mode specifies a positional relationship between a co-located luminance reconstruction pixel and two adjacent pixels (a and b), which calculate a difference value between the co-located luminance reconstruction pixel and the two adjacent pixels.

[0046] Here, the difference from SAO is that when classifying the CCSAO EO, the classification is performed using only the corresponding realistic luminance reconstruction pixels at the same position for the reconstruction values ​​of different input color components. The processing process by such a CCSAO EO may be expressed by the following formula (2).

number

number

[0047] For the CCSAO EO type, the encoding side needs to select the optimal classification mode from the four classification modes shown in Figure 5 by rate-distortion optimization (RDO). The optimal threshold value Th is selected from a set of predefined candidate thresholds, and the corresponding index is transferred to the decoding side. The encoding side also needs to select the reconstructed values ​​of the reconstructed pixels for generating the band categories from "cur", "col1", and "col2" by RDO, and transfer them to the decoding side. In other words, the rate-distortion optimization is performed for each of the four classification modes shown in Figure 5, and the rate-distortion loss corresponding to each classification mode is obtained, and the classification mode with the smallest rate-distortion loss can be the optimal classification mode to be transferred to the decoding side at the end. Similarly, the rate-distortion optimization is performed for "cur", "col1", and "col2" in Equation (3), and the color component with the smallest rate-distortion loss can be transferred to the decoding side as the reconstructed values ​​of the reconstructed pixels for generating the band categories.

[0048] For video data that needs to be coded, different CCSAO classifiers can be used for different video contents, and different classifiers can also be used for different positions in one image. The type and parameters of each classifier need to be explicitly transferred to the decoding side at the slice level. At the coding tree unit (CTU) level, it is necessary to indicate whether CCSAO is used for the current CTU, and if CCSAO is used, it is necessary to further indicate the selection of the corresponding classifier, among which CCSAO supports up to four sets of different classifiers per frame. Note that, if the rate-distortion loss when CCSAO is not used for a certain coding tree unit is smaller than the rate-distortion loss when CCSAO is used, it can be determined that CCSAO is not used for the coding tree unit. On the other hand, if the rate-distortion loss when CCSAO is not used is larger than the rate-distortion loss when CCSAO is used, it can be determined that CCSAO is used for the coding tree unit, and further classifier selection can be performed.

[0049] (7) Adaptive Loop Filtering (ALF): ALF is a Wiener filter that can adaptively identify filter coefficients according to the content of different video components to reduce the mean square error (MSE) between the reconstructed color components and the original color components. As an adaptive filter, the Wiener filter can add different filter coefficients to video contents with different characteristics, so ALF must first classify the video contents and apply corresponding filters to the video contents by category. The inputs to the ALF are the reconstructed pixel values ​​that have undergone DBF, BIF, BIF-Chroma, SAO, and CCSAO filtering, and the outputs are the extended reconstructed luma image and reconstructed chroma image. The ALF of the luma reconstruction pixel can support 25 different categories of filters, and the ALF of each chroma reconstruction pixel can support up to 8 different categories of filters.

[0050] Among them, for luminance reconstruction pixels, ALF adaptively uses different filters at the subblock level (e.g., the subblock level may be a luminance block of size 4*4), i.e., each pixel block of size 4*4 needs to be classified into one of the categories 25. The classification index C of a luminance pixel block is calculated by the directional feature D (Directionality) of the pixel block and the quantized activity feature (outside 1) This was jointly obtained from TIFF2025506036000005.tif14119 (Activity). Here, the classification index C may be expressed by the following formula (4).

number

number

number

number

number

[0051] Based on the pixel gradient values ​​represented by equations (5) to (8), the calculation of the global horizontal, vertical, diagonal and anti-diagonal gradient values ​​of each pixel block of size 4*4 may be shown in the following equations (9) and (10).

number

number

[0052] Furthermore, after the pixel gradient values ​​of the pixel blocks are obtained, the maximum value of the horizontal pixel gradient value and the vertical pixel gradient value of each pixel block is calculated as (Outside 3) TIFF2025506036000014.tif1263, and the minimum value (outside 4) It can also be written as TIFF2025506036000015.tif1263. The maximum value of the diagonal pixel gradient value and the anti-diagonal pixel gradient value of each pixel block is (outside 5) TIFF2025506036000016.tif1263, and the minimum value (outside 6) It can also be written as TIFF2025506036000017.tif1263.

[0053] The directional feature D is the maximum value of the horizontal pixel gradient value and the vertical pixel gradient value. (outer 7) TIFF2025506036000018.tif1263Min (outside 8) TIFF2025506036000019.tif1263 and the maximum of the diagonal pixel gradient value and the anti-diagonal pixel gradient value (outer 9) TIFF2025506036000020.tif1263Min (Outside 10) It may be derived from TIFF2025506036000021.tif1263. The derivation steps are as follows:

[0054] Step 1: (Outside 11) If TIFF2025506036000022.tif1497 is true at the same time, the directional feature D is set to 0. Here, t1 can be expressed as a preset parameter. Step 2: (Outside 12) If the result is TIFF2025506036000023.tif1497, proceed to step 3, otherwise proceed to step 4. Step 3: (Outside 13) If the result is TIFF2025506036000024.tif1497, then the directional feature D is set to 2, otherwise the directional feature D is set to 1. Step 4: (Outside 14) If the result is TIFF2025506036000025.tif1497, then the directional feature D is set to 4, otherwise the directional feature D is set to 3. Here, t1 and t2 are parameters preset according to actual needs, and are not limited in this application.

[0055] Quantized activity features (Outside 15) Before quantizing TIFF2025506036000026.tif14119, it may be expressed as activity feature A. Activity feature A is calculated using the following formula (11).

number

[0056] Note that before filtering for each luminance block of size 4*4 (including the block of luminance reconstruction pixels for each pixel in the pixel block mentioned above), the filter coefficients and corresponding limit values ​​can be geometrically transformed according to the pixel gradient value of the current luminance block according to the rules in Table 1. The geometric transformation can include, but is not limited to, no transformation, diagonal transformation, vertical flip, and rotation transformation. Table 1 is as follows: [Table 1] Here, performing a geometric transformation on the filter coefficients is equivalent to performing a geometric transformation on the pixel values ​​while leaving the coefficients unchanged, and then performing filtering. The purpose of the geometric transformation is to maximize the alignment of the content of different pixel blocks, thereby reducing the amount of classification required for ALF and allowing different pixels to share common filter coefficients. By introducing the geometric transformation, the realistic classification can be increased from category 25 to category 100 without increasing the number of ALF filters, thereby increasing adaptability.

[0057] (8) Cross-Component Adaptive Loop Filtering (CC-ALF): CC-ALF is a Wiener filter like the ALF described above, and its operation is similar, so it will not be described further here. CC-ALF only operates on chroma reconstructed pixels, and the input to CC-ALF is the luminance reconstructed pixel after DBF, BIF, BIF-Chroma, SAO, and CCSAO filtering and before ALF, and the output is the correction value corresponding to the chroma reconstructed pixel. Similarly, in the case of CC-ALF, it is necessary to first classify the video content and apply a corresponding filter to the video content by category, and the CC-ALF of each chroma reconstructed pixel can support up to four different categories of filters. CC-ALF utilizes the correlation between the luminance reconstructed pixel and the chroma reconstructed pixel to perform linear filtering on the luminance reconstructed pixel to obtain a correction value for the chroma reconstructed pixel. The correction value is added to the chroma reconstructed pixel subjected to ALF, and the chroma pixel to be finally reconstructed is the chroma pixel.

[0058] CC-ALF generates a corresponding correction value for each chroma reconstructed pixel by performing linear filtering on the luminance reconstructed pixel. See FIG. 6 for the implementation procedure of CC-ALF and the relationship between CC-ALF and ALF. FIG. 6 is a frame schematic diagram of a cross-component adaptive loop filter according to an embodiment of the present application. The embodiment of the present application will be described by taking the YCbCr color space as an example. As shown in FIG. 6, compensation values ​​are generated by BIF, BIF-Chroma, SAO, and CCSAO processing, and the compensation values ​​generated respectively are simultaneously added to the deblocking filtered reconstructed pixel, thereby generating a compensated luminance component pixel (Y component pixel R Y , etc.), a compensated blue chroma component pixel Cb, and a compensated red chroma component pixel Cr can be obtained.

[0059] The compensated luminance component pixel R Y is input to the ALF on the luminance component, and after ALF processing, the final luminance component pixel Y can be output. Yis input to CC-ALF on the blue chrominance component, and after CC-ALF processing, the reconstructed pixels on the blue chrominance component Cb are output. The difference value ΔR between the blue chrominance component pixel value output from CC-ALF and the deblocking filtered blue chrominance component pixel value is Cb Calculate the compensated luminance component pixel R Y is input to CC-ALF on the red chrominance component, and after CC-ALF processing, the reconstructed pixels on the red chrominance component Cr are output. The difference value ΔR Cr Calculate.

[0060] The compensated blue chrominance pixel and the compensated red chrominance pixel are input to the ALF on the chrominance component, and the reconstructed pixel on the blue chrominance component is output after the ALF process. △R Cb After ALF processing, the reconstructed pixel on the red chrominance component is output, and △R Cr to obtain the final red chroma component pixel Cr.

[0061] Among them, the filtering process by CC-ALF may be expressed by the following equation (12).

number

[0062] Referring to Fig. 7, Fig. 7 is a schematic diagram of the shape of a cross-component adaptive loop filtering filter according to an embodiment of the present application. CC-ALF can support a diamond filter of size 3*4 as shown in Fig. 7. The black circles shown in Fig. 7 can represent luminance component pixels that can be supported by the CC-ALF filter, and the white dotted circle can represent the sample position (x, y) of chroma component pixel f.

[0063] In addition, the filter coefficients of CC-ALF are no longer restricted in symmetry compared to ALF, so they can flexibly adapt to the relative relationships between various luminance and chrominance components. In addition, in order to reduce the number of filter coefficients that need to be transmitted, CC-ALF has two restrictions on its filter coefficients in the current coding framework design.

[0064] (1) The sum of all coefficients of CC-ALF is limited to 0. Therefore, for a diamond filter of size 3*4, only seven filter coefficients need to be calculated and transmitted, and the filter coefficient at the center position can be automatically estimated on the decoding side based on the condition. (2) The absolute value of each filter coefficient that needs to be transmitted must be a power of 2 and be represented by a maximum of 6 bits. Therefore, the absolute value of the filter coefficient of CC-ALF is {0, 2, 4, 8, 16, 32, 64}. This design uses shift operations instead of multiplications, which can reduce the number of multiplications. Among them, unlike the luma ALF that supports subblock-level classification and adaptive selection, CC-ALF only supports CTU-level classification and adaptive selection. For each chroma component pixel, all chroma component pixels in one CTU belong to the same category, so the same filter can be applied to this CTU.

[0065] An Adaptation Parameter Set (APS) can contain up to 25 pairs of luma ALF filter coefficients and corresponding limit indexes, and each chroma component pixel supports up to 8 pairs of chroma ALF filter coefficients and corresponding limit indexes, and each chroma component pixel supports up to 4 pairs of CC-ALF filter coefficients. To save bitrate, filter coefficients of different categories can be merged for luma ALF filters, and one set of filter coefficients can be shared among multiple categories. The encoder determines which categories of coefficients can be merged by rate distortion optimization (RDO). The slice header is marked with the index of the APS used in the current slice. CC-ALF supports CTU-level adaptation, but when there are multiple filters, the CTU level adaptively selects whether to use CC-ALF for each chroma component pixel and the index of the filter to be used.

[0066] Referring to Fig. 8, Fig. 8 is a schematic diagram of a configuration of a data processing system according to an embodiment of the present application. As shown in Fig. 8, the data processing system may include an encoding device 40a (encoding side) and a decoding device 40b (decoding side). The encoding device 40a may be a terminal device or a server. The decoding device 40b may be a terminal device or a server. A direct communication connection between the encoding device 40a and the decoding device 40b can be established by wired communication, or an indirect communication connection can be established by wireless communication.

[0067] Here, examples of the terminal device include, but are not limited to, a smartphone, a tablet, a laptop, a handheld, a mobile internet device (MID), a wearable device (such as a smart watch, a smart handheld), a smart voice dialogue device, a smart home appliance (such as a smart TV), an in-vehicle device, a VR device (such as a VR helmet, a VR glasses), etc. The server may be an independent physical server, a server cluster or a distributed system consisting of multiple physical servers, and may further be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDNs, and big data and artificial intelligence platforms.

[0068] (1) Regarding the encoding device 40a The encoding device 40a can obtain the available video data in a scene capture manner. The scene-captured video data means that a real-world visual scene is collected by a capture device associated with the encoding device 40a to obtain the video data. Here, the capture device may be used to provide a video data acquisition service for the encoding device 40a. The capture device may include, but is not limited to, any of an imaging device, a sensing device, and a scanning device.

[0069] Here, the imaging device may include a general camera, a stereoscopic camera, a light field camera, etc., the sensing device may include a laser device, a radar device, etc., and the scanning device may include a three-dimensional laser scanning device, etc. The capture device associated with the encoding device 40a may be a hardware component provided in the encoding device 40a, for example, the capture device may be a camera or a sensor of a terminal device, and the capture device associated with the encoding device 40a may also be a hardware device connected to the encoding device 40a, for example, a camera connected to a server, etc.

[0070] The encoding device 40a performs an encoding process on an image frame in the video data, and can obtain an encoded bit stream corresponding to the video data. Among them, the embodiment of the present application mainly describes the EO type of the CCSAO in the loop filter. When classifying the CCSAO EO, the encoding device 40a can classify the CCSAO EO based on the extended co-located luminance reconstructed pixel of the reconstructed pixel to be processed (also called the co-located luminance reconstructed pixel in a broad sense, that is, any luminance reconstructed pixel in the target area range centered on the realistic co-located luminance reconstructed pixel), and at the same time, classify the CCSAO EO using other available classification modes other than the four classification modes of horizontal, vertical, diagonal, and anti-diagonal, calculate a compensation value for each EO category, and encode the difference value between the original pixel in the block to be encoded and the reconstructed pixel to which the compensation value has been added, thereby obtaining an encoded bit stream. Furthermore, the encoding device 40a can transmit the obtained encoded bit stream to the decoding device 40b.

[0071] (2) Decryption Device 40b When the decoding device 40b receives the compressed bit rate (encoded bit stream) transmitted from the encoding device 40a, the decoding device 40b can perform a decoding process on the encoded bit stream to reconstruct image frame pixels in the video data. In particular, the decoding device 40b can identify the currently used classification mode by analyzing a syntax element related to the classification mode in the encoded bit stream, and can identify the position of the extended co-located luminance reconstruction pixel performing the CCSAO EO classification and the target classification mode (i.e., the classification mode actually used) based on the mode definition commonly set by the encoding device 40a and the decoding device 40b. Alternatively, the decoding device 40b can identify the position of the extended co-located luminance reconstruction pixel by analyzing a syntax element related to the position of the extended co-located luminance reconstruction pixel in the encoded bit stream, and can further identify the selected target classification mode by analyzing a syntax element related to the classification mode in the encoded bit stream. Furthermore, the decoding device 40b can reconstruct image frame pixels in the video data according to the position of the extended co-located luminance reconstruction pixel and the actual classification mode.

[0072] In the embodiment of the present application, the encoding device performs CCSAO EO classification based on the extended co-located luminance reconstruction pixel of the color component pixel, and at the same time, performs CCSAO EO classification using an extended classification mode (including other available classification modes other than the above-mentioned four classification modes of horizontal, vertical, diagonal, and anti-diagonal), so as to improve the category accuracy of the CCSAO EO of the pixel, and further improve the encoding performance of the video data. The decoding device analyzes the image frame pixels in the video data to obtain the position and realistic classification mode of the extended co-located luminance reconstruction pixel, so as to improve the decoding performance of the video data. The data processing system described in the embodiment of the present application is for more clearly explaining the technical method according to the embodiment of the present application, and is not limited to the technical method according to the embodiment of the present application. It can be understood that the technical method according to the embodiment of the present application can be applied to similar technical problems with the development of system architecture and the emergence of new business scenarios.

[0073] 9, which is a flowchart of a data processing method according to an embodiment of the present application. It can be understood that the data processing method corresponds to an encoding method, since the data processing method is executed by a computer device, and the computer device may be the encoding device 40a shown in FIG. 8. Here, as shown in FIG. 9, the data processing method can include the following steps S101 to S103.

[0074] In step S101, classification mode information corresponding to a block to be coded in the video data is identified; the classification mode information includes an extended co-located luminance reconstruction pixel corresponding to a color component pixel in the block to be coded and a target classification mode; the extended co-located luminance reconstruction pixel belongs to a target region centered on a realistic co-located luminance reconstruction pixel of the color component pixel.

[0075] Specifically, when a computer device performs an encoding process on video data, the computer device can perform frame processing on the video data to obtain a video sequence corresponding to the video data. The video sequence is composed of image frames in the video data, and the image frames in the video sequence are arranged in the temporal order of each image frame in the video data. For an image frame to be encoded in the video data, the image frame to be encoded can be divided into a number of blocks (which may be coding tree blocks or coding tree units). A block of the image frame to be encoded that is currently to be processed can be called a block to be encoded.

[0076] It is understood that the video coding process is performed in units of blocks, and thus when coding a block to be coded, coding operations such as transform, quantization, prediction, loop filtering, entropy coding, etc. can be performed on the block to be coded. For specific implementation procedures, please refer to the descriptions shown in the above-mentioned FIG. 1 to FIG. 7. The technical method according to the embodiment of the present application mainly describes the CCSAO EO classification process in loop filtering, and unlike the existing video coding frame, the CCSAO EO classification is performed only using the fixed realistic co-located luminance reconstructed pixel and the four classification modes (also called basic classification modes) shown in the above-mentioned FIG. 5. The embodiment of the present application proposes to perform the CCSAO EO classification using the extended co-located luminance reconstructed pixel (which can be selected from a target area centered on the realistic co-located luminance reconstructed pixel, and is also called the broad co-located luminance reconstructed pixel), or using the extended classification mode (other available classification modes other than the four classification modes shown in FIG. 5), or using both the extended co-located luminance reconstructed pixel and the extended classification mode. Hereinafter, the CCSAO EO classification process will be described in detail by taking any color component pixel in the block to be coded as an example.

[0077] In an embodiment of the present application, for each original color component pixel in the block to be coded, the loop filtering procedure shown in FIG. 2 above can be performed, that is, the color component pixel input to the CCSAO is substantially the reconstructed pixel obtained by deblocking filtering the original color component pixel in the block to be coded. For the color component pixel input to the CCSAO, the computer device can obtain a realistic co-located luminance reconstructed pixel corresponding to the color component pixel, identify a target region having a target size with the realistic co-located luminance reconstructed pixel as the center of the region, and identify an extended co-located luminance reconstructed pixel corresponding to the color component pixel in the target region, that is, the extended co-located luminance reconstructed pixel can be any luminance reconstructed pixel in the target region. A candidate classification mode set corresponding to the color component pixel can be obtained, a target classification mode corresponding to the color component pixel in the candidate classification mode set can be identified, and the extended co-located luminance reconstructed pixel and the target classification mode can be added to the classification mode information corresponding to the block to be coded.

[0078] Here, the target size is used to represent the coverage of the target area, for example, the target area may be of size 3*3, and the target area is considered to be a selection area of ​​the extended co-located luminance reconstruction pixels. The extended co-located luminance reconstruction pixels corresponding to the color component pixels are optimal luminance reconstruction pixels selected from among the target area. Referring to FIG. 10, FIG. 10 is a schematic diagram of a selection area of ​​the extended co-located luminance reconstruction pixels according to an embodiment of the present application. The position c0 shown in FIG. 10 can be represented as a realistic co-located luminance reconstruction pixel corresponding to the color component pixel, and the position c0 is the center of the target area. The target area may include positions 0 to 7, and the position c0 where the realistic co-located luminance reconstruction pixel itself is located. The extended co-located luminance reconstruction pixel corresponding to the color component pixel may be an optimal luminance reconstruction pixel selected from among positions 0 to 7, or an optimal luminance reconstruction pixel selected from among positions 0 to 7 and position c0.

[0079] Here, the target classification mode may be an optimal classification mode selected from the candidate classification mode set. The candidate classification mode set may refer to a set of all possible classification modes used to calculate difference values ​​between the extended co-located luminance reconstruction pixel and the surrounding adjacent pixels at any possible position where the extended co-located luminance reconstruction pixel may be present in the target area. When performing CCSAO EO classification using the extended co-located luminance reconstruction pixel, the coverage area of ​​the different classification modes in the candidate classification mode set may not be limited, for example, the size of the coverage area of ​​the different classification modes may be 5*5. Therefore, in this situation, when the extended co-located luminance reconstruction pixel is any luminance reconstruction pixel in the target area, the candidate classification mode set may include the basic classification mode and the extended classification mode, or the candidate classification mode may include the extended classification mode.

[0080] Specifically, when the expanded same-position luminance reconstructed pixel is any other luminance reconstructed pixel other than the realistic same-position luminance reconstructed pixel in the target region, i.e., a luminance reconstructed pixel at any of positions 1 to 7 shown in FIG. 10, the candidate classification modes at this time may include only the basic classification mode, or only the extended classification mode, or may simultaneously include the basic classification mode and the extended classification mode. When the expanded same-position luminance reconstructed pixel is a realistic same-position luminance reconstructed pixel in the target region, for example, position c0 shown in FIG. 10, the candidate classification modes at this time may include only the extended classification mode. Among them, the basic classification mode may include a horizontal classification mode, a vertical classification mode, a diagonal classification mode, and an inverse diagonal classification mode, as in the classification modes shown in FIG. 5 described above. The extended classification mode refers to an available classification mode other than the basic classification mode, as in the extended classification mode shown in FIG. 11.

[0081] Referring to Fig. 11, Fig. 11 is a schematic diagram of an extended classification mode according to an embodiment of the present application. The extended classification mode proposed in the present application may include 16 classification modes, from extended classification mode 50a to extended classification mode 50r, shown in Fig. 11. The target classification mode corresponding to a color component pixel may be any of the extended classification modes shown in Fig. 11, that is, the target classification mode corresponding to the color component pixel may be selected from all the extended classification modes shown in Fig. 11.

[0082] In addition, when performing CCSAO EO classification using the extended classification mode, only chroma component pixels may be used, or all color component pixels (chroma component pixels and luminance component pixels) may be used, or only luminance component pixels may be used, but this is not limited in this application.

[0083] The color component pixels in the block to be coded may be either luminance component pixels or chrominance component pixels. For example, only color component pixels may be used when performing CCSAO EO classification using the extended co-located luminance reconstruction pixels, or all color component pixels (chrominance component pixels and luminance component pixels), or only luminance component pixels may be used, but this application is not limited thereto.

[0084] Regardless of whether the color component pixel currently being inputted as a processing target is a luminance component pixel or a chrominance component pixel, the corresponding co-located luminance reconstructed pixel is calculated to perform CCSAO EO classification, but when calculating the co-located luminance reconstructed pixel corresponding to the color component pixel, particularly the co-located luminance reconstructed pixel corresponding to the chrominance component pixel, the position of the actually calculated co-located luminance reconstructed pixel may deviate to a certain extent from the actual position of the color component pixel, so it will be understood that the accuracy of classification of color component pixels by CCSAO EO can be improved by performing CCSAO EO classification using the extended co-located luminance reconstructed pixel. In actual application scenarios, considering the bit rate, CCSAO EO classification may be performed using the extended co-located luminance reconstructed pixel only for the chrominance component pixel, but both are supported by the technical method according to the embodiment of the present application.

[0085] In one or more embodiments, when performing CCSAO EO classification using the extended co-located luminance reconstruction pixel, the coverage area of ​​the different classification modes should not deviate from the range covered by the different classification modes during the existing CCSAO EO classification, i.e., the coverage area of ​​the different classification modes should not deviate from the target area centered on the realistic co-located luminance reconstruction pixel, see FIG. 12 for the coverage area of ​​the different classification modes. FIG. 12 is a schematic diagram of the mode coverage area of ​​the extended co-located luminance reconstruction pixel according to an embodiment of the present application. For example, the position c0 shown in FIG. 12 represents the realistic co-located luminance reconstruction pixel corresponding to the color component pixel, the region 60a represents the unavailable region of the extended co-located luminance reconstruction pixel, and the region 60b is the coverage area defined by each classification mode. That is, the positions of two adjacent elements of the extended co-located luminance reconstruction pixel defined by each classification mode in the candidate classification mode set are limited to within the region 60b and should not deviate from the range of the region 60b, i.e., the region outside the region 60b is the unavailable region.

[0086] Optionally, when the cover region defined by each classification mode is a target region, the luma reconstructed pixels in the target region other than the vertex positions can be identified as a candidate luma pixel set corresponding to the color component pixels. The candidate luma pixel set may include luma reconstructed pixels in a cross-shaped region of the target region, for example, positions 1, 3, 4, 6, and c0 shown in FIG. 10. For each luma reconstructed pixel in the candidate luma pixel set, a rate-distortion optimization process is performed on the color component pixels to obtain a first rate-distortion loss corresponding to each luma reconstructed pixel in the candidate luma pixel set. The luma reconstructed pixel corresponding to the smallest first rate-distortion loss (optimal luma reconstructed pixel) is identified as the extended co-located luma reconstructed pixel corresponding to the color component pixel.

[0087] Also, on the premise that the coverage area defined by each classification mode is the target area, the available classification modes can be restricted by the position of the extended co-located luminance reconstruction pixel. For example, a candidate classification mode set corresponding to a color component pixel can be identified according to the position of the extended co-located luminance reconstruction pixel in the target area, and the coverage range of the classification modes in the candidate classification mode set is equal to or smaller than the target area. When the extended co-located luminance reconstruction pixel is at any one of positions 1, 3, 4, and 6 as shown in FIG. 10, the candidate classification mode set corresponding to any possible position of the extended co-located luminance reconstruction pixel may include a horizontal classification mode or a vertical classification mode. When the extended co-located luminance reconstruction pixel is at position c0 (i.e., a realistic co-located luminance reconstruction pixel) shown in FIG. 10, the candidate classification mode set may include a horizontal classification mode, a vertical classification mode, a diagonal classification mode, and an inverse diagonal classification mode, i.e., the four classification modes shown in FIG. 5. According to each classification mode in the candidate classification mode set, a rate-distortion optimization process is performed on the color component pixel, respectively, to obtain a second rate-distortion loss corresponding to each classification mode in the candidate classification mode set. The classification mode corresponding to the second rate-distortion loss that is the smallest (the optimal classification mode) is identified as the target classification mode corresponding to the color component pixels.

[0088] Now, referring to Fig. 13, Fig. 13 is a schematic diagram of the position and corresponding classification mode of the expanded co-located luminance reconstruction pixel according to an embodiment of the present application. As shown in Fig. 13, when the expanded co-located luminance reconstruction pixel is at position 1 shown in Fig. 10, the corresponding classification mode is the horizontal classification mode 70a shown in Fig. 13. When the expanded co-located luminance reconstruction pixel is at position 3 shown in Fig. 10, the corresponding classification mode is the vertical classification mode 70b shown in Fig. 13. When the expanded co-located luminance reconstruction pixel is at position 4 shown in Fig. 10, the corresponding classification mode is the vertical classification mode 70c shown in Fig. 13. When the expanded co-located luminance reconstruction pixel is at position 6 shown in Fig. 10, the corresponding classification mode is the horizontal classification mode 70d shown in Fig. 13.

[0089] Optionally, assuming that the coverage area defined by each classification mode is the target area, if the expanded co-located luminance reconstruction pixel is one of the luminance reconstruction pixels in the target area and the coverage area of ​​the different classification modes in the candidate classification mode set is larger than the target area, the border pixels in the target area can be used to fill the adjacent area of ​​the target area to obtain a mode coverage area. The mode coverage area is used to identify the target classification mode in the candidate classification mode set. In other words, if the coverage area of ​​the different classification mode is outside the range covered by the different classification modes during the existing CCSAO EO classification, i.e., if there is a classification mode whose coverage area is outside the target area, the range covered by the different classification modes during the existing CCSAO EO classification can be used to fill the unavailable area of ​​the expanded co-located luminance reconstruction pixel. That is, the border pixels in the target area can be copied to fill the unavailable area of ​​the expanded co-located luminance reconstruction pixel.

[0090] Referring to FIG. 14, FIG. 14 is a schematic diagram for filling the unavailable area of ​​the extended co-located luminance reconstruction pixel with the coverage area according to an embodiment of the present application. The area 61b shown in FIG. 14 represents the covered area of ​​different classification modes during the existing CCSAO EO classification, or represents the available area (e.g., the target area) of the extended co-located luminance reconstruction pixel, and the area 61a represents the unavailable area of ​​the extended co-located luminance reconstruction pixel. If the coverage area of ​​the classification mode in the candidate classification mode set is larger than the area 61b (the target area), each boundary pixel in the area 61b can be used to respectively fill the area adjacent to the corresponding boundary pixel in the area 61a. For example, the boundary pixel P0 in the area 61b can be used to fill the area adjacent to the boundary pixel P0 in the area 61a, and the boundary pixel P1 in the area 61b can be used to fill the area adjacent to the boundary pixel P1 in the area 61a.

[0091] In addition, the candidate luminance pixel set corresponding to the color component pixel may include all or a part of the luminance reconstruction pixels in the target area. For example, according to the constraints in the actual application scene, the luminance reconstruction pixel at any position in the target area may be selected as the candidate luminance pixel set corresponding to the color component pixel, but this application is not limited thereto.

[0092] In step S102, a boundary category corresponding to the color component pixel is identified according to the expanded co-located luminance reconstruction pixel and the target classification mode.

[0093] Specifically, after the extended co-located luminance reconstruction pixel corresponding to the color component pixel and the target classification mode are identified, for ease of understanding, the extended co-located luminance reconstruction pixel may be denoted as c and the realistic co-located luminance reconstruction pixel may be denoted as c0. Since the boundary category may be jointly generated by the boundary-based category and the band category, the extended co-located luminance reconstruction pixel may be used only for calculating the boundary category of the color component pixel, or may be used only for calculating the band category of the color component pixel, or may be used for calculating the band category and the boundary category of the color component pixel.

[0094] In the following, an example will be described in which the color component pixels are replaced with luminance component pixels. The expanded same-position luminance reconstruction pixel is assigned to the band category (i B ) and boundary categories (class idxWhen the target classification mode is used to calculate a band category i of the expanded co-located luminance reconstruction pixel c, the computing device identifies a first neighboring pixel (denoted as a) and a second neighboring pixel (denoted as b) corresponding to the expanded co-located luminance reconstruction pixel c based on the target classification mode, obtains a first difference value (denoted as Ea) between the first neighboring pixel and the expanded co-located luminance reconstruction pixel c, obtains a second difference value (denoted as Eb) between the second neighboring pixel and the expanded co-located luminance reconstruction pixel, obtains a first co-located chroma pixel and a second co-located chroma pixel corresponding to the realistic co-located luminance reconstruction pixel, i.e., reconstruction pixels on two chroma components of the color component pixel, and determines a band category i to which the luminance component pixel belongs based on the expanded co-located luminance reconstruction pixel, the first co-located chroma pixel, and the second co-located chroma pixel. B Identify band category i B , according to the first difference value and the second difference value, a boundary category class corresponding to the color component pixel is determined. idx can be identified.

[0095] Here, the band category i to which the color component pixel belongs B The process of obtaining the extended same-position luminance reconstruction pixel (the extended same-position luminance reconstruction pixel in this case corresponds to cur in Equation (3)) and the total number of band categories on the luminance component (N cur ) to obtain the first product of the first co-located chroma pixel (denoted as coll) and the total number of band categories (N col1 ) and the second product of the second co-located chroma pixel (denoted as col2) and the total number of band categories on the second chroma component (N col2 and determining a band category i to which the luminance component pixel belongs based on a numerical relationship between a bit depth (denoted as BD) of the pixel value and each of the first product, the second product, and the third product. BFor example, in the YUV color space, the luminance component may be Y, the first chroma component may be U, and the second chroma component may be V, but the color space used is not limited in this application. In other words, when calculating the band category to which the color component pixel belongs, the extended co-located luminance reconstructed pixel corresponding to the luminance component pixel and the realistic co-located chroma component pixel are used. When calculating the boundary category to which the color component pixel belongs, the extended co-located luminance reconstructed pixel corresponding to the luminance component pixel is used.

[0096] In the following, an example will be described in which the color component pixels are chroma component pixels. The extended same-position luminance reconstruction pixel is in the boundary category class of the color component pixels. idx When the target classification mode is used to calculate only the co-located luminance reconstruction pixel c, the computing device can identify a first neighboring pixel and a second neighboring pixel corresponding to the expanded co-located luminance reconstruction pixel c based on the target classification mode, obtain a first difference value between the first neighboring pixel and the expanded co-located luminance reconstruction pixel, obtain a second difference value between the second neighboring pixel and the expanded co-located luminance reconstruction pixel, obtain a first co-located chroma pixel and a second co-located chroma pixel corresponding to the realistic co-located luminance reconstruction pixel, identify a band category to which the luminance component pixel belongs based on the realistic co-located luminance reconstruction pixel, the first co-located chroma pixel and the second co-located chroma pixel, and identify a boundary category to which the color component pixel belongs according to the band category, the first difference value and the second difference value. In other words, when calculating the band category to which the color component pixel belongs, the realistic co-located luminance reconstruction pixel corresponding to the luminance component pixel (i.e., the deblocking filtered luminance reconstruction pixel itself) and the realistic co-located chroma component pixel are used. When calculating the boundary category to which the color component pixel belongs, the expanded co-located luminance reconstruction pixel corresponding to the luminance component pixel is used.

[0097] In the following, an example will be described in which the color component pixel is a chroma component pixel. The extended same-position luminance reconstruction pixel is in the band category i of the color component pixel. BWhen the target classification mode is used to calculate only the co-located luminance reconstructed pixel, the computing device can identify a third neighboring pixel a and a fourth neighboring pixel b corresponding to the realistic co-located luminance reconstructed pixel based on the target classification mode, obtain a third difference value Ea between the third neighboring pixel and the realistic co-located luminance reconstructed pixel, obtain a fourth difference value Eb between the fourth neighboring pixel and the realistic co-located luminance reconstructed pixel, obtain a first co-located chroma pixel and a second co-located chroma pixel corresponding to the realistic co-located luminance reconstructed pixel, identify a band category to which the luminance component pixel belongs based on the extended co-located luminance reconstructed pixel, the first co-located chroma pixel, and the second co-located chroma pixel, and identify a boundary category to which the color component pixel belongs according to the band category, the third difference value, and the fourth difference value. In other words, the extended co-located luminance reconstructed pixel and the realistic co-located chroma component pixel corresponding to the luminance component pixel are used when calculating the band category to which the color component pixel belongs. The realistic co-located luminance reconstructed pixel is used when calculating the boundary category to which the color component pixel belongs.

[0098] Optionally, the color component pixels may further be chroma component pixels (e.g., U component pixel, V component pixel). In identifying the boundary category of the chroma component pixels, operations similar to those for the luma component pixels described above may be performed. For example, in the YUV color space, the color component pixels are U component pixels, where the U component pixel corresponds to the realistic co-located U component reconstructed pixel (denoted as cur), the co-located luma component pixel (denoted as coll) may be the realistic co-located luma reconstructed pixel or may be the extended co-located luma reconstructed pixel, and the co-located V component reconstructed pixel is the realistic co-located U component reconstructed pixel. Note that the boundary category class idx The implementation procedure may be shown in the above formula (2), and the band category The process of obtaining TIFF2025506036000031.tif33 may be represented by the above formula (3), but will not be described further here.

[0099] In step S103, the reconstructed pixels of the color component pixels are compensated based on the boundary category to obtain the compensated reconstructed pixels, and the encoding process is performed on the encoding target block based on the compensated reconstructed pixels.

[0100] Specifically, computing devices fall into the boundary category class idx The compensation value corresponding to the boundary category class output by CCSAO idx The compensation value corresponding to, for example, σ shown in Eq. (2) CCSAO [class idx ] can be calculated, and the compensation value can be added to the reconstructed pixel of the color component pixel to obtain the compensated reconstructed pixel. In other words, the boundary category class idx The reconstructed pixel of the color component pixel is compensated based on the compensation value corresponding to the color component pixel, and a compensated reconstructed pixel corresponding to the color component pixel is obtained. Here, the reconstructed pixel of the color component pixel may refer to a pixel output through deblocking filtering. The compensated reconstructed pixel may refer to a reconstructed pixel output through the CCSAO EO using an expanded co-position luminance reconstructed pixel corresponding to the color component pixel to be processed in the CCSAO, for example, C'rec in the above formula (2). However, the procedure for realizing the compensated reconstructed pixel is not described here.

[0101] Furthermore, when performing CCSAO EO classification using the extended co-located luminance reconstruction pixels, the positions of the extended co-located luminance reconstruction pixels corresponding to the color component pixels and the target classification mode for calculating the first difference value and the second difference value all need to be encoded and transferred to the decoding end (e.g., the decoding device 40b in the embodiment corresponding to FIG. 8).

[0102] In one or more embodiments, the computing device can set a position syntax element for the extended co-located luminance reconstruction pixel corresponding to the color component pixel, and set a mode syntax element for the target classification mode corresponding to the color component pixel. For the color component pixels in the block to be coded, different color component pixels belong to different boundary categories, and each boundary category can correspond to one compensation value, and further, by adding the corresponding compensation values ​​of each boundary category in the video data and performing an encoding process on the total compensation value, an encoding result corresponding to the total compensation value can be obtained, and further, the position syntax element, the mode syntax element, and the encoding result corresponding to the total compensation value corresponding to each color component pixel in the block to be coded can be identified as an encoded bit stream corresponding to the block to be coded, and the encoded bit stream can be transferred to a decoding end. In other words, the computing device can individually code the position of the extended co-located luminance reconstruction pixel, for example, the position of the extended co-located luminance reconstruction pixel can be indexed by a dedicated syntax element (i.e., a position syntax element). The target classification mode may multiplex a syntax element (i.e., a mode syntax element) for identifying the classification mode in an existing CCSAO EO classification.

[0103] In one or more embodiments, the computing device can set common syntax elements for the extended co-located luminance reconstruction pixels corresponding to the color component pixels and the target classification mode. Based on the common syntax elements and the total compensation values ​​corresponding to each boundary category in the video data, an encoded bitstream of the block to be encoded can be obtained, and the encoded bitstream can be transferred to the decoding end. In other words, the extended co-located luminance reconstruction pixels corresponding to the color component pixels can be combined with their corresponding neighboring pixels (including the first neighboring pixel and the second neighboring pixel) and jointly encoded, for example, any possible position of the extended co-located luminance reconstruction pixels corresponding to the color component pixels and the available classification modes corresponding to each position can be used as a new mode in the CCSAO EO classification, and further a syntax element (common syntax element) for identifying the classification mode in the existing CCSAO EO can be multiplexed and transferred. Therefore, the same common syntax elements can be set for the extended co-located luminance reconstruction pixels corresponding to the color component pixels and the target classification mode.

[0104] For color component pixels in a block to be coded, the CCSAO EO classification may be performed using the extended co-located luminance reconstructed pixels, or may be performed without using the extended co-located luminance reconstructed pixels. If the rate-distortion loss when the extended co-located luminance reconstructed pixels are not used is smaller than the rate-distortion loss when the extended co-located luminance reconstructed pixels are used, the CCSAO EO classification may be performed without using the extended co-located luminance reconstructed pixels. On the other hand, if the rate-distortion loss when the extended co-located luminance reconstructed pixels are not used is larger than the rate-distortion loss when the extended co-located luminance reconstructed pixels are used, the CCSAO EO classification may be performed using the extended co-located luminance reconstructed pixels.

[0105] Optionally, during the encoding process of video data, whether the extended co-located luminance reconstruction pixel (herein, the extended co-located luminance reconstruction pixel particularly refers to a luminance reconstruction pixel at any of positions 0 to 7 shown in FIG. 10 ) and the extended classification pattern are used can be identified by a bit flag, and the bit flag may be called a classification identification field. For example, if it is determined that the CCSAO EO classification is performed using the extended co-located luminance reconstruction pixel and the extended classification mode, the classification identification field can be set to a first identification value. On the other hand, if it is determined that the CCSAO EO classification is performed without using the extended co-located luminance reconstruction pixel and the extended classification mode, the classification identification field can be set to a second identification value. The first identification value and the second identification value are different from each other. The first identification value and the second identification value can include, but are not limited to, symbols, numerical values, signs, and the like.

[0106] The above-mentioned classification identification field can be transferred in a high level syntax (HLS), and the classification identification field may be stored in a sequence parameter set (SPS), a picture header, a slice header, or an APS, but is not limited in this application. When the classification identification field is stored in the SPS, the classification identification field indicates whether the extended co-located luminance reconstruction pixel and the extended classification mode are used in the video data. When the classification identification field is stored in the PictureHeader, the classification identification field indicates whether the extended co-located luminance reconstruction pixel and the extended classification mode are used in the current image frame. When the classification identification field is stored in the SliceHeader, the classification identification field indicates whether the extended co-located luminance reconstruction pixel and the extended classification mode are used in the current slice. When the classification identification field is stored in the APS, the classification identification field indicates whether the extended co-located luminance reconstruction pixel and the extended classification mode are used in the loop filtering.

[0107] Optionally, different bit flags are used to identify whether the extended co-located luminance reconstruction pixel and the extended classification pattern are used to perform the CCSAO EO classification. In this case, the classification identification field may include an extended co-located luminance identification field and an extended mode identification field. For example, when it is determined that the CCSAO EO classification is performed using the extended co-located luminance reconstruction pixel, the extended co-located luminance identification field may be set to a first identification value. On the other hand, when it is determined that the CCSAO EO classification is performed without using the extended co-located luminance reconstruction pixel, the extended co-located luminance identification field may be set to a second identification value. Also, when it is determined that the CCSAO EO classification is performed using the extended classification mode, the extended mode identification field may be set to the first identification value. On the other hand, when it is determined that the CCSAO EO classification is performed without using the extended classification mode, the extended mode identification field may be set to the second identification value. Both the extended co-located luminance identification field and the extended mode identification field can be transmitted in the HLS, and the classification identification field may be stored in the SPS, PictureHeader, SliceHeader, or APS.

[0108] If it is determined that the video data is to be classified using one or both of the extended co-located luminance reconstruction pixels and the extended classification mode, the classification identification field can be set to a first identification value, and the encoding process can be continued for the video data according to the technical method of the embodiment of the present application. On the other hand, if it is determined that the video data is not to be classified using the extended co-located luminance reconstruction pixels and the extended classification mode, the classification identification field can be set to a second identification value, and the encoding process can be continued for the video data according to the existing method.

[0109] In an embodiment of the present application, the encoding device can perform CCSAO EO classification based on the extended co-located luminance reconstruction pixels corresponding to the color component pixels, and at the same time, can also perform CCSAO EO classification using an extended classification mode (including other available classification modes other than the above-mentioned four classification modes of horizontal, vertical, diagonal, and anti-diagonal), thereby improving the category accuracy of the CCSAO EO of the color component pixels and further improving the overall encoding performance of the video data.

[0110] 15, which is a flowchart of another data processing method according to an embodiment of the present application. It can be understood that the data processing method corresponds to the decoding method, since the data processing method is executed by a computer device, and the computer device may be the decoding device 40b shown in FIG. 8. Here, as shown in FIG. 15, the data processing method can include the following steps S201 to S203.

[0111] In step S201, a decoding process is performed on a block to be decoded in video data to obtain classification mode information corresponding to the block to be decoded; the classification mode information includes an extended co-located luminance reconstruction pixel corresponding to a color component pixel in the block to be decoded and a target classification mode; the extended co-located luminance reconstruction pixel belongs to a target area centered on a realistic co-located luminance reconstruction pixel of the color component pixel.

[0112] Specifically, the computer device can analyze a position syntax element corresponding to a color component pixel, obtain an extended co-located luminance reconstruction pixel corresponding to the color component pixel, analyze a mode syntax element corresponding to the color component pixel, obtain a target classification mode corresponding to the color component pixel, and identify the extended co-located luminance reconstruction pixel and the target classification mode as classification mode information corresponding to the block to be decoded. In other words, the computer device can analyze a position syntax element (syntax element related to the position of the extended co-located luminance reconstruction pixel) to identify the position of the extended co-located luminance reconstruction pixel, analyze a mode syntax element (syntax element related to the classification mode) to identify the target classification mode, and calculate a difference value between the position of the extended co-located luminance reconstruction pixel and surrounding pixels of the area covered by the target mode, centered on the position of the extended co-located luminance reconstruction pixel, to perform boundary classification.

[0113] In one or more embodiments, the computer device can analyze a common syntax element corresponding to a color component pixel, obtain an extended co-located luminance reconstruction pixel and a target classification mode corresponding to the color component pixel, and specify the extended co-located luminance reconstruction pixel and the target classification mode as classification mode information corresponding to the block to be decoded. In other words, the computer device can analyze a common syntax element (syntax element related to a classification mode), and define a position of the extended co-located luminance reconstruction pixel and a target classification mode that can be specified to perform classification according to a mode commonly set on the encoding side and the decoding side (the mode is the same on both the encoding side and the decoding side).

[0114] Before analyzing the classification mode information corresponding to the color component pixels in the block to be decoded, the computer device may first analyze a classification identification field in a corresponding sequence parameter set of the video data. If the classification identification field is a first identification value, it is determined that the extended co-located luminance reconstruction pixels are used in the video data. On the other hand, if the classification identification field is a second identification value, it is determined that the extended co-located luminance reconstruction pixels are not used in the video data. For example, if it is analyzed that the classification identification field in the corresponding sequence parameter set of the video data is a first identification value, it is found that the extended co-located luminance reconstruction pixels and the extended classification mode are used in the video data. On the other hand, when performing a decoding process on a certain image frame in the video data, if it is analyzed that the classification identification field stored in the PictureHeader is a second identification value, it is found that the extended co-located luminance reconstruction pixels and the extended classification mode are not used in the image frame currently being processed.

[0115] Optionally, when the classification identification field includes an extended same-position luminance identification field for the extended same-position luminance reconstruction pixel and an extended mode identification field for the extended classification mode, the extended same-position luminance identification field and the extended mode identification field can be sequentially analyzed, and based on the analysis result, it can be determined whether the extended same-position luminance reconstruction pixel and the extended classification mode are used in the video data. If it is determined that the video data is classified using one or both of the extended same-position luminance reconstruction pixel and the extended classification mode, the subsequent decoding process can be continued according to the technical method according to the embodiment of the present application. On the other hand, if it is determined that the video data is classified using the CCSAO EO classification without using the extended same-position luminance reconstruction pixel and the extended classification mode, the subsequent decoding process can be continued according to the existing method.

[0116] In step S202, a boundary category corresponding to the color component pixel is identified according to the expanded co-located luminance reconstruction pixel and the target classification mode.

[0117] Here, the implementation process of step S202 can refer to step S102 in the embodiment corresponding to FIG. 9 described above, so no further description will be given here.

[0118] In step S203, color component pixels in the block to be decoded are reconstructed according to the compensation value corresponding to the boundary category.

[0119] Specifically, the computer device can identify compensated reconstruction pixels according to the compensation values ​​corresponding to the boundary categories, and reconstruct color component pixels in the block to be decoded by the compensated reconstruction pixels. Specifically, the total compensation values ​​corresponding to each boundary category in the video data can be analyzed, and the boundary categories corresponding to the color component pixels in the block to be decoded can be identified according to the analyzed target classification mode and the expanded co-position luminance reconstruction pixels, and the compensation values ​​corresponding to the boundary categories, i.e., the compensation values ​​corresponding to a single boundary category, can be identified from the total compensation values, and the color component pixels belonging to the boundary category can be reconstructed by the compensation values ​​corresponding to the single boundary category.

[0120] In addition, since the video data decoding process and the video data encoding process are processes in the opposite directions to each other, the video data decoding process can be referred to in the embodiment corresponding to Figure 9 above, so no further explanation will be given here.

[0121] In an embodiment of the present application, the decoding device can perform CCSAO EO classification based on the extended co-located luminance reconstruction pixels corresponding to the color component pixels, and at the same time, can also perform CCSAO EO classification using an extended classification mode (including other available classification modes other than the above-mentioned four classification modes of horizontal, vertical, diagonal, and anti-diagonal), so as to improve the category accuracy of the CCSAO EO of the color component pixels and further improve the overall decoding performance of the video data.

[0122] Referring to Figure 16, Figure 16 is a schematic configuration diagram of a data processing device according to an embodiment of the present application. The data processing device may be a computer program (including program code) executed in an encoding device, for example, the data processing device is an application software in an encoding device. Referring to Figure 16, the data processing device 1 may include a classification mode information identifying module 11, a first category identifying module 12, and an encoding processing module 13, where: The classification mode information identification module 11 is configured to identify classification mode information corresponding to a block to be encoded in the video data; the classification mode information includes an extended co-located luminance reconstruction pixel corresponding to a color component pixel in the block to be encoded and a target classification mode; the extended co-located luminance reconstruction pixel belongs to a target region centered on a realistic co-located luminance reconstruction pixel of the color component pixel. The first category identification module 12 is configured to identify a boundary category corresponding to the color component pixel according to the expanded co-located luminance reconstruction pixel and the target classification mode. The encoding processing module 13 is configured to compensate the reconstructed pixels of the color component pixels based on the boundary category, obtain the compensated reconstructed pixels, and perform an encoding process on the encoding target block based on the compensated reconstructed pixels.

[0123] In one or more embodiments, the classification mode information identification module 11 specifically: A step of obtaining color component pixels included in a block to be coded, and obtaining realistic co-located luminance reconstruction pixels corresponding to the color component pixels; identifying a target region having a target size with a realistic co-located luminance reconstruction pixel as a center of the region, and identifying expanded co-located luminance reconstruction pixels corresponding to the color component pixels within the target region; The method is configured to perform the steps of obtaining a candidate classification mode set corresponding to a color component pixel, identifying a target classification mode corresponding to the color component pixel in the candidate classification mode set, and adding the extended co-located luminance reconstruction pixel and the target classification mode to classification mode information corresponding to the block to be encoded.

[0124] In one or more embodiments, the step of identifying, using the classification mode information identification module 11, the expanded co-located luminance reconstruction pixels corresponding to the color component pixels in the target region includes: Identifying luminance reconstruction pixels other than the vertex positions in the target region as candidate luminance pixel sets corresponding to color component pixels; performing a rate-distortion optimization process on the color component pixels according to each luma reconstructed pixel in the candidate luma pixel set, respectively, to obtain a first rate-distortion loss corresponding to each luma reconstructed pixel in the candidate luma pixel set; and identifying the luma reconstructed pixel corresponding to the first rate-distortion loss that is minimal as the extended co-located luma reconstructed pixel corresponding to the color component pixel.

[0125] In one or more embodiments, the step of obtaining a candidate classification mode set corresponding to the color component pixel using the classification mode information identification module 11 and identifying a target classification mode corresponding to the color component pixel in the candidate classification mode set includes: identifying a candidate classification mode set corresponding to a color component pixel according to a position of the expanded co-located luminance reconstruction pixel in the target region, the coverage of the classification modes in the candidate classification mode set is equal to or smaller than the target region, and the candidate classification mode set includes a horizontal classification mode and a vertical classification mode; performing a rate-distortion optimization process on the color component pixels according to each classification mode in the candidate classification mode set, respectively, to obtain a second rate-distortion loss corresponding to each classification mode in the candidate classification mode set; identifying the classification mode corresponding to the second rate-distortion loss that is minimal as a target classification mode corresponding to the color component pixels.

[0126] In one or more embodiments, if the expanded co-located luminance reconstruction pixel is any luminance reconstruction pixel of the target region, The data processing device 1 further comprises: If the candidate classification mode set includes a classification mode whose coverage range is larger than the target region, the method is configured to execute a step of filling adjacent areas of the target region with boundary pixels in the target region to obtain a mode coverage range for identifying the target classification mode in the candidate classification mode set.

[0127] In one or more embodiments, if the expanded co-located luminance reconstruction pixel is any luminance reconstruction pixel of the target region, the set of candidate classification modes includes a base classification mode and an expanded classification mode, or the set of candidate classification modes includes an expanded classification mode, where the base classification mode includes a horizontal classification mode, a vertical classification mode, a diagonal classification mode, and an anti-diagonal classification mode, and the expanded classification mode includes any available classification mode other than the base classification mode.

[0128] In one or more embodiments, the color component pixels include luma component pixels or chroma component pixels.

[0129] In one or more embodiments, if the color component pixels include luminance component pixels, the first category identification module 12 specifically: identifying a first neighboring pixel and a second neighboring pixel corresponding to the expanded co-located luminance reconstruction pixel according to a target classification mode; obtaining a first difference value between a first adjacent pixel and the expanded co-located luminance reconstruction pixel, and obtaining a second difference value between a second adjacent pixel and the expanded co-located luminance reconstruction pixel; obtaining a first co-located chroma pixel and a second co-located chroma pixel corresponding to a realistic co-located luminance reconstruction pixel; determining a band category to which the luminance component pixel belongs based on the expanded co-located luminance reconstruction pixel, the first co-located chroma pixel, and the second co-located chroma pixel; and determining a boundary category corresponding to the color component pixel based on the band category, the first difference value, and the second difference value.

[0130] Here, the step of identifying a band category to which the luminance component pixel belongs based on the expanded co-located luminance reconstruction pixel, the first co-located chroma pixel, and the second co-located chroma pixel using the first category identification module 12 includes the steps of: obtaining a first product of the expanded co-located luminance reconstruction pixel and a total number of band categories on the luminance component, obtaining a second product of the first co-located chroma pixel and a total number of band categories on the first chroma component, and obtaining a third product of the second co-located chroma pixel and a total number of band categories on the second chroma component; and identifying a band category to which the luminance component pixel belongs based on a numerical relationship between a bit depth of the pixel value and each of the first product, the second product, and the third product.

[0131] In one or more embodiments, if the color component pixels include luminance component pixels, the first category identification module 12 specifically: identifying a first neighboring pixel and a second neighboring pixel corresponding to the expanded co-located luminance reconstruction pixel according to a target classification mode; obtaining a first difference value between a first adjacent pixel and the expanded co-located luminance reconstruction pixel, and obtaining a second difference value between a second adjacent pixel and the expanded co-located luminance reconstruction pixel; obtaining a first co-located chroma pixel and a second co-located chroma pixel corresponding to a realistic co-located luminance reconstruction pixel; determining a band category to which the luminance component pixel belongs based on the realistic co-located luminance reconstruction pixel, the first co-located chroma pixel, and the second co-located chroma pixel; and determining a boundary category corresponding to the color component pixel based on the band category, the first difference value, and the second difference value.

[0132] In one or more embodiments, if the color component pixels include luminance component pixels, the first category identification module 12 specifically: identifying third and fourth neighboring pixels corresponding to realistic co-located luminance reconstruction pixels according to a target classification mode; obtaining a third difference value between the third adjacent pixel and the realistic co-located luminance reconstruction pixel, and obtaining a fourth difference value between the fourth adjacent pixel and the realistic co-located luminance reconstruction pixel; obtaining a first co-located chroma pixel and a second co-located chroma pixel corresponding to a realistic co-located luminance reconstruction pixel; determining a band category to which the luminance component pixel belongs based on the expanded co-located luminance reconstruction pixel, the first co-located chroma pixel, and the second co-located chroma pixel; and determining a boundary category corresponding to the color component pixel based on the band category, the third difference value, and the fourth difference value.

[0133] In one or more embodiments, the step of performing an encoding process on the current block using the compensated reconstructed pixels using the encoding process module 13 includes: setting a location syntax element for the extended co-located luminance reconstruction pixels and setting a mode syntax element for the target classification mode; identifying a compensation value corresponding to the boundary category; and obtaining an encoded bitstream for the current block based on the position syntax element, the mode syntax element, and the compensation value.

[0134] In one or more embodiments, the step of performing an encoding process on the current block using the compensated reconstructed pixels using the encoding process module 13 includes: establishing common syntax elements for the extended co-located luminance reconstruction pixel and target classification modes; identifying a compensation value corresponding to the boundary category, and obtaining an encoded bitstream for the block to be encoded based on the common syntax element and the compensation value.

[0135] In an embodiment of the present application, the encoding device can perform CCSAO EO classification based on the extended co-located luminance reconstruction pixels corresponding to the color component pixels, and at the same time, can also perform CCSAO EO classification using an extended classification mode (including other available classification modes other than the above-mentioned four classification modes of horizontal, vertical, diagonal, and anti-diagonal), thereby improving the category accuracy of the CCSAO EO of the color component pixels and further improving the overall encoding performance of the video data.

[0136] Referring to Figure 17, Figure 17 is a schematic configuration diagram of another data processing device according to an embodiment of the present application. The data processing device may be a computer program (including program code) executed in a decoding device, for example, the data processing device is an application software in the decoding device. Referring to Figure 17, the data processing device 2 may include a decoding processing module 21, a second category identification module 22, and a pixel reconstruction module 23. Wherein: The decoding processing module 21 is configured to perform a decoding process on a block to be decoded in the video data, and obtain classification mode information corresponding to the block to be decoded. The classification mode information includes an extended co-located luminance reconstruction pixel corresponding to a color component pixel in the block to be decoded, and a target classification mode. The extended co-located luminance reconstruction pixel belongs to a target region centered on a realistic co-located luminance reconstruction pixel of the color component pixel. The second category identification module 22 is configured to identify a boundary category corresponding to the color component pixel according to the expanded co-located luminance reconstruction pixel and the target classification mode. The pixel reconstruction module 23 is configured to reconstruct color component pixels in the block to be decoded according to the compensation values ​​corresponding to the boundary categories.

[0137] In one or more embodiments, the decryption processing module 21 specifically: parsing positional syntax elements corresponding to color component pixels to obtain extended co-located luminance reconstruction pixels corresponding to the color component pixels; The method is configured to execute the steps of: parsing a mode syntax element corresponding to a color component pixel; obtaining a target classification mode corresponding to the color component pixel; and identifying the extended co-located luminance reconstruction pixel and the target classification mode as classification mode information corresponding to the block to be decoded.

[0138] In one or more embodiments, the decryption processing module 21 specifically: The method is configured to perform the steps of analyzing a common syntax element corresponding to a color component pixel, obtaining an extended co-located luminance reconstruction pixel and a target classification mode corresponding to the color component pixel, and identifying the extended co-located luminance reconstruction pixel and the target classification mode as classification mode information corresponding to the block to be decoded.

[0139] Here, the data processing device 2 further includes: analyzing a classification identification field in a sequence parameter set corresponding to the video data; determining that the video data uses extended co-located luminance reconstruction pixels if the classification identification field is a first identification value; and determining that the video data does not use extended co-located luminance reconstruction pixels if the classification identification field is a second identification value.

[0140] In an embodiment of the present application, the decoding device can perform CCSAO EO classification based on the extended co-located luminance reconstruction pixels corresponding to the color component pixels, and at the same time, can also perform CCSAO EO classification using an extended classification mode (including other available classification modes other than the above-mentioned four classification modes of horizontal, vertical, diagonal, and anti-diagonal), so as to improve the category accuracy of the CCSAO EO of the color component pixels and further improve the overall decoding performance of the video data.

[0141] According to the embodiment of the present application, each or all of the units of the data processing device 1 shown in FIG. 16 and the data processing device 2 shown in FIG. 17 may be integrated into one or more other units, or one (or some) of the units may be functionally divided into multiple smaller units, which can achieve the same operation without affecting the realization of the technical effect of the embodiment of the present application. The above units are divided based on logic functions, and in actual applications, the function of one unit may be realized by multiple units, or the function of multiple units may be realized by one unit. In other embodiments of the present application, the data processing device 2 may also include other units, and in actual applications, these functions may be realized in cooperation with other units and may be realized in cooperation with multiple units.

[0142] Further, referring to FIG. 18, FIG. 18 is a schematic diagram of a computer device according to an embodiment of the present application. As shown in FIG. 18, the computer device 1000 may be a terminal device or a server, but is not limited in the present application. For ease of understanding, the present application describes the computer device as a terminal device. The computer device 1000 may include a processor 1001, a network interface 1004, and a memory 1005. Furthermore, the computer device 1000 may include a user interface 1003 and at least one communication bus 1002. Here, the communication bus 1002 is used to realize the connection communication between these components. Here, the user interface 1003 may also include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a non-volatile memory such as at least one disk memory. The memory 1005 may optionally be at least one storage device separate from the processor 1001 described above. As shown in Figure 18, the memory 1005, which is a computer-readable storage medium, may include an operating system, a network communication module, a user interface module, and a device control application program.

[0143] Here, the network interface 1004 in the computer device 1000 can provide a network communication function, and the user interface 1003 can optionally include a display and a keyboard. In the computer device 1000 shown in Fig. 18, the network interface 1004 can provide a network communication function, while the user interface 1003 mainly provides an interface for input to the user.

[0144] When the computing device 1000 is an encoding device, the processor 1001 may be used to call a device control application program stored in the memory 1005 to realize a data processing method on the encoding device side relating to an embodiment of the present application.

[0145] When the computer device 1000 is a decoding device, the processor 1001 may be used to call a device control application program stored in the memory 1005 to realize a data processing method on the decoding device side relating to an embodiment of the present application.

[0146] It should be understood that the computer device 1000 described in the embodiment of the present application can execute the description of the data processing method in the embodiment corresponding to Fig. 9 or Fig. 15 described above, and can also execute the description of the data processing device 1 in the embodiment corresponding to Fig. 16 described above or the description of the data processing device 2 in the embodiment corresponding to Fig. 17 described above, and therefore will not be further described here. Also, the description of the beneficial effects using the same method will not be further described.

[0147] It should be noted here that the embodiment of the present application provides a computer-readable storage medium on which a computer program executed by the data processing device 1 or data processing device 2 mentioned above is stored. And the computer program includes computer instructions, and when the processor executes the computer instructions, the data processing method in the embodiment corresponding to FIG. 9 or FIG. 15 above can be executed, so no further description will be given here. Also, the beneficial effects of the same method will not be described any further. For technical details not disclosed in the embodiment of the computer-readable storage medium of the present application, please refer to the description of the embodiment of the method of the present application. As an example, the computer instructions may be located and executed on one computing device, or may be executed on multiple computing devices located in one place, or may be executed on multiple computing devices located in multiple places and interconnected via a communication network, and the multiple computing devices located in multiple places and interconnected via a communication network may constitute a blockchain system.

[0148] In addition, the embodiments of the present application further provide a computer program product or a computer program including computer instructions, which may be stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and execute the computer instructions to cause the computer device to execute the data processing method described in the embodiment corresponding to FIG. 9 or FIG. 15 above, so no further description is given here. Also, no further description is given about the beneficial effects of the same method. For technical details not disclosed in the embodiments of the computer program product or computer program of the present application, please refer to the description of the embodiments of the method of the present application.

[0149] In addition, the above-mentioned embodiments of the methods are described as a combination of a series of operations for the sake of simplicity, but according to the present application, some steps may adopt other orders or may be performed in parallel, so it should be recognized by those skilled in the art that the present application is not limited by the order of operations described. Next, it should be recognized by those skilled in the art that all the embodiments described in this specification are preferred embodiments, and the related operations and modules are not necessarily required for the present application.

[0150] The steps in the method according to the embodiments of the present application can be reordered, merged and reduced according to practical needs.

[0151] The modules in the apparatus according to the embodiments of the present application can be merged, split and reduced according to actual needs.

[0152] It should be understood by those skilled in the art that the implementation of all or part of the steps of the method according to the above-mentioned embodiments can be achieved by instructing relevant hardware by a computer program, and the computer program can be stored in a computer-readable storage medium, and when the program is executed, the program can include the steps of each of the above-mentioned method embodiments. Here, the storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), etc.

[0153] It is clear that the above disclosure is merely a preferred embodiment of the present application, and cannot limit the technical scope of the present application. Any equivalent changes according to the claims of the present application are still considered to belong to the technical scope of the present application.

Claims

1. 1. A data processing method performed by a computing device, comprising: identifying classification mode information corresponding to a block to be coded in the video data, the classification mode information including an extended co-located luminance reconstruction pixel corresponding to a color component pixel in the block to be coded and a target classification mode, the extended co-located luminance reconstruction pixel belonging to a target region centered on a realistic co-located luminance reconstruction pixel of the color component pixel; determining a boundary category corresponding to the color component pixel according to the expanded co-located luminance reconstruction pixel and the target classification mode; compensating reconstructed pixels of the color component pixels based on the boundary category, obtaining compensated reconstructed pixels, and performing an encoding process on the encoding target block based on the compensated reconstructed pixels.

2. The step of identifying classification mode information corresponding to a block to be coded in the video data includes: obtaining color component pixels included in the encoding target block, and obtaining realistic co-located luminance reconstructed pixels corresponding to the color component pixels; identifying a target area having a target size with the realistic co-located luminance reconstructed pixel as a center of the area, and identifying an expanded co-located luminance reconstructed pixel corresponding to the color component pixel within the target area; 2. The method of claim 1 , further comprising: obtaining a candidate classification mode set corresponding to the color component pixel; identifying a target classification mode corresponding to the color component pixel in the candidate classification mode set; and adding the expanded co-located luminance reconstruction pixel and the target classification mode to classification mode information corresponding to the block to be encoded.

3. The step of identifying expanded co-located luminance reconstruction pixels corresponding to the color component pixels within the target region comprises: identifying luminance reconstruction pixels other than vertex positions in the target region as candidate luminance pixel sets corresponding to the color component pixels; performing a rate-distortion optimization process on the color component pixels for each luma reconstructed pixel in the candidate luma pixel set to obtain a first rate-distortion loss corresponding to each luma reconstructed pixel in the candidate luma pixel set; and identifying a luma reconstructed pixel corresponding to a first rate-distortion loss that is minimal as the extended co-located luma reconstructed pixel corresponding to the color component pixel.

4. The step of obtaining a candidate classification mode set corresponding to the color component pixel and identifying a target classification mode corresponding to the color component pixel from the candidate classification mode set includes: identifying a candidate classification mode set corresponding to the color component pixels according to a position of the expanded co-located luminance reconstruction pixel in the target area, the coverage of classification modes in the candidate classification mode set being equal to or smaller than the target area, and the candidate classification mode set including a horizontal classification mode and a vertical classification mode; performing a rate-distortion optimization process on the color component pixels for each classification mode in the candidate classification mode set to obtain a second rate-distortion loss corresponding to each classification mode in the candidate classification mode set; and identifying a classification mode corresponding to a second rate-distortion loss that is minimal as a target classification mode corresponding to the color component pixel.

5. the expanded co-located luminance reconstruction pixel is any luminance reconstruction pixel in the target region, The method further comprises:

3. The method of claim 2, further comprising the step of: if the candidate classification mode set includes a classification mode whose coverage range is larger than the target area, filling boundary pixels in the target area with adjacent areas of the target area to obtain a mode coverage range for identifying the target classification mode in the candidate classification mode set.

6. 3. The method of claim 2, wherein when the expanded co-located luminance reconstruction pixel is any luminance reconstruction pixel in the target region, the candidate classification mode set includes both a base classification mode and an expanded classification mode, or includes only an expanded classification mode, the base classification modes include a horizontal classification mode, a vertical classification mode, a diagonal classification mode, and an inverse diagonal classification mode, and the expanded classification mode includes an available classification mode other than the base classification mode.

7. The method of claim 1 , wherein the color component pixels include luma component pixels or chroma component pixels.

8. When the color component pixels include a luminance component pixel, The step of identifying a boundary category corresponding to the color component pixel according to the expanded co-located luminance reconstruction pixel and the target classification mode includes: identifying a first neighboring pixel and a second neighboring pixel corresponding to the expanded co-located luminance reconstruction pixel according to the target classification mode; obtaining a first difference value between the first neighboring pixel and the expanded co-located luminance reconstruction pixel, and obtaining a second difference value between the second neighboring pixel and the expanded co-located luminance reconstruction pixel; obtaining a first co-located chroma pixel and a second co-located chroma pixel corresponding to the realistic co-located luma reconstruction pixel; determining a band category to which the luminance component pixel belongs based on the expanded co-located luminance reconstructed pixel, the first co-located chroma pixel, and the second co-located chroma pixel; and determining a boundary category corresponding to the color component pixel based on the band category, the first difference value, and the second difference value.

9. The step of identifying a band category to which the luminance component pixel belongs based on the expanded co-located luminance reconstructed pixel, the first co-located chroma pixel, and the second co-located chroma pixel, further comprising: obtaining a first product of the expanded co-located luminance reconstruction pixel and a total number of band categories on a luminance component, obtaining a second product of the first co-located chroma pixel and a total number of band categories on a first chroma component, and obtaining a third product of the second co-located chroma pixel and a total number of band categories on a second chroma component; and determining a band category to which the luminance component pixel belongs based on a bit depth of a pixel value and a numerical relationship between the first product, the second product, and each of the third products.

10. When the color component pixels include a luminance component pixel, The step of identifying a boundary category corresponding to the color component pixel according to the expanded co-located luminance reconstruction pixel and the target classification mode includes: identifying a first neighboring pixel and a second neighboring pixel corresponding to the expanded co-located luminance reconstruction pixel according to the target classification mode; obtaining a first difference value between the first neighboring pixel and the expanded co-located luminance reconstruction pixel, and obtaining a second difference value between the second neighboring pixel and the expanded co-located luminance reconstruction pixel; obtaining a first co-located chroma pixel and a second co-located chroma pixel corresponding to the realistic co-located luma reconstruction pixel; determining a band category to which the luminance component pixel belongs based on the realistic co-located luminance reconstructed pixel, the first co-located chroma pixel, and the second co-located chroma pixel; and determining a boundary category corresponding to the color component pixel based on the band category, the first difference value, and the second difference value.

11. When the color component pixels include a luminance component pixel, The step of identifying a boundary category corresponding to the color component pixel according to the expanded co-located luminance reconstruction pixel and the target classification mode includes: identifying third and fourth neighboring pixels corresponding to the realistic co-located luminance reconstruction pixel according to the target classification mode; obtaining a third difference value between the third adjacent pixel and the realistic co-located luminance reconstructed pixel, and obtaining a fourth difference value between the fourth adjacent pixel and the realistic co-located luminance reconstructed pixel; obtaining a first co-located chroma pixel and a second co-located chroma pixel corresponding to the realistic co-located luma reconstruction pixel; determining a band category to which the luminance component pixel belongs based on the expanded co-located luminance reconstructed pixel, the first co-located chroma pixel, and the second co-located chroma pixel; and determining a boundary category corresponding to the color component pixel based on the band category, the third difference value, and the fourth difference value.

12. The step of performing an encoding process on the encoding target block based on the compensated reconstructed pixels includes: setting a position syntax element to the expanded co-located luminance reconstruction pixels and a mode syntax element to the target classification mode; and determining a compensation value corresponding to the boundary category, and obtaining an encoded bitstream for the current block based on the position syntax element, the mode syntax element, and the compensation value.

13. The step of performing an encoding process on the encoding target block based on the compensated reconstructed pixels includes: setting common syntax elements for the extended co-located luminance reconstruction pixels and the target classification mode; and determining a compensation value corresponding to the boundary category, and obtaining an encoded bitstream for the current block based on the common syntax element and the compensation value.

14. 1. A data processing method performed by a computing device, comprising: performing a decoding process on a block to be decoded in the video data, and obtaining classification mode information corresponding to the block to be decoded, the classification mode information including an extended co-located luminance reconstructed pixel corresponding to a color component pixel in the block to be decoded and a target classification mode, the extended co-located luminance reconstructed pixel belonging to a target area centered on a realistic co-located luminance reconstructed pixel of the color component pixel; determining a boundary category corresponding to the color component pixel according to the expanded co-located luminance reconstruction pixel and the target classification mode; reconstructing color component pixels in the block to be decoded according to compensation values ​​corresponding to the boundary categories.

15. The step of performing a decoding process on a decoding target block in video data and obtaining classification mode information corresponding to the decoding target block includes: parsing a positional syntax element corresponding to the color component pixel to obtain an extended co-located luminance reconstruction pixel corresponding to the color component pixel; 15. The method of claim 14, further comprising: parsing a mode syntax element corresponding to the color component pixel to obtain a target classification mode corresponding to the color component pixel; and identifying the extended co-located luminance reconstruction pixel and the target classification mode as classification mode information corresponding to the block to be decoded.

16. The step of performing a decoding process on a decoding target block in video data and obtaining classification mode information corresponding to the decoding target block includes:

15. The method of claim 14, further comprising: parsing a common syntax element corresponding to the color component pixel to obtain an extended co-located luminance reconstruction pixel and a target classification mode corresponding to the color component pixel; and identifying the extended co-located luminance reconstruction pixel and the target classification mode as classification mode information corresponding to the block to be decoded.

17. moreover, analyzing a classification identification field in a sequence parameter set corresponding to the video data; determining that the video data uses extended co-located luminance reconstruction pixels if the classification identification field is a first identification value; 17. The method of claim 14, further comprising: if the classification identification field is a second identification value, determining that the extended co-located luminance reconstruction pixels are not used in the video data.

18. a memory for storing a computer program; A processor connected to the memory, the processor causing, when the computer program is invoked, to perform the method according to any one of claims 1 to 11 or the method according to any one of claims 14 to 16.

19. A computer program product for implementing the method according to any one of claims 1 to 11 or for carrying out the method according to any one of claims 14 to 16, when the computer program product is executed by a processor.

Images