A video coding cross-component prediction method and system based on Gaussian process regression

Abstract

This invention discloses a video coding cross-component prediction method and system based on Gaussian process regression. The method includes: obtaining reference pixels of reconstructed blocks from the luma component, the reference pixels being used as input to a cross-component prediction model; selecting a reference region based on the reference pixels using a codec, and then selecting a first reconstructed pixel based on the reference region; establishing a prediction model for luma pixels of different intensities based on a given first threshold and the current pixel intensity, and then selecting a second reconstructed pixel based on the prediction model; constructing a Gaussian regression model based on the first and second reconstructed pixels; and performing cross-component prediction on the current coding unit based on the Gaussian regression model to obtain the video coding cross-component prediction result. This invention can improve prediction accuracy and solve the correlation problem between luma and chroma in video coding, and can be widely applied in the field of computer technology.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a method and system for cross-component prediction in video coding based on Gaussian process regression. Background Technology

[0002] Traditional cross-component linear regression models, such as the Cross-Component Linear Model (CCLM), often only capture simple linear relationships when dealing with the relationship between luminance and chrominance components, making it difficult to handle complex nonlinear relationships. In video encoding, there is a correlation between luminance and chrominance; therefore, how to deeply capture the complex relationship between luminance and chrominance components in video signals has become an urgent technical problem to be solved. Summary of the Invention

[0003] The main objective of this invention is to propose a video coding cross-component prediction method and system based on Gaussian process regression, which can improve the accuracy of prediction and solve the correlation problem between luminance and chrominance in the video coding process.

[0004] To achieve the above objectives, one aspect of this invention proposes a video coding cross-component prediction method based on Gaussian process regression, comprising the following steps:

[0005] Reference pixels for the reconstructed block are obtained from the luminance component, and these reference pixels are used as input to the cross-component prediction model.

[0006] Based on the reference pixel, a reference region is selected by a codec, and then a first reconstructed pixel is selected based on the reference region.

[0007] Based on a given first threshold, a prediction model can be established for brightness pixels of different intensities based on the size of the first threshold and the current pixel intensity, and then the second reconstructed pixel can be selected through the prediction model.

[0008] Based on the first reconstructed pixel and the second reconstructed pixel, a Gaussian regression model is constructed;

[0009] Based on the Gaussian regression model, cross-component prediction is performed on the current coding unit to obtain the video coding cross-component prediction result.

[0010] In some embodiments, obtaining the reference pixel of the reconstructed block from the luminance component includes the following steps:

[0011] Obtain the luminance reconstruction cache and chrominance reconstruction cache. In the YUV420 format, if the chrominance area corresponds to multiple luminance blocks, then take the luminance encoding unit corresponding to the center point.

[0012] The brightness reconstruction pixels on the top and left sides are downsampled to match the corresponding chromaticity components;

[0013] Specifically, when downsampling the brightness reference pixel, the selection of the sampling filter is determined by the sequence parameter set or the bitstream syntax.

[0014] In some embodiments, the step of selecting the reference region based on the reference pixel using a codec includes the following steps:

[0015] Specify the width and height values ​​for the reference region, and perform pixel reconstruction using the top and left rows of the reference region; expand the pixels from the current WxH coding unit to obtain the reconstructed luminance pixels;

[0016] Select a reference area by using the reconstructed brightness pixels from the top or left.

[0017] In some embodiments, the step of establishing a prediction model for brightness pixels of different intensities based on a given first threshold and the current pixel intensity includes the following steps:

[0018] In the process of selecting the reference area, the mean value of the luminance component is calculated as the first threshold, and the luminance component is divided into two parts according to the first threshold: the part that is lower than or equal to the first threshold and the part that is higher than the first threshold.

[0019] The two brightness components are used to train a Gaussian regression model; the portion below or equal to the first threshold is used to train the first model, and the portion above the first threshold is used to train the second model.

[0020] During cross-component prediction, the appropriate model is selected for prediction based on the current brightness pixel of the statement; specifically: if the brightness value of the current pixel is lower than or equal to the first threshold, the first model is used for prediction; if the brightness value of the current pixel is higher than the first threshold, the second model is used for prediction.

[0021] In some embodiments, constructing a Gaussian regression model based on the first reconstructed pixel and the second reconstructed pixel includes the following steps:

[0022] The expression for the Gaussian process regression model is defined as: C = f(Y) = GP(μ(Y), k(Y,Y')), where μ(Y) is the mean function, k(Y,Y') is the covariance function, Y is the luminance pixel, C is the juxtaposed chrominance pixel; GP() represents the Gaussian process;

[0023] Based on the Gaussian process regression model, the Gaussian process is initialized and configured to determine the initialization results of the mean function and the covariance function.

[0024] Based on the Gaussian process regression model, the Gaussian process model is conditionalized, and then the posterior distribution of the Gaussian process is updated based on the result of the conditionalization.

[0025] In some embodiments, the step of conditionalizing the Gaussian process model based on the Gaussian process regression model, and then updating the posterior distribution of the Gaussian process based on the result of the conditionalization, includes the following steps:

[0026] Based on the newly added sample points, the mean function is updated according to the observed data. The update formula for the mean function is: μ post (Y)=μ prior (Y)+K y,Y (K Y,Y +σ 2 I) -1 (C-μ prior (Y)), where μ post (Y) represents the updated mean function; μ prior (T) represents the prior mean function; K y,Y K represents the covariance matrix between the new sample point y and the existing sample points Y; Y,Y σ represents the covariance matrix among existing sample points Y; σ represents the noise parameter; I represents the identity matrix; C represents the new observation.

[0027] When there is no observation data, the covariance matrix is ​​a diagonal matrix. When new sample points are included in the model, the covariance matrix is ​​updated using the Cholesky decomposition method. When new observation data is available, the new sample points are included in the model by updating the Cholesky decomposition matrix.

[0028] The expression for the autocovariance a of the new sample points is a = k(y,y) + σ, where k(y,y) is the autocovariance of the new sample y, and σ is the noise parameter.

[0029] The expression for the covariance vector b between the new sample points and the existing sample points is: b = [k(y, y1), k(y, y2), ..., k(y, y3)]. n )] T , where y n These are the brightness values ​​of existing sample points;

[0030] During the update of the Cholesky decomposition matrix L, the expression for the autocovariance term a′ of the new sample points is: Therefore, the matrix L is updated as follows:

[0031] L 11 =a′

[0032] L 12 =-a′·b·L

[0033] L 21 =-L·b T ·a′

[0034] L 22 =L+L·b T ·a′·b·L

[0035] Among them, b T ·L·b is the inner product of the covariance vectors of existing sample points and new sample points.

[0036] In some embodiments, the step of performing cross-component prediction of the current coding unit based on the Gaussian regression model includes the following steps:

[0037] Pixel prediction is performed using the updated mean function, and the covariance matrix between the new sample point Y* and the existing sample points Y is calculated, as well as the covariance matrix between the existing sample points Y.

[0038] Determine the mean function with predicted values ​​to obtain the cross-component prediction results;

[0039] The expression for the mean function with predicted values ​​is:

[0040]

[0041] The expression for the prediction result across components is:

[0042] predC=μ ppst (Y * )

[0043] Where, μ post () represents the mean function with predicted values; μ prior () represents the prior mean function; Represents the prediction point Y * The covariance matrix with existing sample points Y; K Y，Y σ represents the covariance matrix among existing sample points Y; I represents the noise parameter; C represents the identity matrix; and predC represents the prediction result across components.

[0044] Another aspect of this invention provides a video coding cross-component prediction system based on Gaussian process regression, comprising:

[0045] The first module is used to obtain reference pixels for the reconstructed blocks from the luminance component, and the reference pixels are used as input to the cross-component prediction model:

[0046] The second module is used to select a reference region based on the reference pixel using a codec, and then select a first reconstructed pixel based on the reference region.

[0047] The third module is used to establish a prediction model for brightness pixels of different intensities based on a given first threshold and the size of the first threshold and the current pixel intensity, and then select the second reconstructed pixel through the prediction model.

[0048] The fourth module is used to construct a Gaussian regression model based on the first reconstructed pixel and the second reconstructed pixel;

[0049] The fifth module is used to perform cross-component prediction on the current coding unit based on the Gaussian regression model to obtain the cross-component prediction result of video coding.

[0050] To achieve the above objectives, another aspect of the present invention provides an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method described above.

[0051] To achieve the above objectives, another aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods described above.

[0052] This invention also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform the aforementioned method.

[0053] The embodiments of this invention include at least the following beneficial effects: This invention provides a video coding cross-component prediction method and system based on Gaussian process regression. This scheme obtains reference pixels for reconstructed blocks from the luminance component, and these reference pixels are used as input to a cross-component prediction model. Based on the reference pixels, a reference region is selected by the codec, and then a first reconstructed pixel is selected based on this reference region. Based on a given first threshold, a prediction model is established for luminance pixels of different intensities based on the size of the first threshold and the current pixel intensity, and then a second reconstructed pixel is selected using this prediction model. A Gaussian regression model is constructed based on the first and second reconstructed pixels. Cross-component prediction is performed on the current coding unit based on the Gaussian regression model to obtain the video coding cross-component prediction result. The embodiments of this invention can improve the accuracy of prediction and solve the correlation problem between luminance and chrominance during video coding. Attached Figure Description

[0054] Figure 1 This is a schematic diagram of an implementation environment provided by an embodiment of the present invention;

[0055] Figure 2 This is a flowchart of the overall steps provided in the embodiments of the present invention;

[0056] Figure 3 This is a flowchart illustrating the specific implementation steps provided in the embodiments of the present invention;

[0057] Figure 4 This is a schematic diagram of the encoding region provided in an embodiment of the present invention;

[0058] Figure 5 This is an example diagram of the reference area selected in the embodiments of the present invention;

[0059] Figure 6 This is an example of multi-model extension based on brightness pixel intensity provided in the embodiments of the present invention;

[0060] Figure 7 This is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of this invention; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this invention as detailed in the appended claims.

[0062] It is understood that the terms “first,” “second,” etc., used in this invention may be used herein to describe various concepts, but unless specifically stated otherwise, these concepts are not limited by these terms. These terms are used only to distinguish one concept from another. For example, first information may also be referred to as second information without departing from the scope of embodiments of the invention, and similarly, second information may also be referred to as first information. Depending on the context, the words “if,” “when,” or “in response to determination” as used herein may be interpreted as “when…” or “when…” or “in response to determination.”

[0063] The terms “at least one,” “multiple,” “each,” “any,” etc., used in this invention, “at least one” includes one, two, or more than two; “multiple” includes two or more than two; “each” refers to each of the corresponding multiple; and “any” refers to any one of the multiple.

[0064] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein is for the purpose of describing embodiments of the invention only and is not intended to limit the invention.

[0065] Before providing a detailed description of the embodiments of the present invention, some related technologies involved in the embodiments of the present invention will be described first, as follows:

[0066] I. Cross-Component Linear Model (CCLM):

[0067] Cross-Component Linear Model (CCLM) is a method used in video coding for chroma prediction, which obtains chroma values ​​from reconstructed luminance pixels through linear transformation.

[0068] The specific implementation process of CCLM involves the following steps:

[0069] 1. Use the reconstructed luminance component values ​​(obtained through a downsampling process) to predict chromaticity values.

[0070] 2. The prediction function is defined as pred_C(i,j)=α*rec_L'(i,j)+β, where pred_C is the predicted chromaticity value, and rec_L' represents the reconstructed value of the luminance component.

[0071] 3. The coefficients α and β are determined based on the luminance and chrominance values ​​of neighboring coding units (CUs), and can be calculated using the following formula:

[0072] α = (Ya - Yb) / (Xa - Xb),

[0073] β = Yb - α * Xb,

[0074] Where Ya is the maximum downsampled luminance pixel value, Xa is the chrominance value corresponding to Ya, Yb is the minimum downsampled luminance pixel value, and Xb is the chrominance value corresponding to Yb.

[0075] In addition to using all the top and left luminance reference pixels to calculate the linear model parameters, CCLM provides two additional modes called LMA and LML modes. LMA mode uses only the top reference pixels, while LML mode uses only the left reference pixels. This increases the flexibility of chroma prediction and allows adaptation to different video content. The calculation of the linear model parameters occurs not only at the encoder end but also at the decoder. Therefore, the values ​​of α and β do not need to be recorded in the bitstream, reducing additional bitstream overhead.

[0076] II. Convolutional Cross-Component Model (CCCM):

[0077] The Convolutional Cross-Component Model (CCCM) is a convolutional model for chromaticity prediction. Similar to the Cross-Component Linear Model (CCLM), it is based on the idea of ​​predicting chromaticity pixels from reconstructed luminance pixels. CCCM considers the relationship between luminance and chromaticity components and uses a downsampled version of the luminance reconstruction block to match the size of the chromaticity block.

[0078] The basic steps of the module are as follows:

[0079] 1. Before predicting chroma pixels, the luminance reconstruction block is downsampled to match the size of the chroma block.

[0080] 2. Apply a 7-tap convolutional filter to process the luminance samples and calculate the chromaticity prediction pixels.

[0081] The specific prediction function is expressed in the following form:

[0082] predChromaVal=c0 C+c1 N+c2 S+c3 E+c4 W+c5 P+c6 B

[0083] Where C is the luminance sample at the corresponding position of the current chromaticity sample, N, S, E, W are the neighboring samples of the current luminance sample, and P is a non-linear term, calculated as follows:

[0084] P = (C * C + midVal) >> bitDepth

[0085] Here, midVal is an offset term that represents a scalar offset between the input and the output, and is usually set to an intermediate chroma value.

[0086] 3. The filter coefficients c_n are calculated by minimizing the mean square error (MSE) between the predicted and reconstructed chromaticity samples. This can be accomplished by solving the equations using numerical methods such as LDL decomposition.

[0087] Similar to CCLM, CCCM offers the option to use a single model or multiple model variants. The multiple model variant uses two models: one derived for samples above the average brightness reference value, and another for the remaining samples (below the average brightness reference value). The multiple model CCCM mode can be selected if the prediction unit (PU) has at least 128 available reference samples.

[0088] The reference region consists of 2 or 6 rows of chromaticity samples above and to the left of the PU. The number of neighbor samples used to derive the CCCM model parameters for a single-model CCCM is determined by the template cost. For multi-model CCCM, the two candidate models derive their average values ​​using either 6 rows of adjacent luminance samples or luminance samples corresponding to the current chromaticity block position, dividing the samples into two groups. The cost is calculated by applying candidate CCPs (2 or 6 rows) to the template and calculating the total absolute difference (SAD) between the CCP prediction samples and the reconstructed samples in the template.

[0089] The video coding cross-component prediction method and system based on Gaussian process regression provided in this invention relates to the field of computer technology. The video coding cross-component prediction method based on Gaussian process regression provided in this invention can be applied to a terminal, a server, or software running on a terminal or server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or vehicle terminal, but is not limited to these. The server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network. The software can be an application implementing the video coding cross-component prediction method based on Gaussian process regression, but is not limited to the above forms.

[0090] This invention can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This invention can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This invention can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0091] like Figure 1 The diagram shown is a schematic representation of an implementation environment provided by an embodiment of the present invention. (Refer to...) Figure 1The implementation environment includes at least one terminal 102 and a server 101. The terminal 102 and the server 101 can be connected via a network, either wirelessly or via a wired connection, to complete data transmission and exchange.

[0092] Server 101 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0093] Additionally, server 101 can also be a node server in a blockchain network. Blockchain is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms.

[0094] Terminal 102 can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, etc. It can also be a vehicle-mounted terminal of the various device types described above, but is not limited to these. Terminal 102 and server 101 can be directly or indirectly connected via wired or wireless communication, and this embodiment of the invention does not impose any limitations.

[0095] Exemplary based on Figure 1 The implementation environment shown in this embodiment of the invention provides a video coding cross-component prediction method based on Gaussian process regression. The following description uses the application of this video coding cross-component prediction method based on Gaussian process regression in server 101 as an example. It can be understood that this method can also be applied to terminal 102.

[0096] Reference Figure 2 , Figure 2 The flowchart illustrates a Gaussian process regression-based video coding cross-component prediction method applied to a server, as provided in this embodiment of the invention. The execution entity of this method can be any of the aforementioned computer devices (including servers or terminals). (Refer to...) Figure 2 The method may include the following steps:

[0097] Reference pixels for the reconstructed block are obtained from the luminance component, and these reference pixels are used as input to the cross-component prediction model.

[0098] Based on the reference pixel, a reference region is selected by a codec, and then a first reconstructed pixel is selected based on the reference region.

[0099] Based on a given first threshold, a prediction model can be established for brightness pixels of different intensities based on the size of the first threshold and the current pixel intensity, and then the second reconstructed pixel can be selected through the prediction model.

[0100] Based on the first reconstructed pixel and the second reconstructed pixel, a Gaussian regression model is constructed;

[0101] Based on the Gaussian regression model, cross-component prediction is performed on the current coding unit to obtain the video coding cross-component prediction result.

[0102] In some embodiments, obtaining the reference pixel of the reconstructed block from the luminance component includes the following steps:

[0103] Obtain the luminance reconstruction cache and chrominance reconstruction cache. In the YUV420 format, if the chrominance area corresponds to multiple luminance blocks, then take the luminance encoding unit corresponding to the center point.

[0104] The brightness reconstruction pixels at the top and left are downsampled to match the corresponding chromaticity components;

[0105] Specifically, when downsampling the brightness reference pixel, the selection of the sampling filter is determined by the sequence parameter set or the bitstream syntax.

[0106] In some embodiments, the selection of the reference region based on the reference pixel using a codec includes the following steps:

[0107] Specify the width and height values ​​for the reference region, and perform pixel reconstruction using the top and left rows of the reference region; expand the pixels from the current WxH coding unit to obtain the reconstructed luminance pixels;

[0108] Select a reference area by using the reconstructed brightness pixels from the top or left.

[0109] In some embodiments, the step of establishing a prediction model for brightness pixels of different intensities based on a given first threshold and the current pixel intensity includes the following steps:

[0110] In the process of selecting the reference area, the mean value of the luminance component is calculated as the first threshold, and the luminance component is divided into two parts according to the first threshold: the part that is lower than or equal to the first threshold and the part that is higher than the first threshold.

[0111] The two brightness components are used to train a Gaussian regression model; the portion below or equal to the first threshold is used to train the first model, and the portion above the first threshold is used to train the second model.

[0112] During cross-component prediction, the appropriate model is selected for prediction based on the current brightness pixel of the statement; specifically: if the brightness value of the current pixel is lower than or equal to the first threshold, the first model is used for prediction; if the brightness value of the current pixel is higher than the first threshold, the second model is used for prediction.

[0113] In some embodiments, constructing a Gaussian regression model based on the first reconstructed pixel and the second reconstructed pixel includes the following steps:

[0114] The expression for the Gaussian process regression model is defined as: C = f(Y) = GP(μ(Y), k(Y,Y')), where μ(Y) is the mean function, k(Y,Y') is the covariance function, Y is the luminance pixel, C is the juxtaposed chrominance pixel; GP() represents the Gaussian process;

[0115] Based on the Gaussian process regression model, the Gaussian process is initialized and configured to determine the initialization results of the mean function and the covariance function.

[0116] Based on the Gaussian process regression model, the Gaussian process model is conditionalized, and then the posterior distribution of the Gaussian process is updated based on the result of the conditionalization.

[0117] In some embodiments, the step of conditionalizing the Gaussian process model based on the Gaussian process regression model, and then updating the posterior distribution of the Gaussian process based on the result of the conditionalization, includes the following steps:

[0118] Based on the newly added sample points, the mean function is updated according to the observed data. The update formula for the mean function is: μ post (Y)=μ prior (Y)+K y，Y (K Y，Y +σ 2 I) -1 (C-μ prior (Y)), where μ post (Y) represents the updated mean function; μ prior (Y) represents the prior mean function; K y，Y K represents the covariance matrix between the new sample point y and the existing sample points Y; Y，Y σ represents the covariance matrix among existing sample points Y; σ represents the noise parameter; I represents the identity matrix; C represents the new observation.

[0119] When there is no observation data, the covariance matrix is ​​a diagonal matrix. When new sample points are included in the model, the covariance matrix is ​​updated using the Cholesky decomposition method. When new observation data is available, the new sample points are included in the model by updating the Cholesky decomposition matrix.

[0120] The expression for the autocovariance a of the new sample points is a = k(y, y) + σ, where k(y, y) is the autocovariance of the new sample y, and σ is the noise parameter.

[0121] The expression for the covariance vector b between the new sample points and the existing sample points is: b = [k(y, y1), k(y, y2), ..., k(y, y3)]. n )] T , where y n These are the brightness values ​​of existing sample points;

[0122] During the update of the Cholesky decomposition matrix L, the expression for the autocovariance term a′ of the new sample points is: Therefore, the matrix L is updated as follows:

[0123] L 11 =a′

[0124] L 12 =-a′·b·L

[0125] L 21 =-L·b T ·a′

[0126] L 22 =L+L·b T ·a′·b·L

[0127] Among them, b T ·L·b is the inner product of the covariance vectors of existing sample points and new sample points.

[0128] In some embodiments, the step of performing cross-component prediction of the current coding unit based on the Gaussian regression model includes the following steps:

[0129] Pixel prediction is performed using the updated mean function, and the covariance matrix between the new sample point Y* and the existing sample points Y is calculated, as well as the covariance matrix between the existing sample points Y.

[0130] Determine the mean function with predicted values ​​to obtain the cross-component prediction results;

[0131] The expression for the mean function with predicted values ​​is:

[0132]

[0133] The expression for the prediction result across components is:

[0134] predC=μ post (Y * )

[0135] Where, μpost () represents the mean function with predicted values; μ prior () represents the prior mean function; Represents the prediction point Y * The covariance matrix with existing sample points Y; K Y,Y σ represents the covariance matrix among existing sample points Y; I represents the noise parameter; C represents the identity matrix; and predC represents the prediction result across components.

[0136] The specific implementation process of this invention will be described in detail below using a specific application scenario as an example:

[0137] The cross-component prediction method based on Gaussian process regression described in this invention is mainly used in the intra-frame image reconstruction process in video coding. This method predicts the chrominance component using the reconstructed luminance component, thereby achieving efficient cross-component prediction. Figure 3 As shown, on both the encoder and decoder sides, this method sequentially executes the same S100, S200, S300, S400, and S500 processes to ensure the consistency and accuracy of the encoding and decoding prediction algorithm.

[0138] A.S100 Reference Area

[0139] The primary responsibility of the S100 module is to obtain reference pixels for the reconstructed blocks from the luma component. These reference pixels are the input to the cross-component prediction model used to predict the chroma component. In this process:

[0140] 1. Obtain the luminance reconstruction buffer and chrominance reconstruction buffer. The current encoding context is as follows: Figure 4 As shown. In formats such as YUV420, if a chroma area corresponds to multiple luma blocks, the luma coding unit (Luma CU) corresponding to the center point is taken.

[0141] 2. To ensure that the luma and chroma blocks have the same width and height, the luma reconstructed pixels on the top and left sides can be downsampled to match the corresponding chroma components. When downsampling the luma reference pixels, the selection of the sampling filter can be specified by the Sequence Parameter Set (SPS) or by the bitstream syntax.

[0142] B.S200 Reference Area Selection

[0143] The S200 module will complete the selection of the encoding region for the codec implementation, and the use of reference pixels will follow a selectable method, such as... Figure 5 As shown, it includes:

[0144] 1. The reference area can be specified with certain width and height values, and pixels are reconstructed using n rows from the top and left. Simultaneously, it can be expanded from the current WxH coding unit (CU) by a certain amount, such as by 0 pixels, W pixels, or H pixels.

[0145] In the implementation, the reference area consists of 2 or 6 rows of chromaticity samples above and to the left of the PU, which are aligned with the reference area obtained by the CCCM method.

[0146] 2. The reference area can be selected in the following ways, such as... Figure 5 As shown:

[0147] Using only the reconstructed brightness pixels from the top or left side is called Top (T, top mode) and Left (L, left mode), respectively. Using all top and left reference pixels is denoted as LT (Left & Top) mode.

[0148] C.S300 Multi-Model Extension

[0149] In addition to using reference region templates in different directions from S200, this algorithm is also applicable to multi-model extension implementations. Based on a given threshold, prediction models can be established for pixels of different brightness intensities according to the threshold value and the current pixel intensity, further improving the accuracy of model predictions.

[0150] 1. During the process of acquiring the reference area in S200, the mean value of the luminance component can be calculated and used as a threshold (modelThr). Based on the threshold, the luminance component is divided into two parts: the part below or equal to the threshold (Model 1) and the part above the threshold (Model 2), such as... Figure 6 As shown, the horizontal axis (Colocated Luma Value) represents the luminance value, and the vertical axis (Chroma Value) represents the chromaticity value.

[0151] 2. During the training of the S400 prediction model, two models are trained based on the threshold.

[0152] 1) For Model 1, only samples with luminance components below or equal to the threshold are used to train the Gaussian process regression model.

[0153] 2) For Model 2, only samples with luminance components above the threshold are used to train the Gaussian process regression model.

[0154] 3. During the cross-component prediction process of S500, the corresponding model is selected for prediction of the current brightness pixel of the statement.

[0155] 1) If the brightness value of the current pixel is lower than or equal to the threshold modelThr, then use Model 1 for prediction.

[0156] 2) If the brightness value of the current pixel is higher than the threshold modelThr, then use Model 2 for prediction.

[0157] Training the D.S400 Gaussian process regression model

[0158] Based on the reconstructed pixels selected by S200 and S300, a Gaussian regression model can be established.

[0159] Gaussian Process Regression (GPR) is a nonparametric regression method based on Bayesian inference. Its core idea is to model the spatial relationship between component pixels as a stochastic process. Any finite-dimensional distribution of this process is a Gaussian distribution. Using the mean function of the Gaussian stochastic process, we can predict and output chroma pixels based on the input luminance pixels.

[0160] The training process of a Gaussian regression model is the process of conditionalizing new samples. The Gaussian process regression model can continuously update its posterior distribution based on the input cross-component pixel pairs [Luma, Chroma] to capture the data fitting relationship.

[0161] Let Y be the luminance pixel and C be its juxtaposed chrominance pixel. The Gaussian process regression model can be expressed as:

[0162] C = f(Y) = GP(μ(Y), k(Y, Y'))

[0163] Where μ(Y) is the mean function and k(Y,Y') is the covariance function (or kernel function), used to describe the similarity between different input points.

[0164] 1. Gaussian process initialization

[0165] 1) Initialization of the mean function

[0166] The mean function is typically set to a constant or zero, representing a prior hypothesis about the input data. In implementation, it can be optionally...

[0167] μ(Y)=0

[0168] Alternatively, the mean chromaticity value of the template region can be used as the mean function, i.e.

[0169]

[0170] Where N is the number of pixels in the template region.

[0171] 2) Kernel function definition

[0172]

[0173] Used to measure the similarity of luminance pixels.

[0174] 2. Conditioning of Gaussian Process Models

[0175] Conditioning is the training process of a Gaussian model, which updates the posterior distribution of the Gaussian process with new data (Y, C).

[0176] 2.1 Update the mean function

[0177] As new sample points are added, the mean function is updated based on the observed data. Specifically, the update formula for the mean function is:

[0178] μ post (Y)=μ prior (Y)+K y，Y (K Y,Y +σ 2 I) -1 (C-μ prior (Y))

[0179] Where: μ_prior(Y) is the prior mean function. K_{y, Y} is the covariance matrix between the new sample point y and the existing sample points Y. K_{Y, Y} is the covariance matrix between the existing sample points Y. σ is the noise parameter. I is the identity matrix. C is the new observation.

[0180] 2.2 Update the covariance matrix

[0181] When no observation data is available, the covariance matrix is ​​a diagonal matrix. When new sample points are included in the model, the Cholesky decomposition method can be used to update the covariance matrix K, achieving an efficient solution.

[0182] K = L·L T

[0183] When new observation data (y, c) is available, the new sample points can be incorporated into the model by updating the Cholesky decomposition matrix L. Thus:

[0184] 1) Autocovariance of new sample points

[0185] a=k(y,y)+σ

[0186] Where k(y, y) is the autocovariance of the new sample y, and σ is the noise parameter.

[0187] 2) Covariance vector of new sample points and existing sample points

[0188] b=[k(y,y1),k(y,y2),…,k(y,yn )] T

[0189] Where y_i is the brightness value of the existing sample points.

[0190] 3) Update the Cholesky decomposition matrix L

[0191] Denote the autocovariance term of the new sample points.

[0192]

[0193] Among them, b T ·L·b is the inner product of the covariance vectors of existing sample points and new sample points.

[0194] The L matrix can be updated:

[0195] L 11 =a′

[0196] L 12 =-a′·b·L

[0197] L 21 =-L·b T ·a′

[0198] L 22 =L+L·b T ·a′·b·L

[0199] For each reconstructed cross-component pixel pair [Luma, Chroma] selected by S200 and S300, the Gaussian process regression model training process described above will be performed as a training sample. This allows the Gaussian process model to continuously adapt to new observation data and achieve more accurate predictions.

[0200] In implementation, Gaussian process regression models can be trained for the chromaticity components Cb and Cr respectively.

[0201] E.S500 uses Gaussian process regression for cross-component prediction.

[0202] The Gaussian process model established in S400 allows for cross-component prediction of the current Cu. For the reconstructed luminance pixel Y* of the current Cu, it is used as input to the Gaussian process model to predict its concatenated chromaticity value. When using a downsampling method for the luminance component, the operation should be consistent with that in S100.

[0203] In the pixel prediction process, the updated mean function can be used for prediction. There are:

[0204] 1. Calculate the covariance matrix K_{Y*, Y} of the new sample point Y* and the existing sample points Y.

[0205] 2. Calculate the covariance matrix K_{Y,Y} among the existing sample points Y.

[0206] Mean function with predicted values

[0207]

[0208] Using this as the result of cross-component prediction, we have

[0209] predC=μ post (Y * )

[0210] At the same time, the predicted values ​​are clipped to ensure they fall within the effective pixel intensity range [0, 2]. bitdepth Within [-1], for example, in a 10-bit deep YUV sequence, the effective range of pixel intensity extends from 0 to 1023. That is,

[0211] predC′=min(max(predC, 0), 1023).

[0212] C. Solution Implementation

[0213] The cross-component prediction method described in this invention is applicable to intra-frame and inter-frame prediction modules in video coding, and can be applied to video coding reference software platforms such as VTM, ECM, AV1, and HPM. It is suitable for encoding and decoding sequences such as YUV420 and YUV444.

[0214] The encoding and decoding methods described in the embodiments of the present invention can be evaluated for quality using metrics such as bpp, PSNR, SSIM, VMAF, and BDRate.

[0215] In intra-frame prediction, the method of this embodiment can be enabled by a flag in the bitstream. When the flag is positive, it indicates that this module is enabled in intra-frame prediction, and a chroma prediction process based on Gaussian process regression is performed.

[0216] In summary, to address the problems existing in the prior art, this embodiment proposes a cross-component prediction method for video coding based on Gaussian process regression. This method aims to solve the correlation problem between luminance and chrominance in the video coding process. The algorithm can deeply capture the complex relationship between luminance and chrominance components in the video signal. By modeling within a stochastic process framework, the algorithm not only provides high interpretability but also achieves more accurate results in the prediction process.

[0217] This embodiment presents a novel cross-component prediction algorithm. Traditional cross-component linear regression models such as CCLM and CCCM often only capture simple linear relationships when dealing with the relationship between luminance and chrominance components, making it difficult to handle complex nonlinear relationships. In contrast, Gaussian process regression, as a powerful nonlinear modeling tool, can accurately capture and predict the complex nonlinear relationship between luminance and chrominance, and adapts to different types of data distributions and noise characteristics, thus providing more accurate prediction results.

[0218] Another aspect of this invention provides a video coding cross-component prediction system based on Gaussian process regression, comprising:

[0219] The first module is used to obtain reference pixels for the reconstructed block from the luminance component, and the reference pixels are used as input to the cross-component prediction model.

[0220] The second module is used to select a reference region based on the reference pixel using a codec, and then select a first reconstructed pixel based on the reference region.

[0221] The third module is used to establish a prediction model for brightness pixels of different intensities based on a given first threshold and the size of the first threshold and the current pixel intensity, and then select the second reconstructed pixel through the prediction model.

[0222] The fourth module is used to construct a Gaussian regression model based on the first reconstructed pixel and the second reconstructed pixel;

[0223] The fifth module is used to perform cross-component prediction on the current coding unit based on the Gaussian regression model to obtain the cross-component prediction result of video coding.

[0224] It is understood that the content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0225] This invention also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the aforementioned video coding cross-component prediction method based on Gaussian process regression. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0226] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0227] Please see Figure 7 , Figure 7 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes:

[0228] The processor 701 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention.

[0229] The memory 702 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 702 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 702 and is called and executed by the processor 701 to execute the video coding cross-component prediction method based on Gaussian process regression of the embodiments of this invention.

[0230] The input / output interface 703 is used to implement information input and output;

[0231] The communication interface 704 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0232] Bus 705 transmits information between various components of the device (e.g., processor 701, memory 702, input / output interface 703, and communication interface 704);

[0233] The processor 701, memory 702, input / output interface 703, and communication interface 704 are connected to each other within the device via bus 705.

[0234] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described video coding cross-component prediction method based on Gaussian process regression.

[0235] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0236] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0237] It should be noted that in various specific embodiments of the present invention, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when embodiments of the present invention require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirection to a confirmation page. Only after obtaining the user's separate permission or consent is the necessary user-related data for the normal operation of the embodiments of the present invention acquired.

[0238] The embodiments described in this invention are for the purpose of more clearly illustrating the technical solutions of the embodiments of this invention, and do not constitute a limitation on the technical solutions provided by the embodiments of this invention. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this invention are also applicable to similar technical problems.

[0239] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present invention, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0240] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0241] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0242] The terms "first," "second," "third," "fourth," etc. (if present) in the specification and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0243] It should be understood that in this invention, "at least one (item)" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0244] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0245] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0246] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0247] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0248] The preferred embodiments of the present invention have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and spirit of the present invention should be within the scope of the claims of the present invention.

Claims

1. A video coding cross-component prediction method based on Gaussian process regression, characterized in that, Includes the following steps: Reference pixels for the reconstructed block are obtained from the luminance component, and these reference pixels are used as input to the cross-component prediction model. Based on the reference pixel, a reference region is selected by a codec, and then a first reconstructed pixel is selected based on the reference region. Given a first threshold, a prediction model is established for brightness pixels of different intensities based on the size of the first threshold and the current pixel intensity, and then the second reconstructed pixel is selected through the prediction model. Based on the first reconstructed pixel and the second reconstructed pixel, a Gaussian regression model is constructed; Based on the Gaussian regression model, cross-component prediction is performed on the current coding unit to obtain the video coding cross-component prediction result.

2. The video coding cross-component prediction method based on Gaussian process regression according to claim 1, characterized in that, The step of obtaining the reference pixel of the reconstructed block from the luminance component includes the following steps: Obtain the luminance reconstruction cache and chrominance reconstruction cache. In the YUV420 format, if the chrominance area corresponds to multiple luminance blocks, then take the luminance encoding unit corresponding to the center point. The brightness reconstruction pixels on the top and left sides are downsampled to match the corresponding chromaticity components; Specifically, when downsampling the brightness reference pixel, the selection of the sampling filter is determined by the bitstream syntax.

3. The video coding cross-component prediction method based on Gaussian process regression according to claim 1, characterized in that, The step of selecting the reference region based on the reference pixel using a codec includes the following steps: Specify the width and height values ​​for the reference region, and perform pixel reconstruction using the top and left rows of the reference region; expand the pixels from the current WxH coding unit to obtain the reconstructed luminance pixels; Select a reference area by using the reconstructed brightness pixels from the top or left.

4. The video coding cross-component prediction method based on Gaussian process regression according to claim 1, characterized in that, Given a first threshold, a prediction model is established for pixels of different brightness intensities based on the size of the first threshold and the current pixel intensity, including the following steps: In the process of selecting the reference area, the mean value of the luminance component is calculated as the first threshold, and the luminance component is divided into two parts according to the first threshold: the part that is lower than or equal to the first threshold and the part that is higher than the first threshold. The two brightness components are used to train a Gaussian regression model; the portion below or equal to the first threshold is used to train the first model, and the portion above the first threshold is used to train the second model. During cross-component prediction, the appropriate model is selected for prediction based on the current brightness pixel. Specifically, if the brightness value of the current pixel is lower than or equal to the first threshold, the first model is used for prediction; if the brightness value of the current pixel is higher than the first threshold, the second model is used for prediction.

5. The video coding cross-component prediction method based on Gaussian process regression according to claim 1, characterized in that, The step of constructing a Gaussian regression model based on the first reconstructed pixel and the second reconstructed pixel includes the following steps: The expression for the Gaussian process regression model is defined as: C=f(Y)=GP(μ(Y), k(Y, Y')), where μ(Y) is the mean function, k(Y, Y') is the covariance function, Y is the luminance pixel, C is the juxtaposed chrominance pixel; GP() represents the Gaussian process; Based on the Gaussian process regression model, the Gaussian process is initialized and configured to determine the initialization results of the mean function and the covariance function. Based on the Gaussian process regression model, the Gaussian process model is conditionalized, and then the posterior distribution of the Gaussian process is updated based on the result of the conditionalization.

6. The video coding cross-component prediction method based on Gaussian process regression according to claim 5, characterized in that, The step of conditionalizing the Gaussian process regression model and updating the posterior distribution of the Gaussian process based on the conditionalization result includes the following steps: Based on the newly added sample points, the mean function is updated according to the observed data. The update formula for the mean function is: ,in, This represents the updated mean function; Represents the prior mean function; The covariance matrix represents the difference between the new sample point y and the existing sample points Y; This represents the covariance matrix among existing sample points Y; Represents noise parameters; Represents the identity matrix; This represents a new observation; When there is no observation data, the covariance matrix is ​​a diagonal matrix. When new sample points are included in the model, the covariance matrix is ​​updated using the Cholesky decomposition method. When new observation data is available, the new sample points are included in the model by updating the Cholesky decomposition matrix. The expression for the autocovariance a of the new sample points is a = k(y,y) + σ, where k(y,y) is the autocovariance of the new sample y, and σ is the noise parameter. Covariance vector of new sample points and existing sample points The expression is: ,in, These are the brightness values ​​of existing sample points; During the process of updating the Cholesky decomposition matrix L, the autocovariance term of the new sample points The expression is: Therefore, the matrix L is updated as follows: in, It is the inner product of the covariance vectors of existing sample points and new sample points.

7. The video coding cross-component prediction method based on Gaussian process regression according to claim 1, characterized in that, The step of performing cross-component prediction on the current coding unit based on the Gaussian regression model includes the following steps: Pixel prediction is performed using the updated mean function, and the covariance matrix between the predicted point Y* and the existing sample points Y is calculated, as well as the covariance matrix between the existing sample points Y. Determine the mean function with predicted values ​​to obtain the cross-component prediction results; The expression for the mean function with predicted values ​​is: The expression for the prediction result across components is: in, A mean function representing predicted values; Represents the prior mean function; Representative prediction point The covariance matrix with respect to the existing sample points Y; This represents the covariance matrix among existing sample points Y; Represents noise parameters; Represents the identity matrix; This represents a new observation; This represents the prediction results across components.

8. A video coding cross-component prediction system based on Gaussian process regression, characterized in that, include: The first module is used to obtain reference pixels for the reconstructed block from the luminance component, and the reference pixels are used as input to the cross-component prediction model. The second module is used to select a reference region based on the reference pixel using a codec, and then select a first reconstructed pixel based on the reference region. The third module is used to establish a prediction model for brightness pixels of different intensities based on a given first threshold and the current pixel intensity, and then select the second reconstructed pixel through the prediction model. The fourth module is used to construct a Gaussian regression model based on the first reconstructed pixel and the second reconstructed pixel; The fifth module is used to perform cross-component prediction on the current coding unit based on the Gaussian regression model to obtain the cross-component prediction result of video coding.

9. An electronic device, characterized in that, Including the processor and memory; The memory is used to store programs; The processor executes the program to implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a program that is executed by a processor to implement the method as described in any one of claims 1 to 7.

Images