Encoding method and device, decoding method and device, encoding device, decoding device, and storage medium

Abstract

The embodiments of the present application disclose an encoding method and device, a decoding method and device, and an encoding device, a decoding device and a storage medium, the method includes: determining a reference sample value of a first color component of a current block; determining a weighting coefficient by a neural network model according to the reference sample value of the first color component of the current block, the input of the neural network model including the reference sample value of the first color component of the current block; determining a predicted block of a second color component of the current block according to the weighting coefficient and the reference sample value of a second color component of the current block; and determining a reconstruction value of a sampling point of the second color component of the current block according to the predicted block. In this way, the accuracy of chrominance prediction can be improved, and the code rate can be saved, as well as the encoding and decoding performance can be improved.

Description

[Technical field]

[0001] TECHNICAL FIELD The embodiments of the present application relate to the technical field of video encoding and decoding, and in particular to an encoding method and apparatus, a decoding method and apparatus, and an encoding device, a decoding device, and a storage medium. [Background technology]

[0002] As people's demands for video display quality increase, new video application formats such as high definition video and super high definition video emerge. The Joint Video Exploration Team (JVET) of the international standardization organizations ISO / IEC and ITU-T formulates the next-generation video coding standard H.266 / Versatile Video Coding (VVC).

[0003] H.266 / VVC includes inter-color component prediction technology. However, there is a large deviation between the predicted value of the coding block calculated by the inter-color component prediction technology of H.266 / VVC and the original value, which results in low prediction accuracy, low decoded video quality, and poor coding performance. Summary of the Invention [Means for solving the problem]

[0004] The embodiments of the present application provide an encoding method and apparatus, a decoding method and apparatus, as well as an encoding device, a decoding device, and a storage medium, which can not only improve the accuracy of chromaticity prediction and save the code rate, but also improve the encoding and decoding performance.

[0005] The technical solutions of the embodiments of the present application can be realized as follows.

[0006] In a first aspect, embodiments of the present application provide a decoding method, the decoding method comprising: determining a reference sample value of a first color component of the current block; determining a weighting coefficient by a neural network model based on a reference sample value of a first color component of the current block, where an input of the neural network model includes the reference sample value of the first color component of the current block; determining a prediction block of a second color component of the current block based on the weighting coefficients and the reference sample values ​​of the second color component of the current block; determining, based on the predicted block, reconstructed values ​​of the second color component sampling points of the current block.

[0007] In a second aspect, embodiments of the present application provide an encoding method, the encoding method comprising: determining a reference sample value of a first color component of the current block; determining a weighting coefficient by a neural network model based on a reference sample value of a first color component of the current block, where an input of the neural network model includes the reference sample value of the first color component of the current block; determining a prediction block of a second color component of the current block based on the weighting coefficients and the reference sample values ​​of the second color component of the current block; determining a prediction difference value of a second color component sampling point of the current block based on the prediction block.

[0008] In a third aspect, an embodiment of the present application provides an encoding device, the encoding device including a first determination unit, a first model unit and a first prediction unit; The first determining unit is configured to determine a reference sample value of a first color component of the current block; The first model unit is configured to determine a weighting coefficient by a neural network model based on a reference sample value of a first color component of the current block, where an input of the neural network model includes the reference sample value of the first color component of the current block; The first prediction unit is configured to determine a prediction block of a second color component of the current block according to the weighting coefficient and the reference sample value of the second color component of the current block; The first determining unit is further configured to determine, based on the predicted block, a prediction difference value of a second color component sampling point of the current block.

[0009] In a fourth aspect, an embodiment of the present application provides an encoding device, the encoding device including a first memory and a first processor; the first memory is adapted to store a computer program executable on the first processor; The first processor, when executing the computer program, is used to carry out the method according to the second aspect.

[0010] In a fifth aspect, an embodiment of the present application provides a decoding device, the decoding device including a second determining unit, a second model unit and a second predicting unit; The second determining unit is configured to determine a reference sample value of a first color component of the current block; The second model unit is configured to determine a weighting coefficient by a neural network model based on the reference sample value of the first color component of the current block, where an input of the neural network model includes the reference sample value of the first color component of the current block; The second prediction unit is configured to determine a prediction block of a second color component of the current block according to the weighting coefficients and the reference sample values ​​of the second color component of the current block; The second determining unit is further configured to determine, based on the predicted block, reconstructed values ​​of the second color component sampling points of the current block.

[0011] In a sixth aspect, embodiments of the present application provide a decoding device, the decoding device including a second memory and a second processor; the second memory is adapted to store a computer program executable on the second processor; The second processor, when executing a computer program, is used to carry out the method according to the first aspect.

[0012] In a seventh aspect, an embodiment of the present application provides a computer readable storage medium having stored thereon a computer program, the computer program, when executed, causing a computer to perform the method according to the first aspect or to perform the method according to the second aspect.

[0013] The embodiments of the present application provide an encoding method and device, a decoding method and device, and an encoding device, a decoding device, and a storage medium, in which whether the encoding side or the decoding side determines a reference sample value of a first color component of a current block, and determines a weighting coefficient by a neural network model based on the reference sample value of the first color component of the current block, where the input of the neural network model includes the reference sample value of the first color component of the current block, and determines a predicted block of the second color component of the current block based on the weighting coefficient and the reference sample value of the second color component of the current block. In this way, the encoding side can determine a predicted difference value of the second color component sampling point of the current block based on the predicted block. Thus, the decoding side can determine a reconstructed value of the second color component sampling point of the current block based on the predicted block. In this way, by using the color component information in the neighboring area of ​​the current block and the color component information in the current block, various kinds of effective input information can be provided for the chrominance prediction based on the neural network, and the chrominance prediction can be performed better. In addition, by using the luminance difference between the reference pixel and the current block, or the result after correlation processing of this luminance difference, as one of the inputs of the neural network, the neural network can also learn the correlation between the reference chrominance and the predicted chrominance of the current block, and this correlation can be included in the chrominance prediction of the current block, thereby further improving the accuracy of chrominance prediction, saving the code rate, and simultaneously improving the encoding and decoding performance. [Brief description of the drawings]

[0014] [Figure 1] FIG. 13 is a schematic diagram showing the distribution of effective adjacent regions. [Diagram 2] FIG. 13 is a schematic diagram showing the distribution of selected regions in different prediction modes. [Diagram 3] 1 is a schematic flow chart of a model parameter derivation solution. [Figure 4A] FIG. 1 is a schematic diagram of a configuration block diagram of an encoder according to an embodiment of the present application. [Figure 4B] FIG. 2 is a schematic diagram of a configuration block diagram of a decoder according to an embodiment of the present application; [Diagram 5] 1 is a first schematic flowchart of a decoding method according to an embodiment of the present application; [Figure 6A] FIG. 2 is a schematic diagram of a reference area of ​​a current block according to an embodiment of the present application; [Figure 6B] FIG. 2 is a schematic diagram of weighted prediction for NWCP mode and other prediction modes according to an embodiment of the present application; [Figure 7] FIG. 1 is a schematic diagram of a framework for weight-based chromaticity prediction according to an embodiment of the present application. [Figure 8] 2 is a second schematic flowchart of a decoding method according to an embodiment of the present application; [Figure 9] 4 is a third schematic flowchart of a decoding method according to an embodiment of the present application; [Figure 10] FIG. 2 is a first schematic diagram of a chromaticity prediction block diagram based on a neural network model according to an embodiment of the present application; [Figure 11] FIG. 2 is a second schematic diagram of a chromaticity prediction block diagram based on a neural network model according to an embodiment of the present application; [Figure 12] FIG. 13 is a third schematic diagram of a chromaticity prediction block diagram based on a neural network model according to an embodiment of the present application; [Figure 13] FIG. 4 is a fourth schematic diagram of a chromaticity prediction block diagram based on a neural network model according to an embodiment of the present application. [Figure 14] FIG. 5 is a fifth schematic diagram of a chromaticity prediction block diagram based on a neural network model according to an embodiment of the present application. [Figure 15] FIG. 6 is a sixth schematic diagram of a chromaticity prediction block diagram based on a neural network model according to an embodiment of the present application. [Figure 16] FIG. 7 is a seventh schematic diagram of a chromaticity prediction block diagram based on a neural network model according to an embodiment of the present application. [Figure 17] FIG. 8 is an eighth schematic diagram of a chromaticity prediction block diagram based on a neural network model according to an embodiment of the present application. [Figure 18] FIG. 9 is a ninth schematic diagram of a chromaticity prediction block diagram based on a neural network model according to an embodiment of the present application. [Figure 19] FIG. 10 is a schematic diagram of a chromaticity prediction block diagram based on a neural network model according to an embodiment of the present application. [Figure 20] 1 is a first schematic flowchart of an encoding method according to an embodiment of the present application; [Figure 21] 2 is a second schematic flowchart of an encoding method according to an embodiment of the present application; [Figure 22] FIG. 2 is a structural schematic diagram of an encoding device according to an embodiment of the present application; [Figure 23] FIG. 2 is a schematic diagram of a specific hardware structure of an encoding device according to an embodiment of the present application; [Figure 24] FIG. 2 is a structural schematic diagram of a decoding device according to an embodiment of the present application; [Diagram 25] FIG. 2 is a schematic diagram of a specific hardware structure of a decoding device according to an embodiment of the present application; [Figure 26] FIG. 1 is a structural schematic diagram of an encoding and decoding system according to an embodiment of the present application; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] In order to provide a more detailed understanding of the features and technical contents of the embodiments of the present application, the implementation of the embodiments of the present application will be described in detail below with reference to the accompanying drawings, which are not intended to limit the embodiments of the present application but are for reference and description only.

[0016] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. The terms used herein are not intended to limit the present application, but are merely intended to describe the embodiments of the present application.

[0017] With respect to "some embodiments" described below, a subset of all possible embodiments is described, but it is understood that "some embodiments" may be the same or different subsets of all possible embodiments, and may be combined with each other if not inconsistent. It should be further pointed out that the terms "first / second / third" described in the embodiments of the present application merely distinguish similar objects, and do not represent a specific order for the objects, and it is understood that "first / second / third" may be interchanged with a specific order or back and forth order when permitted, such that the embodiments of the present application described herein may be implemented in an order other than that shown or described herein.

[0018] In a video image, a first color component, a second color component, and a third color component are generally used to represent a coding block (CB). Here, these three color components are a luminance component, a blue chrominance component, and a red chrominance component, respectively. Exemplarily, the luminance component is usually represented by the symbol Y, the blue chrominance component is usually represented by the symbol Cb or U, and the red chrominance component is usually represented by the symbol Cr or V. In this way, the video image can be represented in a YCbCr format, and can also be represented in a YUV format. In addition, the video image can be in an RGB format, a YCgCo format, etc., and the embodiment of the present application does not make any limitation.

[0019] It can be understood that in the current video image or video encoding and decoding process, the cross-component prediction technology mainly includes a cross-component linear model (CCLM) prediction mode and a multi-directional linear model (MDLM) prediction mode. Whether the model parameters are derived based on the CCLM prediction mode or the MDLM prediction mode, the corresponding prediction model can realize prediction between color components, such as prediction from the first color component to the second color component, prediction from the second color component to the first color component, prediction from the first color component to the third color component, prediction from the third color component to the first color component, prediction from the second color component to the third color component, or prediction from the third color component to the second color component.

[0020] Taking the prediction from the first color component to the second color component as an example, assuming that the first color component is a luma component and the second color component is a chroma component, in order to reduce the redundancy between the luma component and the chroma component, CCLM prediction mode is used in VVC, that is, the prediction value of chroma is constructed based on the luma reconstruction value of the same coding block, e.g., Pred C (i,j)=α·Rec L (i,j)+β.

[0021] Here, i, j represent the position coordinates of the pixel to be predicted in the coding block, i represents the horizontal direction, j represents the vertical direction, and Pred C (i,j) represents the chromaticity prediction value corresponding to the pixel to be predicted at the position coordinate (i,j) in the coding block, and Rec L (i,j) denotes the luminance reconstruction value corresponding to the pixel to be predicted at (downsampled) position coordinates (i,j) in the same coding block, and α and β denote model parameters that can be derived from reference pixels.

[0022] For a coding block, its neighboring region can be divided into five parts: left neighboring region, upper neighboring region, lower left neighboring region, upper left neighboring region, and upper right neighboring region. H.266 / VVC includes three cross-component linear model prediction modes, which are the left and upper neighboring intra CCLM prediction mode (which can be represented by INTRA_LT_CCLM), the left and lower left neighboring intra CCLM prediction mode (which can be represented by INTRA_L_CCLM), and the upper and upper right neighboring intra CCLM prediction mode (which can be represented by INTRA_T_CCLM). In these three prediction modes, each prediction mode can select a preset number (for example, four) of reference pixels to derive the model parameters α and β, but the biggest difference between these three prediction modes is that the selection regions corresponding to the reference pixels for deriving the model parameters α and β are different.

[0023] Specifically, assuming that the coding block size corresponding to the chrominance components is W×H, the upper selection area corresponding to the reference pixels is W′, and the left selection area corresponding to the reference pixels is H′, then: For the INTRA_LT_CCLM mode, the reference pixels may be selected in the upper and left neighboring regions, ie, W'=W, H'=H.

[0024] For the INTRA_L_CCLM mode, the reference pixels may be selected in the left-neighboring region and the bottom-left neighboring region, ie, H'=W+H, and set W'=0.

[0025] For INTRA_T_CCLM mode, the reference pixels may be selected in the upper neighboring region and the upper right neighboring region, ie, W'=W+H, and set H'=0.

[0026] It should be noted that in VTM, only pixel points in the W range at most are stored in the upper right adjacent region, and only pixel points in the H range at most are stored in the lower left adjacent region. The range of the selection region of INTRA_L_CCLM mode and INTRA_T_CCLM mode is defined as W+H, but in actual applications, the selection region of INTRA_L_CCLM mode is limited to H+H, and the selection region of INTRA_T_CCLM mode is limited to W+W. In this way, For the INTRA_L_CCLM mode, the reference pixels may be selected in the left-neighboring region and the bottom-left neighboring region, where H'=min{W+H,H+H}.

[0027] For the INTRA_T_CCLM mode, the reference pixels may be selected in the upper neighboring region and the upper right neighboring region, where W'=min{W+H, W+W}.

[0028] Referring to Figure 1, a schematic distribution diagram of valid adjacent regions is shown. In Figure 1, the left adjacent region, the lower left adjacent region, the upper adjacent region and the upper right adjacent region are all valid. In addition, the block filled with gray is the prediction target pixel whose position coordinates in the coding block are (i, j).

[0029] Thus, based on FIG. 1, the selection regions for the three prediction modes are as shown in FIG. 2. Here, in FIG. 2, (a) shows the selection region of the INTRA_LT_CCLM mode including the left adjacent region and the upper adjacent region, (b) shows the selection region of the INTRA_L_CCLM mode including the left adjacent region and the lower left adjacent region, and (c) shows the selection region of the INTRA_T_CCLM mode including the upper adjacent region and the upper right adjacent region. After the selection regions of the three prediction modes are determined in this way, pixel selection for deriving model parameters can be performed within the selection region. The pixels selected in this way can be called reference pixels, and the number of reference pixels is usually four, and the positions of the reference pixels are generally determined for one coding block of W×H whose size is determined.

[0030] After obtaining the preset number of reference pixels, currently, chromaticity prediction is performed according to the schematic flowchart of the model parameter derivation solution shown in Figure 3. Based on the flow shown in Figure 3, assuming that the preset number is four, the flow can include the following S301 to S308.

[0031] In S301, reference pixels are obtained in the selected region.

[0032] In S302, the number of valid reference pixels is determined.

[0033] In S303, if the number of valid reference pixels is 0, the model parameter α is set to 0, and β is set to a default value.

[0034] In S304, the chromaticity prediction values ​​are filled in with default values.

[0035] In S305, if the number of valid reference pixels is four, two reference pixels with larger values ​​and two reference pixels with smaller values ​​of the luminance component are obtained by comparison.

[0036] In S306, the average point corresponding to the larger value and the average point corresponding to the smaller value are calculated.

[0037] In S307, model parameters α and β are derived based on the two average points.

[0038] In S308, chromaticity prediction is performed using the prediction model constructed by α and β.

[0039] It should be noted that in VVC, the step where the valid reference pixel is 0 is determined based on the validity of the neighboring regions.

[0040] It should be further explained that the prediction model is built using the principle of "two points determine a line", where the two points can be called fitting points. In the current technical solution, after obtaining the four reference pixels, two reference pixels with large values ​​and two reference pixels with small values ​​of the luminance component are obtained by comparison, and then the mean value point (mean) is calculated based on the two reference pixels with large values. max Then, based on the two reference pixels with the smaller values, another mean point is calculated. min (which can be expressed as max and mean min Furthermore, we can obtain the mean max and mean min are two fitting points, and model parameters (represented by α and β) can be derived. Finally, a prediction model is constructed based on α and β, and prediction processing of chromaticity components is performed based on the prediction model.

[0041] However, in the related art, a simple linear model Pred is used for each coding block. C (i,j)=α·Rec L (i,j)+β is used to predict the chrominance components, and all pixels at any position in each coding block are predicted using the same model parameters α and β. This leads to both of the following defects: On the one hand, coding blocks with different content characteristics all use a simple linear model to map from luminance to chrominance, thereby realizing chrominance prediction, but the mapping functions from luminance to chrominance in any coding block cannot all be accurately fitted by this simple linear model, which makes the prediction results of some coding blocks not accurate enough. On the other hand, in the prediction process, pixel points at different positions in the coding block all use the same model parameters α and β, and there is also a large difference in the prediction accuracy of different positions in the coding block. Simply put, in the current CCLM technology, there is a large deviation between the predicted values ​​and original values ​​of some coding blocks, which leads to low prediction accuracy, poor quality, and further poor coding and decoding efficiency.

[0042] Based on this, an embodiment of the present application provides an encoding method, including determining a reference sample value of a first color component of a current block; determining a weighting coefficient by a neural network model based on the reference sample value of the first color component of the current block, where the input of the neural network model includes the reference sample value of the first color component of the current block; determining a predicted block of the second color component of the current block based on the weighting coefficient and the reference sample value of the second color component of the current block; and determining a predicted difference value of a sampling point of the second color component of the current block based on the predicted block.

[0043] An embodiment of the present application further provides a decoding method, including determining a reference sample value of a first color component of a current block; determining a weighting coefficient by a neural network model based on the reference sample value of the first color component of the current block, where an input of the neural network model includes the reference sample value of the first color component of the current block; determining a predicted block of the second color component of the current block based on the weighting coefficient and the reference sample value of the second color component of the current block; and determining a reconstructed value of a sampling point of the second color component of the current block based on the predicted block.

[0044] In this way, by fully utilizing the color component information in the neighboring region of the current block and the color component information in the current block, a variety of effective input information can be provided for the chrominance prediction based on the neural network, so that the chrominance prediction can be performed better. In addition, by using the luminance difference between the reference pixel and the current block, or the result after the correlation processing of this luminance difference as one of the inputs of the neural network, the neural network can also learn the correlation between the reference chrominance and the chrominance to be predicted of the current block, and this correlation is used in the chrominance prediction of the current block, thereby further improving the accuracy of the chrominance prediction, saving the code rate, and simultaneously improving the encoding and decoding performance.

[0045] Hereinafter, each embodiment of the present application will be described in detail with reference to the drawings.

[0046] Referring to Fig. 4A, a schematic diagram of a configuration block diagram of an encoder according to an embodiment of the present application is shown. As shown in Fig. 4A, an encoder (specifically a "video encoder") 100 can include a transform and quantization unit 101, an intra estimation unit 102, an intra prediction unit 103, a motion compensation unit 104, a motion estimation unit 105, an inverse transform and inverse quantization unit 106, a filter control analysis unit 107, a filtering unit 108, an encoding unit 109, and a decoded image cache unit 110, etc. Here, the filtering unit 108 can realize deblock filtering and sample adaptive offset (SAO: Sample Adaptive Offset) filtering, and the encoding unit 109 can realize header information coding and context-based adaptive binary arithmetic coding (CABAC: Context-based Adaptive Binary Arithmetic Coding). For an input original video signal, a video coding block can be obtained by division of a coding tree unit (CTU), and then the video coding block is transformed by a transform and quantization unit 101 for residual pixel information obtained after intra or inter prediction, including transforming the residual information from pixel domain to transform domain, and quantizing the obtained transform coefficients to further reduce the bit rate. The intra estimation unit 102 and the intra prediction unit 103 are used to perform intra prediction on the video coding block, specifically, the intra estimation unit 102 and the intra prediction unit 103 are used to determine the intra prediction mode used to code the video coding block. The motion compensation unit 104 and the motion estimation unit 105 are used to perform inter prediction coding of the received video coding block against one or more blocks of one or more reference intra to provide temporal prediction information.The motion estimation performed by the motion estimation unit 105 is a process of generating a motion vector, which can estimate the motion of the video coding block, and then the motion compensation unit 104 performs motion compensation based on the motion vector determined by the motion estimation unit 105. After determining the intra prediction mode, the intra prediction unit 103 is further used to provide selected intra prediction data to the coding unit 109, and the motion estimation unit 105 also sends the calculated and determined motion vector data to the coding unit 109. And the inverse transform and inverse quantization unit 106 is for reconstruction of the video coding block, which reconstructs a residual block in the pixel domain, and the reconstructed residual block is filtered out by the filter control analysis unit 107 and the filtering unit 108 to remove block effect artifacts, and then the reconstructed residual block is added to the intra predictive block of the decoded image cache unit 110 to generate a reconstructed video coding block. The coding unit 109 is for coding various coding parameters and quantized transform coefficients, and in a CABAC-based coding algorithm, the context content may be used to code information indicating a determined intra-prediction mode based on neighboring coding blocks, and output a bitstream of the video signal. The decoded image cache unit 110 is used to store reconstructed video coding blocks for prediction reference. As the video image coding progresses, new reconstructed video coding blocks are continuously generated, and all these reconstructed video coding blocks are stored in the decoded image cache unit 110.

[0047] Referring to FIG. 4B, a schematic diagram of a configuration block diagram of a decoder according to an embodiment of the present application is shown. As shown in FIG. 4B, a decoder (specifically a "video decoder") 200 can include a decoding unit 201, an inverse transform and inverse quantization unit 202, an intra prediction unit 203, a motion compensation unit 204, a filtering unit 205, and a decoded image cache unit 206, etc. Here, the decoding unit 201 can realize header information decoding and CABAC decoding, and the filtering unit 205 can realize deblock filtering and SAO filtering. After the input video signal undergoes the encoding process of FIG. 4A, it outputs a bitstream of the video signal, which is input to the decoder 200 and first passes through the decoding unit 201 to obtain decoded transform coefficients. The transform coefficients are processed by the inverse transform and inverse quantization unit 202 to generate residual blocks in the pixel domain. The intra prediction unit 203 may be used to generate prediction data of the current video decoded block based on the determined intra prediction mode and data from the previous decoded block from the current frame or picture. The motion compensation unit 204 is used to determine prediction information for the video decoded block by analyzing the motion vector and other related syntax elements, and to generate a predictive block of the video decoded block being decoded using the prediction information. The residual block from the inverse transform and inverse quantization unit 202 is added to the corresponding predictive block generated by the intra prediction unit 203 or the motion compensation unit 204 to form a decoded video block. The decoded video signal can be filtered by the filtering unit 205 to remove block effect artifacts and improve the quality of the video. The decoded video block is then stored in the decoded image cache unit 206, which stores the reference image for subsequent intra prediction or motion compensation, and is also used to output the video signal at the same time, i.e., the restored original video signal is obtained.

[0048] It should be noted that the method of the embodiment of the present application is mainly applied to the part of the intra prediction unit 103 shown in Fig. 4A and the part of the intra prediction unit 203 shown in Fig. 4B. That is, the embodiment of the present application can be applied to not only the encoder but also the decoder, and can be applied to both the encoder and the decoder at the same time, but is not specifically limited in the embodiment of the present application.

[0049] It should be further explained that when applied to the portion of intra prediction unit 103, the "current block" specifically refers to the encoded block on which intra prediction is currently being performed, and when applied to the portion of intra prediction unit 203, the "current block" specifically refers to the decoded block on which intra prediction is currently being performed.

[0050] In one embodiment of the present application, a first schematic flowchart of a decoding method according to an embodiment of the present application is shown in FIG. 5. As shown in FIG. 5, the method may include the following steps S501 to S504.

[0051] In S501, the reference sample value of the first color component of the current block is determined.

[0052] It should be noted that the decoding method of the embodiment of the present application is applied to a decoding device or a decoding device (which may be abbreviated as a "decoder") integrated with the decoding device. In addition, the decoding method of the embodiment of the present application may specifically refer to an intra prediction method, more specifically, a Network-based Weight Chroma Prediction (NWCP) method.

[0053] In the embodiment of the present application, a video image can be divided into a plurality of decoded blocks, each of which can include a first color component, a second color component, and a third color component, and the current block refers to a decoded block in the video image on which intra prediction is currently performed. In addition, if the current block performs prediction of the first color component, and the first color component is a luma component, i.e., the component to be predicted is a luma component, the current block can also be called a luma prediction block. Or, if the current block performs prediction of the second color component, and the second color component is a chroma component, i.e., the component to be predicted is a chroma component, the current block can also be called a chroma prediction block.

[0054] It should be further explained that in the embodiments of the present application, the reference information of the current block may include a value of a first color component sampling point in a neighboring region of the current block and a value of a second color component sampling point in a neighboring region of the current block, and these sampling points (Samples) may be determined based on decoded pixels in the neighboring region of the current block. In some embodiments, the neighboring region of the current block may include at least one of an upper neighboring region, an upper right neighboring region, a left neighboring region, and a lower left neighboring region.

[0055] Here, the upper adjacent region and the upper right adjacent region can be regarded as an upper region, and the left adjacent region and the lower left adjacent region can be regarded as a left region. The adjacent region can further include an upper left region as shown in Fig. 6. Here, when performing prediction of the second color component on the current block, the upper region, the left region, and the upper left region of the current block as the adjacent regions can all be called the reference region of the current block, and the pixels in the reference region are all decoded reference pixels.

[0056] It should be further explained that in the embodiment of the present application, the adjacent region of the current block can include multiple rows or multiple columns adjacent to the current block, for example, the left region can include one or more columns, the upper region can include one or more rows, and the increase or decrease of the number of rows or columns is not limited in the embodiment of the present application.

[0057] In some embodiments, the method may further include determining a neighboring reference value of the first color component of the current block based on the first color component sampling points in a neighboring region of the current block.

[0058] It should be noted that in the embodiment of the present application, the reference pixel of the current block may specifically refer to the reference pixel point adjacent to the current block. The adjacent here may be, but is not limited to, spatial adjacent. For example, the adjacent may be temporal adjacent, spatial and temporal adjacent, and the reference pixel of the current block may further be a reference pixel obtained after performing a certain process on the reference pixel point adjacent in the spatial domain, adjacent in the temporal domain, adjacent in the spatial domain and adjacent in the temporal domain, etc., but the embodiment of the present application does not make any limitation.

[0059] It should be further explained that in the embodiment of the present application, both the first color component sampling point and the second color component sampling point in the adjacent region of the current block can be referred to as the reference pixel of the current block. Assuming that the first color component is a luminance component and the second color component is a chrominance component, the value of the first color component sampling point in the adjacent region of the current block is the reference luminance information corresponding to the reference pixel of the current block, and the value of the second color component sampling point in the adjacent region of the current block is the reference chrominance information corresponding to the reference pixel of the current block.

[0060] It should be further explained that in the embodiments of the present application, the adjacent regions of the current block may include only the upper adjacent region, or may include only the left adjacent region, or may include the upper adjacent region and the upper right adjacent region, or may include the left adjacent region and the lower left adjacent region, or may include the upper adjacent region and the left adjacent region, or may further include the upper adjacent region, the upper right adjacent region and the left adjacent region, etc., but the embodiments of the present application do not make any limitations thereon.

[0061] It should be further noted that in the embodiment of the present application, the neighboring region may be determined based on the prediction mode of the current block. If the prediction mode of the current block is a horizontal mode, determining reference pixels based on pixels in an upper neighboring region and / or an upper right neighboring region; If the prediction mode of the current block is a vertical mode, determining the reference pixels based on pixels in the left neighboring region and / or the lower-left neighboring region.

[0062] For example, if the prediction mode of the current block is a horizontal mode, only the upper neighboring region and / or the upper right neighboring region may be selected as the neighboring region in the chrominance component prediction, and if the prediction mode of the current block is a vertical mode, only the left neighboring region and / or the lower left neighboring region may be selected as the neighboring region in the chrominance component prediction.

[0063] Further, in some embodiments, for determining the adjacent reference value of the first color component, the method may further include performing a screening process on values ​​of the first color component sampling points in the adjacent region to determine the adjacent reference value of the first color component of the current block.

[0064] It should be noted that the first color component sampling points in the neighboring region may have some insignificant sampling points (e.g., these sampling points have poor correlation) or some abnormal sampling points, and these sampling points need to be removed to obtain valid first color component sampling point values ​​to ensure the accuracy of prediction. That is, in the embodiment of the present application, a first sampling point set is formed based on the first color component sampling points in the neighboring region of the current block. A screening process can be performed on the first sampling point set to determine the neighbor reference value of the first color component of the current block.

[0065] In one specific embodiment, performing a screening process on the values ​​of the first color component sampling points in the neighboring region to determine the neighboring reference value of the first color component of the current block includes: determining a location of a selected sampling point based on a location and / or a color component intensity of a first color component sampling point in the adjacent region; determining a neighboring reference value of the first color component of the current block from the neighboring region based on the position of the selected sampling point.

[0066] It should be noted that in the embodiment of the present application, the color component intensity can be represented by color component information such as reference luminance information, reference chrominance information, etc., where the larger the value of the color component information, the higher the color component intensity. In this way, screening the value of the first color component sampling point in the neighboring region can be based on the position of the sampling point or based on the color component intensity, thereby determining the valid first color component sampling point value based on the sampling point obtained by screening, and further determining the neighboring reference value of the first color component of the current block.

[0067] In some embodiments, for the adjacent reference value of the first color component, the method may further include performing a first filtering process on values ​​of the first color component sampling points in the adjacent region to determine an adjacent reference value of the first color component of the current block.

[0068] It should be noted that in the embodiment of the present application, the first filtering process may be a downsampling filtering process, where the first color component is a luminance component, and the downsampling filtering process is performed on the reference luminance information, so that the spatial resolution of the reference luminance information after filtering can be the same as the spatial resolution of the reference chromaticity information. Alternatively, the first filtering process may be an upsampling filtering process, so as to preserve the reference luminance information without loss, but no limitation is made here.

[0069] It should be further noted that in the embodiments of the present application, the reference information of the current block may further include a reconstructed value of a first color component sampling point in the current block, in which case, in some embodiments, the method may further include determining a reference value of the first color component in the current block based on the reconstructed value of the first color component sampling point in the current block.

[0070] In the embodiment of the present application, assuming that the first color component is a luminance component, the reconstructed value of the first color component sampling point in the current block is, namely, the reconstructed luminance information of the current block.

[0071] In some embodiments, for the reference value of the first color component in the current block, the method may further include performing a second filtering process on the reconstructed values ​​of the first color component sampling points in the current block to determine the reference value of the first color component in the current block.

[0072] It should be noted that in the embodiment of the present application, the second filtering process can be a downsampling filtering process, where the first color component is a luminance component, and the downsampling filtering process can also be performed on the reconstructed luminance information in the current block. Or, the second filtering process can be an upsampling filtering process, so as to preserve the reconstructed luminance information without loss, and there is no limitation here.

[0073] In some embodiments, for S501, determining a reference sample value of a first color component of the current block includes: Determining a neighboring reference value of the first color component of the current block based on the first color component sampling points in the neighboring region of the current block; and determining a reference sample value of the first color component of the current block based on a reconstructed value of the first color component sampling point in the current block and an adjacent reference value of the first color component of the current block.

[0074] In one specific embodiment, determining a reference sample value of a first color component of a current block according to a reconstruction value of a first color component sampling point in the current block and a neighboring reference value of the first color component of the current block includes: determining a reference value of the first color component in the current block based on the reconstructed value of the first color component sampling point in the current block; determining a reference sample value of the first color component of the current block based on the reference value of the first color component in the current block and a neighboring reference value of the first color component of the current block.

[0075] It is to be noted that in the embodiment of the present application, the reference sample value of the first color component of the current block may be luminance difference information, where the reference sample value of the first color component of the current block may be an absolute value of a difference between an adjacent reference value of the first color component of the current block and a reference value of the first color component within the current block.

[0076] It should be further explained that in the embodiment of the present application, the calculation of the luminance difference information may be the absolute value of the difference between the reference luminance information and the reconstructed luminance information, the absolute value of the difference between the reference luminance information and the reconstructed luminance information after the filtering process, the absolute value of the difference between the reference luminance information and the reconstructed luminance information after the filtering process, or the absolute value of the difference between the reference luminance information and the reconstructed luminance information after the filtering process. Therefore, in the embodiment of the present application, in addition to the reference sample value of the first color component of the current block being the absolute value of the difference between the adjacent reference value of the first color component of the current block and the reference value of the first color component in the current block, the reference sample value of the first color component of the current block may further be the absolute value of the difference between the reconstructed value of the first color component sampling point in the current block and the adjacent reference value of the first color component of the current block. Or, the reference sample value of the first color component of the current block may be the absolute value of the difference between the reconstructed value of the first color component sampling point in the current block and the value of the first color component sampling point in the adjacent area of ​​the current block. Alternatively, the reference sample value of the first color component of the current block may be the absolute value of the difference between the reference value of the first color component in the current block and the value of the first color component sampling point in an adjacent area of ​​the current block, but no limitation is made here.

[0077] It can be further understood that in embodiments of the present application, for a second color component sampling point in the adjacent region, in some embodiments, the method may further include performing a third filtering process on the value of the second color component sampling point in the adjacent region to determine a reference sample value of the second color component of the current block.

[0078] It should be noted that in the embodiment of the present application, the third filtering process can be an upsampling filtering process, or the third filtering process can be a downsampling filtering process, but no limitation is made herein, where the second color component is a chromaticity component, and in this case, the upsampling filtering process or the downsampling filtering process is performed on the reference chromaticity information, so that the spatial resolution of the reference chromaticity information after filtering can be finally made the same as the spatial resolution of the reference luminance information.

[0079] That is, the reference chrominance information and the reference luma information of the current block may be obtained from the neighboring regions. Here, the obtained reference chrominance information includes, but is not limited to, the reference chrominance reconstructed value of the upper region of the selected current block and / or the reference chrominance reconstructed value of the left region. The obtained reference luma information includes, but is not limited to, the corresponding reference luma information obtained based on the position of the reference chrominance information. The manner of obtaining the reconstructed luma information in the current block includes, but is not limited to, obtaining the corresponding reconstructed luma information based on the position of the chrominance information in the current block as the reconstructed luma information of the current block.

[0080] It can be further understood that in the embodiment of the present application, under certain conditions, the reference chrominance information, the reference luma information, or the reconstructed luma information can be used as the target information of the current block after a correlation filtering process. In addition, the correlation filtering operation here can be realized based on the manner of a neural network structure. Exemplarily, in order to reduce the complexity of the calculation, a point selection operation or a downsampling filtering operation based on a neural network structure can be performed on the reference chrominance information, the reference luma information, or the reconstructed luma information. Exemplarily, under the condition that the spatial resolutions of the chrominance components and the luma components are not consistent, an upsampling filtering / downsampling filtering operation based on a neural network structure can be performed on the reference chrominance information, the reference luma information, or the reconstructed luma information to facilitate the calculation of the subsequent chrominance prediction.

[0081] In this way, the target information of the current block can include the neighbor reference value of the first color component of the current block (i.e., the reference luminance information), the reference value of the first color component in the current block (i.e., the reconstructed luminance information), and the reference sample value of the second color component of the current block (i.e., the reference chromaticity information). These pieces of information may be information after filtering or information without filtering, and are not specifically limited in the embodiments of the present application.

[0082] It can be further understood that in the embodiments of the present application, the luminance difference information is composed of a plurality of luminance difference matrices. Thus, in some embodiments, the reference sample value of the first color component of the current block can include N matrices, where N represents the number of reference sample values ​​of the second color component of the current block, and the k-th matrix includes an absolute value of the difference between the k-th value of the adjacent reference value of the first color component of the current block and the reference value of the first color component in the current block, where k=0,1,...,N-1.

[0083] As an example, the pseudo code is described as follows:

[0084] for(k=0;k<=numRefChroma-1;k++) { for(y=0;y<=hLuma-1;y++) for(x=0;x<=wLuma-1;x++) deltaLuma[k][y][x]=abs(recLuma[y][x]-refLuma[k]); } where numRefChroma represents the number of reference sample values ​​of the second color component of the current block, wLuma×hLuma represents the number of reference values ​​of the first color component in the current block, where wLuma is the width and hLuma is the height, and deltaLuma[k][y][x] represents the kth matrix of reference sample values ​​of the first color component of the current block, i.e., the absolute value of the difference between the kth value of the adjacent reference value of the first color component of the current block and the reference value of the first color component in the current block.

[0085] In S502, a weighting coefficient is determined by a neural network model based on a reference sample value of a first color component of a current block, where an input of the neural network model includes the reference sample value of the first color component of the current block.

[0086] It should be noted that in the embodiment of the present application, after determining the reference sample value of the first color component of the current block, the reference sample value of the first color component of the current block can be used as the input of a neural network model, and the output of the neural network model is a weighting coefficient.

[0087] In one possible implementation form, when the reference sample value of the first color component of the current block includes N matrices, for S502, determining a weighting coefficient by a neural network model based on the reference sample value of the first color component of the current block may include taking the N matrices as inputs of the neural network model and determining the output of the neural network model as the weighting coefficient.

[0088] It should be further explained that in the present embodiment, the N matrices are used in their entirety as inputs to a neural network model, the output of which is the weight coefficients.

[0089] As an example, the pseudo code is described as follows:

[0090] mWeight = NNFunction(deltaLuma) Here, deltaLuma represents a three-dimensional array or matrix, where the whole is used as the input of the neural network model, and mWeight represents the weighting coefficient. Note that if deltaLuma is a three-dimensional data structure, the output mWeight is also a three-dimensional data structure.

[0091] In another possible implementation, the luminance difference information (e.g., N matrices) can be subjected to nonlinear processing, and then the processed result can be taken as the reference sample of the first color component of the current block. In some embodiments, the method includes: determining values ​​corresponding to the N matrices in a preset mapping relationship; The method may further include determining a reference sample value of a first color component of the current block based on values ​​corresponding to the N matrices.

[0092] It is to be noted that the preset mapping relationship may be a preset function relationship, in one specific embodiment, the preset mapping relationship is a Softmax function.

[0093] As can be understood, in probability theory and related fields, the normalized exponential function, also known as the Softmax function, is a popular logic function. It can "compress" an N-dimensional vector z, which contains any real numbers, into another N-dimensional vector σ(z), such that the range of each element is all between (0,1) and the sum of all elements is 1, and it is often used as a nonlinear activation function in multi-classification neural networks. Exemplarily, let deltaLuma[k][y][x] be used to represent the kth matrix, and the Softmax function is shown as follows:

[0094]

number

[0095]

number

[0096] Correspondingly, in some embodiments, for S502, determining a weighting coefficient by a neural network model based on a reference sample value of the first color component of the current block may include taking values ​​corresponding to the N matrices in the preset mapping relationship as inputs of the neural network model, and determining the output of the neural network model as the weighting coefficient.

[0097] It should be noted that after determining the values ​​corresponding to the N matrices in the preset mapping relationship, the values ​​corresponding to the N matrices are also used as the input of the neural network model, and the output of the neural network model is the weighting coefficients.

[0098] It should be further explained that in the embodiment of the present application, the predetermined mapping relationship may be a predetermined functional relationship such as a Softmax function, or may be a predetermined look-up table method, for example, an array element look-up table method is used to reduce some calculation operations, but is not specifically limited in the embodiment of the present application.

[0099] It should be further explained that in the embodiment of the present application, the reference sample value of the first color component of the current block is a three-dimensional data structure. Specifically, after calculating the absolute value of the difference between the adjacent reference value of the first color component of the current block and the reference value of the first color component in the current block, the absolute value of the difference value can be stored in N matrices of the three-dimensional data structure. Or, each element in the adjacent reference value of the first color component of the current block is duplicated multiple times to obtain the adjacent reference value of the first color component of the three-dimensional data structure, and the whole reference value of the first color component in the current block is duplicated multiple times to obtain the reference value of the first color component of the three-dimensional data structure, and then the absolute value of the difference value can be calculated according to the adjacent reference value of the first color component of the three-dimensional data structure and the reference value of the first color component of the three-dimensional data structure to obtain the N matrices of the three-dimensional data structure, and no limitation is made here.

[0100] It can be understood that in the embodiment of the present application, the input of the neural network model, in addition to being the luminance difference information or the information obtained by performing nonlinear processing on the luminance difference information, can also be other types of information. For example, the reference value of the first color component in the current block and the adjacent reference value of the first color component of the current block are directly input to the neural network model, or the information obtained by performing concatenation processing on them is input to the neural network model, or the information obtained by combining them with the reference sample value of the second color component of the current block is input to the neural network model. The following provides relevant descriptions of these situations.

[0101] In one possible implementation form, for S502, determining a weighting coefficient by a neural network model based on a reference sample value of a first color component of the current block includes: The method may include taking the reference value of the first color component in the current block and the adjacent reference value of the first color component of the current block as inputs of a neural network model, and determining the output of the neural network model as the weighting coefficients.

[0102] It should be noted that in this case, there is no need to perform a correlation operation on the reference values ​​of the first color component in the current block and the adjacent reference values ​​of the first color component of the current block, but they can be directly used as inputs to the neural network model.

[0103] In another possible implementation form, for S502, determining a weighting coefficient by a neural network model based on a reference sample value of a first color component of the current block includes: performing a linking process on the reference value of the first color component in the current block and the adjacent reference value of the first color component of the current block to obtain first linking information; The method may include: determining the first link information as an input of a neural network model and an output of the neural network model as a weighting coefficient.

[0104] It should be noted that in this case, a first-dimension connection process can be performed on the reference value of the first color component in the current block and the adjacent reference value of the first color component of the current block, but there is no need to perform absolute value calculation of the difference between the two. At this time, the obtained first connection information is used as the input of a neural network model, and the first connection information is processed by the neural network model, and the final output of the neural network model is a weighting coefficient.

[0105] In yet another possible implementation form, for S502, the method includes: The method may further include: taking the reference sample value of the first color component of the current block and the reference sample value of the second color component of the current block as inputs of a neural network model, and determining the output of the neural network model as a weighting coefficient.

[0106] In one specific embodiment, the method may include taking the reference value of the first color component in the current block, the adjacent reference value of the first color component of the current block, and the reference sample value of the second color component of the current block as inputs of a neural network model, and determining the output of the neural network model as a weighting coefficient.

[0107] It should be noted that in this case, all of the target information of the current block (including reference luma information, reference chroma information in adjacent regions, and reconstructed luma information in the current block) can be input to the neural network model, and then these information are processed by the neural network model, and finally the output of the neural network model is the weighting coefficient.

[0108] In another specific embodiment, the method may include: performing a connection process on the reference value of the first color component in the current block and the adjacent reference value of the first color component of the current block to obtain second connection information; taking the second connection information and the reference sample value of the second color component of the current block as the input of a neural network model, and determining the output of the neural network model as a weighting coefficient.

[0109] It should be noted that in this case, a first-dimension connection process can be performed on the reference value of the first color component in the current block and the adjacent reference value of the first color component of the current block, but there is no need to perform absolute value calculation of the difference between the two; then, the obtained second connection information and the reference sample value of the second color component of the current block are jointly input into a neural network model, and this information is processed by the neural network model; and finally, the output of the neural network model is a weighting coefficient.

[0110] It should be further noted that in the embodiment of the present application, the first connection information and the second connection information are all for performing connection processing on the reference value of the first color component in the current block and the adjacent reference value of the first color component in the current block, but the connection information of both may be the same or different. For example, after each element in the adjacent reference value of the first color component of the current block is duplicated multiple times to obtain the adjacent reference value of the first color component of the three-dimensional data structure, for the first connection information, the whole reference value of the first color component in the current block is duplicated multiple times to obtain the reference value of the first color component of the three-dimensional data structure, and then a first-dimensional connection in the first dimension is performed according to the adjacent reference value of the first color component of the three-dimensional data structure and the reference value of the first color component of the three-dimensional data structure. As for the second connection information, a new dimension can be added along the first dimension for the reference value of the first color component in the current block to obtain the reference value of the first color component of the three-dimensional data structure, and then a connection on the first dimension can be performed in the first dimension based on the adjacent reference value of the first color component of the three-dimensional data structure and the reference value of the first color component of the three-dimensional data structure, but is not specifically limited.

[0111] In yet another specific embodiment, the method may include performing a connection process on the reference value of the first color component in the current block, the adjacent reference value of the first color component of the current block, and the reference sample value of the second color component of the current block to obtain third connection information; using the third connection information as the input of a neural network model, and determining the output of the neural network model as a weighting coefficient.

[0112] It should be noted that in this case, a first-dimensional connection can be performed on the three elements, namely, the reference value of the first color component in the current block, the adjacent reference value of the first color component of the current block, and the adjacent reference value of the second color component of the current block; then, the obtained third connection information is input to a neural network model, and the third connection information is processed by the neural network model; and finally, the output of the neural network model is a weighting coefficient.

[0113] It should be further explained that in the embodiment of the present application, the input of the neural network model needs to be a three-dimensional data structure, that is, the reference value of the first color component in the current block, the adjacent reference value of the first color component in the current block, and the reference sample value of the second color component in the current block need to be converted into a three-dimensional data structure before being used as the input of the neural network model, for example, the reference value of the first color component in the current block is globally replicated, the adjacent reference value of the first color component in the current block is element-by-element replicated, or the reference sample value of the second color component in the current block is element-by-element replicated, etc., before being used as the input of the neural network model.

[0114] It should also be noted that for the neural network model, when the input of the neural network model requires more dimensional information, the embodiment of the present application can further perform dimensional expansion on the input information, and then, after processing the expanded information by the neural network model, the output of the neural network model can be subjected to dimensional reduction processing to return to the original dimension, and the weighting coefficient of the three-dimensional data structure can also be obtained.

[0115] With respect to the neural network model, it can be appreciated that in some embodiments, the neural network model may be a convolutional neural network model.

[0116] Wherein, the convolutional neural network model may include at least one base unit and one target unit, where: The basic unit includes at least a first convolutional layer and a first activation function.

[0117] The target unit includes at least a second convolutional layer and a second activation function.

[0118] In an embodiment of the present application, the first activation function is a ReLU function, and the second activation function is a softmax function. In one specific embodiment, the convolutional neural network model may be configured by seven basic units and a target unit formed by a final layer. Here, the basic units may be configured by a convolutional layer with a stride of 1, same mode padding, convolution kernel of 3×3, and a ReLU first activation function, and the target unit may be configured by a convolutional layer with a stride of 1, same mode padding, convolution kernel of 3×3, and a softmax activation function.

[0119] It should be noted that same mode padding indicates that the size of the feature map after convolution does not change, i.e. the size of the input image is the same as the size of the output image.

[0120] It should be further explained that the number of layers of the convolutional neural network model used herein is not limited. The network structure includes, but is not limited to, a convolutional layer, a pooling layer, an activation function, etc. The size of the convolution kernel can be 3×3, 5×5 or other sizes, the convolution type can be normal convolution, transposed convolution, transformable convolution, etc., whether it is the first activation function or the second activation function, includes, but is not limited to, a ReLU activation function, a softmax activation function, etc., and is not specifically limited in the embodiments of the present application.

[0121] For example, the convolutional neural network model may be a convolutional neural network consisting of two downsampling convolutional units and two upsampling convolutional units, where the downsampling convolutional unit may be configured with a convolutional layer having a stride of 2 and a same mode padding and a ReLU activation function, and the upsampling convolutional unit may be configured with a transposed convolutional layer having a stride of 2 and a ReLU activation function.

[0122] For example, the convolutional neural network model may be a convolutional neural network composed of a number of basic units, where the basic unit may be composed of a convolutional layer with stride 1 and same mode padding and a ReLU activation function, and the size of the convolutional kernel may be 3×3, 5×5, or other sizes.

[0123] For example, a convolutional neural network model may also be used as the neural network model of the embodiment of the present application, which is a convolutional neural network composed of a plurality of basic units and an attention mechanism. Here, the basic unit may be composed of a convolutional layer with stride 1 and same mode padding and a ReLU activation function, and the size of the convolution kernel may be 3×3, 5×5 or other sizes. The attention mechanism may be composed of a convolutional layer and a ReLU and a Sigmoid activation function, or may be composed of a convolutional layer and a ReLU and a Softmax activation function.

[0124] With respect to the neural network model, it can be further appreciated that in some embodiments, the neural network model is a fully-connected neural network model.

[0125] Wherein, the fully-connected neural network model may include at least one basic unit and one target unit, where: The basic unit includes at least a first fully connected layer and a first activation function.

[0126] The target unit includes at least a second fully connected layer and a second activation function.

[0127] In an embodiment of the present application, the first activation function is a ReLU function, and the second activation function is a softmax function. In one specific embodiment, the fully-connected neural network model can be configured to use a fully-connected network with two basic units and a target unit formed by a final layer, where the basic units can be configured by a fully-connected layer and a ReLU activation function, and the target unit can be configured by a fully-connected layer and a Softmax activation function.

[0128] It should be further explained that the number of layers of the fully connected network model used is not limited. The network structure includes, but is not limited to, a fully connected layer, an activation function, etc. There is no fixed limit on the number of neurons in the fully connected layer. The activation function includes, but is not limited to, a ReLU activation function, a softmax activation function, a Sigmoid activation function, etc., and is not specifically limited in the embodiments of the present application.

[0129] In the present embodiment, since deltaLuma[k][y][x] is a three-dimensional matrix data structure, it can be further understood that deltaLuma[k][y][x] can also be considered as a W×H one-dimensional matrix. Thus, in some embodiments, the method further comprises: The reference sample values ​​of the first color component of the current block may include W×H one-dimensional matrices, and the one-dimensional matrix includes N weighting coefficients, where W×H represents the number of sampling points of the second color component of the current block, and N represents the number of reference sample values ​​of the second color component of the current block.

[0130] It should be noted that in the embodiment of the present application, W×H here represents the number of second color component sampling points of the current block, or may be the number of second color component prediction points of the current block, or may be the number of second color component prediction points determined by calculation.

[0131] It should be further explained that in this case, each one-dimensional matrix includes N weighting coefficients, and the predicted value of the second color component predicted point corresponding to the one-dimensional matrix in the current block is determined according to the N weighting coefficients. Here, in one one-dimensional matrix, the sum of the N weighting coefficients is equal to 1, and each weighting coefficient is greater than or equal to 0 and less than or equal to 1. However, it should be noted that "the sum of the N weighting coefficients is equal to 1" is only a theoretical concept, and in the actual fixed-point realization process, the absolute value of the weighting coefficient may be greater than 1.

[0132] In S503, a prediction block of the second color component of the current block is determined according to the weighting coefficient and the reference sample value of the second color component of the current block.

[0133] It should be noted that after the weighting coefficient is output by the neural network model, a weighting calculation can be performed based on the weighting coefficient and the reference sample value of the second color component of the current block, and then a predicted block of the second color component of the current block can be determined.

[0134] In some embodiments, for S503, determining a prediction block of a second color component of the current block based on the weighting coefficients and reference sample values ​​of the second color component of the current block includes: determining a first one-dimensional matrix for calculating a second color component sampling point to be predicted according to a position index and a weighting coefficient of the second color component sampling point to be predicted in the current block; determining a prediction block of the second color component of the current block based on the first one-dimensional matrix and the reference sample values ​​of the second color component of the current block.

[0135] In an embodiment of the present application, the first one-dimensional matrix represents a one-dimensional matrix corresponding to the position index in the three-dimensional matrix data structure of weighting coefficients.

[0136] That is, the weighting coefficient has a three-dimensional matrix data structure. If the position index of the second color component sampling point to be predicted in the current block is the wth column and the hth row, the first one-dimensional matrix is ​​the wth column, the hth row one-dimensional matrix in the three-dimensional matrix data structure of the weighting coefficient.

[0137] In some other embodiments, for S503, determining a prediction block of a second color component of the current block based on the weighting coefficient and a reference sample value of the second color component of the current block includes: The method may include determining a prediction block of the second color component of the current block based on a position index of the second color component sampling point to be predicted in the current block and the weighting coefficient.

[0138] In an embodiment of the present application, the predicted value of the second color component sampling point to be predicted in the prediction block can be set equal to a weighted sum of the reference sample value of the second color component of the current block and a coefficient in a second one-dimensional matrix, where the second one-dimensional matrix represents a one-dimensional matrix corresponding to a position index in the three-dimensional matrix data structure of the weighting coefficients.

[0139] That is, still take the position index of the second color component sampling point to be predicted in the current block as an example, the position index is at the wth column and the hth row, for this second color component sampling point to be predicted, it correspondingly includes N weighting coefficients in the one-dimensional matrix at the wth column and the hth row in the three-dimensional matrix data structure of the weighting coefficients, and then determine the weighting value between the reference sample value of the second color component of the current block and the corresponding coefficient among the N weighting coefficients, and the predicted value of this second color component sampling point to be predicted can be set equal to the sum of the N weighting values, where N represents the number of the reference sample values ​​of the second color component, and N is a positive integer.

[0140] In the embodiment of the present application, the second color component is a chrominance component, so the second color component sampling point is a chrominance sampling point.

[0141] for(y=0;y<=hChroma-1;y++) for(x=0;x<=wChroma-1;x++) { / / Determines the weighting factor used by the (x,y) chromaticity sampling point for(k=0;k<=numRefChroma-1;k++) mWeightSample[k]=mWeight[k][y][x]; / / Calculate the predicted value for the (x,y) chromaticity sampling point for(k=0;k<=numRefChroma-1;k++) predChroma[y][x]+=mWeightSample[k]×refChroma[k]; } The above prediction process may be equivalent to the following:

[0142] for(y=0;y<=hChroma-1;y++) for(x=0;x<=wChroma-1;x++) { for(k=0;k<=numRefChroma-1;k++) predChroma[y][x]+=mWeight[k][y][x]×refChroma[k]; } where numRefChroma represents the number of reference sample values ​​of the second chrominance component in the current block, wLuma×hLuma represents the number of reference values ​​of the first chrominance component in the current block, where wLuma is the width, hLuma is the height, predChroma[y][x] represents the predicted value of the (x,y) chrominance sampling point, where mWeight[k][y][x] represents the kth weighting coefficient corresponding to the (x,y) chrominance sampling point, refChroma[k] represents the kth reference sample value of the second chrominance component, and there are a total of numRefChroma weighting values ​​of mWeight[k][y][x]×refChroma[k], and the sum of these weighting values ​​is the predicted value of the (x,y) chrominance sampling point.

[0143] It should be further noted that in the embodiments of the present application, the weighting coefficients output by the neural network model can be nonlinearly processed before being weighted with the reference sample value of the second color component of the current block. In some embodiments, the method further includes: determining a value corresponding to the weighting coefficient in a preset mapping relationship after determining the weighting coefficient through the neural network model; and updating the weighting coefficients based on values ​​corresponding to the weighting coefficients in the preset mapping relationship.

[0144] Here, the weighting coefficients output by the neural network model can be nonlinearly processed in one dimension or multiple dimensions. Exemplarily, the preset mapping relationship of the nonlinear processing is as follows:

[0145]

number

[0146] It should be further noted that in the embodiment of the present application, for the weighting coefficients output by the neural network model, if the weighting coefficients are floating-point weighting coefficients, fixed-point processing can be performed on them, or if the predicted value calculated according to the second color component sampling points to be predicted is a floating-point number, fixed-point processing can be performed on it.

[0147] Also, the output of the neural network model may be a weighting coefficient that participates in the calculation of the chrominance prediction value, or may be a prediction block of the chrominance components of the current block directly. determining a predicted block of the second color component of the current block by a neural network model according to the reference sample value of the first color component of the current block and the reference sample value of the second color component of the current block; and determining, based on the predicted block, reconstructed values ​​of the second color component sampling points of the current block.

[0148] In an embodiment of the present application, the method may include using a reference sample value of a first color component of a current block and a reference sample value of a second color component of the current block as inputs to a neural network model, and determining an output of the neural network model as a predicted block.

[0149] It should be noted that, for the input of the neural network model, the reference value of the first color component in the current block, the adjacent reference value of the first color component in the current block, and the reference sample value of the second color component in the current block can be the input of the neural network model. Alternatively, the reference value of the first color component in the current block, the adjacent reference value of the first color component in the current block, and the reference sample value of the second color component in the current block can be concatenated, and the concatenated information can be the input of the neural network model. Alternatively, the reference value of the first color component in the current block and the adjacent reference value of the first color component in the current block can be concatenated, and the concatenated information and the reference sample value of the second color component in the current block can be the input of the neural network model. Alternatively, the reference value of the first color component in the current block and the adjacent reference value of the first color component in the current block can be subtracted to calculate the absolute value, and then the absolute value of the difference value and the reference sample value of the second color component in the current block can be the input of the neural network model. In this case, the output of the neural network model can be directly used as the predicted block of the second color component of the current block.

[0150] In S504, a reconstruction value of a second color component sampling point of the current block is determined based on the predicted block.

[0151] It should be noted that in the embodiments of the present application, the prediction block may include predicted values of at least some of the second color component sampling points within the current block. Here, when the prediction block includes the predicted values of all the second color component sampling points within the current block, at this time, there is no need to perform any processing on the prediction block, and the prediction block can be directly used as the final prediction block. When the prediction block includes the predicted values of some of the second color component sampling points within the current block, at this time, in order to obtain the final prediction block, it is necessary to perform upsampling filtering on the prediction block. Although the number of predicted values of the second color component included in the prediction block is the same as the number of second color component sampling points included in the current block, but when the predicted values of the second color component sampling points of the current block are not included, at this time, in order to obtain the final prediction block, it is necessary to enhance the predicted values by filtering. Therefore, in some embodiments, the method may further include performing a correlation process on the prediction block after determining the prediction block of the second color component of the current block, and using the processed prediction block as the prediction block.

[0152] In one specific embodiment, the predicted values of the second color component sampling points within the prediction block usually need to be restricted within a preset range. Therefore, in some embodiments, performing a correlation process on the prediction block may include performing a correction operation on the predicted values of the second color component sampling points within the prediction block and determining the prediction block of the second color component of the current block.

[0153] It should be noted that in the embodiments of the present application, the preset range may be between 0 and (1<<BitDepth)-1. Here, BitDepth is the bit depth required by the chrominance component. When the predicted value within the prediction block (predWcp) exceeds the value of the preset range, it is necessary to perform a corresponding correction operation on the predicted value. Exemplarily, a clamp operation can be performed on predWcp[i][j], and specifically, it is as follows.

[0154] If the value of predWcp[i][j] is less than 0, it is set to 0.

[0155] If the value of predWcp[i][j] is greater than or equal to 0 and less than or equal to (1<<BitDepth)-1, it is equal to predWcp[i][j].

[0156] If the value of predWcp[i][j] is greater than (1<<BitDepth)-1, it is set to (1<<BitDepth)-1.

[0157] In this way, after performing the correction operation on the predicted value, it can be guaranteed that the predicted values of all the second color component sampling points in the prediction block are all between 0 and (1<<BitDepth)-1.

[0158] In another specific embodiment, post-processing operations can also be performed on the prediction block. Therefore, in some embodiments, performing correlation processing on the prediction block performing filtering enhancement processing on the predicted values of the second color component sampling points in the prediction block, performing correction processing on the predicted values of the second color component sampling points in the prediction block by using a preset compensation value, using the predicted value of the second color component of the current block in at least one prediction mode to perform weighted fusion processing on the predicted values of the second color component sampling points in the prediction block, can further include at least one of these.

[0159] It should be noted that in the NWCP mode, in order to reduce the instability caused by independent and parallel prediction for each pixel, at this time, filtering enhancement processing can be performed on the predicted values of the second color component sampling points in the prediction block. For example, smoothing filtering processing can be performed on the prediction block, and then the processed prediction block can be used as the final prediction block.

[0160] It should be further noted that, in order to further improve prediction accuracy in the NWCP mode, a position-related modification process can be performed on the prediction block using a preset compensation value, for example, a chromaticity compensation value is calculated for each second color component sampling point to be predicted using a reference pixel that is spatially close, the second color component sampling point in the prediction block is modified using the chromaticity compensation value, and a final prediction value of the second color component sampling point is determined according to the modified prediction value, and then a final prediction block is obtained.

[0161] It should be further explained that in order to further improve the prediction accuracy in the NWCP mode, the chrominance prediction value calculated in the other prediction mode and the chrominance prediction value calculated in the NWCP mode may be weighted and merged, and the final prediction block may be determined based on the fusion result. Exemplarily, as shown in FIG. 6B, the other prediction modes may include a planar mode, a direct current (DC) mode, a vertical mode, a horizontal mode, and a CCLM mode, etc., and each prediction mode is all connected to one switch, which is used to control whether the chrominance prediction value in this prediction mode participates in the weighted fusion process. Assuming that the weight value corresponding to the planar mode is W_Planar, the weight value corresponding to the DC mode is W_DC, the weight value corresponding to the vertical mode is W_Ver, the weight value corresponding to the horizontal mode is W_Hor, the weight value corresponding to the CCLM mode is W_CCLM, and the weight value corresponding to the NWCP mode is W_Wcp, for the chromaticity prediction value in the planar mode, the chromaticity prediction value in the DC mode, the chromaticity prediction value in the vertical mode, the chromaticity prediction value in the horizontal mode, and the chromaticity prediction value in the CCLM mode, when only the switch connected to the CCLM mode is in a closed state, a weighted fusion can be performed on the chromaticity prediction value in the CCLM mode and the chromaticity prediction value in the NWCP mode based on W_CCLM and W_Wcp, and it can be determined to perform weighted fusion with equal weights or unequal weights based on the values ​​of W_CCLM and W_Wcp, and the weighted result is, that is, the final chromaticity prediction value of the second color component sampling point, and then a final prediction block is obtained.

[0162] In some embodiments, after determining a predicted block, determining reconstructed values ​​of second color component sampling points of the current block based on the predicted block includes: determining a predicted difference value of a second color component sampling point of the current block; determining a predicted value of a second color component sampling point of the current block based on the predicted block; and determining a reconstructed value of the second color component sampling point of the current block based on the predicted difference value of the second color component sampling point of the current block and the predicted value of the second color component sampling point of the current block.

[0163] It should be noted that in an embodiment of the present application, determining a prediction residual value of a second color component sampling point of the current block may be determining a prediction residual value of a second color component sampling point of the current block by analyzing the bitstream.

[0164] It should be further explained that in the embodiment of the present application, the prediction block may include predicted values ​​of at least some second color component sampling points in the current block. Wherein, if the prediction block includes predicted values ​​of all second color component sampling points in the current block, the predicted values ​​of the second color component sampling points of the current block may be set equal to the values ​​of the prediction block. If the prediction block includes predicted values ​​of some second color component sampling points in the current block, the values ​​of the prediction block may be up-sampling filtered, and the predicted values ​​of the second color component sampling points of the current block may be set equal to the output values ​​after the up-sampling filtering.

[0165] That is, in an embodiment of the present application, determining a predicted value of the second color component sampling point of the current block based on a predicted block may be to set the predicted value of the second color component sampling point of the current block equal to the value of the predicted block, or to filter the value of the predicted block and then set the predicted value of the second color component sampling point of the current block equal to the output value of the filtering.

[0166] In this way, taking the chrominance component as an example, the bitstream can be analyzed to determine the chrominance prediction difference value of the current block, and then based on the predicted block, the chrominance prediction value of the current block can be determined, and then the chrominance prediction value and the chrominance prediction difference value can be added to obtain the chrominance reconstruction value of the current block.

[0167] It can be understood that the embodiment of the present application is a neural network-based chrominance prediction NWCP technology, which mainly includes the following (1) and (2): (1) fully obtains existing information of the current block, including reconstructed luminance information of the neighboring region, reconstructed chrominance information of the neighboring region, and reconstructed luminance information of the current block; (2) provides a variety of effective input information for neural network-based chrominance prediction, i.e., the reconstructed luminance information difference between the current block and the reference pixel or the result after the reconstructed luminance information difference is processed, so that chrominance prediction can be performed better.

[0168] It can be further understood that the embodiment of the present application further provides a weight-based chrominance prediction framework, as shown in FIG. 7, for each sampling point recY[i][j] in the reconstructed luminance information (i.e., downsampled luminance block) in the downsampled current block, first obtain a luminance difference vector diffY[i][j] based on the absolute value of the difference between recY[i][j] and the neighboring luminance vector refY[k]. Then, derive a normalized weight vector mWeight[i][j] based on a nonlinear mapping model related to diffY[i][j]. Furthermore, the weight vector can be used to vector-multiply the weight vector by the neighboring chrominance vector of the current block to obtain a predicted chrominance prediction value predWcp[i][j]. It should be noted that the normalized weight vector mWeight[i][j] here can be obtained based on a neural network model.

[0169] The embodiment of the present application provides a decoding method, which determines a reference sample value of a first color component of a current block, and determines a weighting coefficient by a neural network model according to the reference sample value of the first color component of the current block, where the input of the neural network model includes the reference sample value of the first color component of the current block, and determines a predicted block of the second color component of the current block according to the weighting coefficient and the reference sample value of the second color component of the current block, and determines a reconstruction value of the second color component sampling point of the current block according to the predicted block. In this way, by fully utilizing the color component information in the neighboring region of the current block and the color component information in the current block, a variety of effective input information can be provided for chrominance prediction based on the neural network, so that chrominance prediction can be performed better. In addition, by using the luminance difference between the reference pixel and the current block, or the result after correlation processing of this luminance difference, as one of the inputs of the neural network, the neural network can also learn the correlation between the reference chrominance and the predicted chrominance of the current block, and this correlation can be included in the chrominance prediction of the current block, thereby further improving the accuracy of chrominance prediction, saving the code rate, and simultaneously improving the encoding and decoding performance.

[0170] In another embodiment of the present application, based on the decoding method described in the above embodiment, take the current block as an example for performing chrominance prediction, the reconstructed luma information of the current block, the reference luma information and the reference chroma information of the neighboring region are all decoded reference information, therefore, the embodiment of the present application proposes a chroma prediction technique based on a neural network using the above information.

[0171] Here, the detailed steps of the chromaticity prediction process of the WCP mode are as follows:

[0172] WCP mode input: current block position (xTbCmp, yTbCmp), current block width nTbW and current block height nTbH.

[0173] Output of WCP mode: Predictions of the current block, predSamples[x][y], where x = 0, …, nTbW-1, y = 0, …, nTbH-1, with the top-left corner of the current block as the origin.

[0174] Here, the prediction process of the WCP mode can include steps such as determining core parameters, obtaining target information, chrominance prediction based on neural network, post-processing process, etc. After these steps, the chrominance prediction value of the current block can be obtained.

[0175] In one specific embodiment, referring to Figure 8, a second schematic flowchart of a decoding method according to an embodiment of the present application is shown. As shown in Figure 8, the method may include the following S801 to S803.

[0176] In S801, target information is obtained.

[0177] It should be noted that, for S801, when predicting the current block, the upper region, the upper left region, and the left region of the current block are taken as the neighboring regions (which may also be called "reference regions") of the current block, and all pixels in the neighboring regions are decoded reference pixels, as shown in the above-mentioned FIG. 6A.

[0178] Here, reference chrominance information refC and reference luma information refY are obtained from adjacent regions. The obtained reference chrominance information includes, but is not limited to, reference chrominance reconstruction values ​​of the upper region and / or reference chrominance reconstruction values ​​of the left region of the selected current block. The obtained reference luma information includes, but is not limited to, corresponding reference luma information obtained based on the position of the reference chrominance information.

[0179] The manner of obtaining the luminance reconstruction information recY of the current prediction block includes, but is not limited to, obtaining corresponding reconstructed luminance information as the reconstructed luminance information of the current block based on the position of the chrominance information in the current block.

[0180] It should be noted that in the embodiment of the present application, refC, refY, or recY can be the target information of neural network-based chromaticity prediction input after being pre-processed under certain conditions, where the pre-processing of refC, refY, or recY can be realized by artificial or neural network-based methods.

[0181] Exemplarily, to reduce the computational complexity of NWCP prediction, an artificial or neural network-based point selection operation or downsampling filtering operation can be performed on refC, refY, or recY. For example, a convolutional neural network can be configured using multiple basic units to perform a downsampling operation on refC, refY, or recY, and the basic units may be configured by a convolutional layer with stride 2 and same mode padding and a ReLU activation function.

[0182] For example, for current blocks of different sizes, refC, refY, or recY may perform filtering operations based on different artificial or neural networks. For example, a convolutional neural network may be configured using multiple basic units to perform filtering operations on refC, refY, or recY, and the basic units may be configured by a convolutional layer with stride 1 and same mode padding and a ReLU activation function.

[0183] Exemplarily, in a condition where the spatial resolutions of the video chrominance and luma components are not consistent, an artificial or neural network-based upsampling or downsampling filtering operation can be performed on refC, refY, or recY to facilitate subsequent chrominance prediction calculation. Note: In NWCP prediction, when the spatial resolutions of the chrominance and luma components are not consistent, it is not necessary to perform an upsampling or downsampling pre-processing filtering operation on refC, refY, or recY to align the components.

[0184] For example, under the condition that the spatial resolution of the video chrominance and luminance components is the same, multiple row / column reference pixels can be used. In this case, a filtering operation such as upsampling or downsampling based on artificial or neural network can be performed on refC, refY, or recY to facilitate subsequent chrominance prediction calculation.

[0185] That is, the acquired target information may include reference chromaticity information refC (after a preprocessing operation if a preprocessing operation is required) for obtaining the outSize number, reference luma information refY (after a preprocessing operation if a preprocessing operation is required) for obtaining the inSize number, and reconstructed luma information recY within the current block for obtaining the LSizew x LSizeh size (after a preprocessing operation if a preprocessing operation is required).

[0186] In S802, a chromaticity prediction calculation based on a neural network is performed according to the target information to determine a prediction block of the current block.

[0187] It should be noted that, in S802, when predicting the current block, first, construct the input of the neural network model based on the acquired target information, and perform vector multiplication on the output of the neural network model and the reference chromaticity information refC to determine the chromaticity prediction value of the pixel to be predicted in the current block.

[0188] In one specific embodiment, referring to Figure 9, a third schematic flowchart of a decoding method according to an embodiment of the present application is shown. As shown in Figure 9, the method may include the following steps S901 to S903.

[0189] In S901, the input of the neural network model is determined.

[0190] It should be explained that in one possible implementation, the input of the neural network model may be to calculate the luminance difference between each reference luminance information value and all pixels in the current block, and store the luminance difference in a diffY vector, and the specific pseudocode is as follows:

[0191] For k=0,1,…,outSize-1, For i=0,1,…,LSizew-1,j=0,1,…,LSizeh-1, diffY[k][i][j]=abs(refY[k]-recY[i][j]) (5) Where:

number

[0192] In another possible implementation form, the input of the neural network model can be as follows: Each element in the inSize number of one-dimensional reference luminance information array refY is replicated LSizeh*LSizew times and arranged in the second and third dimensions in the form of LSizew×LSizeh to obtain refY' in the form of a three-dimensional array of inSize×LSizew×LSizeh; The entire two-dimensional array recY is replicated inSize times to obtain recY' in the form of a three-dimensional array of inSize×LSizew×LSizeh.

[0193] In this way, the three-dimensional luminance difference matrix diffY is obtained by subtracting the matrix elements corresponding to refY' and recY' and taking the absolute value, and the formula is as follows:

[0194] For k=0,1,…,outSize-1, For i=0,1,…,LSizew-1,j=0,1,…,LSizeh-1, diffY[k][i][j]=abs(refY'[k][i][j]-recY'[k][i][j]) (6) In the embodiment of the present application, the three-dimensional luminance difference matrix diffY[k][i][j] is the input of the neural network model.

[0195] In S902, a weighting factor is determined based on a neural network model, and a chrominance prediction value of a pixel to be predicted in the current block is calculated based on the weighting factor.

[0196] For illustrative purposes, in this embodiment, we assume that the vector shape is 1st dimension, 2nd dimension, ... Nth dimension from left to right. We add one new dimension to the 1st dimension of diffY, i.e., the shape of diffY becomes [1, inSize, LSizew, LSizeh], and this 4-dimensional diffY is used as the input of the neural network model.

[0197] It should be further explained that the embodiment of the present application uses a convolutional neural network model, for example, a convolutional neural network formed by seven basic units and a final layer can be the neural network model of the embodiment of the present application, where the basic units may be configured by a convolutional layer with a stride of 1, same mode padding, convolutional kernel of 3×3 and ReLU activation function, and the final layer may be configured by a convolutional layer with a stride of 1, same mode padding, convolutional kernel of 3×3 and Softmax activation function.

[0198] It should be noted that the number of layers of the convolutional neural network model used is not limited. The network structure includes, but is not limited to, a convolutional layer, a pooling layer, an activation function, etc. The size of the convolution kernel can be 3×3, 5×5 or other sizes, the convolution type can be normal convolution, transposed convolution, transformable convolution, etc., and the activation function includes, but is not limited to, a ReLU activation function, a softmax activation function, etc., and no limitations are made here.

[0199] Furthermore, if the neural network model computes an output vector cWeightFloat with shape [1, outSize, predSizeW, predSizeH], we need to further remove the first dimension of cWeightFloat and modify its shape to [outSize, predSizeW, predSizeH].

[0200] In the embodiment of the present application, when the weighting coefficients are floating-point weighting coefficients, fixed-point processing is performed on the floating-point weighting coefficients to obtain fixed-point weighting coefficients. That is, fixed-point processing can be performed on the above cWeightFloat. For example, the following fixed-point processing is performed on each element in cWeightFloat:

[0201] For i=0,1,…,LSizew-1,j=0,1,…,LSizeh-1, For k=0,1,…,outSize-1, cWeight[k][i][j]=round(cWeightFloat[k][i][j]×2 Shift ) (7) Here, Round(x) = Sign(x) × Floor(abs(x) + 0.5), where Floor(x) is the largest integer less than or equal to x.

number

[0202] Then, the first dimension of cWeight or cWeightFloat (i.e., the channel dimension in the neural network model) is multiplied with refC to obtain a neural network-based prediction block predWcp. Here, the pseudocode for realizing the vector multiplication is written as follows:

[0203] For i=0,1,…,LSizew-1,j=0,1,…,LSizeh-1, For k=0,1,…,outSize-1, subCFloat[k][i][j]=(cWeightFloat[k][i][j]×refC[k]) (8) After the calculation is completed, fixed-point processing can be performed on subCFloat[k][i][j]. In order to maintain a certain calculation accuracy during the fixed-point process, a coefficient may be multiplied, as follows:

[0204] subC[k][i][j]=round(subCFloat[k][i][j]×2Shift) (9) For i=0,1,…,LSizew-1,j=0,1,…,LSizeh-1,

number

[0205] predWcp[i][j]=round(predWcpFloat[i][j]) (11) Or, in another possible implementation, the formula is:

[0206] For k=0,1,…,outSize-1, subC[k][i][j]=(cWeight[k][i][j]×refC[k]) (12) Then, the calculation is performed using fixed-point subC[k][i][j], specifically as follows:

[0207] For i=0,1,…,LSizew-1,j=0,1,…,LSizeh-1,

number

[0208] In S903, for the pixels to be predicted within the current block, a correction process is performed on the calculated chrominance prediction value, and the prediction block of the current block is determined.

[0209] It should be noted that the chrominance prediction values in predWcp should be restricted within a preset range. If it exceeds the preset range, it is necessary to perform corresponding correction operations. Exemplarily, in one possible implementation, a clamp operation can be performed on the chrominance prediction values in predWcp, specifically as follows.

[0210] ● If the value of predWcp[i][j] is less than 0, it is set to 0.

[0211] ● If the value of predWcp[i][j] is greater than (1 << BitDepth) - 1, it is set to (1 << BitDepth) - 1.

[0212] Here, BitDepth is the bit depth required by the chrominance pixel value to ensure that all chrominance prediction values in the prediction block predWcp are between 0 and (1 << BitDepth) - 1.

[0213] That is, predWcp[i][j] = Clip3(0, (1 << BitDepth) - 1, predWcp[i][j]) (14) Here,

Equation

[0214]

number

[0215] It should be noted that the weight-based chrominance prediction output predWcp[i][j] needs to be post-processed under certain conditions before it becomes the final chrominance prediction value (predSamples[i][j]), otherwise the final chrominance prediction value predSamples[i][j] is the same as predWcp[i][j]. Here, the neural network-based NWCP technique can perform post-processing in a non-neural network or neural network manner.

[0216] For example, if the size predSizeW of the chrominance prediction output predWcp block is not equal to the width nTbW of the current block, or predSizeH is not equal to the height nTbH of the current block, it is necessary to perform an artificial or neural network-based upsampling or downsampling operation on predWcp[i][j] to obtain the final chrominance prediction value predSamples[i][j].

[0217] For example, to mitigate the instability caused by the pixel-by-pixel independent and parallel prediction of NWCP, artificial or neural network-based smoothing filtering can be performed on predWcp[i][j] to obtain the final chromaticity prediction value predSamples[i].

[0218] For example, to further improve the accuracy of the NWCP prediction value, a position-related correction process based on artificial or neural network may be performed on predWcp[i][j]. For example, a chrominance compensation value is calculated for each pixel to be predicted using a reference pixel that is spatially close, and predWcp[i][j] is corrected using the chrominance compensation value, and the corrected prediction value is the final chrominance prediction value predSamples.

[0219] For example, in order to further improve the accuracy of the NWCP predicted value, the chrominance predicted value calculated in another chrominance prediction mode and the chrominance predicted value predWcp[i][j] calculated by the NWCP may be weighted artificially or based on a neural network, and the fusion result may be the final chrominance predicted value predSamples[i][j]. For example, the chrominance predicted value predicted in the CCLM mode and the chrominance predicted value predWcp[i][j] calculated by the NWCP may be weighted artificially or based on a neural network with equal or unequal weights, and the weighted result may be the final chrominance predicted value predSamples[i][j].

[0220] For example, in order to improve the NWCP prediction performance, the prediction output predWcp[i][j] of the NWCP can be modified using a neural network, but the embodiment of the present application does not impose any limitation thereon.

[0221] It can be further understood that in the embodiment of the present application, for the input of the neural network model, the reference luminance information refY in the form of a one-dimensional array of inSize numbers can be subtracted from the reconstructed luminance information at each position in the current block, and the absolute value can be taken, so that the one-dimensional array result is transposed as one column of a two-dimensional luminance difference matrix diffY. At this time, the pseudo code for the construction process of diffY is as follows:

[0222] For k in range(inSize): For i in range(LSizew): For j in range(LSizeh): diffY[k][i+j×LSizew]=abs(refY[k]-recY[i][j]) At this time, the two-dimensional luminance difference matrix diffY[k][i+j×LSizew] is used as the input of the neural network model. In this technical solution, a fully connected neural network model can be used to configure a fully connected network using two basic units and a target unit. Here, the basic unit can be configured by a fully connected layer+ReLU activation function, and the target unit can be configured by a fully connected layer+Softmax activation function.

[0223] It should also be noted that the number of layers of the fully connected neural network model used herein is not limited. The network structure includes, but is not limited to, a fully connected layer, an activation function, etc. There is no fixed limit on the number of neurons in the fully connected layer. The activation function includes, but is not limited to, a ReLU activation function, a softmax activation function, or a Sigmoid activation function, etc.

[0224] In this way, the two-dimensional output vector cWeight is calculated and obtained by the fully connected neural network model, and the cWeight and the reference chromaticity information refC are multiplied by a matrix to obtain the prediction block of the current block. The pseudo code of the calculation process is as follows:

[0225] For i in range(LSizew): For j in range(LSizeh): sum=0; For k in range(inSize): sum=sum+cWeight[k][i+j×LSizew]×refC[k]; predWcp[i][j] = sum; It can be further understood that in the embodiment of the present application, the brightness difference matrix diffY can be nonlinearly processed before being input to the network. For example, one or more dimensions of the brightness difference matrix diffY can be subjected to a softmax function normalization process, and the data after the nonlinear processing can be input to the network. Assuming that the first dimension of the brightness difference matrix diffY is subjected to a softmax process, the specific nonlinear processing process is illustratively as follows:

[0226]

number

[0227]

number

[0228] For example, a convolutional neural network model may be a convolutional neural network consisting of two downsampling convolutional units and two upsampling convolutional units, which may be used as the neural network model in the embodiments of the present application. The downsampling convolutional unit may be configured by a convolutional layer with stride of 2 and same mode padding + ReLU activation function, and the upsampling convolutional unit may be configured by a transposed convolutional layer with stride of 2 + ReLU activation function.

[0229] For example, the convolutional neural network model may be a convolutional neural network composed of a number of basic units, where the basic unit may be composed of a convolutional layer with stride 1 and same mode padding + ReLU activation function, and the size of the convolutional kernel may be 3x3, 5x5 or other sizes.

[0230] For example, a convolutional neural network model may also be used as the neural network model of the embodiment of the present application, which is a convolutional neural network composed of a plurality of basic units and an attention mechanism. Here, the basic unit may be composed of a convolutional layer with stride 1 and same mode padding + ReLU activation function, and the size of the convolutional kernel may be 3x3, 5x5 or other sizes. The attention mechanism may be composed of a convolutional layer and ReLU and Sigmoid activation functions, or may be composed of a convolutional layer and ReLU and Softmax activation functions.

[0231] It can be further understood that in the embodiment of the present application, for the aforementioned neural network model, the target information here can include reference luma information refY, reconstructed luma information recY, and reference chroma information refC, where the size of the reference luma information refY is inSize×1, the size of the reconstructed luma information recY is LSizew×LSizeh, and the size of the reference chroma information refC is outSize×1. Then, by performing related processing on refY, recY, and refC, various kinds of different input information can be provided to the neural network model.

[0232] In the following, several application scenarios of neural network based chromaticity prediction are taken as examples in detail.

[0233] In one specific embodiment, referring to FIG. 10, a first schematic diagram of a chromaticity prediction block diagram based on a neural network model according to an embodiment of the present application is shown. As shown in FIG. 10, for the input information of the neural network model, the reference luminance information refY is replicated LSizew×LSizeh times for each element, to obtain the size of the reference luminance information refY' inSize×LSizew×LSizeh, the reconstructed luminance information recY is replicated inSize times overall, to obtain the size of the reconstructed luminance information recY' inSize×LSizew×LSizeh, and the absolute value of the difference between the reference luminance information refY' and the reconstructed luminance information recY' is calculated to obtain a luminance difference matrix diffY, the size of which is inSize×LSizew×LSizeh. In this way, the luminance difference matrix diffY is taken as the input of the neural network model, and the output of the neural network model is the weighting coefficient cWeight, and the size of the weighting coefficient cWeight is outSize×LSizew×LSizeh. Then, the vector product of each column vector in the channel dimension of the weighting coefficient cWeight and the reference chrominance information refC is calculated to obtain a prediction block of the current block.

[0234] In another specific embodiment, referring to FIG. 11, a second schematic diagram of a chrominance prediction block diagram based on a neural network model according to an embodiment of the present application is shown. As shown in FIG. 11, for the input information of the neural network model, the reference luminance information refY is replicated LSizew×LSizeh times for each element to obtain the size of the reference luminance information refY' inSize×LSizew×LSizeh, and the reconstructed luminance information recY is replicated inSize times overall to obtain the size of the reconstructed luminance information recY' inSize×LSizew×LSizeh. In this way, the reference luminance information refY' and the reconstructed luminance information recY' are independently input to the neural network model, and at this time, the output of the neural network model is the weighting coefficient cWeight, and the size of the weighting coefficient cWeight is outSize×LSizew×LSizeh. Then, the vector product of each column vector in the channel dimension of the weighting coefficient cWeight and the reference chrominance information refC can be calculated to obtain a prediction block of the current block.

[0235] Specifically, in the embodiment of the present application, each element in the reference luminance information refY in the one-dimensional array format of the inSize number can be replicated LSizeh×LSizew times, and arranged in the second and third dimensions in the format of LSizew×LSizeh to obtain refY' in the three-dimensional array format of inSize×LSizew×LSizeh. The two-dimensional array recY is entirely replicated inSize times along the first dimension to obtain recY' in the three-dimensional array format of inSize×LSizew×LSizeh. It should be noted that when the input of the neural network model has more dimensions, refY' and recY' can be further independently used as the input of the neural network model after adding one new dimension to the first dimension.

[0236] In yet another specific embodiment, referring to FIG. 12, a third schematic diagram of a chrominance prediction block diagram based on a neural network model according to an embodiment of the present application is shown. As shown in FIG. 12, for the input information of the neural network model, the reference luminance information refY is duplicated LSizew×LSizeh times for each element to obtain the size of the reference luminance information refY' inSize×LSizew×LSizeh, and one dimension is newly added to the reconstructed luminance information recY before performing concatenation, so as to obtain the size of the concatenated information (1+inSize)×LSizew×LSizeh. In this way, the concatenated information is taken as the input of the neural network model, and at this time, the output of the neural network model is still the weighting coefficient cWeight, and the size of the weighting coefficient cWeight is outSize×LSizew×LSizeh. Then, the vector product of each column vector in the channel dimension of the weighting coefficient cWeight and the reference chrominance information refC can be calculated to obtain the predicted block of the current block.

[0237] Specifically, in the embodiment of the present application, each element in the reference luminance information refY in the one-dimensional array format of inSize number is replicated LSizeh×LSizew times, and arranged in the second and third dimensions in the format of LSizew×LSizeh to obtain refY' in the three-dimensional array format of inSize×LSizew×LSizeh. A new dimension is added along the first dimension to the two-dimensional array recY to obtain recY' in the three-dimensional array format of 1×LSizew×LSizeh, and refY' and recY' are concatenated to add a new dimension to the first dimension to form an input vector, which is the input of the neural network model.

[0238] In yet another specific embodiment, referring to Fig. 13, a fourth schematic diagram of a chrominance prediction block diagram based on a neural network model according to an embodiment of the present application is shown. As shown in Fig. 13, for the input information of the neural network model, the reference luminance information refY is replicated LSizew×LSizeh times for each element to obtain the size inSize×LSizew×LSizeh of the reference luminance information refY', the reconstructed luminance information recY is replicated inSize times overall to obtain the size inSize×LSizew×LSizeh of the reconstructed luminance information recY', and the reference chrominance information refC is replicated LSizew×LSizeh times for each element to obtain the size outSize×LSizew×LSizeh of the reference luminance information refC'. In this way, refY', recY' and refC' are concatenated, and the concatenated information is used as the input of the neural network model.

[0239] In yet another specific embodiment, referring to Fig. 14, a fifth schematic diagram of a chrominance prediction block diagram based on a neural network model according to an embodiment of the present application is shown. As shown in Fig. 14, for the input information of the neural network model, the reference luminance information refY is replicated LSizew×LSizeh times for each element to obtain the size of the reference luminance information refY' inSize×LSizew×LSizeh, the reconstructed luminance information recY is replicated inSize times overall to obtain the size of the reconstructed luminance information recY' inSize×LSizew×LSizeh, and the reference chrominance information refC is replicated LSizew×LSizeh times for each element to obtain the size of the reference luminance information refC' outSize×LSizew×LSizeh. In this way, refY' and recY' are concatenated, and the concatenated information and refC' are independently input to the neural network model together.

[0240] In yet another specific embodiment, referring to FIG. 15, a sixth schematic diagram of a chrominance prediction block diagram based on a neural network model according to an embodiment of the present application is shown. As shown in FIG. 15, for the input information of the neural network model, the reference luminance information refY is replicated LSizew×LSizeh times for each element to obtain the size of the reference luminance information refY' inSize×LSizew×LSizeh, the reconstructed luminance information recY is replicated overall inSize times to obtain the size of the reconstructed luminance information recY' inSize×LSizew×LSizeh, and the reference chrominance information refC is replicated LSizew×LSizeh times for each element to obtain the size of the reference luminance information refC' outSize×LSizew×LSizeh. In this way, refY' and recY' are subtracted element by element to obtain a luminance difference matrix, and the size of the luminance difference matrix is ​​inSize×LSizew×LSizeh. In this way, the luminance difference matrix and refC' are independently input together to the neural network model.

[0241] In yet another specific embodiment, referring to Fig. 16, a seventh schematic diagram of a chrominance prediction block diagram based on a neural network model according to an embodiment of the present application is shown. As shown in Fig. 16, for the input information of the neural network model, the reference luminance information refY is replicated LSizew×LSizeh times for each element to obtain the size inSize×LSizew×LSizeh of the reference luminance information refY', the reconstructed luminance information recY is replicated inSize times overall to obtain the size inSize×LSizew×LSizeh of the reconstructed luminance information recY', and the reference chrominance information refC is replicated LSizew×LSizeh times for each element to obtain the size outSize×LSizew×LSizeh of the reference luminance information refC'. In this way, refY', recY' and refC' can be independently input together to the neural network model.

[0242] That is, in the embodiment of the present application, each element in the reference luminance information refY in the one-dimensional array format of the inSize number is replicated LSizeh*LSizew times and arranged in the second and third dimensions in the format of LSizew×LSizeh to obtain refY' in the three-dimensional array format of inSize×LSizew×LSizeh, the reconstructed luminance information recY in the two-dimensional array format is replicated inSize times along the first dimension as a whole to obtain recY' in the three-dimensional array format of inSize×LSizew×LSizeh, and each element in the reference chromaticity information refC in the one-dimensional array format of the outSize number is replicated LSizeh*LSizew times and arranged in the second and third dimensions in the format of LSizew×LSizeh to obtain refC' in the three-dimensional array format of outSize×LSizew×LSizeh. In this way, recY', refY' and refC' are directly input to the neural network model, where there can be multiple combinations of recY', refY' and refC' as model inputs. For example, recY', refY', and refC' may be concatenated in the first dimension, and then a new dimension may be added to the concatenated vector in the first dimension to form one input vector as an input to the neural network model, see FIG. 13 for details. Alternatively, recY' and refY' may be concatenated in the first dimension, and then a new dimension may be added to each of the concatenated vector and refC' in the first dimension to form one input vector as an input to the neural network model, see FIG. 14 for details. Alternatively, recY' and refY' may be subtracted element by element to form one vector, and then a new dimension may be added to each of the concatenated vector and refC' in the first dimension to form one input vector as an input to the neural network model, see FIG. 15 for details. Alternatively, recY', refY', and refC' may be concatenated in the first dimension to form one input vector as an input to the neural network model, see FIG. 16 for details.

[0243] In yet another specific embodiment, referring to Fig. 17, an eighth schematic diagram of a chromaticity prediction block diagram based on a neural network model according to an embodiment of the present application is shown. As shown in Fig. 11, for the input information of the neural network model, the reference luminance information refY is duplicated LSizew×LSizeh times for each element to obtain the size of the reference luminance information refY' inSize×LSizew×LSizeh, and one dimension is newly added to the reconstructed luminance information recY before concatenation to obtain the size of the concatenated information (1+inSize)×LSizew×LSizeh, and the reference chromaticity information refC is duplicated LSizew×LSizeh times for each element to obtain the size of the reference luminance information refC' outSize×LSizew×LSizeh. In this way, the concatenated information and refC' are independently input to the neural network model.

[0244] That is, in the embodiment of the present application, each element in the reference luminance information refY in the one-dimensional array format of the inSize number is duplicated LSizeh*LSizew times, and arranged in the second and third dimensions in the format of LSizew×LSizeh to obtain refY' in the three-dimensional array format of inSize×LSizew×LSizeh. A new dimension is added to the first dimension of the two-dimensional array recY to form recY' in the three-dimensional array format of 1×LSizew×LSizeh. Each element in the reference chromaticity information refC in the one-dimensional array format of the outSize number is duplicated LSizeh*LSizew times, and arranged in the second and third dimensions in the format of LSizew×LSizeh to obtain refC' in the three-dimensional array format of outSize×LSizew×LSizeh. Here, recY', refY', and refC' are directly input together to the neural network model. Here, recY', refY', and refC' can have multiple combination methods as inputs to the neural network model. For example, recY' can be firstly added with one new dimension to the first dimension, and then concatenated with refY' in the first dimension. Then, one new dimension is added to the first dimension for each of the concatenated vector and refC', and these are independently input to the network together. For details, see FIG. 17.

[0245] Also, referring to FIG. 13 to FIG. 17, at this time, the output of the neural network model can be a weighting coefficient cWeight, and the size of the weighting coefficient cWeight is outSize×LSizew×LSizeh. Then, calculate the vector product of each column of the vector of the channel dimension of the weighting coefficient cWeight and the reference chromaticity information refC, thereby obtaining the prediction block of the current block. Or, the output of the neural network model can be a direct prediction block, and there is no limitation here.

[0246] For example, when the output of the neural network model is a weighting coefficient, refer to FIG. 18 to show a ninth schematic diagram of a chromaticity prediction block diagram based on a neural network model according to an embodiment of the present application. As shown in FIG. 18, for the input information of the neural network model, the reference luminance information refY is replicated LSizew×LSizeh times for each element to obtain the size of the reference luminance information refY' inSize×LSizew×LSizeh, the reconstructed luminance information recY is replicated inSize times overall to obtain the size of the reconstructed luminance information recY' inSize×LSizew×LSizeh, and the reference chromaticity information refC is replicated LSizew×LSizeh times for each element to obtain the size of the reference luminance information refC' outSize×LSizew×LSizeh. In this way, refY', recY' and refC' are independently input to the neural network model together, and at this time, the output of the neural network model may be a weighting coefficient cWeight, and the size of the weighting coefficient cWeight is outSize×LSizew×LSizeh. Then, the vector product of each column vector in the channel dimension of the weighting coefficient cWeight and the reference chrominance information refC is calculated, thereby obtaining a prediction block of the current block.

[0247] For example, when the output of the neural network model is a prediction block, refer to FIG. 19 to show a tenth schematic diagram of a chrominance prediction block diagram based on a neural network model according to an embodiment of the present application. As shown in FIG. 19, for the input information of the neural network model, the reference luma information refY is replicated LSizew×LSizeh times for each element to obtain the size of the reference luma information refY' inSize×LSizew×LSizeh, the reconstructed luma information recY is replicated inSize times overall to obtain the size of the reconstructed luma information recY' inSize×LSizew×LSizeh, and the reference chroma information refC is replicated LSizew×LSizeh times for each element to obtain the size of the reference luma information refC' outSize×LSizew×LSizeh. In this way, refY', recY' and refC' can be independently input to the neural network model together, and the output of the neural network model can be a prediction block of the current block directly.

[0248] This embodiment provides a decoding method, and the specific implementation of the above embodiment is detailed in this embodiment. Based on the technical solution of the above embodiment, it can be seen that the chrominance prediction based on neural network can provide a variety of effective network input information, and can perform chrominance prediction better. In addition, the reconstructed luminance information difference between adjacent regions and the current block, or the result after the correlation processing of this reconstructed luminance information difference, is used as one of the inputs of the neural network, so that the network plays a leading role in learning the correlation between the reference chrominance and the predicted chrominance of the current predicted block, thereby further improving the accuracy of chrominance prediction, saving the code rate, and simultaneously improving the encoding and decoding performance.

[0249] In another embodiment of the present application, referring to Fig. 20, a first schematic flowchart of an encoding method according to an embodiment of the present application is shown. As shown in Fig. 20, the method may include the following S2001 to S2004.

[0250] In S2001, a reference sample value of the first color component of the current block is determined.

[0251] It should be noted that the encoding method of the embodiment of the present application is applied to an encoding device or an encoding device (which may be abbreviated as "encoder") integrated with the encoding device. In addition, the encoding method of the embodiment of the present application may specifically refer to an intra prediction method, more specifically, a neural network-based chromaticity prediction (NWCP) method.

[0252] In an embodiment of the present application, a video image can be divided into a plurality of coding blocks, each coding block can include a first color component, a second color component, and a third color component, where the current block refers to a coding block in a video image on which intra prediction is currently performed. Also, assuming that the current block performs prediction of the first color component, and the first color component is a luma component, i.e., the component to be predicted is a luma component, the current block can also be called a luma prediction block. Or, assuming that the current block performs prediction of the second color component, and the second color component is a chroma component, i.e., the component to be predicted is a chroma component, the current block can also be called a chroma prediction block.

[0253] It should be further illustrated that in the embodiments of the present application, the reference information of the current block may include values ​​of first color component sampling points and second color component sampling points in the neighboring regions of the current block, and these sampling points may be determined based on the encoded pixels in the neighboring regions of the current block. In some embodiments, the neighboring regions of the current block may include at least one of an upper neighboring region, a top-right neighboring region, a left neighboring region, and a lower-left neighboring region.

[0254] In some embodiments, the method may further include determining a neighboring reference value of the first color component of the current block based on the first color component sampling points in a neighboring region of the current block.

[0255] It should be noted that in the embodiment of the present application, the reference pixel of the current block may specifically refer to the reference pixel point adjacent to the current block. The adjacent here may be, but is not limited to, spatial adjacent. For example, the adjacent may be temporal adjacent, spatial and temporal adjacent, and the reference pixel of the current block may further be a reference pixel obtained after performing a certain process on the reference pixel point adjacent in the spatial domain, adjacent in the temporal domain, adjacent in the spatial domain and adjacent in the temporal domain, etc., but the embodiment of the present application does not make any limitation.

[0256] It should be further explained that in the embodiment of the present application, both the first color component sampling point and the second color component sampling point in the adjacent region of the current block can be referred to as the reference pixel of the current block. Assuming that the first color component is a luminance component and the second color component is a chrominance component, the value of the first color component sampling point in the adjacent region of the current block is the reference luminance information corresponding to the reference pixel of the current block, and the value of the second color component sampling point in the adjacent region of the current block is the reference chrominance information corresponding to the reference pixel of the current block.

[0257] It should be further explained that in the embodiments of the present application, the adjacent regions of the current block may include only the upper adjacent region, or may include only the left adjacent region, or may include the upper adjacent region and the upper right adjacent region, or may include the left adjacent region and the lower left adjacent region, or may include the upper adjacent region and the left adjacent region, or may further include the upper adjacent region, the upper right adjacent region and the left adjacent region, etc., but the embodiments of the present application do not make any limitations thereon.

[0258] It should be further explained that in the embodiment of the present application, the neighboring region may be determined based on the prediction mode of the current block. For example, if the prediction mode of the current block is a horizontal mode, only the upper neighboring region and / or the upper right neighboring region can be selected as the neighboring region in the prediction of the chrominance component. If the prediction mode of the current block is a vertical mode, only the left neighboring region and / or the lower left neighboring region can be selected as the neighboring region in the prediction of the chrominance component.

[0259] Further, in some embodiments, for determining the adjacent reference value of the first color component, the method may further include performing a screening process on values ​​of the first color component sampling points in the adjacent region to determine the adjacent reference value of the first color component of the current block.

[0260] It should be noted that the first color component sampling points in the adjacent region may have some insignificant sampling points (e.g., these sampling points have poor correlation) or some abnormal sampling points, and in order to ensure the accuracy of the prediction, these sampling points need to be deleted so as to obtain valid first color component sampling point values.

[0261] In one specific embodiment, performing a screening process on the values ​​of the first color component sampling points in the neighboring region to determine the neighboring reference value of the first color component of the current block includes: determining a location of a selected sampling point based on a location and / or a color component intensity of a first color component sampling point in the adjacent region; determining a neighboring reference value of the first color component of the current block from the neighboring region based on the position of the selected sampling point.

[0262] It should be noted that in the embodiment of the present application, the color component intensity can be represented by color component information such as reference luminance information, reference chrominance information, etc., where the larger the value of the color component information, the higher the color component intensity. In this way, screening the value of the first color component sampling point in the neighboring region can be based on the position of the sampling point or based on the color component intensity, thereby determining the valid first color component sampling point value based on the sampling point obtained by screening, and further determining the neighboring reference value of the first color component of the current block.

[0263] In some embodiments, for a neighboring reference value of the first color component, the method may further include performing a first filtering process on values ​​of the first color component sampling points in the neighboring region to determine a neighboring reference value of the first color component of the current block.

[0264] It should be noted that in the embodiment of the present application, the first filtering process may be a downsampling filtering process, where the first color component is a luminance component, and the downsampling filtering process is performed on the reference luminance information, so that the spatial resolution of the reference luminance information after filtering can be the same as the spatial resolution of the reference chromaticity information. Alternatively, the first filtering process may be an upsampling filtering process, so as to preserve the reference luminance information without loss, but no limitation is made here.

[0265] It should be further noted that in the embodiments of the present application, the reference information of the current block may further include a reconstructed value of a first color component sampling point in the current block, in which case, in some embodiments, the method may further include determining a reference value of the first color component in the current block based on the reconstructed value of the first color component sampling point in the current block.

[0266] In the embodiment of the present application, assuming that the first color component is a luminance component, the reconstructed value of the first color component sampling point in the current block is, namely, the reconstructed luminance information of the current block.

[0267] In some embodiments, for the reference value of the first color component in the current block, the method may further include performing a second filtering process on the reconstructed values ​​of the first color component sampling points in the current block to determine the reference value of the first color component in the current block.

[0268] It should be noted that in the embodiment of the present application, the second filtering process can be a downsampling filtering process, where the first color component is a luminance component, and the downsampling filtering process can also be performed on the reconstructed luminance information in the current block. Or, the second filtering process can be an upsampling filtering process, so as to preserve the reconstructed luminance information without loss, and there is no limitation here.

[0269] In some embodiments, for S2001, determining a reference sample value of a first color component of the current block includes: Determining a neighboring reference value of the first color component of the current block based on the first color component sampling points in the neighboring region of the current block; and determining a reference sample value of the first color component of the current block based on a reconstructed value of the first color component sampling point in the current block and an adjacent reference value of the first color component of the current block.

[0270] In one specific embodiment, determining a reference sample value of a first color component of a current block according to a reconstruction value of a first color component sampling point in the current block and a neighboring reference value of the first color component of the current block includes: determining a reference value of the first color component in the current block based on the reconstructed value of the first color component sampling point in the current block; determining a reference sample value of the first color component of the current block based on the reference value of the first color component in the current block and a neighboring reference value of the first color component of the current block.

[0271] It should be noted that in the embodiment of the present application, the reference sample value of the first color component of the current block may be luminance difference information. Here, the reference sample value of the first color component of the current block may be an absolute value of the difference between the adjacent reference value of the first color component of the current block and the reference value of the first color component in the current block. In addition, in the embodiment of the present application, the reference sample value of the first color component of the current block may further be an absolute value of the difference between the reconstructed value of the first color component sampling point in the current block and the adjacent reference value of the first color component of the current block. Or, the reference sample value of the first color component of the current block may further be an absolute value of the difference between the reconstructed value of the first color component sampling point in the current block and the value of the first color component sampling point in the adjacent area of ​​the current block. Or, the reference sample value of the first color component of the current block may further be an absolute value of the difference between the reference value of the first color component in the current block and the value of the first color component sampling point in the adjacent area of ​​the current block, etc., but no limitation is made herein.

[0272] It can be further understood that in embodiments of the present application, for a second color component sampling point in the adjacent region, in some embodiments, the method may further include performing a third filtering process on the value of the second color component sampling point in the adjacent region to determine a reference sample value of the second color component of the current block.

[0273] It should be noted that in the embodiment of the present application, the third filtering process can be an upsampling filtering process, or the third filtering process can be a downsampling filtering process, but no limitation is made herein, where the second color component is a chromaticity component, and in this case, the upsampling filtering process or the downsampling filtering process is performed on the reference chromaticity information, so that the spatial resolution of the reference chromaticity information after filtering can be finally made the same as the spatial resolution of the reference luminance information.

[0274] That is, the reference chrominance information and the reference luma information of the current block may be obtained from the neighboring regions. Here, the obtained reference chrominance information includes, but is not limited to, the reference chrominance reconstructed value of the upper region of the selected current block and / or the reference chrominance reconstructed value of the left region. The obtained reference luma information includes, but is not limited to, the corresponding reference luma information obtained based on the position of the reference chrominance information. The manner of obtaining the reconstructed luma information in the current block includes, but is not limited to, obtaining the corresponding reconstructed luma information based on the position of the chrominance information in the current block as the reconstructed luma information of the current block.

[0275] It can be further understood that in the embodiment of the present application, whether the reference chrominance information, the reference luma information, or the reconstructed luma information can be the target information of the current block after a correlation filtering process is performed under certain conditions. In addition, the correlation filtering operation here can be realized based on the manner of a neural network structure. No limitation is made here. In this way, the target information of the current block can include the neighbor reference value (i.e., reference luma information) of the first color component of the current block, the reference value (i.e., reconstructed luma information) of the first color component in the current block, and the reference sample value (i.e., reference chrominance information) of the second color component of the current block. These information may be information after filtering or information without filtering, and are not specifically limited in the embodiment of the present application.

[0276] It can be further understood that in the embodiments of the present application, the luminance difference information is composed of a plurality of luminance difference matrices. Thus, in some embodiments, the reference sample value of the first color component of the current block can include N matrices, where N represents the number of reference sample values ​​of the second color component of the current block, and the k-th matrix includes an absolute value of the difference between the k-th value of the adjacent reference value of the first color component of the current block and the reference value of the first color component in the current block, where k=0,1,...,N-1.

[0277] In S2002, a weighting coefficient is determined by a neural network model based on a reference sample value of a first color component of a current block, where an input of the neural network model includes the reference sample value of the first color component of the current block.

[0278] It should be noted that in the embodiment of the present application, after determining the reference sample value of the first color component of the current block, the reference sample value of the first color component of the current block can be used as the input of a neural network model, and the output of the neural network model is a weighting coefficient.

[0279] In one possible implementation form, when the reference sample value of the first color component of the current block includes N matrices, for S2002, determining a weighting coefficient by a neural network model based on the reference sample value of the first color component of the current block may include taking the N matrices as inputs of the neural network model and determining the output of the neural network model as the weighting coefficient.

[0280] It should be further explained that in the embodiment of the present application, the N matrices are used as the input of the neural network model in their entirety, and the output of the neural network model is the weight coefficient. Illustratively, deltaLuma represents a three-dimensional array or three-dimensional matrix, where deltaLuma is used as the input of the neural network model in its entirety, and mWeight represents the weight coefficient.

[0281] In another possible implementation, the luminance difference information (e.g., N matrices) can be subjected to nonlinear processing, and then the processed result can be taken as the reference sample of the first color component of the current block. In some embodiments, the method includes: determining values ​​corresponding to the N matrices in a preset mapping relationship; The method may further include determining a reference sample value of a first color component of the current block based on values ​​corresponding to the N matrices.

[0282] Here, in one specific embodiment, the preset mapping relationship is a Softmax function as shown in the above formula (1) and formula (17). In another specific embodiment, the preset mapping relationship is a weighting function having an inverse proportional relationship with the i-th matrix as shown in the above formula (2) and formula (3).

[0283] Correspondingly, in some embodiments, for S2002, determining a weighting coefficient by a neural network model based on a reference sample value of the first color component of the current block may include taking values ​​corresponding to the N matrices in the preset mapping relationship as inputs of the neural network model, and determining the output of the neural network model as the weighting coefficient.

[0284] It should be noted that after determining the values ​​corresponding to the N matrices in the preset mapping relationship, the values ​​corresponding to the N matrices are also used as the input of the neural network model, and the output of the neural network model is the weighting coefficients.

[0285] It should be further explained that in the embodiment of the present application, the predetermined mapping relationship may be a predetermined functional relationship such as a Softmax function, or may be a predetermined look-up table method, for example, an array element look-up table method is used to reduce some calculation operations, but is not specifically limited in the embodiment of the present application.

[0286] It should be further explained that in the embodiment of the present application, the reference sample value of the first color component of the current block is a three-dimensional data structure. Specifically, after calculating the absolute value of the difference between the adjacent reference value of the first color component of the current block and the reference value of the first color component in the current block, the absolute value of the difference value can be stored in N matrices of the three-dimensional data structure. Or, each element in the adjacent reference value of the first color component of the current block is duplicated multiple times to obtain the adjacent reference value of the first color component of the three-dimensional data structure, and the whole reference value of the first color component in the current block is duplicated multiple times to obtain the reference value of the first color component of the three-dimensional data structure, and then the absolute value of the difference value can be calculated according to the adjacent reference value of the first color component of the three-dimensional data structure and the reference value of the first color component of the three-dimensional data structure to obtain the N matrices of the three-dimensional data structure, and no limitation is made here.

[0287] It can be understood that in the embodiment of the present application, in addition to being luminance difference information or information obtained by performing nonlinear processing on luminance difference information, the input of the neural network model may also be other types of information. For example, the reference value of the first color component in the current block and the adjacent reference value of the first color component of the current block are directly input to the neural network model, or the information obtained by performing concatenation processing on them is input to the neural network model, or the information obtained by combining them with the reference sample value of the second color component of the current block is input to the neural network model. The following provides relevant descriptions of these situations.

[0288] In one possible implementation form, for S2002, determining a weighting coefficient by a neural network model based on a reference sample value of a first color component of the current block includes: The method may include taking the reference value of the first color component in the current block and the adjacent reference value of the first color component of the current block as inputs of a neural network model, and determining the output of the neural network model as the weighting coefficients.

[0289] It should be noted that in this case, there is no need to perform a correlation operation on the reference values ​​of the first color component in the current block and the adjacent reference values ​​of the first color component of the current block, but they can be directly used as inputs to the neural network model.

[0290] In another possible implementation form, for S2002, determining a weighting coefficient by a neural network model based on a reference sample value of a first color component of the current block includes: performing a linking process on the reference value of the first color component in the current block and the adjacent reference value of the first color component of the current block to obtain first linking information; The method may include: determining the first link information as an input of a neural network model and an output of the neural network model as a weighting coefficient.

[0291] It should be noted that in this case, a first-dimension connection process can be performed on the reference value of the first color component in the current block and the adjacent reference value of the first color component of the current block, but there is no need to perform absolute value calculation of the difference between the two. At this time, the obtained first connection information is used as the input of a neural network model, and the first connection information is processed by the neural network model, and the final output of the neural network model is a weighting coefficient.

[0292] In yet another possible implementation form, for S2002, the method includes: The method may further include: taking the reference sample value of the first color component of the current block and the reference sample value of the second color component of the current block as inputs of a neural network model, and determining the output of the neural network model as a weighting coefficient.

[0293] In one specific embodiment, the method may include taking the reference value of the first color component in the current block, the adjacent reference value of the first color component of the current block, and the reference sample value of the second color component of the current block as inputs of a neural network model, and determining the output of the neural network model as a weighting coefficient.

[0294] It should be noted that in this case, all of the target information of the current block (including reference luma information, reference chroma information in adjacent regions, and reconstructed luma information in the current block) can be input to the neural network model, and then these information are processed by the neural network model, and finally the output of the neural network model is the weighting coefficient.

[0295] In another specific embodiment, the method may include: performing a connection process on the reference value of the first color component in the current block and the adjacent reference value of the first color component of the current block to obtain second connection information; taking the second connection information and the reference sample value of the second color component of the current block as the input of a neural network model, and determining the output of the neural network model as a weighting coefficient.

[0296] It should be noted that in this case, a first-dimension connection process can be performed on the reference value of the first color component in the current block and the adjacent reference value of the first color component of the current block, but there is no need to perform absolute value calculation of the difference between the two; then, the obtained second connection information and the reference sample value of the second color component of the current block are jointly input into a neural network model, and this information is processed by the neural network model; and finally, the output of the neural network model is a weighting coefficient.

[0297] In yet another specific embodiment, the method may include performing a connection process on the reference value of the first color component in the current block, the adjacent reference value of the first color component of the current block, and the reference sample value of the second color component of the current block to obtain third connection information; using the third connection information as the input of a neural network model, and determining the output of the neural network model as a weighting coefficient.

[0298] It should be noted that in this case, a first-dimensional connection can be performed on the three elements, namely, the reference value of the first color component in the current block, the adjacent reference value of the first color component of the current block, and the adjacent reference value of the second color component of the current block; then, the obtained third connection information is input to a neural network model, and the third connection information is processed by the neural network model; and finally, the output of the neural network model is a weighting coefficient.

[0299] It should be further explained that in the embodiment of the present application, the input of the neural network model needs to be a three-dimensional data structure, that is, the reference value of the first color component in the current block, the adjacent reference value of the first color component in the current block, and the reference sample value of the second color component in the current block need to be converted into a three-dimensional data structure before being used as the input of the neural network model, for example, the reference value of the first color component in the current block is globally replicated, the adjacent reference value of the first color component in the current block is element-by-element replicated, or the reference sample value of the second color component in the current block is element-by-element replicated, etc., before being used as the input of the neural network model.

[0300] It should also be noted that for the neural network model, when the input of the neural network model requires more dimensional information, the embodiment of the present application can further perform dimensional expansion on the input information, and then, after processing the expanded information by the neural network model, perform dimensional reduction processing on the output of the neural network model to return to the original dimension, and further obtain the weighting coefficient of the three-dimensional data structure.

[0301] With respect to the neural network model, it can be appreciated that in some embodiments, the neural network model may be a convolutional neural network model.

[0302] Wherein, the convolutional neural network model may include at least one base unit and one target unit, where: The basic unit includes at least a first convolutional layer and a first activation function.

[0303] The target unit includes at least a second convolutional layer and a second activation function.

[0304] In an embodiment of the present application, the first activation function is a ReLU function, and the second activation function is a softmax function. In one specific embodiment, the convolutional neural network model may be configured by seven basic units and a target unit formed by a final layer. Here, the basic units may be configured by a convolutional layer with a stride of 1, same mode padding, convolution kernel of 3×3, and a ReLU first activation function, and the target unit may be configured by a convolutional layer with a stride of 1, same mode padding, convolution kernel of 3×3, and a softmax activation function.

[0305] It should be noted that same mode padding indicates that the size of the feature map after convolution does not change, i.e. the size of the input image is the same as the size of the output image.

[0306] It should be further explained that the number of layers of the convolutional neural network model used herein is not limited. The network structure includes, but is not limited to, a convolutional layer, a pooling layer, an activation function, etc. The size of the convolution kernel can be 3×3, 5×5 or other sizes, the convolution type can be normal convolution, transposed convolution, transformable convolution, etc., whether it is the first activation function or the second activation function, includes, but is not limited to, a ReLU activation function, a softmax activation function, etc., and is not specifically limited in the embodiments of the present application.

[0307] With respect to the neural network model, it can be further appreciated that in some embodiments, the neural network model is a fully-connected neural network model.

[0308] Wherein, the fully-connected neural network model may include at least one basic unit and one target unit, where: The basic unit includes at least a first fully connected layer and a first activation function.

[0309] The target unit includes at least a second fully connected layer and a second activation function.

[0310] In an embodiment of the present application, the first activation function is a ReLU function, and the second activation function is a softmax function. In one specific embodiment, the fully-connected neural network model can be configured to use a fully-connected network with two basic units and a target unit formed by a final layer, where the basic units can be configured by a fully-connected layer and a ReLU activation function, and the target unit can be configured by a fully-connected layer and a Softmax activation function.

[0311] It should be further explained that the number of layers of the fully connected network model used is not limited. The network structure includes, but is not limited to, a fully connected layer, an activation function, etc. There is no fixed limit on the number of neurons in the fully connected layer. The activation function includes, but is not limited to, a ReLU activation function, a softmax activation function, a Sigmoid activation function, etc., and is not specifically limited in the embodiments of the present application.

[0312] In the present embodiment, since deltaLuma[k][y][x] is a three-dimensional matrix data structure, it can be further understood that deltaLuma[k][y][x] can also be considered as a W×H one-dimensional matrix. Thus, in some embodiments, the method further comprises: The reference sample values ​​of the first color component of the current block may include W×H one-dimensional matrices, and the one-dimensional matrix includes N weighting coefficients, where W×H represents the number of sampling points of the second color component of the current block, and N represents the number of reference sample values ​​of the second color component of the current block.

[0313] It should be noted that in the embodiment of the present application, W×H here represents the number of second color component sampling points of the current block, or may be the number of second color component prediction points of the current block, or may be the number of second color component prediction points determined by calculation.

[0314] It should be further explained that in this case, each one-dimensional matrix includes N weighting coefficients, and the predicted value of the second color component predicted point corresponding to the one-dimensional matrix in the current block is determined according to the N weighting coefficients. Here, in one one-dimensional matrix, the sum of the N weighting coefficients is equal to 1, and each weighting coefficient is greater than or equal to 0 and less than or equal to 1. However, it should be noted that "the sum of the N weighting coefficients is equal to 1" is only a theoretical concept, and in the actual fixed-point realization process, the absolute value of the weighting coefficient may be greater than 1.

[0315] In S2003, a prediction block of the second color component of the current block is determined based on the weighting coefficient and the reference sample value of the second color component of the current block.

[0316] It should be noted that after the weighting coefficient is output by the neural network model, a weighting calculation can be performed based on the weighting coefficient and the reference sample value of the second color component of the current block, and then a predicted block of the second color component of the current block can be determined.

[0317] In some embodiments, for S2003, determining a prediction block of a second color component of the current block based on the weighting coefficients and the reference sample values ​​of the second color component of the current block includes: determining a first one-dimensional matrix for calculating a second color component sampling point to be predicted according to a position index and a weighting coefficient of the second color component sampling point to be predicted in the current block; determining a prediction block of the second color component of the current block based on the first one-dimensional matrix and the reference sample values ​​of the second color component of the current block.

[0318] Here, the first one-dimensional matrix represents a one-dimensional matrix corresponding to the position index in the three-dimensional matrix data structure of weighting coefficients.

[0319] That is, the weighting coefficient has a three-dimensional matrix data structure. If the position index of the second color component sampling point to be predicted in the current block is the wth column and the hth row, the first one-dimensional matrix is ​​the wth column, the hth row one-dimensional matrix in the three-dimensional matrix data structure of the weighting coefficient.

[0320] In some other embodiments, for S2003, determining a prediction block of the second color component of the current block based on the weighting coefficient and a reference sample value of the second color component of the current block may include determining a prediction block of the second color component of the current block based on a position index of the second color component sampling point to be predicted within the current block and the weighting coefficient.

[0321] In an embodiment of the present application, the predicted value of the second color component sampling point to be predicted in the prediction block can be set equal to a weighted sum of the reference sample value of the second color component of the current block and a coefficient in a second one-dimensional matrix, where the second one-dimensional matrix represents a one-dimensional matrix corresponding to a position index in the three-dimensional matrix data structure of the weighting coefficients.

[0322] That is, still take the position index of the second color component sampling point to be predicted in the current block as an example, the position index is at the wth column and the hth row, for this second color component sampling point to be predicted, it correspondingly includes N weighting coefficients in the one-dimensional matrix at the wth column and the hth row in the three-dimensional matrix data structure of the weighting coefficients, and then determine the weighting value between the reference sample value of the second color component of the current block and the corresponding coefficient among the N weighting coefficients, and the predicted value of this second color component sampling point to be predicted can be set equal to the sum of the N weighting values, where N represents the number of the reference sample values ​​of the second color component, and N is a positive integer.

[0323] It should be further noted that in the embodiments of the present application, the weighting coefficients output by the neural network model can be nonlinearly processed before being weighted with the reference sample value of the second color component of the current block. In some embodiments, the method further includes: determining a value corresponding to the weighting coefficient in a preset mapping relationship after determining the weighting coefficient through the neural network model; and updating the weighting coefficients based on values ​​corresponding to the weighting coefficients in the preset mapping relationship.

[0324] Here, the weighting coefficients output by the neural network model can be nonlinearly processed in one dimension or multiple dimensions. For example, the preset mapping relationship of the nonlinear processing is as shown in the above formula (4) or formula (18).

[0325] It should be further noted that in the embodiment of the present application, for the weighting coefficients output by the neural network model, if the weighting coefficients are floating-point weighting coefficients, fixed-point processing can be performed on them, or if the predicted value calculated according to the second color component sampling points to be predicted is a floating-point number, fixed-point processing can be performed on it.

[0326] Also, the output of the neural network model may be a weighting coefficient that participates in the calculation of the chrominance prediction value, or may be a prediction block of the chrominance components of the current block directly. determining a predicted block of the second color component of the current block by a neural network model according to the reference sample value of the first color component of the current block and the reference sample value of the second color component of the current block; and determining, based on the predicted block, reconstructed values ​​of the second color component sampling points of the current block.

[0327] In an embodiment of the present application, the method may include using a reference sample value of a first color component of a current block and a reference sample value of a second color component of the current block as inputs to a neural network model, and determining an output of the neural network model as a predicted block.

[0328] It should be noted that, for the input of the neural network model, the reference value of the first color component in the current block, the adjacent reference value of the first color component in the current block, and the reference sample value of the second color component in the current block can be the input of the neural network model. Alternatively, the reference value of the first color component in the current block, the adjacent reference value of the first color component in the current block, and the reference sample value of the second color component in the current block can be concatenated, and the concatenated information can be the input of the neural network model. Alternatively, the reference value of the first color component in the current block and the adjacent reference value of the first color component in the current block can be concatenated, and the concatenated information and the reference sample value of the second color component in the current block can be the input of the neural network model. Alternatively, the reference value of the first color component in the current block and the adjacent reference value of the first color component in the current block can be subtracted to calculate the absolute value, and then the absolute value of the difference value and the reference sample value of the second color component in the current block can be the input of the neural network model. In this case, the output of the neural network model can be directly used as the predicted block of the second color component of the current block.

[0329] In S2004, a reconstructed value of a second color component sampling point of the current block is determined based on the predicted block.

[0330] It should be noted that in the embodiment of the present application, the prediction block may include prediction values ​​of at least some second color component sampling points in the current block. Here, if the prediction block includes prediction values ​​of all second color component sampling points in the current block, then there is no need to perform any processing on the prediction block, and the prediction block can be directly used as the final prediction block. If the prediction block includes prediction values ​​of some second color component sampling points in the current block, then it is necessary to perform upsampling filtering on the prediction block to obtain the final prediction block. If the number of prediction values ​​of the second color component included in the prediction block is the same as the number of second color component sampling points included in the current block, but does not include prediction values ​​of the second color component sampling points of the current block, then it is necessary to enhance the prediction value by filtering to obtain the final prediction block. Therefore, in some embodiments, the method may further include: after determining the prediction block of the second color component of the current block, performing a correlation process on the prediction block, and using the processed prediction block as the prediction block.

[0331] In one specific embodiment, the predicted value of the second color component sampling point in the predicted block generally needs to be limited within a preset range. Therefore, in some embodiments, performing correlation processing on the predicted block can include performing a correction operation on the predicted value of the second color component sampling point in the predicted block to determine the predicted block of the second color component of the current block.

[0332] It should be noted that in the embodiments of the present application, the preset range can be between 0 and (1<<BitDepth)-1. Here, BitDepth is the bit depth required by the chrominance component. When the predicted value in the prediction block (predWcp) exceeds the value in the preset range, it is necessary to perform a corresponding correction operation on the predicted value. Exemplarily, a clamp operation can be performed on predWcp[i][j] to ensure that the predicted values of all the second color component sampling points in the prediction block are all between 0 and (1<<BitDepth)-1.

[0333] In another specific embodiment, a post-processing operation can also be performed on the prediction block. Therefore, in some embodiments, performing a correlation process on the prediction block performing a filtering enhancement process on the predicted value of the second color component sampling point in the prediction block, performing a correction process on the predicted value of the second color component sampling point in the prediction block by using a preset compensation value, performing a weighted fusion process on the predicted value of the second color component sampling point in the prediction block by using the predicted value of the second color component of the current block in at least one prediction mode, can further include at least one of them.

[0334] It should be noted that in the NWCP mode, in order to reduce instability caused by the independent and parallel prediction of each pixel of the NWCP in the correlation process of the predicted block, a smoothing filtering process may be performed on the predicted block to obtain a final chrominance predicted value. Alternatively, in order to further improve the accuracy of the NWCP predicted value, a position-related correction process may be performed on the predicted block. For example, a chrominance compensation value is calculated for each prediction target pixel using a reference pixel that is spatially close, and the predicted block is corrected using the chrominance compensation value, and the corrected predicted value is the final chrominance predicted value. Alternatively, in order to further improve the accuracy of the NWCP predicted value, a weighted fusion is performed on the chrominance predicted value calculated in another chrominance prediction mode and the fusion result is the final chrominance predicted value. Alternatively, in order to improve the NWCP prediction performance, the chrominance predicted value calculated in the NWCP may be corrected using a neural network model, and the embodiment of the present application does not limit this.

[0335] In some embodiments, after determining the prediction block, as shown in FIG. 21, after S2004, the following steps S2101 to S2103 may be further included.

[0336] In S2101, a predicted value of the second color component sampling point of the current block is determined based on the predicted block.

[0337] In S2102, a predicted difference value of the second color component sampling point of the current block is determined based on the original value of the second color component sampling point of the current block and the predicted value of the second color component sampling point of the current block.

[0338] In S2103, the prediction difference value of the second color component sampling point of the current block is coded, and the resulting coded bits are written into the bitstream.

[0339] It should be noted that in the embodiment of the present application, the prediction block may include predicted values ​​of at least some second color component sampling points in the current block. Wherein, if the prediction block includes predicted values ​​of all second color component sampling points in the current block, the predicted values ​​of the second color component sampling points of the current block may be set equal to the values ​​of the prediction block. If the prediction block includes predicted values ​​of some second color component sampling points in the current block, the values ​​of the prediction block may be up-sampling filtered, and the predicted values ​​of the second color component sampling points of the current block may be set equal to the output values ​​after the up-sampling filtering.

[0340] It should be further explained that after the predicted value of the second color component sampling point of the current block is determined, a predicted difference value of the second color component sampling point can be determined based on the original value of the second color component sampling point and the predicted value of the second color component sampling point. Specifically, a subtraction is performed between the original value of the second color component sampling point and the predicted value of the second color component sampling point, thereby determining a predicted difference value of the second color component sampling point of the current block. In this way, after the predicted difference value of the second color component sampling point is written into the bitstream, the predicted difference value of the second color component sampling point can be obtained by decoding at the subsequent decoding side, thereby restoring the reconstructed value of the second color component sampling point of the current block.

[0341] From the above, it can be seen that the embodiment of the present application is a neural network-based chrominance prediction NWCP technology, which mainly includes: on the one hand, fully obtaining existing information of the current block, including reconstructed luminance information of neighboring regions, reconstructed chrominance information of neighboring regions, and reconstructed luminance information of the current block; on the other hand, providing a variety of effective input information for neural network-based chrominance prediction, i.e., the reconstructed luminance information difference between the current block and the reference pixel, or the result after the reconstructed luminance information difference is processed, so that chrominance prediction can be performed better.

[0342] An embodiment of the present application provides an encoding method, which determines a reference sample value of a first color component of a current block, and determines a weighting coefficient by a neural network model according to the reference sample value of the first color component of the current block, where the input of the neural network model includes the reference sample value of the first color component of the current block, determines a predicted block of the second color component of the current block according to the weighting coefficient and the reference sample value of the second color component of the current block, and determines a predicted difference value of the second color component sampling point of the current block according to the predicted block. In this way, by fully utilizing the color component information in the neighboring region of the current block and the color component information in the current block, a variety of effective input information can be provided for chrominance prediction based on the neural network, so that chrominance prediction can be performed better. In addition, by using the luminance difference between the reference pixel and the current block, or the result after correlation processing of this luminance difference, as one of the inputs of the neural network, the neural network can also learn the correlation between the reference chrominance and the predicted chrominance of the current block, and this correlation can be included in the chrominance prediction of the current block, thereby further improving the accuracy of chrominance prediction, saving the code rate, and simultaneously improving the encoding and decoding performance.

[0343] In another embodiment of the present application, the embodiment of the present application further provides a bitstream, where the bitstream is generated by performing bit encoding based on encoding target information, where the encoding target information includes at least a prediction difference value of a second color component sampling point of the current block.

[0344] In the embodiment of the present application, after the prediction difference value of the second color component sampling point of the current block is transmitted from the encoding side to the decoding side, the decoding side can obtain the prediction difference value of the second color component sampling point by decoding, and then combine the prediction value of the second color component sampling point of the current block, thereby restoring the reconstruction value of the second color component sampling point of the current block. In this way, by fully utilizing the color component information in the neighboring area of ​​the current block and the color component information in the current block, a variety of effective input information can be provided for the chrominance prediction based on the neural network, so that the chrominance prediction can be performed better. In addition, the neural network can also learn the correlation between the reference chrominance and the prediction target chrominance of the current block, and the correlation can be included in the chrominance prediction of the current block, thereby further improving the accuracy of the chrominance prediction, saving the code rate, and simultaneously improving the encoding and decoding performance.

[0345] In yet another embodiment of the present application, based on the same inventive concept as the previous embodiment, referring to Figure 22, a structural schematic diagram of an encoding device 300 according to an embodiment of the present application is shown. As shown in Figure 22, the encoding device 300 can include a first decision unit 3001, a first model unit 3002 and a first prediction unit 3003.

[0346] The first determining unit 3001 is configured to determine a reference sample value of a first color component of a current block.

[0347] The first model unit 3002 is configured to determine weighting coefficients by a neural network model based on the reference sample values ​​of the first color component of the current block, where the input of the neural network model includes the reference sample values ​​of the first color component of the current block.

[0348] The first prediction unit 3003 is configured to determine, according to the weighting coefficient and the reference sample value of the second color component of the current block, a prediction block of the second color component of the current block.

[0349] The first determining unit 3001 is further configured to determine, based on the prediction block, a prediction difference value of a second color component sampling point of the current block.

[0350] In some embodiments, the first determination unit 3001 is further configured to determine a neighboring reference value of the first color component of the current block based on the first color component sampling point in an adjacent region of the current block, and determine a reference sample value of the first color component of the current block based on the reconstructed value of the first color component sampling point in the current block and the neighboring reference value of the first color component of the current block, where the neighboring region includes at least one of an upper neighboring region, an upper right neighboring region, a left neighboring region, and a lower left neighboring region.

[0351] In some embodiments, the first determination unit 3001 is further configured to determine a reference value of the first color component in the current block based on a reconstructed value of a first color component sampling point in the current block, and to determine a reference sample value of the first color component of the current block based on the reference value of the first color component in the current block and a neighboring reference value of the first color component of the current block.

[0352] In some embodiments, the first determination unit 3001 is further configured to determine that the reference sample value of the first color component of the current block is an absolute value of the difference between the adjacent reference value of the first color component of the current block and the reference value of the first color component in the current block.

[0353] In some embodiments, the first determination unit 3001 is further configured to determine that the reference sample values ​​of the first color component of the current block include N matrices, where N represents the number of reference sample values ​​of the second color component of the current block, and the kth matrix includes an absolute value of the difference between the kth value among the adjacent reference values ​​of the first color component of the current block and the reference value of the first color component in the current block, where k=0, 1, ..., N-1.

[0354] In some embodiments, the first model unit 3002 is further configured to take the N matrices as inputs of a neural network model and determine outputs of the neural network model as weighting coefficients.

[0355] In some embodiments, the first determination unit 3001 is further configured to determine values ​​corresponding to N matrices in a preset mapping relationship, and determine a reference sample value of a first color component of the current block based on the values ​​corresponding to the N matrices in the preset mapping relationship.

[0356] In some embodiments, the first model unit 3002 is further configured to determine values ​​corresponding to the N matrices in the preset mapping relationship as inputs of a neural network model, and outputs of the neural network model as weighting coefficients.

[0357] In some embodiments, the pre-defined mapping relationship is a Softmax function.

[0358] In some embodiments, the predefined mapping relationship is a weighting function that has an inverse proportional relationship with the kth matrix.

[0359] In some embodiments, the first determining unit 3001 is further configured to perform a screening process on values ​​of the first color component sampling points in the neighboring region to determine neighboring reference values ​​of the first color component of the current block.

[0360] In some embodiments, the first determination unit 3001 is further configured to determine a position of a selected sampling point based on a position and / or a color component intensity of a first color component sampling point in the adjacent region, and to determine a neighboring reference value of the first color component of the current block from the adjacent region based on the position of the selected sampling point.

[0361] In some embodiments, the first determining unit 3001 is further configured to perform a first filtering operation on values ​​of the first color component sampling points in the neighboring region to determine neighboring reference values ​​of the first color component of the current block.

[0362] In some embodiments, the first determination unit 3001 is further configured to perform a second filtering operation on the reconstructed values ​​of the first color component sampling points in the current block to determine a reference value of the first color component in the current block.

[0363] In some embodiments, the first model unit 3002 is further configured to take the reference value of the first color component in the current block and the neighboring reference value of the first color component of the current block as inputs of a neural network model, and determine the output of the neural network model as the weighting coefficients.

[0364] In some embodiments, the first model unit 3002 is further configured to perform a connection process on the reference value of the first color component in the current block and the adjacent reference value of the first color component of the current block to obtain first connection information, take the first connection information as an input of a neural network model, and determine the output of the neural network model as the weighting coefficient.

[0365] In some embodiments, the first model unit 3002 is further configured to take the reference sample value of the first color component of the current block and the reference sample value of the second color component of the current block as inputs of a neural network model, and determine the output of the neural network model as the weighting coefficients.

[0366] In some embodiments, the first model unit 3002 is further configured to take the reference value of the first color component in the current block, the neighboring reference value of the first color component of the current block, and the reference sample value of the second color component of the current block as inputs of a neural network model, and determine the output of the neural network model as the weighting coefficients.

[0367] In some embodiments, the first model unit 3002 is further configured to perform a connection process on the reference value of the first color component in the current block and the adjacent reference value of the first color component of the current block to obtain second connection information, take the second connection information and the reference sample value of the second color component of the current block as inputs of a neural network model, and determine the output of the neural network model as the weighting coefficient.

[0368] In some embodiments, the first model unit 3002 is further configured to perform a connection process on the reference value of the first color component in the current block, the adjacent reference value of the first color component of the current block, and the reference sample value of the second color component of the current block to obtain third connection information, take the third connection information as the input of a neural network model, and determine the output of the neural network model as the weighting coefficient.

[0369] In some embodiments, the neural network model is a convolutional neural network model, the convolutional neural network model including at least one base unit and one target unit, where: The base unit includes at least a first convolutional layer and a first activation function, and the target unit includes at least a second convolutional layer and a second activation function.

[0370] In some embodiments, the neural network model is a fully-connected neural network model, the fully-connected neural network model including at least one basic unit and one target unit, where: The base unit includes at least a first fully connected layer and a first activation function, and the target unit includes at least a second fully connected layer and a second activation function.

[0371] In some embodiments, the first activation function is a ReLU function and the second activation function is a softmax function.

[0372] In some embodiments, the reference sample values ​​of the first color component of the current block include a one-dimensional matrix of W×H numbers, and the one-dimensional matrix includes N weighting coefficients, where W×H represents the number of sampling points of the second color component of the current block, and N represents the number of reference sample values ​​of the second color component of the current block.

[0373] In some embodiments, the first determining unit 3001 is further configured to determine, based on the second color component sampling points in the neighboring region of the current block, a reference sample value of the second color component of the current block.

[0374] In some embodiments, the first determination unit 3001 is further configured to perform a third filtering operation on values ​​of second color component sampling points in adjacent regions of the current block to determine reference sample values ​​of the second color component of the current block.

[0375] In some embodiments, the first prediction unit 3003 further determines a first one-dimensional matrix for calculating the predicted second color component sampling point according to a position index and a weighting coefficient of the predicted second color component sampling point in the current block; The method is configured to determine a prediction block of a second color component of the current block based on the first one-dimensional matrix and the reference sample values ​​of the second color component of the current block.

[0376] In some embodiments, the first one-dimensional matrix represents a one-dimensional matrix that corresponds to a position index in a three-dimensional matrix data structure of weighting coefficients.

[0377] In some embodiments, the first prediction unit 3003 is further configured to determine a prediction block of the second color component of the current block based on the position index of the second color component sampling point to be predicted in the current block and the weighting coefficient.

[0378] In some embodiments, the predicted value of the second color component sampling point to be predicted in the prediction block is set equal to a weighted sum of the reference sample value of the second color component of the current block and the coefficients in a second one-dimensional matrix, where the second one-dimensional matrix represents a one-dimensional matrix corresponding to a position index in a three-dimensional matrix data structure of weighting coefficients.

[0379] In some embodiments, the first model unit 3002 is further configured to determine a value corresponding to the weighting coefficient in a preset mapping relationship after determining the weighting coefficient by the neural network model, and update the weighting coefficient based on the value corresponding to the weighting coefficient in the preset mapping relationship.

[0380] In some embodiments, the first prediction unit 3003 is further configured to determine a prediction block of a second color component of the current block by a neural network model according to the reference sample values ​​of the first color component of the current block and the reference sample values ​​of the second color component of the current block; The first determining unit 3001 is further configured to determine, based on the predicted block, reconstructed values ​​of the second color component sampling points of the current block.

[0381] In some embodiments, the first prediction unit 3003 is further configured to take the reference sample value of the first color component of the current block and the reference sample value of the second color component of the current block as inputs of a neural network model, and determine the output of the neural network model as a predicted block.

[0382] In some embodiments, the first prediction unit 3003 is further configured to, after determining the prediction block of the second color component of the current block, perform a correlation operation on the prediction block, and obtain the processed prediction block as the prediction block.

[0383] In some embodiments, performing correlation on the predicted block includes: performing enhanced filtering on predicted values ​​of second color component sampling points in the predicted block; performing a correction process on the predicted value of the second color component sampling point in the prediction block by using the preset compensation value; and performing a weighted blending process on predicted values ​​of second color component sampling points in the predicted block using predicted values ​​of the second color component of the current block in at least one prediction mode.

[0384] In some embodiments, the first determination unit 3001 is further configured to determine a predicted difference value of a second color component sampling point of the current block, determine a predicted value of the second color component sampling point of the current block based on the predicted block, and determine a reconstructed value of the second color component sampling point of the current block based on the predicted difference value of the second color component sampling point of the current block and the predicted value of the second color component sampling point of the current block.

[0385] In some embodiments, the first determination unit 3001 is further configured to determine a predicted value of a second color component sampling point of the current block based on the predicted block, and to determine a predicted difference value of the second color component sampling point of the current block based on the original value of the second color component sampling point of the current block and the predicted value of the second color component sampling point of the current block.

[0386] In some embodiments, as shown in FIG. 22, the encoding device 300 may further include an encoding unit 3004 configured to encode the prediction difference values ​​of the second color component sampling points of the current block and write the resulting encoded bits to a bitstream.

[0387] As can be understood, in the embodiments of the present application, a "unit" may be a circuit portion, a processor portion, a program portion, or software portion, and may be a module or non-modular. In addition, each component in the embodiments may be integrated into one processing unit, each unit may exist physically alone, and two or more units may be integrated into one unit. The integrated unit may be realized in the form of hardware or in the form of a software functional module.

[0388] The integrated unit may be realized in the form of a software function module and stored in a computer-readable storage medium when not sold or used as an independent product. Based on this understanding, the technical solution of the present embodiment may be essentially or partly contributed to the prior art or all or part of the technical solution may be embodied in the form of a software product, and the computer software product is stored in a storage medium and includes some instructions for making a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor execute all or part of the steps of the method described in the present embodiment. The aforementioned storage medium includes various media that can store program code, such as U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0389] Therefore, an embodiment of the present application provides a computer-readable storage medium applied to the encoding device 300, the computer-readable storage medium storing a computer program, which, when executed by the first processor, realizes the method according to any one of the above embodiments.

[0390] Based on the above-mentioned configuration of the encoding device 300 and the computer-readable storage medium, referring to FIG. 23, a schematic diagram of a specific hardware structure of an encoding device 310 according to an embodiment of the present application is shown. As shown in FIG. 23, the encoding device 310 may include a first communication interface 3101, a first memory 3102, and a first processor 3103, and each component is coupled together via a first bus system 3104. It can be understood that the first bus system 3104 is used to realize the connection communication between these components. In addition to the data bus, the first bus system 3104 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, various buses are labeled as the first bus system 3104 in FIG. 23. Here, The first communication interface 3101 is used to send and receive signals during the process of sending and receiving information with other external network elements; The first memory 3102 is used to store a computer program executable on the first processor 3103; When executing the computer program, the first processor 3103 determining a reference sample value of a first color component of the current block; determining a weighting coefficient by a neural network model based on a reference sample value of a first color component of the current block, where an input of the neural network model includes the reference sample value of the first color component of the current block; determining a prediction block of a second color component of the current block based on the weighting coefficients and the reference sample values ​​of the second color component of the current block; and determining a prediction difference value of the second color component sampling point of the current block based on the prediction block.

[0391] It may be understood that the first memory 3102 in the embodiment of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memory. Here, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM) that functions as an external cache. By way of example and not of limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchlink dynamic random access memory (SLDRAM), and direct Rambus random access memory (DRRAM). The first memory 3102 of the systems and methods described herein is intended to include, without being limited to, these and any other suitable types of memory.

[0392] The first processor 3103 may be an integrated circuit chip with signal processing capabilities. In the realization process, each step of the above method may be completed by an instruction in the form of a hardware integrated logic circuit or software in the first processor 3103. The above first processor 3103 may be a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. Each method, step and logic block diagram disclosed in the embodiments of the present application may be realized or executed. The general-purpose processor may be a microprocessor, or the processor may be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present application may be directly embodied to be executed and completed by a hardware decode processor, or executed and completed by a combination of hardware and software modules in the decode processor. The software module may be located in a mature storage medium in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the first memory 3102, and the first processor 3103 reads the information in the first memory 3102 and completes the steps of the above method in combination with its hardware.

[0393] It is understood that the embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. When implemented in hardware, the processing unit may be implemented in one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processors (DSPDs), Programmable Logic Devices (PLDs), Field-Programmable Gate Arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units, or combinations thereof for performing the functions described herein. When implemented in software, the techniques described herein may be implemented by modules (e.g., processes, functions, etc.) that perform the functions described herein. The software code may be stored in a memory and executed by a processor. The memory may be implemented within the processor or external to the processor.

[0394] Optionally, as another embodiment, the first processor 3103 is further configured to execute the method according to any one of the previous embodiments when executing the computer program.

[0395] This embodiment provides an encoding device, which may further include the encoding device 300 described in the above embodiment. The encoding device can fully utilize the color component information in the neighboring region of the current block and the color component information in the current block to provide a variety of effective input information for the chrominance prediction based on the neural network, so that the chrominance prediction can be performed better. In addition, the neural network can learn the correlation between the reference chrominance and the chrominance to be predicted of the current block, and the correlation can be added to the chrominance prediction of the current block, thereby further improving the accuracy of the chrominance prediction, saving the code rate, and simultaneously improving the encoding and decoding performance.

[0396] Based on the same inventive concept as the above embodiment, referring to Figure 24, a structural schematic diagram of a decoding device 320 according to an embodiment of the present application is shown. As shown in Figure 24, the decoding device 320 can include a second determination unit 3021, a second model unit 3202 and a second prediction unit 3203.

[0397] The second determining unit 3021 is configured to determine a reference sample value of a first color component of the current block.

[0398] The second model unit 3202 is configured to determine a weighting coefficient by a neural network model based on the reference sample value of the first color component of the current block, where the input of the neural network model includes the reference sample value of the first color component of the current block.

[0399] The second prediction unit 3203 is configured to determine, according to the weighting coefficient and the reference sample value of the second color component of the current block, a prediction block of the second color component of the current block.

[0400] The second determining unit 3021 is further configured to determine, based on the prediction block, reconstructed values ​​of the second color component sampling points of the current block.

[0401] In some embodiments, the second determination unit 3201 is further configured to determine a neighboring reference value of the first color component of the current block based on the first color component sampling point in an adjacent region of the current block, and determine a reference sample value of the first color component of the current block based on the reconstructed value of the first color component sampling point in the current block and the neighboring reference value of the first color component of the current block, where the neighboring region includes at least one of an upper neighboring region, an upper right neighboring region, a left neighboring region, and a lower left neighboring region.

[0402] In some embodiments, the second determination unit 3201 is further configured to determine a reference value of the first color component in the current block based on a reconstructed value of a first color component sampling point in the current block, and determine a reference sample value of the first color component of the current block based on the reference value of the first color component in the current block and a neighboring reference value of the first color component of the current block.

[0403] In some embodiments, the second determination unit 3201 is further configured to determine that the reference sample value of the first color component of the current block is an absolute value of the difference between the adjacent reference value of the first color component of the current block and the reference value of the first color component in the current block.

[0404] In some embodiments, the second determination unit 3201 is further configured to determine that the reference sample values ​​of the first color component of the current block include N matrices, where N represents the number of reference sample values ​​of the second color component of the current block, and the kth matrix includes an absolute value of the difference between the kth value among the adjacent reference values ​​of the first color component of the current block and the reference value of the first color component in the current block, where k=0, 1, ..., N-1.

[0405] In some embodiments, the second model unit 3202 is further configured to take the N matrices as inputs of a neural network model and determine outputs of the neural network model as weighting coefficients.

[0406] In some embodiments, the second determination unit 3201 is further configured to determine values ​​corresponding to the N matrices in the preset mapping relationship, and determine a reference sample value of the first color component of the current block based on the values ​​corresponding to the N matrices in the preset mapping relationship.

[0407] In some embodiments, the second model unit 3202 is further configured to determine values ​​corresponding to the N matrices in the preset mapping relationship as inputs of a neural network model, and outputs of the neural network model as weighting coefficients.

[0408] In some embodiments, the pre-defined mapping relationship is a Softmax function.

[0409] In some embodiments, the predefined mapping relationship is a weighting function that has an inverse proportional relationship with the kth matrix.

[0410] In some embodiments, the second determining unit 3201 is further configured to perform a screening process on the values ​​of the first color component sampling points in the neighboring region to determine the neighboring reference value of the first color component of the current block.

[0411] In some embodiments, the second determination unit 3201 is further configured to determine a position of a selected sampling point based on a position and / or a color component intensity of a first color component sampling point in the adjacent region, and to determine a neighboring reference value of the first color component of the current block from the adjacent region based on the position of the selected sampling point.

[0412] In some embodiments, the second determining unit 3201 is further configured to perform a first filtering operation on the values ​​of the first color component sampling points in the neighboring region to determine neighboring reference values ​​of the first color component of the current block.

[0413] In some embodiments, the second determination unit 3201 is further configured to perform a second filtering operation on the reconstructed values ​​of the first color component sampling points in the current block to determine a reference value of the first color component in the current block.

[0414] In some embodiments, the second model unit 3202 is further configured to take the reference value of the first color component in the current block and the neighboring reference value of the first color component of the current block as inputs of a neural network model, and determine the output of the neural network model as the weighting coefficients.

[0415] In some embodiments, the second model unit 3202 is further configured to perform a connection process on the reference value of the first color component in the current block and the adjacent reference value of the first color component of the current block to obtain first connection information, take the first connection information as the input of a neural network model, and determine the output of the neural network model as the weighting coefficient.

[0416] In some embodiments, the second model unit 3202 is further configured to take the reference sample value of the first color component of the current block and the reference sample value of the second color component of the current block as inputs of a neural network model, and determine the output of the neural network model as the weighting coefficients.

[0417] In some embodiments, the second model unit 3202 is further configured to take the reference value of the first color component in the current block, the neighboring reference value of the first color component of the current block, and the reference sample value of the second color component of the current block as inputs of a neural network model, and determine the output of the neural network model as the weighting coefficients.

[0418] In some embodiments, the second model unit 3202 is further configured to perform a connection process on the reference value of the first color component in the current block and the adjacent reference value of the first color component of the current block to obtain second connection information, take the second connection information and the reference sample value of the second color component of the current block as the input of a neural network model, and determine the output of the neural network model as the weighting coefficient.

[0419] In some embodiments, the second model unit 3202 is further configured to perform a connection process on the reference value of the first color component in the current block, the adjacent reference value of the first color component of the current block, and the reference sample value of the second color component of the current block to obtain third connection information, take the third connection information as the input of a neural network model, and determine the output of the neural network model as the weighting coefficient.

[0420] In some embodiments, the neural network model is a convolutional neural network model, the convolutional neural network model including at least one base unit and one target unit, where: The base unit includes at least a first convolutional layer and a first activation function, and the target unit includes at least a second convolutional layer and a second activation function.

[0421] In some embodiments, the neural network model is a fully-connected neural network model, the fully-connected neural network model including at least one basic unit and one target unit, where: The base unit includes at least a first fully connected layer and a first activation function, and the target unit includes at least a second fully connected layer and a second activation function.

[0422] In some embodiments, the first activation function is a ReLU function and the second activation function is a softmax function.

[0423] In some embodiments, the reference sample values ​​of the first color component of the current block include a one-dimensional matrix of W×H numbers, and the one-dimensional matrix includes N weighting coefficients, where W×H represents the number of sampling points of the second color component of the current block, and N represents the number of reference sample values ​​of the second color component of the current block.

[0424] In some embodiments, the second determining unit 3201 is further configured to determine, based on the second color component sampling points in the neighboring region of the current block, a reference sample value of the second color component of the current block.

[0425] In some embodiments, the second determination unit 3201 is further configured to perform a third filtering operation on values ​​of the second color component sampling points in the adjacent region of the current block to determine reference sample values ​​of the second color component of the current block.

[0426] In some embodiments, the second prediction unit 3203 is further configured to determine a first one-dimensional matrix for calculating the second color component sampling point to be predicted based on a position index and a weighting coefficient of the second color component sampling point to be predicted in the current block, and determine a prediction block of the second color component of the current block based on the first one-dimensional matrix and the reference sample value of the second color component of the current block.

[0427] In some embodiments, the first one-dimensional matrix represents a one-dimensional matrix that corresponds to a position index in a three-dimensional matrix data structure of weighting coefficients.

[0428] In some embodiments, the second prediction unit 3203 is further configured to determine a prediction block of the second color component of the current block based on the position index of the second color component sampling point to be predicted in the current block and the weighting coefficient.

[0429] In some embodiments, the predicted value of the second color component sampling point to be predicted in the prediction block is set equal to a weighted sum of the reference sample value of the second color component of the current block and the coefficients in a second one-dimensional matrix, where the second one-dimensional matrix represents a one-dimensional matrix corresponding to a position index in a three-dimensional matrix data structure of weighting coefficients.

[0430] In some embodiments, the second model unit 3202 is further configured to determine a value corresponding to the weighting coefficient in a preset mapping relationship after determining the weighting coefficient by the neural network model, and update the weighting coefficient based on the value corresponding to the weighting coefficient in the preset mapping relationship.

[0431] In some embodiments, the second determination unit 3201 is further configured to determine a predicted block of the second color component of the current block by a neural network model based on the reference sample value of the first color component of the current block and the reference sample value of the second color component of the current block, and determine a reconstructed value of the second color component sampling point of the current block based on the predicted block.

[0432] In some embodiments, the second prediction unit 3203 is further configured to take the reference sample value of the first color component of the current block and the reference sample value of the second color component of the current block as inputs of a neural network model, and determine the output of the neural network model as a predicted block.

[0433] In some embodiments, the second prediction unit 3203 is further configured to, after determining the prediction block of the second color component of the current block, perform a correlation operation on the prediction block, and obtain the processed prediction block as the prediction block.

[0434] In some embodiments, performing correlation on the predicted block includes: performing enhanced filtering on predicted values ​​of second color component sampling points in the predicted block; performing a correction process on the predicted value of the second color component sampling point in the prediction block by using the preset compensation value; and performing a weighted blending process on predicted values ​​of second color component sampling points in the predicted block using predicted values ​​of the second color component of the current block in at least one prediction mode.

[0435] In some embodiments, the second determination unit 3201 is further configured to determine a predicted difference value of a second color component sampling point of the current block, determine a predicted value of the second color component sampling point of the current block based on the predicted block, and determine a reconstructed value of the second color component sampling point of the current block based on the predicted difference value of the second color component sampling point of the current block and the predicted value of the second color component sampling point of the current block.

[0436] In some embodiments, referring to FIG. 24, the decoding device 320 may further include a decoding unit 3204 configured to analyze the bitstream and determine a prediction difference value of a second color component sampling point of the current block.

[0437] As can be understood, in the embodiments of the present application, a "unit" may be a circuit portion, a processor portion, a program portion, or software portion, and may be a module or non-modular. In addition, each component in the embodiments may be integrated into one processing unit, each unit may exist physically alone, and two or more units may be integrated into one unit. The integrated unit may be realized in the form of hardware or in the form of a software functional module.

[0438] The integrated unit may be realized in the form of a software functional module and stored in one computer-readable storage medium when not sold or used as an independent product. Based on this understanding, this embodiment provides a computer-readable storage medium applied to the decoding device 320, the computer-readable storage medium stores a computer program, and the computer program, when executed by the second processor, realizes the method according to any one of the above embodiments.

[0439] Based on the above-mentioned configuration of the decoding device 320 and the computer-readable storage medium, referring to FIG. 25, a schematic diagram of a specific hardware structure of a decoding device 330 according to an embodiment of the present application is shown. As shown in FIG. 25, the decoding device 330 may include a second communication interface 3301, a second memory 3302, and a second processor 3303, and each component is coupled together via a second bus system 3304. It can be understood that the second bus system 3304 is used to realize the connection communication between these components. In addition to the data bus, the second bus system 3304 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, various buses are labeled as the second bus system 3304 in FIG. 25. Here, The second communication interface 3301 is used to send and receive signals during the process of sending and receiving information to and from other external network elements; The second memory 3302 is used to store a computer program executable on the second processor 3303; When executing the computer program, the second processor 3303 determining a reference sample value of a first color component of the current block; determining a weighting coefficient by a neural network model based on a reference sample value of a first color component of the current block, where an input of the neural network model includes the reference sample value of the first color component of the current block; determining a prediction block of a second color component of the current block based on the weighting coefficients and the reference sample values ​​of the second color component of the current block; and determining, based on the predicted block, reconstructed values ​​of the second color component sampling points of the current block.

[0440] Optionally, as another embodiment, the second processor 3303 is further configured to, when executing the computer program, perform the method according to any one of the previous embodiments.

[0441] It can be understood that the hardware functions of the second memory 3302 and the first memory 3102 are similar, and the hardware functions of the second processor 3303 and the first processor 3103 are similar, and will not be described in detail here.

[0442] This embodiment provides a decoding device, which may further include the decoding device 320 described in the above embodiment. In the decoding device, by fully utilizing the color component information in the neighboring region of the current block and the color component information in the current block, a variety of effective input information can be provided for the chrominance prediction based on the neural network, so that the chrominance prediction can be performed better. In addition, the neural network can learn the correlation between the reference chrominance and the chrominance to be predicted of the current block, and the correlation can be included in the chrominance prediction of the current block, thereby further improving the accuracy of the chrominance prediction, saving the code rate, and simultaneously improving the encoding and decoding performance.

[0443] In yet another embodiment of the present application, referring to Fig. 26, a structural schematic diagram of an encoding and decoding system according to an embodiment of the present application is shown. As shown in Fig. 26, the encoding and decoding system 340 can include an encoder 3401 and a decoder 3402. Here, the encoder 3401 may be an apparatus integrated with the encoding device 300 described in the above embodiment, or may be the encoding device 310 described in the above embodiment. The decoder 3402 may be an apparatus integrated with the decoding device 320 described in the above embodiment, or may be the decoding device 330 described in the above embodiment.

[0444] In the embodiment of the present application, in the encoding and decoding system 340, whether the encoder 3401 or the decoder 3402, can fully utilize the color component information in the neighboring region of the current block and the color component information in the current block, and can provide various effective input information for the chrominance prediction based on the neural network, so as to make the chrominance prediction better. In addition, the neural network can learn the correlation between the reference chrominance and the chrominance to be predicted of the current block, and the correlation can be added to the chrominance prediction of the current block, thereby further improving the accuracy of the chrominance prediction, saving the code rate, and simultaneously improving the encoding and decoding performance.

[0445] It should be understood that, as used herein, the terms "comprises" and "has" or any variation thereof are intended to cover a non-exclusive inclusion, whereby a process, method, article, or apparatus comprising a set of elements includes not only those elements, but also other elements not expressly listed or inherent to such process, method, article, or apparatus. In the absence of further limitations, an element qualified by the phrase "comprises a" does not exclude the presence of other identical elements in the process, method, article, or apparatus that comprises the element.

[0446] The serial numbers of the above embodiments of the present application are for illustrative purposes only and do not represent the superiority or inferiority of the embodiments.

[0447] The methods disclosed in the several method embodiments provided herein may be combined in any manner, where not inconsistent, to obtain new method embodiments.

[0448] The features disclosed in the several product embodiments provided herein may be combined in any combination, where not inconsistent, to obtain new product embodiments.

[0449] Features disclosed in any method or apparatus embodiment provided herein may be combined in any manner not inconsistent to yield new method or apparatus embodiments.

[0450] The above description is merely a specific embodiment of the present application, and the scope of protection of the present application is not limited thereto. Any changes or replacements that any person skilled in the art can easily think of within the technical scope disclosed in the present application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be subject to the scope of protection of the claims. [Industrial Applicability]

[0451] In the embodiment of the present application, whether at the encoding side or the decoding side, a reference sample value of the first color component of the current block is determined, and a weighting factor is determined by a neural network model based on the reference sample value of the first color component of the current block, where the input of the neural network model includes the reference sample value of the first color component of the current block, and a predicted block of the second color component of the current block is determined based on the weighting factor and the reference sample value of the second color component of the current block. In this way, at the encoding side, a predicted difference value of the second color component sampling point of the current block can be determined based on the predicted block, and the predicted difference value can be written into the bitstream. Thus, at the decoding side, a reconstructed value of the second color component sampling point of the current block can be determined based on the predicted block and the predicted difference value obtained by decoding. In this way, by fully obtaining the existing information of the current block, including the color component information in the neighboring area of ​​the current block and the color component information in the current block, a variety of effective input information can be provided for the chrominance prediction based on the neural network, so that the chrominance prediction can be performed better. In addition, by using the luminance difference between the reference pixel and the current block, or the result after correlation processing of this luminance difference, as one of the inputs of the neural network, the neural network can also learn the correlation between the reference chrominance and the predicted chrominance of the current block, and this correlation can be included in the chrominance prediction of the current block, thereby further improving the accuracy of chrominance prediction, saving the code rate, and simultaneously improving the encoding and decoding performance.

Claims

1. 1. A method of decoding, comprising: determining a reference sample value of a first color component of the current block; determining weighting coefficients by a neural network model based on reference sample values ​​of a first color component of the current block, where an input of the neural network model includes the reference sample values ​​of the first color component of the current block; determining a prediction block of a second color component of the current block based on the weighting coefficients and reference sample values ​​of a second color component of the current block; determining reconstructed values ​​of second color component sampling points of the current block based on the predicted block.

2. Determining a reference sample value of a first color component of the current block comprises: determining a neighboring reference value of a first color component of the current block based on a first color component sampling point in a neighboring region of the current block; determining a reference sample value of the first color component of the current block based on a reconstructed value of a first color component sampling point in the current block and a neighboring reference value of the first color component of the current block; The adjacent region includes at least one of an upper adjacent region, an upper right adjacent region, a left adjacent region, and a lower left adjacent region. The method of decoding according to claim 1 .

3. Determining a reference sample value of a first color component of the current block based on a reconstructed value of a first color component sampling point in the current block and a neighboring reference value of the first color component of the current block includes: determining a reference value of a first color component in the current block based on a reconstructed value of a first color component sampling point in the current block; determining a reference sample value of the first color component of the current block based on a reference value of the first color component in the current block and a neighboring reference value of the first color component of the current block; The decoding method according to claim 2.

4. determining that the reference sample value of the first color component of the current block is an absolute value of a difference between a neighboring reference value of the first color component of the current block and a reference value of the first color component within the current block; The decoding method according to claim 3.

5. determining that the reference sample values ​​of the first color component of the current block include N matrices, where N represents the number of reference sample values ​​of the second color component of the current block, and the k-th matrix includes an absolute value of a difference between the k-th value among the adjacent reference values ​​of the first color component of the current block and the reference value of the first color component in the current block, where k=0, 1, ..., N-1; The decoding method according to claim 3.

6. determining a weighting coefficient by a neural network model based on a reference sample value of a first color component of the current block; determining the N matrices as inputs to the neural network model and the outputs of the neural network model as the weighting coefficients; The decoding method according to claim 5.

7. determining values ​​corresponding to the N matrices in a preset mapping relationship; and determining a reference sample value of a first color component of the current block according to values ​​corresponding to the N matrices in the preset mapping relationship. The decoding method according to claim 5.

8. determining a weighting coefficient by a neural network model based on a reference sample value of a first color component of the current block; determining values ​​corresponding to the N matrices in the preset mapping relationship as inputs of the neural network model, and determining outputs of the neural network model as the weighting coefficients; The decoding method according to claim 7.

9. The predetermined mapping relationship is a Softmax function. The decoding method according to claim 7.

10. The predetermined mapping relationship is a weighting function having an inverse proportional relationship with the k-th matrix. The decoding method according to claim 7.

11. performing a screening process on values ​​of the first color component sampling points in the neighboring region to determine a neighboring reference value of the first color component of the current block; The decoding method according to claim 2.

12. performing a screening process on values ​​of first color component sampling points in the neighboring region to determine a neighboring reference value of the first color component of the current block; determining a location of a selected sampling point based on a location and / or a color component intensity of a first color component sampling point within the adjacent region; determining a neighboring reference value of a first color component of the current block from the neighboring region based on a position of the selected sampling point; The method of decoding according to claim 11.

13. performing a first filtering process on values ​​of first color component sampling points in the neighboring region to determine a neighboring reference value of the first color component of the current block; The decoding method according to claim 2.

14. performing a second filtering process on the reconstructed values ​​of the first color component sampling points in the current block to determine a reference value of the first color component in the current block; The decoding method according to claim 3.

15. determining a weighting coefficient by a neural network model based on a reference sample value of a first color component of the current block; a reference value of a first color component in the current block and a neighboring reference value of the first color component of the current block are input to the neural network model, and an output of the neural network model is determined as the weighting coefficient. The decoding method according to claim 3.

16. performing a linking process on the reference value of a first color component in the current block and the adjacent reference value of the first color component of the current block to obtain first linking information; and determining the first link information as an input of the neural network model and an output of the neural network model as the weighting coefficient.

16. The method of decoding according to claim 15.

17. and further comprising: taking a reference sample value of a first color component of the current block and a reference sample value of a second color component of the current block as inputs of the neural network model; and determining an output of the neural network model as the weighting coefficient. The decoding method according to claim 3.

18. The method further includes taking a reference value of a first color component in the current block, a neighbor reference value of the first color component of the current block, and a reference sample value of a second color component of the current block as inputs of the neural network model, and determining an output of the neural network model as the weighting coefficient.

18. The method of decoding according to claim 17.

19. performing a linking process on the reference value of the first color component in the current block and the adjacent reference value of the first color component of the current block to obtain second linking information; The second connection information and the reference sample value of the second color component of the current block are input to the neural network model, and the output of the neural network model is determined as the weighting coefficient.

20. The method of decoding according to claim 18.

20. performing a linking process on the reference value of a first color component in the current block, the adjacent reference value of the first color component of the current block, and the reference sample value of a second color component of the current block to obtain third linking information; and determining the third link information as an input of the neural network model and an output of the neural network model as the weighting coefficient.

20. The method of decoding according to claim 18.

21. The neural network model is a convolutional neural network model, the convolutional neural network model including at least one basic unit and one target unit, where: The basic unit includes at least a first convolutional layer and a first activation function; The target unit includes at least a second convolutional layer and a second activation function. The method of decoding according to claim 1 .

22. The neural network model is a fully-connected neural network model, the fully-connected neural network model including at least one basic unit and one target unit, where: The basic unit includes at least a first fully connected layer and a first activation function; the target unit includes at least a second fully connected layer and a second activation function; The method of decoding according to claim 1 .

23. The first activation function is a ReLU function, and the second activation function is a softmax function.

23. A method for decoding according to claim 21 or 22.

24. The reference sample value of the first color component of the current block includes a W×H one-dimensional matrix, and the one-dimensional matrix includes N weighting coefficients, where W×H represents the number of second color component sampling points of the current block, and N represents the number of reference sample values ​​of the second color component of the current block. The method of decoding according to claim 1 .

25. determining a reference sample value of a second color component of the current block based on second color component sampling points in an adjacent region of the current block; The method of decoding according to claim 1 .

26. performing a third filtering process on values ​​of second color component sampling points in an adjacent region of the current block to determine reference sample values ​​of the second color component of the current block; 26. The method of decoding according to claim 25.

27. determining a prediction block of a second color component of the current block based on the weighting coefficients and reference sample values ​​of a second color component of the current block, determining a first one-dimensional matrix for calculating the predicted second color component sampling point based on a position index of the predicted second color component sampling point in the current block and the weighting coefficient; determining a prediction block of a second color component of the current block based on the first one-dimensional matrix and reference sample values ​​of a second color component of the current block; The method of decoding according to claim 1 .

28. the first one-dimensional matrix represents a one-dimensional matrix corresponding to the position index in the three-dimensional matrix data structure of the weighting coefficients; 28. The method of decoding according to claim 27.

29. determining a prediction block of a second color component of the current block based on the weighting coefficients and reference sample values ​​of a second color component of the current block, determining a prediction block of a second color component of the current block based on a position index of a second color component sampling point to be predicted in the current block and the weighting coefficient; The method of decoding according to claim 1 .

30. a predicted value of the second color component sampling point to be predicted in the prediction block is set equal to a weighted sum of a reference sample value of the second color component of the current block and a coefficient in a second one-dimensional matrix, the second one-dimensional matrix representing a one-dimensional matrix corresponding to the position index in a three-dimensional matrix data structure of weighting coefficients.

30. The method of decoding according to claim 29.

31. determining a weighting coefficient according to a neural network model, and then determining a value corresponding to the weighting coefficient in a preset mapping relationship; updating the weighting coefficients based on values ​​corresponding to the weighting coefficients in the preset mapping relationship. The method of decoding according to claim 1 .

32. determining a predicted block of the second color component of the current block by a neural network model according to the reference sample values ​​of the first color component of the current block and the reference sample values ​​of the second color component of the current block; determining a reconstructed value of a second color component sampling point of the current block based on the predicted block. The method of decoding according to claim 1 .

33. and further comprising: taking a reference sample value of a first color component of the current block and a reference sample value of a second color component of the current block as inputs of the neural network model; and determining an output of the neural network model as the predicted block.

33. The method of decoding according to claim 32.

34. and further comprising: after determining the predicted block of the second color component of the current block, performing a correlation process on the predicted block, and setting the processed predicted block as the predicted block. The method of decoding according to claim 1 .

35. performing correlation processing on the predicted block, performing enhanced filtering on predicted values ​​of second color component sampling points in the predicted block; performing a correction process on the predicted value of the second color component sampling point in the prediction block using a preset compensation value; performing a weighted blending process on a predicted value of a second color component sampling point in the prediction block using a predicted value of a second color component of the current block in at least one prediction mode; 35. The method of decoding according to claim 34.

36. Determining reconstructed values ​​of second color component sampling points of the current block based on the predicted block includes: determining a prediction difference value of a second color component sampling point of the current block; and determining a predicted value of a second color component sampling point of the current block based on the predicted block; determining a reconstructed value of the second color component sampling point of the current block based on a predicted difference value of the second color component sampling point of the current block and a predicted value of the second color component sampling point of the current block; The method of decoding according to claim 1 .

37. 1. A method of encoding, comprising: determining a reference sample value of a first color component of the current block; determining weighting coefficients by a neural network model based on reference sample values ​​of a first color component of the current block, where an input of the neural network model includes the reference sample values ​​of the first color component of the current block; determining a prediction block of a second color component of the current block based on the weighting coefficients and reference sample values ​​of a second color component of the current block; determining a prediction difference value of a second color component sampling point of the current block based on the prediction block.

38. Determining a reference sample value of a first color component of the current block comprises: determining a neighboring reference value of a first color component of the current block based on a first color component sampling point in a neighboring region of the current block; determining a reference sample value of the first color component of the current block based on a reconstructed value of a first color component sampling point in the current block and a neighboring reference value of the first color component of the current block; The adjacent region includes at least one of an upper adjacent region, an upper right adjacent region, a left adjacent region, and a lower left adjacent region.

38. The encoding method of claim 37.

39. Determining a reference sample value of a first color component of the current block based on a reconstructed value of a first color component sampling point in the current block and a neighboring reference value of the first color component of the current block includes: determining a reference value of a first color component in the current block based on a reconstructed value of a first color component sampling point in the current block; determining a reference sample value of the first color component of the current block based on a reference value of the first color component in the current block and a neighboring reference value of the first color component of the current block; 39. The encoding method of claim 38.

40. determining that the reference sample value of the first color component of the current block is an absolute value of a difference between a neighboring reference value of the first color component of the current block and a reference value of the first color component within the current block; 40. The encoding method of claim 39.

41. determining that the reference sample values ​​of the first color component of the current block include N matrices, where N represents the number of reference sample values ​​of the second color component of the current block, and the k-th matrix includes an absolute value of a difference between the k-th value among the adjacent reference values ​​of the first color component of the current block and the reference value of the first color component in the current block, where k=0, 1, ..., N-1; 40. The encoding method of claim 39.

42. determining a weighting coefficient by a neural network model based on a reference sample value of a first color component of the current block; determining the N matrices as inputs to the neural network model and the outputs of the neural network model as the weighting coefficients; 42. The encoding method of claim 41.

43. determining values ​​corresponding to the N matrices in a preset mapping relationship; and determining a reference sample value of a first color component of the current block according to values ​​corresponding to the N matrices in the preset mapping relationship.

42. The encoding method of claim 41.

44. determining a weighting coefficient by a neural network model based on a reference sample value of a first color component of the current block; determining values ​​corresponding to the N matrices in the preset mapping relationship as inputs of the neural network model, and determining outputs of the neural network model as the weighting coefficients; 44. The encoding method of claim 43.

45. The predetermined mapping relationship is a Softmax function.

44. The encoding method of claim 43.

46. The predetermined mapping relationship is a weighting function having an inverse proportional relationship with the k-th matrix.

44. The encoding method of claim 43.

47. performing a screening process on values ​​of the first color component sampling points in the neighboring region to determine a neighboring reference value of the first color component of the current block; 39. The encoding method of claim 38.

48. performing a screening process on values ​​of first color component sampling points in the neighboring region to determine a neighboring reference value of the first color component of the current block; determining a location of a selected sampling point based on a location and / or a color component intensity of a first color component sampling point within the adjacent region; determining a neighboring reference value of a first color component of the current block from the neighboring region based on a position of the selected sampling point; 48. The encoding method of claim 47.

49. performing a first filtering process on values ​​of first color component sampling points in the neighboring region to determine a neighboring reference value of the first color component of the current block; 39. The encoding method of claim 38.

50. performing a second filtering process on the reconstructed values ​​of the first color component sampling points in the current block to determine a reference value of the first color component in the current block; 40. The encoding method of claim 39.

51. determining a weighting coefficient by a neural network model based on a reference sample value of a first color component of the current block; a reference value of a first color component in the current block and a neighboring reference value of the first color component of the current block are input to the neural network model, and an output of the neural network model is determined as the weighting coefficient.

40. The encoding method of claim 39.

52. performing a linking process on the reference value of a first color component in the current block and the adjacent reference value of the first color component of the current block to obtain first linking information; and determining the first link information as an input of the neural network model and an output of the neural network model as the weighting coefficient.

52. The encoding method of claim 51.

53. and further comprising: taking a reference sample value of a first color component of the current block and a reference sample value of a second color component of the current block as inputs of the neural network model; and determining an output of the neural network model as the weighting coefficient.

40. The encoding method of claim 39.

54. The method further includes taking a reference value of a first color component in the current block, a neighbor reference value of the first color component of the current block, and a reference sample value of a second color component of the current block as inputs of the neural network model, and determining an output of the neural network model as the weighting coefficient.

54. The encoding method of claim 53.

55. performing a linking process on the reference value of the first color component in the current block and the adjacent reference value of the first color component of the current block to obtain second linking information; The second connection information and the reference sample value of the second color component of the current block are input to the neural network model, and the output of the neural network model is determined as the weighting coefficient.

55. The encoding method of claim 54.

56. performing a linking process on the reference value of a first color component in the current block, the adjacent reference value of the first color component of the current block, and the reference sample value of a second color component of the current block to obtain third linking information; and determining the third link information as an input of the neural network model and an output of the neural network model as the weighting coefficient.

55. The encoding method of claim 54.

57. The neural network model is a convolutional neural network model, the convolutional neural network model including at least one basic unit and one target unit, where: The basic unit includes at least a first convolutional layer and a first activation function; The target unit includes at least a second convolutional layer and a second activation function.

38. The encoding method of claim 37.

58. The neural network model is a fully-connected neural network model, the fully-connected neural network model including at least one basic unit and one target unit, where: The basic unit includes at least a first fully connected layer and a first activation function; the target unit includes at least a second fully connected layer and a second activation function; 38. The encoding method of claim 37.

59. The first activation function is a ReLU function, and the second activation function is a softmax function.

59. An encoding method according to claim 57 or 58.

60. The reference sample value of the first color component of the current block includes a W×H one-dimensional matrix, and the one-dimensional matrix includes N weighting coefficients, where W×H represents the number of second color component sampling points of the current block, and N represents the number of reference sample values ​​of the second color component of the current block.

38. The encoding method of claim 37.

61. determining a reference sample value of a second color component of the current block based on second color component sampling points in an adjacent region of the current block; 38. The encoding method of claim 37.

62. performing a third filtering process on values ​​of second color component sampling points in an adjacent region of the current block to determine reference sample values ​​of the second color component of the current block; 62. The encoding method of claim 61.

63. determining a prediction block of a second color component of the current block based on the weighting coefficients and reference sample values ​​of a second color component of the current block, determining a first one-dimensional matrix for calculating the predicted second color component sampling point based on a position index of the predicted second color component sampling point in the current block and the weighting coefficient; determining a prediction block of a second color component of the current block based on the first one-dimensional matrix and reference sample values ​​of a second color component of the current block; 38. The encoding method of claim 37.

64. the first one-dimensional matrix represents a one-dimensional matrix corresponding to the position index in the three-dimensional matrix data structure of the weighting coefficients; 64. The encoding method of claim 63.

65. determining a prediction block of a second color component of the current block based on the weighting coefficients and reference sample values ​​of a second color component of the current block, determining a prediction block of a second color component of the current block based on a position index of a second color component sampling point to be predicted in the current block and the weighting coefficient; 38. The encoding method of claim 37.

66. a predicted value of the second color component sampling point to be predicted in the prediction block is set equal to a weighted sum of a reference sample value of the second color component of the current block and a coefficient in a second one-dimensional matrix, the second one-dimensional matrix representing a one-dimensional matrix corresponding to the position index in a three-dimensional matrix data structure of weighting coefficients.

66. The encoding method of claim 65.

67. determining a weighting coefficient according to a neural network model, and then determining a value corresponding to the weighting coefficient in a preset mapping relationship; updating the weighting coefficients based on values ​​corresponding to the weighting coefficients in the preset mapping relationship.

38. The encoding method of claim 37.

68. determining a predicted block of the second color component of the current block by a neural network model according to the reference sample values ​​of the first color component of the current block and the reference sample values ​​of the second color component of the current block; determining a reconstructed value of a second color component sampling point of the current block based on the predicted block.

38. The encoding method of claim 37.

69. and further comprising: taking a reference sample value of a first color component of the current block and a reference sample value of a second color component of the current block as inputs of the neural network model; and determining an output of the neural network model as the predicted block.

69. The encoding method of claim 68.

70. and further comprising: after determining the predicted block of the second color component of the current block, performing a correlation process on the predicted block, and setting the processed predicted block as the predicted block.

38. The encoding method of claim 37.

71. performing correlation processing on the predicted block, performing enhanced filtering on predicted values ​​of second color component sampling points in the predicted block; performing a correction process on the predicted value of the second color component sampling point in the prediction block using a preset compensation value; performing a weighted blending process on a predicted value of a second color component sampling point in the prediction block using a predicted value of a second color component of the current block in at least one prediction mode; 71. The encoding method of claim 70.

72. Determining reconstructed values ​​of second color component sampling points of the current block based on the predicted block includes: determining a prediction difference value of a second color component sampling point of the current block; and determining a predicted value of a second color component sampling point of the current block based on the predicted block; determining a reconstructed value of the second color component sampling point of the current block based on a predicted difference value of the second color component sampling point of the current block and a predicted value of the second color component sampling point of the current block; 38. The encoding method of claim 37.

73. determining a prediction difference value of a second color component sampling point of the current block based on the prediction block, determining a predicted value of a second color component sampling point of the current block based on the predicted block; determining a predicted difference value of the second color component sampling point of the current block based on an original value of the second color component sampling point of the current block and a predicted value of the second color component sampling point of the current block; 38. The encoding method of claim 37.

74. encoding the prediction difference values ​​of the second color component sampling points of the current block and writing the resulting encoded bits into a bitstream.

74. The encoding method of claim 73.

75. An encoding device, comprising: a first decision unit, a first model unit and a first prediction unit, The first determining unit is configured to determine a reference sample value of a first color component of a current block; The first model unit is configured to determine weighting coefficients by a neural network model based on reference sample values ​​of a first color component of the current block, where an input of the neural network model includes the reference sample values ​​of the first color component of the current block; The first prediction unit is configured to determine a prediction block of a second color component of the current block based on the weighting coefficients and reference sample values ​​of a second color component of the current block; The first determination unit is further configured to determine, based on the prediction block, a prediction difference value of a second color component sampling point of the current block.

76. A coding device including a first memory and a first processor; the first memory is adapted to store a computer program executable on the first processor; Coding device, wherein said first processor is used to perform the method according to any one of claims 37 to 74 when executing said computer program.

77. A decoding device, comprising: a second determination unit, a second model unit and a second prediction unit, The second determining unit is configured to determine a reference sample value of a first color component of a current block; The second model unit is configured to determine weighting coefficients by a neural network model based on reference sample values ​​of a first color component of the current block, where an input of the neural network model includes the reference sample values ​​of the first color component of the current block; The second prediction unit is configured to determine a prediction block of a second color component of the current block based on the weighting coefficients and reference sample values ​​of a second color component of the current block; The second determination unit is further configured to determine, based on the prediction block, a reconstructed value of a second color component sampling point of the current block.

78. a decoding device including a second memory and a second processor; the second memory is adapted to store a computer program executable on the second processor; A decoding device, wherein the second processor, when executing the computer program, is used to carry out the method according to any one of claims 1 to 36.

79. A computer readable storage medium having stored thereon a computer program which, when executed, causes a computer to perform the method according to any one of claims 1 to 36 or to perform the method according to any one of claims 37 to 74.

Images