Prediction model training method and device and rate-distortion cost calculation method and device

Abstract

The present disclosure relates to a prediction model training method and device and a rate-distortion cost calculation method and device. The training method includes: obtaining a training data set, wherein the training data set includes feature information of each CTU in at least one video and a true ratio of a first correlation coefficient to a second correlation coefficient of each CTU; inputting the feature information of each CTU into a prediction model to obtain a predicted ratio of the first correlation coefficient to the second correlation coefficient of each CTU; determining a loss value of the prediction model based on the true ratio and the predicted ratio of the first correlation coefficient to the second correlation coefficient of each CTU; and training the prediction model by adjusting parameters of the prediction model according to the loss value.

Description

Technical Field

[0001] The present disclosure relates to the field of video coding and decoding, and in particular to a method and device for training a prediction model and a method and device for calculating a rate-distortion cost. Background Art

[0002] Video coding standards use rate-distortion optimization methods to encode videos. Related technologies for calculating the rate-distortion cost use methods based on Video Multimethod Assessment Fusion (VMAF) or methods that use convolutional neural networks to predict ΔV. However, these methods can result in excessively long computation times or high computational complexity. Summary of the Invention

[0003] The present disclosure provides a prediction model training method and device, and a rate-distortion cost calculation method and device, to at least solve the problems in the above-mentioned related technologies, or not to solve any of the above-mentioned problems. The technical solutions of the present disclosure are as follows:

[0004] According to a first aspect of an embodiment of the present disclosure, a method for training a prediction model is provided, comprising: obtaining a training data set, wherein the training data set includes feature information of each coding tree unit (CTU) in at least one video and a true ratio of a first correlation coefficient to a second correlation coefficient of each CTU, the first correlation coefficient representing the correlation between a first objective evaluation index and a second objective evaluation index at a CTU level, and the second correlation coefficient representing the correlation between a first objective evaluation index and a second objective evaluation index at a frame level; inputting the feature information of each CTU into a prediction model to obtain a predicted ratio of the first correlation coefficient to the second correlation coefficient of each CTU; determining a loss value of the prediction model based on the true ratio and the predicted ratio of the first correlation coefficient to the second correlation coefficient of each CTU; and training the prediction model by adjusting parameters of the prediction model based on the loss value.

[0005] Optionally, the first correlation coefficient of each CTU of each video in the at least one video is obtained by the following steps: obtaining a constant rate factor CRF list of each video; encoding each CTU according to the CRF list to obtain the first correlation coefficient of each CTU.

[0006] Optionally, obtaining the constant rate factor CRF list of each video includes: obtaining an initial CRF list of each video, wherein the initial CRF list includes at least one CRF, and the at least one CRF is arranged in ascending order of numerical value; determining an initial baseline CRF in the initial CRF list; determining a first subset in the initial CRF list based on the initial baseline CRF, wherein the first subset includes the initial baseline CRF and a preset number of CRFs after the initial baseline CRF; determining a baseline CRF from the first subset; and obtaining the CRF list based on the baseline CRF.

[0007] Optionally, determining the benchmark CRF from the first subset includes: using the CRFs in the first subset to encode in sequence in the order of arrangement of the CRFs in the first subset to obtain a first objective evaluation index for each frame in each video; using the CRF used when a predetermined condition is met as the benchmark CRF, wherein when the first objective evaluation index of more than a preset number of frames is within a first preset numerical range, it is determined that the predetermined condition is met; if the first objective evaluation index of each frame in each video obtained by encoding using the CRFs in the first subset does not meet the predetermined condition, then using the last CRF in the first subset as the benchmark CRF.

[0008] Optionally, obtaining the CRF list based on the benchmark CRF includes: obtaining all CRFs within a second preset numerical interval in the initial CRF list, wherein the second preset numerical interval is determined based on the difference between the benchmark CRF and a first preset value and the sum of the benchmark CRF and a second preset value; and obtaining the CRF list for each video based on all CRFs within the second preset numerical interval, wherein the CRF list includes the benchmark CRF and at least one regular CRF.

[0009] Optionally, encoding each CTU according to the CRF list to obtain the first correlation coefficient of each CTU includes: encoding each CTU according to the benchmark CRF to obtain a benchmark first objective evaluation index and a benchmark second objective evaluation index of each CTU; encoding each CTU according to the at least one conventional CRF to obtain at least one conventional first objective evaluation index and at least one conventional second objective evaluation index of each CTU; obtaining the first correlation coefficient of each CTU by least squares method based on the benchmark first objective evaluation index, the benchmark second objective evaluation index, the at least one conventional first objective evaluation index and the at least one conventional second objective evaluation index.

[0010] Optionally, the second correlation coefficient of each frame of each video in the at least one video is obtained by the following steps: determining a first type of CTU and a second type of CTU based on a size relationship between the first correlation coefficient of each CTU and a third preset value, wherein the first correlation coefficient of the first type of CTU is less than or equal to the third preset value, and the first correlation coefficient of the second type of CTU is greater than the third preset value; and obtaining the second correlation coefficient of each frame based on the first correlation coefficient of the second type of CTU.

[0011] According to a second aspect of an embodiment of the present disclosure, a rate-distortion cost calculation method is provided, including: obtaining feature information of each CTU of a precoded frame; inputting the feature information of each CTU of the precoded frame into a prediction model trained by the prediction model training method of the present disclosure, and obtaining a predicted ratio of a first correlation coefficient to a second correlation coefficient of each CTU of the precoded frame; and calculating the rate-distortion cost of each CTU based on the predicted ratio.

[0012] According to a third aspect of an embodiment of the present disclosure, a training device for a prediction model is provided, comprising: an acquisition unit, configured to acquire a training data set, wherein the training data set includes feature information of each coding tree unit (CTU) in at least one video and a true ratio of a first correlation coefficient to a second correlation coefficient of each CTU, the first correlation coefficient representing the correlation between a first objective evaluation index and a second objective evaluation index at a CTU level, and the second correlation coefficient representing the correlation between a first objective evaluation index and a second objective evaluation index at a frame level; a prediction unit, configured to input the feature information of each CTU into a prediction model to obtain a predicted ratio of the first correlation coefficient to the second correlation coefficient of each CTU; a loss determination unit, configured to determine a loss value of the prediction model based on the true ratio and the predicted ratio of the first correlation coefficient to the second correlation coefficient of each CTU; and an adjustment unit, configured to train the prediction model by adjusting parameters of the prediction model based on the loss value.

[0013] Optionally, the acquisition unit is configured to: acquire a constant rate factor crf list of each video; and encode each CTU according to the crf list to obtain a first correlation coefficient of each CTU.

[0014] Optionally, the acquisition unit is configured to: acquire an initial CRF list for each video, wherein the initial CRF list includes at least one CRF, and the at least one CRF is arranged in ascending order of numerical value; determine an initial baseline CRF in the initial CRF list; determine a first subset in the initial CRF list based on the initial baseline CRF, wherein the first subset includes the initial baseline CRF and a preset number of CRFs after the initial baseline CRF; determine a baseline CRF from the first subset; and acquire the CRF list based on the baseline CRF.

[0015] Optionally, the acquisition unit is configured to: encode the first objective evaluation index of each frame in each video in sequence using the CRFs in the first subset in the order of arrangement of the CRFs in the first subset; use the CRF used when the predetermined condition is met as a benchmark CRF, wherein when the first objective evaluation index of more than a preset number of frames is in a first preset numerical range, it is determined that the predetermined condition is met; if the first objective evaluation index of each frame in each video obtained by encoding using the CRFs in the first subset does not meet the predetermined condition, the last CRF in the first subset is used as the benchmark CRF.

[0016] Optionally, the acquisition unit is configured to: acquire all CRFs within a second preset numerical interval in the initial CRF list, wherein the second preset numerical interval is determined based on the difference between the baseline CRF and the first preset value and the sum of the baseline CRF and the second preset value; and obtain a CRF list for each video based on all CRFs within the second preset numerical interval, wherein the CRF list includes the baseline CRF and at least one regular CRF.

[0017] Optionally, the acquisition unit is configured to: encode each CTU according to the benchmark CRF to obtain a benchmark first objective evaluation index and a benchmark second objective evaluation index of each CTU; encode each CTU according to the at least one conventional CRF to obtain at least one conventional first objective evaluation index and at least one conventional second objective evaluation index of each CTU; obtain the first correlation coefficient of each CTU by the least squares method based on the benchmark first objective evaluation index, the benchmark second objective evaluation index, the at least one conventional first objective evaluation index and the at least one conventional second objective evaluation index.

[0018] Optionally, the acquisition unit is configured to: determine the first type of CTU and the second type of CTU based on the size relationship between the first correlation coefficient of each CTU and a third preset value, wherein the first correlation coefficient of the first type of CTU is less than or equal to the third preset value, and the first correlation coefficient of the second type of CTU is greater than the third preset value; and obtain the second correlation coefficient of each frame based on the first correlation coefficient of the second type of CTU.

[0019] According to a fourth aspect of an embodiment of the present disclosure, a rate-distortion cost calculation device is provided, including: an acquisition unit, configured to: acquire feature information of each CTU of a pre-coded frame; a prediction unit, configured to: input the feature information of each CTU of the pre-coded frame into a prediction model trained by the prediction model training method of the present disclosure, and obtain a predicted ratio of the first correlation coefficient to the second correlation coefficient of each CTU of the pre-coded frame; and a calculation unit, configured to: calculate the rate-distortion cost of each CTU according to the predicted ratio.

[0020] According to a fifth aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: at least one processor; and at least one memory storing computer-executable instructions, wherein the computer-executable instructions, when executed by the at least one processor, prompt the at least one processor to execute a training method for a prediction model or a rate-distortion cost calculation method according to the present disclosure.

[0021] According to the sixth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When the instructions stored in the computer-readable storage medium are executed by at least one processor, the at least one processor is prompted to execute the training method of the prediction model or the rate-distortion cost calculation method according to the present disclosure.

[0022] According to a seventh aspect of an embodiment of the present disclosure, a computer program product is provided, comprising computer instructions, which, when executed by at least one processor, implement the prediction model training method or rate-distortion cost calculation method according to the present disclosure.

[0023] The technical solutions provided by the embodiments of the present disclosure bring at least the following beneficial effects:

[0024] According to the prediction model training method and device and rate-distortion cost calculation method and device disclosed in the present invention, the prediction model is trained with the ratio of the first correlation coefficient to the second correlation coefficient as the target, and the prediction ratio is applied to the rate-distortion cost calculation process. The correlation coefficient ratio can be obtained without replacing the CTU multiple times to calculate the VMAF, thereby reducing the calculation time complexity and shortening the calculation time.

[0025] In addition, according to the prediction model training method and device and rate-distortion cost calculation method and device disclosed in the present invention, the prediction ratio is applied to the rate-distortion cost calculation process, which can reduce the BD-Rate-VMAF value while increasing the encoding time complexity by no more than 20%, thereby obtaining encoding benefits.

[0026] In addition, according to the prediction model training method and device and rate-distortion cost calculation method and device disclosed in the present invention, the prediction model can be trained in an offline training manner, avoiding the need to retrain each network.

[0027] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute an improper limitation of the present disclosure.

[0029] Figure 1 The figure is a flowchart of a method for training a prediction model according to an exemplary embodiment.

[0030] Figure 2 The figure is a schematic diagram showing the structure of a prediction model according to an exemplary embodiment.

[0031] Figure 3 The figure is a flowchart of a method for calculating a rate-distortion cost according to an exemplary embodiment.

[0032] Figure 4 The figure is a block diagram of a training device for a prediction model according to an exemplary embodiment.

[0033] Figure 5 The figure is a block diagram of a rate-distortion cost calculation device according to an exemplary embodiment.

[0034] Figure 6 is a block diagram of an electronic device 600 according to an exemplary embodiment. DETAILED DESCRIPTION

[0035] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0036] It should be noted that the terms "first," "second," and the like in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the numbers used in this manner are interchangeable where appropriate so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The implementation methods described in the following examples do not represent all implementation methods consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with certain aspects of the present disclosure as detailed in the appended claims.

[0037] It should be noted that the phrase "at least one of the items" in this disclosure includes three types of parallel situations: "any one of the items", "a combination of any multiple items of the items", and "all of the items". For example, "including at least one of A and B" includes the following three parallel situations: (1) including A; (2) including B; (3) including A and B. For another example, "performing at least one of step 1 and step 2" includes the following three parallel situations: (1) performing step 1; (2) performing step 2; and (3) performing steps 1 and 2.

[0038] Video encoding standards use rate-distortion optimization technology to encode videos. Rate-distortion optimization technology requires measuring distortion, and the currently widely used calculation method is based on the sum of square errors (SSE). However, there is a large difference between measuring distortion using SSE and subjective quality. VMAF is currently the mainstream objective video quality assessment algorithm. However, since VMAF can only be calculated for the entire frame, it cannot be calculated for each coding tree unit (CTU), making it difficult to apply to rate-distortion optimization of video encoding. Existing methods for calculating rate-distortion costs based on VMAF require multiple VMAF calculations for each frame, which has a high time complexity and cannot be used for real-time encoding.

[0039] Based on this, a related technology first uses a block-by-block local fitting method to study the correlation between VMAF and SSE. The correlation coefficient of ΔVMAF and ΔSSE of each CTU is calculated block by block, and then the correlation coefficient is processed offline and applied to the encoding process.

[0040] First, the video is encoded using multiple different quantization parameters (QPs), with QP = 25 being used as the base QP. The pre-encoding result at the base QP is used as the base reconstruction, while the pre-encoding results at other QPs are used as the standard reconstruction. This method then calculates the rate-distortion cost by establishing a relationship between VMAF distortion and SSE.

[0041] For example, the baseline / conventional reconstruction V(V b / V r ) has an approximate linear relationship with the SSE:

[0042] V=100-VMAF (1)

[0043] V r =V b +ΔV, ΔV≈k i *SSE i，r (2)

[0044] Where V is VMAF distortion, VMAF is VMAF score, and V r is the VMAF distortion of conventional reconstruction, V b is the VMAF distortion of the baseline reconstruction, ΔV is V r and V b Deviation, k i is the correlation coefficient.

[0045] The correlation coefficient establishes a link between the frame-level perceptual distortion and the block-level conventional distortion, which will be used in the subsequent rate-distortion optimization. By assuming that ΔV is generated independently by each coding block, the correlation coefficient of each coding block can be obtained by the following substitution method: when deriving the correlation coefficient of the i-th block, only the corresponding block of the baseline reconstruction is replaced by the same position block in the conventional reconstruction to generate a temporary frame. In this case, V r It is the V of the temporary frame, and then according to V b The corresponding correlation coefficient is obtained by replacing the CTU at the same position multiple times and calculating the ΔVMAF and ΔSSE of each CTU. The least squares method is used for linear fitting to obtain the correlation coefficient (ratio) between the two. The fitted correlation coefficient is applied to the rate-distortion optimization process. First, the fitted correlation coefficient is processed to remove the negative and 0 values. The cleaned correlation coefficient is then used to calculate the frame-level λ and CTU-level k for the rate-distortion cost.

[0046] However, this method requires multiple encoding cycles. After each encoding, each CTU in each frame must be replaced, the VMAF of the replaced frames calculated, and the correlation coefficients fitted before being applied to the encoder. For a 720x1280 resolution sequence, this requires 220x4 VMAF calculations per frame, which is too time-consuming and difficult to apply to actual video encoding and transmission.

[0047] Another related technique uses online training of a convolutional neural network (CNN) to learn VMAF characteristics. Since SSE is available during the encoding process, the CNN's primary task is to output the ΔV for each block. The true ΔV value is obtained by block replacement using a baseline reconstruction of the keyframe and a single baseline reconstruction of the adjacent QP point. The CNN input is the encoded block of the unencoded keyframe. To achieve fast convergence, the CNN is a simple network consisting of three convolutional layers. Through this online training method, the ΔV and correlation coefficients for subsequent frames can be derived from the CNN and used to calculate the rate-distortion cost. However, this method has a high time complexity.

[0048] In order to solve the problems existing in the above-mentioned related technologies, the present disclosure proposes a prediction model training method and device and a rate-distortion cost calculation method and device. By training the prediction model with the ratio of the first correlation coefficient to the second correlation coefficient as the target, and applying the prediction ratio to the rate-distortion cost calculation process, the correlation coefficient ratio can be obtained without multiple replacements of CTUs to calculate VMAF, thereby reducing the calculation time complexity and shortening the calculation time.

[0049] Below, we will refer to Figures 1 to 6 The present invention describes in detail a method and apparatus for training a prediction model and a method and apparatus for calculating a rate-distortion cost.

[0050] Figure 1 FIG. 1 is a flow chart of a method for training a prediction model according to an exemplary embodiment. Figure 1 In step 101, a training data set may be obtained, wherein the training data set includes feature information of each coding tree unit (CTU) in at least one video and a true ratio of a first correlation coefficient to a second correlation coefficient of each CTU, the first correlation coefficient representing a correlation between a first objective evaluation indicator and a second objective evaluation indicator at a CTU level, and the second correlation coefficient representing a correlation between the first objective evaluation indicator and the second objective evaluation indicator at a frame level.

[0051] According to an exemplary embodiment of the present disclosure, the first objective evaluation index may be VMAF, and the second objective evaluation index may be SSE.

[0052] According to an exemplary embodiment of the present disclosure, the feature information may be any one of a pixel value, variance, edge information, gradient information, and motion information of the Y component.

[0053] According to an exemplary embodiment of the present disclosure, a first correlation coefficient of each CTU of each video in at least one video may be obtained by the following steps: first, a constant rate factor (CRF) list of each video may be obtained. Then, each CTU may be encoded according to the CRF list to obtain the first correlation coefficient of each CTU.

[0054] According to an exemplary embodiment of the present disclosure, to obtain a constant rate factor (CRF) list for each video, an initial CRF list for each video can be first obtained. The initial CRF list includes at least one CRF, which is arranged in ascending numerical order. An initial baseline CRF can then be determined from the initial CRF list. Next, based on the initial baseline CRF, a first subset can be determined from the initial CRF list. The first subset includes the initial baseline CRF and a preset number of CRFs following the initial baseline CRF. A baseline CRF can then be determined from the first subset. Finally, a CRF list can be obtained based on the baseline CRF.

[0055] According to an exemplary embodiment of the present disclosure, for determining a benchmark CRF from the first subset, first, the CRFs in the first subset can be used sequentially in the order of arrangement of the CRFs in the first subset to encode the first objective evaluation index of each frame in each video. Then, the CRF used when the predetermined condition is met can be used as the benchmark CRF, wherein when the first objective evaluation index of more than a preset number of frames is in a first preset numerical range, it is determined that the predetermined condition is met. It should be noted that when there are multiple CRFs that meet the predetermined condition, the first CRF that meets the predetermined condition is used as the benchmark CRF. Here, if the first objective evaluation index of each frame in each video obtained by encoding using the CRFs in the first subset does not meet the predetermined condition, the last CRF in the first subset is used as the benchmark CRF.

[0056] According to an exemplary embodiment of the present disclosure, to obtain a CRF list based on a baseline CRF, all CRFs within a second preset numerical range may be obtained from an initial CRF list, where the second preset numerical range is determined based on the difference between the baseline CRF and a first preset value and the sum of the baseline CRF and the second preset value. Then, based on all CRFs within the second preset numerical range, a CRF list for each video may be obtained, where the CRF list includes the baseline CRF and at least one regular CRF.

[0057] For example, to determine the CRF list for any video, one can first obtain an initial CRF list of {17, 19, 21, 23, 25, 27, 29, 31, 33, 35} for any video. Then, an initial baseline CRF = 25 can be determined from the initial CRF list. Next, based on the initial baseline CRF, a first subset {25, 27, 29} can be determined from the initial CRF list. The CRFs in the first subset can then be used sequentially to encode the VMAF of each frame in the video. A predetermined condition is determined to be met when the VMAF of more than 90% of the frames is between [85, 99]. The CRF used when the predetermined condition is met can then be used as the baseline CRF. If multiple CRFs meet the predetermined condition, the first CRF that meets the predetermined condition is used as the baseline CRF. If the VMAF of each frame in the video encoded using the CRFs in the first subset does not meet the predetermined condition, the last CRF in the first subset, CRF = 29, is used as the baseline CRF. Then, all CRFs in [29-6, 29+6] can be obtained from the initial CRF list. Finally, based on all CRFs in [29-6, 29+6], the CRF list {23, 25, 27, 29, 31, 33, 35} can be obtained.

[0058] According to an exemplary embodiment of the present disclosure, it can be determined that the predetermined condition is met when the first objective evaluation index of no more than a first preset number of frames is in a third preset numerical range, wherein the first preset number can be 10% of the frames and the third preset numerical range can be (99, 100].

[0059] According to an exemplary embodiment of the present disclosure, each CTU may first be encoded based on a baseline CRF to obtain a baseline first objective evaluation indicator and a baseline second objective evaluation indicator for each CTU. Each CTU may then be encoded based on at least one regular CRF to obtain at least one regular first objective evaluation indicator and at least one regular second objective evaluation indicator for each CTU. Finally, a first correlation coefficient for each CTU may be obtained using a least squares method based on the baseline first objective evaluation indicator, the baseline second objective evaluation indicator, the at least one regular first objective evaluation indicator, and the at least one regular second objective evaluation indicator.

[0060] For example, each CTU may first be encoded based on a baseline CRF to obtain a baseline VMAF and a baseline SSE for each CTU. Then, each CTU may be encoded based on at least one normal CRF to obtain at least one normal VMAF and at least one normal SSE for each CTU. Finally, a first correlation coefficient for each CTU may be obtained using the least squares method based on the baseline VMAF, the baseline SSE, the at least one normal VMAF, and the at least one normal SSE.

[0061] According to an exemplary embodiment of the present disclosure, ΔVMAF between a reference VMAF and each of at least one regular VMAF, and ΔSSE between a reference SSE and each of at least one regular SSE may be calculated.

[0062] For example, the first correlation coefficient of each CTU obtained by the least squares method can be expressed as the following equations (3) and (4):

[0063]

[0064]

[0065] in, is the first correlation coefficient of the i-th CTU.

[0066] According to an exemplary embodiment of the present disclosure, the second correlation coefficient of each frame of each video in at least one video is obtained by the following steps: first, a first-category CTU and a second-category CTU are determined based on a magnitude relationship between the first correlation coefficient of each CTU and a third preset value, wherein the first correlation coefficient of the first-category CTU is less than or equal to the third preset value, and the first correlation coefficient of the second-category CTU is greater than the third preset value. Then, the second correlation coefficient of each frame is obtained based on the first correlation coefficient of the second-category CTU.

[0067] According to an exemplary embodiment of the present disclosure, the third preset value may be set to 0. For example, the first correlation coefficient of the first type CTU may be set to be less than or equal to 0, and the first correlation coefficient of the second type CTU may be set to be greater than 0.

[0068] According to an exemplary embodiment of the present disclosure, a second correlation coefficient may be obtained according to the first correlation coefficient of the second-category CTU and the number of the second-category CTU.

[0069] For example, the second correlation coefficient of each frame can be obtained by the following equations (5) and (6):

[0070]

[0071]

[0072] Where k is the second correlation coefficient of any frame, is the first correlation coefficient of the i-th CTU in any frame, E represents the set of second-category CTUs in any frame, and |E| is the number of second-category CTUs in any frame.

[0073] Back to Figure 1 In step 102, the feature information of each CTU may be input into a prediction model to obtain a predicted ratio of the first correlation coefficient to the second correlation coefficient of each CTU.

[0074] Figure 2 FIG. 1 is a schematic diagram showing a structure of a prediction model according to an exemplary embodiment. Figure 2 , the prediction model can be CNN, Figure 2 In the structure shown, the squares from left to right are the input layer, convolution layer 1 (5*5 convolution kernel), pooling layer 1 (2*2), convolution layer 2 (3*3 convolution kernel), pooling layer 2 (2*2), convolution layer 3 (3*3 convolution kernel), pooling layer 3 (2*2), convolution layer 4, pooling layer 4 (2*2), and fully connected layer.

[0075] In step 103 , the loss value of the prediction model may be determined according to the actual ratio and the predicted ratio of the first correlation coefficient to the second correlation coefficient of each CTU.

[0076] According to an exemplary embodiment of the present disclosure, the loss value may be determined by, but not limited to, mean-squared loss (MSE Loss).

[0077] In step 104, the prediction model can be trained by adjusting the parameters of the prediction model according to the loss value. It should be noted that the prediction model can be trained using an offline training method.

[0078] Figure 3 FIG. 1 is a flow chart showing a method for calculating a rate-distortion cost according to an exemplary embodiment. Figure 3 In step 301, characteristic information of each CTU of the pre-coded frame may be obtained.

[0079] According to an exemplary embodiment of the present disclosure, the feature information may be any one of a pixel value, variance, edge information, gradient information, and motion information of the Y component.

[0080] In step 302, feature information of each CTU of the precoded frame may be input into a prediction model trained by the prediction model training method in the exemplary embodiment of the present disclosure to obtain a predicted ratio of the first correlation coefficient to the second correlation coefficient of each CTU of the precoded frame.

[0081] In step 303 , the rate-distortion cost of each CTU may be calculated according to the prediction ratio.

[0082] According to an exemplary embodiment of the present disclosure, the rate-distortion cost may be calculated according to the ratio, the SSE of each CTU, the encoding parameters, and the code rate of each CTU.

[0083] For example, the rate-distortion cost of the i-th CTU of the precoded frame is calculated as follows (7):

[0084]

[0085] Among them, RDCost is the rate-distortion cost, is the ratio of the i-th CTU, SSE i is the SSE of the i-th CTU, is the encoding parameter, r i is the bit rate of the i-th CTU.

[0086] According to an exemplary embodiment of the present disclosure, the model output can be set to a predicted first correlation coefficient value. Specifically, a training dataset can be first obtained, where the training dataset includes feature information of each CTU in at least one video and the true first correlation coefficient value of each CTU. The feature information of each CTU can then be input into the prediction model to obtain a predicted first correlation coefficient value for each CTU. Next, a loss value of the prediction model can be determined based on the true first correlation coefficient value and the predicted first correlation coefficient value for each CTU. Finally, the prediction model parameters can be adjusted based on the loss value to train the prediction model.

[0087] Furthermore, feature information for each CTU of the precoded frame can first be obtained. The feature information for each CTU of the precoded frame can then be input into the prediction model trained using the prediction model training method of the exemplary embodiment of the present disclosure to obtain a first correlation coefficient prediction value for each CTU of the precoded frame. Next, based on the relationship between the first correlation coefficient prediction value of each CTU and a third preset value, first-class CTUs and second-class CTUs can be determined, where the first correlation coefficient prediction value of the first-class CTU is less than or equal to the third preset value, and the first correlation coefficient prediction value of the second-class CTU is greater than the third preset value. Based on the first correlation coefficient prediction value of the second-class CTU, a second correlation coefficient for each frame can then be obtained. The ratio of the first correlation coefficient prediction value to the second correlation coefficient can then be obtained. Finally, based on the ratio for each CTU of the precoded frame, the rate-distortion cost of each CTU can be obtained.

[0088] Figure 4 FIG1 is a block diagram of a training device for a prediction model according to an exemplary embodiment. Figure 4 The training device 400 of the prediction model includes an acquisition unit 401, a prediction unit 402, a loss determination unit 403 and an adjustment unit 404.

[0089] The acquisition unit 401 can acquire a training data set, wherein the training data set includes feature information of each coding tree unit CTU in at least one video and a true ratio of a first correlation coefficient to a second correlation coefficient of each CTU, the first correlation coefficient represents the correlation between the first objective evaluation indicator and the second objective evaluation indicator at the CTU level, and the second correlation coefficient represents the correlation between the first objective evaluation indicator and the second objective evaluation indicator at the frame level.

[0090] According to an exemplary embodiment of the present disclosure, the first objective evaluation index may be VMAF, and the second objective evaluation index may be SSE.

[0091] According to an exemplary embodiment of the present disclosure, the feature information may be any one of a pixel value, variance, edge information, gradient information, and motion information of the Y component.

[0092] According to an exemplary embodiment of the present disclosure, first, the acquisition unit 401 may acquire a constant rate factor (CRF) list for each video. Then, the acquisition unit 401 may encode each CTU according to the CRF list to obtain a first correlation coefficient for each CTU.

[0093] According to an exemplary embodiment of the present disclosure, the acquisition unit 401 may first acquire an initial CRF list for each video, wherein the initial CRF list includes at least one CRF, which is arranged in ascending numerical order. The acquisition unit 401 may then determine an initial baseline CRF in the initial CRF list. Next, the acquisition unit 401 may determine a first subset from the initial CRF list based on the initial baseline CRF, wherein the first subset includes the initial baseline CRF and a preset number of CRFs following the initial baseline CRF. The acquisition unit 401 may then determine a baseline CRF from the first subset. Finally, the acquisition unit 401 may acquire a CRF list based on the baseline CRF.

[0094] According to an exemplary embodiment of the present disclosure, the acquisition unit 401 first uses the CRFs in the first subset to encode in sequence in the order of arrangement of the CRFs in the first subset to obtain the first objective evaluation index of each frame in each video. The acquisition unit 401 can then use the CRF used when the predetermined condition is met as the reference CRF, wherein when the first objective evaluation index of more than a preset number of frames is in a first preset numerical range, it is determined that the predetermined condition is met. It should be noted that when there are multiple CRFs that meet the predetermined condition, the first CRF that meets the predetermined condition is used as the reference CRF. Here, if the first objective evaluation index of each frame in each video obtained by encoding using the CRFs in the first subset does not meet the predetermined condition, the last CRF in the first subset is used as the reference CRF.

[0095] According to an exemplary embodiment of the present disclosure, first, acquisition unit 401 may obtain all CRFs within a second preset numerical range from an initial CRF list, where the second preset numerical range is determined based on the difference between a baseline CRF and the first preset value, and the sum of the baseline CRF and the second preset value. Then, acquisition unit 401 may obtain a CRF list for each video based on all CRFs within the second preset numerical range, where the CRF list includes the baseline CRF and at least one regular CRF.

[0096] For example, to determine the CRF list for any video, acquisition unit 401 may first obtain an initial CRF list {17, 19, 21, 23, 25, 27, 29, 31, 33, 35} for any video. Acquisition unit 401 may then determine an initial baseline CRF of 25 from the initial CRF list. Next, acquisition unit 401 may determine a first subset {25, 27, 29} from the initial CRF list based on the initial baseline CRF. Acquisition unit 401 may then sequentially use the CRFs in the first subset to encode and obtain the VMAF of each frame in any video. When the VMAF of more than 90% of the frames is between [85, 99], it is determined that a predetermined condition has been met. Acquisition unit 401 may then use the CRF used when the predetermined condition was met as the baseline CRF. Here, when there are multiple CRFs that meet the predetermined conditions, acquisition unit 401 may use the first CRF that meets the predetermined conditions as the base CRF. If the VMAF of any frame in any video encoded using the CRFs in the first subset does not meet the predetermined conditions, acquisition unit 401 uses the last CRF in the first subset, i.e., CRF = 29, as the base CRF. Acquisition unit 401 then acquires all CRFs within [29-6, 29+6] from the initial CRF list. Finally, acquisition unit 401 obtains the CRF list {23, 25, 27, 29, 31, 33, 35} based on all CRFs within [29-6, 29+6].

[0097] According to an exemplary embodiment of the present disclosure, the acquisition unit 401 may determine that a predetermined condition is satisfied when the first objective evaluation index of no more than a first preset number of frames is in a third preset numerical range, wherein the first preset number may be 10% of the frames and the third preset numerical range may be (99, 100].

[0098] According to an exemplary embodiment of the present disclosure, the acquisition unit 401 may first encode each CTU based on a baseline CRF to obtain a baseline first objective evaluation indicator and a baseline second objective evaluation indicator for each CTU. The acquisition unit 401 may then encode each CTU based on at least one regular CRF to obtain at least one regular first objective evaluation indicator and at least one regular second objective evaluation indicator for each CTU. Finally, the acquisition unit 401 may obtain a first correlation coefficient for each CTU using a least squares method based on the baseline first objective evaluation indicator, the baseline second objective evaluation indicator, the at least one regular first objective evaluation indicator, and the at least one regular second objective evaluation indicator.

[0099] For example, the acquisition unit 401 may first encode each CTU based on a baseline CRF to obtain a baseline VMAF and a baseline SSE for each CTU. The acquisition unit 401 may then encode each CTU based on at least one regular CRF to obtain at least one regular VMAF and at least one regular SSE for each CTU. Finally, the acquisition unit 401 may obtain a first correlation coefficient for each CTU using a least squares method based on the baseline VMAF, the baseline SSE, the at least one regular VMAF, and the at least one regular SSE.

[0100] According to an exemplary embodiment of the present disclosure, the acquisition unit 401 may calculate ΔVMAF between a reference VMAF and each of at least one regular VMAF, and ΔSSE between a reference SSE and each of at least one regular SSE.

[0101] For example, the first correlation coefficient of each CTU obtained by the least square method can be expressed as the above equations (3) and (4).

[0102] According to an exemplary embodiment of the present disclosure, the acquisition unit 401 may first determine a first-category CTU and a second-category CTU based on a magnitude relationship between a first correlation coefficient of each CTU and a third preset value, wherein the first correlation coefficient of the first-category CTU is less than or equal to the third preset value, and the first correlation coefficient of the second-category CTU is greater than the third preset value. Then, the acquisition unit 401 may obtain a second correlation coefficient for each frame based on the first correlation coefficient of the second-category CTU.

[0103] According to an exemplary embodiment of the present disclosure, the acquiring unit 401 may set the third preset value to 0. For example, the first correlation coefficient of the first type CTU may be set to be less than or equal to 0, and the first correlation coefficient of the second type CTU may be set to be greater than 0.

[0104] According to an exemplary embodiment of the present disclosure, the acquiring unit 401 may obtain the second correlation coefficient according to the first correlation coefficient of the second type of CTU and the number of the second type of CTU.

[0105] For example, the second correlation coefficient of each frame can be obtained by the above equations (5) and (6).

[0106] Back to Figure 4 The prediction unit 402 may input the feature information of each CTU into the prediction model to obtain a predicted ratio of the first correlation coefficient to the second correlation coefficient of each CTU.

[0107] The loss determination unit 403 may determine a loss value of the prediction model according to a true ratio and a predicted ratio of the first correlation coefficient to the second correlation coefficient of each CTU.

[0108] According to an exemplary embodiment of the present disclosure, the loss value may be determined by, but not limited to, mean-squared loss (MSE Loss).

[0109] The adjustment unit 404 may train the prediction model by adjusting parameters of the prediction model according to the loss value.

[0110] Figure 5 FIG1 is a block diagram of a rate-distortion cost calculation device according to an exemplary embodiment. Figure 5 The rate-distortion cost calculation device 500 includes an acquisition unit 501, a prediction unit 502 and a calculation unit 503.

[0111] The acquisition unit 501 may acquire feature information of each CTU of the pre-coded frame.

[0112] According to an exemplary embodiment of the present disclosure, the feature information may be any one of a pixel value, variance, edge information, gradient information, and motion information of the Y component.

[0113] The prediction unit 502 may input feature information of each CTU of the precoded frame into the prediction model trained by the prediction model training method in the exemplary embodiment of the present disclosure to obtain a predicted ratio of the first correlation coefficient to the second correlation coefficient of each CTU of the precoded frame.

[0114] The calculation unit 503 may calculate the rate-distortion cost of each CTU according to the prediction ratio.

[0115] According to an exemplary embodiment of the present disclosure, the rate-distortion cost may be calculated according to the ratio, the SSE of each CTU, the encoding parameters, and the code rate of each CTU.

[0116] For example, the calculation formula of the rate-distortion cost of the i-th CTU of the pre-coded frame is the above formula (7).

[0117] Figure 6 is a block diagram of an electronic device 600 according to an exemplary embodiment.

[0118] Reference Figure 6 The electronic device 600 includes at least one memory 601 and at least one processor 602, wherein the at least one memory 601 stores a set of computer-executable instructions. When the computer-executable instruction set is executed by the at least one processor 602, the training method of the prediction model or the rate-distortion cost calculation method according to the exemplary embodiments of the present disclosure is executed.

[0119] As an example, the electronic device 600 may be a PC, a tablet device, a personal digital assistant, a smart phone, or other device capable of executing the above-mentioned instruction set. Here, the electronic device 600 is not necessarily a single electronic device, but may also be any device or circuit collection capable of executing the above-mentioned instructions (or instruction set) individually or in combination. The electronic device 600 may also be part of an integrated control system or system manager, or may be configured as a portable electronic device interconnected with a local or remote (e.g., via wireless transmission) interface.

[0120] In electronic device 600, processor 602 may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller, or a microprocessor. By way of example and not limitation, the processor may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc.

[0121] The processor 602 can execute instructions or codes stored in the memory 601, wherein the memory 601 can also store data. Instructions and data can also be sent and received over the network via the network interface device, wherein the network interface device can use any known transmission protocol.

[0122] The memory 601 may be integrated with the processor 602, for example, by placing RAM or flash memory within an integrated circuit microprocessor or the like. Furthermore, the memory 601 may comprise a separate device, such as an external disk drive, a storage array, or any other storage device usable by a database system. The memory 601 and the processor 602 may be operatively coupled or may communicate with each other, for example, via an I / O port, a network connection, or the like, such that the processor 602 can access files stored in the memory.

[0123] In addition, the electronic device 600 may further include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, a mouse, a touch input device, etc.) All components of the electronic device 600 may be connected to each other via a bus and / or a network.

[0124] According to an exemplary embodiment of the present disclosure, a computer-readable storage medium may also be provided, wherein, when the instructions stored in the computer-readable storage medium are executed by at least one processor, the at least one processor is prompted to perform the training method of the prediction model or the rate-distortion cost calculation method according to the exemplary embodiment of the present disclosure. Examples of computer-readable storage media here include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disk storage, hard disk drive (HDD), solid state drive (SSD), card storage (such as, multimedia card, secure digital (SD) card or ultra-fast digital (XD) card), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk and any other device, any other device configured to store the computer program and any associated data, data files and data structures in a non-transitory manner and provide the computer program and any associated data, data files and data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the above-mentioned computer-readable storage medium can be run in an environment deployed in a computer device such as a client, a host, an agent device, a server, etc. In addition, in one example, the computer program and any associated data, data files and data structures are distributed on a networked computer system so that the computer program and any associated data, data files and data structures are stored, accessed and executed in a distributed manner by one or more processors or computers.

[0125] According to an exemplary embodiment of the present disclosure, a computer program product may also be provided. Instructions in the computer program product may be executed by a processor of a computer device to complete the prediction model training method or rate-distortion cost calculation method according to the exemplary embodiment of the present disclosure.

[0126] According to the prediction model training method and device and rate-distortion cost calculation method and device disclosed in the present invention, the prediction model is trained with the ratio of the first correlation coefficient to the second correlation coefficient as the target, and the prediction ratio is applied to the rate-distortion cost calculation process. The correlation coefficient ratio can be obtained without replacing the CTU multiple times to calculate the VMAF, thereby reducing the calculation time complexity and shortening the calculation time.

[0127] In addition, according to the prediction model training method and device and rate-distortion cost calculation method and device disclosed in the present invention, the prediction ratio is applied to the rate-distortion cost calculation process, which can reduce the BD-Rate-VMAF value while increasing the encoding time complexity by no more than 20%, thereby obtaining encoding benefits.

[0128] In addition, according to the prediction model training method and device and rate-distortion cost calculation method and device disclosed in the present invention, the prediction model can be trained in an offline training manner, avoiding the need to retrain each network.

[0129] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0130] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for training a prediction model, characterized in that: include: Acquire a training data set, wherein the training data set includes feature information of each coding tree unit (CTU) in at least one video and a true ratio of a first correlation coefficient to a second correlation coefficient of each CTU, The first correlation coefficient represents the correlation between the first objective evaluation index and the second objective evaluation index at the CTU level, and the second correlation coefficient represents the correlation between the first objective evaluation index and the second objective evaluation index at the frame level; Inputting the feature information of each CTU into a prediction model to obtain a predicted ratio of the first correlation coefficient to the second correlation coefficient of each CTU; determining a loss value of the prediction model according to a true ratio and a predicted ratio of the first correlation coefficient to the second correlation coefficient of each CTU; The prediction model is trained by adjusting parameters of the prediction model according to the loss value.

2. The training method according to claim 1, wherein: The first correlation coefficient of each CTU of each video in the at least one video is obtained by the following steps: Obtain a constant rate factor crf list for each video; Each CTU is encoded according to the CRF list to obtain a first correlation coefficient of each CTU.

3. The training method according to claim 2, wherein: The obtaining of a constant rate factor crf list for each video includes: Obtaining an initial CRF list for each video, wherein the initial CRF list includes at least one CRF, and the at least one CRF is arranged in ascending order of value; Determining an initial reference crf in the initial crf list; Determine a first subset in the initial CRF list according to the initial reference CRF, wherein the first subset includes the initial reference CRF and a preset number of CRFs subsequent to the initial reference CRF; determining a reference crf from said first subset; According to the reference CRF, the CRF list is obtained.

4. The training method according to claim 3, wherein: The determining of a reference crf from the first subset includes: Using the CRFs in the first subset in sequence for encoding in the order in which the CRFs in the first subset are arranged, to obtain a first objective evaluation index for each frame in each video; using the crf used when a predetermined condition is satisfied as a benchmark crf, wherein when a first objective evaluation indicator of more than a preset number of frames is within a first preset value interval, determining that the predetermined condition is satisfied; If the first objective evaluation index of each frame in each video obtained by encoding using the CRF in the first subset does not meet the predetermined condition, the last CRF in the first subset is used as the benchmark CRF.

5. The training method according to claim 3, wherein: The step of obtaining the CRF list according to the benchmark CRF includes: Obtaining, from the initial CRF list, all CRFs within a second preset numerical range, wherein the second preset numerical range is determined based on a difference between the reference CRF and the first preset value and a sum of the reference CRF and the second preset value; A CRF list for each video is obtained according to all CRFs within the second preset value range, wherein the CRF list includes the reference CRF and at least one regular CRF.

6. The training method according to claim 5, wherein: The encoding of each CTU according to the CRF list to obtain a first correlation coefficient of each CTU includes: Encoding each CTU according to the benchmark crf to obtain a benchmark first objective evaluation index and a benchmark second objective evaluation index of each CTU; Encode each CTU according to the at least one conventional CRF to obtain at least one conventional first objective evaluation indicator and at least one conventional second objective evaluation indicator of each CTU; A first correlation coefficient of each CTU is obtained by a least squares method according to the benchmark first objective evaluation indicator, the benchmark second objective evaluation indicator, the at least one conventional first objective evaluation indicator, and the at least one conventional second objective evaluation indicator.

7. The training method according to claim 1, wherein: The second correlation coefficient of each frame of each video in the at least one video is obtained by the following steps: Determine a first-category CTU and a second-category CTU based on a relationship between a first correlation coefficient of each CTU and a third preset value, wherein the first correlation coefficient of the first-category CTU is less than or equal to the third preset value, and the first correlation coefficient of the second-category CTU is greater than the third preset value; A second correlation coefficient of each frame is obtained according to the first correlation coefficient of the second type CTU.

8. A rate-distortion cost calculation method, characterized in that: include: Obtain characteristic information of each CTU of the pre-coded frame; Inputting the feature information of each CTU of the precoded frame into a prediction model trained by the prediction model training method according to any one of claims 1 to 7, to obtain a predicted ratio of the first correlation coefficient to the second correlation coefficient of each CTU of the precoded frame; The rate-distortion cost of each CTU is calculated based on the predicted ratio.

9. A training device for a prediction model, characterized in that: include: an acquiring unit configured to: acquire a training data set, wherein the training data set includes feature information of each coding tree unit (CTU) in at least one video and a true ratio of a first correlation coefficient to a second correlation coefficient of each CTU, wherein the first correlation coefficient represents a correlation between a first objective evaluation indicator and a second objective evaluation indicator at a CTU level, and the second correlation coefficient represents a correlation between the first objective evaluation indicator and the second objective evaluation indicator at a frame level; The prediction unit is configured to: input the feature information of each CTU into a prediction model to obtain a predicted ratio of the first correlation coefficient to the second correlation coefficient of each CTU; a loss determining unit configured to: determine a loss value of the prediction model according to a true ratio and a predicted ratio of the first correlation coefficient to the second correlation coefficient of each CTU; An adjustment unit is configured to train the prediction model by adjusting parameters of the prediction model according to the loss value.

10. The training device according to claim 9, characterized in that The acquiring unit is configured to acquire a first correlation coefficient of each CTU of each video in the at least one video by: Obtain a constant rate factor crf list for each video; Each CTU is encoded according to the CRF list to obtain a first correlation coefficient of each CTU.

11. The training device according to claim 10, wherein: The acquisition unit is configured as follows: Obtaining an initial CRF list for each video, wherein the initial CRF list includes at least one CRF, and the at least one CRF is arranged in ascending order of value; Determining an initial reference crf in the initial crf list; Determine a first subset in the initial CRF list according to the initial reference CRF, wherein the first subset includes the initial reference CRF and a preset number of CRFs subsequent to the initial reference CRF; determining a reference crf from said first subset; According to the reference CRF, the CRF list is obtained.

12. The training device according to claim 11, wherein The acquisition unit is configured as follows: Using the CRFs in the first subset in sequence for encoding in the order in which the CRFs in the first subset are arranged, to obtain a first objective evaluation index for each frame in each video; using the crf used when a predetermined condition is satisfied as a benchmark crf, wherein when a first objective evaluation indicator of more than a preset number of frames is within a first preset value interval, determining that the predetermined condition is satisfied; If the first objective evaluation index of each frame in each video obtained by encoding using the CRF in the first subset does not meet the predetermined condition, the last CRF in the first subset is used as the benchmark CRF.

13. The training device according to claim 11, wherein: The acquisition unit is configured as follows: Obtaining, from the initial CRF list, all CRFs within a second preset numerical range, wherein the second preset numerical range is determined based on a difference between the reference CRF and the first preset value and a sum of the reference CRF and the second preset value; A CRF list for each video is obtained according to all CRFs within the second preset value range, wherein the CRF list includes the reference CRF and at least one regular CRF.

14. The training device according to claim 13, wherein: The acquisition unit is configured as follows: Encoding each CTU according to the benchmark crf to obtain a benchmark first objective evaluation index and a benchmark second objective evaluation index of each CTU; Encode each CTU according to the at least one conventional CRF to obtain at least one conventional first objective evaluation indicator and at least one conventional second objective evaluation indicator of each CTU; A first correlation coefficient of each CTU is obtained by a least squares method according to the benchmark first objective evaluation indicator, the benchmark second objective evaluation indicator, the at least one conventional first objective evaluation indicator, and the at least one conventional second objective evaluation indicator.

15. The training device according to claim 9, wherein: The acquiring unit is configured to acquire the second correlation coefficient of each frame of each video in the at least one video by: Determine a first-category CTU and a second-category CTU based on a relationship between a first correlation coefficient of each CTU and a third preset value, wherein the first correlation coefficient of the first-category CTU is less than or equal to the third preset value, and the first correlation coefficient of the second-category CTU is greater than the third preset value; A second correlation coefficient of each frame is obtained according to the first correlation coefficient of the second type CTU.

16. A rate-distortion cost calculation device, characterized in that: include: The acquiring unit is configured to: acquire characteristic information of each CTU of the pre-coded frame; The prediction unit is configured to: input feature information of each CTU of the precoded frame into a prediction model trained by the prediction model training method according to any one of claims 1 to 7, to obtain a predicted ratio of the first correlation coefficient to the second correlation coefficient of each CTU of the precoded frame; The calculation unit is configured to calculate the rate-distortion cost of each CTU according to the prediction ratio.

17. An electronic device, characterized in that: include: at least one processor; at least one memory storing computer-executable instructions, When the computer executable instructions are executed by the at least one processor, the at least one processor is prompted to execute the prediction model training method according to any one of claims 1 to 7 or the rate-distortion cost calculation method according to claim 8.

18. A computer-readable storage medium, characterized in that When the instructions stored in the computer-readable storage medium are executed by at least one processor, the at least one processor is prompted to perform the prediction model training method according to any one of claims 1 to 7 or the rate-distortion cost calculation method according to claim 8.

19. A computer program product comprising computer instructions, characterized in that When the computer instructions are executed by at least one processor, the prediction model training method according to any one of claims 1 to 7 or the rate-distortion cost calculation method according to claim 8 is implemented.

Images