Video encoding method, apparatus and device

Abstract

This disclosure provides a video encoding method, apparatus, and device, relating to the field of artificial intelligence, specifically cloud computing, video encoding, and media cloud technologies, applicable to intelligent cloud scenarios. A specific implementation of the quantization decision model training method includes: acquiring a set of sample video sequences, wherein the set of sample video sequences includes sample video sequences; acquiring a target feature set and quantization labels of the current transform unit encoding the sample video sequences; using the target feature set as input and the quantization labels as output, training an initial quantization decision model to obtain the quantization decision model. This implementation, by training the quantization decision model, enables quantization decisions to be made in advance based on machine learning models, saving significant time and improving quantization accuracy.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence, specifically cloud computing, video encoding, and media cloud technologies, which can be applied in intelligent cloud scenarios. Background Technology

[0002] Quantization is an indispensable and crucial step in video coding. Quantization refers to the process of mapping continuous values ​​(or a large number of possible discrete values) of a signal to a finite number of discrete amplitude values, achieving a many-to-one mapping of signal values. Quantization inevitably introduces distortion. For example, if RDOQ (Rate Distortion Optimized Quantization) is chosen, it involves a complex RDO (Rate Distortion Optimized) process, making quantization relatively time-consuming. Therefore, determining when to use RDOQ becomes a critical aspect of video coding.

[0003] Currently, quantitative decision-making methods mainly include empirical formulas, experimental data fitting, and formula derivation, which use statistical characteristics to determine whether to use RDOQ quantization. Summary of the Invention

[0004] This disclosure provides a video encoding method, apparatus, device, storage medium, and program product.

[0005] In a first aspect, embodiments of this disclosure propose a method for training a quantization decision model, comprising: acquiring a set of sample video sequences, wherein the set of sample video sequences includes sample video sequences; acquiring a target feature set and a quantization label of the current transform unit in encoding the sample video sequences; and training an initial quantization decision model by using the target feature set as input and the quantization label as output to obtain a quantization decision model.

[0006] Secondly, embodiments of this disclosure propose a video coding method, including: acquiring a video sequence; acquiring a target feature set of the current transform unit in encoding the video sequence; inputting the target feature set into a quantization decision model to obtain a quantization decision result of the current transform unit, wherein the quantization decision model is trained using the method described in the first aspect; and quantizing the current transform unit based on the quantization decision result.

[0007] Thirdly, embodiments of this disclosure propose a quantization decision model training device, comprising: a first acquisition module configured to acquire a set of sample video sequences, wherein the set of sample video sequences includes sample video sequences; a second acquisition module configured to acquire a target feature set and a quantization label of the current transform unit encoding the sample video sequences; and a first training module configured to train an initial quantization decision model by taking the target feature set as input and the quantization label as output, thereby obtaining a quantization decision model.

[0008] Fourthly, embodiments of this disclosure propose a video encoding apparatus, comprising: a first acquisition module configured to acquire a video sequence; a second acquisition module configured to acquire a target feature set of a current transform unit encoding the video sequence; a first decision module configured to input the target feature set into a quantization decision model to obtain a quantization decision result of the current transform unit, wherein the quantization decision model is trained using the apparatus described in the third aspect; and a first quantization module configured to quantize the current transform unit based on the quantization decision result.

[0009] Fifthly, embodiments of this disclosure provide an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform a method as described in any implementation of the first or second aspect.

[0010] In a sixth aspect, embodiments of this disclosure provide a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method described in any implementation of the first or second aspect.

[0011] In a seventh aspect, embodiments of this disclosure provide a computer program product, including a computer program that, when executed by a processor, implements the method as described in either the first or second aspect.

[0012] The quantitative decision-making model training method provided in this disclosure enables quantitative decision-making based on machine learning models to be made in advance, which not only saves a lot of time but also improves the accuracy of quantification.

[0013] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0014] Other features, objects, and advantages of this disclosure will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings. The drawings are provided for a better understanding of the invention and are not intended to limit the scope of this disclosure. Wherein:

[0015] Figure 1 This is a flowchart of an embodiment of the quantitative decision model training method according to the present disclosure;

[0016] Figure 2 This is a flowchart of yet another embodiment of the quantitative decision-making model training method according to the present disclosure;

[0017] Figure 3 This is an example flowchart for tag acquisition.

[0018] Figure 4 This is a flowchart of another embodiment of the quantitative decision model training method according to this disclosure;

[0019] Figure 5 This is a flowchart of an embodiment of the video coding method according to the present disclosure;

[0020] Figure 6 This is a flowchart of yet another embodiment of the video coding method according to the present disclosure;

[0021] Figure 7 This is an exemplary flowchart for quantitative decision-making;

[0022] Figure 8 This is yet another exemplary flowchart for quantitative decision-making;

[0023] Figure 9 This is an exemplary flowchart of video encoding;

[0024] Figure 10 This is a schematic diagram of the structure of an embodiment of a quantitative decision model training device according to the present disclosure;

[0025] Figure 11 This is a schematic diagram of the structure of an embodiment of the video encoding apparatus according to the present disclosure;

[0026] Figure 12 This is a block diagram of an electronic device used to implement the quantitative decision model training method or video encoding method of the embodiments of this disclosure. Detailed Implementation

[0027] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0028] It should be noted that, unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other. This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0029] Figure 1 A flowchart 100 is shown as an embodiment of a quantitative decision model training method according to the present disclosure. The quantitative decision model training method includes the following steps:

[0030] Step 101: Obtain the set of sample video sequences.

[0031] In this embodiment, the entity executing the quantization decision model training method can obtain a set of sample video sequences. This set of sample video sequences can consist of a large number of sample video sequences.

[0032] Typically, a sample video sequence set can include multiple sample video sequences with different temporal and / or spatial complexities. To ensure that the quantization decision model can cover the quantization decisions of as many videos as possible, multiple video sequences can be selected based on their different temporal and / or spatial complexities to generate a sample video sequence set.

[0033] Step 102: Obtain the target feature set and quantization label of the current transform unit in encoding the sample video sequence.

[0034] In this embodiment, the aforementioned execution entity can obtain the target feature set and quantization label of the current transform unit in encoding the sample video sequence.

[0035] In encoding sample video sequences, the images in the sequence can be divided into CTUs (Coding Tree Units). Each CTU can be further subdivided into CUs (Coding Units), PUs (Prediction Units), and TUs (Transform Units). For each transform unit, a set of target features can be collected. This set may include, but is not limited to, at least one of the following target features: the size of the current transform unit, its mean and variance of brightness, the sum of absolute values, mean and variance of the transform matrix of the current transform unit, and the sum of absolute values, mean and variance of the transform matrix after convolution, etc. The mean and variance of brightness reflect the brightness distribution. When convolving the transform matrix, different kernel sizes are used for different sizes of current transform units, such as 4x4, 8x8, 16x16, 32x32, etc.

[0036] Quantization is an essential and crucial step in encoding sample video sequences. Quantization refers to the process of mapping continuous values ​​(or a large number of possible discrete values) of a signal to a finite number of discrete amplitudes, achieving a many-to-one mapping of signal values. Quantization methods can include, but are not limited to, RDOQ and HDQ (Hard Decision Quantization). RDOQ has higher accuracy than HDQ. Quantization labels can indicate whether a particular quantization method is used. For example, a quantization label indicates whether RDOQ is used, with 1 indicating RDOQ and 0 indicating HDQ.

[0037] Step 103: Using the target feature set as input and the quantization label as output, train the initial quantization decision model to obtain the quantization decision model.

[0038] In this embodiment, the aforementioned execution entity can use the target feature set as input and the quantization label as output to train the initial quantization decision model and obtain the quantization decision model.

[0039] First, an initial quantization decision model is obtained by initializing the model parameters. For example, the xgBoost model, which performs well in various data competitions, can be selected for parameter initialization as the initial quantization decision model.

[0040] Then, by inputting the target feature set into the initial quantization decision model, the probability of using a certain quantization method (such as RDOQ) can be learned.

[0041] Finally, based on the difference between the probability and the quantization label, the parameters of the initial quantization decision model are adjusted to make the difference sufficiently small, thus obtaining the quantization decision model. This quantization decision model is a binary classification model that can determine the probability of using a certain quantization method (such as RDOQ).

[0042] The quantitative decision-making model training method provided in this disclosure enables quantitative decision-making based on machine learning models to be made in advance, which not only saves a lot of time but also improves the accuracy of quantification.

[0043] Continue to refer to Figure 2 This illustrates a flow 200 of yet another embodiment of the quantitative decision model training method according to the present disclosure. The quantitative decision model training method includes the following steps:

[0044] Step 201: Obtain the set of sample video sequences.

[0045] In this embodiment, the specific operation of step 201 has been described. Figure 1 The steps in step 101 of the illustrated embodiment are described in detail and will not be repeated here.

[0046] Step 202: Obtain the reference feature set of the current transformation unit.

[0047] In this embodiment, the entity executing the quantization decision model training method can obtain the reference feature set of the current transformation unit.

[0048] Typically, the number of reference features included in the reference feature set is no less than the number of target features included in the target feature set, and the target feature set is a subset of the reference feature set.

[0049] Step 203: Using the reference feature set as input and the quantization label as output, train the initial quantization decision model and obtain the importance of the reference features in the reference feature set during model training.

[0050] In this embodiment, the aforementioned execution entity can use the reference feature set as input and the quantization label as output to train the initial quantization decision model and obtain the importance of the reference features in the reference feature set during the model training process.

[0051] Here, the reference feature set is taken as input and the quantization label is taken as output. During the training of the initial quantization decision model, the importance of the reference features in the reference feature set will be given, and the reference features in the reference feature set will be sorted according to their importance.

[0052] Step 204: Select reference features from the reference feature set based on importance to generate the target feature set.

[0053] In this embodiment, the aforementioned execution entity can select reference features from the reference feature set based on importance to generate the target feature set.

[0054] Here, more important features can be selected to be added to the target feature set. For example, the reference features can be sorted in descending order of importance, and a preset number of reference features can be selected starting from the side with the highest importance, or reference features with an importance greater than a preset importance threshold can be selected to be added to the target feature set.

[0055] Typically, the accuracy of the trained quantization decision-making model is directly determined by the correctness of feature selection. Here, using a machine learning model (such as the xgBoost model) to select features can identify more important features.

[0056] Step 205: Perform a transformation operation on the residual matrix of the current transformation unit to generate the transformation matrix of the current transformation unit.

[0057] In this embodiment, the execution entity can perform a transformation operation on the residual matrix of the current transformation unit to generate the transformation matrix of the current transformation unit.

[0058] Step 206: Perform first quantization and second quantization on the transformation matrix to obtain the first quantization matrix and the second quantization matrix.

[0059] In this embodiment, the aforementioned execution entity can perform a first quantization (such as RDOQ) and a second quantization (such as HDQ) on the transformation matrix to obtain a first quantization matrix (such as an RDOQ matrix) and a second quantization matrix (such as an HDQ matrix).

[0060] Step 207: Compare the differences between the first quantization matrix and the second quantization matrix to obtain the quantization label.

[0061] In this embodiment, the aforementioned execution entity can compare the differences between the first quantization matrix and the second quantization matrix to obtain the quantization label.

[0062] For example, quantization labels are obtained by comparing the differences between the RDOQ matrix and the HDQ matrix. Specifically, if the RDOQ matrix and the HDQ matrix are identical, it means that RDOQ quantization is not needed; otherwise, the more accurate RDOQ quantization is used. In practice, even if there is a slight difference between the RDOQ matrix and the HDQ matrix, they can be considered to be identical.

[0063] When comparing the differences between the first and second quantization matrices, the coefficients of the first and second quantization matrices are first subtracted to obtain the difference matrix. Then, the difference matrix is ​​convolved using a kernel corresponding to the size of the current transform unit to obtain the convolution result. Finally, the difference between the convolution result and a preset threshold is compared to obtain the quantization label. After the transform operation, the coefficients are mainly concentrated in the upper left corner (the upper left corner represents low-frequency coefficients). Different convolution kernels (such as 4x4, 8x8, 16x16, and 32x32) are set for transform units of different sizes. The closer to the upper left corner, the larger the coefficient.

[0064] When comparing the convolution result with a preset threshold, the convolution result can be normalized first to obtain a normalized result; then, the difference between the normalized result and the preset threshold is compared to obtain the quantization label. Specifically, if the normalized result is less than the preset threshold, the difference matrix can be considered as a 0 matrix, indicating that the first quantization matrix (such as the RDOQ matrix) and the second quantization matrix (such as the HDQ matrix) are basically equal. In this case, the value of the quantization label is set to indicate that the second quantization (such as HDQ) is used. If the normalized result is not less than the preset threshold, it indicates that the first quantization matrix (such as the RDOQ matrix) and the second quantization matrix (such as the HDQ matrix) are not equal. In this case, the value of the quantization label is set to indicate that the first quantization (such as RDOQ) is used.

[0065] Here, by comparing the differences between the results of the first quantization (such as RDOQ) and the second quantization (such as HDQ), quantization labels are generated, which improves the accuracy of the quantization labels.

[0066] For ease of understanding, Figure 3 An exemplary flowchart for tag acquisition is shown.

[0067] The first step is to calculate the residual of the current TU.

[0068] The second step is to perform transformation operations on the residual matrix;

[0069] The third step is to extract the features of the transformation matrix;

[0070] The fourth step is to use RDOQ and HDQ to quantize the transformation matrix respectively, and output the RDOQ matrix and HDQ matrix.

[0071] Fifth step: Compare the differences between the RDOQ matrix and the HDQ matrix, and output the quantization label.

[0072] Step 208: Take the target feature set as input and the quantization label as output to train the initial quantization decision model to obtain the quantization decision model.

[0073] In this embodiment, the specific operation of step 208 has been described. Figure 1Step 103 in the illustrated embodiment is described in detail and will not be repeated here.

[0074] from Figure 2 It can be seen from this that, with Figure 1 Compared to the corresponding embodiments, the flowchart 200 of the quantization decision model training method in this embodiment emphasizes the feature selection step and the quantization label generation step. Therefore, the scheme described in this embodiment uses a machine learning model (such as the xgBoost model) to select features, which can select more important features, thereby improving the accuracy of the trained quantization decision model. Simultaneously, by comparing the differences between the results of the first quantization (such as RDOQ) and the second quantization (such as HDQ), quantization labels are generated, improving the accuracy of the quantization labels.

[0075] Further reference Figure 4 The diagram illustrates flow 400 of another embodiment of the quantitative decision model training method according to this disclosure. This quantitative decision model training method includes the following steps:

[0076] Step 401: Obtain the set of sample video sequences.

[0077] Step 402: Obtain the target feature set and quantization label of the current transform unit in encoding the sample video sequence.

[0078] Step 403: Using the target feature set as input and the quantization label as output, train the initial quantization decision model to obtain the quantization decision model.

[0079] In this embodiment, the specific operations of steps 401-403 have been described. Figure 1 The steps 101-103 in the illustrated embodiments are described in detail and will not be repeated here.

[0080] Step 404: Select at least one subset of target features from the target feature set.

[0081] In this embodiment, the entity executing the quantization decision model training method can select at least one subset of target features from the target feature set. The number of target feature subsets is equal to the number of auxiliary quantization decision models, and each target feature subset serves as a model input for training one auxiliary quantization decision model.

[0082] Step 405: Take at least one subset of target features as input and quantization labels as output to train at least one auxiliary quantization decision model.

[0083] In this embodiment, the aforementioned execution entity can use at least one subset of target features as input and quantization labels as output to train at least one auxiliary quantization decision model.

[0084] For a subset of target features, using it as input and the quantization label as output, an auxiliary quantization decision model is trained. Specifically, first, the model parameters are initialized to obtain an initial auxiliary quantization decision model; then, the target feature subset is input into the initial auxiliary quantization decision model, which learns the probability of using a certain quantization method (such as RDOQ); finally, based on the difference between the probability and the quantization label, the parameters of the initial auxiliary quantization decision model are adjusted so that the difference is small enough to obtain the final auxiliary quantization decision model. The auxiliary quantization decision model is a binary classification model that can determine the probability of using a certain quantization method (such as RDOQ).

[0085] Since decision support models are only used to assist in decision-making, relatively simple models such as SVM (Support Vector Machine) and LR (Logistic Regression) can be chosen for decision support. This can improve the accuracy of decision-making.

[0086] It should be noted that, to improve robustness, different models use slightly different input features. Here, a more scientific feature selection method (feature engineering) is used. This involves ranking the importance of features using machine learning models, while also incorporating transform quantization expertise to comprehensively select features that have a greater correlation with the results. By using higher-performing machine learning models and a more systematic feature selection approach, and employing model fusion, features with higher relevance to the results can be selected, leading to the training of a better model.

[0087] from Figure 4 It can be seen from this that, with Figure 1 Compared to the corresponding embodiments, the quantitative decision-making model training method in this embodiment adds an auxiliary quantitative decision-making model training step to process step 400. Therefore, the solution described in this embodiment improves decision accuracy by training an auxiliary quantitative decision-making model for assisted decision-making.

[0088] Further reference Figure 5 The diagram illustrates a flow 500 of an embodiment of a video encoding method according to the present disclosure. The video encoding method includes the following steps:

[0089] Step 501: Obtain the video sequence.

[0090] In this embodiment, the entity executing the video encoding method can obtain a video sequence. Here, the video sequence can be a video sequence that needs to be encoded.

[0091] Step 502: Obtain the target feature set of the current transform unit in encoding the video sequence.

[0092] In this embodiment, the aforementioned execution entity can obtain the target feature set of the current transform unit in encoding the video sequence.

[0093] In encoding video sequences, images within the sequence can be divided into CTUs. Each CTU can be further subdivided into CUs, PUs, and TUs. For each current transform unit, its target feature set can be acquired. This target feature set may include, but is not limited to, at least one of the following target features: the size of the current transform unit, its mean and variance of luminance, the sum of absolute values, mean and variance of the transform matrix of the current transform unit, and the sum of absolute values, mean and variance of the transform matrix after convolution, etc. The mean and variance of luminance reflect the luminance distribution. When convolving the transform matrix, different kernel sizes are used for different sizes of current transform units, such as 4x4, 8x8, 16x16, 32x32, etc.

[0094] Step 503: Input the target feature set into the quantization decision model to obtain the quantization decision result of the current transformation unit.

[0095] In this embodiment, the execution entity can input the target feature set into the quantization decision model to obtain the quantization decision result of the current transformation unit. The quantization decision result can be the probability of using a certain quantization method (such as RDOQ). The quantization decision model can employ... Figure 1 , Figure 2 and Figure 4 The method shown is used for training, and will not be elaborated further here.

[0096] Step 504: Quantize the current transformation unit based on the quantization decision results.

[0097] In this embodiment, the aforementioned execution entity can quantize the current transformation unit based on the quantization decision result. For example, if the probability of using a certain quantization method (such as RDOQ) is greater than a certain threshold, then that quantization method (such as RDOQ) is used; otherwise, that quantization method (such as RDOQ) is not used.

[0098] The video coding method provided in this disclosure is based on a quantization decision model, enabling quantization decisions to be made in advance using a machine learning model. This not only saves a significant amount of time but also improves the accuracy of quantization. Utilizing a quantization decision model effectively simplifies the complex calculation process in quantization. By determining in advance whether the current transform unit should use a certain quantization method (such as RDOQ), redundant calculations are eliminated, reducing the computational load of quantization. This achieves the goal of skipping redundant quantization processes while maintaining consistent coding quality, thereby improving both coding speed and quality.

[0099] Further reference Figure 6 This illustrates a flow 600 of yet another embodiment of the video encoding method according to the present disclosure. The video encoding method includes the following steps:

[0100] Step 601: Obtain the video sequence.

[0101] Step 602: Obtain the target feature set of the current transform unit in encoding the video sequence.

[0102] Step 603: Input the target feature set into the quantization decision model to obtain the quantization decision result of the current transformation unit.

[0103] In this embodiment, the specific operations of steps 601-603 have been described. Figure 5 The steps 501-503 in the illustrated embodiment are described in detail and will not be repeated here.

[0104] Step 604: Select at least one subset of target features from the target feature set.

[0105] In this embodiment, the entity executing the video coding method can select at least one subset of target features from the target feature set. The number of target feature subsets is equal to the number of auxiliary quantization decision models, and each target feature subset serves as input to one auxiliary quantization decision model.

[0106] Step 605: Input at least one subset of target features into at least one auxiliary quantization decision model to obtain at least one auxiliary quantization decision result of the current transformation unit.

[0107] In this embodiment, the execution entity can input at least one subset of target features into at least one auxiliary quantization decision model to obtain at least one auxiliary quantization decision result of the current transformation unit.

[0108] For a subset of target features, it is input into an auxiliary quantization decision model to obtain an auxiliary quantization decision result. This auxiliary quantization decision result can be the probability of using a certain quantization method (such as RDOQ). Figure 5 The method shown is used for training, and will not be elaborated further here.

[0109] Step 606: Merge the quantization decision result and at least one auxiliary quantization decision result, and quantize the current transformation unit based on the merged result.

[0110] In this embodiment, the aforementioned execution entity can merge the quantization decision result and at least one auxiliary quantization decision result, and quantize the current transformation unit based on the merged result.

[0111] For example, if the probability of using the first quantization (such as RDOQ) in the quantization decision result is greater than a preset probability threshold, the quantization method (such as RDOQ or HDQ) of the current transformation unit is determined based on at least one auxiliary quantization decision result, and the current transformation unit is quantized; if the probability of using the first quantization in the quantization decision result is not greater than the preset probability threshold, the current transformation unit is quantized in the second quantization (such as HDQ).

[0112] In some embodiments, decisions can be made in a serial manner. Specifically, if the probability of using the first quantization (e.g., RDOQ) in the quantization decision results is greater than a preset probability threshold, and if the probability of using the first quantization (e.g., RDOQ) in all auxiliary quantization decision results is greater than the preset probability threshold, the current transform unit is subjected to the first quantization (e.g., RDOQ); if at least one of the auxiliary quantization decision results has a probability of using the first quantization (e.g., RDOQ) that is not greater than the preset probability threshold, the current transform unit is subjected to the second quantization (e.g., HDQ).

[0113] For ease of understanding, Figure 7 An exemplary flowchart for quantifying decision-making is shown.

[0114] The first step is to determine whether the RDOQ probability output by Model 1 (quantized decision model) is greater than the threshold 1;

[0115] The second step is to determine whether the RDOQ probability output by model 1 is greater than threshold 1 if the RDOQ probability output by model 2 (an auxiliary quantization decision model) is greater than threshold 2.

[0116] The third step is to determine whether the RDOQ probability output by model 2 is greater than threshold 2 if the RDOQ probability output by model 3 (another auxiliary quantification decision model) is greater than threshold 3.

[0117] Fourth step: If the RDOQ probability output by model 3 is greater than the threshold 3, select RDOQ for quantization;

[0118] Fifth step: If the RDOQ probability output by model 1 is not greater than threshold 1, or the RDOQ probability output by model 2 is not greater than threshold 2, or the RDOQ probability output by model 3 is not greater than threshold 3, then select HDQ for quantization.

[0119] In some embodiments, decision-making can be performed in parallel. Specifically, if the probability of using the first quantization (e.g., RDOQ) in the quantization decision results is greater than a preset probability threshold, and if at least one of the auxiliary quantization decision results has a probability of using the first quantization (e.g., RDOQ) greater than the preset probability threshold, the current transform unit is subjected to the first quantization (e.g., RDOQ); if the probability of using the first quantization (e.g., RDOQ) in all auxiliary quantization decision results is not greater than the preset probability threshold, the current transform unit is subjected to the second quantization (e.g., HDQ).

[0120] For ease of understanding, Figure 8 This shows another exemplary flowchart for quantifying decision-making.

[0121] The first step is to determine whether the RDOQ probability output by Model 1 (quantized decision model) is greater than the threshold 1;

[0122] The second step is to determine whether the RDOQ probability output by model 1 is greater than threshold 1 if the RDOQ probability output by model 2 (an auxiliary quantization decision model) is greater than threshold 2.

[0123] The third step is to determine whether the RDOQ probability output by model 2 is greater than threshold 2 if the RDOQ probability output by model 3 (another auxiliary quantification decision model) is greater than threshold 3.

[0124] Fourth step: If the RDOQ probability output by model 2 is greater than threshold 2, or the RDOQ probability output by model 3 is greater than threshold 3, select RDOQ for quantization.

[0125] Fifth step: If the RDOQ probability output by model 1 is not greater than threshold 1, or the RDOQ probability output by model 3 is not greater than threshold 3, select HDQ for quantization.

[0126] from Figure 6 It can be seen from this that, with Figure 5 Compared to the corresponding embodiments, the video encoding method in this embodiment adds an auxiliary quantization decision-making step to process 600. Therefore, the scheme described in this embodiment, based on an auxiliary quantization decision-making model, can improve decision accuracy. Decision-making based on machine learning models (such as the xgBoost model) offers superior performance compared to traditional methods. Furthermore, by selecting simple auxiliary decision-making models (such as SVM and LR models), using the output of the main model as the input of the auxiliary model, and combining the outputs of multiple models to synthesize the quantization algorithm, the quantization decision becomes more accurate, reducing performance loss and increasing speed.

[0127] For ease of understanding, Figure 9 An exemplary flowchart of video encoding is shown;

[0128] The first step is to calculate the residual of the current TU.

[0129] The second step is to perform transformation operations on the residual matrix;

[0130] The third step is to extract the features of the transformation matrix;

[0131] The fourth step is to input the corresponding features into Model 1, Model 2, and Model 3 to obtain the output results of Model 1, Model 2, and Model 3.

[0132] The fifth step is to comprehensively evaluate the merged results and determine whether to use RDOQ.

[0133] Step 6: If it is determined that RDOQ is to be used, then RDOQ is used to quantize the transformation matrix; if it is determined that RDOQ is not to be used, then HDQ is used to quantize the transformation matrix.

[0134] Step 7: Output the quantization matrix.

[0135] Further reference Figure 10 As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of a quantitative decision model training device, which is similar to... Figure 1 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0136] like Figure 10 As shown, the quantization decision model training device 1000 of this embodiment may include: a first acquisition module 1001, a second acquisition module 1002, and a first training module 1003. The first acquisition module 1001 is configured to acquire a set of sample video sequences, wherein the set of sample video sequences includes sample video sequences; the second acquisition module 1002 is configured to acquire the target feature set and quantization label of the current transform unit encoding the sample video sequences; the first training module 1003 is configured to use the target feature set as input and the quantization label as output to train the initial quantization decision model to obtain the quantization decision model.

[0137] In this embodiment, the specific processing of the first acquisition module 1001, the second acquisition module 1002, and the first training module 1003 in the quantization decision model training device 1000, and the resulting technical effects, can be referred to respectively. Figure 1 The relevant descriptions of steps 101-103 in the corresponding embodiments will not be repeated here.

[0138] In some optional implementations of this embodiment, the sample video sequence set includes multiple sample video sequences with different temporal and / or spatial complexities.

[0139] In some optional implementations of this embodiment, the second acquisition module 1002 includes: a first acquisition submodule configured to acquire a reference feature set of the current transformation unit; a second acquisition submodule configured to use the reference feature set as input and quantization labels as output to train an initial quantization decision model, and to acquire the importance of reference features in the reference feature set during model training; and a selection submodule configured to select reference features from the reference feature set based on importance to generate a target feature set.

[0140] In some optional implementations of this embodiment, the target feature set includes at least one of the following target features: the size, mean and variance of the current transform unit, the sum of absolute values, mean and variance of the transform matrix of the current transform unit, and the sum of absolute values, mean and variance of the transform matrix after convolution.

[0141] In some optional implementations of this embodiment, the second acquisition module 1002 includes: a transformation submodule configured to perform a transformation operation on the residual matrix of the current transformation unit to generate a transformation matrix of the current transformation unit; a quantization submodule configured to perform a first quantization and a second quantization on the transformation matrix to obtain a first quantization matrix and a second quantization matrix; and a comparison submodule configured to compare the difference between the first quantization matrix and the second quantization matrix to obtain a quantization label.

[0142] In some optional implementations of this embodiment, the comparison submodule includes: a subtraction unit configured to subtract the coefficients of the first quantization matrix from those of the second quantization matrix to obtain a difference matrix; a convolution unit configured to perform convolution on the difference matrix using a kernel corresponding to the size of the current transform unit to obtain a convolution result; and a comparison unit configured to compare the difference between the convolution result and a preset threshold to obtain a quantization label.

[0143] In some optional implementations of this embodiment, the comparison unit is further configured to: normalize the convolution result to obtain a normalized result; if the normalized result is less than a preset threshold, set the value of the quantization label to indicate that the second quantization is used; if the normalized result is not less than the preset threshold, set the value of the quantization label to indicate that the first quantization is used.

[0144] In some optional implementations of this embodiment, the quantization decision model training device 1000 further includes: a selection module configured to select at least one subset of target features from the target feature set; and a second training module configured to take the at least one subset of target features as input and the quantization label as output to train at least one auxiliary quantization decision model.

[0145] Further reference Figure 11As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of a video encoding apparatus, which is similar to... Figure 5 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0146] like Figure 11 As shown, the video encoding device 1100 of this embodiment may include: a first acquisition module 1101, a second acquisition module 1102, a first decision module 1103, and a first quantization module 1104. The first acquisition module 1101 is configured to acquire a video sequence; the second acquisition module 1102 is configured to acquire the target feature set of the current transform unit in encoding the video sequence; the first decision module 1103 is configured to input the target feature set into a quantization decision model to obtain the quantization decision result of the current transform unit, wherein the quantization decision model adopts... Figure 10 The device shown is trained; the first quantization module 1104 is configured to quantize the current transformation unit based on the quantization decision result.

[0147] In this embodiment, the specific processing of the first acquisition module 1101, the second acquisition module 1102, the first decision module 1103, and the first quantization module 1104 in the video encoding device 1100, and the resulting technical effects, can be found in reference to [reference needed]. Figure 5 The relevant descriptions of steps 501-504 in the corresponding embodiments will not be repeated here.

[0148] In some optional implementations of this embodiment, the video encoding apparatus 1100 further includes: a selection module configured to select at least one subset of target features from a set of target features; a second decision module configured to input the at least one subset of target features into at least one auxiliary quantization decision model to obtain at least one auxiliary quantization decision result of the current transform unit, wherein the at least one auxiliary quantization decision model is trained using the apparatus of claim 21; and a second quantization module configured to merge the quantization decision result and at least one auxiliary quantization decision result, and to quantize the current transform unit based on the merged result.

[0149] In some optional implementations of this embodiment, the second quantization module includes: a first quantization submodule configured to determine the quantization method of the current transformation unit based on at least one auxiliary quantization decision result and quantize the current transformation unit if the probability of adopting the first quantization in the quantization decision result is greater than a preset probability threshold; and a second quantization submodule configured to perform a second quantization on the current transformation unit if the probability of adopting the first quantization in the quantization decision result is not greater than a preset probability threshold.

[0150] In some optional implementations of this embodiment, the first quantization submodule is further configured to: perform first quantization on the current transformation unit if the probability of using the first quantization in all auxiliary quantization decision results is greater than a preset probability threshold; and perform second quantization on the current transformation unit if at least one of the probability of using the first quantization in all auxiliary quantization decision results is not greater than a preset probability threshold.

[0151] In some optional implementations of this embodiment, the first quantization submodule is further configured to: perform first quantization on the current transformation unit if at least one of the auxiliary quantization decision results has a probability of adopting first quantization greater than a preset probability threshold; and perform second quantization on the current transformation unit if the probability of adopting first quantization on all auxiliary quantization decision results is not greater than the preset probability threshold.

[0152] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0153] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0154] Figure 12 A schematic block diagram of an example electronic device 1200 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0155] like Figure 12 As shown, device 1200 includes a computing unit 1201, which can perform various appropriate actions and processes according to a computer program stored in read-only memory (ROM) 1202 or a computer program loaded from storage unit 1208 into random access memory (RAM) 1203. The RAM 1203 may also store various programs and data required for the operation of device 1200. The computing unit 1201, ROM 1202, and RAM 1203 are interconnected via bus 1204. Input / output (I / O) interface 1205 is also connected to bus 1204.

[0156] Multiple components in device 1200 are connected to I / O interface 1205, including: input unit 1206, such as keyboard, mouse, etc.; output unit 1207, such as various types of monitors, speakers, etc.; storage unit 1208, such as disk, optical disk, etc.; and communication unit 1209, such as network card, modem, wireless transceiver, etc. Communication unit 1209 allows device 1200 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0157] The computing unit 1201 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1201 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1201 performs the various methods and processes described above, such as quantization decision model training methods or video encoding methods. For example, in some embodiments, the quantization decision model training method or video encoding method can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 1208. In some embodiments, part or all of the computer program can be loaded and / or installed on device 1200 via ROM 1202 and / or communication unit 1209. When the computer program is loaded into RAM 1203 and executed by the computing unit 1201, one or more steps of the quantization decision model training method or video encoding method described above can be performed. Alternatively, in other embodiments, the computing unit 1201 may be configured by any other suitable means (e.g., by means of firmware) to perform a quantization decision model training method or a video coding method.

[0158] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0159] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0160] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0161] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0162] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0163] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, distributed system servers, or servers incorporating blockchain technology.

[0164] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution provided in this disclosure can be achieved, and this is not limited herein.

[0165] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for training a quantitative decision-making model, comprising: Obtain a set of sample video sequences, wherein the set of sample video sequences includes multiple sample video sequences with different temporal and / or spatial complexities; Obtain the target feature set and quantization label of the current transform unit in encoding the sample video sequence. The target feature set includes at least one of the following target features: the size, mean luminance, and variance of the current transform unit; the sum of absolute values, mean, and variance of the transform matrix of the current transform unit; and the sum of absolute values, mean, and variance of the transform matrix after convolution. The target feature set is used as input, and the quantization label is used as output to train the initial quantization decision model, thereby obtaining the quantization decision model.

2. The method according to claim 1, wherein, The step of obtaining the target feature set of the current transformation unit includes: Obtain the reference feature set of the current transformation unit; Using the reference feature set as input and the quantization label as output, the initial quantization decision model is trained, and the importance of the reference features in the reference feature set is obtained during the model training process. Reference features are selected from the reference feature set based on their importance to generate the target feature set.

3. The method according to claim 1, wherein, The step of obtaining the quantization label of the current transform unit includes: A transformation operation is performed on the residual matrix of the current transformation unit to generate the transformation matrix of the current transformation unit; The transformation matrix is ​​subjected to first quantization and second quantization respectively to obtain a first quantization matrix and a second quantization matrix; The quantization label is obtained by comparing the difference between the first quantization matrix and the second quantization matrix.

4. The method according to claim 3, wherein, The step of comparing the difference between the first quantization matrix and the second quantization matrix to obtain the quantization label includes: Subtract the coefficients of the first quantization matrix from those of the second quantization matrix to obtain the difference matrix; The difference matrix is ​​convolved using a kernel corresponding to the size of the current transformation unit to obtain the convolution result; The quantization label is obtained by comparing the difference between the convolution result and the preset threshold.

5. The method according to claim 4, wherein, The step of comparing the convolution result with a preset threshold to obtain the quantization label includes: The convolution result is normalized to obtain the normalized result; If the normalization result is less than the preset threshold, the value of the quantization label is set to indicate that the second quantization is used; If the normalization result is not less than the preset threshold, the value of the quantization label is set to indicate that the first quantization is used.

6. The method according to any one of claims 1-5, wherein, The method further includes: Select at least one subset of target features from the set of target features; At least one auxiliary quantization decision model is trained by taking the at least one subset of target features as input and the quantization label as output.

7. A video encoding method, comprising: Obtain the video sequence; Obtain the target feature set of the current transform unit in the encoding of the video sequence; The target feature set is input into the quantization decision model to obtain the quantization decision result of the current transformation unit, wherein the quantization decision model is trained using the method described in any one of claims 1-6; The current transformation unit is quantized based on the quantization decision result.

8. The method according to claim 7, wherein, The method further includes: Select at least one subset of target features from the set of target features; The at least one subset of target features is input into at least one auxiliary quantization decision model to obtain at least one auxiliary quantization decision result of the current transformation unit, wherein the at least one auxiliary quantization decision model is trained using the method described in claim 6; and The quantization of the current transformation unit based on the quantization decision result includes: The quantization decision result and the at least one auxiliary quantization decision result are combined, and the current transformation unit is quantized based on the combined result.

9. The method according to claim 8, wherein, The process of merging the quantization decision result and the at least one auxiliary quantization decision result, and quantizing the current transformation unit based on the merged result, includes: If the probability of using the first quantization in the quantization decision result is greater than a preset probability threshold, the quantization method of the current transformation unit is determined based on the at least one auxiliary quantization decision result, and the current transformation unit is quantized. If the probability of using the first quantization in the quantization decision result is not greater than the preset probability threshold, the current transformation unit is subjected to the second quantization.

10. The method according to claim 9, wherein, The step of determining the quantization mode of the current transformation unit based on the at least one auxiliary quantization decision result, and quantizing the current transformation unit, includes: If the probability of using the first quantization in all auxiliary quantization decision results is greater than the preset probability threshold, the current transformation unit is subjected to the first quantization. If at least one of the auxiliary quantization decision results has a probability of using the first quantization that is not greater than the preset probability threshold, then the current transformation unit is subjected to the second quantization.

11. The method according to claim 9, wherein, The step of determining the quantization mode of the current transformation unit based on the at least one auxiliary quantization decision result, and quantizing the current transformation unit, includes: If at least one of the auxiliary quantization decision results has a probability of using the first quantization greater than the preset probability threshold, the current transformation unit is subjected to the first quantization. If the probability of adopting the first quantization for all auxiliary quantization decision results is not greater than the preset probability threshold, the current transformation unit is subjected to the second quantization.

12. A quantitative decision-making model training device, comprising: The first acquisition module is configured to acquire a set of sample video sequences, wherein the set of sample video sequences includes multiple sample video sequences with different temporal and / or spatial complexities. The second acquisition module is configured to acquire the target feature set and quantization label of the current transform unit in encoding the sample video sequence. The target feature set includes at least one of the following target features: the size, mean luminance and variance of the current transform unit, the sum of absolute values, mean and variance of the transform matrix of the current transform unit, and the sum of absolute values, mean and variance of the transform matrix after convolution. The first training module is configured to take the target feature set as input and the quantization label as output to train the initial quantization decision model to obtain the quantization decision model.

13. A video encoding apparatus, comprising: The first acquisition module is configured to acquire video sequences; The second acquisition module is configured to acquire the target feature set of the current transform unit in encoding the video sequence; The first decision module is configured to input the target feature set into the quantization decision model to obtain the quantization decision result of the current transformation unit, wherein the quantization decision model is trained using the device of claim 12; The first quantization module is configured to quantize the current transformation unit based on the quantization decision result.

14. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the method of any one of claims 1-6 or any one of claims 7-11.

15. A non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1-6 or any one of claims 7-11.

16. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-6 or any one of claims 7-11.

Images