Video coding dynamic parameter optimization method and device based on deep learning

Abstract

The invention provides a video coding dynamic parameter optimization method and device based on deep learning, and the method comprises the steps: obtaining original video stream data, carrying out the framing of the original video stream data, carrying out the preprocessing and feature extraction according to each frame of data, inputting the extracted features into a pre-trained parameter prediction model, and carrying out the prediction of the parameters. The parameter prediction model outputs the adjustment parameter vector corresponding to the corresponding frame data for the extracted vector, and adjusts the parameters in the encoder according to the adjustment parameter vector, thereby achieving the dynamic adjustment and optimization of the parameters in the encoder, guaranteeing the real-time performance and adaptability of the optimization of the parameters of the encoder, and improving the precision of the encoder. And the video coding efficiency and quality are improved.

Description

Technical Field

[0001] This invention belongs to the field of video processing technology, and more specifically, relates to a method and apparatus for optimizing dynamic parameters of video coding based on deep learning. Background Technology

[0002] With the development and progress of the Internet age, the increase in network bandwidth, and the continuous improvement of user experience, ultra-high-definition video, virtual reality video, panoramic video, and intelligent surveillance video are developing rapidly, and the amount of video data is also growing rapidly, which poses greater challenges to video encoding technology.

[0003] Traditional video coding techniques rely on fixed rules or statistical models to adjust parameters, which has the following problems: (1) Coding efficiency bottleneck: Traditional methods have compression efficiency close to the theoretical limit when compressing high-resolution and high dynamic range videos, which is difficult to meet the transmission requirements of 5G and ultra-high-definition videos; (2) Insufficient real-time performance: In complex scenes (such as fast motion and complex textures), traditional algorithms need to perform multiple iterations of calculation, which leads to increased video latency; (3) Poor adaptability: Fixed parameter strategies are difficult to adapt to the coding requirements of different content complexities (such as alternating static and dynamic scenes), resulting in unreasonable bitrate allocation and loss of image details under high compression rates.

[0004] Therefore, overcoming the shortcomings of the existing technology is an urgent problem to be solved in this technical field. Summary of the Invention

[0005] The problem this invention aims to solve is how to improve the efficiency, adaptability, and real-time performance of video encoding.

[0006] Firstly, a deep learning-based method for optimizing dynamic parameters in video coding is provided, including: The original video stream data is acquired, preprocessed, and feature set is obtained by feature extraction based on the preprocessed original video stream data. The feature set is input into a pre-trained parameter prediction model, and the parameter prediction model outputs an adjustment parameter vector corresponding to the original video stream data. The encoder is adjusted according to the adjustment parameter vector to achieve dynamic optimization of the encoding parameters.

[0007] Preferably, the step of preprocessing the original video stream data and extracting features from the preprocessed original data stream data to obtain a feature set specifically includes: The original video stream data is divided into frames according to a preset frame rate, and each frame is scaled to a preset size to obtain preprocessed frame data. Each frame of data after preprocessing is converted to grayscale to obtain the space complexity features corresponding to the frame data. Obtain the motion vectors of all adjacent frame data, and obtain the time complexity features of the corresponding frame data based on the motion vectors; The static feature vector corresponding to each frame of data is extracted from each preprocessed frame of data. The space complexity features, time complexity features, and static feature vectors corresponding to the corresponding frame data are collectively used as the feature set corresponding to the corresponding frame data.

[0008] Preferably, the step of performing grayscale processing on each preprocessed frame of data to obtain the spatial complexity features corresponding to the frame data specifically includes: The preprocessed frame data is converted to grayscale to obtain the mean grayscale gradient of the corresponding frame data, and the mean grayscale gradient is used as the spatial complexity feature of the corresponding frame data. The expression for the mean gray-level gradient is: ; Where SI is the mean grayscale gradient and N is the total number of pixels. The gradient is in the horizontal direction. This represents the gradient in the vertical direction.

[0009] Preferably, the step of obtaining the motion vectors of all adjacent frame data and obtaining the time complexity features corresponding to the frame data based on the motion vectors specifically includes: The optical flow vector variance corresponding to the corresponding frame data is obtained from the motion vector of adjacent frame data, and the optical flow vector variance is used as the time complexity feature of the corresponding frame data. The expression for the variance of the optical flow vector is: ; Where TI is the variance of the optical flow vector, and M is the number of motion vectors. This is the k-th vector.

[0010] Preferably, the step of inputting the feature set into a pre-trained parameter prediction model, wherein the parameter prediction model outputs an adjustment parameter vector corresponding to the original video stream data, specifically includes: The spatial complexity features and the static feature vector are input into the LSTM layer of the parameter prediction model to obtain the spatial redundancy pattern; The time complexity features are input into the LSTM layer of the parameter prediction model to obtain motion continuity features; The spatial redundancy mode and the motion continuity feature are concatenated and input into the fully connected layer of the parameter prediction model to obtain the quantization parameters, frame rate adjustment coefficients, prediction spatial complexity features, and prediction time complexity features corresponding to the frame data. The prediction quantization parameters, frame rate adjustment coefficients, prediction spatial complexity features, and prediction time complexity features corresponding to the frame data are used together as the adjustment parameter vector corresponding to the frame data.

[0011] Preferably, the loss function used to train the parameter prediction model is as follows: ; Where L is the total loss value. and All are balanced hyperparameters. To predict the QP value, This is the actual QP value. To predict institutional similarity, For the similarity of actual institutions.

[0012] Preferably, adjusting the encoder according to the adjustment parameter vector specifically includes: The adjusted quantization parameters are obtained based on the prediction quantization parameters, prediction space complexity features, and prediction time complexity features corresponding to the corresponding frame data, and the adjusted quantization parameters are applied to the encoder. The adjusted bitrate is obtained based on the frame rate adjustment coefficient corresponding to the frame data, and the adjusted bitrate is applied to the encoder.

[0013] Preferably, the expression for adjusting the quantization parameter is: ; in, To adjust the quantization parameters, To predict quantization parameters, To predict time complexity features, To predict space complexity features; The expression for adjusting the bit rate is: ; in, To adjust the bitrate, The initial bitrate is α, and the frame rate adjustment factor is α. Available broadband, For the demand for broadband.

[0014] In a second aspect, a deep learning-based video coding dynamic parameter optimization apparatus is provided, comprising at least one processor and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the processor to perform the deep learning-based video coding dynamic parameter optimization method.

[0015] Thirdly, the present invention also provides a non-volatile computer storage medium storing computer-executable instructions that are executed by one or more processors to perform the method described in the first aspect.

[0016] Fourthly, a chip is provided, comprising: a processor and an interface for calling and running a computer program stored in memory, performing the method as described in the first aspect.

[0017] Fifthly, a computer program product containing instructions is provided that, when executed on a computer or processor, causes the computer or processor to perform the method as described in the first aspect.

[0018] In a sixth aspect, a deep learning-based video coding dynamic parameter optimization system is provided, including the deep learning-based video coding dynamic parameter optimization device as described in the second aspect, and using the deep learning-based video coding dynamic parameter optimization method as described in the first aspect.

[0019] Unlike existing technologies, the present invention has at least the following beneficial effects: The original video stream data is acquired and segmented into frames. Each frame is preprocessed and its features are extracted. The extracted features are then input into a pre-trained parameter prediction model. The parameter prediction model outputs an adjustment parameter vector corresponding to the extracted vector for each frame. The parameters in the encoder are adjusted based on the adjustment parameter vector, thus achieving dynamic adjustment and optimization of the encoder parameters. This ensures the real-time performance and adaptability of the encoder parameter optimization, improving the efficiency and quality of video encoding. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments of the present invention will be briefly described below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0021] Figure 1 This is a flowchart of a video coding dynamic parameter optimization method based on deep learning provided in an embodiment of the present invention; Figure 2 This is a flowchart of a method for obtaining the feature set of the original video stream data in a video coding dynamic parameter optimization method based on deep learning provided in an embodiment of the present invention; Figure 3 This is a flowchart of a method for obtaining the adjustment parameter vector in a deep learning-based video coding dynamic parameter optimization method provided by an embodiment of the present invention; Figure 4 This is a flowchart of an encoder adjustment method for a video coding dynamic parameter optimization method based on deep learning, provided in an embodiment of the present invention. Figure 5 This is a schematic diagram of a video coding dynamic parameter optimization device based on deep learning provided in an embodiment of the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0023] Unless the context otherwise requires, throughout the specification and claims, the term "comprising" is interpreted as openly inclusive, meaning "including, but not limited to." In the description of the specification, terms such as "one embodiment," "some embodiments," "exemplary embodiment," "example," "specific example," or "some examples" are intended to indicate that a particular feature, structure, material, or characteristic associated with that embodiment or example is included in at least one embodiment or example of this disclosure. The illustrative representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics mentioned may be included in any suitable manner in any one or more embodiments or examples; that is, although they may be incorporated into embodiments or examples using the above terms due to reasons such as order and position, it does not limit them to be incorporated in combination by a single embodiment or example.

[0024] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of this disclosure, unless otherwise stated, "a plurality of" means two or more. Furthermore, for example, the description may use the prefix "A" or "B" to describe the same type of nouns as two independent entities. In this case, the corresponding features defined with "A" and "B" are used only to distinguish between similar entities and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features.

[0025] In the description of this invention, the expression “A and / or B” (where A and B are used to formally represent specific features) will be used. The corresponding expression includes the following three combinations: only A, only B, and a combination of A and B.

[0026] As used in this invention, “about,” “approximately,” or “approximately” includes the stated value and the average value within an acceptable range of deviation from a particular value, wherein the acceptable range of deviation is determined by a person skilled in the art taking into account the measurement under discussion and the error associated with the measurement of the particular quantity (i.e., the limitations of the measurement system).

[0027] Furthermore, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0028] Example 1: This embodiment provides a deep learning-based method for optimizing dynamic parameters in video coding, such as... Figure 1 As shown, the method flow includes: In step 101, the original video stream data is obtained, the original video stream data is preprocessed, and a feature set is obtained by feature extraction based on the preprocessed original video stream data.

[0029] In this embodiment, the original video stream data can be an H.264 bitstream, an H.265 bitstream, or raw YUV data, and the original video stream data includes multiple frames of video data. The preprocessing involves dividing the original video stream data into multiple frames of video data and performing resolution unification processing on each frame of video data, which is used for subsequent feature extraction on each frame of data. The encoder is then optimized according to the features of each frame of data to improve the real-time performance and adaptability of video encoding parameter adjustment.

[0030] The feature extraction process involves obtaining feature vectors for each frame of the preprocessed original data stream, predicting encoder parameter adjustments based on the feature vectors extracted from each frame, obtaining relevant prediction data for encoder parameter adjustments, and using this data for subsequent encoder parameter adjustments.

[0031] In step 102, the feature set is input into a pre-trained parameter prediction model, and the parameter prediction model outputs the adjustment parameter vector corresponding to the original video stream data.

[0032] In this embodiment, the parameter prediction model is pre-trained. Based on the characteristics of each frame of data, the parameter prediction model obtains the prediction coding parameters corresponding to the corresponding frame data, namely the adjustment parameter vector, which includes quantization parameters and frame rate coefficients, etc., and is used to adjust and optimize the corresponding parameters of the encoder.

[0033] In step 103, the encoder is adjusted according to the adjustment parameter vector to achieve dynamic optimization of the encoding parameters.

[0034] Through the above design, the original video stream data is acquired and segmented into frames (e.g., 30 frames per second). Preprocessing and feature extraction are performed on each frame. The extracted features are input into a pre-trained parameter prediction model. The parameter prediction model outputs an adjustment parameter vector corresponding to the extracted vector for each frame. The parameters in the encoder are adjusted according to the adjustment parameter vector, realizing dynamic adjustment and optimization of the encoder parameters. This ensures the real-time performance and adaptability of the encoder parameter optimization, and improves the efficiency and quality of video encoding.

[0035] Furthermore, in this embodiment, since the acquired original video stream data contains multiple frames, in order to ensure the real-time performance and quality of subsequent optimization, it is necessary to perform individual feature extraction and parameter prediction on each frame of the original video stream data. Therefore, this embodiment requires preprocessing the acquired original video stream data, and the corresponding design is as follows: The original video stream data is preprocessed, and a feature set is obtained by feature extraction based on the preprocessed original video stream data, such as... Figure 2 As shown, the method flow includes the following.

[0036] In step 201, the original video stream data is divided into frames according to a preset frame rate, and each frame is scaled to a preset size to obtain preprocessed frame data.

[0037] In this embodiment, the preset frame rate is set by those skilled in the art based on actual conditions, and the preset size is set by those skilled in the art based on actual conditions.

[0038] In step 202, each frame of data after preprocessing is converted to grayscale to obtain the spatial complexity features corresponding to the frame data.

[0039] The preprocessed frame data is converted to grayscale to obtain the mean grayscale gradient of the corresponding frame data, and the mean grayscale gradient is used as the spatial complexity feature of the corresponding frame data. The expression for the mean gray-level gradient is: ; Where SI is the mean grayscale gradient and N is the total number of pixels. The gradient is in the horizontal direction. This represents the gradient in the vertical direction.

[0040] In step 203, the motion vectors of all adjacent frame data are obtained, and the time complexity features corresponding to the frame data are obtained based on the motion vectors.

[0041] The optical flow vector variance corresponding to the corresponding frame data is obtained from the motion vector of adjacent frame data, and the optical flow vector variance is used as the time complexity feature of the corresponding frame data. The expression for the variance of the optical flow vector is: ; Where TI is the variance of the optical flow vector, and M is the number of motion vectors. This is the k-th vector.

[0042] In step 204, the static feature vector corresponding to each frame of data is extracted based on the preprocessed frame data.

[0043] In this embodiment, each frame of preprocessed data can be input into a pre-trained ResNet-50 model to extract a 64-channel deep feature vector, i.e., the static feature vector.

[0044] In step 205, the space complexity feature, time complexity feature, and static feature vector corresponding to the corresponding frame data are used together as the feature set corresponding to the corresponding frame data.

[0045] Furthermore, in this embodiment, after obtaining the feature set corresponding to the corresponding frame data, the feature vector needs to be input into the parameter prediction model to output the adjustment parameter vector corresponding to the corresponding frame data for encoder parameter adjustment. The corresponding design is as follows: the feature set is input into the pre-trained parameter prediction model, and the parameter prediction model outputs the adjustment parameter vector corresponding to the original video stream data, such as... Figure 3 As shown, the method flow includes: In step 301, the spatial complexity feature and the static feature vector are input into the LSTM layer of the parameter prediction model to obtain the spatial redundancy pattern.

[0046] In step 302, the time complexity features are input into the LSTM layer of the parameter prediction model to obtain motion continuity features.

[0047] In step 303, the spatial redundancy mode and the motion continuity feature are concatenated and input into the fully connected layer of the parameter prediction model to obtain the quantization parameters, frame rate adjustment coefficients, prediction spatial complexity features and prediction time complexity features corresponding to the corresponding frame data. The prediction quantization parameters, frame rate adjustment coefficients, prediction spatial complexity features and prediction time complexity features corresponding to the corresponding frame data are used together as the adjustment parameter vector corresponding to the corresponding frame data.

[0048] In this embodiment, the frame rate adjustment coefficient can be 0.8 to 1.2. In addition to the quantization parameters, frame rate adjustment coefficient, prediction space complexity features and prediction time complexity features, the data output by the parameter prediction model also includes a reference frame number adjustment coefficient, which can be 1 to 4.

[0049] The vector obtained by concatenating the spatial redundancy pattern and the motion continuity feature is: ; in, For feature fusion vector, For spatial redundancy mode, For continuous motion features, Concat() is a concatenation operation.

[0050] The feature fusion vector is then subjected to a nonlinear transformation through a fully connected layer, expressed as: ; Where P is the original predicted value. W is the weight matrix of the fully connected layer, and b is the bias term. The activation function is, for example, ReLU or tanh.

[0051] In this embodiment, since the prediction quantization parameter, frame rate adjustment coefficient and reference frame number adjustment coefficient all have actual value range constraints, it is usually necessary to map the original prediction value to the target interval.

[0052] The prediction quantization parameters are: ,in For the truncation function, the range of the predicted quantization parameter is usually 0~51.

[0053] The frame rate adjustment coefficient is: The output constraint for the frame rate adjustment coefficient is 0.8~1.2; The reference frame number adjustment factor is: The reference frame number adjustment factor can be 1 to 4, obtained by rounding.

[0054] In addition, spatial redundancy mode This affects the predictive quantization parameter QP and the reference frame number adjustment factor β, as well as the motion continuity feature. This will affect the frame rate adjustment factor α and the reference frame number adjustment factor β.

[0055] Furthermore, in this embodiment, the parameter prediction model needs to be trained using a corresponding loss function. Therefore, in this embodiment, the loss function for the parameter prediction model involves the following design: The loss function used to train the parameter prediction model is as follows: ; Where L is the total loss value. and All are balanced hyperparameters. To predict the QP value, This is the actual QP value. To predict institutional similarity, For the similarity of actual institutions.

[0056] In this embodiment, during the training of the parameter prediction model, a smaller overall loss value indicates a more accurate prediction. Furthermore, during model training, random noise and motion blur are added to the training data to improve the model's robustness.

[0057] Furthermore, in this embodiment, after obtaining the adjustment parameter vector of the corresponding frame data, it is necessary to adjust the corresponding parameters in the encoder according to the adjustment parameter vector to achieve control of the encoder based on the corresponding frame data. Therefore, this embodiment also involves the following design: the adjustment of the encoder according to the adjustment parameter vector, such as... Figure 4 As shown, the method flow includes: In step 401, the adjusted quantization parameters are obtained based on the prediction quantization parameters, prediction space complexity features, and prediction time complexity features corresponding to the corresponding frame data, and the adjusted quantization parameters are applied to the encoder.

[0058] The expression for adjusting the quantization parameters is: ; in, To adjust the quantization parameters, To predict quantization parameters, To predict time complexity features, To predict space complexity features, the quantization parameters are adjusted. This is used directly in the quantization process of the encoder. The predicted quantization parameters... These are typically the initial quantization parameters of the encoder predicted by the parameter prediction model or the original quantization parameters corresponding to the current frame data.

[0059] In step 402, the adjusted bitrate is obtained according to the frame rate adjustment coefficient corresponding to the corresponding frame data, and the adjusted bitrate is applied to the encoder.

[0060] The expression for adjusting the bit rate is: ; in, To adjust the bitrate, The initial bitrate is α, and the frame rate adjustment factor is α. Available broadband, For the demand for broadband.

[0061] Furthermore, in this embodiment, by continuously monitoring the output data and feeding back the peak signal-to-noise ratio (PSNR) and structural similarity of each frame of output data, if the PSNR and structural similarity of the output data for a consecutive preset number of frames are lower than a preset threshold, an incremental learning algorithm can be used to update the weights of the parameter prediction model to avoid catastrophic forgetting. In this embodiment, the preset number of frames is set by those skilled in the art based on actual conditions, and the preset number of frames can be 5 frames.

[0062] It is worth mentioning that, in this embodiment, the encoder parameter adjustment includes dynamically setting quantization parameters, dynamically setting the number of reference frames, and dynamically setting the division level.

[0063] In this embodiment, the above parameter adjustment method can adaptively allocate bitrates for different scenes based on the actual video footage. Higher bitrates are allocated to static scenes, while lower bitrates are allocated to dynamic scenes.

[0064] Furthermore, the BD-rate and peak signal-to-noise ratio corresponding to each frame of output data are monitored and transmitted back to the encoder via a protocol. If the peak signal-to-noise ratio drops by more than 1dB for three consecutive frames, the corresponding model parameters are rolled back to the historical best version to ensure that the video encoding process proceeds normally.

[0065] The above design typically reduces the bit rate by 28.3%, increases the peak signal-to-noise ratio by 1.5dB, and reduces latency to 7.2ms.

[0066] Through the above design, this embodiment can achieve the following beneficial effects: (1) High-efficiency compression: By adjusting the dynamic parameters, redundant information is reduced, resulting in a 15%-30% reduction in bit rate.

[0067] (2) Image quality improvement: Reduces block artifacts and blur at low bitrates, resulting in a significant improvement in peak signal-to-noise ratio.

[0068] (3) Low latency: A lightweight model is used to adapt to edge computing devices to meet real-time requirements.

[0069] (4) Wide applicability: It is compatible with existing encoding standards and can be integrated into terminals such as cameras and streaming media servers.

[0070] Example 2: Based on the deep learning-based video coding dynamic parameter optimization method provided in the foregoing embodiments, the present invention also provides a deep learning-based video coding dynamic parameter optimization apparatus that can be used to implement the above method, such as... Figure 5 The diagram shown is a schematic representation of the device architecture according to an embodiment of the present invention. The device for optimizing dynamic parameters of video coding based on deep learning in this embodiment includes one or more processors 41 and a memory 42. Figure 5 Take a processor 41 as an example.

[0071] Processor 41 and memory 42 can be connected via a bus or other means. Figure 5 Taking the example of a connection between China and Israel via a bus.

[0072] The memory 42, as a non-volatile computer-readable storage medium for a deep learning-based video coding dynamic parameter optimization method, can be used to store non-volatile software programs and non-volatile computer-executable programs, such as the deep learning-based video coding dynamic parameter optimization method in the foregoing embodiments. The processor 41 executes various functional applications and data processing of the deep learning-based video coding dynamic parameter optimization device by running the non-volatile software programs, instructions, and modules stored in the memory 42, thereby implementing the deep learning-based video coding dynamic parameter optimization method of the foregoing embodiments.

[0073] Memory 42 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, memory 42 may include memory remotely located relative to processor 41, which can be connected to processor 41 via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0074] The program instructions / modules are stored in memory 42. When executed by one or more processors 41, they perform the deep learning-based video coding dynamic parameter optimization method described in the foregoing embodiments, for example, performing the steps shown in the figures above.

[0075] This invention also provides a non-volatile computer storage medium storing computer-executable instructions that are executed by one or more processors, for example... Figure 5 One of the processors 41 can enable the one or more processors to execute the deep learning-based video coding dynamic parameter optimization method in the foregoing embodiments, for example, to perform the steps shown in the figures described above.

[0076] It is worth noting that the information interaction and execution process between the modules and units in the above-mentioned device and system are based on the same concept as the processing method embodiment of the present invention. For details, please refer to the description in the method embodiment of the present invention, and will not be repeated here.

[0077] Those skilled in the art will understand that all or part of the steps in the various methods of the embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0078] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for optimizing dynamic parameters of video coding based on deep learning, characterized in that, include: The original video stream data is acquired, preprocessed, and feature set is obtained by feature extraction based on the preprocessed original video stream data. The feature set is input into a pre-trained parameter prediction model, and the parameter prediction model outputs an adjustment parameter vector corresponding to the original video stream data. The encoder is adjusted according to the adjustment parameter vector to achieve dynamic optimization of the encoding parameters.

2. The video coding dynamic parameter optimization method based on deep learning according to claim 1, characterized in that, The preprocessing of the original video stream data and the extraction of features from the preprocessed original data stream data to obtain a feature set specifically include: The original video stream data is divided into frames according to a preset frame rate, and each frame is scaled to a preset size to obtain preprocessed frame data. Each frame of data after preprocessing is converted to grayscale to obtain the space complexity features corresponding to the frame data. Obtain the motion vectors of all adjacent frame data, and obtain the time complexity features of the corresponding frame data based on the motion vectors; The static feature vector corresponding to each frame of data is extracted from each preprocessed frame of data. The space complexity features, time complexity features, and static feature vectors corresponding to the corresponding frame data are collectively used as the feature set corresponding to the corresponding frame data.

3. The video coding dynamic parameter optimization method based on deep learning according to claim 2, characterized in that, The step of performing grayscale processing on each preprocessed frame of data to obtain the space complexity features corresponding to the frame data specifically includes: The preprocessed frame data is converted to grayscale to obtain the mean grayscale gradient of the corresponding frame data, and the mean grayscale gradient is used as the spatial complexity feature of the corresponding frame data. The expression for the mean gray-level gradient is: ； Where SI is the mean grayscale gradient and N is the total number of pixels. The gradient is in the horizontal direction. This represents the gradient in the vertical direction.

4. The video coding dynamic parameter optimization method based on deep learning according to claim 2, characterized in that, The process of acquiring motion vectors from all adjacent frame data and obtaining the time complexity features corresponding to the respective frame data based on the motion vectors specifically includes: The optical flow vector variance corresponding to the corresponding frame data is obtained from the motion vector of adjacent frame data, and the optical flow vector variance is used as the time complexity feature of the corresponding frame data. The expression for the variance of the optical flow vector is: ； Where TI is the variance of the optical flow vector, and M is the number of motion vectors. This is the k-th vector.

5. The video coding dynamic parameter optimization method based on deep learning according to claim 1, characterized in that, The step of inputting the feature set into a pre-trained parameter prediction model, and the parameter prediction model outputting an adjustment parameter vector corresponding to the original video stream data, specifically includes: The spatial complexity features and the static feature vector are input into the LSTM layer of the parameter prediction model to obtain the spatial redundancy pattern; The time complexity features are input into the LSTM layer of the parameter prediction model to obtain motion continuity features; The spatial redundancy mode and the motion continuity feature are concatenated and input into the fully connected layer of the parameter prediction model to obtain the quantization parameters, frame rate adjustment coefficients, prediction spatial complexity features, and prediction time complexity features corresponding to the frame data. The prediction quantization parameters, frame rate adjustment coefficients, prediction spatial complexity features, and prediction time complexity features corresponding to the frame data are used together as the adjustment parameter vector corresponding to the frame data.

6. The video coding dynamic parameter optimization method based on deep learning according to claim 1, characterized in that, The loss function used to train the parameter prediction model is as follows: ； Where L is the total loss value. and All are balanced hyperparameters. To predict the QP value, This is the actual QP value. To predict institutional similarity, For the similarity of actual institutions.

7. The video coding dynamic parameter optimization method based on deep learning according to claim 1, characterized in that, The adjustment of the encoder based on the adjustment parameter vector specifically includes: The adjusted quantization parameters are obtained based on the prediction quantization parameters, prediction space complexity features, and prediction time complexity features corresponding to the corresponding frame data, and the adjusted quantization parameters are applied to the encoder. The adjusted bitrate is obtained based on the frame rate adjustment coefficient corresponding to the frame data, and the adjusted bitrate is applied to the encoder.

8. The video coding dynamic parameter optimization method based on deep learning according to claim 1, characterized in that, The expression for adjusting the quantization parameters is: ； in, To adjust the quantization parameters, To predict quantization parameters, To predict time complexity features, To predict space complexity features; The expression for adjusting the bit rate is: ； in, To adjust the bitrate, The initial bitrate is α, and the frame rate adjustment factor is α. Available broadband, For the demand for broadband.

9. A video coding dynamic parameter optimization device based on deep learning, characterized in that, The method includes at least one processor and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the processor for performing the deep learning-based video coding dynamic parameter optimization method according to any one of claims 1-8.

10. A non-volatile computer storage medium, characterized in that, The computer storage medium stores computer program instructions that, when executed by one or more processors, implement the deep learning-based video coding dynamic parameter optimization method as described in any one of claims 1-8.

Images