Video encoding method, apparatus, electronic device, and storage medium

Abstract

Embodiments of the present disclosure provide a video encoding method, apparatus, electronic device, and storage medium, extract video encoding features of initial video data by a hardware video editor, where the video encoding features characterize the complexity of the video content of the initial video data, process the video encoding features of the initial video data by a pre-trained prediction neural network model, obtain a target quality rank corresponding to the initial video data, where the target quality rank characterizes the image quality level when encoding the initial video in a constant quality variable bit rate mode, encode the initial video data with the target quality rank using a hardware video editor to generate a target video. The bit rate of the generated target video is adapted to the video content, avoiding the problem that the bit rate is too high or too low, improving the image quality of the video, and avoiding waste of the bit rate.

Description

Technical Field

[0001] This application was filed on December 7, 2022, claims the priority of a Chinese patent application with the title "Video Encoding Method, Apparatus, Electronic Device and Storage Medium" and the application number 202211567316.X, and the entire content thereof is incorporated herein by reference.

[0002] Embodiments of the present disclosure relate to the field of image processing technology, and in particular, to video encoding methods, apparatuses, electronic devices, and storage media.

Background Art

[0003] A graphics processing unit (GPU) is a processor that plays a role in executing image processing tasks in terminal devices such as mobile phones and personal computers. The graphics processing unit has powerful digital computing capabilities and parallel processing capabilities. In application scenarios such as video encoding and decoding, by using the graphics processing unit, the quality and efficiency of image encoding and decoding can be effectively improved.

[0004] Some graphics processing units include a hardware-based video encoder, also called a hardware video editor, such as an Nvenc unit. The hardware video editor can encode data in YUV / RGB format into a video conforming to the H.264 / HEVC standard, realizing efficient video encoding.

[0005] In actual applications, when calling a hardware video editor to encode a video, it is necessary to set parameters characterizing the quality rank. However, in the prior art, usually, encoding is performed using a quality rank set fixedly based on experience, resulting in the problem that the bit rate of the encoded video is unreasonable.

Summary of the Invention

[0006] Embodiments of this disclosure provide video encoding methods, apparatus, electronic devices, and storage media for overcoming the problem of unreasonable bitrates in encoded video caused by encoding using a fixed quality rank.

[0007] In a first embodiment, an embodiment of the present disclosure provides a video encoding method, the method is Extracting video coding features of initial video data using a hardware video editor, wherein the video coding features characterize the complexity of the video content of the initial video data; processing the video coding features of the initial video data using a pre-trained predictive neural network model to obtain a target quality rank corresponding to the initial video data, wherein the target quality rank characterizes the image quality level when the initial video is encoded in constant quality dynamic bitrate mode; and encoding the initial video data with the target quality rank using the hardware video editor to generate a target video.

[0008] In a second embodiment, an embodiment of the present disclosure provides a video encoding apparatus, which comprises A parameter acquisition module for extracting video encoding features of initial video data using a hardware video editing machine, wherein the video encoding features characterize the complexity of the video content of the initial video data, A parameter optimization module for processing the video encoding features of the initial video data using a pre-trained predictive neural network model and obtaining a target quality rank corresponding to the initial video data, wherein the target quality rank is a parameter optimization module that characterizes the image quality level when the initial video is encoded in a constant quality dynamic bitrate mode. The system includes an encoding module for encoding the initial video data at the target quality rank using the aforementioned hardware video editing machine and generating a target video.

[0009] In a third embodiment, the embodiments of the present disclosure provide an electronic device, which is The system comprises a processor and a memory that is communicably connected to the processor, The aforementioned memory stores computer execution instructions, The processor executes computer execution instructions stored in the memory to perform the video encoding method described in the first embodiment and various possible designs of the first embodiment.

[0010] In a fourth embodiment, an embodiment of the present disclosure provides a computer-readable storage medium in which computer execution instructions are stored, and when a processor executes the computer execution instructions, it performs the video encoding method described in the first embodiment and various possible designs of the first embodiment.

[0011] In a fifth embodiment, an embodiment of the present disclosure provides a computer program product that, when executed by a processor, includes a computer program that performs the video encoding method described in the first embodiment and various possible designs of the first embodiment.

[0012] The video encoding method, apparatus, electronic device, and storage medium provided by this embodiment extract video encoding features of initial video data using a hardware video editor, the video encoding features characterize the complexity of the video content of the initial video data, the video encoding features of the initial video data are processed by a pre-trained predictive neural network model to obtain a target quality rank corresponding to the initial video data, the target quality rank characterizes the image quality level when encoding the initial video in constant quality dynamic bitrate mode, and the initial video data is encoded with the target quality rank using the hardware video editor to generate a target video. By utilizing the characteristic of the hardware video editor to conveniently output encoding parameters to obtain video encoding features corresponding to the initial video data, then obtaining a target image quality level that matches the video encoding features using a pre-trained predictive neural network model, and further encoding the video with the target quality rank using the hardware video editor, the bitrate of the generated target video is adapted to the video content, problems of the bitrate being too high or too low are avoided, the image quality of the video is improved, and bitrate is avoided. [Brief explanation of the drawing]

[0013] To more clearly illustrate the embodiments of this disclosure or the technical proposals in the prior art, the following is a brief introduction to the accompanying drawings that may be used in the description of the embodiments or the prior art. The accompanying drawings in the following description are some embodiments of this disclosure, and it is obvious to an ordinary person of the art that other drawings can be obtained by following these accompanying drawings without expending any creative effort.

[0014] [Figure 1] This is a schematic diagram of an application scenario provided by the embodiments of this disclosure. [Figure 2] This is a schematic flowchart 1 of a video encoding method provided by an embodiment of the present disclosure. [Figure 3]Figure 2 is a flowchart showing the specific steps for implementing step S101 in the embodiment shown. [Figure 4] Figure 3 is a flowchart showing the specific steps for implementing step S1013 in the embodiment shown. [Figure 5] This is a schematic diagram of a process for generating video coding features provided by embodiments of the present disclosure. [Figure 6] This is a schematic diagram of the data structure of the video coding features provided by the embodiments of this disclosure. [Figure 7] This is a schematic flowchart 2 of a video encoding method provided by an embodiment of the present disclosure. [Figure 8] This is a schematic diagram of the process for generating an evaluation value corresponding to the current quality rank of the current frame, as provided by the embodiments of this disclosure. [Figure 9] This is a schematic flowchart of the steps involved in training a predictive neural network model. [Figure 10] This is a schematic diagram of the configuration of a video encoding device provided by an embodiment of the present disclosure. [Figure 11] This is a schematic diagram of the configuration of an electronic device provided by the embodiments of this disclosure. [Figure 12] This is a schematic diagram of the hardware configuration of an electronic device provided by an embodiment of the present disclosure. [Modes for carrying out the invention]

[0015] To further clarify the purpose, technical proposals, and advantages of the embodiments of this disclosure, the technical proposals of the embodiments of this disclosure will be described clearly and completely below, together with the accompanying drawings of the embodiments of this disclosure. However, it is clear that the embodiments described are only a portion of the embodiments of this disclosure, not all of them. All other embodiments that a person skilled in the art could obtain based on the embodiments of this disclosure without expending any creative effort are included within the scope of this disclosure.

[0016] The application scenarios of the embodiments of this disclosure are described below.

[0017] The video encoding method provided by the embodiments of the present disclosure can be applied to various application scenarios that require video encoding, such as video editing, preview, playback, etc. More specifically, for example, it is applied to video editing software, video editing cloud platforms, and live streaming software. Exemplarily, the method provided by the embodiments of the present disclosure may be applied to terminal devices such as smartphones, tablet computers, and personal computers, or may be applied to cloud servers. FIG. 1 is a schematic diagram of an application scenario provided by the embodiments of the present disclosure. Taking the application scenario of running video editing software on the terminal device side as an example, as shown in FIG. 1, specifically, the terminal device runs the video editing software, performs video editing on the original video, for example, adds video effects, adds audio tracks, adds subtitles, and then generates video data, which may include a plurality of video frames and editing information corresponding to each of the frames. Then, the terminal device calls the hardware video editor in the graphics processing unit to encode the video data, generates a target video (completed video) with editing effects for playback, and completes the workflow of video editing. Here, the hardware video editor is, for example, an Nvenc unit.

[0018] In the prior art, when calling a hardware video editor to process video data, it is necessary to set corresponding parameters to control a specific encoding method. Among them, variable bit rate (VBR) encoding, also called dynamic bit rate encoding or non-fixed bit rate encoding, is a commonly used encoding method. Since it can dynamically adjust the bit rate according to the content of the video, the bit rate of the encoded video can change according to the complexity of the image. Therefore, its encoding efficiency is relatively high, the capacity of the video in a static picture is compressed, there are fewer mosaics in a dynamic moving picture, and the capacity and quality of the video are balanced. Among these, constant quality (CQ) variable bit rate (CQ-VBR) is an encoding method based on variable bit rate. By using the quality rank to control the image quality (bit rate) in the process of variable bit rate encoding, more precise control of the capacity and quality of the video generated after encoding is realized, further improving the flexibility and practicality of video encoding control.

[0019] However, in the actual application process, when calling a hardware video editor to encode using the constant quality variable bit rate (CQ-VBR) mode, the setting of the quality rank (cq value) is usually determined based on the user's experience. As a result, the quality rank is often set inappropriately, leading to problems such as the video bit rate being too high (the video capacity becomes too large, wasting storage and network resources) or too low (the video image quality is low, affecting the viewing experience of the video).

[0020] Embodiments of the present disclosure provide a video encoding method that solves the above problems by automatically generating a reasonable target quality rank (cq value) and performing video encoding based on this target quality rank.

[0021] Referring to Figure 2, Figure 2 is a schematic flowchart 1 of a video encoding method provided by an embodiment of the present disclosure. The method of this embodiment is applicable to electronic devices equipped with hardware video editing equipment, such as terminal devices and servers. In this embodiment, a terminal device is introduced as the implementing body, and exemplarily, this video encoding method includes the following steps.

[0022] Step S101: A hardware video editor extracts video encoding features from the initial video data, which characterize the complexity of the video content in the initial video data.

[0023] For example, after step S101, the terminal device first acquires initial video data, which may be data generated based on a video editing application or a live streaming application, and may include, for example, video data in YUV / RGB format, as well as data such as video effects, added audio tracks, and subtitles. The initial video data is the data to be encoded, and after encoding the initial video data, a corresponding playable video can be generated.

[0024] Subsequently, the initial video data is processed by calling a hardware video editor. Specifically, for example, the initial video data is used as input parameters, the application interface of the hardware video editor is called, and the corresponding processing function is executed to process this initial video data and obtain video coding features corresponding to this initial video data. Here, the video coding features characterize the complexity of the video content of the initial video data by the video parameters of the initial video data and the coding parameters corresponding to the video parameters. Specifically, in one possible implementation, the video coding features are processed based on the video parameters and coding parameters, where the video parameters are information that characterizes this initial video data, such as the video height, width (i.e., resolution), frame rate (fps), etc. The coding parameters characterize the parameters used when the hardware video editor encodes the initial video data, such as the frame coding size (size), the type of each frame corresponding to the video data (including I-frames, P-frames, and B-frames), the degree of image distortion, the degree of image detail, etc.

[0025] Furthermore, the video parameters corresponding to the video coding features are obtained directly based on the video information of the initial video data, and their explanation will not be repeated. The video parameters corresponding to the video coding features are obtained by taking the video parameters as input and calling the interface provided by the hardware video editor for generating the coding parameters; therefore, the coding parameters have a correspondence with the initial video data. In one possible implementation, as shown in Figure 3, the specific implementation steps of step S101 include the following steps.

[0026] Step S1011: Obtain the initial video data quality rank selection.

[0027] Exemplary, based on the previous introduction of application scenarios and prior art, when encoding using a hardware video editor with a constant quality dynamic bitrate mode, it is necessary to set a quality rank, and the quality rank option may be a predetermined default value, or more specifically, a value within the interval [18,35], for example, 25. The smaller the quality rank, the higher the image quality level of the video generated after encoding, the sharper the video, and relatively, the larger the video file size.

[0028] Step S1012: Input the video parameters and quality rank selection of the initial video data into the hardware video editor to obtain the encoding parameters of the initial video data.

[0029] Furthermore, by calling the interface provided by the hardware video editor, the video parameters and quality rank selection of the initial video data are input to the hardware video editor, and the encoding parameters output by the hardware video editor, such as the frame encoding size (size), the type of each frame corresponding to the video data (including I-frames, P-frames, and B-frames), the degree of image distortion, and the image resolution, are obtained. Here, as an example, the degree of image distortion may be represented by the Sum of Absolute Transformed Difference (SATD) of each frame, and the image resolution may be represented by the Quantizer Parameter (QP) of each frame. The specific calculation processes for the Sum of Absolute Transformed Difference and the Quantizer Parameter are existing techniques that can be implemented by functions provided by the hardware video editor's driver, and will not be explained again here.

[0030] Step S1013: Generate corresponding video coding features based on coding parameters.

[0031] For example, the encoding parameters obtained by inputting video parameters and quality rank options into a hardware video editor correspond to the pre-encoding of the initial video data by the hardware video editor; that is, the hardware video editor predicts the corresponding encoding parameters based on the initial video data, but does not actually perform the encoding. Subsequently, based on the encoding parameters, it combines the video parameters and quality rank options to generate video encoding features. These video encoding features can represent the complexity of the video content of the initial video data, and subsequently, based on these video encoding features, a corresponding quality rank can be determined for encoding. This achieves the objective of matching the image quality to the complexity of the video content and avoids problems of wasted or too low bitrate.

[0032] Furthermore, in one possible implementation, the initial video data includes multiple video frames, and as shown in Figure 4, the specific implementation step of step S1013 is: Step S1013A: Obtain the encoding parameters corresponding to each video frame, Step S1013B: Based on the encoding parameters corresponding to each video frame, obtain the encoding feature mean and encoding feature variance, where the encoding feature mean is the mean of the encoding parameters corresponding to each video frame, and the encoding feature variance is the variance of the encoding parameters corresponding to each video frame. Step S1013C includes generating video coding features based on video parameters, quality rank selections, and the mean and variance values ​​of coding features corresponding to each video frame.

[0033] Exemplary, initial video data includes multiple video frames. By obtaining the video parameters and encoding parameters corresponding to each video frame in the initial video data, frame encoding features corresponding to the video frames are obtained. Furthermore, the complexity of the video content of the initial video data is determined based on the average level and variation between the multiple frame encoding features, and video encoding features of the initial video data are obtained. Figure 5 is a schematic diagram of the process for generating video encoding features provided by the embodiment of this disclosure, and the above process will be described below in conjunction with Figure 5. As shown in Figure 5, the initial video data includes N video frames, where N is an integer greater than 1. Here, taking the M-th frame as an example (where M is an integer less than N and greater than 1), first, the video parameters of the M-th frame, for example, the height, width, and frame rate of the M-th frame are obtained. These video parameters and encoding level selections are input to the interface of a hardware video editor, and the encoding parameters of the M-th frame, for example, the type (shown as Type), frame encoding size (shown as Size), absolute error sum (shown as SATD), and quantization parameter (shown as QP) of the M-th frame are obtained using the hardware video editor. Subsequently, the mean and variance of the coding parameters corresponding to the Mth frame are calculated to obtain the coding feature mean and coding feature variance. Specifically, the mean and variance of the coding parameters corresponding to the Mth frame are the mean and variance of the coding parameters of each video frame in the set formed by at least one adjacent video frame prior to the Mth frame (shown as the Lth frame in the figure, where L is a smaller integer than M and greater than or equal to 1) and the Mth frame. The specific calculation process for calculating the corresponding mean and variance for a specific implementation of the coding parameters, and for obtaining the coding feature mean and coding feature variance, is not repeated.Subsequently, based on a combination of video parameters, quality rank options, encoded feature mean, and encoded feature variance, a video encoded feature corresponding to the Mth frame is obtained, where the video encoded feature indicates the complexity of the video content of the video segment corresponding to frames L through M in the initial video data, where L=1 is possible, and the video encoded feature indicates the complexity of the video content of the video segment prior to the Mth frame in the initial video data.

[0034] Furthermore, Figure 6 is a schematic diagram of the data structure of a video coding feature provided by an embodiment of the present disclosure. Referring to Figures 5 and 6, the video coding feature includes a total of 21 data fields, which are shown in the figures as fields #1 to #21.

[0035] Field #1 indicates the height of the images from frame 1 to frame M in the initial video data.

[0036] Field #2 indicates the width of the images from frame 1 to frame M in the initial video data.

[0037] Field #3 shows the frame rate from frame 1 to frame M in the initial video data.

[0038] Field #4 shows the quality rank options from frame 1 to frame M in the initial video data.

[0039] Field #5 indicates the number of I frames from frame 1 to frame M in the initial video data.

[0040] Field #6 indicates the number of P-frames from frame 1 to frame M in the initial video data.

[0041] Field #7 indicates the number of B frames from frame 1 to frame M in the initial video data.

[0042] Field #8 shows the average size of I-frames from frame 1 to frame M in the initial video data.

[0043] Field #9 shows the average size of P-frames from frame 1 to frame M in the initial video data.

[0044] Field #10 shows the average size of B-frames from frame 1 to frame M in the initial video data.

[0045] Field #11 shows the mean absolute error sum of I frames from frame 1 to frame M in the initial video data.

[0046] Field #12 shows the mean absolute error sum of P frames from frame 1 to frame M in the initial video data.

[0047] Field #13 shows the mean absolute error sum of B frames from frame 1 to frame M in the initial video data.

[0048] Field #14 shows the average quantization parameters from frame 1 to frame M in the initial video data.

[0049] Field #15 shows the size distribution of I-frames from frame 1 to frame M in the initial video data.

[0050] Field #16 shows the size distribution of P frames from frame 1 to frame M in the initial video data.

[0051] Field #17 shows the size distribution of B frames from frame 1 to frame M in the initial video data.

[0052] Field #18 shows the mean absolute error sum variance of I frames from frame 1 to frame M in the initial video data.

[0053] Field #19 shows the mean absolute error sum variance of P frames from frame 1 to frame M in the initial video data.

[0054] Field #20 shows the mean absolute error sum variance of B frames from frame 1 to frame M in the initial video data.

[0055] Field #21 shows the quantization parameter variance from frame 1 to frame M in the initial video data.

[0056] Step S102: A pre-trained predictive neural network model processes the video encoding features of the initial video data to obtain a target quality rank corresponding to the initial video data, the target quality rank characterizes the image quality level when the initial video is encoded in constant quality dynamic bitrate mode.

[0057] For example, after obtaining the video encoding features of the initial video data, it is necessary to determine a suitable target quality rank for these video encoding features. Specifically, the quality rank, or CQ value, is one of the parameters that characterize the image quality level when encoding the initial video in constant quality dynamic bitrate (CQ-VBR) mode and needs to be used when calling a hardware video editor. In the steps of this embodiment, the video encoding features of the initial video data are processed by a pre-trained predictive neural network model, and a quality rank, or target quality rank, that matches the complexity of the video content it characterizes is predicted.

[0058] Step S103: Use a hardware video editor to encode the initial video data at the target quality rank and generate the target video.

[0059] For example, after obtaining the target quality rank, a hardware video editor can be called in to process the initial video data using the target quality rank as a parameter, and a corresponding playback-ready video, i.e., the target video, can be generated.

[0060] In one possible implementation, the target quality rank may be a level sequence containing multiple level identifiers that characterize specific quality ranks (i.e., CQ values). Here, each level identifier in the level sequence corresponds to one or more video frames in the initial video data, and in a more specific possible implementation, each level identifier corresponds to one video frame. When encoding the initial video data with the target quality rank using a hardware video editor, each video frame in the initial video data is acquired sequentially (in parallel or serially), and based on the level identifier corresponding to each video frame, the hardware video editor is invoked to encode the corresponding initial video data at a constant quality dynamic bitrate. This results in each encoded video frame having a different image quality level, achieving more accurate encoding and improving encoding efficiency. Of these, the specific implementation process of invoking a hardware video editor to encode at a constant quality dynamic bitrate is prior art and will not be repeated here.

[0061] In this embodiment, a hardware video editor is used to extract video encoding features from the initial video data, which characterize the complexity of the video content of the initial video data. A pre-trained predictive neural network model processes the video encoding features of the initial video data to obtain a target quality rank corresponding to the initial video data. The target quality rank characterizes the image quality level when encoding the initial video in constant quality dynamic bitrate mode. The hardware video editor is then used to encode the initial video data with the target quality rank to generate the target video. By utilizing the hardware video editor's ability to conveniently output encoding parameters, video encoding features corresponding to the initial video data are obtained. Subsequently, a pre-trained predictive neural network model is used to obtain a target image quality level that matches the video encoding features. Furthermore, by using the hardware video editor to encode the video with the target quality rank, the bitrate of the generated target video is adapted to the video content, avoiding problems with excessively high or low bitrates, improving video image quality, and avoiding wasted bitrate.

[0062] Referring to Figure 7, Figure 7 is a schematic flowchart 2 of the video encoding method provided by an embodiment of the present disclosure. This embodiment further subdivides the implementation process of steps S101 and S102 of the embodiment shown in Figure 2, and this video encoding method includes the following steps.

[0063] Step S201: Obtain a quality rank sequence that characterizes the quality rank options, where the quality rank sequence is a set of multiple quality ranks arranged in an ordered manner.

[0064] For illustrative purposes, referring to the introduction to quality rank options in the embodiment shown in Figure 2, in this embodiment, there are multiple quality rank options, and these multiple quality rank options are characterized by a pre-defined quality rank sequence, that is, the quality rank sequence is one form of implementation of the quality rank options. Specifically, for example, the quality rank sequence is cq_data=[15:40], that is, an ordered arrangement of multiple quality ranks including quality rank 15 to quality rank 40, and the multiple quality ranks from quality rank 15 to quality rank 40 arranged in the above ordered arrangement are the quality rank options. It is understood that the quality rank sequence can be implemented in various ways, such as an enumerated data structure such as an array, matrix, key-value pairs, or structure, or a set of numbers expressed as a function, so no examples are given here.

[0065] Step S202: Get the current frame of the initial video data.

[0066] Step S203: Get the video parameters of the current frame and the current quality rank in the quality rank sequence.

[0067] Step S204: Input the video parameters and quality rank of the current frame into the hardware video editor to obtain the encoding parameters corresponding to the current quality rank of the current frame.

[0068] As an example, steps S202 and S203 respectively cycle through two dimensions (video frame dimension and quality rank dimension) to obtain a matching target quality rank corresponding to each video frame in the initial video parameters. Therefore, by encoding each frame based on the target quality rank corresponding to each video frame, dynamic encoding of each frame within the video is achieved, improving encoding efficiency and encoding quality.

[0069] Specifically, video parameters can be directly obtained for each current frame, starting from the first frame of the initial video data and continuing to the last frame of the initial video data; this explanation will not be repeated here. Subsequently, based on the sequential input of multiple quality ranks and video parameters in the quality data selection to the hardware video editor, encoding parameters corresponding to the current quality rank of the current frame can be obtained. Of these, the encoding parameters for the first frame of the initial video data can be obtained by calling the corresponding interface of the hardware video editor in combination with the video parameters based on a predetermined default quality rank; the specific implementation details have been explained in the above example and will not be repeated here.

[0070] Step S205: Generate a first feature corresponding to the current quality rank of the current frame from the video parameters of the current frame, the current quality rank of the current frame, and the encoding parameters corresponding to the current quality rank.

[0071] Furthermore, after obtaining the video parameters of the current frame, the current quality rank of the current frame, and the encoding parameters corresponding to the current quality rank, the above video parameters, current quality rank, and encoding parameters are combined to generate a frame encoding feature corresponding to the current quality rank, i.e., a first feature, i.e., the first feature includes the video parameters, current quality rank, and encoding parameters. Specifically, for example, the video parameters include the height, width, resolution, and frame rate of the image, and the encoding parameters include the frame encoding size, the type of each frame corresponding to the video data (including I-frames, P-frames, and B-frames), the degree of image distortion, and the fineness of the image. The video parameters of the current frame, the current quality rank of the current frame, and the encoding parameters corresponding to the current quality rank of the current frame are integrated to generate the first feature. F={h,w,fps,cq,type,size,satd,gp}. Here, F represents the first feature of the current frame, h represents the height of the image, w represents the width of the image, and fps represents the frame rate of the image; h, w, and fps are video parameters. cq represents the current quality rank, type represents the type of the current frame, size represents the size of the current frame, satd represents the absolute error sum of the current frame, and gp represents the quantization parameters of the current frame. type, size, satd, and gp are coding parameters.

[0072] Step S206: Obtain a second feature corresponding to the target quality rank of the preceding frame corresponding to the current frame, wherein the preceding frame is a predetermined number of video frames adjacent to the current frame.

[0073] Step S207: Based on the first and second features, generate coding feature options corresponding to the current quality rank of the current frame.

[0074] Furthermore, after obtaining a first feature corresponding to the current frame, a second feature is obtained that was generated based on the target quality rank of the preceding frame corresponding to the current frame. The second feature is similar to the first feature and is data that characterizes the frame coding features of the preceding frame. Specifically, the preceding frame corresponding to the current frame is a predetermined number of video frames adjacent to the current frame, for example, 30 video frames preceding the current frame. A concrete implementation of the preceding frame can be seen in the embodiment corresponding to Figure 5, i.e., the set of direct video frames from the Lth frame to the Mth frame is the preceding frame. More specifically, if the current frame is the second frame of the initial video data, the preceding frame of the current frame is the first frame of the initial video data, and the second feature corresponding to the first frame is the frame coding features generated based on the default quality rank, the specific process of which is not repeated. Subsequently, based on the frame coding features corresponding to multiple video frames (the first and second frames of the initial video data) composed of the second feature of the first frame and the first feature of the second frame, the mean and variance values ​​of the coding parameters in the frame coding feature corresponding to the current frame are calculated to obtain the video coding feature corresponding to the second frame, i.e., the coding feature option corresponding to the current quality rank of the current frame. The specific implementation process can be found in the detailed explanation of the process for obtaining the video coding features in the embodiment shown in Figure 5, so it will not be repeated here. When the current frame is the third or later video frame of the initial video data, each video frame in the initial video data is processed sequentially as the current frame, so by the time the processing reaches the third frame, the target quality rank (i.e., the optimized quality rank) corresponding to the preceding frame (e.g., the second frame) has already been obtained. At this time, the frame coding feature corresponding to the preceding frame of the current frame, i.e., the second feature, is generated based on the target quality rank corresponding to this preceding frame.

[0075] In this embodiment, by obtaining a second feature corresponding to the target quality rank of the preceding frame corresponding to the current frame, an encoding feature choice corresponding to the current quality rank characterizing the complexity of the video content is generated based on the second feature and the first feature. Since the second feature of the preceding frame is generated based on the optimized target quality rank, the encoding feature choice generated based on the second feature can more accurately represent the complexity of the video content, improving the accuracy of the encoding feature choice, and ultimately improving the accuracy of the target quality rank obtained based on the encoding feature choice, thereby improving the efficiency of video encoding.

[0076] Step S208: The encoded feature selection is input into the predictive neural network model to obtain a corresponding first evaluation value and a corresponding second evaluation value, where the first evaluation value characterizes the video quality evaluation value based on multi-scheme evaluation fusion of video, and the second evaluation value characterizes the video bitrate.

[0077] Step S209: If the current quality rank is at the end of the quality rank sequence, continue executing step S210; otherwise, return to executing step S203.

[0078] Furthermore, for each current frame in the cyclic process, as the current quality rank changes cyclically, the generation of the corresponding first feature also changes, and the coding feature choice corresponding to the generated current quality rank also changes. In each periodic round corresponding to the current quality rank, the coding feature choice obtained in step S207 is input to the predictive neural network model, and a first evaluation value and a second evaluation value output by the predictive neural network model can be obtained. Here, the first evaluation value characterizes the video quality evaluation value based on video multi-scheme evaluation fusion, and the second evaluation value characterizes the video bitrate.

[0079] Here, Video Multimethod Assessment Fusion (VMAF) is a video quality assessment metric used to measure the perception of streaming video quality in large-scale environments, and it can solve the problem that conventional metrics cannot reflect video situations with multiple scenes or multiple features. The specific implementation form of Video Multimethod Assessment Fusion is conventional technology, and the video bitrate can be obtained from the quantization parameters in the encoding parameters, but this will not be repeated here. A pre-trained predictive neural network model can map the input video encoding features (encoding feature choices) to the corresponding Video Multimethod Assessment Fusion metric and video bitrate. Figure 8 is a schematic diagram of the process for generating an evaluation value corresponding to the current quality rank of the current frame, as provided by an embodiment of the present disclosure. As shown in Figure 8, first, a video frame traversal is performed on the initial video data, then a quality rank traversal is performed on each video frame, the current quality rank is obtained when the traversal is performed on the current frame, and then, based on the steps of the embodiment, an encoded feature choice corresponding to the current quality rank is obtained, the encoded feature choice is input into a predictive neural network model, the predictive neural network model outputs a first evaluation value and a second evaluation value, and then these first evaluation value and second evaluation value are stored together with the corresponding encoded feature choice (and / or current quality rank) as a set of encoded feature choice-evaluation value mapping data. Subsequently, if the current quality rank is at the end of the quality rank sequence, it means that all quality ranks in the quality rank sequence corresponding to the current frame have been traversed, so step S210 is executed to select a target quality rank from multiple quality rank options, and if the current quality rank is not at the end of the quality rank sequence, the process returns to step S203, cycles to the next set of quality ranks (updating the current quality rank), and repeats the above process until all quality ranks in the quality rank sequence have been traversed.

[0080] Step S210: A target quality rank for the current frame is generated based on the first evaluation value corresponding to each of the encoding feature choices for the current frame and the corresponding second evaluation value.

[0081] Since the coding feature options are generated based on the quality rank, each coding feature option corresponds to one quality rank. After obtaining the first and second evaluation values ​​corresponding to each quality rank in the quality rank sequence of the current frame, each quality rank in the quality rank sequence is evaluated based on the first and second evaluation values ​​corresponding to each quality rank in the quality rank sequence of the current frame, and the optimal quality rank, i.e., the target quality rank, is obtained.

[0082] For example, the specific implementation steps of step S210 include the following:

[0083] Step S2101: Based on the first evaluation value corresponding to each coding feature option, a first target coding feature is obtained, the first target coding feature being a coding feature option whose first evaluation value is greater than the first threshold.

[0084] Step S2102: Based on the second evaluation value of the first target coding feature, the second target coding feature is determined, and the second target coding feature is the video coding feature with the smallest second evaluation value among the first target coding features.

[0085] Step S2103: Obtain the target quality rank based on the quality rank corresponding to the second target feature.

[0086] For example, the first and second evaluation values ​​characterize the video quality and bitrate after the video has been encoded, respectively. A higher first evaluation value indicates higher video quality, and a higher second evaluation value indicates a higher bitrate, i.e., a larger video file size. To improve the efficiency of video encoding, it is necessary to select a quality rank that can reduce the bitrate most significantly while still satisfying the video quality requirements of a given video.

[0087] To solve the above problem, in this embodiment, first, based on the first evaluation value corresponding to each coding feature option, the coding feature option whose first evaluation value (i.e., video quality) is greater than the first threshold is determined as the first target coding feature. Then, from the first target coding feature, the video coding feature with the smallest second evaluation value (i.e., the smallest bitrate) is selected, and then the quality rank corresponding to that video coding feature is obtained as the target quality rank. Specifically, the above process can be realized using the coding feature option-evaluation value mapping data saved in the previous step, and the specific process is not repeated.

[0088] In this embodiment, by combining multi-format video evaluation fusion and a bitrate index, a matching target quality rank is obtained. Assuming that the target video encoded based on the target quality rank satisfies the video quality requirements of a given video, the bitrate can be reduced, video capacity can be compressed, and the efficiency of video encoding can be improved.

[0089] Step S211: Use a hardware video editor to encode the current frame at the target quality rank and generate target frames to make up the target video.

[0090] Step S212: If the current frame is not the last frame of the initial video data, set the frame following the current frame as the new current frame and return to step S202.

[0091] For example, after obtaining the target quality rank, the current frame is encoded based on the target quality rank to obtain a target frame with better video quality and a lower bitrate. At the same time, if the current frame is not the last frame of the initial video data, the process returns to step S202, traverses all video frames, and continues the above process for the next video frame until a corresponding target frame is generated and the target video is constructed. Since dynamic encoding is performed using a different target quality rank for each video frame, it is possible to improve video quality while simultaneously reducing video size and improving video encoding efficiency.

[0092] Selectively, and as required by specific needs, a step of training a predictive neural network model may be included prior to step S208. Exemplarily, as shown in Figure 9, the step of training a predictive neural network model includes the following:

[0093] Step S2001: Obtain the original video data and quality rank sequence, the quality rank sequence containing at least two different quality ranks.

[0094] Step S2002: Based on the quality rank sequence, the original video data is processed sequentially using a hardware video editor to obtain video encoding features corresponding to each quality rank.

[0095] Step S2003: Calculate the first and second evaluation values ​​corresponding to each of the video encoding features.

[0096] Step S2004: Based on each video encoding feature, the corresponding first evaluation value, and the corresponding second evaluation value, training samples are generated, and a predetermined neural network model is trained based on the training samples to obtain a predictive neural network model.

[0097] For example, the original video data is a video data sample, and the quality rank sequence is a predetermined parameter. By calling a hardware video editor with the original video data and quality rank sequence as input parameters, the corresponding encoding parameters output by the hardware video editor can be obtained, and video encoding features can be acquired. The specific process for generating video encoding features can be found in the description in the above embodiment and will not be repeated here. Subsequently, based on the calculation method for the multi-format video evaluation fusion index and the encoding parameters output by calling the hardware video editor, a first evaluation value and a second evaluation value corresponding to each video encoding feature are obtained. The first and second evaluation values ​​are used as sample labels, and the original video data and level sequence are used as the original samples to generate training samples. Based on the training samples, a predetermined neural network model is trained until convergence is achieved, and a predictive neural network model is obtained. A detailed explanation of the specific process is omitted.

[0098] In this embodiment, by utilizing the feature that a hardware video editor can easily and quickly output encoding parameters (without actually encoding), and by generating training samples by combining multi-format video evaluation fusion metrics, efficient and high-quality training of a neural network model is achieved, and the model can be rapidly converged. This improves the prediction accuracy and efficiency of the model.

[0099] Corresponding to the video encoding method of the above embodiment, Figure 10 is a block diagram of the configuration of a video encoding device provided by an embodiment of the present disclosure. For the sake of ease of explanation, only the parts relevant to the embodiment of the present disclosure are shown. Referring to Figure 10, the video encoding device 3 comprises a parameter acquisition module 31, a parameter optimization module 32, and an encoding module 33.

[0100] The parameter acquisition module 31 is used by the hardware video editor to extract video encoding features from the initial video data, which characterize the complexity of the video content of the initial video data.

[0101] The parameter optimization module 32 is used to process the video encoding features of the initial video data using a pre-trained predictive neural network model to obtain a target quality rank corresponding to the initial video data, the target quality rank characterizing the image quality level when encoding the initial video in constant quality dynamic bitrate mode.

[0102] The encoding module 33 is used to encode the initial video data at the target quality rank using a hardware video editor to generate the target video.

[0103] In the embodiments of this disclosure, the parameter acquisition module 31 is specifically used to acquire quality rank options for initial video data, input the video parameters and quality rank options of the initial video data to a hardware video editor to acquire encoding parameters for the initial video data, and generate corresponding video encoding features based on the encoding parameters.

[0104] In the embodiments of this disclosure, the initial video data includes a plurality of video frames, and the parameter acquisition module 31 is used to generate corresponding video coding features based on coding parameters, specifically by acquiring coding parameters corresponding to each video frame, acquiring coding feature mean values ​​and coding feature variances based on coding parameters corresponding to each video frame, wherein the coding feature mean value is the mean value of the coding parameters corresponding to each video frame, and the coding feature variance is the variance of the coding parameters corresponding to each video frame, and to generate video coding features based on video parameters, quality rank options, and the coding feature mean values ​​and coding feature variances corresponding to each video frame.

[0105] In the embodiments of this disclosure, the parameter acquisition module 31 is used to input the video parameters and quality rank selection of the initial video data to a hardware video editor to acquire the encoding parameters of the initial video data, and is used to repeatedly perform the following steps until a predetermined condition is met: the steps of acquiring the current frame of the initial video data; acquiring the video parameters and quality rank of the current frame; inputting the video parameters and quality rank of the current frame to a hardware video editor to acquire the encoding parameters corresponding to the current frame; and setting the frame following the current frame as the new current frame.

[0106] In embodiments of the present disclosure, after obtaining the encoding parameters corresponding to the current frame, the parameter acquisition module 31 is further used to generate a frame encoding feature corresponding to the current frame from the video parameters of the current frame, the quality rank of the current frame, and the encoding parameters, and to obtain a frame encoding feature of a preceding frame corresponding to the current frame, wherein the preceding frame is a first predetermined number of video frames adjacent to the current frame, and the frame encoding feature of the preceding frame is generated based on the target quality rank corresponding to the preceding frame.

[0107] The parameter acquisition module 31 is used to generate corresponding video coding features based on coding parameters, specifically by generating video coding features corresponding to the current frame based on the frame coding features corresponding to the current frame and the frame coding features of the preceding frame.

[0108] In embodiments of this disclosure, the encoding parameters include at least one of the frame type, frame size, image distortion, and image resolution.

[0109] In embodiments of the present disclosure, the video coding features of the initial video data include at least two coding feature options, each coding feature option corresponding to one different quality rank, and the parameter optimization module 32 specifically inputs each coding feature option sequentially into a predictive neural network model to obtain a corresponding first evaluation value and a corresponding second evaluation value, the first evaluation value characterizing a video quality evaluation value based on video multi-scheme evaluation fusion, and the second evaluation value characterizing the video bitrate, and is used to generate a target quality rank based on the first evaluation value and the corresponding second evaluation value corresponding to each coding feature option.

[0110] In embodiments of the present disclosure, the parameter optimization module 32 generates a target quality rank based on a first evaluation value and a corresponding second evaluation value for each of the coding feature options, specifically by obtaining a first target coding feature based on the first evaluation value for each of the coding feature options, wherein the first target coding feature is a coding feature option whose first evaluation value is greater than a first threshold; by determining a second target coding feature based on the second evaluation value of the first target coding feature, wherein the second target coding feature is a video coding feature among the first target coding features that has the smallest second evaluation value; and by obtaining a target quality rank based on the quality rank corresponding to the second target coding feature.

[0111] In embodiments of the present disclosure, before processing the video coding features of the initial video data with a pre-trained predictive neural network model to obtain a target quality rank corresponding to the initial video data, the parameter optimization module 32 is further used to obtain the original video data and a quality rank sequence, wherein the quality rank sequence includes at least two different quality ranks; to sequentially process the original video data using a hardware video editor based on the quality rank sequence to obtain video coding features corresponding to each quality rank; to calculate a first evaluation value and a second evaluation value corresponding to each video coding feature; to generate training samples based on each video coding feature, the corresponding first evaluation value and the corresponding second evaluation value; and to train a predetermined neural network model based on the training samples to obtain a predictive neural network model.

[0112] Here, the parameter acquisition module 31, the parameter optimization module 32, and the encoding module 33 are connected in order. The video encoding device 3 provided by this embodiment can implement the technical proposal of the embodiment relating to the above method, and its implementation principle and technical effect are the same, so the description of this embodiment will not be repeated here.

[0113] Figure 11 is a schematic diagram of the configuration of an electronic device provided by an embodiment of the present disclosure, and as shown in Figure 11, this electronic device 4 is The system comprises a processor 41 and a memory 42 that is communicatively connected to the processor 41. Memory 42 stores computer execution instructions. The processor 41 executes computer execution instructions stored in the memory 42 to perform the video encoding method shown in the embodiments in Figures 2 to 9.

[0114] Here, the processor 41 and memory 42 are selectively connected via the bus 43.

[0115] Related explanations can be understood by referring to the related explanations and effects corresponding to the steps in the embodiments shown in Figures 2 to 9, so these explanations are omitted here.

[0116] Referring to Figure 12, which shows a schematic diagram of the configuration of an electronic device 900 suitable for carrying out an embodiment of the present disclosure, this electronic device 900 may be a terminal device or a server. Here, the terminal device includes, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, personal digital assistants (abbreviated as PDAs), tablet computers (abbreviated as Portable Android Devices (PADs)), portable media players (abbreviated as PMPs), in-vehicle terminals (e.g., vehicle navigation terminals), digital TVs, and desktop computers. Note that the electronic device shown in Figure 12 is merely an example and does not in any way limit the functions and scope of use of the embodiments of the present disclosure.

[0117] As shown in Figure 12, the electronic device 900 may include a processing unit (e.g., a central processing unit, a graphics processor, etc.) 901 capable of performing various appropriate operations and processes based on a program stored in a read-only memory (ROM) 902, or a program loaded from a storage device 908 into a random access memory (RAM) 903. The RAM 903 also stores various programs and data necessary for the operation of the electronic device 900. The processing unit 901, ROM 902, and RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0118] Typically, the I / O interface 905 may be connected to an input device 906, which may include, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, output device 907, which may include, for example, a liquid crystal display (LCD), speaker, vibrator, storage device 908, which may include, for example, magnetic tape, hard disk, and communication device 909. The communication device 909 may enable the electronic device 900 to communicate with other devices wirelessly or via a wired connection to exchange data. Figure 12 illustrates an electronic device 900 with various devices, but it should be understood that it is not necessary to implement or include all of the devices shown. More or fewer devices may be implemented or included as alternatives.

[0119] In particular, according to embodiments of the present disclosure, the process described above with reference to the flowchart may be implemented as a computer software program. For example, embodiments of the present disclosure have a computer program product which includes a computer program stored on a computer-readable medium, and which includes program code for performing the method shown in the flowchart. In such embodiments, the computer program may be downloaded and installed from a network via a communication device 909, or installed from a storage device 908, or downloaded from a ROM 902. When the computer program is executed by the processing unit 901, it performs the functions defined in the methods of embodiments of the present disclosure.

[0120] In this disclosure, the computer-readable medium described above may be a computer-readable signal medium, a computer-readable storage medium, or any combination thereof. The computer-readable storage medium may be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage mediums include, but are not limited to, electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, CD-ROMs, optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, the computer-readable storage medium may be any tangible medium containing or storing a program that is used by or can be used in combination with an instruction execution system, apparatus, or device. In this disclosure, the computer-readable signal medium may also include data signals propagated in baseband or as part of a carrier carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium for transmitting, propagating, or transmitting programs used by or in combination with instruction execution systems, apparatus, or devices. Program code contained in a computer-readable medium can be transmitted using any suitable medium, including but not limited to wires, optical fiber cables, RF (radio frequency), or any suitable combination thereof.

[0121] The computer-readable medium described above may be included in the electronic device described above, or it may be a separate component and not be assembled into the electronic device.

[0122] The computer-readable medium described above stores one or more programs, and when the one or more programs are executed by this electronic device, the electronic device is made to perform the method shown in the above embodiment.

[0123] Computer program code for performing the operations of this disclosure may be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code may run entirely on a user computer, partially on a user computer, as a standalone software package, partially on a user computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or it may be connected to an external computer (for example, connected via the Internet using an Internet service provider).

[0124] The flowcharts and block diagrams in the accompanying drawings illustrate architectures, functions, and operations that may be implemented according to the systems, methods, and computer program products relating to each embodiment of this disclosure. In this regard, each box in the flowchart or block diagram may represent a module, program segment, or part of code, which contains one or more executable instructions for implementing a specified logical function. Note that in some alternative implementations, the functions associated with a box may occur in a different order than those associated with the accompanying drawings. For example, two consecutive boxes may actually be executed substantially in parallel, or in reverse order depending on the functions involved. Also note that each box in the block diagram and / or flowchart, and combinations of boxes in the block diagram and / or flowchart, may be implemented in a dedicated hardware-based system that performs a given function or operation, or in a combination of dedicated hardware and computer instructions.

[0125] The units described in the embodiments of this disclosure may be implemented in software or in hardware. Here, the names of the units do not constitute an limitation to the unit itself in a given context; for example, the first acquisition unit may also be described as "a unit for acquiring at least two Internet Protocol addresses."

[0126] In this specification, the functions described above may be performed, at least in part, by one or more hardware logic units. For example, non-limiting examples of usable and exemplary hardware logic units include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standards (ASSPs), systems-on-a-chip (SOCs), and composite programmable logic devices (CPLDs).

[0127] In the context of this disclosure, a machine-readable medium may be a tangible medium that contains or can store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium includes, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, CD-ROMs, optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0128] In a first embodiment, one or more embodiments of the present disclosure provide a video encoding method, which is: Extracting video coding features of initial video data using a hardware video editor, wherein the video coding features characterize the complexity of the video content of the initial video data; processing the video coding features of the initial video data using a pre-trained predictive neural network model to obtain a target quality rank corresponding to the initial video data, wherein the target quality rank characterizes the image quality level when the initial video is encoded in constant quality dynamic bitrate mode; and encoding the initial video data with the target quality rank using the hardware video editor to generate a target video.

[0129] According to one or more embodiments of the present disclosure, extracting video coding features of initial video data using a hardware video editor includes obtaining a quality rank option for the initial video data, inputting the video parameters of the initial video data and the quality rank option into the hardware video editor to obtain coding parameters for the initial video data, and generating corresponding video coding features based on the coding parameters.

[0130] According to one or more embodiments of the present disclosure, the initial video data comprises a plurality of video frames, and generating corresponding video coding features based on the coding parameters comprises obtaining the coding parameters corresponding to each of the video frames, and obtaining a coding feature mean and a coding feature variance based on the coding parameters corresponding to each of the video frames, wherein the coding feature mean is the mean of the coding parameters corresponding to each of the video frames, and the coding feature variance is the variance of the coding parameters corresponding to each of the video frames, and generating the video coding features based on the video parameters, the quality rank selection, and the coding feature mean and coding feature variance corresponding to each of the video frames.

[0131] According to one or more embodiments of the present disclosure, inputting the video parameters of the initial video data and the quality rank selection into the hardware video editor to obtain the encoding parameters of the initial video data includes repeatedly performing the following steps until a predetermined condition is met: the steps of obtaining the current frame of the initial video data; obtaining the video parameters of the current frame and the quality rank of the current frame; inputting the video parameters of the current frame and the quality rank of the current frame into the hardware video editor to obtain the encoding parameters corresponding to the current frame; and setting the frame following the current frame as the new current frame.

[0132] According to one or more embodiments of the present disclosure, after obtaining coding parameters for the current frame, the method includes generating a frame coding feature for the current frame from the video parameters of the current frame, the quality rank of the current frame, and the coding parameters, and obtaining a frame coding feature for a preceding frame corresponding to the current frame, wherein the preceding frame is a first predetermined number of adjacent video frames preceding the current frame, and the frame coding feature for the preceding frame is generated based on the target quality rank of the preceding frame, and generating a corresponding video coding feature based on the coding parameters includes generating a video coding feature for the current frame based on the frame coding feature for the current frame and the frame coding feature for the preceding frame.

[0133] According to one or more embodiments of the present disclosure, the encoding parameter includes at least one of the frame type, frame size, image distortion, and image resolution.

[0134] According to one or more embodiments of the present disclosure, the video coding features of the initial video data include at least two coding feature options, each of which corresponds to a different quality rank, and processing the video coding features of the initial video data with a pre-trained predictive neural network model to obtain a target quality rank corresponding to the initial video data includes sequentially inputting each of the coding feature options into the predictive neural network model to obtain a corresponding first evaluation value and a corresponding second evaluation value, wherein the first evaluation value characterizes a video quality evaluation value based on video multi-scheme evaluation fusion, and the second evaluation value characterizes the video bitrate, and generating the target quality rank based on the first evaluation value and the corresponding second evaluation value corresponding to each of the coding feature options.

[0135] According to one or more embodiments of the present disclosure, generating the target quality rank based on a first evaluation value and a second evaluation value corresponding to each of the coding feature options includes obtaining a first target coding feature based on a first evaluation value corresponding to each of the coding feature options, wherein the first target coding feature is a coding feature option whose first evaluation value is greater than a first threshold; determining a second target coding feature based on a second evaluation value of the first target coding feature, wherein the second target feature is a video coding feature among the first target coding features whose second evaluation value is the smallest; and obtaining the target quality rank based on the quality rank corresponding to the second target feature.

[0136] According to one or more embodiments of the present disclosure, the process involves processing the video coding features of the initial video data with a pre-trained predictive neural network model to obtain the original video data and a quality rank sequence, wherein the quality rank sequence includes at least two different quality ranks; sequentially processing the original video data using a hardware video editor based on the quality rank sequence to obtain video coding features corresponding to each of the quality ranks; calculating a first evaluation value and a second evaluation value corresponding to each of the video coding features; generating training samples based on each of the video coding features, the corresponding first evaluation value, and the corresponding second evaluation value; and training a predetermined neural network model based on the training samples to obtain the predictive neural network model.

[0137] In a second embodiment, according to one or more embodiments of the present disclosure, a video encoding device is provided, which is A parameter acquisition module for extracting video encoding features of initial video data using a hardware video editing machine, wherein the video encoding features characterize the complexity of the video content of the initial video data, A parameter optimization module for processing the video encoding features of the initial video data using a pre-trained predictive neural network model and obtaining a target quality rank corresponding to the initial video data, wherein the target quality rank is a parameter optimization module that characterizes the image quality level when the initial video is encoded in a constant quality dynamic bitrate mode. The system includes an encoding module for encoding the initial video data at the target quality rank using the aforementioned hardware video editing machine and generating a target video.

[0138] According to one or more embodiments of the present disclosure, the parameter acquisition module is used specifically to acquire quality rank options for the initial video data, to input the video parameters of the initial video data and the quality rank options into the hardware video editor to obtain the encoding parameters for the initial video data, and to generate corresponding video encoding features based on the encoding parameters.

[0139] According to one or more embodiments of the present disclosure, the initial video data comprises a plurality of video frames, and the parameter acquisition module is used to generate corresponding video coding features based on the coding parameters, specifically by acquiring coding parameters corresponding to each of the video frames, and obtaining coding feature mean values ​​and coding feature variance values ​​based on the coding parameters corresponding to each of the video frames, wherein the coding feature mean value is the mean value of the coding parameters corresponding to each of the video frames, and the coding feature variance value is the variance value of the coding parameters corresponding to each of the video frames, and by generating the video coding features based on the video parameters, the quality rank selection, and the coding feature mean value and coding feature variance values ​​corresponding to each of the video frames.

[0140] According to one or more embodiments of the present disclosure, the parameter acquisition module is used to input the video parameters of the initial video data and the quality rank selection to the hardware video editor and to obtain the encoding parameters of the initial video data, by repeatedly performing the following steps until a predetermined condition is met, the steps being: acquiring the current frame of the initial video data; acquiring the video parameters and the quality rank of the current frame; inputting the video parameters and the quality rank of the current frame to the hardware video editor and obtaining the encoding parameters corresponding to the current frame; and setting the frame following the current frame as the new current frame.

[0141] According to one or more embodiments of the present disclosure, after obtaining coding parameters corresponding to the current frame, the parameter acquisition module is further used to generate a frame coding feature corresponding to the current frame from the video parameters of the current frame, the quality rank of the current frame, and the coding parameters, and to acquire a frame coding feature of a preceding frame corresponding to the current frame, wherein the preceding frame is a first predetermined number of adjacent video frames preceding the current frame, and the frame coding feature of the preceding frame is generated based on the target quality rank corresponding to the preceding frame, and the parameter acquisition module is further used to generate a corresponding video coding feature based on the coding parameters, specifically, to generate a video coding feature corresponding to the current frame based on the frame coding feature corresponding to the current frame and the frame coding feature of the preceding frame.

[0142] According to one or more embodiments of the present disclosure, the encoding parameter includes at least one of the frame type, frame size, image distortion, and image resolution.

[0143] According to one or more embodiments of the present disclosure, the video coding features of the initial video data include at least two coding feature options, each of which corresponds to a different quality rank, and the parameter optimization module is used to sequentially input each of the coding feature options into the predictive neural network model to obtain a corresponding first evaluation value and a corresponding second evaluation value, the first evaluation value characterizing a video quality evaluation value based on video multi-scheme evaluation fusion, the second evaluation value characterizing the video bitrate, and to generate the target quality rank based on the first evaluation value and the corresponding second evaluation value corresponding to each of the coding feature options.

[0144] According to one or more embodiments of the present disclosure, the parameter optimization module generates the target quality rank based on a first evaluation value and a second evaluation value corresponding to each of the coding feature options, specifically by obtaining a first target coding feature based on a first evaluation value corresponding to each of the coding feature options, wherein the first target coding feature is a coding feature option whose first evaluation value is greater than a first threshold, and by determining a second target coding feature based on a second evaluation value of the first target coding feature, wherein the second target feature is a video coding feature among the first target coding features whose second evaluation value is the smallest, and by obtaining the target quality rank based on the quality rank corresponding to the second target feature.

[0145] According to one or more embodiments of the present disclosure, the parameter optimization module is further used to obtain the original video data and a quality rank sequence, wherein the quality rank sequence includes at least two different quality ranks, before processing the video coding features of the initial video data with a pre-trained predictive neural network model to obtain a target quality rank corresponding to the initial video data, the parameter optimization module obtains the original video data and a quality rank sequence, wherein the quality rank sequence includes at least two different quality ranks, and based on the quality rank sequence, it processes the original video data sequentially using the hardware video editor to obtain a video coding feature corresponding to each of the quality ranks, calculates a first evaluation value and a second evaluation value corresponding to each of the video coding features, generates training samples based on each of the video coding features, the corresponding first evaluation value and the corresponding second evaluation value, and trains a predetermined neural network model based on the training samples to obtain the predictive neural network model.

[0146] In a third embodiment, according to one or more embodiments of the present disclosure, an electronic device is provided which comprises a processor and a memory communicably connected to the processor, The aforementioned memory stores computer execution instructions, The processor executes computer execution instructions stored in the memory to perform the video encoding method described in the first embodiment and various possible designs of the first embodiment.

[0147] In a fourth embodiment, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, a computer execution instruction is stored in the computer-readable storage medium, and when a processor executes the computer execution instruction, it performs the video encoding method described in the first embodiment and various possible designs of the first embodiment.

[0148] In a fifth aspect, embodiments of the present disclosure provide a computer program product that, when executed by a processor, includes a computer program that performs the video encoding method described in the first aspect and various possible designs of the first aspect.

[0149] The above description is merely an explanation of preferred embodiments of the present disclosure and the technical principles employed. Those skilled in the art will understand that the scope of the disclosure is not limited to any technical solution formed by a specific combination of the above technical features, but also covers other technical solutions formed by any combination of the above technical features or equivalent features without departing from the disclosed concepts. For example, this includes technical solutions formed by replacing the above features with (but not limited to) similarly functional technical features disclosed in this disclosure.

[0150] Furthermore, although each operation is described using a specific order, this should not be interpreted as requiring these operations to be executed in a specific order or sequentially. Multitasking and parallel processing may be advantageous in certain environments. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of this disclosure. Some features described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, each feature described in the context of a single embodiment can also be implemented in multiple embodiments, individually or in any appropriate subcombination.

[0151] While this subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter as defined in the attached claims is not necessarily limited to the specific features or behaviors described above. Rather, the specific features and behaviors described above are merely exemplary forms of implementing the claims.

Claims

1. Extracting video encoding features of initial video data using a hardware video editing machine, wherein the video encoding features characterize the complexity of the video content of the initial video data. The method involves processing the video encoding features of the initial video data using a pre-trained predictive neural network model to obtain a target quality rank corresponding to the initial video data, wherein the target quality rank characterizes the image quality level when the initial video data is encoded in a constant quality dynamic bitrate mode. The process involves encoding the initial video data at the target quality rank using the aforementioned hardware video editing machine to generate the target video, A video encoding method, including...

2. Extracting video encoding features from initial video data using a hardware video editing machine is possible. To obtain the quality rank options for the initial video data, The video parameters of the initial video data and the quality rank selection are input to the hardware video editing machine to obtain the encoding parameters of the initial video data. To generate corresponding video coding features based on the aforementioned coding parameters, The method according to claim 1, including the method described in claim 1.

3. The initial video data includes multiple video frames, Generating corresponding video coding features based on the aforementioned coding parameters is: Obtain the encoding parameters corresponding to each of the aforementioned video frames, The method involves obtaining an average encoding feature value and an average encoding feature value based on the encoding parameters corresponding to each of the video frames, wherein the average encoding feature value is the average value of the encoding parameters corresponding to each of the video frames, and the average encoding feature value is the variance of the encoding parameters corresponding to each of the video frames. The video coding features are generated based on the video parameters, the quality rank selection, and the average and variance values ​​of the coding features corresponding to each of the video frames. The method according to claim 2, including the method described in claim 2.

4. Inputting the video parameters of the initial video data and the quality rank selection into the hardware video editor and obtaining the encoding parameters of the initial video data includes repeatedly performing the following steps until a predetermined condition is met: The aforementioned step is, The steps include obtaining the current frame of the initial video data, The steps include obtaining the video parameters of the current frame and the quality rank of the current frame, The steps include inputting the video parameters and quality rank of the current frame into the hardware video editing machine to obtain encoding parameters corresponding to the current frame, The steps include setting the frame following the current frame as the new current frame, The method according to claim 2.

5. After obtaining the encoding parameters corresponding to the current frame, To generate frame coding features corresponding to the current frame from the video parameters of the current frame, the quality rank of the current frame, and the coding parameters, The method further includes obtaining the frame coding features of preceding frames corresponding to the current frame, wherein the preceding frames are a first predetermined number of video frames adjacent to the current frame, and the frame coding features of the preceding frames are generated based on the target quality rank corresponding to the preceding frames. Generating corresponding video coding features based on the aforementioned coding parameters is: To generate a video encoding feature corresponding to the current frame based on the frame encoding feature corresponding to the current frame and the frame encoding feature of the preceding frame, The method according to claim 4, including the method described in claim 4.

6. The method according to claim 2, wherein the encoding parameter includes at least one of frame type, frame size, image distortion, and image resolution.

7. The video encoding features of the initial video data include at least two encoding feature options, each of which corresponds to a different quality rank. Processing the video encoding features of the initial video data using a pre-trained predictive neural network model and obtaining a target quality rank corresponding to the initial video data is: Each of the aforementioned coding feature options is sequentially input into the predictive neural network model to obtain a corresponding first evaluation value and a corresponding second evaluation value, wherein the first evaluation value characterizes the video quality evaluation value based on multi-syntax video evaluation fusion, and the second evaluation value characterizes the video bitrate. The target quality rank is generated based on the first evaluation value and the corresponding second evaluation value corresponding to each of the coding feature options. The method according to claim 1, including the method described in claim 1.

8. To generate the target quality rank based on the first evaluation value and the corresponding second evaluation value for each of the coding feature options, A first target coding feature is obtained based on a first evaluation value corresponding to each of the coding feature options, wherein the first target coding feature is a coding feature option whose first evaluation value is greater than a first threshold. The second target coding feature is determined based on the second evaluation value of the first target coding feature, wherein the second target coding feature is the video coding feature among the first target coding features that has the smallest second evaluation value. Based on the quality rank corresponding to the second target coding feature, the target quality rank is obtained, The method according to claim 7, including the method described in claim 7.

9. Before processing the video coding features of the initial video data using a pre-trained predictive neural network model and obtaining a target quality rank corresponding to the initial video data, The process involves obtaining the original video data and a quality rank sequence, wherein the quality rank sequence includes at least two different quality ranks. Based on the quality rank sequence, the original video data is sequentially processed using the hardware video editor to obtain video encoding features corresponding to each of the quality ranks. To calculate a first evaluation value and a second evaluation value corresponding to each of the aforementioned video encoding features, The process involves generating training samples based on each of the video encoding features, the corresponding first evaluation value, and the corresponding second evaluation value, training a predetermined neural network model based on the training samples, and obtaining the predictive neural network model. The method according to claim 7, including the method described in claim 7.

10. A parameter acquisition module for extracting video encoding features of initial video data using a hardware video editing machine, wherein the video encoding features characterize the complexity of the video content of the initial video data, A parameter optimization module for processing the video encoding features of the initial video data using a pre-trained predictive neural network model and obtaining a target quality rank corresponding to the initial video data, wherein the target quality rank is a parameter optimization module that characterizes the image quality level when the initial video data is encoded in a constant quality dynamic bitrate mode. An encoding module for encoding the initial video data at the target quality rank using the aforementioned hardware video editing machine and generating a target video, A video encoding device equipped with the following features.

11. The system comprises a processor and a memory that is communicably connected to the processor, The aforementioned memory stores computer execution instructions, The processor is an electronic device that executes computer execution instructions stored in the memory to perform the video encoding method according to any one of claims 1 to 9.

12. A computer-readable storage medium that stores computer execution instructions, and when a processor executes the computer execution instructions, performs the video encoding method according to any one of claims 1 to 9.

13. A computer program that, when executed by a processor, performs the video encoding method described in any one of claims 1 to 9.

Images