Video encoding method, computing device, storage medium and program product

Abstract

The present application provides a video encoding method, computing device, storage medium, and program product. The method includes: extracting features from a video to be encoded to obtain video features of multiple dimensions related to the video to be encoded; based on the video features of multiple dimensions, predicting corresponding video quality curves and video bit rate curves when encoding the video to be encoded based on different constant quality factors (CRFs); obtaining CRF target values ​​selected from the video quality curves and the video bit rate curves that match the video quality and video bit rate required for video encoding of the video to be encoded; and encoding the video to be encoded based on the CRF target values ​​to obtain a target video.

Description

Technical Field

[0001] One or more embodiments of the present application relate to the field of video coding technology, and in particular, to a video coding method, computing device, storage medium, and program product. Background Art

[0002] The Constant Rate Factor (CRF) is a parameter used in constant quality encoding mode. By setting a fixed CRF value, the encoder automatically adjusts the bitrate during video encoding to maintain consistent visual quality of the output video as much as possible. Bitrate refers to the number of bits transmitted per second. The higher the bitrate, the more data is transmitted per second, resulting in clearer video quality.

[0003] The larger the CRF value, the more severely the source video is compressed during the encoding process, and the smaller the bit rate of the encoded video is. In other words, the smaller the amount of encoded video data is, and accordingly, the worse the video quality is.

[0004] Therefore, for the source video to be encoded, considering the encoded video quality and bit rate comprehensively, it is particularly important to quickly and accurately select a suitable CRF value for it to perform video encoding. Summary of the Invention

[0005] In view of this, one or more embodiments of the present application provide a video encoding method, a computing device, a computer-readable storage medium, and a computer program product.

[0006] In a first aspect, the present application provides a video encoding method, the method comprising:

[0007] Extracting features from the video to be encoded to obtain video features of multiple dimensions related to the video to be encoded;

[0008] Based on the video features of the multiple dimensions, predicting corresponding video quality curves and video bit rate curves when encoding the video to be encoded based on different constant quality factor (CRF) values;

[0009] Obtaining a CRF target value selected from the video quality curve and the video bit rate curve that matches the video quality and video bit rate required for video encoding of the video to be encoded;

[0010] The video to be encoded is encoded based on the CRF target value to obtain a target video.

[0011] In one illustrated embodiment, the video quality includes a VMAF value, a multi-method fusion evaluation index of video quality based on subjective perception of the human eye; the multiple dimensional video features include a CRF feature value, a VMAF feature value, and a video bitrate feature value; and the feature extraction of the to-be-encoded video to obtain multiple dimensional video features related to the to-be-encoded video includes:

[0012] Performing video encoding on the video to be encoded based on the selected CRF feature value to obtain an encoded video;

[0013] A VMAF value and a video bit rate of the encoded video are obtained, and the VMAF value and the video bit rate are determined as the VMAF characteristic value and the video bit rate characteristic value corresponding to the CRF characteristic value.

[0014] In an illustrated embodiment, the predicting, based on the video features of the multiple dimensions, corresponding video quality curves and video bit rate curves when encoding the video to be encoded based on different CRF values, includes:

[0015] The video features of the multiple dimensions are input into a pre-trained deep learning model, and the deep learning model predicts the corresponding video quality curve and video bit rate curve when the video to be encoded is encoded based on different CRF values ​​based on the video features of the multiple dimensions.

[0016] In an illustrated embodiment, the deep learning model includes a first basic network; inputting the video features of the multiple dimensions into a pre-trained deep learning model, and predicting, by the deep learning model based on the video features of the multiple dimensions, corresponding video quality curves and video bitrate curves when encoding the video to be encoded based on different CRF values, including:

[0017] Perform feature splicing on the video features of the multiple dimensions to obtain target features;

[0018] The target feature is input into the first basic network, and the first basic network predicts, based on the target feature, a video quality curve and a video bit rate curve corresponding to when the video to be encoded is encoded based on different CRF values.

[0019] In an illustrated embodiment, the first basic network includes a batch normalization layer, a self-attention mechanism layer, a residual feature layer, and a fully connected layer connected in sequence; inputting the target feature into the first basic network, and predicting, by the first basic network, based on the target feature, corresponding video quality curves and video bitrate curves when encoding the to-be-encoded video based on different CRF values, including:

[0020] Inputting the target feature into a batch normalization layer, wherein the batch normalization layer performs batch normalization processing on the video features of multiple dimensions contained in the target feature;

[0021] Inputting the normalized video features of the multiple dimensions into a self-attention mechanism layer, the self-attention mechanism layer extracts correlation information between the video features of the multiple dimensions, and integrating the video features of the multiple dimensions into deep features based on the correlation information;

[0022] Inputting the depth feature into a residual feature layer, and performing residual correction on the depth feature by the residual feature layer to obtain a residual feature;

[0023] The residual features are input into a fully connected layer, and the fully connected layer predicts, based on the residual features, a video quality curve and a video bit rate curve corresponding to when the video to be encoded is encoded based on different CRF values.

[0024] In an illustrated embodiment, the deep learning model further includes a hanging operator, and the method further includes:

[0025] Inputting the video quality curve and the video bit rate curve predicted by the first basic network into the suspension operator, and having the suspension operator perform suspension processing on the video quality curve based on the CRF feature value and the VMAF feature value, so that the video quality curve after suspension processing passes through a first anchor point formed by the CRF feature value and the VMAF feature value; and

[0026] The video bit rate curve is suspended based on the CRF feature value and the video bit rate feature value, so that the video bit rate curve after the suspension process passes through a second anchor point formed by the CRF feature value and the video bit rate feature value.

[0027] In an illustrated embodiment, the deep learning model further includes a second basic network, and the method further includes:

[0028] Inputting the video quality curve and the video bit rate curve after the suspension processing into the second basic network, and performing residual correction on the video quality curve and the video bit rate curve after the suspension processing by the second basic network to obtain the residual-corrected video quality curve and the video bit rate curve;

[0029] The obtaining of a CRF target value selected from the video quality curve and the video bit rate curve and matching the video quality and video bit rate required for video encoding of the video to be encoded includes:

[0030] A CRF target value is obtained, which is selected from the residual-corrected video quality curve and the video bit rate curve and matches the VMAF value and video bit rate required for video encoding of the video to be encoded.

[0031] In an illustrated embodiment, obtaining a CRF target value selected from the video quality curve and the video bit rate curve that matches the video quality and video bit rate required for video encoding of the video to be encoded includes:

[0032] Determining a first CRF value range corresponding to a video quality value range required for video encoding of the video to be encoded from the video quality curve; and

[0033] Determining a second CRF value range from the video bit rate curve corresponding to a video bit rate value range required for video encoding of the video to be encoded;

[0034] A CRF target value is selected from the intersection of the first CRF value range and the second CRF value range.

[0035] In one illustrated embodiment, the method further comprises:

[0036] Acquire multiple sample videos, perform feature extraction on each of the multiple sample videos, and obtain video features of multiple dimensions related to each sample video;

[0037] Performing video encoding on each sample video multiple times based on multiple preset CRF values ​​to obtain multiple encoded videos corresponding to each sample video, and obtaining video quality and video bit rate of the multiple encoded videos;

[0038] The deep learning model is supervisedly trained using video features of multiple dimensions related to each video sample as training samples, and video quality and video bit rate of multiple encoded videos corresponding to each sample video as sample labels.

[0039] In a second aspect, the present application provides a video encoding device, the device comprising:

[0040] A feature extraction unit, configured to extract features from the video to be encoded, and obtain video features of multiple dimensions related to the video to be encoded;

[0041] A prediction unit, configured to predict, based on the video features of the multiple dimensions, corresponding video quality curves and video bit rate curves when encoding the video to be encoded based on different CRF values;

[0042] an acquiring unit, configured to acquire a CRF target value selected from the video quality curve and the video bit rate curve and matching the video quality and video bit rate required for video encoding of the video to be encoded;

[0043] The encoding unit is configured to perform video encoding on the video to be encoded based on the CRF target value to obtain a target video.

[0044] In one illustrated embodiment, the video quality includes a VMAF value, a multi-method fusion evaluation index of video quality based on subjective perception of the human eye; the video features in the multiple dimensions include a CRF feature value, a VMAF feature value, and a video bitrate feature value; and the feature extraction unit is specifically configured to:

[0045] Performing video encoding on the video to be encoded based on the selected CRF feature value to obtain an encoded video;

[0046] A VMAF value and a video bit rate of the encoded video are obtained, and the VMAF value and the video bit rate are determined as the VMAF characteristic value and the video bit rate characteristic value corresponding to the CRF characteristic value.

[0047] In an illustrated embodiment, the prediction unit is specifically configured to:

[0048] The video features of the multiple dimensions are input into a pre-trained deep learning model, and the deep learning model predicts the corresponding video quality curve and video bit rate curve when the video to be encoded is encoded based on different CRF values ​​based on the video features of the multiple dimensions.

[0049] In one illustrated embodiment, the deep learning model includes a first basic network; the prediction unit is specifically configured to:

[0050] Perform feature splicing on the video features of the multiple dimensions to obtain target features;

[0051] The target feature is input into the first basic network, and the first basic network predicts, based on the target feature, a video quality curve and a video bit rate curve corresponding to when the video to be encoded is encoded based on different CRF values.

[0052] In one illustrated embodiment, the first basic network includes a batch normalization layer, a self-attention mechanism layer, a residual feature layer, and a fully connected layer connected in sequence; the prediction unit is specifically configured to:

[0053] Inputting the target feature into a batch normalization layer, wherein the batch normalization layer performs batch normalization processing on the video features of multiple dimensions contained in the target feature;

[0054] Inputting the normalized video features of the multiple dimensions into a self-attention mechanism layer, the self-attention mechanism layer extracts correlation information between the video features of the multiple dimensions, and integrating the video features of the multiple dimensions into deep features based on the correlation information;

[0055] Inputting the depth feature into a residual feature layer, and performing residual correction on the depth feature by the residual feature layer to obtain a residual feature;

[0056] The residual features are input into a fully connected layer, and the fully connected layer predicts, based on the residual features, a video quality curve and a video bit rate curve corresponding to when the video to be encoded is encoded based on different CRF values.

[0057] In an illustrated embodiment, the deep learning model further includes a suspension operator, and the apparatus further includes a suspension processing unit configured to:

[0058] Inputting the video quality curve and the video bit rate curve predicted by the first basic network into the suspension operator, and having the suspension operator perform suspension processing on the video quality curve based on the CRF feature value and the VMAF feature value, so that the video quality curve after suspension processing passes through a first anchor point formed by the CRF feature value and the VMAF feature value; and

[0059] The video bit rate curve is suspended based on the CRF feature value and the video bit rate feature value, so that the video bit rate curve after the suspension process passes through a second anchor point formed by the CRF feature value and the video bit rate feature value.

[0060] In an illustrated embodiment, the deep learning model further includes a second basic network, and the apparatus further includes a residual correction unit configured to:

[0061] Inputting the video quality curve and the video bit rate curve after the suspension processing into the second basic network, and performing residual correction on the video quality curve and the video bit rate curve after the suspension processing by the second basic network to obtain the residual-corrected video quality curve and the video bit rate curve;

[0062] The acquisition unit is specifically configured to:

[0063] A CRF target value is obtained, which is selected from the residual-corrected video quality curve and the video bit rate curve and matches the VMAF value and video bit rate required for video encoding of the video to be encoded.

[0064] In an illustrated embodiment, the acquisition unit is specifically configured to:

[0065] Determining a first CRF value range corresponding to a video quality value range required for video encoding of the video to be encoded from the video quality curve; and

[0066] Determining a second CRF value range from the video bit rate curve corresponding to a video bit rate value range required for video encoding of the video to be encoded;

[0067] A CRF target value is selected from the intersection of the first CRF value range and the second CRF value range.

[0068] In an illustrated embodiment, the apparatus further comprises a model training unit, configured to:

[0069] Acquire multiple sample videos, perform feature extraction on each of the multiple sample videos, and obtain video features of multiple dimensions related to each sample video;

[0070] Performing video encoding on each sample video multiple times based on multiple preset CRF values ​​to obtain multiple encoded videos corresponding to each sample video, and obtaining video quality and video bit rate of the multiple encoded videos;

[0071] The deep learning model is supervisedly trained using video features of multiple dimensions related to each video sample as training samples, and video quality and video bit rate of multiple encoded videos corresponding to each sample video as sample labels.

[0072] In a third aspect, the present application provides a model training method, the method comprising:

[0073] Acquire multiple sample videos, perform feature extraction on each of the multiple sample videos, and obtain video features of multiple dimensions related to each sample video;

[0074] Performing video encoding on each sample video multiple times based on multiple preset CRF values ​​to obtain multiple encoded videos corresponding to each sample video, and obtaining video quality and video bit rate of the multiple encoded videos;

[0075] The deep learning model is supervisedly trained using video features of multiple dimensions related to each video sample as training samples, and the video quality and video bit rate of multiple encoded videos corresponding to each sample video as sample labels; wherein the trained deep learning model is used to predict the video quality curve and video bit rate curve corresponding to the video to be encoded based on different CRF values ​​when the video to be encoded is encoded based on the input video features of multiple dimensions related to the video to be encoded.

[0076] In a fourth aspect, the present application provides a model training device, comprising:

[0077] A feature extraction unit is used to obtain multiple sample videos, perform feature extraction on each of the multiple sample videos, and obtain video features of multiple dimensions related to each sample video;

[0078] an acquiring unit, configured to perform video encoding on each sample video multiple times based on a plurality of preset CRF values, obtain a plurality of encoded videos corresponding to each sample video, and obtain video quality and video bit rate of the plurality of encoded videos;

[0079] The training unit is used to perform supervised training on the deep learning model using video features of multiple dimensions related to each video sample as training samples, and using the video quality and video bit rate of multiple encoded videos corresponding to each sample video as sample labels; wherein the trained deep learning model is used to predict the video quality curve and video bit rate curve corresponding to the video to be encoded based on different CRF values ​​when the video to be encoded is encoded based on the input video features of multiple dimensions related to the video to be encoded.

[0080] Accordingly, the present application also provides a computing device, comprising: a processor and a memory for storing processor executable instructions; when the processor executes the executable instructions, it implements the video encoding method described in the first aspect or the model training method described in the third aspect.

[0081] Accordingly, the present application also provides a computer-readable storage medium on which computer instructions are stored. When the computer instructions are executed by a processor, the video encoding method described in the first aspect or the model training method described in the third aspect is implemented.

[0082] Accordingly, the present application also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the video encoding method described in the first aspect or the model training method described in the third aspect.

[0083] In summary, the present application obtains video features of multiple dimensions related to the video to be encoded by extracting features from the video to be encoded. Furthermore, based on the extracted video features of multiple dimensions, the corresponding video quality curves and video bit rate curves are predicted when the video to be encoded is encoded based on different CRF values. Based on this, the user can combine their own actual needs, comprehensively consider the video quality and bit rate, and quickly and accurately select a suitable CRF value from the curve to encode the video to be encoded. Furthermore, based on the above-mentioned video quality curve and video bit rate curve, the gradient information between video quality and bit rate can also be discovered, such as how much bit rate needs to be increased to improve the video quality by one unit, how much bit rate can be saved by reducing the video quality by one unit, and so on. In this way, users can be assisted in designing quality improvement or cost reduction strategies to maximize profits. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] Figure 1 This is a schematic diagram of a video coding system architecture provided by an exemplary embodiment;

[0085] Figure 2 This is a flowchart of a video encoding method provided by an exemplary embodiment;

[0086] Figure 3 is a schematic structural diagram of a deep learning model provided by an exemplary embodiment;

[0087] Figure 4 is a schematic diagram of a video quality curve provided by an exemplary embodiment;

[0088] Figure 5 This is a schematic diagram of a video bit rate curve provided by an exemplary embodiment;

[0089] Figure 6 is a flowchart of a model training method provided by an exemplary embodiment;

[0090] Figure 7 This is a schematic structural diagram of a video encoding device provided by an exemplary embodiment;

[0091] Figure 8 is a structural diagram of a model training device provided by an exemplary embodiment;

[0092] Figure 9 It is a structural diagram of a computing device provided by an exemplary embodiment. DETAILED DESCRIPTION

[0093] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numbers in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible implementations consistent with one or more embodiments of the present application. Rather, they are merely examples of apparatuses and methods consistent with certain aspects of one or more embodiments of the present application, as detailed in the appended claims.

[0094] It should be noted that in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this application. In some other embodiments, the method may include more or fewer steps than those described in this application. In addition, a single step described in this application may be broken down into multiple steps for description in other embodiments; and multiple steps described in this application may be combined into a single step for description in other embodiments.

[0095] First, some of the terms used in the embodiments of the present application are explained to facilitate understanding by those skilled in the art.

[0096] (1) CRF is a parameter in constant quality encoding mode. After setting a fixed CRF value, the encoder will automatically adjust the bitrate when encoding the source video to maintain the consistency of the visual quality of the output video as much as possible.

[0097] (2) Bit rate (or bit rate) refers to the number of bits transmitted per second. The higher the bit rate, the more data is transmitted per second, and the clearer the video quality. The larger the CRF value, the more severely the source video is compressed during the encoding process, and the smaller the bit rate of the encoded video, that is, the smaller the amount of encoded video data, and accordingly, the worse the video quality.

[0098] (3) Video Muitimethod Assessment Fusion (VMAF) is a common video quality evaluation metric. VMAF can effectively assess the subjective quality loss of the encoded video (or transcoded video) compared to the source video based on the human eye's subjective perception. The larger the VMAF value, the better the video quality.

[0099] As mentioned above, it's easy to understand that if you want to reduce the amount of data during video encoding by lowering the video bitrate to achieve greater cost savings, you'll inevitably have to sacrifice video quality. If you need to maintain the quality of the encoded video, you'll inevitably have to increase the bitrate, which increases costs. Therefore, it's crucial to quickly and accurately select an appropriate CRF value for the video being encoded, taking into account both the quality and bitrate of the encoded video.

[0100] However, in conventional video encoding schemes, an exhaustive search for the corresponding CRF values ​​at each resolution is generally used. Through a large number of transcoding operations, transcoding parameters (including CRF values) and resolution levels are selected so that the video quality (i.e., VMAF value) remains constant within a certain range. This obviously consumes a lot of transcoding resources and is not worth the effort. Alternatively, in some other schemes, a model is used to predict the CRF value corresponding to each video to be encoded at a fixed VMAF value, and the video to be encoded is encoded based on the CRF value. Although this can control the video quality of the encoded video within the expected range, it does not take the video bit rate into consideration. As a result, it is impossible to estimate the bit rate cost when improving video quality, and it is also impossible to evaluate the bit rate benefit when reducing video quality. As a result, users cannot formulate reasonable quality improvement or cost reduction strategies based on actual needs.

[0101] Based on this, the present application provides a video encoding method, which first extracts features from the video to be encoded to obtain video features of multiple dimensions related to the video to be encoded. Furthermore, based on the extracted video features of multiple dimensions, the corresponding video quality curves and video bit rate curves are predicted when the video to be encoded is encoded based on different CRF values. In this way, by combining the above-mentioned video quality curve and video bit rate curve, the user can combine their actual needs, comprehensively consider the video quality and bit rate, and quickly and accurately select a suitable CRF value from the curve to encode the video to be encoded. Furthermore, based on the above-mentioned video quality curve and video bit rate curve, the gradient information between video quality and bit rate can also be discovered, such as how much bit rate needs to be increased to improve the video quality by one unit, how much bit rate can be saved by reducing the video quality by one unit, and so on. In this way, users can be assisted in designing quality improvement or cost reduction strategies to maximize profits.

[0102] See also Figure 1 , Figure 1 This is a schematic diagram of a video coding system architecture provided by an exemplary embodiment. The technical solution of the embodiment of the present application can be Figure 1 The system architecture shown in FIG. 1 or a similar system architecture is specifically implemented. Figure 1As shown, the system architecture may include a server 100 and at least one client 200. The server 100 and the at least one client 200 may establish a communication connection through any possible means, which is not specifically limited in this application.

[0103] The server 100 may be a program installed in a background device to provide services to users. For example, Figure 1 As shown, the background device can be a server, which can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0104] The client 200 may be a program installed in a user terminal to provide services to the user, wherein the client 200 includes but is not limited to an application APP, a web page, a small program, a plug-in or a component, etc. For example, Figure 1 As shown, user terminals include but are not limited to smart phones, personal digital assistants, tablet computers, personal computers, laptops, virtual reality terminal devices, augmented reality terminal devices, etc.

[0105] In an illustrated embodiment, a video encoding method provided by the present application can be applied to the server 100. Exemplarily, users can upload videos they have shot or produced through the client 200 to publish corresponding video works in the client. The client 200 can send the videos uploaded by the user to the server 100. Accordingly, the server 100 can use the video encoding method provided by the present application to predict the corresponding video quality curve and video bit rate curve when the video is encoded based on different CRF values. Among them, the video quality curve can be used to describe the changing relationship between CRF and video quality, and the video bit rate curve can be used to describe the changing relationship between CRF and video bit rate. Furthermore, the server 100 can select a suitable CRF value from the video quality curve and the video bit rate curve for video encoding, and return the encoded video to the client 200, which will display the encoded video to the user through the corresponding interface, etc. For details, please refer to the following Figure 2 The description of the corresponding embodiments will not be elaborated here.

[0106] In some possible implementations, a video encoding method provided by this application can also be applied to Figure 1In the client 200 shown. For example, after obtaining the video uploaded by the user, the client 200 can use the video encoding method provided by this application to predict the video quality curve and video bit rate curve related to the video to be encoded, select a suitable CRF value from the curve to perform video encoding, and display the encoded video to the user through a corresponding interface, etc. This application does not make specific limitations on this.

[0107] The following will Figure 1 The server 100 in the system architecture shown in the figure is used as an example to perform the video encoding method provided in the embodiment of the present application. Figure 2 , Figure 2 FIG. 1 is a flow chart of a video encoding method provided by an exemplary embodiment. Figure 2 As shown, the method may specifically include the following steps S201 to S204.

[0108] Step S201 : extracting features from the video to be encoded to obtain video features of multiple dimensions related to the video to be encoded.

[0109] First, it should be noted that this application does not specifically limit the source, type, or content of the video to be encoded. In one embodiment, the video to be encoded can be a source video uploaded by a user (individual or organization) through a client, such as a video work shot or produced by the user. In another embodiment, the video to be encoded can also be surveillance footage uploaded by the client at a scheduled time, or some videos that have been published on a video website, etc. This application does not specifically limit this.

[0110] In an illustrated embodiment, the video to be encoded may be a complete video, or a video segment of the complete video, etc., which is not specifically limited in this application.

[0111] Furthermore, after obtaining the video to be encoded, the server may perform feature extraction on the video to be encoded to obtain video features of multiple dimensions related to the video to be encoded.

[0112] It should be noted that this application does not impose any particular limitation on the specific implementation method of feature extraction.

[0113] In an illustrated embodiment, feature extraction may include three parts: video content feature extraction, online complete encoding feature extraction, and fast pre-encoding feature extraction.

[0114] The video content feature extraction may include a combination of one or more feature extraction methods shown below:

[0115] The video spatial texture features of the video to be encoded are extracted based on the gray-level co-occurrence matrix (GLCM). The gray-level co-occurrence matrix is ​​a commonly used image processing technique for analyzing the texture characteristics of an image. The co-occurrence matrix reflects the spatial co-occurrence probability of different gray levels, thereby capturing the image's texture information, such as roughness and regularity. When applied to video, each color video frame in the video can be first converted into a grayscale image, and then the co-occurrence matrix of the pixel grayscale values ​​in each frame can be calculated. By analyzing these matrices, indicators reflecting the texture characteristics of the video frame, such as contrast, correlation, and entropy, can be extracted. These will not be discussed in detail here.

[0116] Based on a preset quality assessment algorithm, quality information of the video to be encoded is extracted. Exemplarily, the quality information may include one or more of noise information, blind video quality assessment (BVQA) information, block effect information and blur degree information. Among them, noise information can be used to evaluate the intensity and distribution of noise in the video, and noise will affect the clarity and viewing experience of the video. Among them, the blind video quality assessment information refers to the quality score predicted directly from the video to be encoded based on the statistical analysis or machine learning model of the video signal of the video to be encoded, without relying on the reference video. Among them, the block effect information can represent the image blocking effect caused by compression or other processing, which is usually manifested as a clear boundary in the image (i.e., the video frame of the video to be encoded), affecting the visual continuity. Among them, the blur degree information can measure the blur degree of the video, which can usually be determined by analyzing the sharpness or edge information of the video frame.

[0117] The video's spatiotemporal correlation features are extracted based on a preset spatiotemporal algorithm. These algorithms can include motion estimation, optical flow calculation, or time series analysis techniques to capture information such as the dynamic behavior, motion trajectories, and scene transitions of objects in the video. These analyses can extract features in both temporal and spatial dimensions, such as those reflecting the changing patterns of video content, motion coherence, and scene complexity. These features are not further detailed here.

[0118] Among them, the feature extraction of online complete encoding may include: performing complete video encoding on the video to be encoded based on the selected CRF feature value to obtain the encoded video, then obtaining the VMAF value and video bit rate of the encoded video, and determining the VMAF value and video bit rate as the VMAF feature value and video bit rate feature value corresponding to the CRF feature value.

[0119] In one illustrated embodiment, the selected CRF feature value may be a relatively suitable CRF feature value selected based on historical experience, such as 33, 32, or 30, etc., and this application does not impose any specific limitation thereto. For example, taking the selected CRF feature value equal to 32 as an example, a video to be encoded is completely encoded using this CRF feature value, and the VMAF feature value corresponding to the encoded video may be 92, and the video bitrate feature value may be 800 (bit / s).

[0120] For example, the CRF feature value can be recorded as anchor_crf, the video bitrate feature value can be recorded as anchor_bitrate, and the VMAF feature value can be recorded as anchor_vmaf.

[0121] It should be noted that this application does not specifically limit the implementation method of the above-mentioned online complete encoding. In one embodiment shown, this application can use H.265 / HEVC to fully encode the encoded video, or can also use any other possible encoding method to fully encode the encoded video based on the selected CRF feature value.

[0122] Fast precoding feature extraction can include fast encoding of the video to be encoded and, based on the fast encoding results, extracting corresponding intra-frame prediction / inter-frame prediction coding information, motion vector information, quantization step size information, peak signal-to-noise ratio (PSNR), prediction mode (or coding mode) information, block partitioning information, the video bitrate and VMAF value of the fast-encoded video, and the video width, height, and frame rate of the video to be encoded. Intra-frame prediction uses the coded information surrounding the pixel block within the same frame to predict the content of the current block. It is applicable to I-frames (intra-coded frames) and facilitates random access and error recovery. Inter-frame prediction uses the similarity between previous and next frames to predict the content of the current frame. It is commonly used in P-frames (forward prediction frames) and B-frames (bidirectional prediction frames), significantly improving compression efficiency. In inter-frame prediction, the motion vector represents the displacement of the current block relative to the corresponding block in the reference frame. It is used to compensate for motion and is key to reducing spatial redundancy in video compression. The size of the quantization step size during the encoding process directly affects the degree of compression and video quality. Among them, block division information and prediction mode information refer to the macroblock (or sub-macroblock) division of the video frame and the selection of the best prediction mode during the encoding process. The encoding process allows block division in different sizes and shapes (such as 16x16, 8x8, 4x4, etc.), as well as multiple prediction modes (such as DC, planar, directional prediction of various angles, etc.) to achieve the best compression effect, etc., which will not be elaborated here.

[0123] It should be noted that this application does not specifically limit the implementation of the above-mentioned fast pre-encoding. In one embodiment, this application can use x264 to quickly encode the video to be encoded. x264 is an open source H.264 / MPEG-4 AVC video encoding library that can efficiently compress the video to be encoded into a smaller video file while maintaining good video quality.

[0124] To sum up, after a series of feature extractions are performed on the video to be encoded, the video features of multiple dimensions (such as dimensions related to video quality and dimensions related to video content, etc.) obtained may include but are not limited to: anchor_crf, anchor_vmaf, anchor_bitrate, video spatial texture features, noise information, non-parameter video quality assessment information, block effect information, blur degree information, video spatiotemporal correlation features, intra-frame prediction / inter-frame prediction encoding information, motion vector information, quantization step information, peak signal-to-noise ratio, mode / block division information, video bit rate and VMAF value of the video after fast encoding, video width and height and video frame rate of the video to be encoded, etc. This application does not make specific limitations on this.

[0125] In some possible implementations, the server may also perform feature extraction on the encoded video by any other possible method, and the multi-dimensional video features may also include any other possible features in addition to the above features, which is not specifically limited in this application.

[0126] Step S202 : Based on the video features of the multiple dimensions, predicting corresponding video quality curves and video bit rate curves when encoding the video to be encoded based on different constant quality factor (CRF) values.

[0127] Furthermore, after the server obtains video features of multiple dimensions related to the video to be encoded through a series of feature extraction, it can predict the corresponding video quality curve and video bit rate curve when the video to be encoded is encoded based on different CRF values ​​based on the video features of multiple dimensions.

[0128] In one embodiment, the video quality may include a VMAF value. It should be noted that conventional peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) can also be used to evaluate video quality. However, considering that both evaluate the pixel-level differences between the to-be-encoded video and the encoded video at a low level, which differs significantly from the subjective perception of the human eye, the present application may preferably use the VMAF value to represent the video quality of the encoded video obtained after encoding the to-be-encoded video.

[0129] In one illustrated embodiment, the server can perform a prediction every 0.2 steps within the CRF value range of 20-40. That is, the server needs to predict the VMAF value and video bit rate of the encoded video when the video to be encoded is encoded under 101 different CRF values ​​(including 20, 20.2, 20.4, ..., 39.8, 40), thereby predicting 101 VMAF values ​​and 101 video bit rates corresponding to these 101 CRF values.

[0130] In one illustrated embodiment, the server may also perform a prediction every 0.5 steps within the CRF value range of 25-40. That is, the server needs to predict the VMAF value and video bit rate of the encoded video when the to-be-encoded video is encoded at 51 different CRF values ​​(including 25, 25.5, 30, ..., 39.5, 40), thereby predicting 51 VMAF values ​​and 51 video bit rates corresponding to the 51 CRF values.

[0131] Furthermore, the server can generate a video quality curve (which can be recorded as a crf-vmaf curve) for describing the changing relationship between CRF and video quality, and a video bitrate curve (which can be recorded as a crf-bitrate curve) for describing the changing relationship between CRF and video bitrate based on the predicted multiple VMAF values ​​and video bitrates (for example, the 101 VMAF values ​​and 101 video bitrates corresponding to the 101 CRF values ​​mentioned above). The x-axis of the crf-vmaf curve can represent the CRF value, and the y-axis can represent the VMAF value; the x-axis of the crf-bitrate curve can represent the CRF value, and the y-axis can represent the bitrate value.

[0132] It should be noted that this application does not specifically limit the specific implementation method of predicting the above-mentioned crf-vmaf curve and crf-bitrate curve.

[0133] In an illustrated embodiment, the server can predict the above-mentioned crf-vmaf curve and crf-bitrate curve through a pre-trained deep learning model. Exemplarily, the server can input the extracted video features of multiple dimensions into the pre-trained deep learning model, and the deep learning model predicts the above-mentioned crf-vmaf curve and crf-bitrate curve based on the video features of multiple dimensions. Exemplarily, the deep learning model can first predict the corresponding video quality and video bit rate when the video to be encoded is encoded based on different CRF values, and then generate the corresponding crf-vmaf curve and crf-bitrate curve based on the predicted multiple video qualities and video bit rates. For example, still taking the predicted multiple VMAF values ​​and video bit rates including 101 VMAF values ​​and 101 video bit rates corresponding to 101 CRF values ​​as an example, the deep learning model can fit 101 discrete points consisting of 101 CRF values ​​and their corresponding 101 VMAF values ​​into a crf-vmaf curve; and, fit 101 discrete points consisting of 101 CRF values ​​and their corresponding 101 video bit rates into a crf-bitrate curve. This application does not make specific limitations on this.

[0134] It should be noted that this application does not specifically limit the specific type and internal structure of the above-mentioned deep learning model.

[0135] In one illustrated embodiment, see Figure 3 , Figure 3 This is a schematic diagram of the structure of a deep learning model provided by an exemplary embodiment. Figure 3 As shown, the deep learning model can be connected to the feature extraction module used to implement the method flow in the above step S201.

[0136] like Figure 3 As shown, the feature extraction module can include video content feature extraction, online complete encoding feature extraction and fast pre-encoding feature extraction, and output the extracted video features of multiple dimensions. For details, please refer to the description in the above step S201, which will not be repeated here.

[0137] like Figure 3 As shown, the deep learning model may include a first base-net, a suspension operator, and a second base-net connected in sequence.

[0138] In an illustrated embodiment, the video features of multiple dimensions output by the feature extraction module can be input into the first basic network, and the first basic network predicts the crf-vmaf curve and crf-bitrate curve based on the video features of multiple dimensions, that is, Figure 3 Pred 1pass vmaf / bitrate curve shown.

[0139] In one illustrated embodiment, the server can first concatenate the multiple dimensional video features output by the feature extraction module to obtain the corresponding target features, and then input the target features into the first basic network. The first basic network predicts the above-mentioned crf-vmaf curve and crf-bitrate curve based on the target features. It should be understood that feature concatenation can integrate features from different sources or different types to obtain a more comprehensive feature. At the same time, it can allow the network to learn the interaction and combination effects between features to improve the prediction performance of the model, etc.

[0140] like Figure 3 As shown, the first basic network can include a batch normalization layer, a self-attention mechanism layer, a residual feature layer and a fully connected layer. Figure 3 The internal structure of the first basic network is shown, and the process of predicting the crf-vmaf curve and the crf-bitrate curve by the first basic network is described.

[0141] First, the target feature obtained by concatenating the video features of multiple dimensions is input to the batch normalization layer. The batch normalization layer performs batch normalization on the video features of multiple dimensions contained in the target feature, and outputs the normalized multi-dimensional video features. It should be understood that normalization can convert features of different scales to the same scale. For example, the values ​​of video features of multiple dimensions are adjusted to the range of [0, 1], which facilitates the comparison and combination of different video features.

[0142] Then, the normalized video features of multiple dimensions are input into a self-attention mechanism layer, which extracts correlation information between the video features of multiple dimensions and integrates the video features of multiple dimensions into deep features based on the correlation information. In one illustrated embodiment, the correlation information between the video features of multiple dimensions may include correlation coefficients between the video features of multiple dimensions.

[0143] Then, the depth features output from the self-attention mechanism layer are input to the residual feature layer, and the residual feature layer performs residual correction on the depth features to obtain residual features. Figure 3As shown, the residual feature layer can include two residual blocks (res-block). It should be understood that the provision of residual blocks can help the network more easily learn the residual, that is, the difference between the output and the input, promote deeper feature learning, and effectively solve the common gradient vanishing problem in deep networks, etc. This application does not make specific limitations on this.

[0144] Finally, the residual features output by the residual feature layer are input to the fully connected layer, and the fully connected layer predicts the crf-vmaf curve and crf-bitrate curve based on the residual features and outputs them.

[0145] In this way, the present application uses a combination of multi-dimensional video features such as coding information and quality information as the input of the first basic network, and at the same time uses reasonable normalization methods and self-attention mechanisms in the first basic network. Network design can maximize network performance and improve the prediction accuracy of the crf-vmaf curve and the crf-bitrate curve.

[0146] As described above, in the feature extraction process described in step S201 of the present application, a complete encoding of the video to be encoded has been performed based on anchor_crf, and the corresponding anchor_vmaf and anchor_bitrate have been obtained. It is not difficult to understand that the real crf-vmaf curve corresponding to the video to be encoded must pass through the first anchor point (anchor_crf, anchor_vmaf) composed of anchor_crf and anchor_vmaf. Correspondingly, the real anchor_bitrate curve corresponding to the video to be encoded must also pass through the second anchor point (anchor_crf, anchor_bitrate) composed of anchor_crf and anchor_bitrate.

[0147] Based on this, Figure 3 As shown, the present application further provides a suspension operator connected to the above-mentioned first basic network in the deep learning model, through which the crf-vmaf curve and crf-bitrate curve predicted by the first basic network can be further suspended based on the anchor_crf, anchor_vmaf and anchor_bitrate in the above-mentioned video features to obtain the crf-vmaf curve and crf-bitrate curve after suspension processing, that is, Figure 3 Pred1pass VMAF / bitrate hanging curve shown.

[0148] Among them, the crf-vmaf curve after hanging processing passes through the first anchor point (anchor_crf, anchor_vmaf), and the crf-bitrate curve after hanging processing passes through the second anchor point (anchor_crf, anchor_bitrate), thereby improving the prediction accuracy of the deep learning model and making the predicted curve closer to the real curve, that is, Figure 3 The label vmaf / bitrate curve shown.

[0149] In an illustrated embodiment, the suspension processing may specifically include: performing an offset operation on the overall vmaf value of the predicted crf-vmaf curve, wherein the offset value may be equal to anchor_vmaf minus the vmaf value corresponding to anchor_crf in the crf-vmaf curve; and performing an offset operation on the overall bitrate value of the predicted crf-bitrate curve, wherein the offset value may be equal to anchor_bitrate minus the bitrate value corresponding to anchor_crf in the crf-bitrate curve.

[0150] In this way, after the suspension process, the crf-vmaf curve must pass through the first anchor point (anchor_crf, anchor_vmaf), and the crf-bitrate curve must pass through the second anchor point (anchor_crf, anchor_bitrate).

[0151] For example, assuming that anchor_crf=32, anchor_vmaf=92, and the vmaf value corresponding to anchor_crf in the crf-vmaf curve is 91, the suspension processing of the crf-vmaf curve may include: performing an offset operation with an offset value equal to 1 on the overall vmaf value of the crf-vmaf curve, that is, increasing the overall vmaf value of the crf-vmaf curve by 1 unit.

[0152] For example, assuming that anchor_crf=32, anchor_bitrate=800, and the bitrate value corresponding to anchor_crf in the crf-bitrate curve is 760, the suspension processing of the crf-bitrate curve may include: performing an offset operation with an offset value equal to 40 on the overall bitrate value of the crf-bitrate curve, that is, increasing the overall bitrate value of the crf-bitrate curve by 40 units.

[0153] In an embodiment shown, in order to further improve the prediction accuracy of the deep learning model, as shown in FIG. Figure 3 As shown, the present application further provides a second basic network connected to the above-mentioned suspension operator in the deep learning model. The second basic network can perform residual correction on the crf-vmaf curve and crf-bitrate curve after suspension processing to obtain the residual-corrected crf-vmaf curve and crf-bitrate curve, that is, Figure 3 Pred 2pass vmaf / bitrate curve shown.

[0154] It should be noted that the present application does not specifically limit the specific structure of the second basic network. In one embodiment, the second basic network can perform residual learning based on skip connections to enable the model to have better prediction performance, etc., which is not specifically limited in the present application.

[0155] See also Figure 4 , Figure 4 FIG. 1 is a schematic diagram of a video quality curve provided by an exemplary embodiment. Figure 4 As shown, curve ① is the basic crf-vmaf curve predicted by the first basic network in the deep learning model, curve ② is the crf-vmaf curve after the basic crf-vmaf curve is suspended by the suspension operator in the deep learning model, curve ③ is the crf-vmaf curve after the residual correction of the crf-vmaf curve after the suspension processing by the second basic network in the deep learning model, and curve ④ can be the true value, that is, the true crf-vmaf curve corresponding to the video to be encoded. For example, Figure 4 The curve ④ shown may be a curve formed by a plurality of corresponding VMAF true values ​​after the to-be-encoded video is encoded multiple times using different CRF values.

[0156] See also Figure 5 , Figure 5 FIG. 1 is a schematic diagram of a video bit rate curve provided by an exemplary embodiment. Figure 5 As shown, curve ⑤ is the basic crf-bitrate curve predicted by the first basic network in the deep learning model, curve ⑥ is the crf-bitrate curve after the basic crf-bitrate curve is suspended by the suspension operator in the deep learning model, and curve ⑦ is the crf-bitrate curve after the residual correction of the crf-bitrate curve after the suspension processing by the second basic network in the deep learning model. Curve ⑧ can be the true value, that is, the true crf-bitrate curve corresponding to the video to be encoded. For example, Figure 5The curve ⑧ shown may be a curve formed by multiple bitrate true values ​​corresponding to multiple encodings of the to-be-encoded video using different CRF values.

[0157] like Figure 4 and Figure 5 As shown, the basic crf-vmaf curve and crf-bitrate curve predicted only by the first basic network in the model do not necessarily pass through (anchor_crf, anchor_vmaf) and (anchor_crf, anchor_bitrate), and the crf-vmaf curve and crf-bitrate curve after the suspension processing must pass through (anchor_crf, anchor_vmaf) and (anchor_crf, anchor_bitrate). At the same time, the crf-vmaf curve and crf-bitrate curve after further residual correction also pass through (anchor_crf, anchor_vmaf) and (anchor_crf, anchor_bitrate).

[0158] like Figure 4 and Figure 5 As shown in the figure, based on the crf-vmaf curve and crf-bitrate curve predicted by the first basic network, further suspension processing and residual correction can make the crf-vmaf curve and crf-bitrate curve predicted by the model closer and closer to the true value, and the curve prediction accuracy is improved successively.

[0159] Step S203: Obtain a CRF target value selected from the video quality curve and the video bit rate curve that matches the video quality and video bit rate required for video encoding of the video to be encoded.

[0160] Furthermore, after the suspension processing and residual correction, the final crf-vmaf curve and crf-bitrate curve are predicted, and then the CRF target value that matches the video quality and video bit rate required for video encoding of the video to be encoded can be selected from the crf-vmaf curve and crf-bitrate curve according to actual needs.

[0161] It should be noted that this application does not specifically limit the specific implementation method of selecting the CRF target value from the crf-vmaf curve and the crf-bitrate curve.

[0162] In one illustrated embodiment, a CRF target value can be manually selected from the crf-vmaf curve and the crf-bitrate curve. For example, the server can output the predicted crf-vmaf curve and crf-bitrate curve to the platform staff through the corresponding interface. Accordingly, the staff can select a suitable CRF target value from the crf-vmaf curve and the crf-bitrate curve based on actual needs. For example, the staff can enter the corresponding CRF target value in the input box provided in the interface, or can directly click on the corresponding CRF target value in the crf-vmaf curve or the crf-bitrate curve, etc. This application does not make specific limitations on this. Accordingly, the server can obtain the CRF target value selected by the staff from the crf-vmaf curve and the crf-bitrate curve based on the input operation of the staff.

[0163] For example, if the current demand of the platform is to improve the image quality as much as possible, the staff can give priority to the CRF value range corresponding to the high VMAF value (for example, 92-95) in the crf-vmaf curve, and further consider whether the bitrate value range corresponding to the CRF value range in the crf-bitrate curve is reasonable, so as to select a suitable CRF target value from the CRF value range, thereby achieving the goal of improving the image quality as much as possible while avoiding bit rate waste.

[0164] For example, if the current demand of the platform is to reduce the bitrate as much as possible to save costs, the staff can give priority to the CRF value range corresponding to the low bitrate value (for example, 500bit / s-800bit / s) in the crf-bitrat curve, and further consider whether the vmaf value range corresponding to the CRF value range in the crf-vmaf curve is reasonable, so as to select a suitable CRF target value from the CRF value range, thereby achieving the goal of reducing the bitrate as much as possible while avoiding a serious degradation of video quality.

[0165] In one embodiment, the staff can also combine the crf-vmaf curve and the crf-bitrate curve to determine the gradient information between vmaf and bitrate, such as how much bitrate is increased by increasing vmaf by one unit, how much bitrate is saved by reducing vmaf by one unit, etc. In this way, the gradient information between vmaf and bitrate can be used as a reliable reference item to assist users in designing quality improvement or cost reduction strategies to maximize benefits.

[0166] In an illustrated embodiment, the server may also automatically select an appropriate CRF target value based on the predicted crf-vmaf curve and crf-bitrate curve.

[0167] In one illustrated embodiment, the server can first determine the video quality value range and video bitrate range required for encoding the video to be encoded, namely, the VMAF value range and the bitrate value range. Furthermore, the server can determine a first CRF value range corresponding to the VMAF value range from the CRF-VMAF curve, and a second CRF value range corresponding to the bitrate value range from the CRF-bitrate curve. Furthermore, the server can select a CRF target value from the intersection of the first and second CRF value ranges.

[0168] For example, the VMAF value range can be 87-93. Generally speaking, if the VMAF value of the encoded video is lower than 87, the video may have a low bitrate, resulting in a serious degradation of video quality. If the VMAF value of the transcoded video is higher than 93, the video may have a high bitrate, that is, a serious waste of bitrate, which wastes bandwidth and increases unnecessary costs. Accordingly, the first CRF value range corresponding to the VMAF value range of 87-93 determined from the CRF-VMAF curve can be 31-37.5.

[0169] Exemplarily, the bitrate value range may be 750 bit / s-1200 bit / s. Accordingly, the second CRF value range corresponding to the bitrate value range of 600 bit / s-1200 bit / s determined from the CRF-bitrate curve may be 28-33.5.

[0170] For example, the intersection of the first CRF value range (31-37.5) and the second CRF value range (28-33.5) is 31-33.5. Further, the server can select a CRF target value from the CRF value range of 31-33.5 according to the preset actual needs. For example, assuming that the preset actual need is to improve image quality, the server can determine the CRF target value to be 33 or 33.5, etc.; assuming that the preset actual need is to reduce costs, the server can determine the CRF target value to be 31 or 31.5, etc. This application does not make specific limitations on this.

[0171] In one illustrated embodiment, the server may also first determine the target video quality and target video bitrate required for encoding the video to be encoded, namely, the VMAF target value and the bitrate target value. Furthermore, the server may determine a first CRF value corresponding to the VMAF target value from the CRF-VMAF curve; and a second CRF value corresponding to the bitrate target value from the CRF-bitrate curve. Furthermore, the server may determine an appropriate CRF target value based on the first and second CRF values.

[0172] For example, the average value between the first CRF value and the second CRF value can be directly determined as the CRF target value. Assuming that the first CRF value and the second CRF value are 30 and 33 respectively, the average value 31.5 can be determined as the CRF target value.

[0173] Exemplarily, the weight of the first CRF value corresponding to the above-mentioned VMAF target value and the weight of the second CRF value corresponding to the above-mentioned bitrate target value can also be determined according to preset actual needs, and a weighted average of the first CRF value and the second CRF value is calculated based on the weights of the two, and the weighted average is determined as the CRF target value.

[0174] For example, assuming that the preset actual demand is to improve image quality, the server can determine that the weight of the first CRF value is 0.6 and the weight of the second CRF value is 0.4. Correspondingly, still taking the first CRF value and the second CRF value as 30 and 33 respectively as an example, the calculated weighted average value can be 31.2, that is, the selected CRF target value is 31.2.

[0175] For example, assuming that the preset actual demand is to reduce costs, the server can determine that the weight of the first CRF value is 0.4 and the weight of the second CRF value is 0.6. Accordingly, still taking the first CRF value and the second CRF value as 30 and 33 respectively as an example, the calculated weighted average value can be 31.2, that is, the selected CRF target value is 31.8.

[0176] It should be noted that in addition to the manual and automatic methods of selecting the CRF target value described above, this application can also select a suitable CRF target value from the predicted crf-vmaf curve and crf-bitrate curve by any other possible method, and this application does not make specific limitations on this.

[0177] Step S204: encoding the video to be encoded based on the CRF target value to obtain a target video.

[0178] Furthermore, the server can perform video encoding on the video to be encoded based on the selected CRF target value to obtain the encoded target video. It should be noted that the specific implementation method of video encoding cannot be specifically limited in this application. In one embodiment shown, the server can use any possible encoding method such as H.264 / AVC, H.265 / HEVC, etc. to perform video encoding on the video to be encoded based on the CRF target value.

[0179] Furthermore, the server can send the encoded target video to the client connected to it (for example Figure 1 Any one of the multiple clients 200 shown), the client outputs and displays the target video to the user through a corresponding interface (such as a video browsing interface, etc.), etc., and this application does not make specific limitations on this.

[0180] Below, the model training process of the above-mentioned deep learning model will be explained. It should be noted that this application does not specifically limit the training process of the deep learning model.

[0181] In one embodiment, the present application can perform supervised training on the deep learning model through a large number of training samples and corresponding sample labels. Figure 6 , Figure 6 This is a flow chart of a model training method provided by an exemplary embodiment. Figure 6 As shown, the method may specifically include the following steps S601 to S603.

[0182] Step S601: Acquire multiple sample videos, perform feature extraction on each of the multiple sample videos, and obtain video features of multiple dimensions related to each sample video.

[0183] In an illustrated embodiment, the present application may first obtain multiple sample videos, and perform feature extraction on each of the multiple sample videos to obtain video features of multiple dimensions related to each sample video. The specific implementation of feature extraction can refer to the description in step S201 above and will not be repeated here. In addition, it should be noted that the present application does not specifically limit the specific type and content of the sample videos. In an illustrated embodiment, the multiple sample videos can be multiple short videos or long videos with different contents that have been published online, etc.

[0184] Step S602 : performing video encoding on each sample video multiple times based on multiple preset CRF values ​​to obtain multiple encoded videos corresponding to each sample video, and obtaining the video quality and video bit rate of the multiple encoded videos.

[0185] Then, the present application can perform multiple video encoding on each sample video based on multiple preset CRF values ​​to obtain multiple encoded videos corresponding to each sample video, and obtain the video quality and video bit rate of the multiple encoded videos corresponding to each sample video.

[0186] For example, this application can obtain 10,000 online videos and perform feature extraction on each of these 10,000 videos to obtain multi-dimensional video features corresponding to each video. Furthermore, within the CRF value range of 20-40, each video is encoded every 0.2 steps, that is, each video is encoded based on CRF values ​​of 20, 20.2, 20.4, ..., 39.8, and 40, and finally 101 encoded videos corresponding to each video are obtained, that is, a total of 1.01 million encoded videos are obtained. Furthermore, the bitrate information and vmaf information of these 1.01 million encoded videos are extracted.

[0187] In step S603, the deep learning model is supervisedly trained using the video features of multiple dimensions related to each video sample as training samples and the video quality and video bit rate of multiple encoded videos corresponding to each sample video as sample labels.

[0188] Then, the present application can use the video features of multiple dimensions related to each sample video as training samples (i.e., the input data of the model), and use the bitrate information and VMAF information of multiple encoded videos corresponding to each sample video (for example, 101 (bitrate, VMAF) point pairs corresponding to 101 CRF values) as sample labels to perform supervised training on the deep learning model.

[0189] In one illustrated embodiment, the supervised training process may include: using the L1 loss function to supervise the crf-vmaf curve and crf-bitrate curve output by the first basic network in the deep learning model and the difference between each point and the label point pair in the crf-vmaf curve and crf-bitrate curve output by the second basic network, and at the same time using the L1loss loss function to supervise the gradient difference, the gradient can be defined as the vmaf_diff value of the adjacent crf point divided by the bitrate_diff value, the vmaf_diff value is the difference between the two vamaf values ​​corresponding to the adjacent crf points, the bitrate_diff value is the difference between the two bitrate values ​​corresponding to the adjacent crf points, and so on. This application does not make any specific limitations on this.

[0190] Finally, the trained deep learning model can be used to predict the crf-vmaf curve and crf-bitrate curve corresponding to the video to be encoded based on the input video features of multiple dimensions related to the video to be encoded based on different CRF values.

[0191] Corresponding to the above method flow, the embodiment of the present application also provides a video encoding device. Figure 7 , Figure 7 FIG. 1 is a schematic diagram of a video encoding device provided by an exemplary embodiment. Figure 7 As shown, the device 70 includes:

[0192] A feature extraction unit 701 is configured to extract features from the video to be encoded, and obtain video features of multiple dimensions related to the video to be encoded;

[0193] A prediction unit 702 is configured to predict, based on the video features of the multiple dimensions, corresponding video quality curves and video bit rate curves when encoding the video to be encoded based on different CRF values;

[0194] An acquiring unit 703 is configured to acquire a CRF target value selected from the video quality curve and the video bit rate curve, the CRF target value matching the video quality and the video bit rate required for video encoding of the video to be encoded;

[0195] The encoding unit 704 is configured to perform video encoding on the to-be-encoded video based on the CRF target value to obtain a target video.

[0196] In one illustrated embodiment, the video quality includes a VMAF value, a multi-method fusion evaluation index of video quality based on subjective perception of the human eye; the video features in the multiple dimensions include a CRF feature value, a VMAF feature value, and a video bitrate feature value; and the feature extraction unit 701 is specifically configured to:

[0197] Performing video encoding on the video to be encoded based on the selected CRF feature value to obtain an encoded video;

[0198] A VMAF value and a video bit rate of the encoded video are obtained, and the VMAF value and the video bit rate are determined as the VMAF characteristic value and the video bit rate characteristic value corresponding to the CRF characteristic value.

[0199] In an illustrated embodiment, the prediction unit 702 is specifically configured to:

[0200] The video features of the multiple dimensions are input into a pre-trained deep learning model, and the deep learning model predicts the corresponding video quality curve and video bit rate curve when the video to be encoded is encoded based on different CRF values ​​based on the video features of the multiple dimensions.

[0201] In one illustrated embodiment, the deep learning model includes a first basic network; the prediction unit 702 is specifically configured to:

[0202] Perform feature splicing on the video features of the multiple dimensions to obtain target features;

[0203] The target feature is input into the first basic network, and the first basic network predicts, based on the target feature, a video quality curve and a video bit rate curve corresponding to when the video to be encoded is encoded based on different CRF values.

[0204] In one illustrated embodiment, the first basic network includes a batch normalization layer, a self-attention mechanism layer, a residual feature layer, and a fully connected layer connected in sequence; the prediction unit 702 is specifically configured to:

[0205] Inputting the target feature into a batch normalization layer, wherein the batch normalization layer performs batch normalization processing on the video features of multiple dimensions contained in the target feature;

[0206] Inputting the normalized video features of the multiple dimensions into a self-attention mechanism layer, the self-attention mechanism layer extracts correlation information between the video features of the multiple dimensions, and integrating the video features of the multiple dimensions into deep features based on the correlation information;

[0207] Inputting the depth feature into a residual feature layer, and performing residual correction on the depth feature by the residual feature layer to obtain a residual feature;

[0208] The residual features are input into a fully connected layer, and the fully connected layer predicts, based on the residual features, a video quality curve and a video bit rate curve corresponding to when the video to be encoded is encoded based on different CRF values.

[0209] In an illustrated embodiment, the deep learning model further includes a suspension operator, and the apparatus 70 further includes a suspension processing unit 705 configured to:

[0210] Inputting the video quality curve and the video bit rate curve predicted by the first basic network into the suspension operator, and having the suspension operator perform suspension processing on the video quality curve based on the CRF feature value and the VMAF feature value, so that the video quality curve after suspension processing passes through a first anchor point formed by the CRF feature value and the VMAF feature value; and

[0211] The video bit rate curve is suspended based on the CRF feature value and the video bit rate feature value, so that the video bit rate curve after the suspension process passes through a second anchor point formed by the CRF feature value and the video bit rate feature value.

[0212] In an illustrated embodiment, the deep learning model further includes a second basic network, and the apparatus 70 further includes a residual correction unit 706 for:

[0213] Inputting the video quality curve and the video bit rate curve after the suspension processing into the second basic network, and performing residual correction on the video quality curve and the video bit rate curve after the suspension processing by the second basic network to obtain the residual-corrected video quality curve and the video bit rate curve;

[0214] The acquiring unit 703 is specifically configured to:

[0215] A CRF target value is obtained, which is selected from the residual-corrected video quality curve and the video bit rate curve and matches the VMAF value and video bit rate required for video encoding of the video to be encoded.

[0216] In an illustrated embodiment, the obtaining unit 703 is specifically configured to:

[0217] Determining a first CRF value range corresponding to a video quality value range required for video encoding of the video to be encoded from the video quality curve; and

[0218] Determining a second CRF value range from the video bit rate curve corresponding to a video bit rate value range required for video encoding of the video to be encoded;

[0219] A CRF target value is selected from the intersection of the first CRF value range and the second CRF value range.

[0220] In an illustrated embodiment, the apparatus 70 further includes a model training unit 707 configured to:

[0221] Acquire multiple sample videos, perform feature extraction on each of the multiple sample videos, and obtain video features of multiple dimensions related to each sample video;

[0222] Performing video encoding on each sample video multiple times based on multiple preset CRF values ​​to obtain multiple encoded videos corresponding to each sample video, and obtaining video quality and video bit rate of the multiple encoded videos;

[0223] The deep learning model is supervisedly trained using video features of multiple dimensions related to each video sample as training samples, and video quality and video bit rate of multiple encoded videos corresponding to each sample video as sample labels.

[0224] In an illustrated embodiment, the video features of the multiple dimensions include one or more combinations of the following: video spatial texture features, video spatiotemporal correlation features, noise information, non-parametric video quality assessment information, blocking effect information, and blur level information;

[0225] The feature extraction unit 701 is specifically configured to:

[0226] Extracting the video spatial domain texture features of the video to be encoded based on the gray level co-occurrence matrix; and / or,

[0227] Extracting quality information of the video to be encoded based on a preset quality assessment algorithm, where the quality information includes one or more of noise information, non-parametric video quality assessment information, blocking effect information, and blur level information; and / or,

[0228] The video spatiotemporal correlation features of the video to be encoded are extracted based on a preset spatiotemporal domain algorithm.

[0229] In an illustrated embodiment, the video features in multiple dimensions include one or more combinations of the following: video width and height, video frame rate, motion vector information, quantization step size information, peak signal-to-noise ratio, intra-frame prediction / inter-frame prediction encoding information, block partitioning information, video bit rate, and VMAF value;

[0230] The feature extraction unit 701 is specifically configured to:

[0231] The video to be encoded is quickly encoded, and one or more information of video width and height, video frame rate, motion vector information, quantization step information, peak signal-to-noise ratio, intra-frame prediction / inter-frame prediction encoding information, block partitioning information, video bit rate and VMAF value are extracted according to the result of the quick encoding.

[0232] Corresponding to the above method flow, the embodiment of the present application also provides a model training device. Figure 8 , Figure 8 FIG. 1 is a schematic diagram of a model training device provided by an exemplary embodiment. Figure 8As shown, the device 80 includes:

[0233] The feature extraction unit 801 is used to obtain multiple sample videos, perform feature extraction on each of the multiple sample videos, and obtain video features of multiple dimensions related to each sample video;

[0234] An acquiring unit 802 is configured to perform video encoding on each sample video multiple times based on multiple preset CRF values ​​to obtain multiple encoded videos corresponding to each sample video, and to obtain video quality and video bit rate of the multiple encoded videos;

[0235] The training unit 803 is used to perform supervised training on the deep learning model using the video features of multiple dimensions related to each video sample as training samples, and using the video quality and video bit rate of multiple encoded videos corresponding to each sample video as sample labels; wherein the trained deep learning model is used to predict the video quality curve and video bit rate curve corresponding to the video to be encoded based on different CRF values ​​when the video to be encoded is encoded based on the input video features of multiple dimensions related to the video to be encoded.

[0236] The implementation process of the functions and effects of each unit in the above-mentioned device 70 and device 80 is specifically described in detail in the description of the above-mentioned embodiment, and will not be repeated here. It should be understood that the above-mentioned device 70 and device 80 can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a device in the logical sense, it is formed by the processor (CPU) of the device where it is located reading the corresponding computer program instructions into the memory and running them. From the hardware level, in addition to the CPU and memory, the device where the above-mentioned device is located usually also includes other hardware such as chips for wireless signal transmission and reception, and / or other hardware such as boards and cards for implementing network communication functions.

[0237] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical modules, that is, they may be located in one place or distributed across multiple network modules. Some or all of the units or modules may be selected according to actual needs to achieve the purpose of the present application. Those of ordinary skill in the art can understand and implement the present invention without inventive effort.

[0238] The devices, units, and modules described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email transceiver, game console, tablet computer, wearable device, vehicle-mounted computer, or any combination of these devices.

[0239] In some embodiments, an embodiment of the present application further provides a computing device, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor implements any one of the methods described in the above embodiments by running the executable instructions.

[0240] For example, Figure 9 This is a schematic diagram of a computing device provided by an exemplary embodiment. Figure 9 At the hardware level, the device includes a processor 902, an internal bus 904, a network interface 906, a memory 908, and a non-volatile memory 910. Of course, it may also include hardware required for other scenarios. One or more embodiments of the present application can be implemented based on software, such as the processor 902 reading the corresponding computer program from the non-volatile memory 910 into the memory 908 and then running it. Of course, in addition to software implementation, one or more embodiments of the present application do not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0241] In some embodiments, the embodiments of the present application further provide a computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, implement the steps of any of the methods described above.

[0242] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be used to store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory media such as modulated data signals and carrier waves.

[0243] In some embodiments, the embodiments of the present application further provide a computer program product, including a computer program, which implements the steps of any of the methods described above when executed by a processor.

[0244] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0245] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0246] The terms used in one or more embodiments of the present application are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of the present application. The singular forms "a", "the", and "the" used in one or more embodiments of the present application and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0247] The above description is merely a preferred embodiment of one or more embodiments of the present application and is not intended to limit one or more embodiments of the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of one or more embodiments of the present application shall be included in the scope of protection of one or more embodiments of the present application.

Claims

1. A video encoding method, characterized in that: The method comprises: Extracting features from the video to be encoded to obtain video features of multiple dimensions related to the video to be encoded; wherein the video features of the multiple dimensions include a CRF feature value and a VMAF feature value and a video bit rate feature value corresponding to the CRF feature value, the CRF feature value being a feature value of a constant quality factor, and the VMAF feature value being a feature value of a multi-method fusion evaluation index of video quality based on subjective perception of the human eye; Inputting the video features of the multiple dimensions into a pre-trained deep learning model, and having the deep learning model predict, based on the video features of the multiple dimensions, corresponding video quality curves and video bit rate curves when encoding the video to be encoded based on different CRF values; Inputting the predicted video quality curve and the video bit rate curve into a suspension operator of the deep learning model, wherein the suspension operator performs suspension processing on the video quality curve based on the CRF feature value and the VMAF feature value, so that the suspended video quality curve passes through a first anchor point formed by the CRF feature value and the VMAF feature value; and performing suspension processing on the video bit rate curve based on the CRF feature value and the video bit rate feature value, so that the suspended video bit rate curve passes through a second anchor point formed by the CRF feature value and the video bit rate feature value; Obtaining a CRF target value selected from the video quality curve and the video bit rate curve that matches the video quality and video bit rate required for video encoding of the video to be encoded; The video to be encoded is encoded based on the CRF target value to obtain a target video.

2. The method according to claim 1, characterized in that The feature extraction of the video to be encoded is performed to obtain video features of multiple dimensions related to the video to be encoded, including: Performing video encoding on the video to be encoded based on the selected CRF feature value to obtain an encoded video; A VMAF value and a video bit rate of the encoded video are obtained, and the VMAF value and the video bit rate are determined as the VMAF characteristic value and the video bit rate characteristic value corresponding to the CRF characteristic value.

3. The method according to claim 1, characterized in that The deep learning model includes a first basic network; the video features of the multiple dimensions are input into a pre-trained deep learning model, and the deep learning model predicts, based on the video features of the multiple dimensions, corresponding video quality curves and video bit rate curves when encoding the video to be encoded based on different CRF values, including: Perform feature splicing on the video features of the multiple dimensions to obtain target features; The target feature is input into the first basic network, and the first basic network predicts, based on the target feature, a video quality curve and a video bit rate curve corresponding to when the video to be encoded is encoded based on different CRF values.

4. The method according to claim 3, characterized in that The first basic network includes a batch normalization layer, a self-attention mechanism layer, a residual feature layer, and a fully connected layer connected in sequence; the target feature is input into the first basic network, and the first basic network predicts, based on the target feature, corresponding video quality curves and video bit rate curves when encoding the video to be encoded based on different CRF values, including: Inputting the target feature into a batch normalization layer, wherein the batch normalization layer performs batch normalization processing on the video features of multiple dimensions contained in the target feature; Inputting the normalized video features of the multiple dimensions into a self-attention mechanism layer, the self-attention mechanism layer extracts correlation information between the video features of the multiple dimensions, and integrating the video features of the multiple dimensions into deep features based on the correlation information; Inputting the depth feature into a residual feature layer, and performing residual correction on the depth feature by the residual feature layer to obtain a residual feature; The residual features are input into a fully connected layer, and the fully connected layer predicts, based on the residual features, a video quality curve and a video bit rate curve corresponding to when the video to be encoded is encoded based on different CRF values.

5. The method according to claim 4, characterized in that The deep learning model further includes a second basic network, and the method further includes: Inputting the video quality curve and the video bit rate curve after the suspension processing into the second basic network, and performing residual correction on the video quality curve and the video bit rate curve after the suspension processing by the second basic network to obtain the residual-corrected video quality curve and the video bit rate curve; The obtaining of a CRF target value selected from the video quality curve and the video bit rate curve and matching the video quality and video bit rate required for video encoding of the video to be encoded includes: A CRF target value is obtained, which is selected from the residual-corrected video quality curve and the video bit rate curve and matches the VMAF value and video bit rate required for video encoding of the video to be encoded.

6. The method according to claim 1, characterized in that The obtaining of a CRF target value selected from the video quality curve and the video bit rate curve and matching the video quality and video bit rate required for video encoding of the video to be encoded includes: Determining a first CRF value range corresponding to a video quality value range required for video encoding of the video to be encoded from the video quality curve; and Determining a second CRF value range from the video bit rate curve corresponding to a video bit rate value range required for video encoding of the video to be encoded; A CRF target value is selected from the intersection of the first CRF value range and the second CRF value range.

7. The method according to claim 1, characterized in that The method further comprises: Acquire multiple sample videos, perform feature extraction on each of the multiple sample videos, and obtain video features of multiple dimensions related to each sample video; Performing video encoding on each sample video multiple times based on multiple preset CRF values ​​to obtain multiple encoded videos corresponding to each sample video, and obtaining video quality and video bit rate of the multiple encoded videos; The deep learning model is supervisedly trained using video features of multiple dimensions related to each video sample as training samples, and video quality and video bit rate of multiple encoded videos corresponding to each sample video as sample labels.

8. The method according to any one of claims 1 to 7, characterized in that The video features of the multiple dimensions further include one or more combinations of the following: video spatial texture features, video spatiotemporal correlation features, noise information, non-parametric video quality assessment information, block effect information and blur level information; The step of extracting features from the video to be encoded to obtain video features of multiple dimensions related to the video to be encoded further includes: Extracting the video spatial domain texture features of the video to be encoded based on the gray level co-occurrence matrix; and / or, Extracting quality information of the video to be encoded based on a preset quality assessment algorithm, where the quality information includes one or more of noise information, non-parametric video quality assessment information, blocking effect information, and blur level information; and / or, The video spatiotemporal correlation features of the video to be encoded are extracted based on a preset spatiotemporal domain algorithm.

9. The method according to any one of claims 1 to 7, characterized in that: The video features of the multiple dimensions further include one or more combinations of the following: video width and height, video frame rate, motion vector information, quantization step information, peak signal-to-noise ratio, intra-frame prediction / inter-frame prediction encoding information, block partitioning information, video bit rate, and VMAF value; The step of extracting features from the video to be encoded to obtain video features of multiple dimensions related to the video to be encoded further includes: The video to be encoded is quickly encoded, and one or more information of video width and height, video frame rate, motion vector information, quantization step information, peak signal-to-noise ratio, intra-frame prediction / inter-frame prediction encoding information, block partitioning information, video bit rate and VMAF value are extracted according to the result of the quick encoding.

10. A model training method, characterized in that: The method comprises: Acquire multiple sample videos, perform feature extraction on each of the multiple sample videos, and obtain video features of multiple dimensions associated with each sample video; wherein the video features of the multiple dimensions include a CRF feature value and a VMAF feature value and a video bit rate feature value corresponding to the CRF feature value, the CRF feature value being a feature value of a constant quality factor, and the VMAF feature value being a feature value of a multi-method fusion evaluation index of video quality based on subjective perception of the human eye; Performing video encoding on each sample video multiple times based on multiple preset CRF values ​​to obtain multiple encoded videos corresponding to each sample video, and obtaining video quality and video bit rate of the multiple encoded videos; The deep learning model is supervisedly trained using video features of multiple dimensions related to each video sample as training samples, and video quality and video bit rate of multiple encoded videos corresponding to each sample video as sample labels; wherein the trained deep learning model is used to predict the video quality curve and video bit rate curve corresponding to the video to be encoded based on different CRF values ​​based on the input video features of multiple dimensions related to the video to be encoded; and the predicted video quality curve and video bit rate curve are input into the hanging operator of the deep learning model, and the hanging operator hangs the video quality curve based on the CRF feature value and the VMAF feature value, so that the video quality curve after the hanging process passes through the first anchor point formed by the CRF feature value and the VMAF feature value; and the video bit rate curve is hung based on the CRF feature value and the video bit rate feature value, so that the video bit rate curve after the hanging process passes through the second anchor point formed by the CRF feature value and the video bit rate feature value.

11. A video encoding device, characterized in that: The device comprises: A feature extraction unit is configured to extract features from a to-be-encoded video to obtain video features of multiple dimensions related to the to-be-encoded video; wherein the video features of multiple dimensions include a CRF feature value and a VMAF feature value and a video bitrate feature value corresponding to the CRF feature value, wherein the CRF feature value is a feature value of a constant quality factor, and the VMAF feature value is a feature value of a multi-method fusion evaluation index of video quality based on subjective perception of the human eye; A prediction unit is configured to input the video features of the multiple dimensions into a pre-trained deep learning model, and the deep learning model predicts, based on the video features of the multiple dimensions, corresponding video quality curves and video bit rate curves when the video to be encoded is encoded based on different CRF values; input the predicted video quality curve and video bit rate curve into a suspension operator of the deep learning model, and the suspension operator performs suspension processing on the video quality curve based on the CRF feature value and the VMAF feature value, so that the suspended video quality curve passes through a first anchor point formed by the CRF feature value and the VMAF feature value; and perform suspension processing on the video bit rate curve based on the CRF feature value and the video bit rate feature value, so that the suspended video bit rate curve passes through a second anchor point formed by the CRF feature value and the video bit rate feature value; an acquiring unit, configured to acquire a CRF target value selected from the video quality curve and the video bit rate curve and matching the video quality and video bit rate required for video encoding of the video to be encoded; The encoding unit is configured to perform video encoding on the video to be encoded based on the CRF target value to obtain a target video.

12. A model training device, characterized in that: The device comprises: a feature extraction unit configured to obtain a plurality of sample videos, perform feature extraction on each of the plurality of sample videos, and obtain video features of multiple dimensions associated with each sample video; wherein the video features of the multiple dimensions include a CRF feature value and a VMAF feature value and a video bitrate feature value corresponding to the CRF feature value, wherein the CRF feature value is a feature value of a constant quality factor, and the VMAF feature value is a feature value of a multi-method fusion evaluation index of video quality based on subjective perception of the human eye; an acquiring unit, configured to perform video encoding on each sample video multiple times based on a plurality of preset CRF values, obtain a plurality of encoded videos corresponding to each sample video, and obtain video quality and video bit rate of the plurality of encoded videos; A training unit is used to perform supervised training on a deep learning model using video features of multiple dimensions related to each video sample as training samples, and using video quality and video bit rate of multiple encoded videos corresponding to each sample video as sample labels; wherein the trained deep learning model is used to predict the video quality curve and video bit rate curve corresponding to different CRF values ​​when the video to be encoded is encoded based on the input video features of multiple dimensions related to the video to be encoded; and the predicted video quality curve and video bit rate curve are input to the hanging operator of the deep learning model, and the hanging operator hangs the video quality curve based on the CRF feature value and the VMAF feature value, so that the video quality curve after the hanging process passes through a first anchor point formed by the CRF feature value and the VMAF feature value; and the video bit rate curve is hung based on the CRF feature value and the video bit rate feature value, so that the video bit rate curve after the hanging process passes through a second anchor point formed by the CRF feature value and the video bit rate feature value.

13. A computing device, characterized in that include: processor; a memory for storing processor-executable instructions; The processor implements the method according to any one of claims 1 to 10 by running the executable instructions.

14. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the instruction is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

Images