Video coding method and system, electronic equipment and storage medium

Abstract

The invention discloses a video coding method and system, electronic equipment and a storage medium, and relates to the technical field of video coding. The method comprises the following steps: acquiring a current video frame to be coded from a video sequence; based on the current video frame, a video feature set associated with a video scene where the video sequence is located is determined, and video features in the video feature set are used for describing picture features of the current video frame in the video scene; the video feature set is converted into a video feature vector matched with a target model, and the target model is obtained by training based on a video feature vector sample and a corresponding quantization parameter offset sample; predicting the video feature vector by using a target model to obtain a quantization parameter offset of the current video frame; adjusting the quantization parameter of the current video frame by using the quantization parameter offset; and encoding the current video frame by using the adjusted quantization parameter to obtain an encoding result. The technical problem of low efficiency of video coding is solved.

Description

Technical Field

[0001] The present application relates to the field of video encoding technology, and in particular to a video encoding method, system, electronic device and storage medium. Background Art

[0002] Currently, in the video encoding process, a suitable quantization parameter can be selected for an image block in each frame of a video sequence to control a required bit rate.

[0003] In the related art, the rate control performance is usually adjusted. However, this method is mainly used to improve the accuracy of rate control rather than directly improve the coding efficiency. Moreover, this method is mostly designed for standard encoders and lacks application to fast encoders, resulting in low efficiency of video coding.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present application provide a video encoding method, system, electronic device and storage medium to at least solve the technical problem of low efficiency of video encoding.

[0006] According to one aspect of an embodiment of the present application, a video encoding method is provided. The method may include: obtaining a current video frame to be encoded from a video sequence; determining a video feature set associated with a video scene in which the video sequence is located based on the current video frame, wherein the video features in the video feature set are used to describe the picture characteristics of the current video frame in the video scene; converting the video feature set into a video feature vector that matches a target model, wherein the target model is obtained by training based on video feature vector samples and corresponding quantization parameter offset samples; predicting the video feature vector using the target model to obtain a quantization parameter offset of the current video frame; adjusting the quantization parameter of the current video frame using the quantization parameter offset; encoding the current video frame using the adjusted quantization parameter to obtain an encoding result.

[0007] According to another aspect of the embodiment of the present application, a method for determining a model is provided. The method may include: obtaining a video feature vector sample, and obtaining a quantization parameter offset sample corresponding to the video feature vector sample; using the video feature vector sample and the quantization parameter offset sample, training to obtain a target model, wherein the target model is used to predict the video feature vector matching the target model to obtain the quantization parameter offset, the video feature vector is converted from a video feature set associated with the video scene where the video sequence is located, the video features in the video feature set are used to describe the picture characteristics of the current video frame to be encoded in the video sequence in the video scene, the video feature set is determined based on the current video frame, the quantization parameter offset is used to adjust the quantization parameter of the current video frame, and the adjusted quantization parameter is used to encode the current video frame to obtain the encoding result.

[0008] According to another aspect of the embodiment of the present application, a video encoding system is provided. The system may include: an encoder, used to obtain a current video frame to be encoded from a video sequence; calling a pre-analysis module to determine a video feature set associated with a video scene in which the video sequence is located based on the current video frame, wherein the video features in the video feature set are used to describe the picture characteristics of the current video frame in the video scene; converting the video feature set into a video feature vector that matches a target model, wherein the target model is obtained by training based on video feature vector samples and corresponding quantization parameter offset samples; predicting the video feature vector using the target model to obtain a quantization parameter offset of the current video frame; adjusting the quantization parameter of the current video frame using the quantization parameter offset; encoding the current video frame using the adjusted quantization parameter to obtain an encoding result; and a server, used to obtain the encoding result.

[0009] According to another aspect of an embodiment of the present application, an electronic device is also provided. The electronic device may include a memory and a processor: the memory is used to store computer-executable instructions, and the processor is used to execute computer-executable instructions. When the above-mentioned computer-executable instructions are executed by the processor, any one of the above-mentioned methods is implemented.

[0010] According to another aspect of an embodiment of the present application, a processor is further provided, and the processor is used to run a program, wherein any one of the above methods is executed when the program is running.

[0011] According to another aspect of an embodiment of the present application, a computer-readable storage medium is further provided, the computer-readable storage medium including a stored program, wherein when the program is running, the device where the storage medium is located is controlled to execute any of the above methods.

[0012] According to another aspect of an embodiment of the present application, a computer program product is also provided, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, any one of the above methods is implemented.

[0013] In an embodiment of the present application, a current video frame to be encoded is obtained from a video sequence; based on the current video frame, a video feature set associated with a video scene in which the video sequence is located is determined, wherein the video features in the video feature set are used to describe the picture characteristics of the current video frame in the video scene; the video feature set is converted into a video feature vector that matches a target model, wherein the target model is trained based on video feature vector samples and corresponding quantization parameter offset samples; the video feature vector is predicted using the target model to obtain a quantization parameter offset of the current video frame; the quantization parameter of the current video frame is adjusted using the quantization parameter offset; the current video frame is encoded using the adjusted quantization parameter to obtain an encoding result. That is to say, the embodiment of the present application extracts a video feature set of the current video frame, converts the extracted video feature set into a video feature vector that matches the target model, accurately predicts a quantization parameter offset based on the video feature vector, and further adjusts the quantization parameter based on the quantization parameter offset according to actual conditions, so as to encode the current video frame using the adjusted quantization parameter to obtain an encoding result. Since the quantization parameter offset is predicted based on the target model to adjust the quantization parameter, it avoids determining a quantization parameter that meets the conditions through multiple traversals and tests, thereby achieving the technical effect of improving the efficiency of video encoding and solving the technical problem of low efficiency of video encoding.

[0014] It is easy to notice that the above general description and the following detailed description are only for the purpose of exemplifying and explaining the present application, and do not constitute a limitation of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0016] Figure 1 is a schematic diagram of an application scenario of a method for determining a video encoding and model according to an embodiment of the present application;

[0017] Figure 2 is a flowchart of a video encoding method according to an embodiment of the present application;

[0018] Figure 3 is a flow chart of a method for determining a model according to an embodiment of the present application;

[0019] Figure 4 is a schematic diagram of a video encoding system according to an embodiment of the present application;

[0020] Figure 5 is a flow chart of a model training according to an embodiment of the present application;

[0021] Figure 6 is a flowchart of video encoding according to an embodiment of the present application;

[0022] Figure 7 It is a structural block diagram of a computing environment of a method for determining a video encoding and model according to an embodiment of the present application;

[0023] Figure 8 is a schematic diagram of a video encoding device according to an embodiment of the present application;

[0024] Fig. 9 is a schematic diagram of a device for determining a model according to an embodiment of the present application;

[0025] Fig.10 is a structural block diagram of a computer terminal according to an embodiment of the present application;

[0026] Fig.11 is a block diagram of an electronic device for a video encoding and model determination method according to an embodiment of the present application;

[0027] Fig.12 It is a hardware structure block diagram of a computer terminal (or mobile device) for implementing a method for determining a video encoding and model according to an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0030] It should also be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application, for example, the data for verification, are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0031] First, some nouns or terms that appear in the description of the embodiments of the present application are subject to the following explanations:

[0032] Quantization Parameter (QP) is used to derive the quantization step size, which can determine the fineness of intra-frame / inter-frame prediction residual quantization. The quantization parameter will affect the quality of the compressed picture;

[0033] Bit rate, the number of encoded bits per unit time, that is, the ratio of the number of bits in the video stream to the playback time, can also be called bit rate. For the same original video, the lower the encoded bit rate, the more significant the compression effect, and the more effective it is in removing information redundancy;

[0034] Coding Tree Unit (CTU) is the basic unit used to encode each frame of video. In H.266 / Versatile Video Coding (VVC), it usually takes 128*128 pixels. Most of the encoding processes (such as intra / inter prediction, discrete transform, quantization and entropy coding) are performed in CTU. H.266 / VVC is a new generation of video coding standards.

[0035] The temporal identifier (TID), also known as the temporal level, is used to determine the reference relationship between frames in inter-frame coding. A TID of 0 indicates the highest level, and a larger number indicates a lower level. Frames at a lower level can refer to frames at a higher level, but frames at a higher level cannot refer to frames at a lower level.

[0036] The above-mentioned video coding and model determination method provided in the embodiment of the present application can be applied to the following examples: Figure 1 The application scenarios shown are not limited to these. Figure 1 is a schematic diagram of an application scenario of a method for determining a video encoding and model according to an embodiment of the present application. Figure 1 In the application scenario shown, the server 10 can be a cloud. The server 10 can be connected to one or more client devices 20 via a local area network connection, a wide area network connection, an Internet connection, or other types of data networks. The client devices 20 here may include but are not limited to: smart phones, tablet computers, laptops, PDAs, personal computers, smart home devices, vehicle-mounted devices, etc. The client devices together constitute the client relative to the server. An operating interface for obtaining a video sequence can be deployed on the graphical user interface of the client device. The client device 20 can interact with the user through the graphical user interface to implement the video encoding and model determination method provided in the embodiment of the present application.

[0037] In the embodiment of the present application, the system composed of the client device 20 and the server 10 can perform the following steps: the corresponding operation can be performed in the operation interface on the client device 20 to obtain the video sequence. The client device can obtain the video sequence and send it to the server through the network. After receiving the video sequence, the server can perform the following steps: step S102, obtain the current video frame to be encoded from the video sequence; step S104, based on the current video frame, determine the video feature set associated with the video scene where the video sequence is located, wherein the video features in the video feature set are used to describe the picture characteristics of the current video frame in the video scene; step S106, convert the video feature set into a video feature vector matching the target model, wherein the target model is obtained by training based on the video feature vector sample and the corresponding quantization parameter offset sample; step S108, predict the video feature vector using the target model to obtain the quantization parameter offset of the current video frame; step S110, adjust the quantization parameter of the current video frame using the quantization parameter offset; step S112, encode the current video frame using the adjusted quantization parameter to obtain the encoding result.

[0038] Under the above operating environment, this application provides Figure 2 The video encoding method shown. Figure 2is a flowchart of a video encoding method according to an embodiment of the present application, such as Figure 2 As shown, the method may include the following steps:

[0039] Step S202: obtaining a current video frame to be encoded from a video sequence.

[0040] In the technical solution provided in the above step S202 of the present application, the current video frame to be encoded can be obtained from the video sequence. The video sequence can be a video frame sequence composed of a series of continuous video frames arranged in time order, and the video sequence can include video content, such as video content such as movie clips, which is only for example and does not specifically limit the video content of the video sequence.

[0041] In this embodiment, during the video encoding process, a video sequence is input into an encoder, and the encoder processes each video frame in the video sequence. The current video frame may be a video frame currently processed by the encoder. The encoder may determine a method for encoding the current video frame based on content features of the video sequence (e.g., motion, texture complexity, etc.) and encoding parameters.

[0042] Optionally, after acquiring the video sequence, the acquired video sequence may be input into an encoder, and the encoder performs steps such as reading, parsing, buffering, and encoding on the video sequence. After the encoder parses the video sequence, the parsed video frames may be buffered into a buffer, and further, the video frames may be acquired from the buffer as the current video frame to be encoded, so as to perform encoding operations on the current video frame.

[0043] Step S204: determining a video feature set associated with the video scene where the video sequence is located based on the current video frame.

[0044] In the technical solution provided in the above step S204 of the present application, after obtaining the current video frame to be encoded from the video sequence, a video feature set associated with the video scene in which the video sequence is located can be determined based on the obtained current video frame. The video scene can be a picture scene contained in the video sequence, for example, a motion picture scene, a still picture scene, a texture smooth picture scene, a complex picture scene, etc. This is only for example description, and no specific limitation is imposed on the type of the video scene.

[0045] In this embodiment, the video feature set may include multiple video features related to the video scene extracted from the current video frame. The video features in the video feature set may be used to describe the picture characteristics of the current video frame in the video scene. The video features may include intra-frame features and inter-frame features. For example, the intra-frame feature may be the mean intra-frame prediction cost of the current video frame, and the inter-frame feature may be the mean inter-frame content difference between the current video frame and the reference video frame. This is only an example and does not specifically limit the type of video features.

[0046] Optionally, after obtaining the current video frame to be encoded from the video sequence, a pre-analysis module built into the encoder may be called to analyze a video feature set related to the video scene before actual encoding, and the video feature set may include an intra-frame feature set and an inter-frame feature set. The pre-analysis module may include an intra-frame / inter-frame prediction module, and the intra-frame feature set and the inter-frame feature set may be extracted through the intra-frame / inter-frame prediction module.

[0047] It should be noted that in order to save computational effort, the above steps use downsampled low-resolution frames instead of original-resolution frames. The intra / inter prediction module has been simplified compared to actual coding, and it is necessary to approximate video features such as the prediction cost of each frame and each local area, content motion information, and texture complexity. In other words, the intra / inter prediction module is a fast preprocessing process before actual coding, which is an approximate estimate of actual coding. The intra-frame feature set and inter-frame feature set extracted above are also approximate estimated features in pre-analysis.

[0048] Step S206: convert the video feature set into a video feature vector that matches the target model.

[0049] In the technical solution provided in the above step S206 of the present application, after determining the video feature set associated with the video scene in the video sequence based on the current video frame, the determined video feature set can be converted into a video feature vector matching the target model. The target model is obtained by training based on the video feature vector samples and the corresponding quantization parameter offset samples.

[0050] In this embodiment, the target model may be obtained by training based on video feature vector samples and corresponding quantization parameter offset samples. The target model may also be referred to as a fitting model.

[0051] In this embodiment, the video feature vector samples are from the video sequence samples, and the quantization parameter offset samples are used in the process of encoding the video sequence samples so that the performance of the encoder meets the actual application requirements. The performance can be measured by the rate-distortion cost to reflect the coding efficiency of the encoder or algorithm, wherein the smaller the value of the rate-distortion cost is, the better the performance of the encoder in the encoding process is. The rate-distortion cost may include, but is not limited to, being derived from indicators such as bit rate, mean squared error (MSE), peak signal to noise ratio (PSNR), and structural similarity (SSIM).

[0052] Optionally, after determining the video feature set associated with the video scene in the video sequence based on the current video frame, the video feature set can be converted into a video feature vector that matches the fitting model. Optionally, multiple video features in the video feature set can be integrated to obtain a video feature vector.

[0053] Step S208: predict the video feature vector using the target model to obtain the quantization parameter offset of the current video frame.

[0054] In the technical solution provided in the above step S208 of the present application, after converting the video feature set into a video feature vector matching the target model, the target model can be used to predict the video feature vector to obtain the quantization parameter offset of the current video frame. The quantization parameter offset can be an adjustment amount for adjusting the quantization parameter of the current video frame, which can be used to optimize the coding efficiency and picture quality of the video sequence, and can also be called a quantization parameter QP offset value, which can be used To express.

[0055] Optionally, based on the video feature vector samples and the quantization parameter offset samples corresponding to the video feature vector samples, the initial fitting model can be trained to obtain a fitting model. After the fitting model training is completed, the fitting model can be used to determine the QP offset value in actual encoding. The obtained video feature vector can be input into the fitting model, and the video feature vector can be predicted using the fitting model to obtain the QP offset value, and then the QP of the current video frame can be adjusted using the QP offset value.

[0056] Step S210, using the quantization parameter offset, adjusting the quantization parameter of the current video frame. In the technical solution provided in the above step S210 of the present application, after the video feature vector is predicted using the target model to obtain the quantization parameter offset of the current video frame, the quantization parameter of the current video frame can be adjusted using the obtained quantization parameter offset to obtain the adjusted quantization parameter.

[0057] Optionally, after obtaining the QP offset value, the QP of the current video frame may be adjusted using the QP offset value. For example, the QP of the current video frame may be added to the QP offset value, thereby achieving the purpose of adjusting the QP of the current video frame. Has positive and negative values. When the QP bias value When it is positive, it means that the QP of the current video frame increases; when the QP offset value When it is negative, it means that the QP of the current video frame is reduced.

[0058] Step S212: Encode the current video frame using the adjusted quantization parameter to obtain an encoding result.

[0059] In the technical solution provided in the above step S212 of the present application, after the quantization parameter offset is used to adjust the quantization parameter of the current video frame and the adjusted quantization parameter is obtained, the adjusted quantization parameter can be used to encode the current video frame to obtain an encoding result.

[0060] Optionally, after the QP of the current video frame is adjusted to obtain the adjusted QP, the current video frame may be encoded using the adjusted QP to obtain an encoding result.

[0061] Through the above steps S202 to S212 of the present application, a current video frame to be encoded is obtained from a video sequence; based on the current video frame, a video feature set associated with the video scene in which the video sequence is located is determined, wherein the video features in the video feature set are used to describe the picture characteristics of the current video frame in the video scene; the video feature set is converted into a video feature vector that matches a target model, wherein the target model is trained based on video feature vector samples and corresponding quantization parameter offset samples; the video feature vector is predicted using the target model to obtain a quantization parameter offset of the current video frame; the quantization parameter of the current video frame is adjusted using the quantization parameter offset; the current video frame is encoded using the adjusted quantization parameter to obtain an encoding result. That is to say, the embodiment of the present application extracts a video feature set of the current video frame, converts the extracted video feature set into a video feature vector that matches the target model, accurately predicts a quantization parameter offset based on the video feature vector, and further adjusts the quantization parameter based on the quantization parameter offset according to actual conditions, so as to encode the current video frame using the adjusted quantization parameter to obtain an encoding result. Since the quantization parameter offset is predicted based on the target model to adjust the quantization parameter, it avoids determining a quantization parameter that meets the conditions through multiple traversals and tests, thereby achieving the technical effect of improving the efficiency of video encoding and solving the technical problem of low efficiency of video encoding.

[0062] The above method of this embodiment is further introduced below.

[0063] As an optional implementation, step S206, converting the video feature set into a video feature vector that matches the target model, includes: extracting at least one video feature corresponding to at least the frame type of the current video frame from the video feature set; and converting the extracted video feature into a video feature vector that matches the target model.

[0064] In this embodiment, the frame type may include an I frame type and a non-I frame type. Among them, the I frame may be used to indicate that the picture content is obtained through intra-frame prediction, and the I frame may be the first frame of the video sequence. The non-I frame type may be used to indicate that the picture content can be obtained through intra-frame or inter-frame prediction, and the non-I frame type may include a P frame and a B frame. The P frame supports unidirectional prediction, and the B frame supports bidirectional prediction. The frame number of the video sequence can be represented by p, and the frame number of the I frame is p=0.

[0065] Optionally, if the frame type of the current video frame is an I frame type, then the video features such as the mean intra-frame prediction cost corresponding to the I frame type, the mean inter-frame content difference, and the reference QP value used for encoding can be extracted from the video feature set. If the frame type of the current video frame is a non-I frame type, then the video features such as the mean intra-frame prediction cost corresponding to the non-I frame type, the mean inter-frame content difference, the QP offset value of the current video frame output by the encoder's built-in bit rate allocation strategy, and the reference QP value used for encoding can be extracted from the video feature set. After extracting the video features corresponding to different frame types, the extracted video features can be integrated to obtain a video feature vector.

[0066] It should be noted that the above-mentioned bit rate allocation strategy can at least include constant bit rate (CBR), variable bit rate (VBR) and constant rate factor (CRF). Among them, the fixed bit rate adjusts the QP value of each frame or each CTU according to the target bit rate, so that the overall bit rate of the video sequence is close to the target bit rate. The bit rate used by this strategy is relatively stable, but the picture quality fluctuates greatly and the overall picture quality is low. The variable bit rate adjusts the QP value of each frame or each CTU according to the target bit rate and the bit rate upper limit, so that the average bit rate of the video sequence is close to the target bit rate. At the same time, the bit rate used for each frame can be adjusted according to the specific picture content to ensure that the bit rate does not exceed the upper limit. This strategy can flexibly adjust the video bit rate and reduce the picture quality fluctuation, but it requires pre-coding to achieve it, with high calculation cost and long delay. The fixed proportional coefficient determines the QP value of each frame or CTU according to the preset proportional coefficient (which will affect the image quality after video compression) without limiting the specific bit rate. This strategy can directly control the image quality level, but cannot accurately control the bit rate. In complex scenarios, the bit rate is high and it is difficult to transmit stably.

[0067] This embodiment can predict a more reasonable quantization parameter (QP value) by analyzing the frame type of the current video frame and the content features corresponding to the frame type, thereby achieving more efficient bit rate control while ensuring video quality.

[0068] As an optional implementation, the video feature set includes an intra-frame feature set and an inter-frame feature set, the intra-frame features in the intra-frame feature set are used to describe the characteristics of the picture content of the current video frame in the video scene, and the inter-frame features in the inter-frame feature set are used to describe the characteristics of the picture content between the current video frame and the corresponding reference video frame in the video scene, and at least one video feature corresponding to the frame type of the current video frame is extracted from the video feature set, including: in response to the frame type being an I frame type, extracting the mean intra-frame prediction cost of the current video frame from the intra-frame feature set, and / or, extracting the mean inter-frame content difference between the current video frame and the reference video frame from the inter-frame feature set; in response to the frame type being a non-I frame type, extracting the mean intra-frame prediction cost corresponding to the time domain level of the current video frame from the intra-frame feature set, and / or, extracting the mean inter-frame content difference corresponding to the time domain level from the intra-frame feature set.

[0069] In this embodiment, the reference video frame may be another video frame that directly references the current frame. The mean intra-frame prediction cost may be the average rate-distortion cost required when predicting and encoding the internal pixels of the current video frame. The mean inter-frame content difference may be the average value of the pixel content difference between the current video frame and the reference video frame. For the I-frame type, the mean intra-frame prediction cost may be The mean value of the inter-frame content gap can be expressed by the following formula:

[0070]

[0071] in, It can be used to represent the mean value of the content difference between frames. N can be used to represent the number of reference video frames, that is, the number of other video frames that directly refer to the current I frame. n can be used to represent the frame number of the reference video frame. n It can be used to represent the distance between a reference video frame and a current video frame. MAD can be used to represent the mean absolute difference (Mean Absolute Difference) between a reference video frame and a current video frame.

[0072] Optionally, for non-I frame types, the temporal level of the p-th frame of the video sequence can be set to TID=t, and the mean intra-frame prediction cost can be calculated using The mean value of the inter-frame content gap can be expressed by the following formula:

[0073]

[0074] in, Can be used to represent the mean value of the content gap between frames, N t It can be used to indicate the number of reference video frames, that is, the number of other video frames that directly refer to the current video frame, and is related to the time domain level t.

[0075] It should be noted that, for the I frame type, the video features extracted from the video feature set may also include the reference QP value q used for encoding. I For non-I frame types, the video features extracted from the video feature set may also include the QP offset value Δq of the current video frame output by the encoder's built-in bit rate allocation strategy. t (p), and the base QP value q used for encoding t (p).

[0076] This embodiment can significantly improve the efficiency and quality of video encoding by extracting intra-frame / inter-frame features in a video feature set, implement adaptive bit rate control, optimize resource allocation, and improve the stability of the encoding process.

[0077] As an optional implementation, converting the extracted video features into a video feature vector that matches the target model includes: integrating the extracted video features to obtain the video feature vector that matches the target model.

[0078] In this embodiment, after extracting multiple video features corresponding to the I frame type from the video feature set and extracting multiple video features corresponding to the non-I frame type from the video feature set, the multiple video features corresponding to the I frame type and the multiple video features corresponding to the non-I frame type can be integrated to obtain a video feature vector that matches the target model.

[0079] Optionally, for the I frame type, the extracted intra-frame prediction cost mean Mean inter-frame content difference The base QP value q used for encoding I After integration processing, we get For non-I frame types, the extracted intra-frame prediction cost mean can be Mean inter-frame content difference The QP offset value Δq of the current video frame output by the encoder's built-in bit rate allocation strategy t (p), the base QP value q used for encoding t (p) is integrated and processed to obtain The above integrated vectors can be further merged to obtain the video feature vector that matches the target model:

[0080]

[0081] This embodiment integrates the extracted video features to obtain a video feature vector. Since the integrated video feature vector contains key information of the video frame, such as the mean intra-frame prediction cost, the mean inter-frame content gap, and the quantization parameter reference value, etc., by matching with the target model, the quantization parameter bias value in the video encoding process can be effectively predicted, providing more accurate guidance for bit rate control, thereby optimizing encoding efficiency and video quality.

[0082] As an optional implementation, the extracted video features are integrated to obtain a video feature vector that matches the target model, including: integrating the extracted video features to obtain an initial video feature vector; preprocessing the initial video feature vector to obtain a video feature vector that has the same dimension as a quantization parameter and matches the target model.

[0083] In this embodiment, the initial video feature vector can be represented by f. The video feature vector can be a feature vector obtained by preprocessing the initial video feature vector, which can be represented by f * The multiple video features corresponding to the I frame type and the multiple video features corresponding to the non-I frame type can be integrated to obtain the initial video feature vector f.

[0084] Optionally, the initial video feature vector is preprocessed, and the logarithm of the video feature with a larger dynamic range among multiple video features corresponding to the I frame type and the logarithm of the video feature with a larger dynamic range among multiple video features corresponding to the non-I frame type can be taken to ensure that the dimension of the video feature after taking the logarithm is the same as the QP.

[0085] Further, after taking the logarithm of the video features with a larger dynamic range among the multiple video features corresponding to the I frame type and the multiple video features corresponding to the non-I frame type, a scalar 1 can be added to the end of the multiple video features corresponding to the I frame type, and a scalar 1 can be added to the end of the multiple video features corresponding to the non-I frame type to obtain the video feature vector f * , so that the offset of the linear model can be obtained by subsequent fitting. The initial video feature vector f is preprocessed to obtain the video feature vector f * It can be expressed by the following formula:

[0086]

[0087] This embodiment can ensure that the video features and the quantization parameters have the same dimension through preprocessing, such as taking logarithms, normalization or standardization, so that the model is more effective in processing and learning these features, thereby helping the model to more accurately predict the QP value and avoid prediction bias caused by differences in feature scales.

[0088] As an optional implementation, step S208, using the target model to predict the video feature vector to obtain the quantization parameter offset of the current video frame, includes: using the target model and the frame type of the current video frame to predict the video feature vector to obtain the quantization parameter offset.

[0089] In this embodiment, when the target model is used to predict the video feature vector, the obtained quantization parameter offset is a predicted value, and there is an error between the predicted value and the actual value. Therefore, in order to improve the accuracy of the prediction as much as possible, the target model and the frame type of the current video frame can be used to predict the video feature vector to obtain the quantization parameter offset.

[0090] Optionally, based on the frame type of the current video frame, a weight vector matching the frame type can be determined. Further, from the weight vector, a set of weight vectors that minimize the sum of squared prediction errors can be determined, so as to predict the video feature vector using the weight vector that minimizes the sum of squared prediction errors, and obtain a quantization parameter offset, thereby improving the accuracy of the prediction.

[0091] As an optional implementation, the target model and the frame type of the current video frame are used to predict the video feature vector to obtain a quantization parameter offset, including: in the target model, determining a weight vector that matches the frame type, wherein the weight vector includes at least a weight coefficient of the video feature vector; and using the weight vector, adjusting the video feature vector to obtain a quantization parameter offset.

[0092] In this embodiment, in the target model, a weight vector matching the frame type may be determined, and the weight vector may be expressed by the following formula:

[0093]

[0094] Among them, w can be used to represent the weight vector, w I,1 It can be used to represent the weight coefficient of the mean intra-frame prediction cost corresponding to the I frame type, w I,2 It can be used to represent the weight coefficient of the mean value of the inter-frame content difference corresponding to the I frame type, w I,3 A weight coefficient that can be used to represent the reference QP value used for encoding corresponding to the I frame type. t,1 It can be used to represent the weight coefficient of the mean intra prediction cost corresponding to the non-I frame type, w t,2 It can be used to represent the weight coefficient of the mean value of the inter-frame content difference corresponding to the non-I frame type, w t,3 The weight coefficient that can be used to represent the QP bias value of the current video frame corresponding to the non-I frame type, w t,4 A weight coefficient that can be used to represent the base QP value used for encoding corresponding to non-I frame types. Iand b t Can be used to represent the offset of the fitted model respectively.

[0095] Optionally, after determining the weight vector matching the frame type in the target model, in order to further improve the accuracy of predicting the video feature vector, a set of weight vectors that minimize the sum of squared prediction errors can be determined. The weight vector that minimizes the sum of squared prediction errors can be further used to adjust the video feature vector to obtain a quantization parameter offset.

[0096] This embodiment determines the weight vector, which allows the model to adjust the QP value according to different dimensions in the video feature vector (such as the mean intra-frame prediction cost, the mean inter-frame content difference, etc.), thereby achieving refined control of the encoding parameters. The feature-based adjustment can better match the characteristics of the video content and avoid over-quantization or under-quantization.

[0097] As an optional implementation, the current video frame includes multiple coding tree units. Step S210, using the quantization parameter offset, adjusts the quantization parameter of the current video frame, including: determining the picture content of the coding tree unit; adjusting the quantization parameter of the coding tree unit using the quantization parameter offset and the picture content.

[0098] In this embodiment, the picture content of the coding tree unit CTU can be determined, for example, the picture content can be smooth picture content, complex picture content, violent motion picture content, gentle motion picture content, etc. This is only an example, and the type of picture content is not specifically limited. Further, the determined quantization parameter offset and picture content can be used to adjust the quantization parameter of the coding tree unit.

[0099] For example, after determining the picture content of the CTU, if the picture content changes in both the spatial domain and the temporal domain, it indicates that the picture content is not too smooth, and the quantization parameter offset can be used to adjust the quantization parameter of the CTU. For another example, if the picture content is not too complex, the quantization parameter offset can be used to adjust the quantization parameter of the CTU. Optionally, the quantization parameter and the quantization parameter offset are added to achieve the purpose of adjusting the quantization parameter.

[0100] This embodiment automatically adjusts the quantization parameters according to the dynamics and complexity of the video picture, avoiding the use of too high quantization parameters in areas where the picture changes little or is too simple, resulting in waste of bit rate, and using more appropriate quantization parameters in areas where the picture is complex or changes greatly to reduce information loss.

[0101] As an optional implementation, the quantization parameter of the coding tree unit is adjusted using a quantization parameter offset and a picture content, including: in response to the quantization parameter offset being greater than an offset threshold, and the complexity of the picture content in the spatial domain and the temporal domain being greater than a complexity threshold, adjusting the quantization parameter of the coding tree unit using the quantization parameter offset, wherein the complexity is used to represent the complexity of the picture content; in response to the quantization parameter offset being not greater than the offset threshold, and the complexity of the picture content being less than the complexity threshold, adjusting the quantization parameter of the coding tree unit using the quantization parameter offset, wherein the complexity is used to represent the complexity of the picture content.

[0102] In this embodiment, the offset threshold may be a critical value of the offset used to measure the QP offset that is pre-set according to actual conditions. For example, the offset threshold may be 0. This is only an example and does not impose a specific limit on the value of the offset threshold. Complexity may be used to indicate the complexity of the content of the picture. The complexity threshold may be a critical value of the complexity used to measure the complexity that is pre-set according to actual conditions. The complexity of the content of the picture may be determined by the average absolute value difference d0(p) between the current CTU and the corresponding position of the adjacent frame, and the current CTU pixel value variance v0(p).

[0103] For example, the offset threshold is set to 0. For the current video frame whose temporal level TID is 0, if the quantization parameter offset is greater than the offset threshold, that is Then it is necessary to satisfy the average absolute value difference d0(p)>τ between the current CTU and the corresponding position of the adjacent frame M+ And the current CTU pixel value variance v0(p)>τ V+ , that is, the QP value of the CTU is adjusted only when the picture content changes in both the spatial and temporal domains and is not too smooth. M+ It can be used to indicate the decision threshold of the mean absolute value difference of the quantization parameter offset greater than the offset threshold, τ V+ It can be used to indicate that the quantization parameter offset is greater than the decision threshold of the current CTU pixel value variance under the offset threshold.

[0104] For another example, if the quantization parameter offset is not greater than the offset threshold, that is, or Then it is necessary to satisfy the average absolute value difference between the current CTU and the corresponding position of the adjacent frame d0(p) < τ M- And the current CTU pixel value variance v0(p)>τ V- That is, the QP value of the CTU is adjusted only when the picture content is not too complex to avoid excessive increase in bit rate. M- It can be used to indicate that the decision threshold of the mean absolute value difference of the quantization parameter offset is not greater than the offset threshold, τ V-The decision threshold value may be used to indicate that the quantization parameter offset is not greater than the current CTU pixel value variance under the offset threshold value.

[0105] It should be noted that the above is a case where the CTU decision strategy is turned on, and a case where TID is 0 is shown. For frames of other TIDs, similar strategies can also be adopted, or part of the training video sequence can be used as a verification set. The above strategy can be flexibly adjusted according to the encoding performance on the part of the sequence to ensure that most videos have a bit rate change rate (ie, BD-rate) benefit under the same objective quality. The BD-rate benefit can be used to measure the encoding efficiency. A negative value indicates a decrease in the bit rate, and a positive value indicates an increase in the bit rate.

[0106] This embodiment allows the encoder to perform more sophisticated bitrate control at the CTU level, ensuring a more reasonable bitrate distribution. When the video content varies in spatial and temporal dimensions, increasing the QP value can reduce the bitrate, while the content that is not too smooth or complex maintains or increases the QP value to maintain image quality and reduce information redundancy.

[0107] As an optional implementation, the current video frame includes multiple coding tree units, and adjusting the quantization parameter of the current video frame by using the quantization parameter offset includes: adjusting the quantization parameter of each coding tree unit by using the quantization parameter offset.

[0108] It should be noted that this embodiment compares the BD-rate benefits of whether the CTU decision strategy is enabled or not. If the BD-rate benefit is greater after the CTU decision strategy is enabled, it is recommended to enable it. As described above, by enabling the CTU decision strategy, the average BD-rate benefit can be improved and the BD-rate loss caused by inaccurate prediction of the fitting model on some sequences can be reduced.

[0109] In this embodiment, if the CTU decision strategy is not enabled, the frame-level QP value can be adjusted. That is, the QP value of each CTU of the current video frame is increased by the quantization parameter offset. in, If it is positive, it means the actual QP value increases. A negative value indicates that the actual QP value decreases.

[0110] This embodiment avoids complex judgment and processing at the CTU level by adjusting the frame-level QP value, reduces calculation complexity, and makes the encoding process more concise and faster.

[0111] As an optional implementation, the method further includes: in response to the adjusted quantization parameter being greater than a first quantization parameter threshold, or less than a second quantization parameter threshold, maintaining the quantization parameter before adjustment; and encoding the current video frame using the quantization parameter before adjustment to obtain an encoding result.

[0112] In this embodiment, the first quantization parameter threshold may be a lower limit critical value for measuring QP pre-set according to actual conditions. The second quantization parameter threshold may be an upper limit critical value for measuring QP pre-set according to actual conditions. When the adjusted QP value is greater than the first quantization parameter threshold, or less than the second quantization parameter threshold, it means that the adjusted QP value is too large or too small, and the quantization parameter may not be adjusted. Further, the quantization parameter may be directly used to encode the current video frame to obtain an encoding result.

[0113] For example, if the adjusted QP value of the current CTU is too large or too small, for example, Then cancel the adjustment of QP value and keep the QP value unchanged to avoid excessive risk of fitting model prediction. t (p) can be used to represent the QP offset value of the current video frame output by the encoder built-in bit rate allocation strategy corresponding to the non-I frame type, τ Q Can be used to indicate the decision threshold.

[0114] This embodiment prevents the QP offset value predicted by the model from being too large or too small by setting the decision threshold, thereby avoiding the problem of over-quantization or under-quantization that may occur during the video encoding process and ensuring the stability of video quality.

[0115] In an embodiment of the present application, a current video frame to be encoded is obtained from a video sequence; based on the current video frame, a video feature set associated with a video scene in which the video sequence is located is determined, wherein the video features in the video feature set are used to describe the picture characteristics of the current video frame in the video scene; the video feature set is converted into a video feature vector that matches a target model, wherein the target model is trained based on video feature vector samples and corresponding quantization parameter offset samples; the video feature vector is predicted using the target model to obtain a quantization parameter offset of the current video frame; the quantization parameter of the current video frame is adjusted using the quantization parameter offset; the current video frame is encoded using the adjusted quantization parameter to obtain an encoding result. That is to say, the embodiment of the present application extracts a video feature set of the current video frame, converts the extracted video feature set into a video feature vector that matches the target model, accurately predicts a quantization parameter offset based on the video feature vector, and further adjusts the quantization parameter based on the quantization parameter offset according to actual conditions, so as to encode the current video frame using the adjusted quantization parameter to obtain an encoding result. Since the quantization parameter offset is predicted based on the target model to adjust the quantization parameter, it avoids determining a quantization parameter that meets the conditions through multiple traversals and tests, thereby achieving the technical effect of improving the efficiency of video encoding and solving the technical problem of low efficiency of video encoding.

[0116] The present application also provides a method for determining a model. Figure 3 is a flow chart of a method for determining a model according to an embodiment of the present application, such as Figure 3 As shown, the method may include the following steps:

[0117] Step S302: Obtain video feature vector samples, and obtain quantization parameter offset samples corresponding to the video feature vector samples.

[0118] In the technical solution provided in the above step S302 of the present application, a video feature vector sample can be obtained, and a quantization parameter offset sample corresponding to the video feature vector sample can be obtained, wherein the video feature vector sample comes from a video sequence sample.

[0119] Step S304: train and obtain a target model using the video feature vector samples and the quantization parameter offset samples.

[0120] In the technical solution provided in the above step S304 of the present application, after obtaining the video feature vector samples from the video sequence samples and the quantization parameter offset samples corresponding to the video feature vector samples, the video feature vector samples and the quantization parameter offset samples can be used to train and obtain the target model. The target model can be used to predict the video feature vector matching the target model to obtain the quantization parameter offset, and the target model can also be called a fitting model.

[0121] In this embodiment, the video feature vector is converted from a video feature set associated with a video scene in which the video sequence is located. The video features in the video feature set can be used to describe the picture characteristics of the current video frame to be encoded in the video sequence under the video scene, and the video feature set is determined based on the current video frame. The quantization parameter offset can be used to adjust the quantization parameter of the current video frame. The adjusted quantization parameter can be used to encode the current video frame to obtain an encoding result.

[0122] Optionally, a current video frame to be encoded may be obtained from a video sequence. The obtained video sequence may be input into an encoder, and the encoder performs steps such as reading, parsing, buffering, and encoding on the video sequence. After the encoder parses the video sequence, the parsed video frame may be buffered into a buffer, and further, a video frame may be obtained from the buffer as the current video frame to be encoded, so as to perform an encoding operation on the current video frame.

[0123] Optionally, after obtaining the current video frame to be encoded from the video sequence, a pre-analysis module built into the encoder may be called to analyze a video feature set related to the video scene before actual encoding, and the video feature set may include an intra-frame feature set and an inter-frame feature set. The pre-analysis module may include an intra-frame / inter-frame prediction module, and the intra-frame feature set and the inter-frame feature set may be extracted through the intra-frame / inter-frame prediction module.

[0124] Optionally, after determining the video feature set associated with the video scene in the video sequence based on the current video frame, the video feature set can be converted into a video feature vector that matches the fitting model. Optionally, multiple video features in the video feature set can be integrated to obtain a video feature vector.

[0125] Optionally, based on the video feature vector samples and the quantization parameter offset samples corresponding to the video feature vector samples, the initial fitting model can be trained to obtain a fitting model. After the fitting model training is completed, the fitting model can be used to determine the QP offset value in actual encoding. The obtained video feature vector can be input into the fitting model, and the video feature vector can be predicted using the fitting model to obtain the QP offset value, so as to adjust the QP of the current video frame using the QP offset value.

[0126] Optionally, after obtaining the QP offset value, the QP of the current video frame may be adjusted using the QP offset value. For example, the QP of the current video frame may be added to the QP offset value, thereby achieving the purpose of adjusting the QP of the current video frame. Has positive and negative values. When the QP bias value When it is positive, it means that the QP of the current video frame increases; when the QP offset value When it is negative, it indicates that the QP of the current video frame is reduced. After the QP of the current video frame is adjusted to obtain the adjusted QP, the current video frame can be encoded using the adjusted QP to obtain an encoding result.

[0127] Through the above steps S302 to S304 of the present application, video feature vector samples are obtained, and quantization parameter offset samples corresponding to the video feature vector samples are obtained; the target model is trained using the video feature vector samples and the quantization parameter offset samples, wherein the target model is used to predict the video feature vector matching the target model to obtain the quantization parameter offset, the video feature vector is converted from a video feature set associated with the video scene where the video sequence is located, the video features in the video feature set are used to describe the picture characteristics of the current video frame to be encoded in the video sequence under the video scene, the video feature set is determined based on the current video frame, the quantization parameter offset is used to adjust the quantization parameter of the current video frame, and the adjusted quantization parameter is used to encode the current video frame to obtain the encoding result. That is to say, the embodiment of the present application extracts a video feature set of the current video frame, converts the extracted video feature set into a video feature vector that matches the target model, accurately predicts a quantization parameter offset based on the video feature vector, and further adjusts the quantization parameter based on the quantization parameter offset according to actual conditions, so as to encode the current video frame using the adjusted quantization parameter to obtain an encoding result. Since the quantization parameter offset is predicted based on the target model to adjust the quantization parameter, it avoids determining a quantization parameter that meets the conditions through multiple traversals and tests, thereby achieving the technical effect of improving the efficiency of video encoding and solving the technical problem of low efficiency of video encoding.

[0128] The above method of this embodiment is further introduced below.

[0129] As an optional implementation, step S302, obtaining video feature vector samples, includes: obtaining multiple lossless video sequence samples, where different lossless video sequence samples come from different video scenes; and obtaining video feature vector samples from the multiple lossless video sequence samples.

[0130] In this embodiment, multiple lossless video sequence samples may be obtained. Different lossless video sequence samples come from different video scenes, and the video scene may be a picture scene contained in the video sequence, such as a motion picture scene, a still picture scene, a texture smooth picture scene, a complex picture scene, etc. This is only an example, and the type of video scene is not specifically limited. After obtaining multiple lossless video sequence samples, video feature vector samples may be obtained from the multiple lossless video sequence samples.

[0131] As an optional implementation, step S304, obtaining video feature vector samples from multiple lossless video sequence samples, includes: based on the video frame samples in the lossless video sequence samples, determining a video feature sample set associated with the video scene where the lossless video sequence samples are located, wherein the video feature samples in the video feature sample set are used to describe the picture characteristics of the video frame samples in the corresponding video scene; converting the video feature sample set into video feature vector samples.

[0132] In this embodiment, the video feature sample set may include multiple video feature samples extracted from the video frame samples and related to the video scene. The video feature samples in the video feature sample set may be used to describe the picture characteristics of the video frame samples in the video scene. The video feature samples may include intra-frame feature samples or inter-frame feature samples. For example, the intra-frame feature samples may be intra-frame prediction cost mean samples of the video frame samples, and the inter-frame features may be inter-frame content difference mean samples between the video frame samples and the reference video frame samples. This is only an example and does not specifically limit the types of video feature samples.

[0133] Optionally, after obtaining multiple lossless video sequence samples, video frame samples can be extracted from the multiple lossless video sequence samples. The obtained multiple lossless video sequence samples can be input into an encoder, and the encoder performs steps such as reading, parsing, buffering, and encoding the multiple lossless video sequence samples. After the encoder parses the multiple lossless video sequence samples, the parsed video frame samples can be buffered into a buffer, and further the video frame samples can be obtained from the buffer.

[0134] Optionally, after extracting video frame samples from multiple lossless video sequence samples, a pre-analysis module built into the encoder can be called to analyze a video feature sample set related to the video scene before actual encoding, and the video feature sample set can include an intra-frame feature sample set and an inter-frame feature sample set. The pre-analysis module can include an intra-frame / inter-frame prediction module, and the intra-frame feature sample set and the inter-frame feature sample set can be extracted through the intra-frame / inter-frame prediction module. By integrating multiple video feature samples in the video feature sample set, a video feature vector sample can be obtained.

[0135] As an optional implementation, based on the video frame samples in the lossless video sequence samples, a video feature sample set associated with the video scene in which the lossless video sequence samples are located is determined, including: downsampling the video frame samples, wherein the resolution of the downsampled video frame samples is lower than a resolution threshold; and determining the video feature sample set based on the downsampled video frame samples.

[0136] In this embodiment, the resolution threshold may be a resolution critical value preset according to actual conditions for measuring resolution. After extracting video frame samples from the video sequence samples, the video frame samples may be downsampled. Further based on the downsampled video frame samples, a video feature sample set may be determined.

[0137] Optionally, in order to save computational effort, the above steps use downsampled low-resolution frame samples instead of original-resolution frame samples. The intra-frame / inter-frame prediction module has been simplified compared to actual coding, and it is necessary to approximate the prediction cost of each frame and each local area, content motion information, texture complexity and other video feature samples. In other words, the intra-frame / inter-frame prediction module is a fast preprocessing process before actual coding, which is an approximate estimate of actual coding. The above-extracted intra-frame feature sample set and inter-frame feature sample set are also approximate estimated features in pre-analysis.

[0138] As an optional implementation, converting a video feature sample set into a video feature vector sample includes: extracting at least one video feature sample corresponding to at least a frame type of a video frame sample from the video feature sample set; integrating the extracted video feature samples to obtain an initial video feature vector sample; and preprocessing the initial video feature vector sample to obtain a video feature vector sample.

[0139] In this embodiment, if the frame type of the video frame sample is an I frame type, then the video feature samples such as the mean intra-frame prediction cost corresponding to the I frame type, the mean inter-frame content difference, and the reference QP value used for encoding can be extracted from the video feature sample set. If the frame type of the video frame sample is a non-I frame type, then the video feature samples such as the mean intra-frame prediction cost corresponding to the non-I frame type, the mean inter-frame content difference, the QP offset value of the current video frame output by the encoder's built-in bit rate allocation strategy, and the reference QP value used for encoding can be extracted from the video feature sample set. After extracting the video feature samples corresponding to different frame types, the extracted video feature samples can be integrated to obtain the initial video feature vector samples.

[0140] Further, the initial video feature vector samples are preprocessed, and the logarithm of the video feature samples with a larger dynamic range among the multiple video feature samples corresponding to the I frame type can be taken, and the logarithm of the video feature samples with a larger dynamic range among the multiple video feature samples corresponding to the non-I frame type can be taken. Further, a scalar 1 can be added to the end of the multiple video feature samples corresponding to the I frame type, and a scalar 1 can be added to the end of the multiple video feature samples corresponding to the non-I frame type to obtain a video feature vector sample.

[0141] As an optional implementation, the video feature sample set includes an intra-frame feature sample set and an inter-frame feature sample set, the intra-frame feature samples in the intra-frame feature sample set are used to describe the characteristics of the picture content of the video frame sample in the corresponding video scene, and the inter-frame feature samples in the inter-frame feature sample set are used to describe the characteristics of the picture content between the video frame sample and the corresponding reference video frame sample in the corresponding video scene, and at least one video feature sample corresponding to the frame type of the video frame sample is extracted from the video feature sample set, including: in response to the frame type being an I frame type, extracting the mean intra-frame prediction cost of the video frame sample from the intra-frame feature sample set, and / or, extracting the mean inter-frame content difference between the video frame sample and the reference video frame sample from the inter-frame feature sample set; in response to the frame type being a non-I frame type, extracting the mean intra-frame prediction cost corresponding to the time domain level of the video frame sample from the intra-frame feature sample set, and / or, extracting the mean inter-frame content difference corresponding to the time domain level from the intra-frame feature sample set.

[0142] In this embodiment, the reference video frame sample may be a video frame sample of another directly referenced video frame sample. For the I frame type, the intra-frame prediction cost mean of the video frame sample may be extracted from the intra-frame feature sample set. Mean inter-frame content difference The base QP value q used for encoding I .

[0143] Optionally, for non-I frame types, the temporal level of the pth frame of the video sequence sample can be set to TID=t, and the mean intra-frame prediction cost of the video frame sample can be extracted from the intra-frame feature sample set: Mean inter-frame content difference The QP offset value Δq of the current video frame output by the encoder's built-in bit rate allocation strategy t (p), and the base QP value q used for encoding t (p).

[0144] As an optional implementation, step S304, obtaining quantization parameter offset samples corresponding to video feature vector samples, includes: encoding lossless video sequence samples using rate control strategies corresponding to multiple bit rates, to obtain multiple encoding result samples, wherein the multiple encoding result samples correspond to multiple bit rates; determining the quantization parameter offset samples from multiple initial quantization parameter offset samples corresponding to the multiple encoding result samples.

[0145] In this embodiment, each lossless video sequence sample can be encoded at each bit rate point using a corresponding bit rate control strategy to obtain multiple encoding result samples. The bit rate control strategy can be a fixed bit rate, a variable bit rate, a fixed proportional coefficient, etc. Further, using a rate distortion (RD) search method, a quantization parameter offset sample can be determined from multiple initial quantization parameter offset samples corresponding to multiple encoding result samples. The rate distortion search method can include a traversal method and an estimation method, and the rate distortion can be used to comprehensively measure the relationship between the bit rate and the degree of distortion.

[0146] Optionally, when a traversal method is used to determine a quantization parameter offset sample from multiple initial quantization parameter offset samples corresponding to multiple encoding result samples, first assume that the BD-rate that meets the conditions is 0, and the QP of each frame is unchanged, and check each bit rate point of each frame in the encoding order. At each bit rate point, check the situation of multiple QP bias values ​​and encode them separately. If it is found that the current BD-rate is better than the recorded BD-rate, the BD-rate is updated, and the QP bias used for the current frames and each bit rate point is recorded until all video frames are checked. This method can accurately obtain the QP bias value of each frame, but it requires multiple traversals and has a high amount of calculation.

[0147] Optionally, when the estimation method is used to determine the quantization parameter offset sample from multiple initial quantization parameter offset samples corresponding to multiple encoding result samples, first, for each sequence, the frame QP value with the same TID is changed at the same time, and the bit rate and quality evaluation indicators such as PSNR and SSIM at each bit rate point are recorded. The situations formed by each bit rate point and each QP offset value are further compared, and the QP offset value when the BD-rate meets the target is recorded. This method can approximately estimate the QP offset that meets the conditions for various TID situations with less calculation.

[0148] As an optional implementation, a target model is trained using video feature vector samples and quantization parameter offset samples, including: converting the video feature vector samples into an input matrix of an initial model, and converting the quantization parameter offset samples into an output matrix of the initial model; fitting the input matrix and the output matrix to obtain weight vector samples corresponding to the initial model, wherein the weight vector samples include at least weight coefficient samples of the video feature vector samples; and establishing a target model using the weight vector samples and the initial model.

[0149] In this embodiment, the initial model can be other fitting models other than the linear model, such as support vector machine, random forest, multi-layer neural network, etc. First, a training set is constructed. It is assumed that M records are extracted cumulatively from multiple frames of multiple video sequences, where each record (set as sequence number m) includes a preprocessed video feature vector sample and the corresponding quantization parameter offset sample ΔQ m , the video feature vector sample can be converted into the input matrix of the initial model through the following formula:

[0150]

[0151] Among them, X can be used to represent the input matrix of the initial model. The quantization parameter offset sample can be further converted into the output matrix of the initial model by the following formula:

[0152]

[0153] Among them, y can be used to represent the output matrix of the initial model. Further through the formula By fitting the input matrix and the output matrix, we can get the weight vector sample corresponding to the initial model:

[0154]

[0155] The weight vector sample may include a weight coefficient sample of the video feature vector sample and an offset of the initial model. I,1 It can be used to represent the weight coefficient of the mean intra-frame prediction cost corresponding to the I frame type, w I,2 It can be used to represent the weight coefficient of the mean value of the inter-frame content difference corresponding to the I frame type, w I,3 A weight coefficient that can be used to represent the reference QP value used for encoding corresponding to the I frame type. t,1 It can be used to represent the weight coefficient of the mean intra prediction cost corresponding to the non-I frame type, w t,2 It can be used to represent the weight coefficient of the mean value of the inter-frame content difference corresponding to the non-I frame type, w t,3 The weight coefficient that can be used to represent the QP bias value of the current video frame corresponding to the non-I frame type, w t,4 A weight coefficient that can be used to represent the base QP value used for encoding corresponding to non-I frame types. I and b t can be used to represent the offset of the initial model. Expressed in matrix form, we can get:

[0156]

[0157] in, Can be used to represent the quantization parameter offset sample ΔQ m The predicted value of It can be used to represent the predicted value of the output matrix y. Due to the error in the prediction, Because y is usually not equal. In order to improve the accuracy as much as possible, a set of weight vector samples that minimize the sum of squared prediction errors can be determined. That is, the least squares optimization objective:

[0158]

[0159] Among them, the weight vector sample that minimizes the sum of squares of the prediction error can be obtained by matrix operation.

[0160]

[0161] Among them, the target model is established using the weight vector sample and the initial model. After the target model training is completed, the parameters required by the target model are saved in the weight vector sample that minimizes the sum of squared prediction errors. It can be used to adjust the QP value of each frame in actual encoding.

[0162] In an embodiment of the present application, a video feature vector sample is obtained, and a quantization parameter offset sample corresponding to the video feature vector sample is obtained; the video feature vector sample and the quantization parameter offset sample are used to train a target model, wherein the target model is used to predict the video feature vector matching the target model to obtain the quantization parameter offset, the video feature vector is converted from a video feature set associated with a video scene in which a video sequence is located, the video features in the video feature set are used to describe the picture characteristics of a current video frame to be encoded in the video sequence in the video scene, the video feature set is determined based on the current video frame, the quantization parameter offset is used to adjust the quantization parameter of the current video frame, and the adjusted quantization parameter is used to encode the current video frame to obtain an encoding result. That is to say, the embodiment of the present application extracts a video feature set of the current video frame, converts the extracted video feature set into a video feature vector that matches the target model, accurately predicts a quantization parameter offset based on the video feature vector, and further adjusts the quantization parameter based on the quantization parameter offset according to actual conditions, so as to encode the current video frame using the adjusted quantization parameter to obtain an encoding result. Since the quantization parameter offset is predicted based on the target model to adjust the quantization parameter, it avoids determining a quantization parameter that meets the conditions through multiple traversals and tests, thereby achieving the technical effect of improving the efficiency of video encoding and solving the technical problem of low efficiency of video encoding.

[0163] The embodiment of the present application also provides a video encoding system, Figure 4is a schematic diagram of a video encoding system according to an embodiment of the present application, such as Figure 4 As shown, the video encoding system 400 may include: an encoder 402 and a server 404 .

[0164] Encoder 402 is used to obtain a current video frame to be encoded from a video sequence; call a pre-analysis module to determine a video feature set associated with a video scene in which the video sequence is located based on the current video frame, wherein the video features in the video feature set are used to describe the picture characteristics of the current video frame in the video scene; convert the video feature set into a video feature vector that matches a target model, wherein the target model is trained based on video feature vector samples and corresponding quantization parameter offset samples; use the target model to predict the video feature vector to obtain a quantization parameter offset of the current video frame; use the quantization parameter offset to adjust the quantization parameter of the current video frame; use the adjusted quantization parameter to encode the current video frame to obtain an encoding result.

[0165] Optionally, after acquiring the video sequence, the acquired video sequence may be input into an encoder, and the encoder performs steps such as reading, parsing, buffering, and encoding on the video sequence. After the encoder parses the video sequence, the parsed video frames may be buffered into a buffer, and further, the video frames may be acquired from the buffer as the current video frame to be encoded, so as to perform encoding operations on the current video frame.

[0166] Optionally, after obtaining the current video frame to be encoded from the video sequence, a pre-analysis module built into the encoder may be called to analyze a video feature set related to the video scene before actual encoding, and the video feature set may include an intra-frame feature set and an inter-frame feature set. The pre-analysis module may include an intra-frame / inter-frame prediction module, and the intra-frame feature set and the inter-frame feature set may be extracted through the intra-frame / inter-frame prediction module.

[0167] Optionally, after determining the video feature set associated with the video scene in the video sequence based on the current video frame, the video feature set can be converted into a video feature vector that matches the fitting model. Optionally, multiple video features in the video feature set can be integrated to obtain a video feature vector.

[0168] Optionally, based on the video feature vector samples and the quantization parameter offset samples corresponding to the video feature vector samples, the initial fitting model can be trained to obtain a fitting model. After the fitting model training is completed, the fitting model can be used to determine the QP offset value in actual encoding. The obtained video feature vector can be input into the fitting model, and the video feature vector can be predicted using the fitting model to obtain the QP offset value, so as to adjust the QP of the current video frame using the QP offset value.

[0169] Optionally, after obtaining the QP offset value, the QP of the current video frame may be adjusted using the QP offset value. For example, the QP of the current video frame may be added to the QP offset value, thereby achieving the purpose of adjusting the QP of the current video frame. Has positive and negative values. When the QP bias value When it is positive, it means that the QP of the current video frame increases; when the QP offset value When it is negative, it indicates that the QP of the current video frame is reduced. After the QP of the current video frame is adjusted to obtain the adjusted QP, the current video frame can be encoded using the adjusted QP to obtain an encoding result.

[0170] Server 404 is used to obtain the encoding result.

[0171] In the video encoding system, a current video frame to be encoded is obtained from a video sequence through an encoder 402; a pre-analysis module is called to determine a video feature set associated with a video scene in which the video sequence is located based on the current video frame, wherein the video features in the video feature set are used to describe the picture characteristics of the current video frame in the video scene; the video feature set is converted into a video feature vector that matches a target model, wherein the target model is obtained by training based on video feature vector samples and corresponding quantization parameter offset samples; the video feature vector is predicted using the target model to obtain a quantization parameter offset of the current video frame; the quantization parameter of the current video frame is adjusted using the quantization parameter offset; the current video frame is encoded using the adjusted quantization parameter to obtain an encoding result; and the encoding result is obtained through a server 404. That is to say, the embodiment of the present application extracts a video feature set of the current video frame, converts the extracted video feature set into a video feature vector that matches the target model, accurately predicts a quantization parameter offset based on the video feature vector, and further adjusts the quantization parameter based on the quantization parameter offset according to actual conditions, so as to encode the current video frame using the adjusted quantization parameter to obtain an encoding result. Since the quantization parameter offset is predicted based on the target model to adjust the quantization parameter, it avoids determining a quantization parameter that meets the conditions through multiple traversals and tests, thereby achieving the technical effect of improving the efficiency of video encoding and solving the technical problem of low efficiency of video encoding.

[0172] The technical solution of the embodiment of the present disclosure is further introduced below with examples in combination with preferred implementation modes.

[0173] At present, rate control is an important part of the video encoding process. It can be used to select appropriate quantization parameters for each frame of the video sequence or each image block in each frame to control the required bit rate. Mainstream encoders usually support multiple rate control strategies such as fixed bit rate, variable bit rate and fixed proportional coefficient. The specific strategy will affect the final bit rate, stability, image quality and other indicators to meet the needs of different scenarios.

[0174] In recent years, multimedia technology and video scenes are experiencing rapid development, and the amount of video data is showing an explosive growth trend, which poses a great challenge to communication bandwidth and storage resources. In response to this, multiple generations of video compression coding standards have been proposed to improve coding efficiency and reduce video storage and transmission overhead. Compared with the previous generation standard H.265 / High Efficiency Video Coding (HEVC), the new generation standard H.266 / VVC can reduce the bit rate by 30% to 50% under the same subjective and objective visual quality. This efficiency improvement mainly comes from a variety of new coding technologies that expand the rate-distortion search space, but at the same time greatly increases the computational complexity, making it difficult to apply in practice, and has become a major bottleneck hindering the improvement of coding efficiency. Therefore, how to improve coding efficiency without significantly increasing complexity has become a focus of attention from all parties.

[0175] After the new generation H.266 / VVC video coding standard was proposed, some algorithms have been used to optimize the bit rate control performance. However, these algorithms are mainly used to improve the control accuracy, rather than directly improve the coding efficiency. Moreover, most of them are designed for standard encoders, and there is a lack of solutions for fast encoders in actual scenarios.

[0176] In one example, the commonly used bit rate control strategies in open source encoders (such as x265, VVenC, VTM, etc.) mainly include: fixed bit rate, variable bit rate, fixed proportional coefficient, fixed proportional coefficient under bit rate constraint and fixed quantization parameter (Constant Quantization Parameter, referred to as CQP). Among them, fixed bit rate, variable bit rate, and fixed proportional coefficient have been introduced above and will not be repeated here. The fixed proportional coefficient under bit rate constraint limits the upper limit of the bit rate on the basis of the fixed proportional coefficient CRF. This strategy is mainly used to control the image quality level and ensure that the bit rate is not too high. The fixed quantization parameter uses the same QP value. This strategy is simple to implement and the image quality is relatively stable, but it ignores the reference relationship between multiple frames, resulting in large bit rate redundancy, insufficient coding efficiency, and less practical application. Among them, x265 is an open source H.265 / HEVC encoder, VVenC is an open source H.266 / VVC encoder, and VTM is the H.266 / VVC standard reference software, which contains an encoder.

[0177] Based on the above strategies, existing algorithms have further improved the rate control strategy in the new generation H.266 / VVC standard. For example, by comparing and analyzing the impact of zero residual blocks on rate distortion parameters and inter-frame quality dependence, it is proposed to update the parameters of zero residual blocks and regular blocks respectively, and select the rate allocation strategy that meets the conditions, which significantly improves the coding efficiency of VTM. In addition, a machine learning-based intra-frame mode rate allocation optimization algorithm is proposed in the existing algorithm. According to the directional characteristics of CTU texture, multiple machine learning models are used to predict the QP value that meets the conditions, thereby improving the accuracy of rate control and the overall rate distortion performance.

[0178] Furthermore, existing algorithms also predict the rate allocation coefficient that meets the conditions based on the content similarity between the key frame and other reference frames to improve the rate-distortion performance of inter-frame coding. In addition, some existing algorithms use a variety of methods such as the cubic rate-distortion model, Nash equilibrium, and joint decision trees to model the rate control process in VVC to improve control accuracy and coding efficiency.

[0179] The above algorithms provide a variety of optimization ideas and methods for bit rate control in VVC. However, the existing algorithms are usually designed for the standard encoder VTM, and there is a lack of research on the widely used fast encoders such as VVenC. In addition, the computational complexity of some algorithms is large, making them difficult to use for fast or even real-time encoders. Most existing algorithms are designed for fixed bit rate CBR strategies to improve the accuracy of bit rate control, rather than directly optimizing encoding efficiency. However, CBR is not used in many actual video scenarios, and there is a greater need to improve encoding efficiency, which makes it difficult to directly use existing algorithms.

[0180] In order to solve the above problems, the present application proposes a rate control optimization algorithm for adaptive video scenes, which predicts quantization parameters that meet the conditions based on the intra-frame and inter-frame features of the video content, and uses the quantization parameters that meet the conditions in the rate control process to improve the coding efficiency. The encoder used in this embodiment is the S266 encoder, which is a video encoder that complies with the H.266 / VVC standard and has been put into use in short video and live broadcast scenarios. In the process of optimizing the S266 encoder, the existing rate allocation scheme is difficult to adapt to complex and changeable scenes, and the coding efficiency is low. After algorithm optimization, this embodiment can adaptively adjust the quantization parameters used for each frame according to the video scene to improve the rate allocation process and significantly improve the coding efficiency.

[0181] This embodiment can be used to optimize the bit rate allocation process of video encoding, which mainly includes two parts: pre-statisticing the statistical characteristics of the content of each frame in multiple video scenes, and the QP value that meets the rate-distortion performance conditions, using linear regression or machine learning models to fit the relationship therein, and saving the model; reading the above model during actual encoding, predicting a reasonable QP value based on the statistical characteristics in the video scene, and using the QP value for bit rate allocation to improve the overall encoding efficiency.

[0182] Figure 5 is a flow chart of a model training according to an embodiment of the present application, such as Figure 5 As shown, the model training process may include the following steps:

[0183] Step S501: Select multiple lossless video sequences and a basic bit rate control strategy.

[0184] In the above steps, multiple lossless video sequences can be selected, and it is ensured that the selected video sequences contain a variety of video scenes, such as motion pictures, still pictures, smooth texture pictures, complex pictures, etc., for model training. The encoder version and the basic bit rate control strategy used can be further determined. The basic bit rate control strategy is CBR, VBR, CRF, etc. mentioned above. Considering that the bit rate used in actual encoding varies greatly, multiple bit rate points can be selected in this step to cover various situations as much as possible.

[0185] Optionally, 20 1080-pixel video sequences may be selected as training data, and CRF may be selected as a basic bit rate control strategy, which includes four bit rate points CRF {18, 22, 26, 30}.

[0186] Step S502: Pre-analyze the video content of the selected video sequence.

[0187] In the above steps, each sequence is encoded at each bit rate point, and each bit rate point calls the encoder's built-in pre-analysis module to analyze the intra-frame / inter-frame features related to the video scene before actual encoding. To save computational effort, this step can use downsampled low-resolution frames to replace the original resolution frames. Among them, the intra-frame / inter-frame prediction module has been greatly simplified compared to actual encoding, and it is necessary to approximate the prediction cost, content motion information, and texture complexity of each frame and local area.

[0188] Step S503: extract statistical features of each frame.

[0189] In the above steps, the video frames of each sequence are divided into two categories: I frames and non-I frames. Non-I frames are classified according to the time domain level. The video features of each frame (set the frame number as p) are further recorded according to the frame type. For the I frame (generally the first frame of the video, p = 0), the recorded video features include: the average intra-frame prediction cost of each region of the frame The base QP value q used for encoding I ; The average value of the content difference between this frame and other frames that directly reference this frame:

[0190]

[0191] Among them, N can be used to represent the number of other frames that directly reference the current I frame, n can be used to represent the reference frame number, and p n It can be used to represent the distance between the reference frame and the current frame, and MAD can be used to represent the average absolute value difference between two frames.

[0192] For non-I frames (assuming that the temporal level of the pth frame of the video sequence is TID=t), the recorded video features include: the mean intra-frame prediction cost of each region of the frame The current frame QP offset Δq output by the encoder's built-in rate allocation strategy t (p); the base QP value q used for encoding t (p); The average content difference between this frame and other frames that directly reference this frame:

[0193]

[0194] Among them, N t It can be used to represent the number of other frames that directly refer to the current frame, which is related to the time domain level t. Further, the above features are integrated into a vector:

[0195]

[0196] Optionally, a coding configuration with a group of pictures (Group Of Pictures, GOP for short) length of 64 is selected, and the maximum time domain level TID is 6, that is, non-I frames can be divided into 7 categories, namely, TID 0, TID 1, ..., TID 6. This embodiment can select other easy-to-calculate features according to the specific encoder type and application scenario, and is not limited to the features listed above.

[0197] Step S504, traverse the QP offset values ​​of each frame.

[0198] In the above steps, for each sequence, a QP offset value ΔQ satisfying the conditions for each frame is obtained by rate-distortion search. According to actual needs, one of obtaining a QP offset value satisfying the conditions at the frame level and estimating a QP offset value satisfying the conditions at the TID level can be selected.

[0199] Traverse to obtain the QP bias value that meets the conditions at the frame level: First, assume that the BD-rate that meets the conditions is 0, the QP of each frame remains unchanged, and check each bitrate point of each frame in the encoding order. Among them, multiple QP bias values ​​are checked at each bitrate point and encoded separately. If it is found that the current BD-rate is better than the recorded BD-rate that meets the conditions, the recorded BD-rate that meets the conditions is updated, and the QP bias used for each bitrate point of the current frame is recorded until all video frames are checked. This method can accurately obtain the QP bias value that meets the conditions for each frame, but it requires multiple traversals and has a high amount of calculation.

[0200] Estimate the QP offset value that satisfies the conditions at the TID level: For each sequence, change the QP value of the video frame with the same TID at the same time, and record the bit rate and quality evaluation indicators such as PSNR and SSIM at each bit rate point. Further compare the situations composed of each bit rate point and each QP offset value, and record the QP offset value when the BD-rate meets the conditions. This method can approximately estimate the QP offset that meets the conditions for each TID situation with less calculation.

[0201] Optionally, the QP offset value range of the above traversal can be {-2, -1, 0, +1, +2}, that is, five cases are checked: QP unchanged, QP increased by 1, QP increased by 2, QP decreased by 1, and QP decreased by 2. For I frames and non-I frames satisfying TID=0, the frame-level QP offset is obtained by traversal, and for non-I frames with TID>=1, the QP offset that satisfies the conditions at the estimated TID level is selected.

[0202] Step S505: record the QP offset value that meets the condition.

[0203] In the above steps, after obtaining the QP offset value ΔQ satisfying the conditions for each frame by using the rate-distortion search method, the obtained QP offset value satisfying the conditions may be recorded.

[0204] Step S506: Use the model to fit the statistical characteristics and the QP offset value that meets the conditions.

[0205] In the above steps, a linear model is used to fit the relationship between feature f and the QP bias value ΔQ that meets the conditions. First, feature f is preprocessed: for I frames, features with a larger dynamic range are Take the logarithm, for non-I frames, the features with larger dynamic range Take the logarithm to ensure that the dimension of the feature after taking the logarithm is the same as QP, and add a scalar 1 at the end of the feature to facilitate the subsequent fitting to obtain the offset of the linear model. The preprocessed features can be expressed by the following formula:

[0206]

[0207] Secondly, construct a training set: suppose that M records are extracted from multiple frames of multiple video sequences, and each record (set the sequence number to be m) includes the preprocessed features And the corresponding QP offset value ΔQ that satisfies the conditions m . The preprocessed features and the QP offset value ΔQ that satisfies the conditions m Combined into the input matrix X and output vector y of the linear model:

[0208]

[0209] Suppose the weight vector required for the linear model is:

[0210]

[0211] The QP bias value that meets the conditions can be obtained by fitting:

[0212]

[0213] The above is expressed in matrix form as:

[0214]

[0215] in, Can be used to represent ΔQ m The predicted value of Can be used to represent the predicted value of y, w I,1 ~w I,3 and w t,1 ~w t,4 is the weight coefficient of each feature, b I and b t is the offset of the fitted model. Since predictions usually have errors, In order to improve the accuracy as much as possible, we need to find a set of weight vectors that minimize the sum of squared prediction errors. That is, the least squares optimization objective:

[0216]

[0217] The weight vector that meets the conditions is obtained by matrix operation:

[0218]

[0219] In summary, the model training is complete, and the required parameters have been saved to the vector It can be used to adjust the QP value of each frame in actual encoding.

[0220] Optionally, this embodiment may select other fitting models besides linear models, such as support vector machines, random forests, multi-layer neural networks, etc., but the selected model should not be too complex, and the trainable parameters therein should be much less than the number of samples to avoid overfitting.

[0221] Figure 6 is a flowchart of a video encoding according to an embodiment of the present application, such as Figure 6 As shown, the video encoding process may include the following steps:

[0222] Step S601: Check whether all video frames have been read.

[0223] In the above steps, it can be determined whether the video frames of the video sequence have been encoded. If the video frames of the video sequence have not been encoded, the process proceeds to step S602, and the encoding step is continued until all the video frames are encoded. If the video frames of the video sequence have been encoded, the video encoding is terminated.

[0224] Step S602: Pre-analyze the current frame content.

[0225] In the above steps, the pre-analysis module built into the encoder is called to analyze intra-frame / inter-frame features related to the video scene before actual encoding. This step is the same as step S502 of model training.

[0226] Step S603: extract statistical features.

[0227] In the above steps, the content feature f of each frame of the video is recorded according to the frame type and the temporal level. This step is the same as step S503 of model training. The preprocessed feature f is further calculated. * .

[0228] Step S604: calling the model to predict a QP offset value that meets the conditions.

[0229] In the above steps, the feature f * By inputting the fitting model obtained from model training, the QP bias value that meets the conditions at the frame level can be obtained.

[0230] Step S605: Whether to enable the CTU level decision strategy.

[0231] In the above steps, it is determined whether to enable the CTU-level decision strategy. If the CTU-level decision strategy is enabled, the process proceeds to step S606 to determine whether to adjust the QP of each CTU; if the CTU-level decision strategy is not enabled, the process proceeds to step S607 to adjust the frame-level QP value. Optionally, the embodiment may choose to enable the CTU-level decision strategy.

[0232] Step S606, adjusting the CTU-level QP value according to the picture content.

[0233] In the above steps, for each CTU of the current frame, check whether the picture content is too smooth, complex, has violent motion, or has gentle motion, and judge whether the QP value of the current CTU needs to be increased. Bias value. Among them, A positive value indicates that the actual QP value increases. A negative value indicates that the actual QP value is reduced. For example, for a frame with TID 0, the following strategy is adopted:

[0234] like It is necessary to satisfy the average absolute value difference d0(p)>τ between the current CTU and the corresponding position of the adjacent frame M+ And the current CTU pixel value variance v0(p)>τ V+ time (i.e. the content changes in both spatial and temporal domains, but not too smooth, where τ M+ and τ V+ is the decision threshold), the QP value of this CTU is adjusted; if The average absolute value difference between the current CTU and the corresponding position of the adjacent frame needs to be d0(p) < τ M- And the current CTU pixel value variance v0(p)>τ V- When (i.e. the content is not too complicated, τ M- and τ V- is the decision threshold), the QP value of this CTU is adjusted to avoid excessive increase in bit rate.

[0235] Optionally, the above decision threshold may be: M+ =500, τ V+ =1000, τ M- =2000,τ V- =2000 and τ Q =10.

[0236] In addition, if the adjusted QP value of the current CTU is too large or too small (for example, where τ Q is the decision threshold), the adjustment is cancelled and the QP value is kept unchanged to avoid excessively high risk of model prediction.

[0237] For frames of other TIDs, similar strategies can be adopted, or part of the training video sequence can be used as a validation set, and the above strategy can be flexibly adjusted according to the encoding performance on this part of the sequence to ensure that the algorithm has BD-rate benefits on most videos.

[0238] Step S607: directly adjust the frame-level QP value.

[0239] In the above steps, the QP value of the current frame CTU is increased by Bias value, A positive value indicates that the actual QP value increases. A negative value means that the actual QP value decreases.

[0240] Step S608, encode the current frame to obtain an encoding result.

[0241] In the above steps, the current frame may be encoded according to a conventional process.

[0242] In the process of bit rate control, this embodiment accurately predicts the QP bias value that meets the conditions at the frame level according to the intra-frame / inter-frame features in the specific video scene to improve the overall coding efficiency, and selects part of the area in the whole frame through the CTU-level decision strategy to adjust the QP value to reduce the risk of performance loss. In addition, this embodiment has strong versatility, in which the principle of feature extraction and model fitting can be used for encoders of different types and versions, and does not limit whether there are other bit rate control optimization algorithms in the encoder, and the parameters used in the specific steps can also be flexibly adjusted.

[0243] In one example, existing rate control algorithms are mostly used to improve control accuracy, that is, to ensure that the rate of the entire video sequence is always stable, but they do not meet the needs of actual video scenes. In actual scenes, more attention is usually paid to coding efficiency.

[0244] However, the present application proposes to predict a reasonable QP bias value for each frame based on the intra-frame / inter-frame features in a specific video scene to improve the overall coding efficiency. On this basis, the CTU-level decision strategy is further used to determine whether to adjust the QP value of each CTU, filter out situations where the fitting model prediction risk is high, and improve the stability of the algorithm between different video scenes. This embodiment has a low amount of calculation, and only needs to use the frame-level and CTU-level features obtained by the built-in pre-coding link of the encoder to obtain the features required for the fitting model. Further, step S506 can quickly predict the QP bias value that meets the conditions, which does not bring additional calculation costs and is easy to apply in fast encoders.

[0245] This embodiment is applied to the short video end of the S266 encoder, and the coding efficiency is tested on 63 video sequences. As shown in Table 1, the gear position can be different performance modes of the encoder, such as medium mode and slow mode. Slow mode may indicate that the encoder will spend more time optimizing the compression of the video, which may result in higher coding efficiency and video quality. The peak signal-to-noise ratio PSNR can be used to objectively evaluate the error level in the video, and the structural similarity SSIM can be used to measure the similarity of structure and texture information. The average BD-rate shows the average bit rate percentage that can be saved by using the encoder optimized by this embodiment while maintaining the same or similar video quality relative to the benchmark encoder. A negative value of BD-rate indicates bit rate savings, that is, the optimized encoder can use fewer bits on average when encoding videos of the same quality, reflecting the improvement in coding efficiency.

[0246] Table 1 Coding efficiency test table

[0247]

[0248] For the new generation H.266 / VVC video coding standard, the existing rate control optimization algorithm is mainly used to improve control accuracy rather than prioritize coding efficiency, and is usually designed for standard encoders, which is difficult to meet the fast coding requirements in actual scenarios. In response to this, this application proposes a rate control optimization algorithm for adaptive video scenes, which improves coding efficiency by selecting a QP offset value that meets the conditions for each frame.

[0249] This embodiment can extract appropriate intra-frame / inter-frame features for video frames of various types and time domain levels, and fit the relationship between the features and the QP bias that meets the conditions through the prediction model. This embodiment includes two-level QP optimization strategies at the frame level and CTU level, which can filter out situations with high prediction risks on the basis of optimizing the frame-level QP bias and improve the stability of the algorithm.

[0250] This embodiment is a general-purpose bit rate allocation optimization solution. The principle of feature extraction and model fitting is not limited to a specific encoder version. It can be used not only for standard encoders VTM or open source fast encoders VVenC, but also for third-party encoders that support H.266 / VVC or similar standards. It has a wide range of applications. This embodiment is simple to implement and has a low amount of calculation. It only needs to use the information obtained by pre-analysis before encoding each frame to quickly predict the frame-level QP bias. Since the pre-analysis link can use the downsampled frame content instead of the original resolution content, or it is already built into the encoder, it does not require too much additional calculation and is easy to implement in actual scenarios.

[0251] In an embodiment of the present application, a current video frame to be encoded is obtained from a video sequence; based on the current video frame, a video feature set associated with a video scene in which the video sequence is located is determined, wherein the video features in the video feature set are used to describe the picture characteristics of the current video frame in the video scene; the video feature set is converted into a video feature vector that matches a target model, wherein the target model is trained based on video feature vector samples and corresponding quantization parameter offset samples; the video feature vector is predicted using the target model to obtain a quantization parameter offset of the current video frame; the quantization parameter of the current video frame is adjusted using the quantization parameter offset; the current video frame is encoded using the adjusted quantization parameter to obtain an encoding result. That is to say, the embodiment of the present application extracts a video feature set of the current video frame, converts the extracted video feature set into a video feature vector that matches the target model, accurately predicts a quantization parameter offset based on the video feature vector, and further adjusts the quantization parameter based on the quantization parameter offset according to actual conditions, so as to encode the current video frame using the adjusted quantization parameter to obtain an encoding result. Since the quantization parameter offset is predicted based on the target model to adjust the quantization parameter, it avoids determining a quantization parameter that meets the conditions through multiple traversals and tests, thereby achieving the technical effect of improving the efficiency of video encoding and solving the technical problem of low efficiency of video encoding.

[0252] Figure 7 is a structural block diagram of a computing environment of a video encoding and model determination method according to an embodiment of the present application, such as Figure 7 As shown, computing environment 701 includes multiple computing nodes (such as servers) running on a distributed network (shown as 710-1, 710-2, ..., in the figure). The computing nodes all contain local processing and memory resources, and end users 702 can remotely run applications or store data in computing environment 701. Applications can be provided as multiple services 720-1, 720-2, 720-3 and 720-4 in computing environment 701, representing services "A", "D", "E" and "H" respectively.

[0253] The end user 702 can provide and access services through a web browser or other software application on the client, and in some embodiments, the end user 702's provision and / or request can be provided to the entry gateway 730. The entry gateway 730 can include a corresponding agent to handle the provision and / or request for the service (one or more services provided in the computing environment 701).

[0254] Services are provided or deployed based on various virtualization technologies supported by the computing environment 701. In some embodiments, services can be provided based on virtual machine (VM)-based virtualization, container-based virtualization, and / or similar methods. Virtual machine-based virtualization can be to simulate a real computer by initializing a virtual machine, and execute programs and applications without directly contacting any actual hardware resources. While the virtual machine virtualizes the machine, according to container-based virtualization, a container can be started to virtualize the entire operating system (Operating System, referred to as OS) so that multiple workloads can run on a single operating system instance.

[0255] In an embodiment based on container virtualization, several containers of a service can be assembled into a Pod (e.g., a Kubernetes Pod). Figure 7 As shown, service 720-2 can be equipped with one or more Pods 740-1, 740-2, ..., 740-N (collectively referred to as Pods). Pods can include a proxy 745 and one or more containers 742-1, 742-2, ..., 742-M (collectively referred to as containers). One or more containers in the Pod process requests related to one or more corresponding functions of the service, and the proxy 745 generally controls network functions related to the service, such as routing, load balancing, etc. Other services can also be equipped with Pods similar to Pods.

[0256] During operation, executing a user request from the end user 702 may require calling one or more services in the computing environment 701, and executing one or more functions of a service may require calling one or more functions of another service. Figure 7 As shown, service "A" 720-1 receives a user request from end user 702 from ingress gateway 730, service "A" 720-1 may call service "D" 720-2, and service "D" 720-2 may request service "E" 720-3 to perform one or more functions.

[0257] The computing environment described above can be a cloud computing environment, where the allocation of resources is managed by the cloud service provider, allowing the development of functions without considering the implementation, adjustment or expansion of servers. The computing environment allows developers to execute code in response to events without building or maintaining complex infrastructure. Services can be divided into a set of functions that can be automatically and independently scaled, rather than expanding a single hardware device to handle potential loads.

[0258] According to an embodiment of the present application, there is also provided a method for implementing the above Figure 2 A video encoding device according to the video encoding method is shown.

[0259] Figure 8 is a schematic diagram of a video encoding device according to an embodiment of the present application, such as Figure 8 As shown, the video encoding device 800 may include: a first acquisition unit 802 , a determination unit 804 , a conversion unit 806 , a prediction unit 808 , an adjustment unit 810 and an encoding unit 812 .

[0260] The first acquisition unit 802 is used to acquire a current video frame to be encoded from a video sequence.

[0261] The determination unit 804 is configured to determine, based on the current video frame, a video feature set associated with the video scene in which the video sequence is located, wherein the video features in the video feature set are used to describe the picture characteristics of the current video frame in the video scene.

[0262] The conversion unit 806 is used to convert the video feature set into a video feature vector that matches the target model, wherein the target model is trained based on the video feature vector samples and the corresponding quantization parameter offset samples.

[0263] The prediction unit 808 is used to predict the video feature vector using the target model to obtain the quantization parameter offset of the current video frame.

[0264] The adjustment unit 810 is used to adjust the quantization parameter of the current video frame by using the quantization parameter offset.

[0265] The encoding unit 812 encodes the current video frame using the adjusted quantization parameter to obtain an encoding result.

[0266] It should be noted here that the above-mentioned first acquisition unit 802, determination unit 804, conversion unit 806, prediction unit 808, adjustment unit 810 and encoding unit 812 correspond to steps S202 to S212, and the six units are the same as the examples and application scenarios implemented by the corresponding steps, but are not limited to the above-disclosed contents. It should be noted that the above-mentioned units can be hardware components or software components stored in a memory (e.g., memory 1004) and processed by one or more processors (e.g., processors 1002a, 1002b..., 1002n), and the above-mentioned units can also be run in computer terminal A as part of the device.

[0267] In the video encoding device, the current video frame to be encoded is obtained from the video sequence by the first acquisition unit 802. The video feature set associated with the video scene where the video sequence is located is determined by the determination unit 804 based on the current video frame, wherein the video features in the video feature set are used to describe the picture characteristics of the current video frame in the video scene. The video feature set is converted into a video feature vector matching the target model by the conversion unit 806, wherein the target model is obtained by training based on the video feature vector samples and the corresponding quantization parameter offset samples. The video feature vector is predicted by the prediction unit 808 using the target model to obtain the quantization parameter offset of the current video frame. The quantization parameter of the current video frame is adjusted by the adjustment unit 810 using the quantization parameter offset. The current video frame is encoded by the encoding unit 812 using the adjusted quantization parameter to obtain an encoding result. That is to say, the embodiment of the present application extracts a video feature set of the current video frame, converts the extracted video feature set into a video feature vector that matches the target model, accurately predicts a quantization parameter offset based on the video feature vector, and further adjusts the quantization parameter based on the quantization parameter offset according to actual conditions, so as to encode the current video frame using the adjusted quantization parameter to obtain an encoding result. Since the quantization parameter offset is predicted based on the target model to adjust the quantization parameter, it avoids determining a quantization parameter that meets the conditions through multiple traversals and tests, thereby achieving the technical effect of improving the efficiency of video encoding and solving the technical problem of low efficiency of video encoding.

[0268] According to an embodiment of the present application, there is also provided a method for implementing the above Figure 3 The model determination method shown is a model determination device.

[0269] Fig. 9 is a schematic diagram of a model determination device according to an embodiment of the present application, such as Fig. 9 As shown, the model determination device 900 may include: a second acquisition unit 902 and a training unit 904.

[0270] The second acquisition unit 902 is used to acquire video feature vector samples and acquire quantization parameter offset samples corresponding to the video feature vector samples.

[0271] The training unit 904 is used to train a target model using video feature vector samples and quantization parameter offset samples, wherein the target model is used to predict the video feature vector matching the target model to obtain the quantization parameter offset, the video feature vector is converted from a video feature set associated with a video scene where a video sequence is located, the video features in the video feature set are used to describe the picture characteristics of a current video frame to be encoded in the video sequence under the video scene, the video feature set is determined based on the current video frame, the quantization parameter offset is used to adjust the quantization parameter of the current video frame, and the adjusted quantization parameter is used to encode the current video frame to obtain an encoding result.

[0272] Here, the second acquisition unit 902 and the training unit 904 correspond to steps S302 to S304, and the two units and the corresponding steps implement the same examples and application scenarios, but are not limited to the above disclosed content. It should be noted that the above units can be hardware components or software components stored in a memory (e.g., memory 1004) and processed by one or more processors (e.g., processors 1002a, 1002b..., 1002n), and the above units can also be run in computer terminal A as part of the device.

[0273] In the determination device of the model, a video feature vector sample is obtained by a second acquisition unit 902, and a quantization parameter offset sample corresponding to the video feature vector sample is obtained. The training unit 904 uses the video feature vector sample and the quantization parameter offset sample to train a target model, wherein the target model is used to predict the video feature vector matching the target model to obtain the quantization parameter offset, the video feature vector is converted from a video feature set associated with a video scene where a video sequence is located, the video features in the video feature set are used to describe the picture characteristics of a current video frame to be encoded in the video sequence under the video scene, the video feature set is determined based on the current video frame, the quantization parameter offset is used to adjust the quantization parameter of the current video frame, and the adjusted quantization parameter is used to encode the current video frame to obtain an encoding result. That is to say, the embodiment of the present application extracts a video feature set of the current video frame, converts the extracted video feature set into a video feature vector that matches the target model, accurately predicts a quantization parameter offset based on the video feature vector, and further adjusts the quantization parameter based on the quantization parameter offset according to actual conditions, so as to encode the current video frame using the adjusted quantization parameter to obtain an encoding result. Since the quantization parameter offset is predicted based on the target model to adjust the quantization parameter, it avoids determining a quantization parameter that meets the conditions through multiple traversals and tests, thereby achieving the technical effect of improving the efficiency of video encoding and solving the technical problem of low efficiency of video encoding.

[0274] The embodiment of the present application may provide a computer terminal, which may be any computer terminal device in a computer terminal group. Optionally, in this embodiment, the computer terminal may also be replaced by a terminal device such as a mobile terminal.

[0275] Optionally, in this embodiment, the computer terminal may be located in at least one network device among a plurality of network devices of the computer network.

[0276] In this embodiment, the above-mentioned computer terminal can execute the program code of the following steps in the video encoding method: obtaining the current video frame to be encoded from the video sequence; based on the current video frame, determining the video feature set associated with the video scene where the video sequence is located, wherein the video features in the video feature set are used to describe the picture characteristics of the current video frame in the video scene; converting the video feature set into a video feature vector that matches the target model, wherein the target model is trained based on video feature vector samples and corresponding quantization parameter offset samples; using the target model to predict the video feature vector to obtain the quantization parameter offset of the current video frame; using the quantization parameter offset, adjusting the quantization parameter of the current video frame; using the adjusted quantization parameter, encoding the current video frame to obtain the encoding result.

[0277] Optionally, Fig.10 is a structural block diagram of a computer terminal according to an embodiment of the present application, such as Fig.10 As shown, the computer terminal A may include: one or more (only one is shown in the figure) processors 1002 , a memory 1004 and a transmission device 1006 .

[0278] Among them, the memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the method and device for determining the video coding and model in the embodiment of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, the above-mentioned method for determining the video coding and model is realized. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the terminal A via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0279] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: obtain video feature vector samples, and obtain quantization parameter offset samples corresponding to the video feature vector samples; use the video feature vector samples and the quantization parameter offset samples to train a target model, wherein the target model is used to predict the video feature vector matching the target model to obtain the quantization parameter offset, the video feature vector is converted from a video feature set associated with the video scene where the video sequence is located, the video features in the video feature set are used to describe the picture characteristics of the current video frame to be encoded in the video sequence under the video scene, the video feature set is determined based on the current video frame, the quantization parameter offset is used to adjust the quantization parameter of the current video frame, and the adjusted quantization parameter is used to encode the current video frame to obtain the encoding result.

[0280] It can be understood by those skilled in the art that Fig.10 The structure shown is for illustration only, and the computer terminal A may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (Mobile Internet Devices, abbreviated as MID), a personal access display (Personal Access Display, abbreviated as PAD), and other terminal devices. Fig.10 It does not limit the structure of the above-mentioned computer terminal A. For example, the computer terminal A may also include Fig.10 More or fewer components (such as network interfaces, display devices, etc.) shown in, or having Fig.10 Different configurations are shown.

[0281] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0282] The embodiment of the present application further provides a computer-readable storage medium. Optionally, in this embodiment, the computer-readable storage medium can be used to store the program code executed by the video encoding and model determination method provided in the first embodiment.

[0283] Optionally, in this embodiment, the computer-readable storage medium may be located in any one of the computer terminals in a computer terminal group in a computer network, or in any one of the mobile terminals in a mobile terminal group.

[0284] Optionally, in this embodiment, a computer-readable storage medium is configured to store program code for performing the following steps: obtaining a current video frame to be encoded from a video sequence; determining a video feature set associated with a video scene in which the video sequence is located based on the current video frame, wherein the video features in the video feature set are used to describe the picture characteristics of the current video frame in the video scene; converting the video feature set into a video feature vector that matches a target model, wherein the target model is trained based on video feature vector samples and corresponding quantization parameter offset samples; predicting the video feature vector using the target model to obtain a quantization parameter offset of the current video frame; adjusting the quantization parameter of the current video frame using the quantization parameter offset; encoding the current video frame using the adjusted quantization parameter to obtain an encoding result.

[0285] As an optional example, a computer-readable storage medium is configured to store program code for performing the following steps: obtaining a video feature vector sample and obtaining a quantization parameter offset sample corresponding to the video feature vector sample; using the video feature vector sample and the quantization parameter offset sample to train a target model, wherein the target model is used to predict a video feature vector matching the target model to obtain a quantization parameter offset, the video feature vector is converted from a video feature set associated with a video scene in which a video sequence is located, the video features in the video feature set are used to describe the picture characteristics of a current video frame to be encoded in the video sequence in the video scene, the video feature set is determined based on the current video frame, the quantization parameter offset is used to adjust the quantization parameter of the current video frame, and the adjusted quantization parameter is used to encode the current video frame to obtain an encoding result.

[0286] The embodiment of the present application further provides a computer program product. Optionally, in this embodiment, the computer program product may include a computer program, and the computer program implements the method provided in the embodiment when executed by a processor.

[0287] Optionally, the computer program product may include a non-volatile computer-readable storage medium, and the non-volatile computer-readable storage medium may be used to store a computer program. When the computer program is executed by a processor, the method provided in the above embodiment is implemented.

[0288] An embodiment of the present application may provide an electronic device, which may include a memory and a processor.

[0289] Fig.11It is a block diagram of an electronic device according to a method for determining a video encoding and model according to an embodiment of the present application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0290] like Fig.11 As shown, the device 1100 includes a computing unit 1101, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1102 or a computer program loaded from a storage unit 1108 into a random access memory (RAM) 1103. In the RAM 1103, various programs and data required for the operation of the device 1100 can also be stored. The computing unit 1101, the ROM 1102, and the RAM 1103 are connected to each other via a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104.

[0291] A number of components in the device 1100 are connected to the I / O interface 1105, including: an input unit 1106, such as a keyboard, a mouse, etc.; an output unit 1107, such as various types of displays, speakers, etc.; a storage unit 1108, such as a disk, an optical disk, etc.; and a communication unit 1109, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1109 allows the device 1100 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0292] The computing unit 1101 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1101 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSP), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1101 performs the various methods and processes described above, such as a video encoding method and a method for determining a model. For example, in some embodiments, the video encoding and model determination methods may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 1108. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 1100 via the ROM 1102 and / or the communication unit 1109. When the computer program is loaded into the RAM 1103 and executed by the computing unit 1101, one or more steps of the video coding and model determination method described above may be performed. Alternatively, in other embodiments, the computing unit 1101 may be configured to perform the video coding and model determination method in any other appropriate manner (e.g., by means of firmware).

[0293] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard parts (ASSPs), system-on-a-chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that may be executed and / or interpreted on a programmable system including at least one programmable processor that may be a dedicated or general purpose programmable processor that may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0294] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0295] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Fig.12 is a hardware structure block diagram of a computer terminal (or mobile device) for implementing a method for determining a video encoding and model according to an embodiment of the present application, such as Fig.12 As shown, the computer terminal 120 (or mobile device) may include one or more (shown in the figure as 1202a, 1202b, ..., 1202n) processors 1202 (the processor 1202 may include but is not limited to a processing device such as a microprocessor (Microcontroller Unit, referred to as MCU) or a programmable logic device (Field Programmable Gate Array, referred to as FPGA)), a memory 1204 for storing data, and a transmission device 1206 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a Universal Serial Bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Fig.12 The structure shown is only for illustration and does not limit the structure of the above electronic device. Fig.12 More or fewer components as shown, or with Fig.12 Different configurations are shown.

[0296] Fig.12 The hardware structure block diagram shown can be used not only as an exemplary block diagram of the above-mentioned computer terminal 120 (or mobile device), but also as an exemplary block diagram of the above-mentioned server. In an optional embodiment, Fig.12 The block diagram shows the use of the above Fig.12 The computer terminal 120 (or mobile device) is shown as an embodiment of a computing node in the computing environment 701.

[0297] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by an instruction execution system, device or equipment or for use in combination with an instruction execution system, device or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory for short), an optical fiber, a portable compact disc read-only memory (CD-ROM for short), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0298] To provide interaction with a user, the systems and techniques described herein may be implemented on a computer having: a display device (e.g., a cathode ray tube (CRT) or a liquid crystal display (LCD), a monitor for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a path ball), through which the user may provide input to the computer. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0299] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a Local Area Network (LAN), a Wide Area Network (WAN), and the Internet.

[0300] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0301] It should be noted that the serial numbers of the above-mentioned embodiments of the present application are only for description and do not represent the advantages or disadvantages of the embodiments.

[0302] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0303] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0304] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0305] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0306] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, read-only memories, random access memories, mobile hard disks, magnetic disks or optical disks.

[0307] The above are only preferred implementations of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A video encoding method, characterized in that: include: From the video sequence, obtain the current video frame to be encoded; Based on the current video frame, determining a video feature set associated with the video scene where the video sequence is located, wherein the video features in the video feature set are used to describe the picture characteristics of the current video frame in the video scene; Converting the video feature set into a video feature vector that matches a target model, wherein the target model is trained based on video feature vector samples and corresponding quantization parameter offset samples; Predicting the video feature vector using the target model to obtain a quantization parameter offset of the current video frame; Using the quantization parameter offset, adjusting the quantization parameter of the current video frame; The current video frame is encoded using the adjusted quantization parameter to obtain an encoding result.

2. The method according to claim 1, characterized in that Converting the video feature set into a video feature vector matching the target model includes: Extracting at least one video feature corresponding to at least a frame type of the current video frame from the video feature set; The extracted video features are converted into the video feature vector that matches the target model.

3. The method according to claim 2, characterized in that Converting the extracted video features into the video feature vector matching the target model includes: The extracted video features are integrated to obtain the video feature vector that matches the target model.

4. The method according to claim 3, characterized in that Integrating the extracted video features to obtain the video feature vector that matches the target model includes: Integrate the extracted video features to obtain an initial video feature vector; The initial video feature vector is preprocessed to obtain the video feature vector having the same dimension as the quantization parameter and matching the target model.

5. The method according to claim 2, characterized in that: The video feature set includes an intra-frame feature set and an inter-frame feature set, wherein the intra-frame features in the intra-frame feature set are used to describe the characteristics of the picture content of the current video frame in the video scene, and the inter-frame features in the inter-frame feature set are used to describe the characteristics of the picture content between the current video frame and the corresponding reference video frame in the video scene. At least one video feature corresponding to at least a frame type of the current video frame is extracted from the video feature set, including: In response to the frame type being an I frame type, extracting a mean intra-frame prediction cost of the current video frame from the intra-frame feature set, and / or extracting a mean inter-frame content difference between the current video frame and the reference video frame from the inter-frame feature set; In response to the frame type being a non-I frame type, the intra-frame prediction cost mean corresponding to the time domain level of the current video frame is extracted from the intra-frame feature set, and / or the inter-frame content difference mean corresponding to the time domain level is extracted from the intra-frame feature set.

6. The method according to claim 1, characterized in that Predicting the video feature vector using the target model to obtain a quantization parameter offset of the current video frame includes: The video feature vector is predicted using the target model and the frame type of the current video frame to obtain the quantization parameter offset.

7. The method according to claim 6, characterized in that Predicting the video feature vector by using the target model and the frame type of the current video frame to obtain the quantization parameter offset includes: In the target model, determining a weight vector matching the frame type, wherein the weight vector includes at least a weight coefficient of the video feature vector; The video feature vector is adjusted using the weight vector to obtain the quantization parameter offset.

8. The method according to claim 1, characterized in that The current video frame includes a plurality of coding tree units, and adjusting the quantization parameter of the current video frame by using the quantization parameter offset includes: Determining picture content of the coding tree unit; The quantization parameter of the coding tree unit is adjusted using the quantization parameter offset and the picture content.

9. The method according to claim 8, characterized in that Adjusting the quantization parameter of the coding tree unit by using the quantization parameter offset and the picture content includes: In response to the quantization parameter offset being greater than an offset threshold, and the complexity of the picture content in the spatial domain and the temporal domain being greater than a complexity threshold, adjusting the quantization parameter of the coding tree unit by using the quantization parameter offset, wherein the complexity is used to indicate the complexity of the picture content; In response to the quantization parameter offset being not greater than the offset threshold and the complexity of the picture content being less than a complexity threshold, the quantization parameter of the coding tree unit is adjusted using the quantization parameter offset, wherein the complexity is used to represent the complexity of the picture content.

10. The method according to claim 1, characterized in that The current video frame includes a plurality of coding tree units, and adjusting the quantization parameter of the current video frame by using the quantization parameter offset includes: The quantization parameter of each of the coding tree units is adjusted using the quantization parameter offset.

11. The method according to any one of claims 1 to 10, characterized in that The method further comprises: In response to the adjusted quantization parameter being greater than a first quantization parameter threshold or less than a second quantization parameter threshold, maintaining the quantization parameter before adjustment; The current video frame is encoded using the quantization parameter before adjustment to obtain the encoding result.

12. A method for determining a model, characterized in that: include: Obtaining a video feature vector sample, and obtaining a quantization parameter offset sample corresponding to the video feature vector sample; The target model is trained by using the video feature vector samples and the quantization parameter offset samples, wherein the target model is used to predict the video feature vector matching the target model to obtain the quantization parameter offset, the video feature vector is converted from a video feature set associated with the video scene where the video sequence is located, the video features in the video feature set are used to describe the picture characteristics of the current video frame to be encoded in the video sequence under the video scene, the video feature set is determined based on the current video frame, the quantization parameter offset is used to adjust the quantization parameter of the current video frame, and the adjusted quantization parameter is used to encode the current video frame to obtain an encoding result.

13. The method according to claim 12, characterized in that Get video feature vector samples, including: Acquire multiple lossless video sequence samples, wherein different lossless video sequence samples come from different video scenes; The video feature vector samples are obtained from the multiple lossless video sequence samples.

14. The method according to claim 13, characterized in that Acquiring the video feature vector sample from the multiple lossless video sequence samples, comprising: Based on the video frame samples in the lossless video sequence samples, determining a video feature sample set associated with the video scene where the lossless video sequence samples are located, wherein the video feature samples in the video feature sample set are used to describe the picture characteristics of the video frame samples in the corresponding video scene; The video feature sample set is converted into the video feature vector sample.

15. The method according to claim 14, characterized in that Determining a video feature sample set associated with a video scene where the lossless video sequence sample is located based on a video frame sample in the lossless video sequence sample includes: Downsampling the video frame samples, wherein a resolution of the downsampled video frame samples is lower than a resolution threshold; The video feature sample set is determined based on the downsampled video frame samples.

16. The method according to claim 14, characterized in that Converting the video feature sample set into the video feature vector sample comprises: Extracting at least one video feature sample corresponding to at least a frame type of the video frame sample from the video feature sample set; Integrate the extracted video feature samples to obtain initial video feature vector samples; The initial video feature vector samples are preprocessed to obtain the video feature vector samples.

17. The method according to claim 16, characterized in that The video feature sample set includes an intra-frame feature sample set and an inter-frame feature sample set. The intra-frame feature samples in the intra-frame feature sample set are used to describe the characteristics of the picture content of the video frame sample in the corresponding video scene. The inter-frame feature samples in the inter-frame feature sample set are used to describe the characteristics of the picture content between the video frame sample and the corresponding reference video frame sample in the corresponding video scene. At least one video feature sample corresponding to at least the frame type of the video frame sample is extracted from the video feature sample set, including: In response to the frame type being an I frame type, extracting a mean intra-frame prediction cost of the video frame sample from the intra-frame feature sample set, and / or extracting a mean inter-frame content difference between the video frame sample and the reference video frame sample from the inter-frame feature sample set; In response to the frame type being a non-I frame type, the intra-frame prediction cost mean corresponding to the time domain level of the video frame sample is extracted from the intra-frame feature sample set, and / or the inter-frame content difference mean corresponding to the time domain level is extracted from the intra-frame feature sample set.

18. The method according to claim 13, characterized in that Obtaining a quantization parameter offset sample corresponding to the video feature vector sample, including: Encoding the lossless video sequence samples using the rate control strategies corresponding to the multiple bit rates to obtain multiple encoding result samples, wherein the multiple encoding result samples correspond to the multiple bit rates; The quantization parameter offset sample is determined from a plurality of initial quantization parameter offset samples corresponding to a plurality of the encoding result samples.

19. The method according to any one of claims 13 to 18, characterized in that Using the video feature vector samples and the quantization parameter offset samples, training to obtain a target model includes: Converting the video feature vector samples into an input matrix of an initial model, and converting the quantization parameter offset samples into an output matrix of the initial model; Fitting the input matrix and the output matrix to obtain weight vector samples corresponding to the initial model, wherein the weight vector samples at least include weight coefficient samples of the video feature vector samples; The target model is established using the weight vector samples and the initial model.

20. A video encoding system, characterized in that: include: The encoder is used to obtain a current video frame to be encoded from the video sequence; Calling a pre-analysis module to determine a video feature set associated with a video scene where the video sequence is located based on the current video frame, wherein the video features in the video feature set are used to describe the picture characteristics of the current video frame in the video scene; converting the video feature set into a video feature vector that matches a target model, wherein the target model is obtained by training based on video feature vector samples and corresponding quantization parameter offset samples; predicting the video feature vector using the target model to obtain a quantization parameter offset of the current video frame; adjusting the quantization parameter of the current video frame using the quantization parameter offset; encoding the current video frame using the adjusted quantization parameter to obtain an encoding result; The server is used to obtain the encoding result.

21. A computing device, characterized in that include: A memory storing an executable program; A processor, configured to run the program, wherein the program executes the method according to any one of claims 1 to 19 when running.

22. An electronic device, characterized in that: include: A memory storing an executable program; A processor, connected to the memory via a bus, and configured to run the program, wherein the program executes the method described in any one of claims 1 to 19 when running.

23. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored executable program, wherein when the executable program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 19.

24. A computer program product, characterized in that The invention comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 19.

Images