Video encoding method and system, electronic device, and storage medium

Abstract

The present application relates to the technical field of video encoding, and discloses a video encoding method and system, an electronic device, and a storage medium. The method comprises: acquiring, from a video sequence, a current video frame to be encoded; on the basis of the current video frame, determining a video feature set associated with a video scene in which the video sequence is located, each video feature in the video feature set being used for describing a picture characteristic of the current video frame in the video scene; converting the video feature set into a video feature vector that matches a target model, the target model being obtained by training on video feature vector samples and corresponding quantization parameter offset samples; using the target model to perform prediction on the video feature vector, and obtaining a quantization parameter offset of the current video frame; adjusting a quantization parameter of the current video frame by using the quantization parameter offset; and encoding the current video frame by using the adjusted quantization parameter, and obtaining an encoding result. The present application solves the technical problem of low efficiency of video encoding.

Description

Video encoding methods, systems, electronic devices and storage media

[0001] Cross-references

[0002] This application incorporates Chinese Patent Application No. 2025100742601, filed on January 16, 2025, entitled “Video Coding Method, System, Electronic Device and Storage Medium”, which is incorporated herein by reference in its entirety. Technical Field

[0003] This application relates to the field of video coding technology, and more specifically, to a video coding method, system, electronic device, and storage medium. Background Technology

[0004] Currently, in the video encoding process, appropriate quantization parameters can be selected for image blocks in each frame of the video sequence to control the required bit rate.

[0005] In related technologies, bitrate control performance is typically adjusted. However, this method primarily aims to improve bitrate control accuracy rather than directly increasing encoding efficiency. Furthermore, most of these methods are designed for standard encoders and lack application to fast encoders, resulting in low video encoding efficiency.

[0006] There is currently no effective solution to the above problems. Summary of the Invention

[0007] This application provides a video encoding method, system, electronic device, and storage medium to at least solve the technical problem of low efficiency in video encoding.

[0008] According to one aspect of the embodiments of this application, a video encoding method is provided. The method may include: obtaining a current video frame to be encoded from a video sequence; determining a video feature set associated with the video scene in which the video sequence is located based on the current video frame, wherein the video features in the video feature set are used to describe the image characteristics of the current video frame in the video scene; converting the video feature set into a video feature vector matching a target model, wherein the target model is trained based on video feature vector samples and corresponding quantization parameter offset samples; predicting the video feature vector using the target model to obtain the quantization parameter offset of the current video frame; adjusting the quantization parameters of the current video frame using the quantization parameter offset; and encoding the current video frame using the adjusted quantization parameters to obtain an encoding result.

[0009] According to another aspect of the embodiments of this application, a method for determining a model is provided. The method may include: acquiring video feature vector samples and acquiring quantization parameter offset samples corresponding to the video feature vector samples; training a target model using the video feature vector samples and the quantization parameter offset samples, wherein the target model is used to predict video feature vectors matching the target model to obtain quantization parameter offsets, the video feature vectors are converted from a video feature set associated with the video scene in which the video sequence is located, the video features in the video feature set are used to describe the image characteristics of the current video frame to be encoded in the video scene, the video feature set is determined based on the current video frame, the quantization parameter offsets are used to adjust the quantization parameters of the current video frame, and the adjusted quantization parameters are used to encode the current video frame to obtain an encoding result.

[0010] According to another aspect of the embodiments of this application, a video encoding system is provided. The system may include: an encoder, configured to acquire a current video frame to be encoded from a video sequence; invoke a pre-analysis module to determine a video feature set associated with the video scene in which the video sequence is located, based on the current video frame, wherein the video features in the video feature set are used to describe the image characteristics of the current video frame in the video scene; convert the video feature set into a video feature vector matching a target model, wherein the target model is trained based on video feature vector samples and corresponding quantization parameter offset samples; predict the video feature vector using the target model to obtain the quantization parameter offset of the current video frame; adjust the quantization parameters of the current video frame using the quantization parameter offset; encode the current video frame using the adjusted quantization parameters to obtain an encoding result; and a server, configured to acquire the encoding result.

[0011] According to another aspect of the embodiments of this application, an electronic device is also provided, which may include a memory and a processor: the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, wherein when the computer-executable instructions are executed by the processor, the method of any one of the above is implemented.

[0012] According to another aspect of the embodiments of this application, a processor is also provided, which is used to run a program, wherein the method of any one of the above is executed when the program is running.

[0013] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored program, wherein, when the program is running, the device where the storage medium is located executes any of the methods described above.

[0014] According to another aspect of the embodiments of this application, a computer program product is also provided, including a non-volatile computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the method of any one of the above.

[0015] In this embodiment, the current video frame to be encoded is obtained from the video sequence; based on the current video frame, a video feature set associated with the video scene in which the video sequence is located is determined, wherein the video features in the video feature set are used to describe the picture characteristics of the current video frame in the video scene; the video feature set is converted into a video feature vector that matches a target model, wherein the target model is trained based on video feature vector samples and corresponding quantization parameter offset samples; the target model is used to predict the video feature vector to obtain the quantization parameter offset of the current video frame; the quantization parameters of the current video frame are adjusted using the quantization parameter offset; and the current video frame is encoded using the adjusted quantization parameters to obtain the encoding result. In other words, this application embodiment extracts the video feature set of the current video frame, converts the extracted video feature set into a video feature vector that matches the target model, accurately predicts the quantization parameter offset based on the video feature vector, and further adjusts the quantization parameter according to the actual situation based on the quantization parameter offset. The adjusted quantization parameter is then used to encode the current video frame to obtain the encoding result. Since the quantization parameter offset is predicted based on the target model to adjust the quantization parameter, it avoids determining the quantization parameter that meets the conditions through multiple traversals and tests, thereby achieving the technical effect of improving the efficiency of video encoding and solving the technical problem of low efficiency in video encoding.

[0016] It is worth noting that the general description above and the detailed description that follow are merely for illustrative purposes and do not constitute a limitation on this application. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0018] Figure 1 is a schematic diagram of an application scenario of a video coding and model determination method according to an embodiment of this application;

[0019] Figure 2 is a flowchart of a video encoding method according to an embodiment of this application;

[0020] Figure 3 is a flowchart of a method for determining a model according to an embodiment of this application;

[0021] Figure 4 is a schematic diagram of a video encoding system according to an embodiment of this application;

[0022] Figure 5 is a flowchart of a model training method according to an embodiment of this application;

[0023] Figure 6 is a flowchart of a video encoding method according to an embodiment of this application;

[0024] Figure 7 is a structural block diagram of the computing environment for a video coding and model determination method according to an embodiment of this application;

[0025] Figure 8 is a schematic diagram of a video encoding device according to an embodiment of this application;

[0026] Figure 9 is a schematic diagram of a model determining device according to an embodiment of this application;

[0027] Figure 10 is a structural block diagram of a computer terminal according to an embodiment of this application;

[0028] Figure 11 is a block diagram of an electronic device according to an embodiment of the present application of a method for determining video encoding and model;

[0029] Figure 12 is a hardware structure block diagram of a computer terminal (or mobile device) for implementing a method for determining video encoding and models according to an embodiment of this application. Detailed Implementation

[0030] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0031] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0032] It should also be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application, such as the data to be verified, are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entry points for users to choose to authorize or refuse.

[0033] First, some nouns or terms that appear in the description of the embodiments of this application shall be interpreted as follows:

[0034] The quantization parameter (QP) is used to derive the quantization step size, which determines the fineness of the quantization of intra-frame / inter-frame prediction residuals. The quantization parameter affects the quality of the compressed image.

[0035] Bitrate, or encoding bitrate, is the number of bits encoded per unit of time, that is, the ratio of the number of bits in a video stream to the playback time. For the same original video, a lower bitrate after encoding indicates a more significant compression effect and more effective removal of information redundancy.

[0036] The Coding Tree Unit (CTU) is the basic unit used to encode each frame of video. In H.266 / Versatile Video Coding (VVC), it is usually 128*128 pixels. Most of the coding process (such as intra / inter-frame prediction, discrete transformation, quantization and entropy coding) is carried out in the CTU. H.266 / VVC is a new generation video coding standard.

[0037] The Temporal Identifier (TID), also known as the temporal level, is used to determine the reference relationship between frames in inter-frame coding. A TID of 0 indicates the highest level, while a larger number indicates a lower level. Frames at lower levels can reference frames at higher levels, but frames at higher levels cannot reference frames at lower levels.

[0038] The video encoding and model determination method provided in this application embodiment can be applied to the application scenario shown in Figure 1, but is not limited thereto. Figure 1 is a schematic diagram of an application scenario of a video encoding and model determination method according to an embodiment of this application. In the application scenario shown in Figure 1, the server 10 can be in the cloud. The server 10 can connect to one or more client devices 20 through a local area network (LAN), a wide area network (WAN), an internet connection, or other types of data networks. The client devices 20 can include, but are not limited to, smartphones, tablets, laptops, PDAs, personal computers, smart home devices, and in-vehicle devices. The client devices together constitute the client relative to the server. An operation interface for obtaining video sequences can be deployed on the graphical user interface of the client device. The client device 20 can interact with the user through the graphical user interface to implement the video encoding and model determination method provided in this application embodiment.

[0039] In this embodiment, the system consisting of client device 20 and server 10 can perform the following steps: A corresponding operation can be performed on the operation interface of client device 20 to obtain a video sequence. The client device can obtain the video sequence and send it to the server via the network. After receiving the video sequence, the server can perform the following steps: Step S102, obtain the current video frame to be encoded from the video sequence; Step S104, based on the current video frame, determine the video feature set associated with the video scene where the video sequence is located, wherein the video features in the video feature set are used to describe the picture characteristics of the current video frame in the video scene; Step S106, convert the video feature set into a video feature vector matching the target model, wherein the target model is trained based on video feature vector samples and corresponding quantization parameter offset samples; Step S108, use the target model to predict the video feature vector to obtain the quantization parameter offset of the current video frame; Step S110, use the quantization parameter offset to adjust the quantization parameters of the current video frame; Step S112, use the adjusted quantization parameters to encode the current video frame to obtain the encoding result.

[0040] Under the above operating environment, this application provides a video encoding method as shown in Figure 2. Figure 2 is a flowchart of a video encoding method according to an embodiment of this application. As shown in Figure 2, the method may include the following steps:

[0041] Step S202: Obtain the current video frame to be encoded from the video sequence.

[0042] In the technical solution provided by step S202 of this application, the current video frame to be encoded can be obtained from the video sequence. The video sequence can be a sequence of consecutive video frames arranged in chronological order. This video sequence can include video content, such as movie clips, etc. This is merely an example and does not impose specific limitations on the video content of the video sequence.

[0043] In this embodiment, during video encoding, the video sequence is input into the encoder, which processes each video frame in the video sequence. The current video frame can be the video frame currently being processed by the encoder. The encoder can determine the encoding method for the current video frame based on the content features of the video sequence (e.g., motion, texture complexity, etc.) and encoding parameters.

[0044] Optionally, after acquiring the video sequence, it can be input into an encoder, which performs steps such as reading, parsing, buffering, and encoding. After the encoder parses the video sequence, the parsed video frames can be buffered into a buffer, and further, video frames can be retrieved from the buffer as the current video frames to be encoded.

[0045] Step S204: Based on the current video frame, determine the video feature set associated with the video scene where the video sequence is located.

[0046] In the technical solution provided in step S204 of this application, after obtaining the current video frame to be encoded from the video sequence, a video feature set associated with the video scene in which the video sequence is located can be determined based on the obtained current video frame. The video scene can be any scene contained in the video sequence, such as a moving scene, a still scene, a scene with smooth textures, or a complex scene. This is merely an example and does not impose specific limitations on the type of video scene.

[0047] In this embodiment, the video feature set may include multiple video features related to the video scene extracted from the current video frame. The video features in the video feature set can be used to describe the image characteristics of the current video frame within the video scene. These video features may include intra-frame features and inter-frame features. For example, intra-frame features may be the average intra-frame prediction cost of the current video frame, and inter-frame features may be the average inter-frame content difference between the current video frame and a reference video frame. This is merely an illustrative example, and no specific limitations are imposed on the type of video features.

[0048] Optionally, after obtaining the current video frame to be encoded from the video sequence, the encoder's built-in pre-analysis module can be invoked to analyze the video feature set related to the video scene before actual encoding. This video feature set may include intra-frame feature sets and inter-frame feature sets. The pre-analysis module may include an intra-frame / inter-frame prediction module, which can extract the aforementioned intra-frame and inter-frame feature sets.

[0049] It should be noted that, to save computation, the above steps use downsampled low-resolution frames instead of the original resolution frames. The intra / inter-frame prediction module has been simplified compared to the actual encoding, and it needs to approximate the prediction cost, content motion information, and texture complexity of each frame and local region. In other words, the intra / inter-frame prediction module is a fast preprocessing process before the actual encoding, and it is an approximate estimate of the actual encoding. The intra-frame feature sets and inter-frame feature sets extracted above are also approximate estimated features in the pre-analysis.

[0050] Step S206: Convert the video feature set into a video feature vector that matches the target model.

[0051] In the technical solution provided in step S206 of this application, after determining the video feature set associated with the video scene where the video sequence is located based on the current video frame, the determined video feature set can be converted into a video feature vector that matches the target model. The target model is trained based on video feature vector samples and corresponding quantization parameter offset samples.

[0052] In this embodiment, the target model can be trained based on video feature vector samples and corresponding quantization parameter offset samples. The target model can also be called a fitting model.

[0053] In this embodiment, the video feature vector samples are derived from video sequence samples. The quantization parameter offset samples, used in the encoding of the video sequence samples, ensure that the encoder's performance meets the requirements of practical applications. This performance can be measured by rate-distortion cost, reflecting the encoding efficiency of the encoder or algorithm. A smaller rate-distortion cost indicates better encoder performance during the encoding process. Rate-distortion cost can include, but is not limited to, metrics derived from bit rate, mean squared error (MSE), peak signal-to-noise ratio (PSNR), and structural similarity (SSIM).

[0054] Optionally, after determining the video feature set associated with the video scene where the video sequence is located based on the current video frame, the video feature set can be transformed into a video feature vector that matches the fitted model. Optionally, multiple video features in the video feature set can be integrated to obtain a video feature vector.

[0055] Step S208: Use the target model to predict the video feature vector to obtain the quantization parameter offset of the current video frame.

[0056] In the technical solution provided in step S208 of this application, after converting the video feature set into a video feature vector that matches the target model, the target model can be used to predict the video feature vector to obtain the quantization parameter offset of the current video frame. The quantization parameter offset can be an adjustment amount for adjusting the quantization parameters of the current video frame, which can be used to optimize the coding efficiency and picture quality of the video sequence. It can also be called the quantization parameter QP offset value, and can be used... To express.

[0057] Optionally, based on video feature vector samples and corresponding quantization parameter offset samples, an initial fitting model can be trained to obtain a fitted model. After the fitted model is trained, it can be used to determine the QP bias value in actual encoding. The obtained video feature vectors can be input into the fitted model, which predicts the video feature vectors to obtain the QP bias value. This QP bias value can then be used to adjust the QP of the current video frame.

[0058] Step S210: Adjust the quantization parameters of the current video frame using the quantization parameter offset. In the technical solution provided by step S210 of this application, after predicting the video feature vector using the target model to obtain the quantization parameter offset of the current video frame, the quantization parameters of the current video frame can be adjusted using the obtained quantization parameter offset to obtain the adjusted quantization parameters.

[0059] Optionally, after obtaining the QP offset value, the QP of the current video frame can be adjusted using the QP offset value. For example, the QP of the current video frame can be added to the QP offset value to achieve the purpose of adjusting the QP of the current video frame. The QP offset value... It has positive and negative values, depending on the QP bias value. When the value is positive, it indicates that the QP of the current video frame has increased; when the QP offset value is negative... When it is negative, it means that the QP of the current video frame has decreased.

[0060] Step S212: Encode the current video frame using the adjusted quantization parameters to obtain the encoding result.

[0061] In the technical solution provided in step S212 of this application, after adjusting the quantization parameters of the current video frame using the quantization parameter offset to obtain the adjusted quantization parameters, the current video frame can be encoded using the adjusted quantization parameters to obtain the encoding result.

[0062] Optionally, after adjusting the QP of the current video frame to obtain the adjusted QP, the adjusted QP can be used to encode the current video frame to obtain the encoding result.

[0063] Through steps S202 to S212 of this application, the current video frame to be encoded is obtained from the video sequence; based on the current video frame, a video feature set associated with the video scene in which the video sequence is located is determined, wherein the video features in the video feature set are used to describe the picture characteristics of the current video frame in the video scene; the video feature set is converted into a video feature vector that matches the target model, wherein the target model is trained based on video feature vector samples and corresponding quantization parameter offset samples; the target model is used to predict the video feature vector to obtain the quantization parameter offset of the current video frame; the quantization parameters of the current video frame are adjusted using the quantization parameter offset; and the current video frame is encoded using the adjusted quantization parameters to obtain the encoding result. In other words, this application embodiment extracts the video feature set of the current video frame, converts the extracted video feature set into a video feature vector that matches the target model, accurately predicts the quantization parameter offset based on the video feature vector, and further adjusts the quantization parameter according to the actual situation based on the quantization parameter offset. The adjusted quantization parameter is then used to encode the current video frame to obtain the encoding result. Since the quantization parameter offset is predicted based on the target model to adjust the quantization parameter, it avoids determining the quantization parameter that meets the conditions through multiple traversals and tests, thereby achieving the technical effect of improving the efficiency of video encoding and solving the technical problem of low efficiency in video encoding.

[0064] The method described in this embodiment will be further described below.

[0065] As an optional implementation, step S206, converting the video feature set into a video feature vector that matches the target model, includes: extracting at least one video feature from the video feature set that corresponds to at least the frame type of the current video frame; and converting the extracted video feature into a video feature vector that matches the target model.

[0066] In this embodiment, the frame type can include I-frames and non-I-frames. An I-frame indicates that the content is obtained entirely through intra-frame prediction, and it can be the first frame of a video sequence. A non-I-frame indicates that the content can be obtained through intra-frame or inter-frame prediction. Non-I-frames can include P-frames and B-frames; P-frames support unidirectional prediction, while B-frames support bidirectional prediction. The frame number of the video sequence can be represented by p, so the frame number of an I-frame is p = 0.

[0067] Optionally, if the current video frame is an I-frame, then video features such as the mean intra-frame prediction cost, the mean inter-frame content difference, and the baseline QP value used for encoding can be extracted from the video feature set. If the current video frame is a non-I-frame, then video features such as the mean intra-frame prediction cost, the mean inter-frame content difference, the QP bias value of the current video frame output by the encoder's built-in bitrate allocation strategy, and the baseline QP value used for encoding can be extracted from the video feature set. After extracting the video features corresponding to different frame types, the extracted video features can be integrated to obtain a video feature vector.

[0068] It should be noted that the bitrate allocation strategies mentioned above can include at least Constant Bit Rate (CBR), Variable Bit Rate (VBR), and Constant Rate Factor (CRF). Constant Bit Rate adjusts the QP value of each frame or CTU based on the target bitrate, ensuring the overall bitrate of the video sequence is close to the target bitrate. This strategy uses a relatively stable bitrate, but the image quality fluctuates significantly, resulting in lower overall image quality. Variable Bit Rate adjusts the QP value of each frame or CTU based on the target bitrate and the bitrate cap, ensuring the average bitrate of the video sequence is close to the target bitrate. It can also adjust the bitrate used in each frame according to the specific content, ensuring the bitrate does not exceed the cap. This strategy allows for flexible adjustment of the video bitrate and reduces image quality fluctuations, but it requires pre-encoding, resulting in higher computational costs and longer latency. The fixed ratio coefficient determines the QP value of each frame or CTU based on a preset ratio coefficient (which affects the image quality level after video compression) without limiting the specific bit rate. This strategy can directly control the image quality level, but cannot accurately control the bit rate. In complex scenarios, the bit rate is high, making it difficult to transmit stably.

[0069] This embodiment analyzes the frame type of the current video frame and the content features corresponding to the frame type to predict more reasonable quantization parameters (QP value), thereby achieving more efficient bitrate control while ensuring video quality.

[0070] As an optional implementation, the video feature set includes an intra-frame feature set and an inter-frame feature set. The intra-frame features in the intra-frame feature set are used to describe the characteristics of the image content of the current video frame in the video scene, and the inter-frame features in the inter-frame feature set are used to describe the characteristics of the image content between the current video frame and the corresponding reference video frame in the video scene. From the video feature set, at least one video feature corresponding to the frame type of the current video frame is extracted, including: in response to the frame type being I-frame, extracting the mean intra-frame prediction cost of the current video frame from the intra-frame feature set, and / or, extracting the mean inter-frame content difference between the current video frame and the reference video frame from the inter-frame feature set; in response to the frame type being non-I-frame, extracting the mean intra-frame prediction cost corresponding to the temporal level of the current video frame from the intra-frame feature set, and / or, extracting the mean inter-frame content difference corresponding to the temporal level from the intra-frame feature set.

[0071] In this embodiment, the reference video frame can be any other video frame that directly references the current frame. The mean intra-frame prediction cost can be the average rate-distortion cost required to predict and encode pixels within the current video frame. The mean inter-frame content difference can be the average pixel content difference between the current video frame and the reference video frame. For I-frame types, the mean intra-frame prediction cost can be... The mean difference in content between frames can be represented by the following formula:

[0072] in, It can be used to represent the average content difference between frames, N can be used to represent the number of reference video frames, that is, the number of other video frames that directly reference the current I-frame, n can be used to represent the frame number of the reference video frame, p n It can be used to represent the distance between a reference video frame and the current video frame, while MAD can be used to represent the mean absolute difference between the reference video frame and the current video frame.

[0073] Optionally, for non-I-frame types, the temporal level of the p-th frame of the video sequence can be set to TID = t, and the mean intra-frame prediction cost can be used. The mean difference in content between frames can be represented by the following formula:

[0074] in, It can be used to represent the mean difference between frame contents, N t It can be used to represent the number of reference video frames, that is, the number of other video frames that directly reference the current video frame, and is related to the temporal level t.

[0075] It should be noted that, for I-frame types, the video features extracted from the video feature set may also include the baseline QP value q used for encoding. I For non-I-frame types, the video features extracted from the video feature set can also include the QP bias value Δq of the current video frame output by the encoder's built-in bitrate allocation strategy. t (p), and the baseline QP value q used for encoding. t (p).

[0076] This embodiment can significantly improve the efficiency and quality of video coding by extracting intra-frame / inter-frame features from the video feature set, achieving adaptive bitrate control, optimizing resource allocation, and improving the stability of the coding process.

[0077] As an optional implementation, the extracted video features are converted into a video feature vector that matches the target model, including: integrating the extracted video features to obtain a video feature vector that matches the target model.

[0078] In this embodiment, after extracting multiple video features corresponding to I-frame types from the video feature set and multiple video features corresponding to non-I-frame types from the video feature set, the multiple video features corresponding to I-frame types and the multiple video features corresponding to non-I-frame types can be integrated to obtain a video feature vector that matches the target model.

[0079] Optionally, for I-frame types, the extracted average intra-frame prediction cost can be... Mean content difference between frames The baseline QP value q used for encoding I After integration processing, the following results were obtained: For non-I-frame types, the average of the extracted intra-frame prediction costs can be used. Mean content difference between frames The QP bias value Δq of the current video frame output by the encoder's built-in bitrate allocation strategy. t (p) The baseline QP value q used for encoding t (p) is integrated to obtain Furthermore, the integrated vectors can be merged to obtain video feature vectors that match the target model:

[0080] This embodiment integrates the extracted video features to obtain a video feature vector. Since the integrated video feature vector contains key information about the video frames, such as the mean intra-frame prediction cost, the mean inter-frame content difference, and the baseline value of the quantization parameters, it can effectively predict the quantization parameter bias value during the video encoding process by matching it with the target model. This provides more accurate guidance for bitrate control, thereby optimizing encoding efficiency and video quality.

[0081] As an optional implementation, the extracted video features are integrated to obtain a video feature vector that matches the target model. This includes: integrating the extracted video features to obtain an initial video feature vector; and preprocessing the initial video feature vector to obtain a video feature vector with the same dimensions as the quantization parameters and that matches the target model.

[0082] In this embodiment, the initial video feature vector can be represented by f. The video feature vector can be a feature vector obtained after preprocessing the initial video feature vector, and can be represented by f. * This can be represented by integrating multiple video features corresponding to I-frame types with multiple video features corresponding to non-I-frame types to obtain an initial video feature vector f.

[0083] Optionally, the initial video feature vector can be preprocessed by taking the logarithm of the video features with a larger dynamic range among the multiple video features corresponding to I-frame types, and taking the logarithm of the video features with a larger dynamic range among the multiple video features corresponding to non-I-frame types, to ensure that the dimensions of the video features after taking the logarithm are the same as QP.

[0084] Furthermore, after taking the logarithm of the video features with larger dynamic range among the multiple video features corresponding to I-frame types and the multiple video features corresponding to non-I-frame types, a scalar 1 can be added to the end of the multiple video features corresponding to I-frame types and to the end of the multiple video features corresponding to non-I-frame types to obtain the video feature vector f. * This is to obtain the offset of the linear model in subsequent fitting. The initial video feature vector f is preprocessed to obtain the video feature vector f. * It can be expressed by the following formula:

[0085] This embodiment, through preprocessing operations such as taking logarithms, normalization, or standardization, can ensure that video features and quantization parameters have the same dimensions, making the model more effective in processing and learning these features. This helps the model predict QP values ​​more accurately and avoids prediction bias caused by differences in feature scale.

[0086] As an optional implementation, step S208, predicting the video feature vector using the target model to obtain the quantization parameter offset of the current video frame, includes: predicting the video feature vector using the target model and the frame type of the current video frame to obtain the quantization parameter offset.

[0087] In this embodiment, when the target model is used to predict the video feature vector, the resulting quantization parameter offset is a predicted value, which has an error compared to the actual value. Therefore, to improve the prediction accuracy as much as possible, the target model and the frame type of the current video frame can be used to predict the video feature vector to obtain the quantization parameter offset.

[0088] Optionally, based on the frame type of the current video frame, a weight vector matching the frame type can be determined. Further, from the weight vectors, a set of weight vectors that minimize the sum of squared prediction errors can be determined. Using these weight vectors, the video feature vectors are predicted to obtain the quantization parameter offset, thereby improving prediction accuracy.

[0089] As an optional implementation, the video feature vector is predicted using the target model and the frame type of the current video frame to obtain the quantization parameter offset, including: determining a weight vector matching the frame type in the target model, wherein the weight vector includes at least the weight coefficients of the video feature vector; and adjusting the video feature vector using the weight vector to obtain the quantization parameter offset.

[0090] In this embodiment, a weight vector matching the frame type can be determined in the target model, and this weight vector can be represented by the following formula:

[0091] Where w can be used to represent the weight vector, w I,1 The weighting coefficient w can be used to represent the mean of the intra-prediction cost corresponding to the I-frame type. I,2 w can be used as a weighting coefficient to represent the mean of inter-frame content differences corresponding to the I-frame type. I,3 Weighting coefficients can be used to represent the baseline QP value used in encoding corresponding to the I-frame type. t,1 w can be used to represent the weighting coefficients of the mean intra-prediction cost corresponding to non-I-frame types. t,2 w can be used to represent the weighting coefficient of the mean inter-frame content difference corresponding to non-I-frame types. t,3 The weighting coefficient w can be used to represent the QP bias value of the current video frame corresponding to a non-I-frame type. t,4 This can be used to represent the weighting coefficients of the baseline QP value used for encoding corresponding to non-I-frame types. I and b tThese can be used to represent the offset of the fitted model.

[0092] Optionally, after determining the weight vector matching the frame type in the target model, a set of weight vectors that minimizes the sum of squared prediction errors can be determined to further improve the accuracy of predicting video feature vectors. Furthermore, the video feature vectors can be adjusted using the weight vectors that minimize the sum of squared prediction errors to obtain the quantization parameter offset.

[0093] This embodiment, by determining the weight vector, enables the model to adjust the QP value based on different dimensions in the video feature vector (such as the mean intra-frame prediction cost, the mean inter-frame content difference, etc.), thereby achieving fine-grained control over the coding parameters. Feature-based adjustments can better match the characteristics of the video content and avoid over-quantization or under-quantization.

[0094] As an optional implementation, the current video frame includes multiple coding tree units. Step S210 involves adjusting the quantization parameters of the current video frame using quantization parameter offset, including: determining the image content of the coding tree unit; and adjusting the quantization parameters of the coding tree unit using the quantization parameter offset and the image content.

[0095] In this embodiment, the image content of the coding tree unit (CTU) can be determined. For example, the image content can be smooth image content, complex image content, image content with rapid motion, image content with gentle motion, etc. This is only an example and does not impose specific limitations on the type of image content. Furthermore, using the determined quantization parameter offset and image content, the quantization parameters of the coding tree unit can be adjusted.

[0096] For example, after determining the CTU's image content, if the image content changes in both the spatial and temporal domains, it indicates that the image content is not smooth enough, and the CTU's quantization parameters can be adjusted using the quantization parameter offset. As another example, if the image content is not too complex, the CTU's quantization parameters can be adjusted using the quantization parameter offset. Optionally, adding the quantization parameter to the quantization parameter offset can achieve the purpose of adjusting the quantization parameter.

[0097] This embodiment automatically adjusts the quantization parameters based on the dynamism and complexity of the video frame, avoiding the use of excessively high quantization parameters in areas where the frame changes little or is too simple, which would result in wasted bitrate. At the same time, it uses more appropriate quantization parameters in areas where the frame is complex or changes greatly, in order to reduce information loss.

[0098] As an optional implementation, adjusting the quantization parameters of the coding tree unit using the quantization parameter offset and the image content includes: adjusting the quantization parameters of the coding tree unit using the quantization parameter offset when the quantization parameter offset is greater than the offset threshold and the complexity of the image content in the spatial and temporal domains is greater than the complexity threshold, wherein the complexity is used to represent the complexity of the image content; and adjusting the quantization parameters of the coding tree unit using the quantization parameter offset when the quantization parameter offset is not greater than the offset threshold and the complexity of the image content is less than the complexity threshold, wherein the complexity is used to represent the complexity of the image content.

[0099] In this embodiment, the offset threshold can be a pre-set critical value for measuring QP offset based on actual conditions. For example, the offset threshold can be 0; this is merely an example and no specific limitation is made on the value of the offset threshold. Complexity can be used to represent the complexity of the image content, and the complexity threshold can be a pre-set critical value for measuring complexity based on actual conditions. Specifically, the complexity of the image content can be determined using the average absolute difference d0(p) between the current CTU and the corresponding positions in neighboring frames, and the variance v0(p) of the current CTU pixel value.

[0100] For example, let's set the offset threshold to 0. For the current video frame with a temporal layer TID of 0, if the quantization parameter offset is greater than the offset threshold, i.e. Then it is necessary to satisfy the requirement that the average absolute difference d0(p) between the current CTU and the corresponding position in the neighboring frame is greater than τ. M+ And the current CTU pixel value variance v0(p) > τ V+ In other words, the QP value of the CTU is adjusted only when the image content varies in both the spatial and temporal domains and is not overly smooth. Among these, τ M+ It can be used to represent the decision threshold where the quantization parameter offset is greater than the average absolute value difference of the offset threshold, τ V+ It can be used to represent the decision threshold for the variance of the current CTU pixel value when the quantization parameter offset is greater than the offset threshold.

[0101] For another example, if the quantization parameter offset is not greater than the offset threshold, i.e. or Then it is necessary to satisfy the requirement that the average absolute difference d0(p) between the current CTU and the corresponding position in the neighboring frame is less than τ. M- And the current CTU pixel value variance v0(p) > τ V- In other words, the QP value of the CTU is adjusted only when the image content is not too complex, to avoid excessive increase in bitrate. Among these, τ... M- It can be used to represent the decision threshold where the quantization parameter offset is no greater than the average absolute value difference of the offset threshold, τ V-It can be used as a decision threshold to indicate the variance of the current CTU pixel value when the quantization parameter offset is not greater than the offset threshold.

[0102] It should be noted that the above describes the case where the CTU decision strategy is enabled, and shows a case where TID is 0. For frames with other TIDs, a similar strategy can be adopted, or a portion of the training video sequences can be used as a validation set. The above strategy can be flexibly adjusted based on the coding performance of the sequences to ensure that most videos have the same objective quality in terms of bitrate change rate (i.e., BD-rate) gain. BD-rate gain can be used to measure coding efficiency. A negative value indicates a decrease in bitrate, and a positive value indicates an increase in bitrate.

[0103] This embodiment allows for finer bitrate control of the encoder at the CTU level, ensuring more reasonable bitrate allocation. When video content varies in spatial and temporal dimensions, increasing the QP value can reduce the bitrate, while for content that is not too smooth or complex, the QP value is maintained or increased to preserve image quality and reduce information redundancy.

[0104] As an optional implementation, the current video frame includes multiple coding tree units. The quantization parameters of the current video frame are adjusted using quantization parameter offsets, including: adjusting the quantization parameters of each coding tree unit using quantization parameter offsets.

[0105] It should be noted that this embodiment compares the BD-rate gains with and without enabling the CTU decision strategy. If enabling the CTU decision strategy results in a greater BD-rate gain, then enabling it is recommended. As mentioned above, enabling the CTU decision strategy can improve the average BD-rate gain and reduce the BD-rate loss caused by incomplete predictions from the fitted model on some sequences.

[0106] In this embodiment, if the CTU decision strategy is not enabled, the frame-level QP value can be adjusted. That is, the QP value of each CTU in the current video frame is increased by the quantization parameter offset. in, A regular expression indicates that the actual QP value increases. A negative value indicates that the actual QP value has decreased.

[0107] This embodiment avoids complex judgments and processing at the CTU level by adjusting the frame-level QP value, reducing computational complexity and making the encoding process simpler and faster.

[0108] As an optional implementation, the method further includes: maintaining the quantization parameters before adjustment in response to the adjusted quantization parameters being greater than a first quantization parameter threshold or less than a second quantization parameter threshold; and encoding the current video frame using the quantization parameters before adjustment to obtain an encoding result.

[0109] In this embodiment, the first quantization parameter threshold can be a pre-set lower limit threshold for measuring QP based on actual conditions. The second quantization parameter threshold can be a pre-set upper limit threshold for measuring QP based on actual conditions. When the adjusted QP value is greater than the first quantization parameter threshold or less than the second quantization parameter threshold, it indicates that the adjusted QP value is too large or too small, and the quantization parameter does not need to be adjusted. Furthermore, the quantization parameter can be directly used to encode the current video frame to obtain the encoding result.

[0110] For example, if the current CTU-adjusted QP value is too large or too small, for instance, Then, the adjustment of the QP value is cancelled, and the QP value is kept unchanged to avoid excessively high prediction risk from the fitted model. Wherein, Δq t (p) can be used to represent the QP bias value of the current video frame output by the encoder's built-in bitrate allocation strategy for non-I-frame types, τ Q It can be used to represent the decision threshold.

[0111] This embodiment prevents the model's predicted QP offset value from being too large or too small by setting a decision threshold, thereby avoiding over-quantization or under-quantization problems that may occur during video encoding and ensuring stable video quality.

[0112] In this embodiment, the current video frame to be encoded is obtained from the video sequence; based on the current video frame, a video feature set associated with the video scene in which the video sequence is located is determined, wherein the video features in the video feature set are used to describe the picture characteristics of the current video frame in the video scene; the video feature set is converted into a video feature vector that matches a target model, wherein the target model is trained based on video feature vector samples and corresponding quantization parameter offset samples; the target model is used to predict the video feature vector to obtain the quantization parameter offset of the current video frame; the quantization parameters of the current video frame are adjusted using the quantization parameter offset; and the current video frame is encoded using the adjusted quantization parameters to obtain the encoding result. In other words, this application embodiment extracts the video feature set of the current video frame, converts the extracted video feature set into a video feature vector that matches the target model, accurately predicts the quantization parameter offset based on the video feature vector, and further adjusts the quantization parameter according to the actual situation based on the quantization parameter offset. The adjusted quantization parameter is then used to encode the current video frame to obtain the encoding result. Since the quantization parameter offset is predicted based on the target model to adjust the quantization parameter, it avoids determining the quantization parameter that meets the conditions through multiple traversals and tests, thereby achieving the technical effect of improving the efficiency of video encoding and solving the technical problem of low efficiency in video encoding.

[0113] This application also provides a method for determining a model. Figure 3 is a flowchart of a method for determining a model according to an embodiment of this application. As shown in Figure 3, the method may include the following steps:

[0114] Step S302: Obtain video feature vector samples and obtain quantization parameter offset samples corresponding to the video feature vector samples.

[0115] In the technical solution provided in step S302 of this application, video feature vector samples can be obtained, and quantization parameter offset samples corresponding to the video feature vector samples can be obtained. The video feature vector samples are derived from video sequence samples.

[0116] Step S304: The target model is trained using video feature vector samples and quantization parameter offset samples.

[0117] In the technical solution provided in step S304 of this application, after obtaining video feature vector samples from video sequence samples and obtaining quantization parameter offset samples corresponding to the video feature vector samples, a target model can be trained using the video feature vector samples and quantization parameter offset samples. The target model can be used to predict video feature vectors that match the target model to obtain quantization parameter offsets; the target model can also be called a fitting model.

[0118] In this embodiment, the video feature vector is obtained by transforming a video feature set associated with the video scene in which the video sequence is located. The video features in the video feature set can be used to describe the image characteristics of the current video frame to be encoded in the video sequence within the video scene. The video feature set is determined based on the current video frame. The quantization parameter offset can be used to adjust the quantization parameters of the current video frame. The adjusted quantization parameters can be used to encode the current video frame to obtain the encoding result.

[0119] Optionally, the current video frame to be encoded can be obtained from the video sequence. The obtained video sequence can be input into the encoder, which performs steps such as reading, parsing, buffering, and encoding the video sequence. After the encoder parses the video sequence, the parsed video frames can be buffered into the buffer, and further, video frames can be obtained from the buffer as the current video frame to be encoded.

[0120] Optionally, after obtaining the current video frame to be encoded from the video sequence, the encoder's built-in pre-analysis module can be invoked to analyze the video feature set related to the video scene before actual encoding. This video feature set may include intra-frame feature sets and inter-frame feature sets. The pre-analysis module may include an intra-frame / inter-frame prediction module, which can extract the aforementioned intra-frame and inter-frame feature sets.

[0121] Optionally, after determining the video feature set associated with the video scene where the video sequence is located based on the current video frame, the video feature set can be transformed into a video feature vector that matches the fitted model. Optionally, multiple video features in the video feature set can be integrated to obtain a video feature vector.

[0122] Optionally, based on video feature vector samples and corresponding quantization parameter offset samples, an initial fitting model can be trained to obtain a fitted model. After the fitted model is trained, it can be used to determine the QP bias value in actual encoding. The obtained video feature vectors can be input into the fitted model, and the fitted model can be used to predict the video feature vectors to obtain the QP bias value, which can then be used to adjust the QP of the current video frame.

[0123] Optionally, after obtaining the QP offset value, the QP of the current video frame can be adjusted using the QP offset value. For example, the QP of the current video frame can be added to the QP offset value to achieve the purpose of adjusting the QP of the current video frame. The QP offset value... It has positive and negative values, depending on the QP bias value. When the value is positive, it indicates that the QP of the current video frame has increased; when the QP offset value is negative... A negative value indicates a decrease in the QP of the current video frame. After adjusting the QP of the current video frame to obtain the adjusted QP, the adjusted QP can be used to encode the current video frame to obtain the encoded result.

[0124] Through steps S302 to S304 of this application, video feature vector samples are obtained, and quantization parameter offset samples corresponding to the video feature vector samples are also obtained. Using the video feature vector samples and quantization parameter offset samples, a target model is trained. The target model is used to predict the video feature vectors that match the target model to obtain the quantization parameter offset. The video feature vectors are obtained by converting the video feature set associated with the video scene in which the video sequence is located. The video features in the video feature set are used to describe the picture characteristics of the current video frame to be encoded in the video scene. The video feature set is determined based on the current video frame. The quantization parameter offset is used to adjust the quantization parameters of the current video frame. The adjusted quantization parameters are used to encode the current video frame to obtain the encoding result. In other words, this application embodiment extracts the video feature set of the current video frame, converts the extracted video feature set into a video feature vector that matches the target model, accurately predicts the quantization parameter offset based on the video feature vector, and further adjusts the quantization parameter according to the actual situation based on the quantization parameter offset. The adjusted quantization parameter is then used to encode the current video frame to obtain the encoding result. Since the quantization parameter offset is predicted based on the target model to adjust the quantization parameter, it avoids determining the quantization parameter that meets the conditions through multiple traversals and tests, thereby achieving the technical effect of improving the efficiency of video encoding and solving the technical problem of low efficiency in video encoding.

[0125] The method described in this embodiment will be further described below.

[0126] As an optional implementation, step S302, obtaining video feature vector samples, includes: obtaining multiple lossless video sequence samples, wherein different lossless video sequence samples come from different video scenes; and obtaining video feature vector samples from the multiple lossless video sequence samples.

[0127] In this embodiment, multiple lossless video sequence samples can be obtained. These different lossless video sequence samples come from different video scenes. A video scene can be any scene contained within the video sequence, such as a moving scene, a still scene, a scene with smooth textures, or a complex scene. This is merely an example and does not impose specific limitations on the type of video scene. After obtaining multiple lossless video sequence samples, video feature vector samples can be extracted from them.

[0128] As an optional implementation, step S304, obtaining video feature vector samples from multiple lossless video sequence samples, includes: determining a set of video feature samples associated with the video scene where the lossless video sequence samples are located based on the video frame samples in the lossless video sequence samples, wherein the video feature samples in the video feature sample set are used to describe the picture characteristics of the video frame samples in the corresponding video scene; and converting the set of video feature samples into video feature vector samples.

[0129] In this embodiment, the video feature sample set may include multiple video feature samples related to the video scene extracted from video frame samples. The video feature samples in the video feature sample set can be used to describe the image characteristics of the video frame samples in the video scene. The video feature samples may include intra-frame feature samples or inter-frame feature samples. For example, intra-frame feature samples may be the mean intra-frame prediction cost samples of video frame samples, and inter-frame features may be the mean inter-frame content difference samples between video frame samples and reference video frame samples. This is only an example and does not impose specific limitations on the type of video feature samples.

[0130] Optionally, after acquiring multiple lossless video sequence samples, video frame samples can be extracted from them. The acquired lossless video sequence samples can be input into an encoder, which performs steps such as reading, parsing, buffering, and encoding. After the encoder parses the lossless video sequence samples, the parsed video frame samples can be buffered into a buffer, from which video frame samples can be further extracted.

[0131] Optionally, after extracting video frame samples from multiple lossless video sequence samples, the encoder's built-in pre-analysis module can be invoked to analyze a set of video feature samples related to the video scene before actual encoding. This set of video feature samples can include intra-frame feature samples and inter-frame feature samples. The pre-analysis module can include an intra-frame / inter-frame prediction module, which can extract the aforementioned intra-frame and inter-frame feature sample sets. Integrating multiple video feature samples from the video feature sample set yields video feature vector samples.

[0132] As an optional implementation, a video feature sample set associated with the video scene where the lossless video sequence sample is located is determined based on video frame samples in the lossless video sequence sample, including: downsampling the video frame samples, wherein the resolution of the downsampled video frame samples is lower than a resolution threshold; and determining the video feature sample set based on the downsampled video frame samples.

[0133] In this embodiment, the resolution threshold can be a pre-set resolution cutoff value used to measure resolution based on actual conditions. After extracting video frame samples from the video sequence samples, the video frame samples can be downsampled. Further, based on the downsampled video frame samples, a video feature sample set can be determined.

[0134] Optionally, to save computation, the above steps use downsampled low-resolution frame samples instead of original resolution frame samples. The intra / inter-frame prediction module has been simplified compared to the actual encoding, and it needs to approximate the prediction cost, content motion information, and texture complexity of video feature samples for each frame and each local region. In other words, the intra / inter-frame prediction module is a fast preprocessing process before actual encoding, and it is an approximate estimate of the actual encoding. The intra-frame feature sample set and inter-frame feature sample set extracted above are also approximate estimated features in the pre-analysis.

[0135] As an optional implementation, converting a video feature sample set into a video feature vector sample includes: extracting at least one video feature sample from the video feature sample set that corresponds to at least one frame type of the video frame sample; integrating the extracted video feature samples to obtain an initial video feature vector sample; and preprocessing the initial video feature vector sample to obtain a video feature vector sample.

[0136] In this embodiment, if the video frame sample is of type I-frame, then video feature samples such as the mean intra-frame prediction cost, the mean inter-frame content difference, and the baseline QP value used for encoding, corresponding to the I-frame type, can be extracted from the video feature sample set. If the video frame sample is of type non-I-frame, then video feature samples such as the mean intra-frame prediction cost, the mean inter-frame content difference, the QP bias value of the current video frame output by the encoder's built-in bitrate allocation strategy, and the baseline QP value used for encoding, corresponding to the non-I-frame type, can be extracted from the video feature sample set. After extracting the video feature samples corresponding to different frame types, the extracted video feature samples can be integrated to obtain an initial video feature vector sample.

[0137] Furthermore, the initial video feature vector samples are preprocessed by taking the logarithm of the video feature samples with larger dynamic ranges among the multiple video feature samples corresponding to I-frame types, and also taking the logarithm of the video feature samples with larger dynamic ranges among the multiple video feature samples corresponding to non-I-frame types. Additionally, a scalar 1 can be appended to the end of the multiple video feature samples corresponding to I-frame types, and to the end of the multiple video feature samples corresponding to non-I-frame types, to obtain the video feature vector samples.

[0138] As an optional implementation, the video feature sample set includes an intra-frame feature sample set and an inter-frame feature sample set. The intra-frame feature samples in the intra-frame feature sample set are used to describe the characteristics of the image content of the video frame sample in the corresponding video scene. The inter-frame feature samples in the inter-frame feature sample set are used to describe the characteristics of the image content between the video frame sample and the corresponding reference video frame sample in the corresponding video scene. From the video feature sample set, at least one video feature sample corresponding to the frame type of the video frame sample is extracted, including: in response to the frame type being I-frame, extracting the mean intra-frame prediction cost of the video frame sample from the intra-frame feature sample set, and / or, extracting the mean inter-frame content difference between the video frame sample and the reference video frame sample from the inter-frame feature sample set; in response to the frame type being non-I-frame, extracting the mean intra-frame prediction cost corresponding to the temporal level of the video frame sample from the intra-frame feature sample set, and / or, extracting the mean inter-frame content difference corresponding to the temporal level from the intra-frame feature sample set.

[0139] In this embodiment, the reference video frame sample can be a video frame sample from another directly referenced video frame sample. For I-frame type, the mean intra-frame prediction cost of the video frame sample can be extracted from the intra-frame feature sample set. Mean content difference between frames The baseline QP value q used for encoding I .

[0140] Optionally, for non-I-frame types, the temporal level of the p-th frame of the video sequence sample can be set to TID = t, and the mean intra-frame prediction cost of the video frame sample can be extracted from the intra-frame feature sample set. Mean content difference between frames The QP bias value Δq of the current video frame output by the encoder's built-in bitrate allocation strategy. t (p), and the baseline QP value q used for encoding. t (p).

[0141] As an optional implementation, step S304, obtaining the quantization parameter offset sample corresponding to the video feature vector sample, includes: encoding the lossless video sequence sample using the bitrate control strategies corresponding to multiple bitrates respectively, to obtain multiple encoding result samples, wherein the multiple encoding result samples correspond to multiple bitrates; and determining the quantization parameter offset sample from the multiple initial quantization parameter offset samples corresponding to the multiple encoding result samples.

[0142] In this embodiment, each lossless video sequence sample can be encoded using a corresponding bitrate control strategy at various bitrate points to obtain multiple encoded result samples. The bitrate control strategy can be a fixed bitrate, a variable bitrate, or a fixed scaling factor, etc. Further utilizing a rate-distortion (RD) search method, quantization parameter offset samples can be determined from multiple initial quantization parameter offset samples corresponding to the multiple encoded result samples. The rate-distortion search method can include traversal and estimation methods, and rate-distortion can be used to comprehensively measure the relationship between bitrate and distortion degree.

[0143] Optionally, when using a traversal approach to determine the quantization parameter offset samples from multiple initial quantization parameter offset samples corresponding to multiple encoded result samples, it is first assumed that the BD-rate satisfying the condition is 0, and that the QP remains unchanged for each frame. Each bitrate point in each frame is checked according to the encoding order, and multiple QP offset values ​​are checked at each bitrate point and encoded separately. If the current BD-rate is found to be better than the recorded BD-rate, the BD-rate is updated, and the QP offsets used in each frame and bitrate point are recorded, until all video frames have been checked. This method can accurately obtain the QP offset values ​​of each frame, but it requires multiple traversals and has a high computational load.

[0144] Optionally, when using an estimation method to determine the quantization parameter offset samples from multiple initial quantization parameter offset samples corresponding to multiple encoded result samples, firstly, for each sequence segment, simultaneously change the QP value of frames with the same TID, and record the bit rate and quality evaluation indicators such as PSNR and SSIM at each code rate point. Further compare the situations formed by each code rate point and each QP offset value, and record the QP offset value when the BD-rate meets the target condition. This method can approximately estimate the QP offset that meets the conditions for various TID situations with less computation.

[0145] As an optional implementation, a target model is trained using video feature vector samples and quantization parameter offset samples, including: converting the video feature vector samples into the input matrix of the initial model and converting the quantization parameter offset samples into the output matrix of the initial model; fitting the input matrix and the output matrix to obtain weight vector samples corresponding to the initial model, wherein the weight vector samples include at least the weight coefficient samples of the video feature vector samples; and establishing the target model using the weight vector samples and the initial model.

[0146] In this embodiment, the initial model can be a fitting model other than a linear model, such as a support vector machine, random forest, or multilayer neural network. First, a training set is constructed. Let M records be extracted from multiple frames of multiple video sequences, where each record (let's say its sequence number is m) includes a preprocessed video feature vector sample. and the corresponding quantization parameter offset sample ΔQ m The following formula can be used to convert video feature vector samples into the input matrix of the initial model:

[0147] Here, X can be used to represent the input matrix of the initial model. Furthermore, the quantization parameter offset samples can be converted into the output matrix of the initial model using the following formula:

[0148] Where y can be used to represent the output matrix of the initial model. Further, through the formula... By fitting the input and output matrices, we can obtain weight vector samples corresponding to the initial model.

[0149] The weight vector sample can include the weight coefficients of the video feature vector sample and the offset of the initial model. I,1 The weighting coefficient w can be used to represent the mean of the intra-prediction cost corresponding to the I-frame type. I,2 w can be used as a weighting coefficient to represent the mean of inter-frame content differences corresponding to the I-frame type. I,3 Weighting coefficients can be used to represent the baseline QP value used in encoding corresponding to the I-frame type. t,1 w can be used to represent the weighting coefficients of the mean intra-prediction cost corresponding to non-I-frame types. t,2 w can be used to represent the weighting coefficient of the mean inter-frame content difference corresponding to non-I-frame types. t,3 The weighting coefficient w can be used to represent the QP bias value of the current video frame corresponding to a non-I-frame type. t,4 This can be used to represent the weighting coefficients of the baseline QP value used for encoding corresponding to non-I-frame types. I and b t These can be used to represent the offset of the initial model, respectively. Further, the formula... Represented in matrix form, we can obtain:

[0150] in, It can be used to represent the quantization parameter offset sample ΔQ m The predicted value, It can be used to represent the predicted value of the output matrix y. Because predictions contain errors, The values ​​for y are usually not equal. To maximize accuracy, a set of weight vector samples that minimizes the sum of squared prediction errors can be determined. The least squares optimization objective is:

[0151] The weight vector sample that minimizes the sum of squared prediction errors can be obtained through matrix operations.

[0152] The target model is built using weight vector samples and the initial model. After the target model is trained, all the parameters required for the target model are stored in the weight vector samples that minimize the sum of squared prediction errors. In this context, it can be used to adjust the QP value of each frame during actual encoding.

[0153] In this embodiment, video feature vector samples are obtained, and quantization parameter offset samples corresponding to the video feature vector samples are also obtained. A target model is trained using the video feature vector samples and the quantization parameter offset samples. The target model is used to predict the video feature vectors that match the target model to obtain the quantization parameter offset. The video feature vectors are obtained by converting a video feature set associated with the video scene in which the video sequence is located. The video features in the video feature set are used to describe the image characteristics of the current video frame to be encoded in the video scene. The video feature set is determined based on the current video frame. The quantization parameter offset is used to adjust the quantization parameters of the current video frame. The adjusted quantization parameters are used to encode the current video frame to obtain the encoding result. In other words, this application embodiment extracts the video feature set of the current video frame, converts the extracted video feature set into a video feature vector that matches the target model, accurately predicts the quantization parameter offset based on the video feature vector, and further adjusts the quantization parameter according to the actual situation based on the quantization parameter offset. The adjusted quantization parameter is then used to encode the current video frame to obtain the encoding result. Since the quantization parameter offset is predicted based on the target model to adjust the quantization parameter, it avoids determining the quantization parameter that meets the conditions through multiple traversals and tests, thereby achieving the technical effect of improving the efficiency of video encoding and solving the technical problem of low efficiency in video encoding.

[0154] This application also provides a video encoding system. Figure 4 is a schematic diagram of a video encoding system according to an embodiment of this application. As shown in Figure 4, the video encoding system 400 may include an encoder 402 and a server 404.

[0155] Encoder 402 is used to obtain the current video frame to be encoded from the video sequence; call the pre-analysis module to determine the video feature set associated with the video scene in the video sequence based on the current video frame, wherein the video features in the video feature set are used to describe the picture characteristics of the current video frame in the video scene; convert the video feature set into a video feature vector that matches the target model, wherein the target model is trained based on the video feature vector samples and the corresponding quantization parameter offset samples; use the target model to predict the video feature vector to obtain the quantization parameter offset of the current video frame; use the quantization parameter offset to adjust the quantization parameters of the current video frame; use the adjusted quantization parameters to encode the current video frame to obtain the encoding result.

[0156] Optionally, after acquiring the video sequence, it can be input into an encoder, which performs steps such as reading, parsing, buffering, and encoding. After the encoder parses the video sequence, the parsed video frames can be buffered into a buffer, and further, video frames can be retrieved from the buffer as the current video frames to be encoded.

[0157] Optionally, after obtaining the current video frame to be encoded from the video sequence, the encoder's built-in pre-analysis module can be invoked to analyze the video feature set related to the video scene before actual encoding. This video feature set may include intra-frame feature sets and inter-frame feature sets. The pre-analysis module may include an intra-frame / inter-frame prediction module, which can extract the aforementioned intra-frame and inter-frame feature sets.

[0158] Optionally, after determining the video feature set associated with the video scene where the video sequence is located based on the current video frame, the video feature set can be transformed into a video feature vector that matches the fitted model. Optionally, multiple video features in the video feature set can be integrated to obtain a video feature vector.

[0159] Optionally, based on video feature vector samples and corresponding quantization parameter offset samples, an initial fitting model can be trained to obtain a fitted model. After the fitted model is trained, it can be used to determine the QP bias value in actual encoding. The obtained video feature vectors can be input into the fitted model, and the fitted model can be used to predict the video feature vectors to obtain the QP bias value, which can then be used to adjust the QP of the current video frame.

[0160] Optionally, after obtaining the QP offset value, the QP of the current video frame can be adjusted using the QP offset value. For example, the QP of the current video frame can be added to the QP offset value to achieve the purpose of adjusting the QP of the current video frame. The QP offset value... It has positive and negative values, depending on the QP bias value. When the value is positive, it indicates that the QP of the current video frame has increased; when the QP offset value is negative... A negative value indicates a decrease in the QP of the current video frame. After adjusting the QP of the current video frame to obtain the adjusted QP, the adjusted QP can be used to encode the current video frame to obtain the encoded result.

[0161] A 404 error message indicates a problem with the server, which is used to retrieve the encoded result.

[0162] In this video encoding system, encoder 402 obtains the current video frame to be encoded from the video sequence; the pre-analysis module is invoked to determine the video feature set associated with the video scene in which the video sequence is located based on the current video frame, wherein the video features in the video feature set are used to describe the picture characteristics of the current video frame in the video scene; the video feature set is converted into a video feature vector that matches the target model, wherein the target model is trained based on video feature vector samples and corresponding quantization parameter offset samples; the target model is used to predict the video feature vector to obtain the quantization parameter offset of the current video frame; the quantization parameters of the current video frame are adjusted using the quantization parameter offset; the current video frame is encoded using the adjusted quantization parameters to obtain the encoding result; and the encoding result is obtained through server 404. In other words, this application embodiment extracts the video feature set of the current video frame, converts the extracted video feature set into a video feature vector that matches the target model, accurately predicts the quantization parameter offset based on the video feature vector, and further adjusts the quantization parameter according to the actual situation based on the quantization parameter offset. The adjusted quantization parameter is then used to encode the current video frame to obtain the encoding result. Since the quantization parameter offset is predicted based on the target model to adjust the quantization parameter, it avoids determining the quantization parameter that meets the conditions through multiple traversals and tests, thereby achieving the technical effect of improving the efficiency of video encoding and solving the technical problem of low efficiency in video encoding.

[0163] The technical solutions of the present disclosure embodiments will be further illustrated below with reference to preferred embodiments.

[0164] Currently, bitrate control is a crucial step in video encoding. It's used to select appropriate quantization parameters for each frame of a video sequence or for each image block within a frame to control the desired bitrate. Mainstream encoders typically support various bitrate control strategies, such as fixed bitrate, variable bitrate, and fixed scaling factor. The specific strategy affects various metrics, including the final bitrate, stability, and image quality, to adapt to different scenario requirements.

[0165] In recent years, multimedia technology and video scenarios have been developing rapidly, with video data volume experiencing explosive growth, posing significant challenges to communication bandwidth and storage resources. In response, multiple generations of video compression coding standards have been proposed to improve coding efficiency and reduce video storage and transmission overhead. Compared to the previous generation standard H.265 / High Efficiency Video Coding (HEVC), the new generation standard H.266 / VVC can reduce the bitrate by 30% to 50% while maintaining the same subjective and objective visual quality. This efficiency improvement mainly comes from the expansion of the rate-distortion search space by various new coding technologies, but it also greatly increases computational complexity, making it difficult to apply in practice and becoming a major bottleneck hindering the improvement of coding efficiency. Therefore, how to improve coding efficiency without significantly increasing complexity has become a key focus for all parties.

[0166] Since the introduction of the new generation H.266 / VVC video coding standard, existing algorithms have been used to optimize bitrate control performance. However, these existing algorithms mainly focus on improving control accuracy rather than directly improving coding efficiency, and most are designed for standard encoders, lacking solutions for fast encoders in real-world scenarios.

[0167] In one example, common bitrate control strategies in open-source encoders (such as x265, VVenC, VTM, etc.) mainly include: fixed bitrate, variable bitrate, fixed scaling factor, fixed scaling factor under bitrate constraints, and constant quantization parameter (CQP). Fixed bitrate, variable bitrate, and fixed scaling factor have already been introduced above and will not be repeated here. Fixed scaling factor under bitrate constraints limits the upper limit of the bitrate based on the fixed scaling factor (CRF). This strategy is mainly used to control the image quality level and ensure that the bitrate is not too high. The constant quantization parameter uses the same QP value. This strategy is simple to implement and provides relatively stable image quality, but it ignores the reference relationship between multiple frames, resulting in large bitrate redundancy and insufficient encoding efficiency, and is rarely used in practice. x265 is an open-source H.265 / HEVC encoder, VVenC is an open-source H.266 / VVC encoder, and VTM is the H.266 / VVC standard reference software, which includes an encoder.

[0168] Building upon the aforementioned strategies, existing algorithms have further improved the rate control strategy in the next-generation H.266 / VVC standard. For example, by comparing and analyzing the impact of zero-residual blocks on rate-distortion parameters and inter-frame quality dependence, a method is proposed to update parameters for both zero-residual blocks and regular blocks separately, and select a rate allocation strategy that meets the conditions, significantly improving the coding efficiency of VTM. Furthermore, existing algorithms also propose a machine learning-based intra-frame mode rate allocation optimization algorithm. Based on the directional features of CTU textures, this algorithm uses multiple machine learning models to predict QP values ​​that meet the conditions, improving rate control accuracy and overall rate-distortion performance.

[0169] Furthermore, existing algorithms predict rate allocation coefficients that meet certain conditions based on the content similarity between keyframes and other reference frames, thereby improving the rate-distortion performance of inter-frame coding. In addition, some existing algorithms utilize multiple methods such as cubic rate-distortion models, Nash equalization, and joint decision trees to model the rate control process in VVC, in order to improve control accuracy and coding efficiency.

[0170] The aforementioned algorithms offer various optimization approaches and methods for bitrate control in VVC. However, existing algorithms are typically designed for standard encoders (VTM), and research on widely used fast encoders such as VVenC is lacking. Furthermore, some algorithms are computationally intensive, making them unsuitable for fast or even real-time encoders. Most existing algorithms are designed for fixed-bitrate CBR strategies, aiming to improve bitrate control accuracy rather than directly optimizing encoding efficiency. However, many real-world video scenarios do not employ CBR and require improved encoding efficiency, making it difficult to directly apply existing algorithms.

[0171] To address the aforementioned issues, this application proposes an adaptive video scene bitrate control optimization algorithm. This algorithm predicts quantization parameters that meet certain conditions based on intra-frame and inter-frame features of the video content, and then uses these quantization parameters in the bitrate control process to improve encoding efficiency. The encoder used in this embodiment is an S266 encoder, a video encoder compliant with the H.266 / VVC standard, which is already in use in short video and live streaming scenarios. In optimizing the S266 encoder, existing bitrate allocation schemes struggle to adapt to complex and changing scenarios, resulting in low encoding efficiency. This embodiment, after algorithm optimization, can adaptively adjust the quantization parameters used in each frame according to the video scene, thereby improving the bitrate allocation process and significantly enhancing encoding efficiency.

[0172] This embodiment can be used to optimize the bitrate allocation process of video encoding. It mainly includes two parts: pre-statistically calculating the statistical features of each frame content in multiple video scenes, as well as the QP value that meets the rate-distortion performance conditions, fitting the relationship between them using a linear regression or machine learning model, and saving the model; during actual encoding, reading the above model, predicting a reasonable QP value based on the statistical features in the video scene, and using the QP value for bitrate allocation to improve the overall encoding efficiency.

[0173] Figure 5 is a flowchart of a model training process according to an embodiment of this application. As shown in Figure 5, the model training process may include the following steps:

[0174] Step S501: Select multiple lossless video sequences and a basic bitrate control strategy.

[0175] In the above steps, multiple lossless video sequences can be selected, ensuring that the selected video sequences contain diverse video scenes, such as moving images, still images, smooth-textured images, and complex images, for model training. Further steps involve determining the encoder version and the basic bitrate control strategy used, such as CBR, VBR, and CRF as mentioned earlier. Considering the significant variations in bitrate used in actual encoding, this step can select multiple bitrate points to cover as many scenarios as possible.

[0176] Optionally, the training data can be 20 1080 pixel video sequences, with CRF selected as the basic bitrate control strategy, which includes four bitrate points CRF{18,22,26,30}.

[0177] Step S502: Perform pre-analysis on the video content of the selected video sequence.

[0178] In the above steps, each sequence segment is encoded at various bitrates. At each bitrate, the encoder's built-in pre-analysis module is invoked to analyze intra-frame / inter-frame features related to the video scene before actual encoding. To save computation, this step can use downsampled low-resolution frames instead of the original resolution frames. The intra-frame / inter-frame prediction module is significantly simplified compared to the actual encoding, requiring approximate estimation of the prediction cost, content motion information, and texture complexity for each frame and local region.

[0179] Step S503: Extract the statistical features of each frame.

[0180] In the above steps, each video frame sequence is divided into two main categories: I-frames and non-I-frames. Non-I-frames are further classified according to temporal hierarchy. The video features of each frame (with frame number p) are then recorded based on frame type. For an I-frame (generally the first frame of the video, p = 0), the recorded video features include: the average intra-frame prediction cost for each region of that frame. The baseline QP value q used for encoding I The mean inter-frame content difference between this frame and other frames that directly reference it:

[0181] Where N can be used to represent the number of other frames that directly reference the current I-frame, n can be used to represent the reference frame number, and p n It can be used to represent the distance between a reference frame and the current frame, while MAD can be used to represent the average absolute difference between two frames.

[0182] For non-I-frames (assuming the temporal level of the p-th frame in the video sequence is TID = t), the recorded video features include: the mean intra-frame prediction cost for each region of that frame. The current frame QP offset Δq output by the encoder's built-in rate allocation strategy t (p); The baseline QP value q used for encoding t (p); Mean inter-frame content difference between this frame and other frames that directly reference this frame:

[0183] Where, N t This can be used to represent the number of other frames that directly reference the current frame, and is related to the temporal level t. Furthermore, the above features are integrated into a vector:

[0184] Optionally, an encoding configuration with a Group of Pictures (GOP) length of 64 is selected, and the highest temporal layer TID is 6. That is, non-I-frames can be divided into 7 categories: TID 0, TID 1, ..., TID 6. This embodiment can select other easily computed features according to the specific encoder type and application scenario, and is not limited to the features listed above.

[0185] Step S504: Iterate through the QP offset values ​​of each frame.

[0186] In the above steps, for each sequence segment, the QP bias value ΔQ that satisfies the conditions for each frame is obtained through rate-distortion search. Depending on the actual needs, one can choose to traverse to obtain the QP bias value that satisfies the conditions at the frame level or estimate the QP bias value that satisfies the conditions at the TID level.

[0187] The process involves iterating through the frames to obtain the QP offset values ​​that meet the conditions. First, it assumes the BD-rate meeting the conditions is 0, and the QP remains constant for each frame. Then, it checks each bitrate point in the encoding order. At each bitrate point, it checks for multiple QP offset values ​​and encodes them separately. If the current BD-rate is found to be better than a previously recorded BD-rate meeting the conditions, it updates the recorded BD-rate and records the QP offset used for each bitrate point in the current frame, continuing until all video frames have been checked. This method can accurately obtain the QP offset values ​​that meet the conditions for each frame, but it requires multiple iterations and has a high computational cost.

[0188] Estimating the QP offset value for TID-level conditions: For each sequence segment, simultaneously change the QP value of video frames with the same TID, and record the bitrate and quality evaluation indicators such as PSNR and SSIM at each bitrate point. Further compare the combinations of each bitrate point and each QP offset value, and record the QP offset value of BD-rate under the conditions. This method can approximately estimate the QP offset for each TID condition with relatively little computation.

[0189] Optionally, the range of QP offset values ​​for the above traversal can be {-2, -1, 0, +1, +2}, that is, checking five cases: QP unchanged, QP increased by 1, QP increased by 2, QP decreased by 1, and QP decreased by 2. For I-frames and non-I-frames that satisfy TID=0, traversal is used to obtain the frame-level QP offset; for non-I-frames with TID>=1, the estimated TID-level QP offset that meets the condition is used.

[0190] Step S505: Record the QP bias value that meets the conditions.

[0191] In the above steps, after obtaining the QP bias value ΔQ that satisfies the conditions for each frame through rate-distortion search, the obtained QP bias value that satisfies the conditions can be recorded.

[0192] Step S506: Use the model to fit statistical features and QP bias values ​​that meet the conditions.

[0193] In the above steps, a linear model is used to fit the relationship between feature f and the QP bias value ΔQ that meets the conditions. First, feature f is preprocessed: for I-frames, features with a large dynamic range are... Take the logarithm; for non-I-frames, select features with a larger dynamic range. Take the logarithm to ensure that the dimensions of the logarithmic features are the same as those of QP, and add a scalar of 1 to the end of the features so that the offset of the linear model can be obtained in the subsequent fitting. The preprocessed features can be represented by the following formula:

[0194] Secondly, construct the training set: Suppose that M records are extracted cumulatively from multiple frames of multiple video sequences, where each record (let the sequence number be m) includes preprocessed features. And the corresponding QP bias value ΔQ that satisfies the conditions. m The preprocessed features The QP bias value ΔQ that meets the conditions m The input matrix X and output vector y are combined into a linear model:

[0195] Let the weight vector required for the linear model be:

[0196] The QP bias value that meets the conditions can be obtained through fitting:

[0197] The above can be expressed in matrix form as follows:

[0198] in, It can be used to represent ΔQ m The predicted value, It can be used to represent the predicted value of y, w I,1 ~w I,3 and w t,1 ~w t,4 b represents the weight coefficients for each feature. I and b t This represents the offset of the fitted model. Since predictions typically contain errors, The values ​​for y are generally not equal. To maximize accuracy, we need to find a set of weight vectors that minimizes the sum of squared prediction errors. The least squares optimization objective is:

[0199] The weight vector that satisfies the conditions is obtained by matrix operations:

[0200] In summary, the model training is complete, and all required parameters have been saved to vectors. In this context, it can be used to adjust the QP value of each frame during actual encoding.

[0201] Optionally, this embodiment can select other fitting models besides linear models, such as support vector machines, random forests, multilayer neural networks, etc. However, the selected model should not be too complex, and the number of trainable parameters should be much less than the number of samples to avoid overfitting.

[0202] Figure 6 is a flowchart of a video encoding method according to an embodiment of this application. As shown in Figure 6, the video encoding process may include the following steps:

[0203] Step S601: Have all video frames been read?

[0204] In the above steps, it can be determined whether all video frames of the video sequence have been encoded. If not, proceed to step S602 to continue the encoding process until all video frames have been encoded. If all video frames of the video sequence have been encoded, then the video encoding process ends.

[0205] Step S602: Perform pre-analysis on the content of the current frame.

[0206] In the above steps, the pre-analysis module built into the encoder is called to analyze the intra-frame / inter-frame features related to the video scene before actual encoding. This step is the same as step S502 of model training.

[0207] Step S603: Extract statistical features.

[0208] In the above steps, the content features f of each video frame are recorded according to the frame type and temporal hierarchy. This step is the same as step S503 of model training. The preprocessed features f are then further calculated. * .

[0209] Step S604: Call the model to predict the QP bias value that meets the conditions.

[0210] In the above steps, feature f * The fitted model obtained from the input model training can be used to calculate the QP bias value that satisfies the conditions at the frame level.

[0211] Step S605: Whether to enable CTU-level decision strategy.

[0212] In the above steps, it is determined whether to enable the CTU-level decision policy. If the CTU-level decision policy is enabled, proceed to step S606 to determine whether to adjust the QP of each CTU; if the CTU-level decision policy is not enabled, proceed to step S607 to adjust the frame-level QP value. Optionally, this embodiment can choose to enable the CTU-level decision policy.

[0213] Step S606: Adjust the CTU-level QP value according to the screen content.

[0214] In the above steps, for each CTU in the current frame, the image content is checked to see if it is too smooth, complex, has violent motion, or has slow motion, and based on this, it is determined whether the QP value of the current CTU needs to be increased. Bias value. Wherein, A positive value indicates an increase in the actual QP value. A negative value indicates a decrease in the actual QP value. For example, for a frame with a TID of 0, the following strategy is adopted:

[0215] like The average absolute difference d0(p) between the current CTU and the corresponding position in the neighboring frame needs to be greater than τ. M+ And the current CTU pixel value variance v0(p) > τ V+ Time (i.e., the content changes in both the spatial and temporal domains, but is not overly smooth, where τ) M+ and τ V+ The QP value of this CTU is adjusted only if the threshold value is used for the decision; if... The average absolute difference d0(p) between the current CTU and the corresponding position in the neighboring frame needs to be less than τ. M- And the current CTU pixel value variance v0(p) > τ V- Time (i.e., the content is not too complex, where τ) M- and τ V- The QP value of this CTU is adjusted only when the threshold for judgment is reached, in order to avoid excessive increase in bit rate.

[0216] Optionally, the above decision threshold can be: τ M+ =500, τ V+ =1000, τ M- =2000, τ V- =2000 and τ Q =10.

[0217] Furthermore, if the current CTU-adjusted QP value is too large or too small (for example, Where τ Q If the threshold is set (i.e., the decision threshold), the adjustment is canceled and the QP value remains unchanged to avoid excessive risk in model prediction.

[0218] For frames with other TIDs, a similar strategy can be adopted, or a portion of the training video sequences can be used as a validation set. The above strategy can be flexibly adjusted based on the coding performance of this portion of the sequences to ensure that the algorithm has BD-rate gains on most videos.

[0219] Step S607: Directly adjust the frame-level QP value.

[0220] In the above steps, the QP value of the current frame CTU is increased. bias value, A positive value indicates an increase in the actual QP value. A negative value indicates a decrease in the actual QP value.

[0221] Step S608: Encode the current frame to obtain the encoding result.

[0222] In the above steps, the current frame can be encoded according to the conventional process.

[0223] This embodiment accurately predicts the frame-level QP bias value that meets the conditions based on intra-frame / inter-frame features in specific video scenes during bitrate control, thereby improving overall coding efficiency. Furthermore, it selects a portion of the frame using a CTU-level decision strategy to adjust the QP value, reducing the risk of performance loss. In addition, this embodiment has strong versatility; the principle of feature extraction and model fitting can be used with different types and versions of encoders, and it is not limited to whether other bitrate control optimization algorithms already exist in the encoder. The parameters used in the specific steps can also be flexibly adjusted.

[0224] In one example, existing bitrate control algorithms are mostly used to improve control accuracy, that is, to ensure that the bitrate remains stable throughout the entire video sequence, but they do not meet the needs of actual video scenarios, where encoding efficiency is usually the primary concern.

[0225] However, this application proposes to predict reasonable QP bias values ​​for each frame based on intra / inter-frame features in specific video scenes to improve overall coding efficiency. Furthermore, a CTU-level decision strategy is used to determine whether to adjust the QP value for each CTU, filtering out cases with high prediction risk from the fitted model and improving the algorithm's stability across different video scenes. This embodiment has low computational cost, requiring only frame-level and CTU-level features obtained from the encoder's built-in precoding stage to obtain the features needed for the fitted model. Step S506 then allows for rapid prediction of QP bias values ​​that meet the conditions, without incurring additional computational costs and is easily applied in fast encoders.

[0226] This embodiment was applied to the short video end of the S266 encoder, and the coding efficiency was tested on 63 video sequences, as shown in Table 1. The levels represent different performance modes of the encoder, such as Medium and Slow modes. Slow mode indicates that the encoder will spend more time optimizing video compression, potentially resulting in higher coding efficiency and video quality. Peak Signal-to-Noise Ratio (PSNR) can be used to objectively assess the error level in the video, and Structural Similarity (SSIM) can be used to measure the similarity of structural and texture information. The average BD-rate shows the average percentage of bitrate saved by the encoder optimized using this embodiment compared to the benchmark encoder, while maintaining the same or similar video quality. A negative BD-rate value indicates bitrate savings, meaning that the optimized encoder can use fewer bits on average when encoding the same quality video, reflecting an improvement in coding efficiency.

[0227] Table 1 Encoding Efficiency Test Table

[0228] For the new generation H.266 / VVC video coding standard, existing rate control optimization algorithms mainly focus on improving control accuracy rather than prioritizing coding efficiency, and are usually designed for standard encoders, making it difficult to meet the fast coding requirements of real-world scenarios. To address this, this application proposes an adaptive video scene rate control optimization algorithm that improves coding efficiency by selecting a suitable QP offset value for each frame.

[0229] This embodiment can extract suitable intra-frame / inter-frame features for video frames of various types and temporal levels, and fit the relationship between the features and the QP bias that meets the conditions through a prediction model. This embodiment includes a two-level QP optimization strategy at the frame level and CTU level, which can filter out cases with high prediction risk and improve the stability of the algorithm based on optimizing the frame-level QP bias.

[0230] This embodiment presents a general-purpose bitrate allocation optimization scheme. The principles of feature extraction and model fitting are not limited to specific encoder versions. It can be used not only with standard encoders like VTM or open-source fast encoders like VVenC, but also with third-party encoders supporting H.266 / VVC or similar standards, making it widely applicable. This embodiment is simple to implement and has low computational requirements. It utilizes information obtained from pre-analysis before encoding each frame to quickly predict the frame-level QP bias. Since the pre-analysis stage can use downsampled frame content instead of the original resolution content, or is already built into the encoder, it requires minimal additional computation and is easy to implement in real-world scenarios.

[0231] In this embodiment, the current video frame to be encoded is obtained from the video sequence; based on the current video frame, a video feature set associated with the video scene in which the video sequence is located is determined, wherein the video features in the video feature set are used to describe the picture characteristics of the current video frame in the video scene; the video feature set is converted into a video feature vector that matches a target model, wherein the target model is trained based on video feature vector samples and corresponding quantization parameter offset samples; the target model is used to predict the video feature vector to obtain the quantization parameter offset of the current video frame; the quantization parameters of the current video frame are adjusted using the quantization parameter offset; and the current video frame is encoded using the adjusted quantization parameters to obtain the encoding result. In other words, this application embodiment extracts the video feature set of the current video frame, converts the extracted video feature set into a video feature vector that matches the target model, accurately predicts the quantization parameter offset based on the video feature vector, and further adjusts the quantization parameter according to the actual situation based on the quantization parameter offset. The adjusted quantization parameter is then used to encode the current video frame to obtain the encoding result. Since the quantization parameter offset is predicted based on the target model to adjust the quantization parameter, it avoids determining the quantization parameter that meets the conditions through multiple traversals and tests, thereby achieving the technical effect of improving the efficiency of video encoding and solving the technical problem of low efficiency in video encoding.

[0232] Figure 7 is a structural block diagram of a computing environment for a video coding and model determination method according to an embodiment of this application. As shown in Figure 7, the computing environment 701 includes multiple computing nodes (such as servers) running on a distributed network (shown as 710-1, 710-2, ... in the figure). Each computing node contains local processing and memory resources, and the end user 702 can remotely run applications or store data in the computing environment 701. The applications can be provided as multiple services 720-1, 720-2, 720-3, and 720-4 in the computing environment 701, representing services "A", "D", "E", and "H", respectively.

[0233] End user 702 can provide and access services through a web browser or other software application on a client. In some embodiments, the provisioning and / or requests of end user 702 can be provided to ingress gateway 730. Ingress gateway 730 may include a corresponding agent to handle the provisioning and / or requests for services (one or more services provided in computing environment 701).

[0234] The service is provided or deployed based on various virtualization technologies supported by the computing environment 701. In some embodiments, the service may be provided based on virtual machine (VM)-based virtualization, container-based virtualization, and / or similar methods. VM-based virtualization can simulate a real computer by initializing a virtual machine, executing programs and applications without directly accessing any actual hardware resources. While the machine is virtualized by a virtual machine, container-based virtualization can launch containers to virtualize an entire operating system (OS), allowing multiple workloads to run on a single OS instance.

[0235] In one embodiment based on container virtualization, several containers of a service can be assembled into a Pod (e.g., a Kubernetes Pod). For example, as shown in Figure 7, service 720-2 can be equipped with one or more Pods 740-1, 740-2, ..., 740-N (collectively referred to as Pods). A Pod can include a proxy 745 and one or more containers 742-1, 742-2, ..., 742-M (collectively referred to as containers). One or more containers in a Pod handle requests related to one or more corresponding functions of the service. The proxy 745 typically controls service-related network functions such as routing and load balancing. Other services can also be equipped with Pods similar to Pods.

[0236] During operation, executing a user request from end user 702 may require calling one or more services in computing environment 701, and executing one or more functions of one service may require calling one or more functions of another service. As shown in Figure 7, service "A" 720-1 receives a user request from end user 702 from ingress gateway 730. Service "A" 720-1 can call service "D" 720-2, and service "D" 720-2 can request service "E" 720-3 to execute one or more functions.

[0237] The aforementioned computing environment can be a cloud computing environment, where resource allocation is managed by cloud services, allowing functionality development without needing to consider implementation, adjustment, or server scaling. This computing environment allows developers to execute event-responsive code without building or maintaining complex infrastructure. Services can be partitioned into a set of functions that can automatically and independently scale, rather than scaling a single hardware device to handle potential loads.

[0238] According to an embodiment of this application, a video encoding apparatus for implementing the video encoding method shown in FIG2 is also provided.

[0239] Figure 8 is a schematic diagram of a video encoding device according to an embodiment of the present application. As shown in Figure 8, the video encoding device 800 may include: a first acquisition unit 802, a determination unit 804, a conversion unit 806, a prediction unit 808, an adjustment unit 810, and an encoding unit 812.

[0240] The first acquisition unit 802 is used to acquire the current video frame to be encoded from the video sequence.

[0241] The determining unit 804 is used to determine a video feature set associated with the video scene where the video sequence is located based on the current video frame, wherein the video features in the video feature set are used to describe the picture characteristics of the current video frame in the video scene.

[0242] The conversion unit 806 is used to convert the video feature set into a video feature vector that matches the target model, wherein the target model is trained based on the video feature vector samples and the corresponding quantization parameter offset samples.

[0243] Prediction unit 808 is used to predict the video feature vector using the target model to obtain the quantization parameter offset of the current video frame.

[0244] The adjustment unit 810 is used to adjust the quantization parameters of the current video frame using the quantization parameter offset.

[0245] The encoding unit 812 uses the adjusted quantization parameters to encode the current video frame and obtain the encoding result.

[0246] It should be noted that the first acquisition unit 802, determination unit 804, conversion unit 806, prediction unit 808, adjustment unit 810, and encoding unit 812 mentioned above correspond to steps S202 to S212. The six units and the corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed above. It should be noted that the above units can be hardware or software components stored in memory (e.g., memory 1004) and processed by one or more processors (e.g., processors 1002a, 1002b, ..., 1002n). The above units can also be part of a device and run in computer terminal A.

[0247] In this video encoding apparatus, the first acquisition unit 802 acquires the current video frame to be encoded from the video sequence. The determination unit 804 determines a video feature set associated with the video scene in which the video sequence is located, based on the current video frame. The video features in the feature set describe the image characteristics of the current video frame within the video scene. The conversion unit 806 converts the video feature set into a video feature vector that matches a target model, where the target model is trained based on video feature vector samples and corresponding quantization parameter offset samples. The prediction unit 808 uses the target model to predict the video feature vector, obtaining the quantization parameter offset of the current video frame. The adjustment unit 810 adjusts the quantization parameters of the current video frame using the quantization parameter offset. The encoding unit 812 encodes the current video frame using the adjusted quantization parameters to obtain the encoding result. In other words, this application embodiment extracts the video feature set of the current video frame, converts the extracted video feature set into a video feature vector that matches the target model, accurately predicts the quantization parameter offset based on the video feature vector, and further adjusts the quantization parameter according to the actual situation based on the quantization parameter offset. The adjusted quantization parameter is then used to encode the current video frame to obtain the encoding result. Since the quantization parameter offset is predicted based on the target model to adjust the quantization parameter, it avoids determining the quantization parameter that meets the conditions through multiple traversals and tests, thereby achieving the technical effect of improving the efficiency of video encoding and solving the technical problem of low efficiency in video encoding.

[0248] According to an embodiment of this application, a model determination apparatus is also provided for implementing the model determination method shown in FIG3 above.

[0249] Figure 9 is a schematic diagram of a model determination device according to an embodiment of the present application. As shown in Figure 9, the model determination device 900 may include a second acquisition unit 902 and a training unit 904.

[0250] The second acquisition unit 902 is used to acquire video feature vector samples and acquire quantization parameter offset samples corresponding to the video feature vector samples.

[0251] Training unit 904 is used to train a target model using video feature vector samples and quantization parameter offset samples. The target model is used to predict the quantization parameter offset by the video feature vectors that match the target model. The video feature vectors are converted from the video feature set associated with the video scene in the video sequence. The video features in the video feature set are used to describe the picture characteristics of the current video frame to be encoded in the video scene. The video feature set is determined based on the current video frame. The quantization parameter offset is used to adjust the quantization parameters of the current video frame. The adjusted quantization parameters are used to encode the current video frame to obtain the encoding result.

[0252] Here, the second acquisition unit 902 and training unit 904 correspond to steps S302 to S304. The two units and the corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed above. It should be noted that the above units can be hardware or software components stored in memory (e.g., memory 1004) and processed by one or more processors (e.g., processors 1002a, 1002b, ..., 1002n). The above units can also be part of a device and run in computer terminal A.

[0253] In the model determination device, video feature vector samples and corresponding quantization parameter offset samples are acquired by the second acquisition unit 902. The target model is trained by the training unit 904 using the video feature vector samples and quantization parameter offset samples. The target model is used to predict video feature vectors matching the target model to obtain quantization parameter offsets. The video feature vectors are converted from a video feature set associated with the video scene in the video sequence. The video features in the video feature set describe the image characteristics of the current video frame to be encoded in the video scene. The video feature set is determined based on the current video frame. The quantization parameter offset is used to adjust the quantization parameters of the current video frame. The adjusted quantization parameters are used to encode the current video frame to obtain the encoding result. In other words, this application embodiment extracts the video feature set of the current video frame, converts the extracted video feature set into a video feature vector that matches the target model, accurately predicts the quantization parameter offset based on the video feature vector, and further adjusts the quantization parameter according to the actual situation based on the quantization parameter offset. The adjusted quantization parameter is then used to encode the current video frame to obtain the encoding result. Since the quantization parameter offset is predicted based on the target model to adjust the quantization parameter, it avoids determining the quantization parameter that meets the conditions through multiple traversals and tests, thereby achieving the technical effect of improving the efficiency of video encoding and solving the technical problem of low efficiency in video encoding.

[0254] Embodiments of this application may provide a computer terminal, which may be any computer terminal device in a group of computer terminals. Optionally, in this embodiment, the aforementioned computer terminal may also be replaced by a mobile terminal or other terminal device.

[0255] Optionally, in this embodiment, the computer terminal may be located in at least one of a plurality of network devices in a computer network.

[0256] In this embodiment, the computer terminal described above can execute the program code for the following steps in the video encoding method: obtaining the current video frame to be encoded from the video sequence; determining a video feature set associated with the video scene in which the video sequence is located based on the current video frame, wherein the video features in the video feature set are used to describe the picture characteristics of the current video frame in the video scene; converting the video feature set into a video feature vector that matches a target model, wherein the target model is trained based on video feature vector samples and corresponding quantization parameter offset samples; predicting the video feature vector using the target model to obtain the quantization parameter offset of the current video frame; adjusting the quantization parameters of the current video frame using the quantization parameter offset; and encoding the current video frame using the adjusted quantization parameters to obtain the encoding result.

[0257] Optionally, FIG10 is a structural block diagram of a computer terminal according to an embodiment of the present application. As shown in FIG10, the computer terminal A may include: one or more (only one is shown in the figure) processors 1002, memory 1004 and transmission device 1006.

[0258] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the video encoding and model determination method and apparatus in this application embodiment. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby realizing the aforementioned video encoding and model determination method. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to terminal A via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0259] The processor can access information and applications stored in the memory via a transmission device to perform the following steps: acquiring video feature vector samples and acquiring quantization parameter offset samples corresponding to the video feature vector samples; training a target model using the video feature vector samples and quantization parameter offset samples, wherein the target model is used to predict video feature vectors that match the target model to obtain quantization parameter offsets, the video feature vectors are converted from a video feature set associated with the video scene in which the video sequence is located, the video features in the video feature set are used to describe the picture characteristics of the current video frame to be encoded in the video scene, the video feature set is determined based on the current video frame, the quantization parameter offsets are used to adjust the quantization parameters of the current video frame, and the adjusted quantization parameters are used to encode the current video frame to obtain the encoding result.

[0260] Those skilled in the art will understand that the structure shown in Figure 10 is merely illustrative, and computer terminal A can also be a smartphone (such as an Android phone, iOS phone, etc.), tablet computer, PDA, mobile internet device (MID), personal access display (PAD), or other terminal device. Figure 10 does not limit the structure of the computer terminal A described above. For example, computer terminal A may include more or fewer components (such as network interfaces, display devices, etc.) than shown in Figure 10, or have a different configuration than that shown in Figure 10.

[0261] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0262] Embodiments of this application also provide a computer-readable storage medium. Optionally, in this embodiment, the aforementioned computer-readable storage medium can be used to store the program code executed by the video encoding and model determination method provided in Embodiment 1.

[0263] Optionally, in this embodiment, the computer-readable storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.

[0264] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for performing the following steps: obtaining the current video frame to be encoded from the video sequence; determining a video feature set associated with the video scene in which the video sequence is located based on the current video frame, wherein the video features in the video feature set are used to describe the picture characteristics of the current video frame in the video scene; converting the video feature set into a video feature vector that matches a target model, wherein the target model is trained based on video feature vector samples and corresponding quantization parameter offset samples; predicting the video feature vector using the target model to obtain the quantization parameter offset of the current video frame; adjusting the quantization parameters of the current video frame using the quantization parameter offset; and encoding the current video frame using the adjusted quantization parameters to obtain an encoding result.

[0265] As an optional example, a computer-readable storage medium is configured to store program code for performing the following steps: acquiring video feature vector samples and acquiring quantization parameter offset samples corresponding to the video feature vector samples; training a target model using the video feature vector samples and the quantization parameter offset samples, wherein the target model is used to predict video feature vectors that match the target model to obtain quantization parameter offsets, the video feature vectors are converted from a video feature set associated with the video scene in which the video sequence is located, the video features in the video feature set are used to describe the picture characteristics of the current video frame to be encoded in the video scene, the video feature set is determined based on the current video frame, the quantization parameter offsets are used to adjust the quantization parameters of the current video frame, and the adjusted quantization parameters are used to encode the current video frame to obtain an encoding result.

[0266] Embodiments of this application also provide a computer program product. Optionally, in this embodiment, the computer program product may include a computer program that, when executed by a processor, implements the methods provided in the embodiments described above.

[0267] Optionally, the computer program product described above may include a non-volatile computer-readable storage medium, which can be used to store a computer program that, when executed by a processor, implements the method provided in the above embodiments.

[0268] Embodiments of this application may provide an electronic device that may include a memory and a processor.

[0269] Figure 11 is a block diagram of an electronic device for determining a video encoding and model according to an embodiment of this application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0270] As shown in Figure 11, device 1100 includes a computing unit 1101, which can perform various appropriate actions and processes according to a computer program stored in read-only memory (ROM) 1102 or a computer program loaded into random access memory (RAM) 1103 from storage unit 1108. RAM 1103 may also store various programs and data required for the operation of device 1100. The computing unit 1101, ROM 1102, and RAM 1103 are interconnected via bus 1104. Input / output (I / O) interface 1105 is also connected to bus 1104.

[0271] Multiple components in device 1100 are connected to I / O interface 1105, including: input unit 1106, such as a keyboard, mouse, etc.; output unit 1107, such as various types of displays, speakers, etc.; storage unit 1108, such as a disk, optical disk, etc.; and communication unit 1109, such as a network card, modem, wireless transceiver, etc. Communication unit 1109 allows device 1100 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0272] The computing unit 1101 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1101 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. The computing unit 1101 performs the various methods and processes described above, such as video encoding methods and model determination methods. For example, in some embodiments, the video encoding and model determination methods can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 1108. In some embodiments, part or all of the computer program can be loaded and / or installed on device 1100 via ROM 1102 and / or communication unit 1109. When the computer program is loaded into RAM 1103 and executed by computing unit 1101, one or more steps of the video encoding and model determination method described above can be performed. Alternatively, in other embodiments, computing unit 1101 can be configured to perform the video encoding and model determination method by any other suitable means (e.g., by means of firmware).

[0273] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard parts (ASSPs), system-on-a-chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0274] The program code used to implement the methods of this application may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0275] The method embodiments provided in this application can be executed in a mobile terminal, computer terminal, or similar computing device. Figure 12 is a hardware structure block diagram of a computer terminal (or mobile device) for implementing a video encoding and model determination method according to an embodiment of this application. As shown in Figure 12, the computer terminal 120 (or mobile device) may include one or more processors 1202 (shown as 1202a, 1202b, ..., 1202n in the figure) (processor 1202 may include, but is not limited to, a microprocessor (MCU) or a programmable gate array (FPGA), etc.), a memory 1204 for storing data, and a transmission device 1206 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a Universal Serial Bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that the structure shown in Figure 12 is only illustrative and does not limit the structure of the above electronic device. For example, computer terminal 120 may also include more or fewer components than shown in FIG. 12, or have a different configuration than shown in FIG. 12.

[0276] The hardware structure block diagram shown in Figure 12 can serve as an exemplary block diagram not only for the computer terminal 120 (or mobile device) described above, but also as an exemplary block diagram for the server described above. In one optional embodiment, Figure 12 illustrates an embodiment using the computer terminal 120 (or mobile device) shown in Figure 12 as a computing node in the computing environment 701.

[0277] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by an instruction execution system, apparatus, or device, or in combination with such an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0278] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a cathode ray tube (CRT) or liquid crystal display (LCD)) for displaying information to the user; a monitor; and a keyboard and pointing device (e.g., a mouse or pathball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0279] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication (e.g., a communication network) of any form or medium. Examples of communication networks include Local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.

[0280] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0281] It should be noted that the sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0282] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0283] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of units or modules may be electrical or other forms.

[0284] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0285] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0286] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory, random access memory, portable hard drive, magnetic disk, or optical disk.

[0287] The above are merely preferred embodiments of this application. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A video encoding method, characterized in that, include: Obtain the current video frame to be encoded from the video sequence; Based on the current video frame, a video feature set associated with the video scene in which the video sequence is located is determined, wherein the video features in the video feature set are used to describe the picture characteristics of the current video frame in the video scene; The video feature set is converted into a video feature vector that matches the target model, wherein the target model is trained based on video feature vector samples and corresponding quantization parameter offset samples; The target model is used to predict the video feature vector to obtain the quantization parameter offset of the current video frame; The quantization parameters of the current video frame are adjusted using the quantization parameter offset. The current video frame is encoded using the adjusted quantization parameters to obtain the encoding result.

2. The method according to claim 1, characterized in that, Converting the video feature set into a video feature vector that matches the target model includes: From the video feature set, extract at least one video feature that corresponds to the frame type of the current video frame; The extracted video features are converted into video feature vectors that match the target model.

3. The method according to claim 2, characterized in that, The extracted video features are converted into video feature vectors that match the target model, including: The extracted video features are integrated to obtain the video feature vector that matches the target model.

4. The method according to claim 3, characterized in that, The extracted video features are integrated to obtain the video feature vector that matches the target model, including: The extracted video features are integrated to obtain an initial video feature vector; The initial video feature vector is preprocessed to obtain a video feature vector with the same dimensions as the quantization parameters and that matches the target model.

5. The method according to claim 2, characterized in that, The video feature set includes an intra-frame feature set and an inter-frame feature set. Intra-frame features in the intra-frame feature set describe the characteristics of the current video frame's content within the video scene. Inter-frame features in the inter-frame feature set describe the characteristics of the content between the current video frame and a corresponding reference video frame within the video scene. From the video feature set, at least one video feature corresponding to the frame type of the current video frame is extracted, including: In response to the frame type being I-frame, the mean intra-frame prediction cost of the current video frame is extracted from the intra-frame feature set, and / or the mean inter-frame content difference between the current video frame and the reference video frame is extracted from the inter-frame feature set. In response to the frame type being a non-I-frame type, the mean value of the intra-frame prediction cost corresponding to the temporal level of the current video frame is extracted from the intra-frame feature set, and / or the mean value of the inter-frame content difference corresponding to the temporal level is extracted from the intra-frame feature set.

6. The method according to claim 1, characterized in that, The target model is used to predict the video feature vector to obtain the quantization parameter offset of the current video frame, including: Using the target model and the frame type of the current video frame, the video feature vector is predicted to obtain the quantization parameter offset.

7. The method according to claim 6, characterized in that, Using the target model and the frame type of the current video frame, the video feature vector is predicted to obtain the quantization parameter offset, including: In the target model, a weight vector matching the frame type is determined, wherein the weight vector includes at least the weight coefficients of the video feature vector; The video feature vector is adjusted using the weight vector to obtain the quantization parameter offset.

8. The method according to claim 1, characterized in that, The current video frame includes multiple coding tree units. The quantization parameters of the current video frame are adjusted using the quantization parameter offset, including: Determine the screen content of the encoded tree unit; The quantization parameters of the coding tree unit are adjusted using the quantization parameter offset and the image content.

9. The method according to claim 8, characterized in that, Adjusting the quantization parameters of the coding tree unit using the quantization parameter offset and the image content includes: In response to the quantization parameter offset being greater than an offset threshold, and the complexity of the image content in the spatial and temporal domains being greater than a complexity threshold, the quantization parameter of the coding tree unit is adjusted using the quantization parameter offset, wherein the complexity is used to represent the complexity of the image content; In response to the quantization parameter offset being no greater than the offset threshold and the complexity of the image content being less than the complexity threshold, the quantization parameter of the coding tree unit is adjusted using the quantization parameter offset, wherein the complexity is used to represent the complexity of the image content.

10. The method according to claim 1, characterized in that, The current video frame includes multiple coding tree units. The quantization parameters of the current video frame are adjusted using the quantization parameter offset, including: The quantization parameters of each coding tree unit are adjusted using the quantization parameter offset.

11. The method according to any one of claims 1 to 10, characterized in that, The method further includes: In response to the adjusted quantization parameter being greater than a first quantization parameter threshold or less than a second quantization parameter threshold, the quantization parameter before adjustment is maintained. The current video frame is encoded using the quantization parameters before adjustment to obtain the encoding result.

12. A method for determining a model, characterized in that, include: Obtain video feature vector samples, and obtain quantization parameter offset samples corresponding to the video feature vector samples; A target model is trained using the video feature vector samples and the quantization parameter offset samples. The target model is used to predict the video feature vectors that match the target model to obtain the quantization parameter offset. The video feature vectors are converted from a video feature set associated with the video scene in which the video sequence is located. The video features in the video feature set are used to describe the picture characteristics of the current video frame to be encoded in the video sequence under the video scene. The video feature set is determined based on the current video frame. The quantization parameter offset is used to adjust the quantization parameters of the current video frame. The adjusted quantization parameters are used to encode the current video frame to obtain the encoding result.

13. The method according to claim 12, characterized in that, Obtain video feature vector samples, including: Multiple lossless video sequence samples are obtained, wherein the different lossless video sequence samples come from different video scenes; The video feature vector samples are obtained from the multiple lossless video sequence samples.

14. The method according to claim 13, characterized in that, Obtaining the video feature vector samples from the multiple lossless video sequence samples includes: Based on the video frame samples in the lossless video sequence samples, a set of video feature samples associated with the video scene where the lossless video sequence samples are located is determined, wherein the video feature samples in the set of video feature samples are used to describe the picture characteristics of the video frame samples in the corresponding video scene. The video feature sample set is converted into the video feature vector sample.

15. The method of claim 14, characterized in that, Based on the video frame samples in the lossless video sequence samples, a set of video feature samples associated with the video scene where the lossless video sequence samples are located is determined, including: The video frame samples are downsampled, wherein the resolution of the downsampled video frame samples is lower than a resolution threshold; The video feature sample set is determined based on the downsampled video frame samples.

16. The method according to claim 14, characterized in that, Converting the video feature sample set into video feature vector samples includes: From the video feature sample set, extract at least one video feature sample that corresponds to the frame type of the video frame sample; The extracted video feature samples are integrated to obtain initial video feature vector samples; The initial video feature vector samples are preprocessed to obtain the video feature vector samples.

17. The method of claim 16, characterized in that, The video feature sample set includes an intra-frame feature sample set and an inter-frame feature sample set. The intra-frame feature samples in the intra-frame feature sample set describe the characteristics of the video frame sample's content within a corresponding video scene. The inter-frame feature samples in the inter-frame feature sample set describe the characteristics of the video frame sample's content with a corresponding reference video frame sample within a corresponding video scene. From the video feature sample set, at least one video feature sample corresponding to the frame type of the video frame sample is extracted, including: In response to the frame type being I-frame type, the mean intra-frame prediction cost of the video frame sample is extracted from the intra-frame feature sample set, and / or the mean inter-frame content difference between the video frame sample and the reference video frame sample is extracted from the inter-frame feature sample set. In response to the frame type being a non-I-frame type, the mean value of the intra-frame prediction cost corresponding to the temporal level of the video frame sample is extracted from the intra-frame feature sample set, and / or the mean value of the inter-frame content difference corresponding to the temporal level is extracted from the intra-frame feature sample set.

18. The method according to claim 13, characterized in that, Obtaining the quantization parameter offset samples corresponding to the video feature vector samples includes: The lossless video sequence samples are encoded using multiple bitrate control strategies corresponding to multiple bitrates to obtain multiple encoded result samples, wherein the multiple encoded result samples correspond to multiple bitrates; The quantization parameter offset sample is determined from multiple initial quantization parameter offset samples corresponding to multiple encoded result samples.

19. The method according to any one of claims 13 to 18, characterized in that, The target model is trained using the video feature vector samples and the quantization parameter offset samples, including: The video feature vector samples are converted into the input matrix of the initial model, and the quantization parameter offset samples are converted into the output matrix of the initial model. The input matrix and the output matrix are fitted to obtain a weight vector sample corresponding to the initial model, wherein the weight vector sample includes at least the weight coefficient sample of the video feature vector sample; The target model is established using the weight vector samples and the initial model.

20. A video encoding system, characterized in that, include: An encoder is used to extract the current video frame to be encoded from a video sequence. The pre-analysis module is invoked to determine a video feature set associated with the video scene in which the video sequence is located, based on the current video frame. The video features in the feature set describe the image characteristics of the current video frame within the video scene. The video feature set is then converted into a video feature vector that matches a target model, which is trained based on video feature vector samples and corresponding quantization parameter offset samples. The target model is used to predict the video feature vector to obtain the quantization parameter offset of the current video frame. The quantization parameters of the current video frame are adjusted using the quantization parameter offset. Finally, the adjusted quantization parameters are used to encode the current video frame to obtain an encoding result. A server is used to obtain the encoded result.

21. A computing device, characterized in that, include: Memory, which stores executable programs; A processor for running the program, wherein the program, when running, performs the method according to any one of claims 1 to 19.

22. An electronic device, characterized in that, include: Memory, which stores executable programs; A processor, connected to the memory via a bus, is used to run the program, wherein the program, when running, executes the method according to any one of claims 1 to 19.

23. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored executable program, wherein, when the executable program is executed, it controls the device on which the computer-readable storage medium is located to perform the method according to any one of claims 1 to 19.

24. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the method described in any one of claims 1 to 19.

Images