Video encoding method and device, electronic equipment and storage medium

Abstract

The present disclosure provides a video encoding method, device, electronic equipment and computer readable storage medium, relating to the technical field of video processing. The method comprises: obtaining a video frame sequence of a to-be-processed video and a code rate control model; determining a first to-be-encoded frame from the video frame sequence, performing an encoding processing operation corresponding to the first to-be-encoded frame, and the code rate control model having a data operation protection lock at a specific step of the encoding processing operation; while performing the encoding processing operation on the first to-be-encoded frame, performing a code rate control end operation on the encoded frame to obtain an encoded video of the to-be-processed video; and the code rate control model having a data operation protection lock at a specific step of the code rate control end operation. The present disclosure adds a data protection mechanism to the code rate control model at a specific step of the encoding process, avoids redundant waiting in the code rate control process, and improves the encoding speed on the basis of ensuring encoding safety.

Description

Technical Field

[0001] This disclosure relates to the field of video processing technology, and in particular to a video encoding method, a video encoding apparatus, an electronic device, and a computer-readable storage medium. Background Technology

[0002] With the explosive growth of video content on the internet, storing and transmitting videos requires massive amounts of space and bandwidth. To encode higher-quality videos at lower bitrates, video encoding algorithms are constantly being innovated, continuously improving video compression efficiency, saving storage and transmission costs while enhancing the user viewing experience. Video encoding standards specify the set of encoding tools that can be used by encoding algorithms, and an encoder is an implementation of an encoding algorithm that conforms to a specific encoding standard.

[0003] Depending on the specific application scenario, there are various requirements and limitations on the output bitrate of video encoders, including maximum target bitrate and average bitrate. Bitrate control is used to control the quantizer parameter (QP) value of each frame for various application scenarios, so that the bitrate of the encoded video output meets the specific constraints, while optimizing the encoding performance as much as possible.

[0004] Some related video encoding schemes employ a multi-threaded encoding pipeline for video encoding processing. To ensure consistency in the encoding results across multiple encoding operations, such schemes not only need to wait for the current frame to complete its encoding process but also for all other frames in their respective threads to complete their encoding processes before proceeding with the bitrate control process. This results in redundant waiting and low resource utilization. Summary of the Invention

[0005] This disclosure provides a video encoding method, apparatus, electronic device, computer-readable storage medium, and computer program product to at least solve the problem in related technologies where redundant waiting times between frame threads lead to low resource utilization and reduced encoding speed. The technical solution of this disclosure is as follows:

[0006] According to a first aspect of the present disclosure, a video encoding method is provided, comprising: acquiring a video frame sequence and a bitrate control model of a video to be processed; the video frame sequence including frames to be encoded and encoded frames; determining a first frame to be encoded from the video frame sequence; performing an encoding processing operation corresponding to the first frame to be encoded; wherein the bitrate control model has a data operation protection lock at a specific step of the encoding processing operation, the data operation protection lock being used to isolate external resources from data access operations of the bitrate control model; simultaneously performing the encoding processing operation on the first frame to be encoded, performing a bitrate control termination operation on the encoded frames to obtain an encoded video of the video to be processed; wherein the bitrate control model has a data operation protection lock at a specific step of the bitrate control termination operation.

[0007] In one exemplary embodiment of this disclosure, the method further includes: determining the encoded frame number of the encoded frame, the encoded frame number being used to determine the first frame to be encoded; obtaining a pre-configured bitrate control model, the bitrate control model including bitrate control conditions for encoding the video frame sequence; obtaining a pre-configured plurality of frame threads, and generating the frame thread set based on the plurality of frame threads.

[0008] In an exemplary embodiment of this disclosure, the encoding processing operation includes a bitrate control start operation, a frame encoding operation, and a bitrate control end operation. Executing the encoding processing operation corresponding to the first frame to be encoded includes: starting a main thread for video encoding; executing the bitrate control start operation based on the main thread; simultaneously executing the bitrate control start operation, adding a data operation protection lock to the first frame to be encoded based on the main thread; performing the frame encoding operation on the first frame to be encoded to obtain a corresponding frame encoding result; and triggering the main thread to execute the bitrate control end operation corresponding to the first frame to be encoded based on the frame encoding result, generating a corresponding bitrate control end instruction.

[0009] In one exemplary embodiment of this disclosure, performing the frame encoding operation on the first frame to be encoded to obtain the corresponding frame encoding result includes: determining a first frame thread corresponding to the first frame to be encoded from the frame thread set; performing the frame encoding operation on the first frame to be encoded based on the first frame thread; and releasing the first frame thread in response to a frame encoding end instruction corresponding to the frame encoding operation.

[0010] In an exemplary embodiment of this disclosure, the step of triggering the main thread to execute a bitrate control termination operation corresponding to the first frame to be encoded based on the frame encoding result includes: obtaining a first thread state corresponding to the first frame thread based on the main thread; controlling the first frame thread to continue executing the frame encoding operation when the first thread state is detected to be non-idle; and adding an encoded identifier to the first frame to be encoded in response to the detection that the first thread state is idle, thereby triggering the main thread to execute the bitrate control termination operation corresponding to the first frame to be encoded.

[0011] In one exemplary embodiment of this disclosure, the rate control start operation is performed by the following steps: obtaining rate parameters for rate control processing of the first frame to be encoded; and determining the predictive coding result corresponding to the first frame to be encoded based on the rate parameters.

[0012] In an exemplary embodiment of this disclosure, the rate control termination operation is performed through the following steps: obtaining the predicted coding result and the actual coding result corresponding to the first frame to be encoded; obtaining the rate control model at the current moment; updating the rate control model based on the predicted coding result and the actual coding result to obtain the updated rate control model; and performing rate control processing on the remaining second frames to be encoded in the frame sequence based on the updated rate control model.

[0013] In an exemplary embodiment of this disclosure, before performing the bitrate control end operation for the encoded frame, the method further includes: obtaining a first frame thread number corresponding to a first frame thread; determining a second frame thread number based on the first frame thread number, and determining a second frame thread based on the second frame thread number; obtaining a second thread state corresponding to the second frame thread; and determining the video frame type of the video frame to be frame encoded by the second frame thread based on the second thread state; wherein the video frame type includes an encoded frame type.

[0014] According to a second aspect of the present disclosure, a video encoding apparatus is provided, comprising: an information acquisition module, configured to acquire a video frame sequence and a bitrate control model of a video to be processed; the video frame sequence includes frames to be encoded and encoded frames; a first encoding processing module, configured to determine a first frame to be encoded from the video frame sequence, and perform an encoding processing operation corresponding to the first frame to be encoded, wherein the bitrate control model has a data operation protection lock at a specific step of the encoding processing operation, the data operation protection lock being used to isolate external resources from data access operations of the bitrate control model; and a second encoding processing module, configured to perform a bitrate control termination operation for the encoded frames while performing the encoding processing operation on the first frame to be encoded, thereby obtaining an encoded video of the video to be processed; wherein the bitrate control model has a data operation protection lock at a specific step of the bitrate control termination operation.

[0015] In one exemplary embodiment of this disclosure, the video encoding apparatus includes an information acquisition module, configured to: determine the encoded frame number of the encoded frame, the encoded frame number being used to determine the first frame to be encoded; acquire a pre-configured bitrate control model, the bitrate control model including bitrate control conditions for encoding the video frame sequence; acquire a pre-configured plurality of frame threads, and generate the frame thread set based on the plurality of frame threads.

[0016] In an exemplary embodiment of this disclosure, the first encoding processing module includes a first encoding processing unit, configured to: start a main thread for video encoding; execute the bitrate control start operation based on the main thread; while executing the bitrate control start operation, add a data operation protection lock to the first frame to be encoded based on the main thread; perform the frame encoding operation on the first frame to be encoded to obtain a corresponding frame encoding result; and trigger the main thread to execute the bitrate control end operation corresponding to the first frame to be encoded based on the frame encoding result, generating a corresponding bitrate control end instruction.

[0017] In an exemplary embodiment of this disclosure, the first encoding processing unit includes a frame encoding subunit, configured to: determine a first frame thread corresponding to the first frame to be encoded from the set of frame threads; perform the frame encoding operation on the first frame to be encoded based on the first frame thread; and release the first frame thread in response to a frame encoding end instruction corresponding to the frame encoding operation.

[0018] In an exemplary embodiment of this disclosure, the first encoding processing unit includes a rate control termination subunit, configured to: obtain a first thread state corresponding to the first frame thread based on the main thread; control the first frame thread to continue executing the frame encoding operation when the first thread state is detected to be non-idle; and add an encoded identifier to the first frame to be encoded in response to the detection that the first thread state is idle, thereby triggering the main thread to execute the rate control termination operation corresponding to the first frame to be encoded.

[0019] In one exemplary embodiment of this disclosure, the first encoding processing module further includes a bitrate control start operation unit, configured to: acquire bitrate parameters for bitrate control processing of the first frame to be encoded; and determine the predictive encoding result corresponding to the first frame to be encoded based on the bitrate parameters.

[0020] In an exemplary embodiment of this disclosure, the first encoding processing module further includes a rate control termination operation unit, configured to: obtain the predicted encoding result and the actual encoding result corresponding to the first frame to be encoded; obtain the rate control model at the current moment; update the rate control model based on the predicted encoding result and the actual encoding result to obtain an updated rate control model; and perform rate control processing on the remaining second frames to be encoded in the frame sequence based on the updated rate control model.

[0021] In one exemplary embodiment of this disclosure, the video encoding apparatus further includes an encoded frame acquisition module, configured to: acquire a first frame thread number corresponding to a first frame thread; determine a second frame thread number based on the first frame thread number, and determine a second frame thread based on the second frame thread number; acquire a second thread state corresponding to the second frame thread; and determine the video frame type of the video frame to be frame encoded by the second frame thread based on the second thread state; wherein the video frame type includes an encoded frame type.

[0022] According to a third aspect of this disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute instructions to implement the video encoding method described in any of the preceding claims.

[0023] According to a fourth aspect of this disclosure, a computer-readable storage medium is provided, wherein when instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the video encoding method described in any of the preceding claims.

[0024] According to a fifth aspect of the present disclosure, a computer program product is provided, including a computer program / instructions, characterized in that the computer program / instructions, when executed by a processor, implement the video encoding method described in any one of the above claims.

[0025] The technical solutions provided by the embodiments of this disclosure offer at least the following beneficial effects: On the one hand, by adding a data operation protection lock to the rate control model in specific steps of the encoding process, it can be ensured that only one entity accesses the rate control model for data at any given time. Therefore, while ensuring data security, redundant waiting during the encoding process can be reduced, effectively improving resource utilization and increasing encoding speed. On the other hand, for some frames with long encoding times, since frame encoding processes for other encoded frames can be executed in parallel, the impact of long waiting times for individual frames on the overall encoding speed can be avoided, effectively improving encoder performance.

[0026] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0027] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0028] Figure 1 This is a flowchart for bitrate control in a related video encoding scheme.

[0029] Figure 2 This is a flowchart illustrating a video encoding method according to an exemplary embodiment.

[0030] Figure 3 This is an overall flowchart illustrating the encoding process of a video to be processed, according to an exemplary embodiment.

[0031] Figure 4 This is a schematic diagram illustrating the selection of the currently used frame thread from a set of frame threads according to an exemplary embodiment.

[0032] Figure 5 This is a block diagram illustrating a video encoding apparatus according to an exemplary embodiment.

[0033] Figure 6 A block diagram of an electronic device according to an exemplary embodiment of the present disclosure is shown schematically. Detailed Implementation

[0034] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0035] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0036] During video encoding, the output bitrate of the video encoder is controlled according to different application scenarios. Taking x265 as an example, each frame undergoes two processes on the main thread: bitrate control start operation (RC_start) and bitrate control end operation (RC_end). (See reference...) Figure 1 , Figure 1 This is a flowchart illustrating bitrate control using a relevant video encoding scheme. From... Figure 1 As can be seen, a counter value (cnt) can be used for conditional checks before the RC_end process. If a certain condition is met, the process will wait until the cnt value changes, and then re-check whether the condition is met. The RC_end process will only continue when the loop is exited. Figure 1 The bitrate control scheme changes the cnt value only in two places: first, after the encoding process of each frame ends; and second, after RC_end is completed.

[0037] However, this pipeline design does not fully leverage the advantages of multi-threaded parallel computing, spending considerable time waiting for other threads to complete encoding all frames. This waiting is redundant, assuming correct data dependencies between frames. Furthermore, if individual frames take too long to encode due to their complexity, this waiting mechanism significantly impacts the overall encoder speed, causing encoder stuttering and non-real-time issues.

[0038] Based on this, according to embodiments of the present disclosure, a video encoding method, a video encoding apparatus, an electronic device, a computer-readable storage medium, and a computer program product are proposed.

[0039] Figure 2 This is a flowchart illustrating a video encoding method according to an exemplary embodiment, such as... Figure 2As shown, the video encoding method can be used in computer devices. The computer devices described in this disclosure may include mobile terminal devices such as mobile phones, tablets, laptops, PDAs, and personal digital assistants (PDAs), as well as fixed terminal devices such as desktop computers. This exemplary embodiment illustrates the application of the method to a computer device; it is understood that the method can also be applied to a server, and further to a system including both computer devices and a server, and implemented through the interaction between the computer device and the server. Specifically, it includes the following steps.

[0040] In step S210, the video frame sequence and bitrate control model of the video to be processed are obtained; the video frame sequence includes the frame to be encoded and the encoded frame.

[0041] In one exemplary embodiment of this disclosure, the video to be processed may be a video awaiting video encoding processing. The video frame sequence may be a sequence composed of video frames contained in the video to be processed. The bitrate control model may be a data model used to control the bitrate of the frames to be encoded in the video to be processed. The frames to be encoded may be video frames that have not yet undergone frame encoding processing. The encoded frames may be video frames that have already completed frame encoding processing.

[0042] Due to limited network bandwidth and the vast number of video resources online, compression encoding can be performed on video resources before transmission to reduce transmission costs and improve efficiency, thereby decreasing the amount of video data transmitted. Before transmission, some videos to be transmitted online can be used as pre-processed videos, as can locally captured videos. The corresponding video frame sequence and a bitrate control model for encoding these pre-processed frames are determined. The video frame sequence can include already encoded frames and pre-encoded frames, and the bitrate control model includes the bitrate control conditions for encoding the video frames.

[0043] In addition, it is readily understood by those skilled in the art that before encoding a video to be processed, the video frame sequence may contain only the frame to be encoded corresponding to the video to be processed. After the encoding operation of the video to be processed is started, that is, during the encoding process, the content in the video frame sequence will change. For example, the video frame sequence may contain both the frame to be encoded and the encoded frame at the same time.

[0044] In step S220, the first frame to be encoded is determined from the video frame sequence, and the encoding processing operation corresponding to the first frame to be encoded is performed. The bitrate control model has a data operation protection lock in a specific step of the encoding processing operation. The data operation protection lock is used to isolate external resources from data access operations of the bitrate control model.

[0045] In one exemplary embodiment of this disclosure, the first frame to be encoded can be a video frame currently to be encoded, obtained from the frame to be encoded. The encoding processing operation can be a complete process of encoding a frame to be encoded, including a rate control start operation, a frame encoding operation, and a rate control end operation. The data operation protection lock can be a data protection identifier indicating that the rate control model is in a data protection state to isolate external resource access. External resources can be resources capable of accessing the frame to be encoded; for example, external resources can be the main thread and other frame threads in the system. Data access operations can include operations such as reading and modifying the model data of the rate control model.

[0046] When encoding a video frame sequence, the first frame to be encoded can be obtained from the sequence. This first frame can be determined based on the number of the already encoded video frames. For example, if the sequence contains n frames to be encoded and m frames have already been encoded, then the frame with the number m+1 can be obtained as the first frame to be encoded.

[0047] After acquiring the first frame to be encoded, encoding processing operations can be performed on it. Simultaneously, during these operations, a data operation protection lock can be added to the rate control model at specific steps, placing it in a data-protected state and isolating it from external data manipulation. At this point, encoding processing operations can be performed on the first frame, including rate control start, frame encoding, and rate control end operations.

[0048] Specifically, the rate control start operation may include frame information synchronization, Video Buffering Verifier (VBV) state synchronization, preprocessing information acquisition, rate control model estimation of quantization parameter QP value, and frame size estimation, among other steps. The specific steps for adding data operation protection locks during the rate control start operation may include steps for estimating the QP value and frame size. Frame encoding operations may include determining the initial QP value of the encoding block, model updates at the end of the encoding block, prediction, quantization, filtering, entropy coding, and other processing steps. The specific steps for adding data operation protection locks during frame encoding operations may include determining the initial QP value of the encoding block and model updates at the end of the encoding block. The rate control end operation may include frame-level rate control model updates, information synchronization, and VBV state synchronization, among other processing steps. The specific steps for adding data operation protection locks during the rate control end operation may include frame-level rate control model updates.

[0049] Because the rate control model is in a data protection state during specific steps of the encoding process, it can isolate external resources from accessing the data of the rate control model, effectively protecting the security of the rate control model data.

[0050] In step S230, while performing encoding processing on the first frame to be encoded, a bitrate control termination operation is performed on the encoded frame to obtain the encoded video of the video to be processed; the bitrate control model has a data operation protection lock in a specific step of the bitrate control termination operation.

[0051] In one exemplary embodiment of this disclosure, the idle frame thread can be a frame thread that is not currently occupied. The second frame to be encoded can be a video frame in the video frame sequence that has not yet undergone encoding processing and is waiting to be encoded.

[0052] Because the rate control model is in a data protection state during the encoding process of the first frame to be encoded, external resources cannot access it. This effectively ensures that only one process can access and modify the data in the rate control model at any given time, thus guaranteeing its data security. Therefore, while encoding the first frame, the rate control termination operations for already encoded frames in the video frame sequence can be executed in parallel. For example, in some processing steps of the frame encoding operation that do not require access to the rate control model, the data protection state of the rate control model can be released. At this time, the main thread can perform the rate control termination operation for already encoded frames. During this process, the rate control model used for video encoding is updated so that the remaining frames to be encoded can be encoded based on the updated model. Since the rate control termination operations for other already encoded frames can be executed simultaneously with the frame encoding of the first frame, redundant waiting time during the encoding process can be reduced, improving resource utilization.

[0053] According to the video encoding method in this example embodiment, on the one hand, by adding a data operation protection lock to the bitrate control model in specific steps of the encoding process, it can be ensured that only one entity can access the bitrate control model at any given time. Therefore, while ensuring data security, redundant waiting during the encoding process can be reduced, effectively improving resource utilization and increasing encoding speed. On the other hand, for some frames with long encoding times, since the frame encoding processes for other frames can be executed in parallel, the impact of long waiting times for individual frames on the overall encoding speed can be avoided, effectively improving encoder performance.

[0054] The video encoding method in this example embodiment will be further explained below.

[0055] In one exemplary embodiment of this disclosure, the encoded frame number of the encoded frame is determined, and the encoded frame number is used to determine the first frame to be encoded; a pre-configured bitrate control model is obtained, the bitrate control model includes bitrate control conditions for encoding the video frame sequence; a plurality of pre-configured frame threads are obtained, and a frame thread set is generated based on the plurality of frame threads.

[0056] Here, the encoded frame number can be the number of a video frame that has already been encoded. The bitrate control condition can be the constraint condition when encoding the video to be processed. The frame order can be the sequential dependency relationship between multiple video frames in the video to be processed. The frame thread can be the thread used to perform frame encoding processing on the frames to be encoded. The frame thread set can be a collection of multiple frame threads.

[0057] Before encoding the video, due to factors such as network environment, certain encoding constraints need to be imposed on the video to be processed. For example, bitrate control conditions can be added to the bitrate control model to constrain the encoding process of the frames to be encoded. Multiple frames to be encoded within the video are obtained. Since there is a frame sequence dependency among these frames, their order can be determined. Based on the frame sequence and the encoded frame number, the first frame to be encoded can be identified from the video frame sequence. Further, multiple pre-configured frame threads for frame encoding processing of the frames to be encoded can be obtained, generating a corresponding frame thread set. Encoding processing of the frame sequence is then performed based on this frame thread set. Through these steps, the set of frame threads for frame encoding processing of the frame sequence and the currently waiting frames to be encoded in the frame sequence can be determined, completing the preprocessing steps before frame encoding to trigger the encoding processing operation for the first frame to be encoded.

[0058] refer to Figure 3 , Figure 3 This is an overall flowchart illustrating the encoding process of a video to be processed according to an exemplary embodiment. In step S301, the encoding information of the video to be processed is obtained. For example, the encoding information may include the total number of encoded frames that have already been encoded, which can be represented by m; the number of frame threads included in the frame thread set, which can be represented by N; and the sequence number of the first idle frame thread in the frame thread set, which can be represented by i.

[0059] In one exemplary embodiment of this disclosure, step S220, performing the encoding processing operation corresponding to the first frame to be encoded, includes: starting the main thread for video encoding, and performing a bitrate control start operation based on the main thread; while performing the bitrate control start operation, adding a data operation protection lock to the first frame to be encoded based on the main thread; performing a frame encoding operation on the first frame to be encoded to obtain the corresponding frame encoding result; and triggering the main thread to perform a bitrate control end operation corresponding to the first frame to be encoded based on the frame encoding result, generating a corresponding bitrate control end instruction.

[0060] The main thread can be the thread started during the encoding process of the video to be processed. It can be used to execute bitrate control start and end operations for the frames to be encoded. The bitrate control start operation is an operation performed before image encoding of the frames to be encoded, involving frame-level bitrate prediction to meet preset bitrate constraints. The frame encoding operation is the process of encoding the frames to be encoded. The frame encoding result is the result obtained after frame encoding. The bitrate control end operation is performed after the image encoding of the frames to be encoded is completed, used to update the bitrate control model so that the subsequent bitrate allocation for the frames to be encoded is more accurate and reasonable. The bitrate control end command is an operation command generated after the bitrate control end operation.

[0061] Continue to refer to Figure 3 In step S302, the main thread begins. In step S303, the bitrate control operation is initiated. Because only one thread can access the bitrate control model during specific steps of encoding the frame to be encoded, subsequent operations must wait for the main operation using the bitrate control model to complete before proceeding. For example, in certain steps of frame encoding the first frame to be encoded, the bitrate control model is required. In this case, only the first frame thread corresponding to the first frame to be encoded can access the bitrate control model; the main thread or other frame threads cannot access the bitrate control model, otherwise, the data security of the bitrate control model will be compromised.

[0062] To address the aforementioned issues, the main thread can add a data operation protection lock to the rate control model during a specific step of the rate control initiation operation. Adding this lock ensures that the rate control model encoding the first frame to be encoded is in a data-protected state, allowing only one process to access its data at a time, effectively guaranteeing data security. By executing the rate control initiation operation for the first frame to be encoded—that is, performing frame-level rate prediction on the first frame—the predicted encoding result for that frame is obtained.

[0063] In step S304, the first frame to be encoded is obtained. The first frame to be encoded, determined by the main thread from the sequence of frames to be encoded, can be the video frame currently to be encoded. Since m video frames have already been encoded, the first frame to be encoded can be obtained by changing the sequence number of the frame to be encoded, such as by performing an operation of m = m + 1. After determining the first frame to be encoded, a frame encoding operation can be performed on the first frame to be encoded to obtain the corresponding frame encoding result. After obtaining the frame encoding result corresponding to the first frame to be encoded, the main thread can be triggered to perform a bitrate control termination operation for the first frame to be encoded, generating the corresponding bitrate control termination instruction. Through the above processing steps, the bitrate control model is kept in a data protection state during specific steps of the entire encoding process of the first frame to be encoded, preventing other threads or network resources from accessing the data of the bitrate control model during specific steps of the encoding process, thus ensuring the data security of the bitrate control model.

[0064] In one exemplary embodiment of this disclosure, performing a frame encoding operation on a first frame to be encoded to obtain a corresponding frame encoding result includes: determining a first frame thread corresponding to the first frame to be encoded from a set of frame threads; performing a frame encoding operation on the first frame to be encoded based on the first frame thread; and releasing the first frame thread in response to a frame encoding end instruction corresponding to the frame encoding operation.

[0065] The first frame thread can be the frame thread that performs the encoding process on the first frame to be encoded. The frame encoding end instruction can be an instruction generated by the first frame thread after it has completed the frame encoding operation on the first frame to be encoded.

[0066] Before performing frame encoding on the first frame to be encoded, the first frame thread corresponding to the first frame to be encoded can be determined from the frame thread set. For example, based on the sequence numbers of multiple frame threads in the frame thread set, the unused frame thread number can be determined. If frame thread i is not occupied, then frame thread i can be used as the first frame thread. (Continue to refer to...) Figure 3 In step S305, the first frame thread is started. By starting frame thread i, the processing operation of frame thread i can begin. In step S306, frame encoding operation is performed. During the frame encoding operation, the rate control model corresponding to the first frame to be encoded is also in a data protection state.

[0067] In step S307, the frame encoding operation for the first frame to be encoded is terminated, and the first frame thread is released. If the frame encoding operation for the first frame to be encoded is completed, a corresponding frame encoding termination instruction will be generated. Based on this instruction, the first frame thread can be released. Through the above processing steps, the frame encoding operation for the first frame to be encoded can be completed by the first frame thread while the rate control model is in a data protection state, effectively ensuring the data security of the rate control model.

[0068] In one exemplary embodiment of this disclosure, based on the frame encoding result, the main thread is triggered to execute a bitrate control termination operation corresponding to the first frame to be encoded, including: obtaining the first thread state corresponding to the first frame thread based on the main thread; controlling the first frame thread to continue executing the frame encoding operation when the first thread state is detected to be non-idle; and adding an encoded identifier to the first frame to be encoded in response to the detection that the first thread state is idle, and triggering the main thread to execute the bitrate control termination operation corresponding to the first frame to be encoded.

[0069] The first thread state can be defined as whether the first frame thread is currently occupied. The non-idle state can be defined as the state in which the frame thread is currently performing frame encoding processing. The idle state can be defined as the state in which the frame thread has completed the encoding processing for the first frame to be encoded and the frame thread resources have been released.

[0070] During video encoding, the main thread can obtain the thread state of the first frame thread to determine whether it is idle or not. If the main thread detects a non-idle state, it means the first frame thread's encoding operation on the first frame to be encoded has not yet finished. In this case, the first frame thread can continue to perform the encoding operation to complete the encoding of the first frame to be encoded. If the main thread detects an idle state, it means the first frame thread's encoding operation on the first frame to be encoded has finished. In this case, the state of the first frame to be encoded can be updated to "encoded frame," and an encoding flag can be added so that the main thread can subsequently execute the bitrate control termination operation corresponding to the first frame to be encoded. After the encoding operation on the first frame to be encoded is completed, the thread resources of the first frame thread can be released, and the frame thread state can be updated so that the frame thread state can be used to determine whether the encoding operation of the video frame has been completed.

[0071] In an exemplary embodiment of this disclosure, the bitrate control start operation in step S303 is performed by the following steps: obtaining bitrate parameters for bitrate control processing of the first frame to be encoded; and determining the predictive coding result corresponding to the first frame to be encoded based on the bitrate parameters.

[0072] The bitrate parameter can be the parameter used to encode the first frame to be encoded. The predictive coding result can be the coding prediction result of the first frame to be encoded, determined based on the bitrate parameter.

[0073] During the initial bitrate control operation, the main thread can obtain the bitrate parameters for the first frame to be encoded, which may include parameters such as the maximum target bitrate and average bitrate corresponding to the frame. After obtaining these bitrate parameters, the predictive coding result corresponding to the first frame to be encoded can be determined based on these parameters. The predictive coding result satisfies the pre-configured bitrate constraints, so that it can be used as the data basis for updating the bitrate control model during the subsequent bitrate control termination operation.

[0074] In an exemplary embodiment of this disclosure, the rate control termination operation is performed through the following steps: obtaining the predicted coding result and the actual coding result corresponding to the first frame to be encoded; obtaining the rate control model at the current moment; updating the rate control model based on the predicted coding result and the actual coding result to obtain the updated rate control model; and performing rate control processing on the remaining second frames to be encoded in the frame sequence based on the updated rate control model.

[0075] The actual encoding result can be the encoding result obtained after frame encoding processing of the first frame to be encoded. The rate control model can be a data model used to control the encoding of the frame to be encoded. Rate control processing can be a process of limiting and constraining various different rates of the frame to be encoded. For example, during the encoding process of the frame to be encoded, the maximum target rate and average rate of the frame to be encoded can be limited.

[0076] In the rate control termination operation, the main thread can obtain the predicted coding result and the actual coding result corresponding to the first frame to be encoded. The actual coding result can be the result obtained after frame encoding processing of the first frame to be encoded. Simultaneously, a pre-configured rate control model can be obtained, which is used to perform rate control processing on the first frame to be encoded. After obtaining the predicted coding result and the actual coding result, the difference between the two can be compared. Based on the comparison result, the rate control model is updated to obtain the updated rate control model. Based on the updated rate control model, rate control processing is performed on the remaining uncoded second frame in the frame sequence to be encoded, which can reduce the result deviation generated during the encoding process of the frame to be encoded.

[0077] In an exemplary embodiment of this disclosure, for step S230, the first frame thread number corresponding to the first frame thread is obtained; the second frame thread number is determined according to the first frame thread number, and the second frame thread is determined based on the second frame thread number; the second thread state corresponding to the second frame thread is obtained; and the video frame type of the video frame to which the second frame thread performs frame encoding processing is determined according to the second thread state; the video frame type includes the encoded frame type.

[0078] Here, the first frame thread number can be the number corresponding to the first frame thread. The second frame thread number can be the number corresponding to the second frame thread. The second frame thread can be the frame thread immediately adjacent to and following the first frame thread number. The second thread status can be whether the second frame thread is in an idle state. The video frame type can be the type determined based on whether a video frame has completed frame encoding processing. The encoded frame type can be the type of video frames that have completed frame encoding processing.

[0079] Continue to refer to Figure 3 In step S308, the second frame thread is obtained. Since frame threads in the frame thread set can be allocated to corresponding frames to be encoded in a round-robin manner during frame encoding, the frame thread whose allocation time is longest from the first frame thread can be obtained as the second frame thread. For example, the first frame thread number is i. After determining the first frame thread number i, the second frame thread number can be determined based on the first frame thread number. The second frame thread number can be the first frame thread number + 1, so the second frame thread number is i+1. After determining the second frame thread number, the second frame thread is obtained based on the second frame thread number, and the corresponding second thread state is determined. Since the currently allocated first frame thread number is i, the frame thread whose allocation time is longest from the first frame thread can be the frame thread located after its number, i.e., the second frame thread is frame thread i+1.

[0080] In step S309, the state of the second thread corresponding to the second frame thread is determined. During the encoding process of the first frame thread on the first frame to be encoded, the specific state of the second frame thread in the frame thread set can be obtained to determine whether it is in an idle state or a non-idle state. In step S310, if the main thread detects that the second thread's state is non-idle, it means that the second frame thread's frame encoding operation on the previously allocated frame to be encoded has not yet ended. At this time, the second frame thread can continue to execute the frame encoding operation to complete the frame encoding operation on the previously allocated frame to be encoded.

[0081] In step S311, if the main thread detects that the second thread is in an idle state, it triggers the bitrate control termination operation for the video frames previously allocated to the second frame thread. If the main thread detects that the second thread is in an idle state, it means that the frame encoding operation of the second frame thread for the previously allocated video frames has ended, that is, the video frames processed by the second frame thread last time have changed from the frame to be encoded type to the encoded frame type, and then the bitrate control termination operation is performed on the encoded frames. In step S312, the bitrate control termination operation for the encoded frames ends. All video frames in the video frame sequence can be processed cyclically through the above steps to obtain the final video encoding result. This avoids redundant waiting in the video encoding process and also avoids the overall encoder speed being affected by the encoding time of a certain frame to be encoded being too long. Therefore, it can effectively improve the performance of the encoder.

[0082] After the second frame thread completes its frame encoding operation on the previously allocated video frame, if the frame encoding process for the first frame to be encoded is not yet complete, the main thread can continue to obtain frame threads from the frame thread set and perform encoding processing on the remaining second frames to be encoded. (Reference) Figure 4 , Figure 4 This is a schematic diagram illustrating the selection of the currently used frame thread from a set of frame threads according to an exemplary embodiment. Determining a frame thread from idle frame threads can be performed through the following steps: for example, a second frame thread can be determined from idle frame threads by polling. For instance, if the first frame thread is numbered i, the second frame thread is numbered i+1, and the second frame thread has completed the frame encoding processing operation for the previous video frame, then the frame encoding operation for the remaining second frame to be encoded can continue based on the second frame thread.

[0083] In summary, the video encoding method disclosed herein obtains a video frame sequence and a bitrate control model for the video to be processed. The video frame sequence includes frames to be encoded and encoded frames. A first frame to be encoded is determined from the video frame sequence, and the encoding processing operation corresponding to the first frame to be encoded is executed. The bitrate control model has a data operation protection lock at a specific step of the encoding processing operation. This data operation protection lock is used to isolate external resources from accessing the bitrate control model. Simultaneously with the encoding processing operation on the first frame to be encoded, a bitrate control termination operation is executed for the encoded frames, resulting in the encoded video of the video to be processed. The bitrate control model also has a data operation protection lock at a specific step of the bitrate control termination operation. On the one hand, adding a data operation protection lock to the bitrate control model at a specific step of the encoding process ensures that only one entity can access the bitrate control model at any given time. Therefore, while ensuring data security, it can reduce redundant waiting during the encoding process, effectively improve resource utilization, and increase encoding speed. On the other hand, for some frames with long encoding times, the frame encoding processes for other frames can be executed in parallel, which can avoid the impact of long waiting times for individual frames on the overall encoding speed and effectively improve encoder performance.

[0084] Figure 5 This is a block diagram illustrating a video encoding apparatus according to an exemplary embodiment. (Refer to...) Figure 5 The video encoding device 500 includes: an information acquisition module 510, a first encoding processing module 520, and a second encoding processing module 530.

[0085] Specifically, the information acquisition module 510 is used to acquire the video frame sequence and bitrate control model of the video to be processed; the video frame sequence includes frames to be encoded and encoded frames; the first encoding processing module 520 is used to determine the first frame to be encoded from the video frame sequence and execute the encoding processing operation corresponding to the first frame to be encoded; the bitrate control model has a data operation protection lock in a specific step of the encoding processing operation, and the data operation protection lock is used to isolate external resources from data access operations of the bitrate control model; the second encoding processing module 530 is used to perform a bitrate control termination operation for the encoded frames while performing encoding processing operation on the first frame to be encoded, to obtain the encoded video of the video to be processed; the bitrate control model has a data operation protection lock in a specific step of the bitrate control termination operation.

[0086] In one exemplary embodiment of this disclosure, the video encoding apparatus 500 further includes an information acquisition module, configured to: determine the encoded frame number of an encoded frame, the encoded frame number being used to determine a first frame to be encoded; acquire a pre-configured bitrate control model, the bitrate control model including bitrate control conditions for encoding the video frame sequence; acquire a pre-configured plurality of frame threads, and generate a frame thread set based on the plurality of frame threads.

[0087] In one exemplary embodiment of this disclosure, the first encoding processing module 520 includes a first encoding processing unit, configured to: perform encoding processing operations including a bitrate control start operation, a frame encoding operation, and a bitrate control end operation; and execute encoding processing operations corresponding to a first frame to be encoded, including: starting a main thread for video encoding; performing a bitrate control start operation based on the main thread; adding a data operation protection lock to the first frame to be encoded based on the main thread while performing the bitrate control start operation; performing a frame encoding operation on the first frame to be encoded to obtain a corresponding frame encoding result; and triggering the main thread to perform a bitrate control end operation corresponding to the first frame to be encoded based on the frame encoding result, thereby generating a corresponding bitrate control end instruction.

[0088] In one exemplary embodiment of this disclosure, the first encoding processing unit includes a frame encoding subunit, configured to: determine a first frame thread corresponding to a first frame to be encoded from a set of frame threads; perform frame encoding operation on the first frame to be encoded based on the first frame thread; and release the first frame thread in response to a frame encoding end instruction corresponding to the frame encoding operation.

[0089] In an exemplary embodiment of this disclosure, the first encoding processing unit includes a rate control termination subunit, configured to: obtain the first thread state corresponding to the first frame thread based on the main thread; control the first frame thread to continue performing frame encoding operations when the first thread state is detected to be non-idle; and add an encoded identifier to the first frame to be encoded in response to the detection that the first thread state is idle, thereby triggering the main thread to perform the rate control termination operation corresponding to the first frame to be encoded.

[0090] In one exemplary embodiment of this disclosure, the first encoding processing module 520 further includes a bitrate control start operation unit, configured to: acquire bitrate parameters for bitrate control processing of the first frame to be encoded; and determine the predictive encoding result corresponding to the first frame to be encoded based on the bitrate parameters.

[0091] In an exemplary embodiment of this disclosure, the first encoding processing module 520 further includes a rate control termination operation unit, configured to: obtain the predicted encoding result and the actual encoding result corresponding to the first frame to be encoded; obtain the rate control model at the current moment; update the rate control model based on the predicted encoding result and the actual encoding result to obtain the updated rate control model; and perform rate control processing on the remaining second frames to be encoded in the frame sequence based on the updated rate control model.

[0092] In one exemplary embodiment of this disclosure, the video encoding apparatus 500 further includes an encoded frame acquisition module, configured to: acquire a first frame thread number corresponding to a first frame thread; determine a second frame thread number based on the first frame thread number, and determine a second frame thread based on the second frame thread number; acquire a second thread state corresponding to the second frame thread; and determine the video frame type of the video frame to be frame encoded by the second frame thread based on the second thread state; the video frame type includes the encoded frame type.

[0093] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0094] The following is for reference. Figure 6 To describe an electronic device 600 according to such an embodiment of the present disclosure. Figure 6 The electronic device 600 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0095] like Figure 6 As shown, the electronic device 600 is presented in the form of a general-purpose computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different system components (including storage unit 620 and processing unit 610), and a display unit 640.

[0096] The storage unit stores program code that can be executed by the processing unit 610, causing the processing unit 610 to perform the steps described in the "Exemplary Methods" section above, according to various exemplary embodiments of this disclosure.

[0097] Storage unit 620 may include readable media in the form of volatile storage units, such as random access memory (RAM) 621 and / or cache memory 622, and may further include read-only memory (ROM) 623.

[0098] Storage unit 620 may include a program / utility 624 having a set (at least one) program module 625, such program module 625 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0099] Bus 630 can represent one or more of several bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0100] Electronic device 600 can also communicate with one or more external devices 670 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 600, and / or with any device that enables electronic device 600 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 650. Furthermore, electronic device 600 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 660. As shown, network adapter 660 communicates with other modules of electronic device 600 via bus 630. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0101] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory including instructions, which can be executed by a processor of the device to perform the video encoding method described above. Optionally, the computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0102] In an exemplary embodiment, a computer program product is also provided, including a computer program / instructions, characterized in that the computer program / instructions, when executed by a processor, implement the video encoding method described in any one of the preceding embodiments.

[0103] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0104] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method of video coding, the method comprising: include: Obtain the video frame sequence and bitrate control model of the video to be processed; the video frame sequence includes frames to be encoded and encoded frames. The first frame to be encoded is determined from the video frame sequence, and the encoding processing operation corresponding to the first frame to be encoded is executed. The bitrate control model has a data operation protection lock in a specific step of the encoding processing operation. The data operation protection lock is used to isolate external resources from data access operations of the bitrate control model. The encoding processing operation includes a bitrate control start operation, a frame encoding operation, and a bitrate control end operation. While performing the encoding processing operation on the first frame to be encoded, a bitrate control termination operation is performed on the encoded frame to obtain the encoded video of the video to be processed. The bitrate control model has a data operation protection lock at a specific step of the bitrate control end operation. The step of performing the encoding processing operation on the first frame to be encoded while simultaneously executing the bitrate control termination operation for the encoded frame includes: obtaining the first thread state corresponding to the first frame thread based on the main thread; controlling the first frame thread to continue executing the frame encoding operation when the first thread state is detected to be non-idle; and adding an encoded flag to the first frame to be encoded in response to the detection that the first thread state is idle, triggering the main thread to execute the bitrate control termination operation corresponding to the first frame to be encoded.

2. The method according to claim 1, characterized in that, The method further includes: The encoded frame number is determined, and the encoded frame number is used to determine the first frame to be encoded; Obtain the pre-configured bitrate control model, which includes bitrate control conditions for encoding the video frame sequence; Obtain multiple pre-configured frame threads, and generate the frame thread set based on the multiple frame threads.

3. The method according to claim 2, characterized in that, The execution of the encoding processing operation corresponding to the first frame to be encoded includes: Start the main thread for video encoding, and perform the bitrate control start operation based on the main thread; While performing the bitrate control start operation, a data operation protection lock is added to the first frame to be encoded based on the main thread; Perform the frame encoding operation on the first frame to be encoded to obtain the corresponding frame encoding result; Based on the frame encoding result, the main thread is triggered to execute the bitrate control termination operation corresponding to the first frame to be encoded, and generate the corresponding bitrate control termination instruction.

4. The method according to claim 3, characterized in that, The step of performing the frame encoding operation on the first frame to be encoded to obtain the corresponding frame encoding result includes: From the set of frame threads, determine the first frame thread corresponding to the first frame to be encoded; Based on the first frame thread, the frame encoding operation is performed on the first frame to be encoded. In response to the frame encoding end instruction corresponding to the frame encoding operation, the first frame thread is released.

5. The method according to claim 3, characterized in that, The rate control operation is initiated through the following steps: Obtain the bitrate parameters for bitrate control processing of the first frame to be encoded; Based on the bitrate parameter, the predictive coding result corresponding to the first frame to be encoded is determined.

6. The method according to claim 1 or 3, characterized in that, The rate control termination operation is performed through the following steps: Obtain the predicted encoding result and the actual encoding result corresponding to the first frame to be encoded; Obtain the current bitrate control model; Based on the predicted coding results and the actual coding results, the rate control model is updated to obtain the updated rate control model. Based on the updated rate control model, rate control processing is performed on the remaining second frame to be encoded in the frame sequence to be encoded.

7. The method according to claim 1, characterized in that, Before performing the bitrate control termination operation for the encoded frame, the method further includes: Get the first frame thread number corresponding to the first frame thread; The thread number of the second frame is determined based on the thread number of the first frame, and the thread of the second frame is determined based on the thread number of the second frame; Obtain the state of the second thread corresponding to the second frame thread; Based on the second thread state, the video frame type of the video frame to be frame encoded by the second frame thread is determined; the video frame type includes the encoded frame type.

8. A video encoding device, characterized in that, include: The information acquisition module is used to acquire the video frame sequence and bitrate control model of the video to be processed; the video frame sequence includes frames to be encoded and encoded frames. The first encoding processing module is used to determine the first frame to be encoded from the video frame sequence and execute the encoding processing operation corresponding to the first frame to be encoded. The bitrate control model has a data operation protection lock in a specific step of the encoding processing operation. The data operation protection lock is used to isolate external resources from data access operations of the bitrate control model. The encoding processing operation includes a bitrate control start operation, a frame encoding operation, and a bitrate control end operation. The second encoding processing module is used to perform the encoding processing operation on the first frame to be encoded, and at the same time execute the bitrate control end operation on the encoded frame to obtain the encoded video of the video to be processed. The bitrate control model has a data operation protection lock at a specific step of the bitrate control end operation. The second encoding processing module is also used to obtain the first thread state corresponding to the first frame thread based on the main thread; If the first thread is detected to be in a non-idle state, the first frame thread is controlled to continue executing the frame encoding operation; in response to the first thread being detected to be in an idle state, an encoded flag is added to the first frame to be encoded, and the main thread is triggered to execute the bitrate control termination operation corresponding to the first frame to be encoded.

9. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the video encoding method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is able to perform the video encoding method as described in any one of claims 1 to 7.

Images