Video processing method, device, electronic device and readable storage medium

Abstract

The present disclosure relates to a video processing method, device, electronic device and readable storage medium, wherein the method comprises: starting from a specified video frame position, executing a target video editing operation on a video to be processed by a first thread, and obtaining each target video frame obtained by executing the target video editing operation; in response to a play instruction, rendering each target video frame by a second thread to display each target video frame. By executing the target video editing operation on each video frame after the specified video frame position in advance, it is ensured that the video frame position where the target video editing operation is being executed is always ahead of the video frame position of the target video frame being displayed, thereby solving the problem of preview jamming and ensuring the user's preview needs; in addition, by decoupling the execution of the target video editing process from the rendering and executing them by different threads, the problem of the rendering thread being blocked by the execution of the target video editing operation in the streaming framework is solved, which is conducive to improving the preview effect.

Description

Technical Field

[0001] The present disclosure relates to the field of Internet technology, and in particular to a video processing method, device, electronic device, and readable storage medium. Background Art

[0002] With the continuous development of Internet technology, a variety of applications (APPs) have been developed. Among them, applications with video editing functions are particularly popular. Currently, applications that provide video editing functions can implement many video editing operations, such as smart cutout. Such video editing operations require corresponding processing of each video frame image.

[0003] In existing technologies, streaming processing frameworks are often used for this type of video editing. For example, when performing intelligent cutout processing, encoding, cutout processing, rendering, and display are performed frame by frame. However, when using streaming processing frameworks for intelligent cutout processing, users are prone to severe display lag during preview playback, severely impacting the user experience. Summary of the Invention

[0004] In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a video processing method, device, electronic device and readable storage medium.

[0005] In a first aspect, the present disclosure provides a video processing method, comprising:

[0006] Starting from a designated video frame position, a first thread performs a target video editing operation on a video frame at the designated video frame position in a to-be-processed video, and pre-performs the target video editing operation on each video frame after the designated video frame position, thereby obtaining target video frames obtained by performing the target video editing operation.

[0007] In response to the play instruction, each of the target video frames is rendered through the second thread to display each of the target video frames; wherein, the position of the video frame on which the target video editing operation is being performed is ahead of the corresponding video frame position of the target video frame being displayed in the video to be processed.

[0008] As a possible implementation, the specified video frame position is determined based on the video frame position located when obtaining the video editing instruction; wherein, the video editing instruction is used to instruct the execution of the target video editing operation on the video to be processed; the video frame position located when obtaining the video editing instruction is a preset video frame position or a video frame position specified by the first jump instruction.

[0009] As a possible implementation, the designated video frame position is determined according to a video frame position designated by a second jump instruction, and the second jump instruction is an instruction following a video editing instruction for performing the target video editing operation on the video to be processed.

[0010] As a possible implementation manner, if the designated video frame position is determined based on the video frame position located when obtaining the video editing instruction; the method further includes: starting from the designated video frame position, executing the target video editing operation on the video frame at the designated video frame position in the to-be-processed video, and pre-executing the target video editing operation on each video frame after the designated video frame position; and obtaining each target video frame obtained by executing the target video editing operation.

[0011] According to the specified video frame position, the video editing processing mode corresponding to the first thread is determined; wherein, the video editing processing mode is a first processing mode or a second processing mode, the first processing mode is a mode for performing the target video editing operation on the key frame, and the second processing mode is a mode for performing the target video editing operation frame by frame.

[0012] As a possible implementation manner, determining the video editing processing mode corresponding to the first thread according to the specified video frame position includes:

[0013] If the designated video frame position is the preset video frame position, determining that the video editing processing mode corresponding to the first thread is the first processing mode;

[0014] If the designated video frame position is the video frame position designated by the first jump instruction, it is determined that the video editing processing mode corresponding to the first thread is the first processing mode or the second processing mode.

[0015] As a possible implementation manner, determining the video editing processing mode corresponding to the first thread according to the specified video frame position includes:

[0016] Determine whether there is a corresponding target video frame at the specified video frame position;

[0017] If the designated video frame position does not have a corresponding target video frame, the video editing processing mode corresponding to the first thread is determined based on the designated video frame position being the preset video frame position or the video frame position specified by the first jump instruction.

[0018] As a possible implementation, the method further includes:

[0019] If a corresponding target video frame exists at the designated video frame position, determining that the video editing processing mode corresponding to the first thread is the second processing mode;

[0020] The method of performing a target video editing operation on a video frame at a designated video frame position in a to-be-processed video by a first thread starting from a designated video frame position and pre-performing the target video editing operation on each video frame after the designated video frame position, and obtaining each target video frame obtained by performing the target video editing operation, includes:

[0021] Starting from the designated video frame position, determine whether a corresponding target video frame exists frame by frame until a first video frame position where no corresponding target video frame exists is determined;

[0022] Through the first thread, according to the second processing mode, starting from the first video frame position where no corresponding target video frame exists, the target video editing operation is performed on the video frame of the video to be processed.

[0023] As a possible implementation manner, if the designated video frame position is determined according to the video frame position designated by the second jump instruction; the method further includes: starting from the designated video frame position, executing the target video editing operation on the video frame at the designated video frame position in the to-be-processed video, and pre-executing the target video editing operation on each video frame after the designated video frame position; and obtaining each target video frame obtained by executing the target video editing operation.

[0024] If the designated video frame position does not have a corresponding target video frame, it is determined that the video editing processing mode corresponding to the first thread is the second processing mode.

[0025] As a possible implementation, the method further includes:

[0026] If there is a corresponding target video frame at the designated video frame position and the first thread is currently performing the target video editing operation on the video to be processed, the first thread is not interrupted, so that the first thread continues to perform the target video editing operation on the video to be processed.

[0027] As a possible implementation, the method further includes:

[0028] In response to the playback instruction, the video editing processing mode is switched to a third processing mode, and through the first thread, the target video editing operation is performed on the video frame of the video to be processed according to the third processing mode; wherein, the third processing mode is a mode for determining the video frame position for performing the target video editing operation according to the playback speed.

[0029] As a possible implementation, in response to the play instruction, switching the video editing processing mode to a third processing mode, and performing the target video editing operation on the video frame of the to-be-processed video according to the third processing mode by the first thread, includes:

[0030] In response to the play instruction, detecting in real time whether there is a next target video frame to be played;

[0031] When it is detected that there is no next target video frame to be played, the first thread performs the target video editing operation on the video frame of the video to be processed according to the third processing mode, starting from the position of the next target video frame to be played.

[0032] As a possible implementation, in response to the play instruction, switching the video editing processing mode to a third processing mode, and performing the target video editing operation on the video frame of the to-be-processed video according to the third processing mode by the first thread, includes:

[0033] In response to the play instruction, through the first thread, starting from the video frame position where the target video editing operation is being performed when the play instruction is obtained, the target video editing operation is performed on the video frame of the video to be processed according to the third processing mode.

[0034] As a possible implementation, in response to the play instruction, switching the video editing processing mode to a third processing mode, and performing the target video editing operation on the video frame of the to-be-processed video according to the third processing mode by the first thread, includes:

[0035] In response to the play instruction, determining a switching position corresponding to the third processing mode according to a preset processing time length and a play speed for performing the target video editing operation on a single video frame and a position of a video frame currently performing the target video editing operation;

[0036] The target video editing operation is performed on the video frame of the video to be processed according to the third processing mode, starting from the switching position, through the first thread.

[0037] As a possible implementation, the method further includes:

[0038] Get the pause playback instruction;

[0039] In response to the pause playback instruction, the second thread is paused from rendering the target video frame; and the first thread continues to perform the target video editing operation on the video frame of the video to be processed according to the third processing mode.

[0040] As a possible implementation, the method further includes:

[0041] After performing the target video editing operation on the last video frame of the video to be processed, performing the target video editing operation on the video frames of the video to be processed according to a fourth processing mode, starting from the position of the starting video frame of the video to be processed, through the first thread;

[0042] The fourth processing mode is a mode of sequentially performing the target video editing operation on video frames in the video to be processed that do not have corresponding target video frames.

[0043] In a second aspect, the present disclosure provides a video processing device, comprising:

[0044] a first processing module, configured to, through a first thread, start from a specified video frame position, perform a target video editing operation on the video frame at the specified video frame position in the video to be processed, pre-perform the target video editing operation on each video frame after the specified video frame position, and obtain each target video frame obtained by performing the target video editing operation;

[0045] The second processing module is used to render each of the target video frames through the second thread in response to the playback instruction to display each of the target video frames; wherein the position of the video frame on which the target video editing operation is being performed is ahead of the video frame position corresponding to the target video frame being displayed in the video to be processed.

[0046] In a third aspect, the present disclosure provides an electronic device, comprising: a memory and a processor;

[0047] The memory is configured to store computer program instructions;

[0048] The processor is configured to execute the computer program instructions so that the electronic device implements the video processing method as described in any one of the first aspects.

[0049] In a fourth aspect, the present disclosure provides a readable storage medium, comprising: computer program instructions; when the computer program instructions are executed by at least one processor of an electronic device, the electronic device implements the video processing method as described in any one of the first aspects.

[0050] In a fifth aspect, the present disclosure provides a computer program product, which, when executed by a computer, enables the computer to implement the video processing method as described in any one of the first aspects.

[0051] The present disclosure provides a video processing method, device, electronic device and readable storage medium, wherein the method comprises: starting from a specified video frame position, executing a target video editing operation on a video to be processed by a first thread, obtaining each target video frame obtained by executing the target video editing operation; and rendering each target video frame by a second thread in response to a play instruction to display each target video frame. The present disclosure performs the target video editing operation on each video frame after the specified video frame position in advance, thereby ensuring that the video frame position of the target video editing operation being executed is always ahead of the video frame position of the target video frame being displayed in the video frame to be processed, thereby solving the problem of preview jamming during the process of executing the target video editing operation on the video to be processed, thereby ensuring the user's preview needs; in addition, the present disclosure solves the problem of blocking the rendering thread due to the execution of the target video editing operation in the streaming framework by decoupling the execution of video editing processing from rendering and executing them by different threads, which is conducive to improving the preview effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0053] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0054] Figure 1 A flowchart of a video processing method provided by an embodiment of the present disclosure;

[0055] Figure 2 A flowchart of a video processing method provided by another embodiment of the present disclosure;

[0056] Figure 3 A flowchart of a video processing method provided by another embodiment of the present disclosure;

[0057] Figure 4 A flowchart of a video processing method provided by another embodiment of the present disclosure;

[0058] Figure 5 A flowchart of a video processing method provided by another embodiment of the present disclosure;

[0059] Figure 6 A flowchart of a video processing method provided by another embodiment of the present disclosure;

[0060] Figure 7 A flowchart of a video processing method provided by another embodiment of the present disclosure;

[0061] Figure 8 A flowchart of a video processing method provided by another embodiment of the present disclosure;

[0062] Figure 9 A flowchart of a video processing method provided by another embodiment of the present disclosure;

[0063] Figure 10 A schematic diagram of a framework of a video processing device provided by an embodiment of the present disclosure;

[0064] Figure 11 An embodiment of the present disclosure provides Figure 10 A schematic diagram of the life cycle of each module in the embodiment shown;

[0065] Figure 12 A schematic diagram of the structure of a video processing device provided in one embodiment of the present disclosure;

[0066] Figure 13 A schematic structural diagram of an electronic device provided in one embodiment of the present disclosure. DETAILED DESCRIPTION

[0067] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features therein can be combined with each other in the absence of conflict.

[0068] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0069] The video processing method provided in the present disclosure can be executed by a video processing device, wherein the video processing device can be implemented by any software and / or hardware. For example, the video processing device can be a tablet computer, a mobile phone (such as a foldable screen mobile phone, a large screen mobile phone, etc.), a wearable device, an in-vehicle device, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), a smart TV, a smart screen, a high-definition TV, a 4K TV, a smart speaker, a smart projector, and other Internet of Things (IoT) devices. The present disclosure does not impose any restrictions on the specific type of electronic device.

[0070] It should be noted that the video processing method provided by the present disclosure can be applied to at least the following scenarios:

[0071] 1. It is necessary to perform video editing operations on each video frame of the video to be processed, generate and save the corresponding target video frame (i.e., the video editing processing result) for rendering and display.

[0072] 2. Video editing operations are time-consuming, but require better preview and playback effects.

[0073] 3. Video editing operations require real-time follow-up and alignment with user input commands, such as user input jump commands (also called seek commands), update commands (also called refresh commands), play commands, variable speed play commands, pause play commands, etc.

[0074] For scenes with the above characteristics, the video editing and processing method provided by the present disclosure can be adopted to ensure the user preview effect.

[0075] In order to more clearly introduce the video processing method provided by the present disclosure, in the following embodiments, a video editing application (hereinafter referred to as: application) installed in an electronic device is taken as an example to explain the video processing method provided by the present disclosure in detail.

[0076] Figure 1 This is a flow chart of a video processing method provided by an embodiment of the present disclosure. Figure 1 As shown, the method provided in this embodiment includes:

[0077] S101. Starting from a specified video frame position, a first thread performs a target video editing operation on a video frame at the specified video frame position in a video to be processed, and pre-performs a target video editing operation on each video frame after the specified video frame position, to obtain target video frames obtained by performing the target video editing operation.

[0078] The video to be processed is the material on which the target video editing operation is to be performed. This disclosure does not limit the length, storage format, resolution, video content, and acquisition method of the video to be processed.

[0079] The first thread is a thread used to perform a target video editing operation on the processed video. The first thread may also be referred to as an algorithm thread. The algorithm thread may call an algorithm instance corresponding to the target video editing operation to perform the target video editing operation on the processed video. Furthermore, this disclosure does not limit the method for determining the first thread.

[0080] Among them, the present disclosure does not limit the type of target video editing operation. For example, the target video editing operation may include but is not limited to: smart cutout, adding stickers, filters, etc.

[0081] The designated video frame position may be the position of any video frame in the video to be processed. For example, the designated video frame position may be the video frame position located when obtaining the video editing instruction, or may be the video frame position designated by the user through a trigger operation.

[0082] The video editing instruction is used to instruct to perform a target video editing operation on each video frame of the video to be processed. The present disclosure does not limit the implementation method for obtaining the video editing instruction. For example, the user can input the video editing instruction to the application by operating the corresponding control provided by the application.

[0083] The following uses different scenarios as examples to illustrate how to implement this step:

[0084] In one possible implementation, a user may select a video to be processed through a material selection page displayed by an application, and import the video into the application for video editing. The application may display a video editing page on the user interface, wherein the video editing page may include controls corresponding to a variety of different video editing operations, including: a target control corresponding to a target video editing operation (such as smart cutout); when the application receives a user trigger operation (such as a click operation) on the target control, it generates a video editing instruction for instructing to perform smart cutout on each video frame of the video to be processed, and the application responds to the video editing instruction and performs the target video editing operation on the video to be processed starting from the specified video frame position.

[0085] In this scenario, when importing a video to be processed, the application can default to a preset video frame position. Since the user does not subsequently input any triggering operations, the positioned video frame position will not change. When receiving video editing instructions, the positioned video frame position will still be the preset video frame position, that is, the specified video frame position is the preset video frame position. The preset video frame position can be any video frame of the video to be processed, for example, the preset video frame position is the position of the starting video frame of the video to be processed.

[0086] Another possible implementation method is that the user can select the video to be processed through the material selection page displayed by the application, and import it into the application for video editing. The application can display a video editing page on the user interface, wherein the video editing page can include controls corresponding to a variety of different video editing operations, including: a target control corresponding to a target video editing operation (such as smart cutout); when the application receives a trigger operation (such as a click operation) from the user for the target control, it generates a video editing instruction for instructing to perform smart cutout on each video frame of the video to be processed, and the application responds to the video editing instruction and performs the target video editing operation on the video to be processed starting from the specified video frame position. Before the user inputs the trigger operation for the target control, he or she can also input a first jump instruction for jumping the positioned position from a preset video frame position to the video frame position indicated by the first jump instruction.

[0087] In this scenario, the designated video frame position is the video frame position indicated by the first jump instruction. It should be noted that the user can input one or more first jump instructions, and the video frame position indicated by the last jump instruction is usually used as the designated video frame position.

[0088] Another possible implementation method, based on the first possible implementation method mentioned above, the first thread has started to execute the target video editing operation from a preset video frame position. During the execution of the target video editing operation, the user can input a second jump instruction. The application then responds to the second jump instruction, and the first thread needs to execute the target video editing operation from the video frame position indicated by the second jump instruction.

[0089] In this scenario, the specified video frame position is the video frame position indicated by the second jump instruction. The user can enter multiple second jump instructions, and the first thread needs to respond to each second jump instruction respectively and perform the target video editing operation starting from the video frame position indicated by the second jump instruction.

[0090] S102. In response to a play instruction, each of the target video frames is rendered through a second thread to display each of the target video frames; wherein the position of the video frame on which the target video editing operation is being performed is ahead of the position of the video frame corresponding to the target video frame being displayed in the video to be processed.

[0091] While the first thread performs the target video editing operation on the video to be processed, the application supports the user to preview each target video frame obtained by performing the target video editing operation, that is, the application supports the user to preview and play each target video frame.

[0092] In a possible implementation, when the application receives a play instruction, in response to the play instruction, the second thread renders each target video frame according to the play speed to play each target video frame.

[0093] It should be noted that while the first thread is performing a target video editing operation on the video being processed, when the target video editing operation is performed on the video frame at the specified video frame position to obtain the target video frame corresponding to the specified video frame position, the second thread can automatically render the target video frame corresponding to the specified video frame position to display the target video frame corresponding to the specified video frame position. This ensures that the user can preview the processing effect corresponding to the specified video frame position before the application receives a play instruction.

[0094] Another possible implementation method is that during the process of the first thread performing a target video editing operation on the processed video, when the target video editing operation is performed on the video frame at the specified video frame position to obtain the target video frame corresponding to the specified video frame position, the second thread can automatically render each target video frame starting from the specified video frame position, that is, without the user triggering a play instruction.

[0095] Regardless of which of the above methods is adopted, the method provided in this embodiment ensures that there is a target video frame on which the target video editing operation is currently being performed for preview by ensuring that the position of the video frame on which the target video editing operation is currently being performed is always ahead of the position of the target video frame currently being displayed in the video to be processed, thereby ensuring that there is a target video frame on which the target video editing operation has been performed for preview. Therefore, it can solve the problem of user preview freeze.

[0096] In order to make this solution clearer, an example is used here to illustrate the meaning of the video frame position currently performing the target video editing operation being ahead of the position of the target video frame currently being displayed in the video to be processed: assuming that there are 100 frames in the video to be processed, they are frame 1 to frame 100 in chronological order, the target video frame corresponding to frame 20 is currently being displayed, and the video frame position currently performing the target video editing operation is frame 30, which means that the video frame position currently performing the target video editing operation is ahead of the position of the target video frame currently being displayed in the video to be processed.

[0097] The method provided in this embodiment starts from a specified video frame position by a first thread, performs a target video editing operation on the video to be processed, and obtains each target video frame obtained by performing the target video editing operation; in response to a play instruction, each target video frame is rendered by a second thread to display each target video frame. The present disclosure performs the target video editing operation on each video frame after the specified video frame position in advance, ensuring that the video frame position of the target video editing operation being performed is always ahead of the video frame position of the target video frame being displayed in the video frame to be processed, thereby solving the problem of preview jamming during the process of performing the target video editing operation on the video to be processed, and ensuring the user's preview needs; in addition, since the execution of the target video editing operation is relatively time-consuming, if the time consumed by the algorithm processing does not match the time consumed by the rendering thread, it is easy to block the rendering thread. Therefore, the present disclosure solves the problem of blocking the rendering thread due to the execution of the target video editing operation in the streaming framework by decoupling the execution of video editing processing and rendering, and performing them by different threads, which is conducive to improving the preview effect.

[0098] exist Figure 1 Based on the illustrated embodiment, the video processing method provided by the present disclosure can provide at least the following video editing processing modes. When the application receives a video editing instruction, the video editing processing mode corresponding to the first thread can be determined according to a pre-set strategy. The pre-set strategy can be related to, but is not limited to, one or more factors such as the current video frame position of the video to be processed, the overall progress of the target video editing operation being executed on the video to be processed, the playback state (paused or playing), and the user's triggering operation.

[0099] The application supports at least the following four video editing processing modes: a first processing mode, a second processing mode, a third processing mode, and a fourth processing mode.

[0100] The first processing mode is a mode for performing target video editing operations on key frames of the video to be processed. The first processing mode may also be called a pre-processing mode, an anchor mode, an ancher mode, or other names.

[0101] The second processing mode is a mode for performing target video editing operations frame by frame. The second processing mode may also be called a jump processing mode, a seek mode, or other names.

[0102] The third processing mode is a mode for determining the video frame position for performing the target video editing operation based on the playback progress. The third processing mode may also be called a playback processing mode, a playback mode, or other names.

[0103] The fourth processing mode sequentially performs the target video editing operation on video frames in the processing video that do not have a corresponding target video frame. Specifically, the fourth processing mode sequentially performs the target video editing operation on video frames that have not yet been subjected to the target video editing operation, starting from the first video frame to the last video frame in the processing video. The fourth processing mode may also be referred to as a self-propagating processing mode, adaptive mode, or other similar names.

[0104] The application program may determine the video editing processing mode corresponding to the first thread executing the target video editing operation in different scenarios based on the above-mentioned pre-set strategy.

[0105] The following describes in detail the video editing processing mode in which the first thread performs the target video editing operation on the processed video in different scenarios through several specific embodiments.

[0106] Figure 2 This is a flow chart of a video processing method provided by an embodiment of the present disclosure. Figure 2 As shown, the method provided in this embodiment includes:

[0107] S201. Determine a video editing processing mode corresponding to a first thread according to a specified video frame position, where the video editing processing mode is a first processing mode or a second processing mode.

[0108] Combine Figure 1 In the illustrated embodiment, the designated video frame position may be a preset video frame position, a video frame position indicated by the first jump instruction, or a video frame position indicated by the second jump instruction. In different situations, the video editing processing mode corresponding to the first thread may vary.

[0109] The first processing mode performs targeted video editing operations on key frames (i.e., I-frames). I-frames, also known as intra pictures, are important frames in inter-frame compression coding. During the encoding process, some video frame sequences are compressed into I-frames, some into P-frames, and some into B-frames. During decoding, the complete image can be reconstructed based solely on the data from the I-frames, without reference to data from other video frames.

[0110] The second processing mode is a mode for performing target video editing operations on the video to be processed frame by frame. The second processing mode is a video editing processing mode provided by the present disclosure for responding to a seek instruction input by a user.

[0111] In a possible implementation, if the designated video frame position is a preset video frame position (such as the position of the starting video frame of the video to be processed), the video editing processing mode corresponding to the first thread is determined to be the first processing mode.

[0112] By performing target video editing operations on the key frames of the video to be processed, when the user previews the special effects processing results, the complete image can be reconstructed based on the target video frames corresponding to the key frames. Therefore, the user preview effect can be better guaranteed.

[0113] In another possible implementation, if the specified video frame position is the video frame position indicated by the jump instruction, the video editing processing mode corresponding to the first thread is the first processing mode or the second processing mode. For example, if the specified video frame position is the video frame position indicated by the first jump instruction, the video editing processing mode corresponding to the first thread can be the first processing mode or the second processing mode; if the specified video frame position is the video frame position indicated by the first jump instruction, the video editing processing mode corresponding to the first thread can be the second processing mode.

[0114] S202. Using the first thread, in accordance with the determined video editing processing mode, starting from a specified video frame position, perform a target video editing operation on the video frame at the specified video frame position in the video to be processed, and pre-perform the target video editing operation on each video frame after the specified video frame position, and obtain each target video frame obtained by performing the target video editing operation.

[0115] If the determined video editing processing mode is the first processing mode, the application starts from the specified video frame position, performs the target video editing operation on each key frame in the processed video in turn, and stores the target video frame obtained by performing the target video editing operation on each key frame.

[0116] If the determined video editing processing mode is the second processing mode, the application starts from the specified video frame position, performs the target video editing operation on the video to be processed frame by frame, and stores the target video frame obtained by performing the target video editing operation on each video frame.

[0117] S203. In response to the play instruction, each of the target video frames is rendered through the second thread to display each of the target video frames; wherein the position of the video frame on which the target video editing operation is being performed is ahead of the position of the video frame corresponding to the target video frame being displayed in the video to be processed.

[0118] In this embodiment, step S203 and Figure 1 Step S102 in the embodiment shown is similar, and can be referred to Figure 1 For the sake of brevity, the detailed description of the illustrated embodiment will not be repeated here.

[0119] The method provided by this embodiment, in a video editing scenario, obtains and analyzes a specified video frame position to determine the video editing processing mode corresponding to the first thread, and performs a target video editing operation on the corresponding video frame in the video to be processed according to the determined video editing processing mode, so as to ensure that the video frame position for which the target video editing operation is being performed is always ahead of the video frame position of the target video frame being displayed in the video frame to be processed, thereby solving the problem of preview jamming that may occur during the process of performing the target video editing operation on the video to be processed, and ensuring the user's preview needs; in addition, since performing the target video editing operation is relatively time-consuming, if the time consumed by the algorithm processing does not match the time consumed by the rendering thread, the rendering thread is likely to be blocked. Therefore, the present disclosure solves the problem of blocking the rendering thread due to the execution of the target video editing operation in the streaming framework by decoupling the execution of video editing processing from rendering and executing them by different threads, which is conducive to improving the preview effect.

[0120] Figure 3 This is a flow chart of a video processing method provided by another embodiment of the present disclosure. Figure 3 As shown, the method of this embodiment includes:

[0121] exist Figure 2 Based on the embodiment shown, Figure 2 S201 in the illustrated embodiment can be implemented through S301 to S306 in this embodiment.

[0122] S301: Determine whether there is a corresponding target video frame at a specified video frame position.

[0123] If the designated video frame position does not have a corresponding target video frame, then execute S302 ; if the designated video frame position has a corresponding target video frame, then execute S303 .

[0124] S302: Determine a video editing processing mode corresponding to the first thread according to the specified video frame position.

[0125] After the video editing processing mode corresponding to the first thread is determined, S305 is executed.

[0126] S303: Determine whether the current first thread performs a target video editing operation on the video to be processed.

[0127] If the first thread is not currently performing the target video editing operation on the video to be processed, then S304 and S305 are executed. If the first thread is currently performing the target video editing operation on the video to be processed, then S306 is executed.

[0128] S304: Determine that the video editing processing mode is the second processing mode.

[0129] S305 : Execute a target video editing operation on the video to be processed according to the determined video editing processing mode through the first thread, and obtain target video frames on which the target video editing operation is executed.

[0130] S306: Continue to perform the target video editing operation on the video to be processed through the first thread.

[0131] In actual scenarios, when the first thread starts to execute the target video editing operation according to a video editing processing mode, or switches the video editing processing mode, it is necessary to update the decoder, which consumes a lot of computing resources. Therefore, the method provided in this embodiment determines whether to switch the video editing processing mode by analyzing whether there is a corresponding target video frame at a specified video frame position and whether there is a process currently executing the target video editing operation on the video to be processed (that is, whether the first thread is currently executing the target video editing operation on the video to be processed), thereby reducing the waste of computing resources caused by frequent decoder updates and improving the processing efficiency of executing the target video editing operation.

[0132] Among them, the reason for analyzing whether there is a corresponding target video frame at the specified video frame position is that in some cases, the target video editing operation may have been performed on the video to be processed before, so some or all of the video frames in the video to be processed have corresponding target video frames. Taking the target video editing operation as smart cutout as an example, it is assumed that after the user imports the video to be processed, he generates a video editing instruction by operating the control corresponding to the smart cutout; the application performs smart cutout on the video to be processed according to the video editing instruction; when the smart cutout is not fully completed, the user enters the undo instruction to cancel the smart cutout of the video to be processed. Since the video to be processed has not completed the smart cutout when the smart cutout is canceled, some of the video frames have corresponding target video frames (i.e., the cutout result).

[0133] In one possible case, if there is no corresponding target video frame at the specified video frame position, regardless of whether the current first thread is performing the target video editing operation on the video to be processed, it is necessary to determine the video editing processing mode corresponding to the first thread based on the specified video frame position, and the first thread starts from the specified video frame position according to the determined video editing processing mode and performs the target video editing operation on the video to be processed.

[0134] The implementation method of determining the video editing processing mode corresponding to the first thread according to the specified video frame position can be referred to the above description and will not be repeated here.

[0135] In another possible situation, if there is a corresponding target video frame at the specified video frame position and there is currently no process for executing the target special effects on the video to be processed, since video encoding is performed in the order of the video frames, it is possible to start from the next frame of the specified video frame position and determine frame by frame whether there is a corresponding target video frame until the first video frame position where there is no corresponding target video frame is determined, and through the first thread, according to the first processing mode or the second processing mode, starting from the first video frame position where there is no corresponding target video frame, perform the target video editing operation on the video to be processed.

[0136] In this scenario, the first thread adopts the first processing mode or the second processing mode, which can be determined according to whether the designated video frame position is a preset video frame position or a video frame position indicated by the first jump instruction.

[0137] In another possible situation, if the specified video frame position has a corresponding target video frame, and assuming that there is currently a process that performs the target special effects on the video to be processed, the process can continue to be executed, so there is no need to update the decoder, reducing the resource consumption caused by updating the decoder due to switching the special effects processing mode. For example, when the above-mentioned specified video frame position is the video frame position indicated by the second jump instruction, the first thread may be performing the target video editing operation on the video to be processed according to a video editing processing mode. If the video frame position indicated by the second jump instruction has a corresponding target video frame, there is no need to switch the video editing processing mode, that is, the first thread will not be interrupted.

[0138] Among them, if the specified video frame position is the preset video frame position or the video frame position indicated by the first jump instruction, the video editing processing mode can be determined by referring to the method described above; if the specified video frame position is the video frame position indicated by the second jump instruction, the video editing processing mode can be determined as the second processing mode.

[0139] S307. In response to the play instruction, each target video frame is rendered through the second thread to display each target video frame; wherein the position of the video frame on which the target video editing operation is being performed is ahead of the video frame position corresponding to the target video frame being displayed in the video to be processed.

[0140] Whether based on S305 or S306, during the process of the first thread performing the target video editing operation on the processed video, the application can respond to the play instruction and play it through the second thread. Figure 2 Step S203 in the embodiment shown is similar, and can be referred to Figure 2 For the sake of brevity, the detailed description of the illustrated embodiment will not be repeated here.

[0141] The method provided in this embodiment determines whether to switch the video editing processing mode in a video editing scenario by analyzing whether there is a corresponding target video frame at a specified video frame position and whether there is currently a process performing the target video editing operation on the video to be processed. This not only meets the user's preview needs and solves the problem of user preview lag, but also reduces the resource waste caused by frequent decoder updates and improves the processing efficiency of executing target video editing.

[0142] Figure 4 This is a flow chart of a video processing method provided by another embodiment of the present disclosure. Figure 4 As shown, the method provided in this embodiment includes:

[0143] S401. Starting from a designated video frame position, a first thread performs a target video editing operation on a video frame at the designated video frame position in a video to be processed, and pre-performs a target video editing operation on each video frame after the designated video frame position, to obtain target video frames obtained by performing the target video editing operation.

[0144] S402. In response to a play instruction, each of the target video frames is rendered through a second thread to display each of the target video frames; wherein the position of the video frame on which the target video editing operation is being performed is ahead of the position of the video frame corresponding to the target video frame being displayed in the video to be processed.

[0145] In this embodiment, steps S401 and S402 are respectively Figure 1 The embodiments S101 and S102 shown are similar, and can be referred to Figure 1 For the sake of brevity, the detailed description of the illustrated embodiment will not be repeated here.

[0146] S403: In response to the play instruction, switch the video editing processing mode to a third processing mode, and perform a target video editing operation on the video to be processed according to the third processing mode through the first thread.

[0147] The third processing mode is a mode for determining the video frame position for executing the target special effect according to the preview playback speed.

[0148] For example, assuming that there are 100 frames of video to be processed, in steps S401 to S402, the target video editing operation has been performed on the 1st to 10th video frames based on the determined video editing processing mode to obtain the corresponding target video frames. When the application receives the play instruction, the second thread starts to render the target video frames corresponding to the 1st to 10th video frames in sequence, so as to play the target video frames corresponding to the 1st to 10th video frames respectively according to the set play speed. Assuming that the current playback reaches the 10th frame, it is detected that the next video frame to be played (i.e., the 11th frame) does not have a corresponding target video frame. If the time taken to perform the target video editing operation is greater than the play speed, if the target video editing operation is performed on the 11th video frame in the order of the video frames, the playback requirements cannot be met, which will inevitably lead to serious playback freezes or even screen distortion. Therefore, the video editing processing mode is switched to the third processing mode, so that the first thread determines the video frame position for executing the target video editing operation according to the playback speed, so as to ensure that the video frame position for executing the target video editing operation is always ahead of the corresponding video frame position of the target video frame being displayed in the video frame to be processed. By performing the target video editing operation on the video frame that is ahead of the preview playback progress, the needs of preview playback are met, and problems such as severe preview playback freezes and screen distortion are solved.

[0149] It should be noted that when the video editing processing mode is the third processing mode, it is possible to dynamically determine which video frames ahead of the playback progress to perform the target video editing operation on based on one or more factors, such as the preview playback speed, the processing time for a preset single video frame to perform the target video editing operation (i.e., the average time taken for the target video editing operation), the position of the video frame currently performing the target video editing operation, the video frame on which the target video editing operation has been performed, etc.

[0150] Continuing with the example in the previous paragraph, for example, according to the third processing mode, it is determined that the target video editing operation is performed on the 15th video frame through the first thread, but the target special effects have been performed on the 15th frame, then the target video editing operation can be performed on the 16th video frame in the order of the video frames (of course, it is not limited to the 16th frame, it can also be other subsequent video frames with a certain interval from the 15th frame, such as the 18th frame), and if it is determined that there is no corresponding target video frame for the 16th video frame, then the target video editing operation is performed on the 16th video frame. After performing the target video editing operation on the 16th video frame, continue to flexibly determine the video frame position on which the target video editing operation is to be performed according to the third processing mode.

[0151] From the above introduction and examples of the third processing mode, it can be seen that the third processing mode is a video editing processing mode in response to the playback instruction. During the playback of the target video frame, the third processing mode can ensure that the position of the video frame on which the target video editing operation is being performed is always ahead of the corresponding video frame position of the target video frame being played in the video to be processed, that is, ahead of the playback progress.

[0152] In this embodiment, the video editing processing mode is switched to the third processing mode, and the switching timing can be determined by one or more of the following factors.

[0153] 1. The randomness of user operations. For example, a user might enter the play command and the pause command repeatedly. 2. Whether there is a process currently performing the target video editing operation on the video to be processed. 3. The target video frame obtained by performing the target video editing operation.

[0154] The influence of the above factors is illustrated by way of example:

[0155] Assuming that there are 100 frames of video to be processed, in steps S401 to S402, target video editing operations have been performed on the 1st to 10th video frames based on the determined video editing processing mode, and target video frames corresponding to the 1st to 10th video frames are obtained respectively.

[0156] Scenario (1): Assume that the application receives a play command, updates the decoder, and immediately switches the video editing processing mode to the third processing mode; and starts playing the target video frame from the first frame. Then, the application receives a pause command. Because the interval between the play command and the pause command is short, the second thread may not have rendered to the tenth frame. Interrupting the current video editing processing mode, that is, interrupting the first thread, re-updating the decoder, and switching the video editing processing mode will result in a large consumption of computing resources, which will lead to a decrease in video processing efficiency.

[0157] Scenario (2): During playback, it is detected in real time whether there is a corresponding target video frame at the position of the next video frame to be played. If it is determined that there is no corresponding target video frame at the position of the next video frame to be played, the video editing processing mode is switched. Then, during the playback of the 1st to 10th frames, the target video editing operation can continue to be performed on the processed video according to the original video editing processing mode, that is, the process of executing the target video editing operation will not be interrupted, which can improve the utilization of computing resources.

[0158] Furthermore, based on scenario (2), if the application receives a pause instruction while the playback progress has not reached the target video editing operation, for example, the application receives a pause instruction while playing the 8th frame. While playing the 1st to 8th frames, the application also performs the target video editing operation on the 11th and 12th frames using the original video editing processing mode.

[0159] In the above situation (2), although the application receives the play instruction and the pause instruction, since the corresponding target video frame exists in the part to be played, the smoothness of the playback can be guaranteed without switching the video editing processing mode corresponding to the first thread, which can also reduce the large computing resource consumption caused by switching the video editing processing mode; on the other hand, during the playback process, the first thread also performs the target video editing operation on a part of the video frames according to the original video editing processing mode, thereby improving the video processing efficiency.

[0160] Based on the above description, it can be seen that in this step, the application responds to the play instruction and can determine the switching timing of the video editing processing mode based on one or more of the above factors.

[0161] The method provided in this embodiment receives a play instruction during the process of executing the target video editing operation on the video to be processed by the first thread. In response to the play instruction, the video editing processing mode can be switched to a third processing mode to execute the target video editing operation on the video frame of the video to be processed. Through the third processing mode, it can be ensured that the position of the video frame executing the target video editing operation during the playback process is always ahead of the playback progress, thereby solving the problems of severe freezes and screen distortion during user preview playback, improving video processing efficiency, and helping to enhance user experience.

[0162] Optionally, in Figure 4 Based on the embodiment shown, the present invention further includes:

[0163] S404: Receive a pause playback instruction.

[0164] S405: In response to the pause playback instruction, the second thread pauses playback, and the first thread continues to perform the target video editing operation on the video frames of the video to be processed according to the third processing mode.

[0165] Specifically, when the application receives the pause playback instruction, the second thread stops rendering the target video frame, that is, pauses playback of the target video frame.

[0166] Since a process already executing the target special effects on the video being processed is in progress when the pause playback instruction is received, this process can be left uninterrupted. That is, the first thread continues to execute the target video editing operation on the video being processed, i.e., continues to execute the target video editing operation on the video frames of the video being processed according to the third processing mode. This allows the computing resources of the electronic device to be effectively utilized to execute video editing, thereby improving computing resource utilization and thus video processing efficiency.

[0167] Below through Figures 5 to 7 In the illustrated embodiment, several possible implementations corresponding to step S403 are introduced.

[0168] Figure 5 This is a flow chart of a video processing method provided by another embodiment of the present disclosure. Figure 5 As shown, the method of this embodiment includes:

[0169] S501. Starting from a designated video frame position, a first thread executes a target video editing operation on a video frame at the designated video frame position in a video to be processed, and pre-executes a target video editing operation on each video frame after the designated video frame position, to obtain target video frames obtained by executing the target video editing operation.

[0170] S502. In response to a play instruction, each of the target video frames is rendered through a second thread to display each of the target video frames; wherein the position of the video frame on which the target video editing operation is being performed is ahead of the position of the video frame corresponding to the target video frame being displayed in the video to be processed.

[0171] In this embodiment, steps S501 and S502 are respectively Figure 4 The embodiments S401 and S402 are similar, and can be referred to Figure 4 For the sake of brevity, the detailed description of the illustrated embodiment will not be repeated here.

[0172] S503 . In response to the play instruction, starting from the video frame position where the target video editing operation is being performed when the preview play instruction is obtained, perform the target video editing operation on the video frame of the video to be processed according to the third processing mode.

[0173] In this step, when obtaining the playback instruction, the first thread may be performing the target video editing operation on a certain video frame. In this case, the position of the video frame where the video editing processing mode is switched is the position of the video frame where the target video editing operation is currently being performed; the application may also have just completed the target video editing operation on a certain video frame, but has not yet started to perform the target video editing operation on the next video frame. In this case, the position of the video frame where the video editing processing mode is switched is the position of the next video frame where the target video editing operation is to be performed.

[0174] Among them, when the application obtains the playback instruction, it can immediately switch the video editing processing mode corresponding to the first thread to the third processing mode, and perform the target video editing operation on the video frame to be processed according to the third processing mode from the position of the video frame where the video editing processing mode is determined to be switched.

[0175] It should be noted that the execution order of S502 and S503 is not particular.

[0176] The method provided in this embodiment obtains a play instruction during the process of executing the target video editing operation on the video to be processed; in response to the play instruction, the first thread performs the target video editing operation on the video frame of the video to be processed according to the third processing mode, starting from the video frame position where the target video editing operation was being executed when the play instruction was obtained. The implementation method provided by this embodiment has simple logic and does not require complex judgment. In addition, through the third processing mode, it can be ensured that the video frame position where the target video editing operation is being executed during the playback process is always ahead of the playback progress, thereby solving problems such as severe freezes and screen distortion during user preview playback, improving video processing efficiency, and helping to enhance user experience.

[0177] Optionally, in Figure 5 Based on the embodiment shown, the present invention further includes:

[0178] S504: Get a pause playback instruction.

[0179] S505: In response to the pause playback instruction, the second thread pauses playback, and the first thread continues to perform the target video editing operation on the video frames of the video to be processed according to the third processing mode.

[0180] Specifically, when the application receives the pause playback instruction, the second thread stops rendering the target video frame, that is, pauses playback of the target video frame.

[0181] Since a process already executing the target special effects on the video being processed is in progress when the pause playback instruction is received, this process can be left uninterrupted. That is, the first thread continues to execute the target video editing operation on the video being processed, i.e., continues to execute the target video editing operation on the video frames of the video being processed according to the third processing mode. This allows the computing resources of the electronic device to be effectively utilized to execute video editing, thereby improving computing resource utilization and thus video processing efficiency.

[0182] Figure 6 This is a flow chart of a video processing method provided by another embodiment of the present disclosure. Figure 6 As shown, the method provided in this embodiment includes:

[0183] S601. Starting from a specified video frame position, a first thread performs a target video editing operation on a video frame at the specified video frame position in a video to be processed, and pre-performs a target video editing operation on each video frame after the specified video frame position, to obtain target video frames obtained by performing the target video editing operation.

[0184] S602. In response to a play instruction, each of the target video frames is rendered through a second thread to display each of the target video frames; wherein the position of the video frame on which the target video editing operation is being performed is ahead of the position of the video frame corresponding to the target video frame being displayed in the video to be processed.

[0185] In this embodiment, steps S601 and S602 are respectively Figure 4 The embodiments S401 and S402 are similar, and can be referred to Figure 4 For the sake of brevity, the detailed description of the illustrated embodiment will not be repeated here.

[0186] S603: In response to the play instruction, determine a switching position corresponding to the third processing mode according to a preset processing time and play speed for executing the target video editing operation on a single video frame and a position of the video frame currently executing the target video editing operation.

[0187] For example, the maximum duration for which the current video editing processing mode can continue to be executed can be determined based on the position and playback speed of the video frame currently performing the target video editing operation. Then, based on the determined maximum duration and the preset processing duration for performing the target video editing operation on a single video frame, the number S of video frames that can be processed using the current video editing processing mode within the maximum duration is determined. Then, the switching position of the video editing processing mode is determined from the S consecutive video frames starting from the position of the video frame currently performing the target video editing operation. The switching position of the video editing processing mode can be any video frame from the S consecutive video frames starting from the position of the video frame currently performing the target video editing operation.

[0188] In some possible implementations, in order to reduce errors in switching video editing processing modes, such as switching too late so that the processing progress of the target video editing operation cannot meet the rendering progress, the video frame among S consecutive video frames that is closer to the video frame currently executing the target video editing operation can be set as the switching position.

[0189] It should be understood that when determining the above-mentioned maximum duration, since the video frame currently performing the target video editing operation has not completed the target video editing operation, the maximum duration can be calculated based on the previous frame of the video frame currently performing the target video editing operation, thereby improving the accuracy of the calculation result.

[0190] It should be noted that the execution order of S602 and S603 is not particular.

[0191] S604: Using the first thread, starting from the switching position, perform the target video editing operation on the video frame of the video to be processed according to the third processing mode.

[0192] For details about the third processing mode, please refer to Figure 4 For the sake of brevity, the description in the illustrated embodiment will not be repeated here.

[0193] The method provided in this embodiment obtains a playback instruction during the process of executing a target video editing operation on a video to be processed; in response to the playback instruction, the switching position of the video editing processing mode is flexibly determined based on the duration required for executing the target video editing operation on a single video frame, the playback speed, and the position of the video frame currently executing the target video editing operation; when the progress of the target video editing operation reaches the switching position, the video editing processing mode is switched to the third processing mode, thereby improving the flexibility of switching the video editing processing mode. In addition, through the third processing mode, it is possible to ensure that the position of the video frame executing the target video editing operation is always ahead of the playback progress during the playback process, thereby solving problems such as severe freezes and screen distortion during user preview playback, improving video processing efficiency, and being conducive to improving user experience.

[0194] Optionally, in Figure 6 Based on the embodiment shown, the present invention further includes:

[0195] S605: Get a pause playback instruction.

[0196] S606: In response to the pause playback instruction, the second thread pauses playback, and the first thread continues to perform the target video editing operation on the video frames of the video to be processed according to the third processing mode.

[0197] Specifically, when the application receives the pause playback instruction, the second thread stops rendering the target video frame, that is, pauses playback of the target video frame.

[0198] Since a process already executing the target special effects on the video being processed is in progress when the pause playback instruction is received, this process can be left uninterrupted. That is, the first thread continues to execute the target video editing operation on the video being processed, i.e., continues to execute the target video editing operation on the video frames of the video being processed according to the third processing mode. This allows the computing resources of the electronic device to be effectively utilized to execute video editing, thereby improving computing resource utilization and thus video processing efficiency.

[0199] Figure 7 This is a flow chart of a video processing method provided by another embodiment of the present disclosure. Figure 7As shown, the method provided in this embodiment includes:

[0200] S701. Starting from a specified video frame position, a first thread performs a target video editing operation on the video frame at the specified video frame position in the video to be processed, and pre-performs a target video editing operation on each video frame after the specified video frame position, and obtains each target video frame obtained by performing the target video editing operation.

[0201] S702. In response to a play instruction, each of the target video frames is rendered through a second thread to display each of the target video frames; wherein the position of the video frame on which the target video editing operation is being performed is ahead of the position of the video frame corresponding to the target video frame being displayed in the video to be processed.

[0202] In this embodiment, steps S701 and S702 are respectively Figure 4 The embodiments S401 and S402 are similar, and can be referred to Figure 4 For the sake of brevity, the detailed description of the illustrated embodiment will not be repeated here.

[0203] S703 : In response to the play instruction, detect in real time whether the next video frame to be played has a corresponding target video frame.

[0204] When it is detected that the next video frame to be played does not have a corresponding target video frame, S704 is executed.

[0205] It should be noted that the execution order of S702 and S703 is not particular.

[0206] S704: Execute a target video editing operation on the video frame of the video to be processed according to the third processing mode, starting from the position of the next target video frame to be played, through the first thread.

[0207] It should be noted that, starting from the position of the next target video frame to be played, executing the target video editing operation according to the third processing mode does not mean that the next video frame on which the target video editing operation is to be executed is the video frame corresponding to the position of the next video frame to be played. For example, if the next video frame to be played is the 10th frame of the video frame to be processed, but there is no corresponding target video frame for the 10th frame, then starting from the 10th frame, the first thread executes the target video editing operation according to the third processing mode. However, the position of the video frame determined to execute the target video editing operation may be the 13th frame, the 15th frame, and so on in the video to be processed, thereby ensuring that the position of the video frame on which the target video editing operation is being executed is ahead of the playback progress.

[0208] In this embodiment, while a target video editing operation is being performed on a video to be processed, a play instruction is received; in response to the preview play instruction, upon detecting that the next video frame to be played does not have a corresponding target video frame, the video editing processing mode is switched. This minimizes resource consumption caused by the need to frequently switch video editing processing modes due to frequent user operations. In addition, the play processing mode ensures that the position of the video frame performing the target video editing operation is always ahead of the playback progress during playback, thereby resolving issues such as severe playback freezes and screen distortion during user preview, improving video processing efficiency, and ultimately enhancing the user experience.

[0209] Optionally, in Figure 7 Based on the embodiment shown, the present invention further includes:

[0210] S705: Get a pause playback instruction.

[0211] S706: In response to the pause playback instruction, the second thread pauses playback, and the first thread continues to perform the target video editing operation on the video frames of the video to be processed according to the third processing mode.

[0212] Specifically, when the application receives the pause playback instruction, the second thread stops rendering the target video frame, that is, pauses playback of the target video frame.

[0213] Since a process already executing the target special effects on the video being processed is in progress when the pause playback instruction is received, this process can be left uninterrupted. That is, the first thread continues to execute the target video editing operation on the video being processed, i.e., continues to execute the target video editing operation on the video frames of the video being processed according to the third processing mode. This allows the computing resources of the electronic device to be effectively utilized to execute video editing, thereby improving computing resource utilization and thus video processing efficiency.

[0214] Figure 8 This is a flowchart of a video processing method provided by another embodiment.

[0215] The application supports video frame position jump, so the application can also obtain jump instructions during the process of playing each target video frame, that is, in the above Figures 1 to 7 The method of this embodiment can also be executed on the basis of any embodiment.

[0216] Reference Figure 8 As shown, the method of this embodiment includes:

[0217] S801: Get the third jump instruction.

[0218] S802: In response to the third jump instruction, redetermine the video editing processing mode according to whether there is a corresponding target video at the video frame position indicated by the third jump instruction and whether the first thread is performing the target video editing operation on the video to be processed.

[0219] S803: Perform a target video editing operation on the video frames of the video to be processed according to the re-determined video editing processing mode through the first thread.

[0220] And through the second thread, the target video frame corresponding to the video frame position indicated by the jump instruction is rendered, and the target video frame corresponding to the video frame position indicated by the third jump instruction is displayed.

[0221] In some cases, during the playback process, the user inputs a third jump instruction, and the application can switch the playback state to pause; in other cases, during the playback process, the user inputs a third jump instruction, and the application can also start playing the target video frame from the video frame position indicated by the third jump instruction.

[0222] In this embodiment, if the application switches the playback state to the paused playback state in response to the third jump instruction, the following situations may occur:

[0223] In case (a), there is a corresponding target video frame at the video frame position indicated by the third jump instruction, and there is currently a process that performs the target video editing operation on the video to be processed, then the current process will not be interrupted, and the first thread will continue to perform the target video editing operation on the video to be processed according to the original video editing processing mode.

[0224] Because the target video frame exists at the video frame location indicated by the third jump instruction, the application can display the target video frame corresponding to the video frame location indicated by the third jump instruction. Since the application can be in a paused preview playback state, no lag occurs. On this basis, the application can continue to execute the current process.

[0225] Case (b): There is a corresponding target video frame at the video frame position indicated by the third jump instruction, but there is currently no process for executing the target special effects on the video to be processed. Therefore, it is possible to determine frame by frame whether there is a corresponding target video frame starting from the next frame of the video frame position indicated by the third jump instruction, until starting from the first video frame position determined to not have a corresponding target video frame, the target video editing operation is performed on the video to be processed according to the second processing mode.

[0226] In case (c), if there is no corresponding target video frame at the video frame position indicated by the third jump instruction, the target video editing operation is performed on the video frame of the video to be processed starting from the video frame position indicated by the third jump instruction according to the second processing mode.

[0227] If in response to the third jump instruction, the application starts playing the target video frame from the video frame position indicated by the third jump instruction, the video editing processing mode can be switched to the third processing mode to ensure that when the target video frame is played from the video frame position indicated by the third jump instruction, the video frame position where the target video editing operation is being performed is always ahead of the current playback progress, thereby achieving uninterrupted playback and meeting the user's preview needs.

[0228] The method for determining the switching timing of the video editing processing mode can be implemented by using the aforementioned Figures 5 to 7 The method shown in any embodiment can refer to the above Figures 5 to 7 For the sake of brevity, the detailed description of the illustrated embodiment will not be repeated here.

[0229] The method provided in this embodiment determines whether a video editing mode switch is necessary by analyzing whether the target video frame exists at the video frame position indicated by the third jump instruction and whether a process is currently executing the target video editing operation. Furthermore, the playback status is considered when determining whether to switch the video editing mode, ensuring a real-time display effect.

[0230] Figure 9 This is a flow chart of a video processing method provided by another embodiment of the present disclosure. Figure 9 As shown, the method of this embodiment includes:

[0231] S901. Starting from a specified video frame position, using a first thread, perform a target video editing operation on the video frame at the specified video frame position in the video to be processed, and pre-perform a target video editing operation on each video frame after the specified video frame position, and obtain each target video frame obtained by performing the target video editing operation.

[0232] S902. In response to a play instruction, each of the target video frames is rendered through a second thread to display each of the target video frames; wherein the position of the video frame on which the target video editing operation is being performed is ahead of the position of the video frame corresponding to the target video frame being displayed in the video to be processed.

[0233] In this embodiment, steps S901 and S902 are respectively Figure 1 The embodiments S101 and S102 shown are similar, and can be referred to Figure 1 For the sake of brevity, the detailed description of the illustrated embodiment will not be repeated here.

[0234] S903: After performing the target video editing operation on the last video frame of the video to be processed, perform the target video editing operation on the video frames of the video to be processed according to the fourth processing mode, starting from the position of the starting video frame of the video to be processed, through the first thread.

[0235] The fourth processing mode is a mode for sequentially performing target video editing operations on video frames in the video to be processed that do not have corresponding target video frames. The fourth processing mode is an adaptive video editing processing mode.

[0236] When the application executes any one or more video editing processing modes from the first processing mode to the third processing mode to the last video frame position of the video to be processed, it can switch to the fourth processing mode to make full use of computing resources and perform target video editing operations on the video frames in the video to be processed that do not have corresponding target video frames.

[0237] Next, we introduce the two cases of the video to be processed including one video clip and the video to be processed including multiple video clips respectively:

[0238] 1. The video to be processed includes a video clip

[0239] Assume that the video to be processed includes a video segment, which is recorded as video segment A1. When it is detected that the target video editing operation has been performed on the last video frame of video segment A1, the video editing processing mode is switched to the fourth processing mode. Starting from the starting video frame position of video segment A1, the target video editing operation is performed on the video frames on which the target video editing operation has not been performed, in sequence according to the order of the video frames in video segment A1.

[0240] Exemplarily, the video segment A1 frame includes 100 frames. It is assumed that the target video editing operation is performed on the 50th to 100th frames according to the second processing mode; when it is received that the target video editing operation is completed on the 100th frame, it switches to the fourth processing mode, starting from the 1st frame, and performs the target video editing operation on the 1st to 49th frames frame by frame.

[0241] For another example, the video segment A1 frame includes 100 frames. It is assumed that the target video editing operation is performed on the 1st frame, the 5th frame, the 10th frame, the 15th frame, and the 20th frame according to the first processing mode; starting from the 30th frame, the second processing mode is switched to, and the target video editing operation is performed on the 30th frame to the 100th frame; when it is detected that the target video editing operation is completed on the 100th frame, the fourth processing mode is switched to, starting from the 1st frame, and the target video editing operation is performed on the 2nd to 4th frames, the 6th to 9th frames, the 11th to 14th frames, and the 16th to 19th frames in turn, on which the target video editing operation has not been performed.

[0242] 2. The video to be processed includes multiple video clips

[0243] Assume that the video to be processed includes multiple video segments, which are respectively recorded as video segments B1 to B N When the video segment B is detected n After the target video editing operation is performed on the last video frame of the video segment B, the video editing processing mode is switched to the fourth processing mode. n The starting video frame position of video segment B n The target video editing operation is performed on the video frames that have not been subjected to the target video editing operation in the order of the video frames in the video editing process. Wherein, N is an integer greater than or equal to 2. n is an integer greater than or equal to 1 and less than or equal to N.

[0244] That is to say, when the video to be processed includes multiple video clips, when it is detected that the target video editing operation is performed on the last video frame of a video clip, the special effects processing task of the fourth processing mode can be preferentially performed within the video clip.

[0245] In practical applications, each video segment corresponds to a decoder. Executing the target video editing task of the fourth processing mode within a video segment can reduce the computing resource consumption caused by updating the decoder due to video segment switching. Furthermore, executing the target video editing task of the fourth processing mode within a video segment can fully utilize computing resources and improve video processing efficiency.

[0246] For example, assume that the video to be processed includes three video segments, namely video segments B1, B2, and B3. A target video editing operation is currently being performed on video segment B2 according to the second processing mode. When it is detected that the target video editing operation has been performed on the last video frame of video segment B2, the fourth processing mode is switched to, and the target video editing operation is sequentially performed on video frames in video segment B2 that do not have a corresponding target video frame, starting from the starting video frame position of video segment B2.

[0247] When all video frames in the video segment B2 have corresponding target video frames, the video segment B2 may be marked as “completed.” If no other operation instructions are received, the target video editing operation may be performed on the video segments B1 and B3.

[0248] Here, the example is taken as follows: the target video editing operation is completed on the video segment B2, and the next video segment to be processed is the video segment B1.

[0249] The target video editing operation may not have been performed on video segment B1, so no corresponding target video frame exists for any video frame in video segment B1. In this case, starting from the starting video frame of video segment B1, the first processing mode can be used to sequentially perform the target video editing operation on each video frame in video segment B1. Upon detecting that the target video editing operation has been performed on the last video frame of video segment B1, the fourth processing mode is switched to sequentially perform the target video editing operation on the video frames in video segment B1 that do not have corresponding target video frames.

[0250] The target video editing operation may have already been performed on video segment B1, and thus some of the video frames in video segment B1 have corresponding target video frames. In this case, starting from the starting video frame position of video segment B1, the fourth processing mode is used to sequentially perform the target video editing operation on the video frames in video segment B1 that do not have corresponding target video frames.

[0251] It should be noted that if all video frames in the video segment B1 have corresponding target video editing operations, the video segment B1 will be marked as "completed". Based on the above mark, it can be determined that the target video editing operation does not need to be performed on the video segment B1.

[0252] When the next video segment to be processed is video segment B3, its implementation method is similar to the implementation method when the next video segment to be processed is video segment B1. Please refer to the above detailed introduction and will not be repeated here.

[0253] The method provided by this embodiment switches the video editing processing mode to the fourth processing mode when it is detected that the target video editing operation has been performed on the last video frame of the video to be processed during the process of performing the target video editing operation on the video frames of the video to be processed, and sequentially performs the target video editing operation on the video frames in the video to be processed that do not have corresponding target video frames.

[0254] Figure 10 This is a schematic diagram of a video processing device according to an embodiment of the present disclosure. Figure 10 The video processing device shown includes three physical layers: a graph layer, an interaction layer, and an algorithm processing layer.

[0255] The layer includes: a decoding thread and a rendering thread; wherein the decoding thread includes: a decoding control unit (decoder reader unit); the rendering thread includes: a video preprocessing unit (clip preprocess unit).

[0256] The decoding control unit is used to update the algorithm processing tasks being executed in the algorithm processing layer according to the operation instructions input by the user, such as jump instructions, play instructions, etc.; and synchronously update the rendering position.

[0257] The video preprocessing unit is used to determine whether there is a corresponding target video frame at the current video frame position; if so, the target video frame is read, and the texture is uploaded for rendering and screen display; if not, the data of the corresponding target video frame is read based on the previous video frame that performed the target video editing operation, and the texture is uploaded for rendering and screen display.

[0258] The interaction layer includes: task scheduling module and cache control module.

[0259] Among them, the task scheduling module is used to generate video editing tasks and send video editing tasks to the algorithm processing layer.

[0260] It should be noted that the task scheduling module can generate video editing tasks based on user input instructions or based on information returned by the algorithm processing layer indicating a switch to a self-propagation mode. Therefore, the task scheduling module can be viewed as performing task scheduling using both external triggering and self-propagation strategies.

[0261] In the framework of the entire video processing device, the algorithm task is the smallest processing unit. Each time a video editing operation (e.g., clip) is added, an algorithm task (task Param) object is created and a task identifier (task ID) is returned. Each algorithm task object uniquely corresponds to a task identifier.

[0262] The algorithm task (task Param) object is a structure that records all the parameters required for algorithm processing on a clip and holds a reference to an algorithm processing instance (task Process wrapper).

[0263] The task process wrapper holds a one-to-one correspondence with each Lab algorithm model instance and manages its corresponding content. The task process wrapper maintains a master file (MANE File) that records information about all algorithm results (i.e., target video frames) for that path file (e.g., timestamp information indicating the video frame location, such as PTS information) and provides an interface for querying.

[0264] In practical applications, the data volume of the Lab algorithm model instance is large. For example, the Lab algorithm model instance in the intelligent cutout is about 100 megabytes. In order to avoid out of memory (OOM) situations, the Lab algorithm model instance is destroyed immediately after use.

[0265] The cache control module is used to return the target video frame to the rendering thread based on the target video frame stored in the cache; when the corresponding target video frame does not exist in the cache area, the access path of the target video frame in the external storage space is returned to the rendering thread; if the target video frame does not exist in the external storage space, null is returned to the rendering thread (return NULL).

[0266] The algorithm layer may also be called the algorithm interaction layer, the middle layer, or other names, which is not limited in this disclosure.

[0267] The algorithm processing layer includes: an algorithm task management module, which provides an access interface to the outside world. For example, the algorithm task management module provides an interface for adding, deleting, and querying special effect processing tasks and progress.

[0268] The algorithm task management module itself also holds the algorithm task (task Param) object, algorithm processing instance and algorithm thread.

[0269] Among them, the essence of the algorithm thread is a thread object, which is a thread selected from the thread pool to execute the special effect processing tasks in the corresponding message queue.

[0270] When the algorithm thread performs a video editing task, the following steps may be included:

[0271] Step 1: Update the corresponding decoder.

[0272] Step 2: Correct the total number of frames corresponding to the video editing task.

[0273] Step 3: Determine whether the number of processed frames is equal to the total number of frames, or determine whether the starting position of the target video editing operation is the last video frame of the video to be processed.

[0274] It should be noted that whether the number of processed frames is equal to the total number of frames corresponds to Figure 10 Condition 1: Whether the starting position of the target video editing operation is the last video frame of the video to be processed, that is, Figure 10 Medium condition 2.

[0275] Step 4: When none of the above conditions are met, determine whether a corresponding target video frame already exists for the video frame; if so, skip executing the target video editing operation on the video frame; if not, execute step 5.

[0276] Step 5: Determine whether the video frame is a video frame that needs to be discarded according to the corresponding video editing processing mode.

[0277] The video editing processing mode referred to here can be any one of a play processing mode, a jump processing mode, a pre-processing mode and a self-propagation processing mode. For the meanings of the above-mentioned various video editing processing modes, please refer to the above description.

[0278] It should be noted that if the video editing processing mode is the playback processing mode, the video frame is determined to be a video frame that needs to be discarded based on the playback speed. If it is a video frame that needs to be discarded, there is no need to perform the target video editing operation on it; if it is not a video frame that needs to be discarded, execute step 6.

[0279] If the video editing processing mode is the jump processing mode, each video frame is a video frame that does not need to be discarded.

[0280] If the video editing processing mode is the pre-processing mode, the key frame is a video frame that does not need to be discarded, and step 6 is executed; the other frames are video frames that need to be discarded.

[0281] When the video editing processing mode is the self-propagation mode, video frames that do not have corresponding target video frames are all video frames that do not need to be discarded; video frames that have corresponding target video frames are all video frames that need to be discarded.

[0282] Step 6: Perform target video editing operations on the video frames to obtain corresponding target video frames.

[0283] Step 7: Store the target video frame corresponding to the video frame.

[0284] The target video frame corresponding to a video frame can be stored in both external storage and the cache. When the rendering thread is rendering, if the corresponding target video frame is in the cache, the data can be directly read from the cache; if the corresponding target video frame is not in the cache, the corresponding target video frame is read from the external storage space. This approach can reduce data input and output times and improve video processing efficiency.

[0285] Furthermore, when storing the target video frame corresponding to the video frame, the relevant information of the target video frame can be passed to the callback service, so that the callback service can call the service thread to perform related operations. For example, during the process of executing the target video edit, the callback service can provide feedback to the service thread on the current execution progress of the target video edit.

[0286] Using the above Figure 10In the framework shown, the algorithm thread can perform target video operations on the processed video based on various video editing processing modes, ensuring that the algorithm thread's execution progress always stays ahead of the rendering thread's progress, thus resolving the issue of user preview lag. Furthermore, after receiving a task to perform a target video editing operation on a video frame of the processed video, the algorithm thread can execute it independently without affecting the upper-level encoding and rendering threads. This enables parallel processing among the encoding, algorithm, and rendering threads, resolving the issue of the algorithm thread blocking the rendering thread due to time-consuming processing, thereby resolving the issue of user preview lag.

[0287] Figure 11 An embodiment of the present disclosure provides Figure 10 The life cycle diagram of each thread in the embodiment shown is shown. Figure 11 As shown, it includes: frame extraction thread, rendering thread, business thread and algorithm thread.

[0288] Among them, the frame extraction thread is used to perform synchronous frame extraction; the rendering thread is used to render the target video frame for playback; the business thread is used to control the algorithm thread to perform the target video editing operation on each video frame of the video to be processed according to the determined video editing processing mode and to control the pause and start of playback; the algorithm thread is used to perform the target video editing operation on the corresponding video frame according to the instruction of the business thread.

[0289] When receiving a video editing instruction to perform a target video editing operation on each video frame of the video to be processed, first, the service thread sets the video editing processing mode to the pre-processing mode or the jump processing mode, and the control algorithm thread is started and runs the algorithm according to the set video editing processing mode to perform the target video editing operation on the video frame of the video to be processed. Figure 11 As shown, during this process, the corresponding target video frames need to be cached in sequence.

[0290] When playback is required, the service thread first pauses the algorithm thread, sets the video editing processing mode to the playback processing mode, and controls the algorithm thread to run the algorithm according to the playback processing mode to perform the target video editing operation on the video frames to be processed. Figure 11 As shown, in this process, the corresponding target video frames still need to be cached in sequence. Synchronously, the service thread controls the rendering thread to read the corresponding target video frames from the cache for rendering and display for user preview.

[0291] When preview playback is paused, the service thread controls the rendering thread to pause playback.

[0292] When jumping to a video frame position, the service thread first pauses the algorithm thread and sets the video editing processing mode to the jump processing mode, controls the algorithm thread to run the algorithm according to the jump processing mode, and starts from the final jump video frame position, performs the target video editing operation on the video frame to be processed. Figure 11 As shown, during this process, it is still necessary to synchronously cache the corresponding target video frame based on the final jump video frame position. Synchronously, the service thread controls the rendering thread to display the target video frame at the corresponding video frame position.

[0293] After the target video editing operation is performed on the last video frame of the video to be processed, the business thread sets the video editing processing mode to the self-propagating processing mode, and the control algorithm thread starts from the first video frame position of the video to be processed and sequentially performs the target video editing operation on the video frames that have not performed the target video editing operation. Figure 11 As shown, during this process, the corresponding target video frames still need to be cached in sequence.

[0294] When the target video editing operation is canceled or stopped, the service thread controls the algorithm thread to stop running the algorithm and reclaim the resources occupied by the algorithm thread. After that, the rendering thread and algorithm thread are reclaimed.

[0295] Combine Figure 10 as well as Figure 11 As shown, through Figure 10 The framework shown, combined with Figure 11 The interaction between the threads shown can perform target video editing operations on each video frame of the video to be processed, and effectively solve the problem of lag in user preview and playback.

[0296] Figure 12 This is a schematic diagram of the structure of a video processing device provided by an embodiment of the present disclosure. Figure 12 As shown, the video processing device 1200 provided in this embodiment includes:

[0297] The first processing module 1210 is configured to, starting from a specified video frame position through a first thread, perform a target video editing operation on the video frame at the specified video frame position in the processed video and pre-perform the target video editing operation on each video frame after the specified video frame position, and obtain each target video frame obtained by performing the target video editing operation.

[0298] The second processing module 1220 is used to render each of the target video frames through the second thread in response to the play instruction to display each of the target video frames; wherein the position of the video frame on which the target video editing operation is being performed is ahead of the video frame position corresponding to the target video frame being displayed in the video to be processed.

[0299] As a possible implementation, the video processing device 1200 may further include a display module 1240. The display module 1240 is configured to display each target video frame.

[0300] As a possible implementation, the specified video frame position is determined based on the video frame position located when obtaining the video editing instruction; wherein, the video editing instruction is used to instruct the execution of the target video editing operation on the video to be processed; the video frame position located when obtaining the video editing instruction is a preset video frame position or a video frame position specified by the first jump instruction.

[0301] Correspondingly, the video processing device 1200 may further include: an acquisition module 1230 .

[0302] The acquisition module 1230 is configured to acquire a video editing instruction and a first jump instruction.

[0303] As a possible implementation, the designated video frame position is determined according to a video frame position designated by a second jump instruction, and the second jump instruction is an instruction following a video editing instruction for performing the target video editing operation on the video to be processed.

[0304] Correspondingly, the acquisition module 1230 is further configured to acquire a second jump instruction.

[0305] As a possible implementation, if the specified video frame position is determined based on the video frame position located when obtaining the video editing instruction; the first processing module 1210 starts from the specified video frame position through the first thread, performs the target video editing operation on the video frame at the specified video frame position in the processed video, and pre-performs the target video editing operation on each video frame after the specified video frame position, and before obtaining each target video frame obtained by performing the target video editing operation, it is also used to determine the video editing processing mode corresponding to the first thread according to the specified video frame position; wherein the video editing processing mode is a first processing mode or a second processing mode, the first processing mode is a mode for performing the target video editing operation on key frames, and the second processing mode is a mode for performing the target video editing operation frame by frame.

[0306] As a possible implementation, the first processing module 1210 is specifically used to determine that the video editing processing mode corresponding to the first thread is the first processing mode if the specified video frame position is the preset video frame position; if the specified video frame position is the video frame position specified by the first jump instruction, then determine that the video editing processing mode corresponding to the first thread is the first processing mode or the second processing mode.

[0307] As a possible implementation, the first processing module 1210 is specifically used to determine whether there is a corresponding target video frame at the specified video frame position; if there is no corresponding target video frame at the specified video frame position, then based on the specified video frame position being the preset video frame position or the video frame position specified by the first jump instruction, determine the video editing processing mode corresponding to the first thread.

[0308] As a possible implementation, the first processing module 1210 is specifically configured to determine that the video editing processing mode corresponding to the first thread is the second processing mode if a corresponding target video frame exists at the designated video frame position.

[0309] Correspondingly, the second processing module 1220 is specifically used to determine whether there is a corresponding target video frame starting from the specified video frame position and moving backward frame by frame until the first video frame position where no corresponding target video frame exists is determined; through the first thread, in accordance with the second processing mode, starting from the first video frame position where no corresponding target video frame exists, the target video editing operation is performed on the video frame of the video to be processed.

[0310] As a possible implementation, if the specified video frame position is determined according to the video frame position specified by the second jump instruction; the first processing module 1210, starting from the specified video frame position through the first thread, performs the target video editing operation on the video frame at the specified video frame position in the processed video and pre-performs the target video editing operation on each video frame after the specified video frame position, and before obtaining each target video frame obtained by performing the target video editing operation, is also used to determine that the video editing processing mode corresponding to the first thread is the second processing mode when it is determined that there is no corresponding target video frame at the specified video frame position.

[0311] As a possible implementation, if there is a corresponding target video frame at the specified video frame position, and the first thread is currently performing the target video editing operation on the video to be processed, the first processing module 1210 does not interrupt the first thread, so that the first thread continues to perform the target video editing operation on the video to be processed.

[0312] As a possible implementation, the first processing module 1210 is also used to switch the video editing processing mode to a third processing mode in response to the playback instruction, and perform the target video editing operation on the video frame of the video to be processed according to the third processing mode through the first thread; wherein the third processing mode is a mode for determining the video frame position for performing the target video editing operation according to the playback speed.

[0313] As a possible implementation, the first processing module 1210 is specifically used to respond to the playback instruction and detect in real time whether there is a next target video frame to be played; when it is detected that there is no next target video frame to be played, through the first thread, starting from the position of the next target video frame to be played, the target video editing operation is performed on the video frame of the video to be processed according to the third processing mode.

[0314] As a possible implementation, the first processing module 1210 is specifically used to respond to the playback instruction, through the first thread, starting from the video frame position where the target video editing operation is being performed when the playback instruction is obtained, and perform the target video editing operation on the video frame of the video to be processed according to the third processing mode.

[0315] As a possible implementation, the first processing module 1210 is specifically used to respond to the playback instruction, determine the switching position corresponding to the third processing mode according to the processing time, playback speed and video frame position of the preset single video frame for performing the target video editing operation, and through the first thread, start from the switching position, perform the target video editing operation on the video frame of the video to be processed according to the third processing mode.

[0316] As a possible implementation, the first processing module 1210 is also used to pause the second thread's rendering of the target video frame in response to the pause playback instruction; and the first thread continues to perform the target video editing operation on the video frame of the video to be processed according to the third processing mode.

[0317] Correspondingly, the acquisition module 1230 is further configured to acquire a pause playback instruction.

[0318] As a possible implementation manner, the first processing module 1210 is also used to, after performing the target video editing operation on the last video frame of the video to be processed, perform the target video editing operation on the video frames of the video to be processed according to the fourth processing mode, starting from the position of the starting video frame of the video to be processed through the first thread; wherein, the fourth processing mode is a mode of sequentially performing the target video editing operation on video frames in the video to be processed that do not have corresponding target video frames.

[0319] The video processing device provided in this embodiment can be used to execute the technical solution of any of the aforementioned method embodiments. Its implementation principles and technical effects are similar. Please refer to the detailed description of the aforementioned method embodiments. For the sake of brevity, they will not be repeated here.

[0320] Figure 13 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. Figure 13 As shown, the electronic device 1300 provided in this embodiment includes: a memory 1301 and a processor 1302.

[0321] The memory 1301 may be an independent physical unit, and may be connected to the processor 1302 via a bus 1303. The memory 1301 and the processor 1302 may also be integrated together and implemented through hardware.

[0322] The memory 1301 is used to store program instructions, and the processor 1302 calls the program instructions to execute the operations of any of the above method embodiments.

[0323] Optionally, when part or all of the methods of the above embodiments are implemented by software, the electronic device 1300 may also include only the processor 1302. The memory 1301 for storing programs is located outside the electronic device 1300, and the processor 1302 is connected to the memory via circuits / wires to read and execute the programs stored in the memory.

[0324] The processor 1302 may be a central processing unit (CPU), a network processor (NP), or a combination of a CPU and an NP.

[0325] Processor 1302 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0326] The memory 1301 may include a volatile memory, such as a random-access memory (RAM); the memory may also include a non-volatile memory, such as a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); the memory may also include a combination of the above types of memory.

[0327] The present disclosure also provides a readable storage medium, comprising: computer program instructions; when the computer program instructions are executed by at least one processor of an electronic device, the video processing method shown in any of the above method embodiments is implemented.

[0328] The present disclosure also provides a computer program product. When the computer program product is executed by a computer, the computer is enabled to implement the video processing method shown in any of the above method embodiments.

[0329] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0330] The foregoing description is intended only to provide specific embodiments of the present disclosure, intended to enable those skilled in the art to understand and implement the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the embodiments described herein, but rather to be construed in the broadest manner consistent with the principles and novel features disclosed herein.

Claims

1. A video processing method, characterized in that: include: Starting from a designated video frame position, a first thread performs a target video editing operation on a video frame at the designated video frame position in a to-be-processed video, and pre-performs the target video editing operation on each video frame after the designated video frame position, thereby obtaining target video frames obtained by performing the target video editing operation. In response to the play instruction, rendering each of the target video frames through the second thread to display each of the target video frames; wherein the position of the video frame on which the target video editing operation is being performed is ahead of the position of the video frame corresponding to the target video frame being displayed in the video to be processed; In response to the playback instruction, the video editing processing mode corresponding to the first thread is switched to a third processing mode, and the target video editing operation is performed on the video frame of the video to be processed through the first thread according to the third processing mode; wherein, the third processing mode is a mode for determining the video frame position for performing the target video editing operation according to the playback speed.

2. The method according to claim 1, characterized in that The specified video frame position is determined based on the video frame position located when obtaining the video editing instruction; wherein, the video editing instruction is used to instruct the execution of the target video editing operation on the video to be processed; the video frame position located when obtaining the video editing instruction is a preset video frame position or a video frame position specified by the first jump instruction.

3. The method according to claim 1, characterized in that The designated video frame position is determined according to the video frame position designated by the second jump instruction, and the second jump instruction is an instruction following the video editing instruction for performing the target video editing operation on the video to be processed.

4. The method according to claim 2, characterized in that The method further includes: starting from a designated video frame position, executing a target video editing operation on a video frame at the designated video frame position in a to-be-processed video, and pre-executing the target video editing operation on each video frame after the designated video frame position; and obtaining each target video frame obtained by executing the target video editing operation. According to the specified video frame position, the video editing processing mode corresponding to the first thread is determined; wherein, the video editing processing mode is a first processing mode or a second processing mode, the first processing mode is a mode for performing the target video editing operation on the key frame, and the second processing mode is a mode for performing the target video editing operation frame by frame.

5. The method according to claim 4, characterized in that The determining, according to the specified video frame position, a video editing processing mode corresponding to the first thread includes: If the designated video frame position is the preset video frame position, determining that the video editing processing mode corresponding to the first thread is the first processing mode; If the designated video frame position is the video frame position designated by the first jump instruction, it is determined that the video editing processing mode corresponding to the first thread is the first processing mode or the second processing mode.

6. The method according to claim 4, characterized in that The determining, according to the specified video frame position, a video editing processing mode corresponding to the first thread includes: Determine whether there is a corresponding target video frame at the specified video frame position; If the designated video frame position does not have a corresponding target video frame, the video editing processing mode corresponding to the first thread is determined based on the designated video frame position being the preset video frame position or the video frame position specified by the first jump instruction.

7. The method according to claim 6, characterized in that The method further comprises: If a corresponding target video frame exists at the designated video frame position, determining that the video editing processing mode corresponding to the first thread is the second processing mode; The method of performing a target video editing operation on a video frame at a designated video frame position in a to-be-processed video by a first thread starting from a designated video frame position and pre-performing the target video editing operation on each video frame after the designated video frame position, and obtaining each target video frame obtained by performing the target video editing operation, includes: Starting from the designated video frame position, determine whether a corresponding target video frame exists frame by frame until a first video frame position where no corresponding target video frame exists is determined; Through the first thread, according to the second processing mode, starting from the first video frame position where no corresponding target video frame exists, the target video editing operation is performed on the video frame of the video to be processed.

8. The method according to claim 3, characterized in that The method further includes: starting from a designated video frame position, executing a target video editing operation on a video frame at the designated video frame position in a to-be-processed video, and pre-executing the target video editing operation on each video frame after the designated video frame position; and obtaining each target video frame obtained by executing the target video editing operation. If the designated video frame position does not have a corresponding target video frame, it is determined that the video editing processing mode corresponding to the first thread is a second processing mode, where the second processing mode is a mode for performing the target video editing operation frame by frame.

9. The method according to claim 8, characterized in that The method further comprises: If there is a corresponding target video frame at the designated video frame position and the first thread is currently performing the target video editing operation on the video to be processed, the first thread is not interrupted, so that the first thread continues to perform the target video editing operation on the video to be processed.

10. The method according to any one of claims 1 to 9, characterized in that In response to the play instruction, switching the video editing processing mode to a third processing mode, and performing the target video editing operation on the video frame of the to-be-processed video according to the third processing mode by the first thread, includes: In response to the play instruction, detecting in real time whether there is a next target video frame to be played; When it is detected that there is no next target video frame to be played, the target video editing operation is performed on the video frame of the video to be processed according to the third processing mode, starting from the position of the next target video frame to be played, through the first thread.

11. The method according to any one of claims 1 to 9, characterized in that In response to the play instruction, switching the video editing processing mode to a third processing mode, and performing the target video editing operation on the video frame of the to-be-processed video according to the third processing mode by the first thread, includes: In response to the play instruction, through the first thread, starting from the video frame position where the target video editing operation is being performed when the play instruction is obtained, the target video editing operation is performed on the video frame of the video to be processed according to the third processing mode.

12. The method according to any one of claims 1 to 9, characterized in that In response to the play instruction, switching the video editing processing mode to a third processing mode, and performing the target video editing operation on the video frame of the to-be-processed video according to the third processing mode by the first thread, includes: In response to the play instruction, determining a switching position corresponding to the third processing mode according to a preset processing time length and a play speed for performing the target video editing operation on a single video frame and a position of a video frame currently performing the target video editing operation; The target video editing operation is performed on the video frame of the video to be processed according to the third processing mode, starting from the switching position, through the first thread.

13. The method according to any one of claims 1 to 9, characterized in that The method further comprises: Get the pause playback instruction; In response to the pause playback instruction, the second thread is paused from rendering the target video frame; and the first thread continues to perform the target video editing operation on the video frame of the video to be processed according to the third processing mode.

14. The method according to claim 1, wherein The method further comprises: After performing the target video editing operation on the last video frame of the video to be processed, performing the target video editing operation on the video frames of the video to be processed according to a fourth processing mode, starting from the position of the starting video frame of the video to be processed, through the first thread; The fourth processing mode is a mode of sequentially performing the target video editing operation on video frames in the video to be processed that do not have corresponding target video frames.

15. A video processing device, characterized in that: include: a first processing module, configured to, through a first thread, start from a specified video frame position, perform a target video editing operation on the video frame at the specified video frame position in the video to be processed, pre-perform the target video editing operation on each video frame after the specified video frame position, and obtain each target video frame obtained by performing the target video editing operation; a second processing module, configured to render each of the target video frames through a second thread in response to a play instruction, so as to display each of the target video frames; wherein a position of a video frame on which the target video editing operation is being performed is ahead of a position of a video frame corresponding to the target video frame being displayed in the video to be processed; The first processing module is also used to switch the video editing processing mode corresponding to the first thread to a third processing mode in response to the playback instruction, and perform the target video editing operation on the video frame of the video to be processed according to the third processing mode through the first thread; wherein the third processing mode is a mode for determining the video frame position for performing the target video editing operation according to the playback speed.

16. An electronic device, characterized in that: include: memory and processor; The memory is configured to store computer program instructions; The processor is configured to execute the computer program instructions so that the electronic device implements the video processing method according to any one of claims 1 to 14.

17. A readable storage medium, characterized in that include: computer program instructions; When the computer program instructions are executed by at least one processor of an electronic device, the electronic device implements the video processing method according to any one of claims 1 to 14.

18. A computer program product, characterized in that When the computer program product is executed by a computer, the computer is enabled to implement the video processing method according to any one of claims 1 to 14.

Images