Video processing method, device, electronic device and readable storage medium

By separating video editing and rendering operations into different threads and performing editing ahead of the frame presentation, the method addresses display freezes in video editing applications, ensuring smooth playback and improved user experience.

JP2024545005AActive Publication Date: 2024-12-05BEIJING ZITIAO NETWORK TECH CO LTD
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
JP2024529213
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2021-11-15
Filing Date
2022-11-02
Publication Date
2024-12-05
Estimated Expiration
2042-11-02

AI Technical Summary

Technical Problem

Existing video editing applications using streaming processing frameworks often experience display freezes during preview playback due to the time-consuming nature of video editing operations, which disrupt the user experience.

Method used

A method where a first thread performs video editing operations ahead of the specified frame position, and a second thread renders the edited frames in response to playback instructions, ensuring the editing process is always ahead of the frame being presented, utilizing different threads for editing and rendering to prevent blocking.

Benefits of technology

This approach prevents display freezes and ensures smooth preview playback by aligning the video editing process with rendering, improving user experience and processing efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024545005000001_ABST
    Figure 2024545005000001_ABST
Patent Text Reader

Abstract

The present disclosure relates to a video processing method, an apparatus, an electronic device, and a readable storage medium, the method including: performing a target video editing operation on a processing target video from a specified video frame position by a first thread, obtaining each target video frame obtained by performing the target video editing operation, and performing rendering on each target video frame by a second thread in response to a playback command, thereby ensuring that the video frame position where the target video editing operation is performed always precedes the video frame position of the presented target video frame by performing the target video editing operation in advance, solving the problem of preview freezing and ensuring the preview needs of users. In addition, the target video editing process and rendering are separated and performed by different threads, solving the problem of the rendering thread being blocked due to the target video editing operation being performed in a streaming framework, which is favorable for improving the preview effect.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] [CROSS REFERENCE TO RELATED APPLICATIONS] This application is based on and claims priority from a Chinese application having application number 202111347351.6 and filing date November 15, 2021, the entire disclosure of which is incorporated herein by reference.

[0002] [Technical field] The present disclosure relates to the field of Internet technology, and in particular to a video processing method, device, electronic device and readable storage medium. [Background technology]

[0003] With the development of Internet technology, various applications (APPs) have been developed, among which applications with video editing functions are particularly popular. At present, applications that can provide video editing functions can realize many video editing operations, such as intelligent matting, and such video editing operations require corresponding processing for each video frame image of a video.

[0004] In related art, a streaming processing framework is generally adopted for such video editing operations, for example, when performing intelligent matting, encoding, matting, rendering and display on a frame-by-frame basis. Summary of the Invention

[0005] To solve or at least partially solve the above technical problems, the present disclosure provides a video processing method, apparatus, electronic device, and readable storage medium.

[0006] According to a first aspect, the present disclosure provides a method for producing a method for detecting a pulmonary circulation disorder, comprising: performing, by a first thread, a target video editing operation on a video frame at the designated video frame position in a target video from a designated video frame position, and pre-performing the target video editing operation on each video frame after the designated video frame position to obtain each target video frame obtained by performing the target video editing operation; and in response to a playback command, rendering, by a second thread, for each of the target video frames and presenting each of the target video frames, wherein a video frame position at which the target video editing operation is being performed precedes a corresponding video frame position in the processed video of the target video frame being presented.

[0007] In one possible embodiment, the specified video frame position is determined based on a video frame position located when a video editing command is obtained, the video editing command being for instructing the execution of the target video editing operation on the target video, and the video frame position located when the video editing command is obtained is a preset video frame position or a video frame position specified by a first jump command.

[0008] In one possible embodiment, the specified video frame position is determined based on a video frame position specified in a second jump command, and the second jump command is an instruction following a video editing instruction for performing the target video editing operation on the target video.

[0009] In one possible embodiment, when the designated video frame position is determined based on a video frame position determined when a video editing command is obtained, the method further comprises: executing a target video editing operation on a video frame at the designated video frame position in the target video from the designated video frame position by the first thread as described above; and pre-executing the target video editing operation on each video frame after the designated video frame position, before obtaining each target video frame obtained by executing the target video editing operation, the method further comprises: The method further includes determining a video editing processing mode corresponding to the first thread based on the specified video frame position, the video editing processing mode being a first processing mode that performs the target video editing operation on a key frame, or a second processing mode that performs the target video editing operation on a frame-by-frame basis.

[0010] In one possible embodiment, the step of determining a video editing processing mode corresponding to the first thread based on the specified video frame position as described above includes: If the specified video frame position is the preset video frame position, determining a video editing processing mode corresponding to the first thread as the first processing mode; When the specified video frame position is a video frame position specified by a first jump instruction, determining a video editing processing mode corresponding to the first thread as the first processing mode or the second processing mode.

[0011] In one possible embodiment, the step of determining a video editing processing mode corresponding to the first thread based on the specified video frame position as described above includes: determining whether a corresponding target video frame exists at the specified video frame location; and when there is no corresponding target video frame at the specified video frame position, determining a video editing processing mode corresponding to the first thread according to whether the specified video frame position is the preset video frame position or a video frame position specified by the first jump command.

[0012] In one possible embodiment, the method comprises: If a corresponding target video frame exists at the specified video frame position, determining a video editing processing mode corresponding to the first thread as the second processing mode; The above-mentioned, performing a target video editing operation on a video frame at a specified video frame position in a target video by a first thread from a specified video frame position, and pre-performing the target video editing operation on each video frame after the specified video frame position to obtain each target video frame obtained by performing the target video editing operation, determining whether a corresponding target video frame exists for each frame after the designated video frame position until a first video frame position without a corresponding target video frame is determined; The method includes performing, by the first thread, the target video editing operation on video frames of the processing target video from the first video frame position where no corresponding target video frame exists in accordance with the second processing mode.

[0013] In one possible embodiment, when the specified video frame position is determined based on a video frame position specified by a second jump instruction, the method includes: performing a target video editing operation on a video frame at the specified video frame position in the target video from the specified video frame position by the first thread as described above; and performing the target video editing operation in advance on each video frame after the specified video frame position to obtain each target video frame obtained by performing the target video editing operation, before the method includes: The method further includes determining a video editing processing mode corresponding to the first thread as a second processing mode when there is no corresponding target video frame at the specified video frame position.

[0014] In one possible embodiment, the method comprises the steps of: The method further includes, if a corresponding target video frame exists at the specified video frame position and the first thread is currently performing the target video editing operation on the target video, not interrupting the first thread and allowing the first thread to continue performing the target video editing operation on the target video.

[0015] In one possible embodiment, the method comprises: The method further includes, in response to the playback command, switching the video editing processing mode to a third processing mode, in which a video frame position at which to perform the target video editing operation is determined according to a playback speed, and performing the target video editing operation on a video frame of the processed video according to the third processing mode by the first thread.

[0016] In one possible embodiment, the above-mentioned step of switching the video editing processing mode to a third processing mode in response to the playback instruction and performing, by the first thread, the target video editing operation on video frames of the target video according to the third processing mode, comprises: Detecting in real time whether there is a next target video frame to be played in response to said play command; and when it is detected that there is no next target video frame to be played, performing, by the first thread, the target video editing operation on video frames of the processing subject video from the position of the next target video frame to be played according to the third processing mode.

[0017] In one possible embodiment, the above-mentioned step of switching the video editing processing mode to a third processing mode in response to the playback instruction and performing, by the first thread, the target video editing operation on video frames of the target video according to the third processing mode, comprises: In response to the playback command, the first thread performs the target video editing operation on video frames of the subject video in accordance with the third processing mode from the video frame position where the target video editing operation was being performed when the playback command was obtained.

[0018] In one possible embodiment, the above-mentioned step of switching the video editing processing mode to a third processing mode in response to the playback instruction and performing, by the first thread, the target video editing operation on video frames of the target video according to the third processing mode, comprises: determining a switching position corresponding to the third processing mode based on a processing time length for performing the target video editing operation on a single video frame, a playback speed, and a video frame position where the target video editing operation is currently being performed in response to the playback command; and performing, by the first thread, the target video editing operations on video frames of the subject video from the switch position according to the third processing mode.

[0019] In one possible embodiment, the method comprises: Obtaining a playback pause command; The method further includes, in response to the playback pause command, pausing rendering of the target video frames by the second thread, and the first thread continuing to perform the target video editing operations on video frames of the target video in accordance with the third processing mode.

[0020] In one possible embodiment, the method comprises: and performing, by the first thread, the target video editing operation on video frames of the target video from a position of a first video frame of the target video according to a fourth processing mode after performing the target video editing operation on a last video frame of the target video; The fourth processing mode is a mode in which the target video editing operations are sequentially performed on video frames in the current video that have no corresponding target video frames.

[0021] According to a second aspect, the present disclosure provides a method for producing a method for detecting a pulmonary circulation, comprising: a first processing module for performing, by a first thread, a target video editing operation on a video frame at the designated video frame position in a to-be-processed video from a designated video frame position, and pre-performing the target video editing operation on each video frame after the designated video frame position to obtain each target video frame obtained by performing the target video editing operation; and a second processing module for rendering each of the target video frames by a second thread in response to a playback command and presenting each of the target video frames, wherein a video frame position at which the target video editing operation is being performed precedes a corresponding video frame position in the processed video of the target video frame being presented.

[0022] According to a third aspect, the present disclosure provides an electronic device including a memory and a processor, the memory is configured to store computer program instructions; The processor is configured to execute the computer program instructions to cause the electronic device to implement the video processing method of any one of the first aspects.

[0023] According to a fourth aspect, the present disclosure provides a readable storage medium comprising computer program instructions which, when executed by at least one processor of an electronic device, cause the electronic device to implement the video processing method described in any one of the first aspect.

[0024] According to a fifth aspect, the present disclosure provides a computer program product which, when executed by a computer, causes the computer to implement the video processing method according to any one of the first aspect.

[0025] The present disclosure provides a video processing method, apparatus, electronic device, and readable storage medium, the method including: performing, by a first thread, a target video editing operation on a target video from a specified video frame location, and obtaining each target video frame obtained by performing the target video editing operation; and, in response to a playback command, performing, by a second thread, rendering on each target video frame and presenting each target video frame. [Brief description of the drawings]

[0026] The drawings herein are incorporated in and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification serve to explain the principles of the present disclosure.

[0027] In order to more clearly describe the technical solutions in the embodiments or related art of the present disclosure, the following briefly describes the drawings that need to be used in the description of the embodiments or related art. Obviously, a person skilled in the art can obtain other drawings based on these drawings without creative efforts.

[0028] [Figure 1]1 is a flowchart of a video processing method according to one embodiment of the present disclosure. [Diagram 2] 4 is a flowchart of a video processing method according to another embodiment of the present disclosure. [Diagram 3] 4 is a flowchart of a video processing method according to another embodiment of the present disclosure. [Figure 4] 4 is a flowchart of a video processing method according to another embodiment of the present disclosure. [Diagram 5] 4 is a flowchart of a video processing method according to another embodiment of the present disclosure. [Figure 6] 4 is a flowchart of a video processing method according to another embodiment of the present disclosure. [Figure 7] 4 is a flowchart of a video processing method according to another embodiment of the present disclosure. [Figure 8] 4 is a flowchart of a video processing method according to another embodiment of the present disclosure. [Figure 9] 4 is a flowchart of a video processing method according to another embodiment of the present disclosure. [Figure 10] 1 is a schematic framework diagram of a video processing device according to an embodiment of the present disclosure. [Figure 11] FIG. 11 is a schematic diagram of the life cycle of each module in the embodiment shown in FIG. 10 according to one embodiment of the present disclosure. [Figure 12] 1 is a schematic diagram illustrating the configuration of a video processing device according to an embodiment of the present disclosure. [Figure 13] FIG. 1 is a schematic diagram illustrating a configuration of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0029] In order to make the above objectives, features and advantages of the present disclosure more clearly understandable, the present disclosure will be further described below. It should be noted that, unless contradictory, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0030] In order to fully understand the present disclosure, many specific details are described in the following description, but the present disclosure may be embodied in other forms different from those described herein. Apparently, the embodiments in the specification are only some embodiments of the present disclosure, but not all embodiments.

[0031] In the related art, in the process of adopting a streaming processing framework to perform intelligent matting, when a user performs a preview playback, a problem of serious display freezing is likely to occur, which seriously affects the user experience.

[0032] The video processing method according to the present disclosure can be performed by a video processing device, and the video processing device can be realized by any software and / or hardware. For example, the video processing device may be a tablet, a mobile phone (e.g., a flip phone, a large screen phone, etc.), a wearable device, an in-vehicle device, an augmented reality (AR) / virtual reality (VR) device, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), a smart TV, a smart screen, a high-definition TV, a 4K TV, a smart speaker, a smart projector, or other Internet of Things (IOT) device, and the present disclosure does not limit the specific type of electronic device in any way.

[0033] It should be noted that the video processing method according to the present disclosure is applicable to at least scenarios having the following characteristics:

[0034] 1. A video editing operation needs to be performed on each video frame of the target video, and a corresponding target video frame (i.e., the video editing processing result) needs to be generated and stored for rendering and display.

[0035] 2. It takes more time for video editing operations, but it needs to provide a better preview playback effect.

[0036] 3. Video editing operations need to follow and match in real time with operation commands input by a user, such as a jump command (which may also be called a seek command), an update command (which may also be called a refresh command), a play command, a variable speed play command, a play pause command, etc.

[0037] For any scenario having the above characteristics, the video editing processing method according to the present disclosure can be adopted to ensure the preview effect for users.

[0038] In order to more clearly explain the video processing method according to the present disclosure, the following embodiment will explain in detail the video processing method according to the present disclosure using a case in which a video editing application (hereinafter referred to as application) is installed in an electronic device as an example.

[0039] 1 is a flowchart of a video processing method according to an embodiment of the present disclosure. As shown in FIG. 1, the method according to the embodiment includes the following steps:

[0040] S101: A first thread performs a target video editing operation on a video frame at a specified video frame position in a target video from the specified video frame position, and pre-performs a target video editing operation on each video frame after the specified video frame position to obtain each target video frame obtained by performing the target video editing operation.

[0041] The target video is the material that needs to perform the target video editing operation. The present disclosure does not limit the time length, storage format, resolution, video content, acquisition manner, etc. of the target video.

[0042] The first thread is a thread for performing a target video editing operation on the target video. The first thread may be called an algorithm thread. The algorithm thread can call an algorithm instance corresponding to the target video editing operation to realize the execution of the target video editing operation on the target video. In addition, the present disclosure does not limit the manner of determining the first thread.

[0043] Here, the present disclosure does not limit the type of the target video editing operation. Exemplarily, the target video editing operation may include, but is not limited to, intelligent matting, adding stickers, filters, and the like.

[0044] The designated video frame position may be the position of any one of the video frames in the video to be processed. For example, the designated video frame position may be the video frame position located when the video editing command is obtained, or may be the video frame position designated by the user through a trigger operation.

[0045] The video editing instructions are for instructing to perform a target video editing operation on each video frame of the target video. The present disclosure does not limit the implementation of obtaining the video editing instructions, and for example, a user may input the video editing instructions to the application by operating a corresponding control provided by the application.

[0046] In the following, the implementation of this step will be described exemplarily by taking different example scenarios.

[0047] In one possible embodiment, a user may select a target video through a material selection page displayed in the application and introduce it into the application to perform video editing. The application may display a video editing page in a user interface. The video editing page may include controls corresponding to a number of different video editing operations, including a target control corresponding to a target video editing operation (e.g., intelligent matting). When the application receives a trigger operation (e.g., a tap operation) by the user on the target control, the application generates a video editing command instructing to perform intelligent matting on each video frame of the target video. In response to the video editing command, the application performs the target video editing operation on the target video from the specified video frame position.

[0048] In this scenario, when the target video is introduced, the application may position the video frame position at the preset video frame position by default. Since the user has not input any trigger operation later, the positioned video frame position has not changed, and when a video editing command is obtained, the positioned video frame position is still the preset video frame position as described above. That is, the specified video frame position is the preset video frame position. Here, the preset video frame position may be the position of any one of the video frames of the target video. For example, the preset video frame position is the position of the first video frame of the target video.

[0049] In another possible embodiment, a user may select a target video through a material selection page displayed in the application, and introduce the target video into the application to perform video editing. The application may display a video editing page in a user interface. The video editing page may include controls corresponding to a plurality of different video editing operations, including a target control corresponding to a target video editing operation (e.g., intelligent matting). When the application receives a trigger operation (e.g., a tap operation) by the user on the target control, the application generates a video editing command instructing to perform intelligent matting on each video frame of the target video. In response to the video editing command, the application performs the target video editing operation on the target video from the designated video frame position. Here, before the user inputs the trigger operation on the target control, a first jump command may be input to cause the positioned position to jump from a preset video frame position to a video frame position designated by the first jump command.

[0050] In this scenario, the specified video frame position is the video frame position indicated by the first jump command. Note that the user may input one or more first jump commands, and typically the video frame position indicated by the last jump command is the specified video frame position.

[0051] In another possible embodiment, based on the above first possible embodiment, the first thread may execute a target video editing operation from a video frame position already preset, and while executing the target video editing operation, a user may input a second jump instruction, and then, in response to the second jump instruction, the application needs to execute the target video editing operation in the first thread from the video frame position indicated by the second jump instruction.

[0052] In this scenario, the specified video frame position is a video frame position indicated by a second jump instruction, where the user may input multiple second jump instructions, and the first thread needs to execute a target video editing operation from the video frame position indicated by the second jump instruction in response to each second jump instruction, respectively.

[0053] S102: In response to a play command, a second thread performs rendering on each of the target video frames and presents each of the target video frames, where a video frame position at which the target video editing operation is being performed precedes a corresponding video frame position in the target video of the target video frame being presented.

[0054] In the first thread, while performing a target video editing operation on the target video, the application supports a user to preview each target video frame obtained by performing the target video editing operation, i.e., the application supports a user to preview playback each target video frame.

[0055] In one possible embodiment, when the application receives a play command, the application responsively performs rendering for each target video frame according to a play speed in a second thread and plays each target video frame.

[0056] It should be noted that, while the first thread is performing the target video editing operation on the processing target video, when the first thread performs the target video editing operation on the video frame at the designated video frame position to obtain the target video frame corresponding to the designated video frame position, the second thread may automatically perform rendering on the target video frame corresponding to the designated video frame position, and present the target video frame corresponding to the designated video frame position, thereby ensuring that the user can preview the processing effect corresponding to the designated video frame position before the application receives a play command.

[0057] In another possible embodiment, while the first thread is performing the target video editing operation on the processed video, the second thread may perform the target video editing operation on the video frame at the specified video frame position to obtain the target video frame corresponding to the specified video frame position, and then the second thread may automatically render each target video frame from the specified video frame position, i.e., the user does not need to trigger a play command.

[0058] Regardless of which of the above embodiments is adopted, the method according to the present embodiment ensures that the target video frame on which the target video editing operation is performed is always available for preview by ensuring that the video frame position on which the target video editing operation is performed precedes the position of the presented target video frame in the processing video, thus solving the problem of the user's preview freezing.

[0059] In order to clarify the technical solution, an example is taken to describe the meaning of the video frame position where the target video editing operation is performed preceding the position of the presented target video frame in the to-be-processed video. Assuming that the to-be-processed video has 100 frames, starting from the 1st frame to the 100th frame in order of earliest, the presented target video frame corresponds to the 20th frame, and the video frame position where the target video editing operation is performed is the 30th frame, it indicates that the video frame position where the target video editing operation is performed preceding the position of the presented target video frame in the to-be-processed video.

[0060] The method according to the present embodiment performs a target video editing operation on a target video from a specified video frame position by a first thread, obtains each target video frame obtained by performing the target video editing operation, and performs rendering on each target video frame by a second thread in response to a playback command, and presents each target video frame. The present disclosure ensures that the video frame position where the target video editing operation is performed always precedes the video frame position of the presented target video frame in the target video by performing the target video editing operation in advance on each video frame after the specified video frame position. This solves the problem of the preview freezing while the target video editing operation is being performed on the target video, and ensures the preview needs of the user. In addition, since the execution of the target video editing operation takes time, if the time required for the algorithm processing and the time required for the rendering thread are not consistent, the rendering thread is likely to be blocked. For this reason, the present disclosure solves the problem of the rendering thread being blocked due to the execution of the target video editing operation in a streaming framework by separating the executed video editing process and rendering and executing them by different threads, which is advantageous for improving the preview effect.

[0061] Based on the embodiment shown in FIG. 1 , the video processing method according to the present disclosure can provide at least the following several video editing processing modes: When an application receives a video editing command, it may determine a video editing processing mode corresponding to a first thread based on a preset policy, where the preset policy may be related to one or more factors, such as, but not limited to, a current video frame position of the to-be-processed video, an overall progress of performing a target video editing operation on the to-be-processed video, a playback state (playback pause state or play state), and a user's trigger operation.

[0062] Here, the application supports at least four video editing processing modes: a first processing mode, a second processing mode, a third processing mode, and a fourth processing mode.

[0063] Here, the first processing mode is a mode for performing a target video editing operation on a key frame of a target video, and may be called other names such as a pre-processing mode, an anchor mode, or an anchor mode.

[0064] The second processing mode is a mode for performing the target video editing operation on a frame-by-frame basis, and may be called other names such as a jump processing mode, a seek mode, etc.

[0065] The third processing mode is a mode for determining a video frame position for performing a target video editing operation based on a playback progress, and may be called a playback processing mode or other names.

[0066] The fourth processing mode is a mode in which the target video editing operation is sequentially performed on video frames in the to-be-processed video that do not have a corresponding target video frame. That is, the fourth processing mode is a mode in which the target video editing operation is sequentially performed on video frames in which the target video editing operation has not been performed, from the first video frame to the last video frame in the to-be-processed video. The fourth processing mode may also be called other names such as a self-enhancing processing mode, an adaptive mode, etc.

[0067] Based on the above preset policies, the application can determine the video editing processing mode corresponding to the first thread performing the target video editing operation in different scenarios.

[0068] Hereinafter, several specific embodiments will be used to describe in detail how the first thread performs the target video editing operation on the target video in different scenarios in different video editing processing modes.

[0069] 2 is a flowchart of a video processing method according to an embodiment of the present disclosure. As shown in FIG. 2, the method according to the embodiment includes the following steps:

[0070] S201: Determine a video editing processing mode corresponding to a first thread according to a specified video frame position, where the video editing processing mode is a first processing mode or a second processing mode.

[0071] 1, the specified video frame position may be a preset video frame position, may be a video frame position indicated by a first jump command, or may be a video frame position indicated by a second jump command. The video editing processing mode corresponding to the first thread may be different in different cases.

[0072] Here, the first processing mode is a mode in which a target video editing operation is performed on a key frame (i.e., an I frame). An I frame, also called an intra picture, is an important frame in intraframe compression encoding. During encoding, a part of a video frame sequence is compressed into an I frame, a part into a P frame, and a part into a B frame. During decoding, the whole image can be reconstructed based only on the data of the I frame, without needing to refer to the data of other video frames.

[0073] The second processing mode is for performing target video editing operations on the target video on a frame-by-frame basis, where the second processing mode is a video editing processing mode for responding to a user-entered seek command according to the present disclosure.

[0074] In one possible embodiment, if the specified video frame position is a pre-set video frame position (e.g., the position of the first video frame of the video to be processed), the video editing processing mode corresponding to the first thread is determined as the first processing mode.

[0075] By performing a target video editing operation on a key frame of the target video, when a user previews the effect processing result, the entire image can be reconstructed from the target video frame corresponding to the key frame, so that the preview effect performed by the user can be well ensured.

[0076] In another possible embodiment, when the specified video frame position is a video frame position indicated by a jump instruction, the video editing processing mode corresponding to the first thread is the first processing mode or the second processing mode. Exemplarily, when the specified video frame position is a video frame position indicated by a first jump instruction, the video editing processing mode corresponding to the first thread may be the first processing mode or the second processing mode. When the specified video frame position is a video frame position indicated by a first jump instruction, the video editing processing mode corresponding to the first thread may be the second processing mode.

[0077] S202: A first thread performs a target video editing operation on a video frame at a specified video frame position in a target video according to a determined video editing processing mode, from a specified video frame position, and pre-performs the target video editing operation on each video frame after the specified video frame position to obtain each target video frame obtained by performing the target video editing operation.

[0078] If the determined video editing processing mode is the first processing mode, the application sequentially performs a target video editing operation on each key frame in the video to be processed, starting from the specified video frame position, and stores the target video frame obtained by performing the target video editing operation on each key frame.

[0079] If the determined video editing processing mode is the second processing mode, the application performs a target video editing operation on the target video for each frame from the specified video frame position, and stores the target video frame obtained by performing the target video editing operation on each video frame.

[0080] S203: In response to a play command, a second thread performs rendering on each of the target video frames and presents each of the target video frames, where a video frame position at which the target video editing operation is being performed precedes a corresponding video frame position in the target video of the presented target video frame.

[0081] Step S203 in this embodiment is similar to step S102 in the embodiment shown in FIG. 1, and reference may be made to the detailed description of the embodiment shown in FIG. 1, and for the sake of brevity, no further description will be given here.

[0082] In the method according to the present embodiment, in a video editing scenario, the method obtains and analyzes a specified video frame position to determine a video editing processing mode corresponding to the first thread, and performs a target video editing operation on a corresponding video frame in the target video according to the determined video editing processing mode, thereby ensuring that the video frame position where the target video editing operation is performed always precedes the video frame position of the presented target video frame in the target video. This solves the problem of preview freezing while performing a target video editing operation on the target video, and ensures the preview needs of the user. In addition, since the execution of the target video editing operation takes time, if the time taken for algorithm processing and the time taken for the rendering thread are not consistent, the rendering thread is likely to be blocked. Therefore, the present disclosure separates the executed video editing process and rendering and executes them on different threads, thereby solving the problem of the rendering thread being blocked due to the execution of the target video editing operation in the streaming framework, and is favorable for improving the preview effect.

[0083] 3 is a flowchart of a video processing method according to another embodiment of the present disclosure. As shown in FIG. 3, the method of this embodiment includes the following steps:

[0084] Based on the embodiment shown in FIG. 2, S201 in the embodiment shown in FIG. 2 can be realized by S301 to S306 in this embodiment.

[0085] S301: Determine whether there is a corresponding target video frame at a specified video frame position.

[0086] If there is no corresponding target video frame at the specified video frame position, execute S302. If there is a corresponding target video frame at the specified video frame position, execute S303.

[0087] S302: Determine a video editing processing mode corresponding to a first thread based on a specified video frame position.

[0088] Here, after determining the video editing processing mode corresponding to the first thread, S305 is executed.

[0089] S303: Determine whether the current first thread performs a target video editing operation on the target video.

[0090] If the current first thread is not performing a target video editing operation on the current video, execute S304 and S305. If the current first thread is performing a target video editing operation on the current video, execute S306.

[0091] S304: Determine the video editing processing mode as the second processing mode.

[0092] S305: The first thread performs a target video editing operation on the target video according to the determined video editing processing mode, and obtains each target video frame obtained by performing the target video editing operation.

[0093] S306: The first thread continues to perform the target video editing operation on the target video.

[0094] In a practical scenario, when the first thread starts to execute a target video editing operation according to one video editing processing mode or switches the video editing processing mode, the decoder needs to be updated, and a lot of computational resources are required for updating the decoder. Therefore, the method according to the present embodiment determines whether to switch the video editing processing mode by analyzing whether there is a corresponding target video frame at a specified video frame position and whether there is a process currently executing the target video editing operation on the target video (i.e., whether the current first thread is executing the target video editing operation on the target video), thereby reducing the waste of computational resources associated with frequent updates of the decoder and further improving the processing efficiency of executing the target video editing operation.

[0095] Here, the reason for analyzing whether a corresponding target video frame exists at the specified video frame position is that a target video editing operation may have been performed on the target video before, and therefore a corresponding target video frame may exist in some or all of the video frames in the target video. Taking the target video editing operation as an example where intelligent matting is the target video editing operation, it is assumed that a user generates a video editing command by operating a control corresponding to intelligent matting after introducing the target video, and the application performs intelligent matting on the target video according to the video editing command. If intelligent matting is not completely completed, the user inputs a cancel command to cancel intelligent matting on the target video. When canceling intelligent matting, since intelligent matting on the target video is not completely completed, a corresponding target video frame (i.e., a matting result) exists in some video frames.

[0096] In one possible case, if there is no corresponding target video frame at the specified video frame position, regardless of whether the current first thread is performing a target video editing operation on the target video, a video editing processing mode corresponding to the first thread needs to be determined based on the specified video frame position, and the first thread performs the target video editing operation on the target video from the specified video frame position according to the determined video editing processing mode.

[0097] Here, the implementation form for determining the video editing processing mode corresponding to the first thread based on the specified video frame position can be referred to the above content, and will not be described further here.

[0098] In another possible case, if a corresponding target video frame exists at the specified video frame position and there is no process currently performing a target effect on the video to be processed, since video encoding is performed in chronological order of video frames, it is determined whether a corresponding target video frame exists for each frame, starting from the frame next to the specified video frame position, until the first video frame position where no corresponding target video frame exists is determined, and the first thread can perform the target video editing operation on the video to be processed from the first video frame position where no corresponding target video frame exists according to the first processing mode or the second processing mode.

[0099] In this scenario, whether the first thread adopts the first processing mode or the second processing mode may be determined depending on whether the specified video frame position is a preset video frame position or a video frame position indicated by the first jump instruction.

[0100] In another possible case, assuming that there is a corresponding target video frame at the specified video frame position and there is a process currently performing a target effect on the target video, the process can continue to run. This eliminates the need to update the decoder, and can reduce resource consumption associated with updating the decoder due to switching of effect processing modes. For example, if the specified video frame position is the video frame position indicated by the second jump instruction, the first thread is performing a target video editing operation on the target video according to one video editing processing mode, and there may be a corresponding target video frame at the video frame position indicated by the second jump instruction, so there is no need to switch the video editing processing mode, i.e., the first thread is not interrupted.

[0101] Here, if the designated video frame position is a preset video frame position or a video frame position indicated by a first jump command, the video editing processing mode can be determined by referring to the above-mentioned method, and if the designated video frame position is a video frame position indicated by a second jump command, the video editing processing mode can be determined as a second processing mode.

[0102] S307: In response to a play command, a second thread performs rendering on each of the target video frames and presents each of the target video frames, where a video frame position at which the target video editing operation is being performed precedes a corresponding video frame position in the target video of the target video frame being presented.

[0103] Based on S305 or S306, while the first thread performs the target video editing operation on the target video, the application can perform playback by the second thread in response to the playback command. Specifically, step S307 in this embodiment is similar to step S203 in the embodiment shown in Figure 2, and reference may be made to the detailed description of the embodiment shown in Figure 2, and for the sake of simplicity, no further description will be given here.

[0104] The method according to the present embodiment determines whether to switch the video editing processing mode in a video editing scenario by analyzing whether there is a corresponding target video frame at a specified video frame position and whether there is a process currently performing a target video editing operation on the video being processed, thereby satisfying the user's preview needs, and not only solving the problem of the user's preview freezing, but also reducing the resource waste associated with frequent updates of the decoder and improving the processing efficiency of executing the target video editing.

[0105] 4 is a flowchart of a video processing method according to another embodiment of the present disclosure. As shown in FIG. 4, the method according to this embodiment includes the following steps:

[0106] S401: A first thread performs a target video editing operation on a video frame at a specified video frame position in a target video from the specified video frame position, and pre-performs a target video editing operation on each video frame after the specified video frame position to obtain each target video frame obtained by performing the target video editing operation.

[0107] S402: In response to a play command, a second thread performs rendering on each of the target video frames and presents each of the target video frames, where a video frame position at which the target video editing operation is being performed precedes a corresponding video frame position in the target video of the presented target video frame.

[0108] Steps S401 and S402 in this embodiment are similar to steps S101 and S102 in the embodiment shown in FIG. 1, respectively, and reference may be made to the detailed description of the embodiment shown in FIG. 1, and for the sake of brevity, no further description will be given here.

[0109] S403: In response to the play command, switch a video editing processing mode to a third processing mode, and perform a target video editing operation on the subject video according to the third processing mode by a first thread.

[0110] The third processing mode is a mode in which the video frame position at which the target effect is to be executed is determined according to the preview playback speed.

[0111] For example, assuming that the video to be processed has 100 frames, in steps S401-S402, the target video editing operation has already been performed on the first to tenth video frames according to the determined video editing processing mode, and the corresponding target video frames have been obtained. When the application receives a playback command, the second thread sequentially performs rendering on the target video frames corresponding to the first to tenth video frames, and plays the target video frames corresponding to the first to tenth video frames, respectively, according to the set playback speed. Assuming that the tenth frame has been currently played, it is detected that there is no corresponding target video frame in the next video frame to be played (i.e., the eleventh frame). If the time required to perform the target video editing operation is greater than the playback speed, performing the target video editing operation on the eleventh video frame in the order of the earliest video frames cannot meet the playback needs, which inevitably leads to serious playback freezes and further problems such as screen distortion. Therefore, the video editing processing mode is switched to a third processing mode, and the first thread determines the video frame position at which to perform the target video editing operation according to the playback speed, thereby ensuring that the video frame position at which the target video editing operation is being performed always precedes the corresponding video frame position in the processing target video of the presented target video frame, and by performing the target video editing operation on the video frame that precedes the preview playback progress, the preview playback needs are met and problems such as serious preview playback freezing and screen distortion are solved.

[0112] In addition, when the video editing processing mode is the third processing mode, the device may dynamically determine which video frame ahead of the playback progress to perform the target video editing operation on in combination with one or more factors such as the preview playback speed, the processing time length for performing the target video editing operation on a predetermined single video frame (i.e., the average time required for the target video editing operation), the video frame position at which the target video editing operation is currently being performed, and the video frame on which the target video editing operation has already been performed.

[0113] Continuing with reference to the example in the previous paragraph, for example, when it is determined that the first thread performs a target video editing operation on the 15th video frame according to the third processing mode, if the target effect has already been performed on the 15th frame, the target video editing operation may be performed on the 16th video frame in the order of the earliest video frames (obviously, it is not limited to the 16th frame, but may be another video frame that is a certain distance away from the 15th frame, for example, the 18th frame). If it is determined that there is no corresponding target video frame in the 16th video frame, the target video editing operation is performed on the 16th video frame. After performing the target video editing operation on the 16th video frame, the video frame position to perform the target video editing operation is still flexibly determined according to the third processing mode.

[0114] As can be seen from the above description and examples of the third processing mode, the third processing mode is a video editing processing mode responsive to a playback command. During playback of a target video frame, the third processing mode can ensure that the video frame position at which the target video editing operation is being performed always precedes the corresponding video frame position in the target video of the played target video frame, i.e., precedes the playback progress.

[0115] In this embodiment, the video editing processing mode may be switched to the third processing mode, and the switching timing may be determined by one or more of the following factors:

[0116] 1. The randomness of user operations, for example, the user may continuously input play commands and play pause commands; 2. Whether there is currently a process that is performing a target video editing operation on the target video; and 3. The target video frame obtained by performing the target video editing operation.

[0117] The influence of the above factors will now be described by way of example.

[0118] Assuming that the video to be processed has 100 frames, in steps S401 to S402, target video editing operations have already been performed on the first to tenth video frames according to the determined video editing processing mode, and target video frames corresponding to the first to tenth video frames, respectively, have been obtained.

[0119] Case (1): Assume that an application receives a play command, updates the decoder, immediately switches the video editing processing mode to the third processing mode, and plays the target video frame from the position of the first frame. Then, the application receives a play pause command. Because the interval time between the two commands, the play command and the play pause command, is short, the second thread may not have rendered until the tenth frame. If the process of the current video editing processing mode is interrupted, i.e., the first thread is interrupted, the decoder is updated again, and the video editing processing mode is switched, the consumption of computing resources will be large, and thus the video processing efficiency will be reduced.

[0120] Case (2): During playback, detect in real time whether there is a corresponding target video frame at the position of the next video frame to be played. If it is determined that there is no corresponding target video frame at the position of the next video frame to be played, and then switch the video editing processing mode, during playback from the 1st frame to the 10th frame, the target video editing operation can continue to be performed on the target video according to the original video editing processing mode, that is, the process of currently performing the target video editing operation is not interrupted, and the utilization rate of computing resources can be improved.

[0121] Further, based on case (2), when the application receives a playback pause command, the playback progress has not reached the progress of the target video editing operation. For example, when playing to the 8th frame, if the application receives a playback pause command, during the playback from the 1st frame to the 8th frame, the application also performs the target video editing operation on the 11th frame and the 12th frame according to the original video editing processing mode.

[0122] In the above case (2), the application receives a play command and a play pause command, but there are corresponding target video frames in the part to be played, so there is no need to switch the video editing processing mode corresponding to the first thread, which can also ensure smooth playback and reduce the large consumption of computing resources caused by switching the video editing processing mode. Meanwhile, during playback, the first thread also performs target video editing operations on some video frames according to the original video editing processing mode, thereby improving video processing efficiency.

[0123] As is apparent from the above description, in this step, the application may determine the timing of switching the video editing processing mode based on one or more of the above factors in response to the playback command.

[0124] The method according to the present embodiment can receive a playback command while performing a target video editing operation on a target video by a first thread, and in response to the playback command, switch the video editing processing mode to a third processing mode to perform a target video editing operation on a video frame of the target video, and ensure that the video frame position where the target video editing operation is being performed during playback always precedes the playback progress by using the third processing mode, thereby solving the problem of serious freezing of the user preview playback and screen distorts, improving the efficiency of video processing and favoring the improvement of user experience.

[0125] Optionally, based on the embodiment shown in FIG. 4, the method further includes the following steps:

[0126] S404: A playback pause command is received.

[0127] S405: In response to a playback pause command, the second thread pauses playback, and the first thread continues to perform target video editing operations on video frames of the target video according to a third processing mode.

[0128] Specifically, when the application receives a playback pause command, the second thread stops rendering the target video frame, that is, pauses the playback of the target video frame.

[0129] When the playback pause command is received, there is a process performing a target effect on the target video, so the process does not need to be interrupted. That is, the first thread continues to perform the target video editing operation on the target video. That is, the first thread continues to perform the target video editing operation on the video frames of the target video according to the third processing mode. In this way, the computing resources of the electronic device can be effectively utilized to perform video editing, and the utilization rate of the computing resources is improved. Thus, the video processing efficiency is improved.

[0130] Hereinafter, some possible implementation forms corresponding to step S403 will be described with reference to the examples shown in FIGS.

[0131] 5 is a flowchart of a video processing method according to another embodiment of the present disclosure. As shown in FIG. 5, the method of this embodiment includes the following steps:

[0132] S501: A first thread performs a target video editing operation on a video frame at a specified video frame position in a video to be processed, starting from the specified video frame position, and pre-performs a target video editing operation on each video frame after the specified video frame position to obtain each target video frame obtained by performing the target video editing operation.

[0133] S502: In response to a play command, a second thread performs rendering on each of the target video frames and presents each of the target video frames, where a video frame position at which the target video editing operation is being performed precedes a corresponding video frame position in the target video of the presented target video frame.

[0134] Steps S501 and S502 in this embodiment are similar to steps S401 and S402 in the embodiment shown in FIG. 4, respectively, and reference may be made to the detailed description of the embodiment shown in FIG. 4. For the sake of brevity, no further description will be given here.

[0135] S503: In response to the playback command, perform a target video editing operation on a video frame of the processing target video according to a third processing mode from the video frame position where the target video editing operation is being performed when the preview playback command is obtained.

[0136] In this step, when the first thread gets the playback command, the first thread may be performing a target video editing operation on a certain video frame. In such a case, the video frame position at which the video editing processing mode is switched is the video frame position at which the target video editing operation is being performed. The application may have just performed the target video editing operation on a certain video frame, but may not have started to perform the target video editing operation on the next video frame. In such a case, the video frame position at which the video editing processing mode is switched is the position of the next video frame at which the target video editing operation is to be performed.

[0137] Here, when the application obtains a playback command, it may immediately switch the video editing processing mode corresponding to the first thread to a third processing mode, and perform a target video editing operation on a video frame of the target video according to the third processing mode from the position of the video frame determined to switch the video editing processing mode.

[0138] The execution order of S502 and S503 does not matter.

[0139] The method according to the present embodiment obtains a playback command while performing a target video editing operation on the target video, and in response to the playback command, a first thread performs a target video editing operation on a video frame of the target video according to a third processing mode from the video frame position where the target video editing operation is being performed when the playback command is obtained. The implementation according to the present embodiment has a simple logic and does not require complex judgment. In addition, the third processing mode can ensure that the video frame position where the target video editing operation is being performed during playback always precedes the playback progress. This solves the problem of the preview playback performed by the user becoming serious and causing the screen to collapse, improves the video processing efficiency, and is favorable to improving the user experience.

[0140] Optionally, based on the embodiment shown in FIG. 5, the method further includes the following steps:

[0141] S504: A playback pause command is obtained.

[0142] S505: In response to a playback pause command, the second thread pauses playback, and the first thread continues to perform target video editing operations on video frames of the target video according to a third processing mode.

[0143] Specifically, when the application receives a playback pause command, the second thread stops rendering the target video frame, that is, pauses the playback of the target video frame.

[0144] When the playback pause command is received, there is a process performing a target effect on the target video, so the process does not need to be interrupted. That is, the first thread continues to perform the target video editing operation on the target video. That is, the first thread continues to perform the target video editing operation on the video frames of the target video according to the third processing mode. In this way, the computing resources of the electronic device can be effectively utilized to perform video editing, and the utilization rate of the computing resources is improved. Thus, the video processing efficiency is improved.

[0145] 6 is a flowchart of a video processing method according to another embodiment of the present disclosure. As shown in FIG. 6, the method according to this embodiment includes the following steps:

[0146] S601: A first thread performs a target video editing operation on a video frame at a specified video frame position in a target video from the specified video frame position, and pre-performs a target video editing operation on each video frame after the specified video frame position to obtain each target video frame obtained by performing the target video editing operation.

[0147] S602: In response to a play command, a second thread performs rendering on each of the target video frames and presents each of the target video frames, where a video frame position at which the target video editing operation is being performed precedes a corresponding video frame position in the target video of the target video frame being presented.

[0148] Steps S601 and S602 in this embodiment are similar to steps S401 and S402 in the embodiment shown in FIG. 4, respectively, and reference may be made to the detailed description of the embodiment shown in FIG. 4. For the sake of brevity, no further description will be given here.

[0149] S603: In response to a playback command, determine a switching position corresponding to a third processing mode based on a processing time length for performing a target video editing operation on a predetermined single video frame, a playback speed, and a video frame position at which the target video editing operation is currently being performed.

[0150] For example, a maximum time length for which the current video editing processing mode can be continuously executed may be determined based on the position of the video frame where the target video editing operation is currently being executed and the playback speed. Then, based on the determined maximum time length and a processing time length for which the target video editing operation is executed on a single video frame set in advance, a number S of video frames that can be processed in the current video editing processing mode within the maximum time length is determined. Next, a switching position of the video editing processing mode is determined from among S video frames consecutive from the video frame position where the target video editing operation is currently being executed. Here, the switching position of the video editing processing mode may be any one of the S video frames consecutive from the video frame position where the target video editing operation is currently being executed.

[0151] In some possible implementations, in order to reduce errors in switching video editing processing modes, for example, switching timing being too late such that the processing progress of the target video editing operation does not meet the rendering progress, a video frame among S consecutive video frames that is close to the video frame on which the target video editing operation is currently being performed may be set as the switching position.

[0152] It should be understood that, when determining the maximum duration, since the target video editing operation for the video frame on which the target video editing operation is currently being performed has not been completed, the maximum duration may be calculated from a frame immediately preceding the video frame on which the target video editing operation is currently being performed, thereby improving the accuracy of the calculation result.

[0153] The execution order of S602 and S603 does not matter.

[0154] S604: Perform, by a first thread, the target video editing operation on the video frames of the target video according to a third processing mode from the switching position.

[0155] Here, for details of the third processing mode, please refer to the description in the embodiment shown in Figure 4. For the sake of brevity, no further description will be given here.

[0156] The method according to the present embodiment obtains a playback command while performing a target video editing operation on a video to be processed. In response to the playback command, a switching position of a video editing processing mode is flexibly determined based on the time length required to perform the target video editing operation on a single video frame, the playback speed, and the video frame position where the target video editing operation is currently being performed. When the progress of the target video editing operation reaches the switching position, the video editing processing mode is switched to a third processing mode, improving the flexibility of switching the video editing processing mode. In addition, the third processing mode can ensure that the video frame position where the target video editing operation is being performed during playback always precedes the playback progress. This solves the problem of the preview playback performed by the user becoming serious and causing the screen to collapse, improves the video processing efficiency, and is favorable to improving the user experience.

[0157] Optionally, based on the embodiment shown in FIG. 6, the method further includes the following steps:

[0158] S605: A playback pause command is obtained.

[0159] S606: In response to a playback pause command, the second thread pauses playback, and the first thread continues to perform target video editing operations on video frames of the target video according to a third processing mode.

[0160] Specifically, when the application receives a playback pause command, the second thread stops rendering the target video frame, that is, pauses the playback of the target video frame.

[0161] When the playback pause command is received, there is a process performing a target effect on the target video, so the process does not need to be interrupted. That is, the first thread continues to perform the target video editing operation on the target video. That is, the first thread continues to perform the target video editing operation on the video frames of the target video according to the third processing mode. In this way, the computing resources of the electronic device can be effectively utilized to perform video editing, and the utilization rate of the computing resources is improved. Thus, the video processing efficiency is improved.

[0162] 7 is a flowchart of a video processing method according to another embodiment of the present disclosure. As shown in FIG. 7, the method according to this embodiment includes the following steps:

[0163] S701: A first thread performs a target video editing operation on a video frame at a specified video frame position in a video to be processed, from the specified video frame position, and pre-performs a target video editing operation on each video frame after the specified video frame position to obtain each target video frame obtained by performing the target video editing operation.

[0164] S702: In response to a play command, a second thread performs rendering on each of the target video frames and presents each of the target video frames, where a video frame position at which the target video editing operation is being performed precedes a corresponding video frame position in the target video of the presented target video frame.

[0165] Steps S701 and S702 in this embodiment are similar to steps S401 and S402 in the embodiment shown in FIG. 4, respectively, and reference may be made to the detailed description of the embodiment shown in FIG. 4. For the sake of brevity, no further description will be given here.

[0166] S703: In response to a playback command, detect in real time whether there is a corresponding target video frame in the next video frame to be played.

[0167] If it is detected that there is no corresponding target video frame in the next video frame to be played, then S704 is executed.

[0168] The execution order of S702 and S703 does not matter.

[0169] S704: Performing a target video editing operation on a video frame of the processing target video according to a third processing mode by the first thread from the position of the next target video frame to be played.

[0170] It should be noted that performing the target video editing operation according to the third processing mode from the position of the next target video frame to be played does not mean that the next video frame to be performed the target video editing operation is a video frame corresponding to the next video frame position to be played. For example, if the next video frame to be played is the 10th frame of the to-be-processed video frames, but there is no corresponding target video frame at the 10th frame, the first thread performs the target video editing operation according to the third processing mode from the 10th frame. However, the determined video frame position to perform the target video editing operation may be the 13th frame, the 15th frame, etc. in the to-be-processed video. This ensures that the video frame position at which the target video editing operation is being performed is ahead of the playback progress.

[0171] In this embodiment, a playback command is received while a target video editing operation is being performed on a video to be processed. In response to the preview playback command, when it is detected that there is no corresponding target video frame in the next video frame to be played, the video editing processing mode is switched. This can minimize resource consumption caused by frequent switching of the video editing processing mode due to frequent user operations. In addition, the playback processing mode can ensure that the video frame position where the target video editing operation is being performed during playback is always ahead of the playback progress. This solves the problem of serious freezes of preview playback performed by the user, causing screen distorts, etc., improves video processing efficiency, and is favorable to improving user experience.

[0172] Optionally, based on the embodiment shown in FIG. 7, the method further includes the following steps:

[0173] S705: A playback pause command is obtained.

[0174] S706: In response to a playback pause command, the second thread pauses playback, and the first thread continues to perform target video editing operations on video frames of the target video according to a third processing mode.

[0175] Specifically, when the application receives a playback pause command, the second thread stops rendering the target video frame, that is, pauses the playback of the target video frame.

[0176] When the playback pause command is received, there is a process performing a target effect on the target video, so the process does not need to be interrupted. That is, the first thread continues to perform the target video editing operation on the target video. That is, the first thread continues to perform the target video editing operation on the video frames of the target video according to the third processing mode. In this way, the computing resources of the electronic device can be effectively utilized to perform video editing, and the utilization rate of the computing resources is improved. Thus, the video processing efficiency is improved.

[0177] FIG. 8 is a flowchart of a video processing method according to another embodiment of the present disclosure.

[0178] Here, to support video frame position jumping, the application can also obtain a jump command while the application is playing each target video frame, i.e., the method of this embodiment can be performed based on any one of the embodiments of Figures 1 to 7 above.

[0179] As shown in FIG. 8, the method of this embodiment includes the following steps:

[0180] S801: Obtain the third jump instruction.

[0181] S802: In response to the third jump command, re-determine a video editing processing mode depending on whether a corresponding target video exists at the video frame position indicated by the third jump command and whether the first thread is performing the target video editing operation on the video to be processed.

[0182] S803: The first thread performs a target video editing operation on a video frame of the target video according to the redetermined video editing processing mode.

[0183] A second thread performs rendering on a target video frame corresponding to the video frame position indicated by the jump instruction, and presents a target video frame corresponding to the video frame position indicated by a third jump instruction.

[0184] In some cases, when a user inputs a third jump command during playback, the application may switch the playback state to playback paused. In other cases, when a user inputs a third jump command during playback, the application may play the target video frame from the video frame position indicated by the third jump command.

[0185] In this embodiment, when the application switches the playback state to a playback pause state in response to the third jump command, there may be several cases: Case (a): If there is a corresponding target video frame at the video frame position indicated by the third jump command, and there is a process currently performing a target video editing operation on the target video, then the current process is not interrupted, and the first thread continues to perform the target video editing operation on the target video according to the original video editing processing mode.

[0186] Since there is a corresponding target video frame at the video frame position indicated by the third jump command, the application can present the target video frame corresponding to the video frame position indicated by the third jump command. Since the application can be in preview playback pause state, no freezing phenomenon occurs. Based on this, the application can continue to execute the ongoing process.

[0187] Case (b): Since a corresponding target video frame exists at the video frame position indicated by the third jump command, but there is no process currently performing the target effect on the video to be processed, it may be determined whether a corresponding target video frame exists for each frame from the frame next to the video frame position indicated by the third jump command, and the target video editing operation may be performed on the video to be processed according to the second processing mode from the first video frame position where the determined corresponding target video frame does not exist.

[0188] Case (c): If there is no corresponding target video frame at the video frame position indicated by the third jump command, perform the target video editing operation on the video frame of the processing target video from the video frame position indicated by the third jump command according to the second processing mode.

[0189] In response to the third jump command, when the application starts playing the target video frame from the video frame position indicated by the third jump command, the video editing processing mode is switched to the third processing mode, and when the application starts playing the target video frame from the video frame position indicated by the third jump command, the video frame position where the target video editing operation is being performed always precedes the current playback progress, thereby realizing that no freezing occurs in the playback, and meeting the user's needs for preview.

[0190] Here, the implementation form for determining the timing of switching the video editing processing mode may adopt the form shown in any one of the embodiments shown in the above-mentioned Figures 5 to 7, and it is sufficient to refer to the detailed explanation of the embodiment shown in the above-mentioned Figures 5 to 7, and for the sake of simplicity, no further explanation will be given here.

[0191] The method according to the present embodiment determines whether the video editing process mode needs to be switched by analyzing whether there is a corresponding target video frame at the video frame position indicated by the third jump command, and whether there is a process in which the target video editing operation is currently being performed, and also considers the playback state when determining whether to switch the video editing process mode to ensure a real-time display effect.

[0192] 9 is a flowchart of a video processing method according to another embodiment of the present disclosure. As shown in FIG. 9, the method of this embodiment includes the following steps:

[0193] S901: A first thread performs a target video editing operation on a video frame at a specified video frame position in a video to be processed, starting from the specified video frame position, and pre-performs a target video editing operation on each video frame after the specified video frame position to obtain each target video frame obtained by performing the target video editing operation.

[0194] S902: In response to a play command, a second thread performs rendering on each of the target video frames and presents each of the target video frames, where a video frame position at which the target video editing operation is being performed precedes a corresponding video frame position in the target video of the target video frame being presented.

[0195] Steps S901 and S902 in this embodiment are similar to steps S101 and S102 in the embodiment shown in FIG. 1, respectively, and reference may be made to the detailed description of the embodiment shown in FIG. 1, and for the sake of brevity, no further description will be given here.

[0196] S903: After performing the target video editing operation on the last video frame of the to-be-processed video, a first thread performs the target video editing operation on video frames of the to-be-processed video according to a fourth processing mode from the position of the first video frame of the to-be-processed video.

[0197] Wherein, the fourth processing mode is a mode in which the target video editing operation is sequentially performed on a video frame in the to-be-processed video that has no corresponding target video frame, The fourth processing mode is an adaptive video editing processing mode.

[0198] When the application executes according to any one or more video editing processing modes among the first processing mode to the third processing mode up to the last video frame position of the video to be processed, it may switch to a fourth processing mode and make full use of the computing resources to sequentially perform target video editing operations on video frames in the video to be processed for which no corresponding target video frames exist.

[0199] Next, a case where the video to be processed includes one video clip and a case where the video to be processed includes multiple video clips will be described separately.

[0200] 1. The video to be processed contains one video clip.

[0201] Assuming that the processing target video includes one video clip, the video clip is referred to as video clip A1. When it is detected that the target video editing operation has been performed on the last video frame of the video clip A1, the video editing processing mode is switched to a fourth processing mode, and the target video editing operation is sequentially performed on the video frames on which the target video editing operation has not been performed, starting from the first video frame position of the video clip A1, in order of the earliest video frames in the video clip A1.

[0202] For example, a video clip A1 includes 100 frames. Assume that a target video editing operation is performed on the 50th frame to the 100th frame according to the second processing mode. Upon receiving the completion of the target video editing operation on the 100th frame, the processing mode is switched to the fourth processing mode, and the target video editing operation is performed on the 1st frame to the 49th frame, frame by frame, starting from the 1st frame.

[0203] Also, for example, video clip A1 includes 100 frames. Assume that the target video editing operation is performed on the first frame, the fifth frame, the tenth frame, the fifteenth frame, and the twentieth frame according to the first processing mode, and from the thirtieth frame, the processing mode is switched to the second processing mode, and the target video editing operation is performed on the thirtieth to the hundredth frames. When the completion of the execution of the target video editing operation on the hundredth frame is detected, the processing mode is switched to the fourth processing mode, and the target video editing operation is sequentially performed from the first frame on the second to fourth frames, the sixth to ninth frames, the eleventh to fourteenth frames, and the sixteenth to nineteenth frames on which the target video editing operation has not been performed.

[0204] 2. The video to be processed contains multiple video clips.

[0205] Assuming that the video to be processed includes a plurality of video clips, the plurality of video clips are respectively designated as video clips B1 to B2. N Video Clip B n and detecting that the target video editing operation has been performed on the last video frame of video clip B, switch the video editing processing mode to a fourth processing mode, and n From the first video frame position of video clip B n The target video editing operation is sequentially performed on the video frames on which the target video editing operation has not been performed, in the order of the earliest video frames in , where N is an integer equal to or greater than 2. n is an integer equal to or greater than 1 and equal to or less than N.

[0206] In other words, when the video to be processed includes multiple video clips, when it is detected that a target video editing operation has been performed on the last video frame of a certain video clip, an effect processing task of the fourth processing mode may be preferentially executed within the video clip.

[0207] In practical application, each video clip corresponds to a decoder, and executes the target video editing operation task of the fourth processing mode in the video clip, so that the consumption of computing resources caused by updating the decoder due to switching of the video clip can be reduced, and the target video editing operation task of the fourth processing mode can be executed in the video clip, so that the computing resources can be fully utilized and the video processing efficiency can be improved.

[0208] For example, assume that the video to be processed includes three video clips, which are video clips B1, B2, and B3. Currently, the target video editing operation is performed on the video clip B2 according to the second processing mode. When it is detected that the target video editing operation is performed on the last video frame of the video clip B2, the processing mode is switched to the fourth processing mode, and the target video editing operation is sequentially performed on the video frames in the video clip B2 that do not have a corresponding target video frame, starting from the first video frame position of the video clip B2.

[0209] If there are corresponding target video frames in all video frames in video clip B2, video clip B2 may be marked as "completed". If no other operation command has been received, the target video editing operation may be performed on video clip B1 and video clip B3.

[0210] Here, an example will be described in which the target video editing operation for video clip B2 is completed and the next video clip to be processed is video clip B1.

[0211] It is possible that a target video editing operation has never been performed on the video clip B1. Therefore, there is no corresponding target video frame in any video frame in the video clip B1. In this case, the target video editing operation may be first performed sequentially on each video frame of the video clip B1 according to the first processing mode from the first video frame position of the video clip B1. Based on this, after detecting that the target video editing operation has been performed on the last video frame of the video clip B1, the processing mode is switched to the fourth processing mode, and the target video editing operation is sequentially performed on the video frames in the video clip B1 that have no corresponding target video frame.

[0212] It is possible that a target video editing operation has been performed on the video clip B1. Therefore, there are corresponding target video frames in some video frames in the video clip B1. In this case, the target video editing operation is sequentially performed on the video frames in the video clip B1 that do not have corresponding target video frames, starting from the first video frame position of the video clip B1, according to the fourth processing mode.

[0213] If there are corresponding target video frames in all video frames in the video clip B1, the video clip B1 is marked as "completed". Based on the above marking, it can be determined that there is no need to perform a target video editing operation on the video clip B1.

[0214] When the next video clip to be processed is video clip B3, the implementation form is similar to the implementation form when the next video clip to be processed is video clip B1, and reference may be made to the detailed description provided above, and no further description will be given here.

[0215] In the method of this embodiment, when it is detected that a target video editing operation has been performed on the last video frame of the to-be-processed video while performing a target video editing operation on a video frame of the to-be-processed video, the method switches the video editing processing mode to a fourth processing mode, and sequentially performs the target video editing operation on video frames in the to-be-processed video for which no corresponding target video frame exists.

[0216] 10 is a schematic framework diagram of a video processing device according to an embodiment of the present disclosure. Referring to the video processing device shown in FIG. 10, it includes three entity layers, which are a graph layer, an interaction layer, and an algorithm processing layer.

[0217] The graph layer includes a decoding thread and a rendering thread, where the decoding thread includes a decoder reader unit, and the rendering thread includes a clip preprocessing unit.

[0218] The decoding control unit is for updating the algorithm processing tasks being executed in the algorithm processing layer and synchronously updating the rendering position according to operation commands input by a user, such as jump commands, play commands, etc.

[0219] The video pre-processing unit is for judging whether there is a corresponding target video frame at the current video frame position, and if there is, for reading the target video frame, uploading texture, rendering, and displaying on the screen; if not, for reading the data of the corresponding target video frame from the previous video frame on which the target video editing operation is performed, uploading texture, rendering, and displaying on the screen.

[0220] The interaction layer includes a task scheduling module and a cache control module.

[0221] Here, the task scheduling module is for generating video editing tasks and distributing the video editing tasks to the algorithm processing layer.

[0222] In addition, the task scheduling module may generate the video editing task based on an operation command input by a user, or may generate the video editing task based on information for switching the video editing processing mode to the self-expanding processing mode according to an instruction returned from the algorithm processing layer. Therefore, the task scheduling module may be regarded as performing task scheduling through an external trigger policy and a self-expanding policy.

[0223] In the overall framework of the video processing device, an algorithm task is the smallest processing unit. Every time a video editing operation for a clip is added, one algorithm task, i.e., an algorithm task (task Param) object, is created and one task identifier (task ID) is returned. One algorithm task object uniquely corresponds to one task identifier (task ID).

[0224] An algorithm task (task Param) object is a structure in which all parameters required for algorithm processing in one clip are recorded, and which has a reference to one algorithm processing instance (task Process wrapper).

[0225] The algorithm processing instance (task process wrapper) has a Lab algorithm model instance that corresponds to it one-to-one and manages the corresponding contents. The algorithm processing instance holds one main file (MANE File), records information about all algorithm results (i.e., target video frames) of this path file (e.g., timestamp information indicating the video frame position, e.g., PTS information), and provides queries to the interface.

[0226] In practical applications, the Lab algorithm model instance has a large amount of data. For example, in intelligent matting, the Lab algorithm model instance is about 100M. In order to reduce the occurrence of out of memory (OOM), the Lab algorithm model instance is discarded immediately after use.

[0227] The cache control module is for returning a target video frame to the rendering thread based on the target video frame stored in the cache, and for returning an access path in the external memory space of the target video frame to the rendering thread if the corresponding target video frame does not exist in the cache area, and for returning NULL to the rendering thread if the target video frame does not exist in the external memory space.

[0228] The algorithmic layer may be referred to by other names, such as an algorithmic interaction layer, an intermediate layer, etc., and this disclosure is not limited in this respect.

[0229] The algorithm processing layer includes an algorithm task management module, which provides an access interface to the outside, for example, the algorithm task management module provides an interface for adding, deleting, and querying effect processing tasks and progress.

[0230] The algorithm task management module itself further has algorithm task (task Param) objects, algorithm process instances, and algorithm threads.

[0231] Here, the essence of an algorithm thread is a thread object, which is a thread selected from a thread pool and executes effect processing tasks in the corresponding message queue.

[0232] When the algorithm thread performs a video editing task, it may include the following steps.

[0233] Step 1: Update the appropriate decoder.

[0234] Step 2: Correct the total number of frames corresponding to the video editing task.

[0235] Step 3: Determine whether the number of processed frames is equal to the total number of frames, or whether the start position of the target video editing operation is the last video frame of the video to be processed.

[0236] Whether the number of processed frames is equal to the total number of frames corresponds to condition 1 in FIG. 10, and whether the start position of the target video editing operation is the last video frame of the video to be processed corresponds to condition 2 in FIG.

[0237] Step 4: If none of the above conditions are met, determine whether a corresponding target video frame already exists for the video frame, and if so, skip performing the target video editing operation for the video frame; if not, perform step 5.

[0238] Step 5: According to the corresponding video editing processing mode, determine whether the video frame is a video frame to be discarded.

[0239] The video editing processing mode may be any one of a playback processing mode, a jump processing mode, a pre-processing mode, and a self-expansion processing mode. The meanings of the various video editing processing modes mentioned above can be referred to in the above description.

[0240] If the video editing processing mode is a playback processing mode, determine whether the video frame is a video frame to be discarded according to the playback speed. If it is a video frame to be discarded, there is no need to perform the target video editing operation on it; if it is not a video frame to be discarded, execute step 6.

[0241] When the video editing processing mode is a jump processing mode, none of the video frames should be discarded.

[0242] If the video editing processing mode is the pre-processing mode, the key frames are the video frames that should not be discarded, and execute step 6. The other frames are the video frames that should be discarded.

[0243] When the video editing processing mode is the self-expansion mode, any video frame for which there is no corresponding target video frame is a video frame that should not be discarded, and any video frame for which there is a corresponding target video frame is a video frame that should be discarded.

[0244] Step 6: Perform a target video editing operation on the video frame to obtain a corresponding target video frame.

[0245] Step 7: Store the target video frame corresponding to the video frame.

[0246] Here, the target video frame corresponding to the video frame may be stored in the external memory space and the cache at the same time. When rendering in the rendering thread, if the corresponding target video frame exists in the cache, the data may be read directly from the cache. If the corresponding target video frame does not exist in the cache, the corresponding target video frame is read from the external memory space. In this way, the number of data input / output can be reduced, and the video processing efficiency can be improved.

[0247] Furthermore, when storing a target video frame corresponding to a video frame, information about the target video frame may be passed to a callback service, which may then invoke a service thread to perform related operations. For example, while performing a target video edit, the callback service may provide feedback to the service thread about the progress of the execution of the current target video edit.

[0248] By adopting the framework shown in FIG. 10 above, the algorithm thread can perform target video editing operations on the target video according to multiple video editing processing modes, and ensure that the execution progress of the algorithm thread always precedes the progress of the rendering thread, thereby solving the problem of the user preview freezing. In addition, the algorithm thread can be executed independently after accepting the task of performing the target video editing operation on the video frame of the target video, and does not affect the upper coding thread and rendering thread, thereby realizing the parallel processing between the coding thread, the algorithm thread, and the rendering thread, and solving the problem of the rendering thread being blocked due to the algorithm thread taking a long time to process, and solving the problem of the user preview freezing.

[0249] Fig. 11 is a schematic diagram of the life cycle of each thread in the embodiment shown in Fig. 10 according to one embodiment of the present disclosure. As shown in Fig. 11, a frame skipping thread, a rendering thread, a service thread, and an algorithm thread are included.

[0250] The frame skip thread is for performing synchronous frame skipping. The rendering thread is for rendering the target video frame for playback. The service thread is for controlling the algorithm thread to perform the target video editing operation for each video frame of the target video according to the determined video editing processing mode, and for controlling the pausing and starting of playback. The algorithm thread is for performing the target video editing operation for the corresponding video frame according to the instruction of the service thread.

[0251] Upon receiving a video editing command to perform a target video editing operation on each video frame of the target video, the service thread first sets the video editing processing mode to a pre-processing mode or a jump processing mode, and controls the algorithm thread to start, so as to run the algorithm according to the set video editing processing mode, to perform the target video editing operation on the video frames of the target video. As shown in Figure 11, this process requires sequential buffering of the corresponding target video frames.

[0252] When playback is required, the service thread first suspends the algorithm thread, sets the video editing processing mode to the playback processing mode, and controls the algorithm thread to run the algorithm according to the playback processing mode to perform the target video editing operation on the video frames of the target video. As shown in Figure 11, this process still requires sequential buffering of the corresponding target video frames. Synchronously, the service thread controls the rendering thread to read the corresponding target video frames from the cache, and render and display them for user preview.

[0253] When the preview playback is paused, the service thread takes control of the rendering thread and pauses the playback.

[0254] When jumping a video frame position, the service thread first suspends the algorithm thread, sets the video editing processing mode to a jump processing mode, controls the algorithm thread, and runs the algorithm according to the jump processing mode to perform a target video editing operation on the video frame of the target video from the final jumped video frame position. As shown in Figure 11, this process still requires synchronously buffering the corresponding target video frame from the final jumped video frame position. Synchronously, the service thread controls the rendering thread to present the target video frame at the corresponding video frame position.

[0255] After the target video editing operation is performed on the last video frame of the target video, the service thread sets the video editing processing mode to a self-expanding processing mode, and controls the algorithm thread to sequentially perform the target video editing operation on the video frames on which the target video editing operation has not been performed, starting from the first video frame position of the target video. As shown in Figure 11, this process still requires sequential buffering of the corresponding target video frames.

[0256] When the execution of the target video editing operation on the target video frame is cancelled or stopped, the service thread takes control of the algorithm thread, stops the running of the algorithm, and reclaims the resources occupied by the algorithm thread, and then reclaims the rendering thread and the algorithm thread.

[0257] As shown in Figures 10 and 11, through the framework shown in Figure 10, in combination with the inter-thread interactions shown in Figure 11, the execution of the target video editing operation for each video frame of the target video is realized, and the problem of the user's preview playback freezing is effectively solved.

[0258] 12 is a schematic diagram of a configuration of a video processing device according to an embodiment of the present disclosure. As shown in FIG. 12, a video processing device 1200 according to this embodiment includes: a first processing module 1210 for performing a target video editing operation from a designated video frame position on a video frame at the designated video frame position in a processing target video by a first thread, and pre-performing the target video editing operation on each video frame after the designated video frame position to obtain each target video frame obtained by performing the target video editing operation; and a second processing module 1220 for rendering, by a second thread, each of the target video frames in response to a playback command and presenting each of the target video frames, wherein a video frame position at which the target video editing operation is being performed precedes a corresponding video frame position in the processed video of the target video frame being presented.

[0259] As a possible embodiment, the video processing device 1200 may further include a display module 1240. The display module 1240 is for presenting each target video frame.

[0260] In one possible embodiment, the specified video frame position is determined based on a video frame position located when a video editing command is obtained, the video editing command being for instructing the execution of the target video editing operation on the target video, and the video frame position located when the video editing command is obtained is a preset video frame position or a video frame position specified by a first jump command.

[0261] Accordingly, the video processing device 1200 may further include an acquisition module 1230 .

[0262] The obtaining module 1230 is for obtaining a video editing instruction, and obtaining a first jump instruction.

[0263] In one possible embodiment, the specified video frame position is determined based on a video frame position specified in a second jump command, and the second jump command is an instruction following a video editing instruction for performing the target video editing operation on the target video.

[0264] Accordingly, the acquiring module 1230 is further used to acquire a second jump instruction.

[0265] As one possible embodiment, when the specified video frame position is determined based on the video frame position located when the video editing command is acquired, the first processing module 1210 is used to execute a target video editing operation from the specified video frame position to the video frame at the specified video frame position in the video to be processed by a first thread, and to pre-execute the target video editing operation on each video frame after the specified video frame position before obtaining each target video frame obtained by executing the target video editing operation, and further to determine a video editing processing mode corresponding to the first thread based on the specified video frame position, wherein the video editing processing mode is a first processing mode which is a mode of executing the target video editing operation on a key frame, or a second processing mode which is a mode of executing the target video editing operation for each frame.

[0266] As a possible embodiment, the first processing module 1210 is specifically used for determining a video editing processing mode corresponding to the first thread as the first processing mode when the specified video frame position is the preset video frame position, and for determining a video editing processing mode corresponding to the first thread as the first processing mode or the second processing mode when the specified video frame position is a video frame position specified by a first jump command.

[0267] As a possible embodiment, the first processing module 1210 is specifically used for determining whether a corresponding target video frame exists at the specified video frame position, and if there is no corresponding target video frame at the specified video frame position, determining a video editing processing mode corresponding to the first thread according to whether the specified video frame position is the preset video frame position or a video frame position specified by the first jump command.

[0268] In one possible embodiment, the first processing module 1210 is specifically used for determining a video editing processing mode corresponding to the first thread as the second processing mode when a corresponding target video frame exists at the specified video frame position.

[0269] Accordingly, the second processing module 1220 is specifically used to determine whether there is a corresponding target video frame for each frame after the specified video frame position until a first video frame position where no corresponding target video frame exists is determined, and to perform the target video editing operation on the video frames of the processing target video from the first video frame position where no corresponding target video frame exists according to the second processing mode by the first thread.

[0270] As a possible embodiment, when the specified video frame position is determined based on the video frame position specified by the second jump command, the first processing module 1210 performs a target video editing operation on the video frame at the specified video frame position in the to-be-processed video from the specified video frame position by a first thread, and pre-performs the target video editing operation on each video frame after the specified video frame position before obtaining each target video frame obtained by performing the target video editing operation; and when it is determined that there is no corresponding target video frame at the specified video frame position, the first processing module 1210 further performs a video editing processing mode corresponding to the first thread as a second processing mode.

[0271] In one possible embodiment, if a corresponding target video frame exists at the specified video frame position and the first thread is performing the target video editing operation on the target video, the first processing module 1210 does not interrupt the first thread and allows the first thread to continue performing the target video editing operation on the target video.

[0272] As a possible embodiment, the first processing module 1210 is further used to switch the video editing processing mode to a third processing mode in response to the playback command, the third processing mode being a mode for determining a video frame position at which the target video editing operation is to be performed according to a playback speed, and for the first thread to perform the target video editing operation on a video frame of the processing target video according to the third processing mode.

[0273] As one possible embodiment, the first processing module 1210 is specifically used to detect in real time whether there is a next target video frame to be played in response to the playback command, and when it detects that there is no next target video frame to be played, to execute the target video editing operation on the video frames of the processing target video according to the third processing mode from the position of the next target video frame to be played by the first thread.

[0274] As one possible embodiment, the first processing module 1210 is specifically used to, in response to the playback command, execute the target video editing operation on a video frame of the target video according to the third processing mode by the first thread from the video frame position where the target video editing operation was being executed when the playback command was obtained.

[0275] As one possible embodiment, the first processing module 1210 is specifically used to determine a switching position corresponding to the third processing mode based on a processing time length for performing the target video editing operation on a predetermined single video frame, a playback speed, and a video frame position at which the target video editing operation is currently being performed in response to the playback command, and to execute the target video editing operation on the video frame of the processing target video from the switching position according to the third processing mode by the first thread.

[0276] As one possible embodiment, the first processing module 1210 is specifically further adapted to pause rendering of the target video frame by the second thread in response to the playback pause command, and the first thread continues to perform the target video editing operation on the video frame of the processing target video according to the third processing mode.

[0277] Accordingly, the obtaining module 1230 is further used to obtain a playback pause command.

[0278] In one possible embodiment, the first processing module 1210 is further used to, after performing the target video editing operation on the last video frame of the to-be-processed video, perform the target video editing operation on video frames of the to-be-processed video from the position of the first video frame of the to-be-processed video according to a fourth processing mode by the first thread, wherein the fourth processing mode is a mode for sequentially performing the target video editing operation on video frames in the to-be-processed video for which no corresponding target video frame exists.

[0279] The video processing device according to this embodiment can be used to implement the technical solution according to any one of the above method embodiments, and since the implementation principle and technical effect are similar, reference can be made to the detailed description of the above method embodiment, and for the sake of simplicity, no further description will be given here.

[0280] 13 is a schematic diagram of an electronic device according to an embodiment of the present disclosure. As shown in FIG. 13, an electronic device 1300 according to this embodiment includes a memory 1301 and a processor 1302.

[0281] Here, the memory 1301 may be an independent physical unit, or may be connected to the processor 1302 via a bus 1303. The memory 1301 and the processor 1302 may be integrated together and realized by hardware or the like.

[0282] The memory 1301 is used to store program instructions that can be called upon by the processor 1302 to perform the operations of any one of the method embodiments described above.

[0283] Alternatively, when a part or all of the methods in the above-described embodiments are realized by software, the above-described electronic device 1300 may include only the processor 1302. The memory 1301 for storing a program is external to the electronic device 1300, and the processor 1302 is connected to the memory via a circuit / wiring to read and execute the program stored in the memory.

[0284] The processor 1302 may be a central processing unit (CPU), a network processor (NP), or a combination of a CPU and a NP.

[0285] The processor 1302 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0286] The memory 1301 may include volatile memory, such as random-access memory (RAM). The memory may also include non-volatile memory, such as flash memory, a hard disk drive (HDD) or a solid-state drive (SSD). The memory may also include a combination of the above types of memory.

[0287] The present disclosure further provides a readable storage medium comprising computer program instructions which, when executed by at least one processor of an electronic device, cause the video processing method according to any one of the method embodiments described above to be implemented.

[0288] The present disclosure further provides a computer program product which, when executed by a computer, causes the computer to implement the video processing method according to any one of the above method embodiments.

[0289] It should be noted that, in this specification, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that such a physical relationship or sequence exists between those entities or operations. Furthermore, the terms "comprise", "include", or any other variant thereof, are intended to indicate a non-exclusive inclusion, such that a process, method, article, or device that includes a series of elements includes not only those elements, but also other elements not expressly stated or that are inherent in such process, method, article, or device. In the absence of more limitations, an element limited by the words "comprise..." does not exclude the inclusion of other identical elements in the process, method, article, or device that includes the element.

[0290] The above are merely specific embodiments of the present disclosure, and are used to enable those skilled in the art to understand or realize the present disclosure. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure is not limited to these embodiments described herein, but has the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. performing, by a first thread, a target video editing operation on a video frame at the designated video frame position in a target video from a designated video frame position, and pre-performing the target video editing operation on each video frame after the designated video frame position to obtain each target video frame obtained by performing the target video editing operation; and in response to a playback command, rendering, by a second thread, for each of the target video frames and presenting each of the target video frames, wherein a video frame position at which the target video editing operation is being performed precedes a corresponding video frame position in the processed video of the target video frame being presented.

2. 2. The method of claim 1, wherein the specified video frame position is determined based on a video frame position located when a video editing command is obtained, the video editing command being for instructing to perform the target video editing operation on the target video, and the video frame position located when the video editing command is obtained is a preset video frame position or a video frame position specified by a first jump command.

3. 2. The method of claim 1, wherein the specified video frame position is determined based on a video frame position specified in a second jump instruction, and the second jump instruction is an instruction following a video editing instruction for performing the target video editing operation on the target video.

4. When the designated video frame position is determined based on a video frame position located when a video editing command is obtained, the method further comprises: executing a target video editing operation from the designated video frame position to a video frame at the designated video frame position in a processing target video by a first thread as described above; and pre-executing the target video editing operation on each video frame after the designated video frame position to obtain each target video frame obtained by executing the target video editing operation. The method of any one of claims 2 to 3, further comprising determining a video editing processing mode corresponding to the first thread based on the specified video frame position, the video editing processing mode being a first processing mode that performs the target video editing operation on key frames, or a second processing mode that performs the target video editing operation on a frame-by-frame basis.

5. Determining a video editing processing mode corresponding to the first thread based on the specified video frame position as described above includes: determining a video editing processing mode corresponding to the first thread as the first processing mode when the specified video frame position is the preset video frame position; 5. The method of claim 4, further comprising: if the specified video frame position is a video frame position specified by a first jump instruction, determining a video editing processing mode corresponding to the first thread as the first processing mode or the second processing mode.

6. Determining a video editing processing mode corresponding to the first thread based on the specified video frame position as described above includes: determining whether a corresponding target video frame exists at the specified video frame location; 5. The method of claim 4, further comprising: if there is no corresponding target video frame at the specified video frame position, determining a video editing processing mode corresponding to the first thread depending on whether the specified video frame position is the preset video frame position or a video frame position specified by the first jump instruction.

7. The method comprises: If a corresponding target video frame exists at the specified video frame position, determining a video editing processing mode corresponding to the first thread as the second processing mode; The above-mentioned performing a target video editing operation on a video frame at a specified video frame position in a target video by a first thread, from a specified video frame position, and performing the target video editing operation on each video frame after the specified video frame position in advance to obtain each target video frame obtained by performing the target video editing operation, determining whether a corresponding target video frame exists for each frame after the designated video frame position until a first video frame position without a corresponding target video frame is determined; 7. The method of claim 6, further comprising: performing, by the first thread, the target video editing operation on video frames of the processing subject video from a first video frame position for which no corresponding target video frame exists, in accordance with the second processing mode.

8. When the specified video frame position is determined based on the video frame position specified by a second jump instruction, the method further comprises: executing a target video editing operation from the specified video frame position to a video frame at the specified video frame position in a processing target video by a first thread; and pre-executing the target video editing operation on each video frame after the specified video frame position to obtain each target video frame obtained by executing the target video editing operation, the method further comprises: The method according to any one of claims 3 to 7, further comprising: determining a video editing processing mode corresponding to the first thread as a second processing mode if there is no corresponding target video frame at the specified video frame position.

9. 9. The method of claim 8, further comprising: if a corresponding target video frame exists at the specified video frame position and the first thread is currently performing the target video editing operation on the target video, not suspending the first thread and allowing the first thread to continue performing the target video editing operation on the target video.

10. A method according to any one of claims 1 to 9, further comprising: in response to the playback command, switching the video editing processing mode to a third processing mode, in which a video frame position at which the target video editing operation is to be performed is determined according to a playback speed; and performing the target video editing operation on a video frame of the processing subject video according to the third processing mode by the first thread.

11. switching the video editing processing mode to a third processing mode in response to the playback instruction and performing, by the first thread, the target video editing operations on video frames of the subject video in accordance with the third processing mode, as described above, Detecting in real time whether there is a next target video frame to be played in response to said play command; 11. The method of claim 10, further comprising: upon detecting that there is no next target video frame to be played, performing, by the first thread, the target video editing operation on video frames of the subject video from the position of the next target video frame to be played according to the third processing mode.

12. switching the video editing processing mode to a third processing mode in response to the playback instruction and performing, by the first thread, the target video editing operations on video frames of the subject video in accordance with the third processing mode, as described above, A method according to any one of claims 10 to 11, comprising, in response to the playback command, performing, by the first thread, the target video editing operation on video frames of the processing target video according to the third processing mode from the video frame position where the target video editing operation was being performed when the playback command was obtained.

13. switching the video editing processing mode to a third processing mode in response to the playback instruction and performing, by the first thread, the target video editing operations on video frames of the subject video in accordance with the third processing mode, as described above, determining a switching position corresponding to the third processing mode based on a processing time length for performing the target video editing operation on a single video frame, a playback speed, and a video frame position where the target video editing operation is currently being performed in response to the playback command; and executing, by the first thread, the target video editing operation on video frames of the subject video from the switching position in accordance with the third processing mode.

14. Obtaining a playback pause command; The method of any one of claims 10 to 13, further comprising: in response to the playback pause command, pausing rendering of the target video frames by the second thread, and the first thread continuing to perform the target video editing operations on video frames of the subject video in accordance with the third processing mode.

15. The method comprises: and performing, by the first thread, the target video editing operation on video frames of the target video according to a fourth processing mode from a position of a first video frame of the target video after performing the target video editing operation on a last video frame of the target video; The method according to any one of claims 1 to 14, wherein the fourth processing mode is a mode in which the target video editing operations are performed sequentially on video frames in the to-be-processed video for which no corresponding target video frames exist.

16. a first processing module configured to perform, by a first thread, a target video editing operation on a video frame at the specified video frame position in a to-be-processed video from a specified video frame position, and to pre-perform the target video editing operation on each video frame after the specified video frame position to obtain each target video frame obtained by performing the target video editing operation; and a second processing module configured to, in response to a playback command, perform rendering on each of the target video frames by a second thread and present each of the target video frames, wherein a video frame position at which the target video editing operation is being performed precedes a corresponding video frame position in the processed video of the target video frame being presented.

17. An electronic device including a memory and a processor, the memory is configured to store computer program instructions; An electronic device, wherein the processor is configured to execute the computer program instructions to cause the electronic device to implement a video processing method according to any one of claims 1 to 15.

18. A readable storage medium containing computer program instructions, comprising: A readable storage medium, the computer program instructions being, when executed by at least one processor of an electronic device, causing the electronic device to implement the video processing method according to any one of claims 1 to 15.

19. A computer program product which, when executed by a computer, causes the computer to implement the video processing method according to any one of claims 1 to 15.

Citation Information

Patent Citations

  • Video previewing method and apparatus

    CN104967865A

  • To-be-rendered object processing method and device and terminal

    CN111459591A

  • Video data processing method and device

    CN112954459A

  • Rendering in multi-user video editing system

    JP2010252328A

  • Visual effects network-based rendering and steering

    JP2015502678A