Editing methods, equipment and storage media

Abstract

This application provides an editing method, device, and storage medium. In the editing method, upon receiving an editing task for a target object and its editing type, the timeline of the target object can be segmented according to its editing type, thereby dividing the editing task into multiple sub-editing tasks. These sub-editing tasks can then be executed in parallel using multiple editing resources. On one hand, segmenting the timeline of the target object based on its editing type facilitates the reasonable division of the editing task into sub-editing tasks, reducing the risk of timestamps of the same editing type being disrupted, thus affecting the merging effect of the sub-editing results. On the other hand, the parallel slice editing method can fully utilize editing resources and significantly improve editing speed.

Description

Technical Field

[0001] This application relates to the field of multimedia processing, and more particularly to an editing method, apparatus, and storage medium. Background Technology

[0002] With the development of the internet, people are increasingly using videos to record their lives and for educational or entertainment purposes. Before publishing a video, users can edit it to improve its appearance. However, adding numerous effects, filters, transitions, and smart features during editing can make the process very time-consuming. Therefore, a new solution is needed. Summary of the Invention

[0003] This application provides an editing method, apparatus, and storage medium in several aspects to improve editing efficiency and reduce editing time.

[0004] This application provides an editing method, including: receiving an editing task for a target object and an editing type for the target object; dividing the timeline of the target object according to the editing type to obtain multiple sub-editing tasks; using multiple editing resources to execute the multiple sub-editing tasks in parallel to obtain the multiple sub-editing results; and synthesizing the multiple sub-editing results to obtain an edited video of the target object.

[0005] Optionally, the timeline of the target object is segmented according to the editing type to obtain multiple sub-editing tasks, including: identifying at least one time interval with atomic characteristics from the timeline of the target object according to the editing type; determining a separable position from the timeline of the target object according to the at least one time interval with atomic characteristics; and segmenting the timeline of the target object according to the separable position to obtain the multiple sub-editing tasks.

[0006] Optionally, before determining the divisible position from the timeline of the target object based on the at least one time interval with atomic characteristics, the method further includes: if any adjacent time intervals overlap, merging the adjacent time intervals.

[0007] Optionally, determining a divisible position from the timeline of the target object based on at least one time interval having atomic characteristics includes: obtaining a set slice duration; pre-segmenting the timeline of the target object based on the slice duration to obtain at least one pre-segmented position; and correcting the at least one pre-segmented position based on at least one time interval having atomic characteristics to obtain at least one segmented position.

[0008] Optionally, based on the at least one time interval with atomic characteristics, the at least one pre-segmentation position is modified to obtain at least one segmentation position, including: for any of the at least one pre-segmentation positions, determining whether the pre-segmentation position is located in any time interval with atomic characteristics; if the pre-segmentation position is not located in any time interval with atomic characteristics, then the pre-segmentation position is used as the segmentation position; if the pre-segmentation position is located in any time interval with atomic characteristics, then the pre-segmentation position is moved forward or backward in the time dimension to move the pre-segmentation position outside the at least one time interval with atomic characteristics, thereby obtaining the segmentation position.

[0009] Optionally, multiple editing resources are used to execute the multiple sub-editing tasks in parallel to obtain the multiple sub-editing results, including: identifying the algorithm type of the multiple sub-editing tasks according to the editing type corresponding to each of the multiple sub-editing tasks; determining the resource type matched by each of the multiple sub-editing tasks according to the algorithm type of the multiple sub-editing tasks; the resource type includes: GPU resource type or CPU resource type; and using the editing resources matched with the resource types of each of the multiple sub-editing tasks to perform parallel editing on the multiple sub-editing tasks to obtain the multiple sub-editing results.

[0010] Optionally, multiple editing resources are used to execute the multiple sub-editing tasks in parallel to obtain the multiple sub-editing results, including: determining the target file format corresponding to the edited film of the target object; using the multiple editing resources to perform parallel editing on the multiple sub-objects, and outputting the multiple sub-editing results according to the target file format.

[0011] This application embodiment also provides an editing method, including: obtaining a target object and the editing type of the target object; sending the editing task of the target object to a server, so that the server divides the timeline of the target object according to the editing type to obtain multiple sub-editing tasks, and uses multiple editing resources to execute the multiple sub-editing tasks in parallel to obtain the multiple sub-editing results; receiving the edited video synthesized based on the multiple sub-editing results returned by the server.

[0012] This application also provides a terminal device, including: a memory and a processor; the memory is used to store one or more computer instructions; the processor is used to execute the one or more computer instructions to perform the steps in the method provided in this application.

[0013] This application also provides a server, including: a memory and a processor; the memory is used to store one or more computer instructions; the processor is used to execute the one or more computer instructions to perform the steps in the method provided in this application.

[0014] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can implement the steps in the method provided in this application.

[0015] In the editing method provided in this application, upon receiving the editing task and editing type of the target object, the timeline of the target object can be segmented according to the editing type, thereby dividing the editing task of the target object into multiple sub-editing tasks, and using multiple editing resources to execute multiple sub-editing tasks in parallel. On the one hand, segmenting the timeline of the target object based on the editing type is beneficial for reasonably dividing the editing task into sub-editing tasks, reducing the risk that the timestamps of the same editing type will be disrupted, thus affecting the merging effect of the sub-editing results. On the other hand, based on the parallel slice editing method, editing resources can be fully utilized, significantly improving the editing speed. Attached Figure Description

[0016] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0017] Figure 1 A flowchart illustrating an exemplary embodiment of the editing system provided in this application;

[0018] Figure 2 A schematic diagram of video timeline segmentation and compositing provided for an exemplary embodiment of this application;

[0019] Figure 3 A schematic diagram illustrating the architecture hierarchy and functions of each layer of a cloud server provided for an exemplary embodiment of this application;

[0020] Figure 4 A schematic flowchart of the editing method provided in an exemplary embodiment of this application on the terminal device side;

[0021] Figure 5 A schematic diagram of the editing method provided on the server side as another exemplary embodiment of this application;

[0022] Figure 6 A schematic diagram of the structure of a terminal device provided in an exemplary embodiment of this application;

[0023] Figure 7This is a schematic diagram of the structure of a server provided for an exemplary embodiment of this application. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” used in the embodiments of this invention and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. “Multiple” generally includes at least two, but does not exclude the inclusion of at least one.

[0026] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0027] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a product or system comprising a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a product or system. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the product or system that includes said element.

[0028] In view of the technical problem of time-consuming editing in the prior art, this application provides a solution in some embodiments. The technical solutions provided by the embodiments of this application are described in detail below with reference to the accompanying drawings.

[0029] Figure 1 This is a schematic diagram of the structure of an editing system provided in an exemplary embodiment of this application, as shown below. Figure 1 As shown, the editing system 100 mainly includes: terminal device 101 and server 102.

[0030] The terminal device 101 can be a computer, tablet computer, mobile phone, or other similar device. The terminal device 101 can run an application or browser that provides editing services. In this embodiment, the terminal device 101 is mainly used to obtain the target object to be edited according to the user's editing needs, add editing types to the target object according to the user's editing needs, and submit the editing task to the server 102.

[0031] The target object to be edited may include video or audio. Typically, the target object may consist of one or more pieces of material to be edited. For example, a video object to be edited may include one or more video clips. In some embodiments, in addition to video clips, the video object to be edited may also include one or more audio clips. An audio object to be edited may include one or more audio clips.

[0032] The editing type of the target object may include, but is not limited to, at least one of the following: adding filters, background removal, adding stickers, adding effects, speed adjustment, adding subtitles, adding music, adding picture-in-picture, adding transitions, adding watermarks, and removing watermarks. Typically, each editing type has a start timestamp and an end timestamp.

[0033] The server 102 can be implemented as a device capable of receiving editing tasks sent by the terminal device 101 and performing editing operations based on abundant editing resources. The server 102 can be implemented as a conventional server, a cloud server, a cloud computing data center, or an elastic computing instance on the cloud, etc., and this embodiment does not impose any restrictions.

[0034] In this embodiment, server 102 is mainly used to: receive the editing task of the target object to be edited and the editing type of the target object. According to the editing type, server 102 can divide the timeline of the target object to obtain multiple sub-editing tasks.

[0035] The clip type of the target object can be carried in the clipping task of the target object, or it can be sent independently of the clipping task. This embodiment does not impose any restrictions.

[0036] In audio and video editing, a timeline is a data structure used to organize the temporal or spatial information of editing materials such as videos, images, audio, and subtitles. In audio and video editing, editing materials can be organized according to the time dimension, and transitions, effects, filters, subtitles, picture-in-picture, and other editing types can be added to these materials to obtain the audio and video timeline. This allows the computer to output the final product according to the editor's wishes. Vertically, the timeline can consist of layers of tracks, each corresponding to a different editing type. The final output can be a composite of multiple tracks. Horizontally, the timeline is a time axis, used to represent duration information. Materials on different tracks are arranged on the time axis in the order they appear.

[0037] The timeline segmentation operation divides the target object into multiple sub-objects, each sub-object corresponding to a sub-editing task. Server 102 can use multiple editing resources to execute the sub-editing tasks of each sub-object in parallel, obtaining multiple sub-editing results. When executing a sub-editing task, any editing resource can perform at least one of the following operations based on the editing type corresponding to the sub-editing task: special effects compositing, image cutout, watermarking, image overlay, and image stitching. After multiple editing resources execute the sub-editing tasks in parallel, they can output the sub-editing results separately. Based on these multiple sub-editing results, server 102 can synthesize the final edited video of the target object.

[0038] The server 102 may include a cluster server, which can provide various editing resources based on multiple physical machines to improve editing speed. An editing resource may be a single physical machine or a cluster of multiple physical machines; this embodiment does not impose any limitations.

[0039] In this embodiment, upon receiving the editing task and editing type of the target object, the timeline of the target object can be segmented according to the editing type, thereby dividing the editing task of the target object into multiple sub-editing tasks, and using multiple editing resources to execute multiple sub-editing tasks in parallel. On the one hand, segmenting the timeline of the target object based on the editing type is beneficial for reasonably dividing the editing task into sub-editing tasks, reducing the risk that the timestamps of the same editing type will be disrupted, thus affecting the merging effect of the sub-editing results. On the other hand, based on the parallel slice editing method, editing resources can be fully utilized, significantly improving the editing speed.

[0040] In some exemplary embodiments, when server 102 divides the timeline of the target object according to the clip type to obtain multiple sub-editing tasks, it can identify at least one time interval with atomic characteristics from the timeline of the target object based on the clip type. Here, atomic characteristics refer to features that cannot be further divided. In the target object, time intervals with atomic characteristics have editing effects that continuously change over time. Each clip type corresponds to a start timestamp and an end timestamp on the timeline. During the editing process, the clip resource can determine the first and last keyframes and intermediate frames based on the start and end timestamps, and can perform editing operations on consecutive frames starting from the first keyframe according to the clip type, up to the last keyframe. Therefore, the time interval between the first and last keyframes is a time interval with atomic characteristics. Thus, when segmenting the timeline, the time interval between the first and last keyframes can be avoided for cutting, ensuring that consecutive intermediate frames between the first and last keyframes are not split into different slices, reducing the risk of timestamps of the same clip type being disrupted, and improving the final merged effect.

[0041] like Figure 2 As shown, taking video editing as an example, the video to be edited includes material 1, material 2, and material 3. On the video timeline, dynamic effects are added to certain time intervals of material 1, material 2, and material 3; transition effects are added between material 1 and material 3, and also between material 2 and material 3. Figure 2 In the illustrated timeline, time intervals with transition effects and dynamic effects are considered indivisible to avoid disrupting the continuity of the time dimension. All other intervals are considered divisible.

[0042] Based on this, server 102 can determine the separable position from the timeline of the target object according to at least one time interval with atomic characteristics, and divide the timeline of the target object according to the separable position to obtain the multiple sub-editing tasks.

[0043] Optionally, before determining the divisible position from the timeline of the target object based on at least one time interval with atomic characteristics, server 102 may merge overlapping time intervals with atomic characteristics. That is, if any adjacent time intervals overlap within at least one time interval with atomic characteristics, the adjacent time intervals are merged.

[0044] In this implementation, server 102 can first identify indivisible atomic features on the timeline of the target object and calculate the time interval for each atomic feature. If there is an overlap in the time intervals between two adjacent atomic features, the two overlapping time intervals with atomic features can be merged. After obtaining the merged time interval, the server can continue to perform overlap judgment and merging operations with other time intervals with atomic features until there are no overlapping time intervals with atomic features on the timeline.

[0045] Based on this implementation method, overlapping and indivisible intervals can be merged. On the one hand, this improves the recognition speed of divisible intervals, and on the other hand, it reduces the possibility of segmenting time intervals with atomic characteristics, which is beneficial to improving the editing effect.

[0046] In some optional embodiments, when server 102 determines a separable position from the timeline of the target object based on at least one time interval with atomic characteristics, it may consider time intervals on the target object's timeline other than the at least one time interval with atomic characteristics as separable intervals. After determining the separable intervals, a segmentation position can be selected from the separable intervals according to a set strategy. The set strategy can be a randomly selected strategy or a strategy selected at fixed time intervals; this embodiment does not impose any limitations.

[0047] Continue to combine Figure 2 Please provide an explanation. For example... Figure 2 As shown, server 102 can select the segmentation position within the time interval when there are no special effects on material 1, material 2, and material 3.

[0048] In some alternative embodiments, when server 102 determines a divisible position from the timeline of the target object based on at least one time interval with atomic characteristics, it may pre-segment the timeline of the target object and then correct the pre-segmentation result based on the time interval with atomic characteristics.

[0049] Optionally, the server 102 can obtain a set slice duration. This slice duration can be set by the user or by default by the developers; this embodiment does not impose any restrictions. For example, the slice duration can be 3 seconds, 5 seconds, or 10 seconds; this embodiment does not impose any restrictions. The server 102 can pre-segment the timeline of the target object based on the slice duration to obtain at least one pre-segmentation position, and then correct the at least one pre-segmentation position based on the at least one time interval with atomic characteristics to obtain at least one segmentation position.

[0050] Optionally, when modifying the at least one pre-segmentation position, the server 102 may determine whether each pre-segmentation position is located within a divisible interval. The following will illustrate this with an example of any one of the at least one pre-segmentation positions.

[0051] Optionally, for any pre-segmentation position, the server 102 can determine whether the pre-segmentation position is located within any time interval with atomic characteristics; if the pre-segmentation position is not located within at least one time interval with atomic characteristics (i.e., the pre-segmentation position is outside all time intervals with atomic characteristics), then the pre-segmentation position is taken as the segmentation position. Optionally, if the pre-segmentation position is located within any time interval with atomic characteristics, then the pre-segmentation position is moved forward or backward in the time dimension to move the pre-segmentation position outside at least one time interval with atomic characteristics (i.e., the pre-segmentation position is moved to the nearest time interval outside all time intervals with atomic characteristics) to obtain the segmentation position.

[0052] In this implementation, by setting the slice duration, the slice length of the timeline can be constrained, ensuring that the timeline slices corresponding to sub-editing tasks have reasonable lengths to make efficient use of editing resources. Simultaneously, by correcting the pre-segmentation results based on the slice duration according to the editing type on the timeline, the timeline slices can better match the distribution of materials in the editing operation, meeting the editing needs of various scenarios.

[0053] After determining the segmentation position based on the foregoing embodiments, server 102 can segment the timeline of the target object according to the segmentation position. For example... Figure 2 As shown, dividing the timeline according to the position of the dotted line on the timeline yields slice 1, slice 2, slice 3, and slice 4.

[0054] In some exemplary embodiments, when the server 102 uses multiple clip resources to execute the multiple sub-clip tasks in parallel and obtain the multiple sub-clip results, it can allocate different clip resources to the sub-object according to the type of the clip task of the sub-object.

[0055] Optionally, server 102 can identify the algorithm type of each of the multiple sub-editing tasks based on the corresponding editing type, and determine the resource type that matches each of the multiple sub-editing tasks based on the algorithm type. After determining the resource type, server 102 can use the editing resources that match the resource type of each of the multiple sub-editing tasks to perform parallel editing on the multiple sub-editing tasks to obtain the multiple sub-editing results.

[0056] The resource types can include either GPU (Graphics Processing Unit) or CPU (central processing unit) resource types.

[0057] The server 102 can pre-define the correspondence between different editing types and algorithm types. For each sub-editing task, the algorithm type used to execute the sub-editing task can be determined based on the editing type corresponding to that sub-editing task. For example, the correspondence between masking editing type and GPU algorithm, and the correspondence between image matting editing type and GPU algorithm can be pre-defined. When a sub-editing task contains a masking editing operation, the algorithm type of the sub-editing task can be determined to be a GPU algorithm. When a sub-editing task contains a GPU-type algorithm, the server 102 can schedule the sub-editing task to be executed on GPU resources; when a sub-editing task contains a CPU-type algorithm, the server 102 can schedule the sub-editing task to be executed on CPU resources.

[0058] Based on this implementation, sub-editing tasks with high CPU utilization can be scheduled to be executed on a CPU cluster, while sub-editing tasks with high GPU utilization can be scheduled to be executed on a GPU cluster. Thus, different sub-editing tasks can use different computing resources, further reducing editing time compared to the limitations of a single machine's computing resources. It is worth noting that in some optional embodiments, when the server 102 uses multiple editing resources to perform parallel editing on multiple sub-editing tasks, it can determine the target file format corresponding to the edited piece of the target object and specify the target file format to each of the multiple editing resources. The server 102 can then use these multiple editing resources to perform parallel editing on the multiple sub-objects and output the multiple sub-editing results according to the target file format.

[0059] Continue to combine Figure 2 Let's explain. Assuming the target file format for the edited clip of the target object is mp4 (Moving Picture Experts Group 4), then each slice obtained from the time-division segment can be in mp4 format. Accordingly, when merging the slices into a final clip, the edited results of multiple mp4 slices are combined to obtain the final mp4 clip.

[0060] In this implementation, the final video of each sub-editing task has the same encoding format as the target output video. When merging the final videos of multiple sub-editing tasks, there is no need to perform encoding and decoding operations on the final video of each sub-editing task again. The video merging can be completed simply by merging the decapsulated final videos together, thereby greatly improving the merging speed and editing efficiency.

[0061] The editing system provided in this application can be applied to video editing and audio editing. The following will use video editing as an example for further illustrative explanation.

[0062] In some application scenarios, server 102 can be implemented as a clustered server located in the cloud (hereinafter referred to as a cloud server). For example... Figure 3 As shown, a cloud server may include a cloud editing service layer, a scheduling layer, and an engine layer. The cloud editing service layer and scheduling layer can be implemented as microservices on the cloud server. The engine layer may include multiple physical machines or multiple virtual machines to execute editing tasks in parallel.

[0063] The cloud editing service layer is primarily used to perform timeline segmentation operations on the target object and manage the main editing task and sub-editing tasks. The scheduling layer is mainly used to dynamically distribute each sub-editing task to different clusters. The engine layer is mainly responsible for the execution of sub-editing tasks and the merging of sub-editing results.

[0064] like Figure 3 As shown, after the client submits a video editing task, the cloud editing service layer can organize the video's timeline and calculate timeline segments based on the types of clips added to the video. When calculating timeline segments, the cloud editing service layer can divide the original timeline according to the user-specified segment duration. If the cutting position falls within certain uncut segments (such as the time periods of transitions or effects), the cutting position can be postponed until it falls within a cutable area of ​​the timeline.

[0065] In the timeline, indivisible segments are typically those with atomic characteristics. These segments exhibit video effects that change with timestamps, such as transitions between videos, dynamic video effects, and other features where timestamps change due to slicing, affecting the final composite effect. The cloud editing service layer identifies atomic features on the timeline and calculates the time interval for each atomic feature. If two atomic features overlap, it can be assumed that all time points within those intervals are indivisible. In this case, the cloud editing service layer merges the two overlapping atomic feature intervals and then uses the merged interval to further evaluate for overlap with other atomic feature intervals. Time intervals without atomic characteristics are considered divisible intervals. Within these divisible intervals, timeline slicing points can be selected, and the timeline can be sliced.

[0066] After calculating the timeline segments, the cloud editing service layer can submit segment editing tasks (i.e., sub-editing tasks) to the scheduling layer one by one. For example... Figure 3As shown, if there are any remaining unsubmitted segment editing tasks, the cloud editing service layer can repeatedly execute the submission operation of the segment editing tasks.

[0067] When the cloud editing service layer submits a segmented editing task to the scheduling layer, it can identify the algorithm type corresponding to the segmented editing task and add a type identifier to the segmented editing task. After receiving the segmented editing task, the scheduling layer can identify the algorithm type of the segmented editing task based on the type identifier on the segmented editing task. If it is a GPU-type algorithm, the segmented editing task will be scheduled to be executed on the GPU cluster of the engine layer; if it is a CPU-type algorithm, the segmented editing task will be scheduled to be executed on the CPU cluster of the engine layer.

[0068] When the scheduling layer submits a segment editing task to the engine layer, it can also submit the target output format corresponding to the segment editing task. This target output format matches the format of the target edited video. Therefore, the encoding format of the segment editing result output by the engine layer matches the format of the target edited video. When the engine layer processes the segment editing results for merging, it does not need to perform encoding and decoding operations on each segment editing result again; it only needs to merge the decapsulated segment editing results together to complete the segmented video merging, further improving the merging speed.

[0069] like Figure 3 As shown, after the engine layer completes all segment editing tasks, the scheduling layer can execute callback operations, and the cloud editing service layer will handle the sub-task callbacks. If all segment editing tasks are completed, the cloud editing service layer can submit a video merging task to the scheduling layer. The scheduling layer can then submit the video merging task to the distributed media processing engine in the engine layer, which will then perform video merging processing on the segment editing results to obtain the final video.

[0070] In addition to the editing system provided in the foregoing embodiments, this application also provides an editing method, which will be described below in conjunction with the accompanying drawings.

[0071] Figure 4 This is a flowchart illustrating an exemplary embodiment of the editing method provided in this application. When executed on a terminal device, the method may include, for example: Figure 4 The steps shown are as follows:

[0072] Step 401: Obtain the target object and the clip type of the target object.

[0073] Step 402: Send the editing task of the target object to the server, so that the server can divide the timeline of the target object according to the editing type to obtain multiple sub-editing tasks, and use multiple editing resources to execute the multiple sub-editing tasks in parallel to obtain the multiple sub-editing results.

[0074] Step 403: Receive the edited video synthesized based on the multiple sub-editing results returned by the server.

[0075] In this embodiment, when the terminal device receives an editing task, it can utilize the server to segment the timeline of the target object according to the editing type of the target object, thereby dividing the editing task of the target object into multiple sub-editing tasks, and using multiple editing resources to execute multiple sub-editing tasks in parallel. On the one hand, segmenting the timeline of the target object based on the editing type is beneficial for reasonably dividing the editing task into sub-editing tasks, reducing the risk that the timestamps of the same editing type will be disrupted, thus affecting the merging effect of the sub-editing results. On the other hand, based on the parallel slice editing method, editing resources can be fully utilized, significantly improving the editing speed. The server can be implemented as a cloud server to fully utilize the processing power and abundant machine resources of the cloud-based editing cluster, thereby improving editing efficiency.

[0076] Figure 5 This is a flowchart illustrating an editing method provided in another exemplary embodiment of this application. When executed on a server, this method may include, for example... Figure 5 The steps shown are as follows:

[0077] Step 501: Receive the editing task of the target object and the editing type of the target object.

[0078] Step 502: Based on the editing type, the timeline of the target object is segmented to obtain multiple sub-editing tasks.

[0079] Step 503: Use multiple editing resources to execute the multiple sub-editing tasks in parallel to obtain the multiple sub-editing results.

[0080] Step 504: Based on the multiple sub-edit results, synthesize the edited video of the target object.

[0081] In some exemplary embodiments, one method of segmenting the timeline of the target object according to the clip type to obtain multiple sub-editing tasks includes: identifying at least one time interval with atomic characteristics from the timeline of the target object according to the clip type; determining a separable position from the timeline of the target object according to the at least one time interval with atomic characteristics; and segmenting the timeline of the target object according to the separable position to obtain the multiple sub-editing tasks.

[0082] In some exemplary embodiments, before determining the divisible position from the timeline of the target object based on the at least one time interval having atomic characteristics, the method further includes: if any adjacent time intervals overlap, merging the adjacent time intervals.

[0083] In some exemplary embodiments, one method for determining a divisible position from the timeline of the target object based on at least one time interval having atomic characteristics includes: obtaining a set slice duration; pre-segmenting the timeline of the target object based on the slice duration to obtain at least one pre-segmented position; and correcting the at least one pre-segmented position based on the at least one time interval having atomic characteristics to obtain at least one segmented position.

[0084] In some exemplary embodiments, one method of modifying at least one pre-segmentation position based on at least one time interval with atomic characteristics to obtain at least one segmentation position includes: for any of the at least one pre-segmentation positions, determining whether the pre-segmentation position is located within any time interval with atomic characteristics; if the pre-segmentation position is not located within any time interval with atomic characteristics, then using the pre-segmentation position as the segmentation position; if the pre-segmentation position is located within any time interval with atomic characteristics, then moving the pre-segmentation position forward or backward in the time dimension to move the pre-segmentation position outside the at least one time interval with atomic characteristics, thereby obtaining the segmentation position.

[0085] In some exemplary embodiments, one method of using multiple editing resources to execute the multiple sub-editing tasks in parallel to obtain the multiple sub-editing results includes: identifying the algorithm type of the multiple sub-editing tasks based on the editing type corresponding to each of the multiple sub-editing tasks; determining the resource type matched to each of the multiple sub-editing tasks based on the algorithm type of the multiple sub-editing tasks; the resource type includes: GPU resource type or CPU resource type; and using the editing resources matched to the resource types of the multiple sub-editing tasks to perform parallel editing on the multiple sub-editing tasks to obtain the multiple sub-editing results.

[0086] In some exemplary embodiments, one method of using multiple editing resources to execute the multiple sub-editing tasks in parallel to obtain the multiple sub-editing results includes: determining the target file format corresponding to the edited film of the target object; using the multiple editing resources to perform parallel editing on the multiple sub-objects, and outputting the multiple sub-editing results according to the target file format.

[0087] In this embodiment, upon receiving the editing task and editing type of the target object, the timeline of the target object can be segmented according to the editing type, thereby dividing the editing task of the target object into multiple sub-editing tasks, and using multiple editing resources to execute multiple sub-editing tasks in parallel. On the one hand, segmenting the timeline of the target object based on the editing type is beneficial for reasonably dividing the editing task into sub-editing tasks, reducing the risk that the timestamps of the same editing type will be disrupted, thus affecting the merging effect of the sub-editing results. On the other hand, based on the parallel slice editing method, editing resources can be fully utilized, significantly improving the editing speed.

[0088] It should be noted that the execution subject of each step of the method provided in the above embodiments can be the same device, or the method can be executed by different devices. For example, the execution subject of steps 501 to 504 can be device A; or the execution subject of steps 501 and 502 can be device A, and the execution subject of step 503 can be device B; and so on.

[0089] Furthermore, in some of the processes described in the above embodiments and accompanying drawings, multiple operations appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or they may be executed in parallel. The operation numbers, such as 501, 502, etc., are merely used to distinguish different operations and do not represent any execution order. Additionally, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first" and "second" in this document are used to distinguish different messages, devices, modules, etc., and do not represent a chronological order, nor do they limit "first" and "second" to different types.

[0090] Figure 6 This illustration shows a structural diagram of a terminal device provided in an exemplary embodiment of this application. This terminal device is applicable to the editing system provided in the foregoing embodiments. Figure 6 As shown, the terminal device includes: a memory 601, a processor 602, and a communication component 603.

[0091] Memory 601 is used to store computer programs and can be configured to store various other data to support operation on a terminal device. Examples of this data include instructions for any application or method used to operate on the terminal device.

[0092] Processor 602, coupled to memory 601, is used to execute computer programs in memory 601 for: acquiring a target object and the clip type of the target object; sending the clipping task of the target object to a server using communication component 603, so that the server segments the timeline of the target object according to the clip type to obtain multiple sub-cutting tasks, and uses multiple clipping resources to execute the multiple sub-cutting tasks in parallel to obtain the multiple sub-cutting results; and receiving the final edited video synthesized based on the multiple sub-cutting results returned by the server using communication component 603.

[0093] Furthermore, such as Figure 6 As shown, the terminal device also includes other components such as a power supply component 604, a display component 605, and an audio component 606. Figure 6 The diagram only shows some components and does not mean that the terminal device only includes... Figure 6 The components shown.

[0094] The display component 605 includes a screen, which may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touchscreen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation.

[0095] The audio component 606 can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC) configured to receive external audio signals when the device containing the audio component is in an operating mode, such as a call mode, recording mode, or voice recognition mode. The received audio signals can be further stored in memory or transmitted via a communication component. In some embodiments, the audio component also includes a speaker for outputting audio signals.

[0096] In this embodiment, when the terminal device receives an editing task, it can utilize the server to segment the timeline of the target object according to the editing type of the target object, thereby dividing the editing task of the target object into multiple sub-editing tasks, and using multiple editing resources to execute multiple sub-editing tasks in parallel. On the one hand, segmenting the timeline of the target object based on the editing type is beneficial for reasonably dividing the editing task into sub-editing tasks, reducing the risk that the timestamps of the same editing type will be disrupted, thus affecting the merging effect of the sub-editing results. On the other hand, based on the parallel slice editing method, editing resources can be fully utilized, significantly improving the editing speed. The server can be implemented as a cloud server to fully utilize the processing power and abundant machine resources of the cloud-based editing cluster, thereby improving editing efficiency.

[0097] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed, can perform the steps that can be executed by a terminal device in the above method embodiments.

[0098] Figure 7 This illustration shows a structural diagram of a server provided in an exemplary embodiment of this application, which is suitable for the editing system provided in the foregoing embodiments. For example... Figure 7 As shown, the server includes: a memory 701, a processor 702, and a communication component 703.

[0099] Memory 701 is used to store computer programs and can be configured to store various other data to support operations on the server. Examples of this data include instructions for any application or method used to operate on the server.

[0100] The processor 702, coupled to the memory 701, is used to execute a computer program in the memory 701 for: receiving a editing task of a target object and the editing type of the target object using a communication component 703; dividing the timeline of the target object according to the editing type to obtain multiple sub-editing tasks; using multiple editing resources to execute the multiple sub-editing tasks in parallel to obtain the multiple sub-editing results; and synthesizing the multiple sub-editing results to obtain a final edit of the target object.

[0101] Optionally, when the processor 702 segments the timeline of the target object according to the clip type to obtain multiple sub-editing tasks, it is specifically configured to: identify at least one time interval with atomic characteristics from the timeline of the target object according to the clip type; determine a separable position from the timeline of the target object according to the at least one time interval with atomic characteristics; and segment the timeline of the target object according to the separable position to obtain the multiple sub-editing tasks.

[0102] Optionally, before determining the divisible position from the timeline of the target object based on the at least one time interval having atomic characteristics, the processor 702 is further configured to: merge the adjacent time intervals if any adjacent time intervals overlap.

[0103] Optionally, when the processor 702 determines a divisible position from the timeline of the target object based on at least one time interval having atomic characteristics, it is specifically configured to: obtain a set slice duration; pre-segment the timeline of the target object based on the slice duration to obtain at least one pre-segmented position; and correct the at least one pre-segmented position based on at least one time interval having atomic characteristics to obtain at least one segmented position.

[0104] Optionally, when the processor 702 corrects the at least one pre-segmentation position according to the at least one time interval with atomic characteristics to obtain at least one segmentation position, it specifically performs the following steps: for any of the at least one pre-segmentation positions, it determines whether the pre-segmentation position is located in any time interval with atomic characteristics; if the pre-segmentation position is not located in any time interval with atomic characteristics, it takes the pre-segmentation position as the segmentation position; if the pre-segmentation position is located in any time interval with atomic characteristics, it moves the pre-segmentation position forward or backward in the time dimension to move the pre-segmentation position outside the at least one time interval with atomic characteristics, thereby obtaining the segmentation position.

[0105] Optionally, when the processor 702 uses multiple editing resources to execute the multiple sub-editing tasks in parallel and obtain the multiple sub-editing results, it is specifically used to: identify the algorithm type of the multiple sub-editing tasks according to the editing type corresponding to each of the multiple sub-editing tasks; determine the resource type matched by each of the multiple sub-editing tasks according to the algorithm type of the multiple sub-editing tasks; the resource type includes: GPU resource type or CPU resource type; and use the editing resources matched with the resource type of each of the multiple sub-editing tasks to perform parallel editing on the multiple sub-editing tasks to obtain the multiple sub-editing results.

[0106] Optionally, when the processor 702 uses multiple editing resources to execute the multiple sub-editing tasks in parallel and obtain the multiple sub-editing results, it is specifically used to: determine the target file format corresponding to the edited film of the target object; use the multiple editing resources to perform parallel editing on the multiple sub-objects, and output the multiple sub-editing results according to the target file format.

[0107] Furthermore, such as Figure 7 As shown, the server also includes other components such as the power supply component 704. Figure 7 The diagram only shows some components and does not mean that the server only includes... Figure 7 The components shown.

[0108] In this embodiment, upon receiving the editing task and editing type of the target object, the timeline of the target object can be segmented according to the editing type, thereby dividing the editing task of the target object into multiple sub-editing tasks, and using multiple editing resources to execute multiple sub-editing tasks in parallel. On the one hand, segmenting the timeline of the target object based on the editing type is beneficial for reasonably dividing the editing task into sub-editing tasks, reducing the risk that the timestamps of the same editing type will be disrupted, thus affecting the merging effect of the sub-editing results. On the other hand, based on the parallel slice editing method, editing resources can be fully utilized, significantly improving the editing speed.

[0109] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed, can implement the steps that can be executed by the server in the above method embodiments.

[0110] exist Figure 6 as well as Figure 7 In this context, the memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0111] exist Figure 6 as well as Figure 7 In this embodiment, the communication component is configured to facilitate wired or wireless communication between the device containing the communication component and other devices. The device containing the communication component can access wireless networks based on communication standards, such as WiFi, 2G, 3G, 4G, or 5G, or combinations thereof. In one exemplary embodiment, the communication component receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component may be implemented based on Near Field Communication (NFC), Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), Bluetooth (BT), and other technologies.

[0112] exist Figure 6 as well as Figure 7In this context, a power supply component is used to provide power to various components of the device in which it resides. A power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device in which it resides.

[0113] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0114] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0115] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0116] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0117] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0118] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0119] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0120] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0121] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. An editing method, characterized in that, include: Receive the editing task of the target object and the editing type of the target object; Based on the editing type, the timeline of the target object is segmented to obtain multiple sub-editing tasks; Multiple editing resources are used to execute the multiple sub-editing tasks in parallel, resulting in multiple sub-editing results; Based on the multiple sub-edit results, a final edit of the target object is obtained; Each clip type has a start timestamp and an end timestamp on the timeline. The start timestamp and end timestamp are used to determine the first and last keyframes. The time interval between the first and last keyframes has atomic characteristics. When segmenting the timeline of the target object, the clip type is used to identify time intervals with atomic characteristics from the timeline of the target object in order to avoid cutting the time interval between the first and last keyframes.

2. The method according to claim 1, characterized in that, Based on the editing type, the timeline of the target object is segmented to obtain multiple sub-editing tasks, including: Based on the clip type, identify at least one time interval with atomic characteristics from the timeline of the target object; Based on at least one time interval having atomic characteristics, determine the divisible position from the timeline of the target object; Based on the divisible positions, the timeline of the target object is segmented to obtain the multiple sub-editing tasks.

3. The method according to claim 2, characterized in that, Before determining the divisible position from the timeline of the target object based on at least one time interval having atomic characteristics, the method further includes: If any adjacent time intervals overlap within the at least one time interval possessing atomic characteristics, then the adjacent time intervals are merged.

4. The method according to claim 2, characterized in that, Determining a separable position from the timeline of the target object based on at least one time interval having atomic characteristics includes: Get the set slice duration; Based on the slice duration, the timeline of the target object is pre-segmented to obtain at least one pre-segmentation position; Based on at least one time interval with atomic characteristics, the at least one pre-segmented position is corrected to obtain at least one segmentation position.

5. The method according to claim 4, characterized in that, Based on at least one time interval with atomic characteristics, the at least one pre-segmented position is corrected to obtain at least one segmentation position, including: For any one of the at least one pre-segmentation positions, determine whether the pre-segmentation position is located in any time interval with atomic characteristics; If the pre-segmentation position is not located in any time interval with atomic characteristics, then the pre-segmentation position is used as the segmentation position; If the pre-segmentation position is located in any time interval with atomic characteristics, then the pre-segmentation position is moved forward or backward in the time dimension to move the pre-segmentation position outside at least one time interval with atomic characteristics, thereby obtaining the segmentation position.

6. The method according to any one of claims 1-5, characterized in that, Using multiple clip resources, the multiple sub-edit tasks are executed in parallel to obtain the multiple sub-edit results, including: Identify the algorithm type of each of the multiple sub-editing tasks based on the editing type corresponding to each of the multiple sub-editing tasks; Based on the algorithm type of the multiple sub-editing tasks, determine the resource type that each of the multiple sub-editing tasks matches; the resource type includes: GPU resource type or CPU resource type; Using editing resources that match the resource types of each of the multiple sub-editing tasks, the multiple sub-editing tasks are edited in parallel to obtain the multiple sub-editing results.

7. The method according to any one of claims 1-5, characterized in that, Using multiple clip resources, the multiple sub-edit tasks are executed in parallel to obtain the multiple sub-edit results, including: Determine the target file format corresponding to the edited video of the target object; Using the multiple editing resources, multiple sub-objects obtained by segmenting the target object are edited in parallel, and the multiple sub-editing results are output according to the target file format, with each sub-object corresponding to a sub-editing task.

8. An editing method, characterized in that, include: Obtain the target object and the clip type of the target object; The editing task of the target object is sent to the server, so that the server divides the timeline of the target object according to the editing type to obtain multiple sub-editing tasks, and uses multiple editing resources to execute the multiple sub-editing tasks in parallel to obtain multiple sub-editing results; wherein, each editing type has a start timestamp and an end timestamp on the timeline, the start timestamp and end timestamp are used to determine the first and last keyframes, and the time interval between the first and last keyframes has atomic characteristics; when dividing the timeline of the target object, the editing type is used to identify the time intervals with atomic characteristics from the timeline of the target object to avoid cutting the time interval between the first and last keyframes; Receive the edited video synthesized from the multiple sub-edit results returned by the server.

9. A server, characterized in that, include: Memory and processor; The memory is used to store one or more computer instructions; The processor is configured to execute one or more computer instructions for performing the steps of the method according to any one of claims 1-7.

10. A computer-readable storage medium storing a computer program, characterized in that, When a computer program is executed by a processor, it can implement the editing method according to any one of claims 1-8.

Images