Multimedia editing method and device and electronic equipment

Abstract

The embodiment of the invention provides a multimedia editing method and device and electronic equipment, and the method comprises the steps: receiving a plurality of editing operations, and determining a time period corresponding to each editing operation on a time axis, the editing operation being used for editing a multimedia material track fragment; in response to a received editing operation ending instruction, determining a target time period covering each editing operation according to the time period corresponding to each editing operation; updating the target segmented view data of the target time period according to the editing effect indicated by each editing operation; and generating the multimedia content to be displayed according to the target segment view data and the historical segment view data of other segments of the time axis, thereby improving the view creation efficiency and reducing the data redundancy.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of computer technology, and more particularly to a multimedia editing method, device, and electronic device. Background Art

[0002] With the development of short video technology, users can use various video editing tools to edit videos. In some video editing scenarios, the editing operations required for videos are also relatively complex, such as deleting video tracks, deleting audio tracks, deleting video clips, deleting audio clips, moving video clips, adjusting the order between clips, and so on. Summary of the invention

[0003] Embodiments of the present disclosure provide a multimedia editing method, device, and electronic device.

[0004] In a first aspect, an embodiment of the present disclosure provides a multimedia editing method, the method comprising: receiving multiple editing operations, determining a time period corresponding to each editing operation on a timeline, the editing operations being used to edit multimedia material track segments; in response to receiving an editing operation end instruction, determining a target time period covering each editing operation according to the time periods corresponding to each editing operation; updating target segmented view data of the target time period according to the editing effects indicated by each editing operation; and generating multimedia content to be displayed according to the target segmented view data and historical segmented view data of remaining time periods.

[0005] In a second aspect, an embodiment of the present disclosure provides a multimedia editing device, which includes: a first determination unit, used to receive multiple editing operations, and determine the time period corresponding to each editing operation on the timeline, wherein the editing operation is used to edit a multimedia material track segment; a second determination unit, used to determine, in response to receiving an editing operation end instruction, a target time period covering each editing operation according to the time period corresponding to each editing operation; an updating unit, used to update the target segmented view data of the target time period according to the editing effects indicated by each editing operation; and a generating unit, used to generate multimedia content to be displayed according to the target segmented view data and the historical segmented view data of the remaining time periods.

[0006] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising: a processor and a memory; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory, so that the at least one processor executes the first aspect as well as various methods that may be involved in the first aspect.

[0007] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the first aspect and various possible methods involved in the first aspect are implemented.

[0008] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the first aspect as well as various methods that may be involved in the first aspect.

[0009] The multimedia editing method, device and electronic device provided in this embodiment receive multiple editing operations, determine the time period corresponding to each editing operation on the time axis, and the editing operation is used to edit the multimedia material track segment; in response to receiving the editing operation end instruction, determine the target time period covering each editing operation according to the time period corresponding to each editing operation; update the target segment view data of the target time period according to the editing effect indicated by each editing operation; generate the multimedia content to be displayed according to the target segment view data and the historical segment view data of the remaining time periods, so that when receiving multiple editing operations, only the view data of the target time period corresponding to the editing operation can be updated, and the multimedia content responding to the multiple editing operations can be generated. In the above-mentioned editing operation process, it is not necessary to generate view data for each editing operation. Only after receiving the end of the editing operation, the view data in the above-mentioned target time period is updated, so as to generate the comprehensive response content to the multiple editing operations, and it is not necessary to generate the multimedia view data and segment view data on the whole time axis. Data redundancy is reduced, the efficiency of the editing operation is improved, and the response speed to the editing operation is also improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0011] Figure 1 A schematic diagram of a multimedia editing method provided by an embodiment of the present disclosure;

[0012] Figure 2 A schematic diagram of another multimedia editing method provided by an embodiment of the present disclosure;

[0013] Figure 3 A flowchart of a multimedia editing method provided by an embodiment of the present disclosure;

[0014] Figure 4 This is a schematic diagram of an application scenario;

[0015] Figure 5 for Figure 3 A schematic diagram of a process of obtaining historical segmented view data involved in step S303 in the illustrated embodiment;

[0016] Figure 6 This is a schematic diagram of an application scenario;

[0017] Figure 7 A structural block diagram of a multimedia editing device provided by an embodiment of the present disclosure;

[0018] Figure 8 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0020] In a multimedia editing method implementation, a user can use a video editing client to edit a video. For example, the video editing client provides editing operation information for performing editing operations on audio and video materials in a video editing interface. After receiving the editing operation information, the video editing client transmits the editing operation information and the audio and video materials in the video editing interface to a multi-layer data model running in the background.

[0021] Please refer to Figure 1 , Figure 1 FIG. 1 is a schematic diagram of a multimedia editing method. Figure 1 As shown, the multi-layer data model may include multiple data processing layers. For each editing operation of the user, the video editing client may transmit the editing operation and the audio and video data segments in the video editing interface to the multi-layer data model, and each layer in the multi-layer data model creates corresponding audio objects and video objects respectively, and the latter layer of data model consumes the data output by the previous layer of data model.

[0022] Schematically, the multi-layer data model may include a conversion layer 102, an engine layer 103, and a consumer view layer 104. The engine layer is usually designed to process multiple frames of image data in a video. Therefore, before the audio and video data including audio information in the data model layer is input into the engine layer, the conversion layer 102 needs to split the audio and video data including audio information into a first video track including only pure image data and a second video track including only audio data. In this process, the conversion layer needs to create a first video track object and a second video track object according to the editing operation. For pure audio data in the original data layer, the conversion layer can create an audio conversion track object.

[0023] The conversion layer may send a first video track including only pure image data and a second video track including only audio data and an audio conversion track object to the engine layer.

[0024] The engine layer further processes the received data to generate video segment view data, segment video view data, audio segment view data, and segment audio view data that can be generated by the consumption view layer. The engine layer can generate an engine video track object according to the editing operation information in the first video track object. Generate an engine audio track object according to the editing operation in the second video track object. For an audio conversion object, an audio track object can be generated according to the editing operation, and so on.

[0025] The consumer view layer generates video fragment views and segmented video views based on the engine video track object, and generates audio fragment views and segmented views based on the engine audio track and audio track.

[0026] For complex editing scenarios, each editing operation requires the data layer to transfer the audio and video data to the conversion layer, which then performs corresponding processing in the background based on the editing operation. Therefore, there is data redundancy on the one hand, and operational redundancy of the same operations at different layers on the other hand.

[0027] In order to improve the data redundancy and operation redundancy problems of the above embodiment, for multimedia materials, the video editing client determines the engine layer attributes according to the difference between the data of the data layer and the data of the engine layer. The engine layer attributes are associated with the data of the data layer to obtain a data layer including the engine layer attributes, thereby simplifying the multi-layer data model.

[0028] Please refer to Figure 2 , Figure 2 A schematic diagram of another multimedia editing method provided in this embodiment is shown in FIG. Figure 2The multimedia editing method shown uses a data layer and a consumption view layer, and the consumption view layer directly consumes the data in the data layer. The multimedia data in the data layer is associated with the editing effect attributes. Figure 1 The data layer is consumed by the consumption view layer after passing through multiple data layers. The adjustment is made so that the consumption view layer directly consumes the data in the data layer, which can avoid data redundancy caused by the data processing of the data layer by the multi-layer data model.

[0029] When a user performs an editing operation on a multimedia track segment, how can you use Figure 2 The solution shown quickly generates a consumable view for the user's editing operation to generate multimedia content corresponding to the editing operation, which is the technical problem to be solved by this solution.

[0030] The solution provided by the present disclosure receives multiple editing operations, determines the time period corresponding to each editing operation on the timeline, and the editing operations are used to edit multimedia material track segments; in response to receiving an editing operation end instruction, determines a target time period covering each editing operation according to the time period corresponding to each editing operation; updates the target segment view data of the target time period according to the editing effects indicated by each editing operation; generates multimedia content to be displayed according to the target segment view data and the historical segment view data of the remaining time periods, so that when multiple editing operations are received, only the view data of the target time period corresponding to the editing operation can be updated, and the multimedia content corresponding to the editing operation can be generated, thereby improving the response speed to the editing operation.

[0031] Please refer to Figure 3 , Figure 3 Schematic diagram of a multimedia editing method provided by an embodiment of the present disclosure. Figure 3 As shown, the method includes:

[0032] S301: receiving multiple editing operations, and determining a time period corresponding to each editing operation on a timeline, wherein the editing operations are used to edit multimedia material track segments.

[0033] The execution subject of the multimedia editing method in the present disclosure may be a terminal device, specifically a client running in the terminal device for editing multimedia materials. In addition, the execution subject of the multimedia editing method may also be a server, for example, a server providing services for a multimedia editing client running in the terminal device.

[0034] The following description will be made by taking the multimedia editing client as an example.

[0035] The multimedia material track fragments may include audio material track fragments and / or video material track fragments.

[0036] The above editing operations include one or more of the following:

[0037] Delete tracks, delete multimedia material track segments, move multimedia material track segments, change the playback speed of multimedia material track segments, adjust the order between multimedia material track segments, and add multimedia material track segments.

[0038] The time period corresponding to the above-mentioned editing operation includes: the time period corresponding to the multimedia material track segment on which the editing operation is performed before the editing operation and / or the time period corresponding to the editing operation after the editing operation.

[0039] For example, for an editing operation of adding a new video material track segment, a time period between a start point and an end point of the new video material track segment on the timeline is determined as the time period corresponding to the editing operation of adding the new video segment.

[0040] For example, when deleting a video material track segment, it is determined that the time period on the time axis corresponding to the video material track segment before being deleted is the time period corresponding to the editing operation.

[0041] For example, to change the display time of a video material track segment, the time periods of the video segments before and after the change are determined as the time periods corresponding to the change operation.

[0042] S302: In response to receiving an editing operation end instruction, determining a target time period covering each editing operation according to time periods corresponding to each editing operation.

[0043] The target time period may be determined according to the time periods corresponding to the respective editing operations. The target time period may include the time periods corresponding to the respective editing operations.

[0044] As an implementation method, the earliest start point of the time period among the time periods corresponding to the above multiple editing operations can be used as the target start point of the target time period, and the latest end point of the time period among the time periods corresponding to the multiple editing operations can be used as the target end point of the target time period.

[0045] In some embodiments, the above step S302 includes:

[0046] First, determine the first starting point of the earliest time period among the time periods related to each editing operation;

[0047] Secondly, the starting point of the previous segment earlier than the first starting point is used as the starting point of the target time period.

[0048] Combine the following Figure 4 To explain, Figure 4 is a schematic diagram of an application scenario. Figure 4As shown, in the point editing interface of the multimedia editing client, there are tracks 1, 2, 3, 4, 5 and 6. Among them, video clips 10 and 20 are set in track 1, video clip 30 is set in track 2, video clip A1 is set in track 3, videos 40 and B1 are set in track 4; video clip 50 is set in track 5; and video clip 60 is set in track 6.

[0049] During the editing operation, the user successively moved the video material track segment A1 and the video material track segment B1 on their respective tracks. In this way, the time periods corresponding to the video material track segments A1 and B1 on the timeline have changed. For the video material track segment A1, after the video material track segment A1 is moved on the track, the time period T1 corresponding to the video material track segment A1 on the timeline before the editing operation and the time period T2 corresponding to the video material track segment A1 on the timeline after the editing operation can be recorded. For the video material track segment B1, after the video material track segment B1 is moved on track 4, the time period T3 corresponding to the video material track segment B1 on the timeline before the editing operation and the time period T4 corresponding to the video material track segment B1 on the timeline after the editing operation can be recorded. After the user moves the video material track segments A1 and B1 on their respective tracks, the user can click the preview control. After the above-mentioned execution subject receives the above-mentioned trigger operation performed on the preview control, it can be regarded that the user's editing operation is completed. The above-mentioned execution subject can determine the target time period according to the time information of the time periods corresponding to T1, T2, T3 and T4. Among the above-mentioned time periods, T1 is the frontmost time period on the time axis, and the starting point t6 of T1 can be used as the first starting point. In each segment, the starting point t5 of the previous segment of t6 is determined as the starting point of the target time period. Then the end point of the target time period is determined according to the end points of the time periods corresponding to T1, T2, T3 and T4. As shown in the figure, the end point of time period T4 is the latest point on the time axis, so the end point t12 of T4 can be used as the end point of the target time period. In this way, the target time period [t5, t12] can include the time period where the video material track segments A1 and A2 before and after the editing operation are located. The view of the target time period [t5, t12] can be regenerated. Specifically, it can be used Figure 5The method of the illustrated embodiment generates a video material track segment and a segmented view of a target time period. In this way, during the above-mentioned editing operation of the video material track segment, the time information of the time period involved in each editing operation after the editing operation is completed is used to determine the target time period, and then the view data in the target time period is regenerated, including the view data of the video segment and the segmented view data. Since the starting point of the target time period is earlier than the earliest time period involved in each editing operation on the time axis, it can be ensured that the regenerated view data covers the video segment affected by each editing operation, and no omission occurs.

[0050] S303: Update the target segment view data of the target time period according to the editing effects respectively indicated by the editing operations.

[0051] Before receiving the editing operation end instruction, the editing effect information of each editing operation can be recorded and associated with the editing effect information. Figure 2 The data layer shown includes multimedia material track data (such as video material track (video track) fragment data and audio material track (audio track) fragment data).

[0052] Each editing operation may correspond to a corresponding editing effect, and the editing effect may include, for example, display time, display speed, display level, etc. on the timeline.

[0053] The above time axis may be segmented in advance to obtain a plurality of time axis segments.

[0054] For each segment, view data for the segment may be determined.

[0055] In the target time period, the editing effect is applied to the multimedia material track segment data involved in the editing operation to obtain the multimedia segment view data of the target time period. The target segment view data is determined according to the multimedia segment view data of the target time period.

[0056] As an implementation method, the above step S303 includes the following steps:

[0057] First, determine the multimedia material track segment view data of the target time period;

[0058] Secondly, according to the starting point and the ending point of the updated multimedia material track segment data to which the editing effect is applied in the target time period, the target time period is re-segmented to obtain at least one updated segment;

[0059] Finally, for each updated segment, the view data of the updated segment is determined according to the multimedia material track fragment view data corresponding to the updated segment.

[0060] In the above implementation, for the target time period, the multimedia material track segment data and the editing effect can be first obtained from the data layer, and then the editing effect is applied to the multimedia material track segment data, and the multimedia material track segment view data of the target time period is generated according to the multimedia material track segment data to which the editing effect is applied.

[0061] After applying the editing effect to the multimedia material track segment in the target time period, the updated multimedia material track segment is obtained. The target time period is segmented according to the starting point and the ending point of the updated multimedia material track segment to obtain the updated segment. The multimedia material segment view data corresponding to each updated segment is determined, and the view data of the updated segment is determined. Thus, the view data of the target segment is obtained.

[0062] The specific implementation process of the above implementation method can also be referred to Figure 5 Description of the illustrated embodiment.

[0063] S304: Generate multimedia content to be displayed according to the target segment view data and the historical segment view data of other segments of the timeline.

[0064] The historical segment view data is segment view data generated before receiving the editing operation.

[0065] After receiving the above multiple editing operations, other segments on the timeline do not involve editing operations, so the view data of other segments may not be updated. The above other segment view data may be reused. The historical segment view data of other time periods and the target segment view data of the target time period may be used to generate the multimedia content to be displayed corresponding to the editing operation. For example, the historical segment view data and the target segment view data of the target time period are merged to obtain the multimedia content to be displayed.

[0066] In this embodiment, multiple editing operations are received, and the time period corresponding to each editing operation on the time axis is determined. The editing operation is used to edit the multimedia material track segment; in response to receiving the editing operation end instruction, the target time period covering each editing operation is determined according to the time period corresponding to each editing operation; the target segment view data of the target time period is updated according to the editing effect indicated by each editing operation; and the multimedia content to be displayed is generated according to the target segment view data and the historical segment view data of other segments of the time axis. Therefore, when multiple editing operations are received, only the view data of the target time period corresponding to the editing operation can be updated, and multimedia content responding to multiple editing operations can be generated. In the above-mentioned editing operation process, it is not necessary to generate view data for each editing operation. Only after receiving the end of the editing operation, the view data in the above-mentioned target time period is updated, so as to generate comprehensive response content to multiple editing operations, and it is not necessary to generate multimedia view data and segment view data on the entire time axis. Data redundancy is reduced, the efficiency of editing operations is improved, and the response speed to editing operations is also improved.

[0067] Please refer to Figure 5 , Figure 5 FIG. 1 is a schematic diagram of the process of obtaining the historical segment view data involved in step S303. Figure 5 As shown, the historical segment view data is determined based on the following steps:

[0068] S501: extracting multimedia material track segment data from the data layer using the consumption view layer, and creating multimedia material track segment view data according to each multimedia material track segment data.

[0069] The above data layer is Figure 2 The data layers shown. Figure 2 The data layer shown includes multimedia material track segment data. Before receiving the above-mentioned multiple editing operations, each multimedia material track segment may correspond to a historical editing effect. The historical editing effect may include the effect of the multimedia material track segment itself or the editing effect generated by the historical editing operation.

[0070] The consumer view layer can obtain multimedia material track fragment data and historical editing effects from the data layer.

[0071] Then, the historical editing effect is applied to the multimedia material track segment data, such as applying the layering information and / or the presentation speed to the multimedia material segment data, thereby constructing the multimedia segment view data.

[0072] In addition, the created multimedia material track fragment view data includes time information, that is, start time information and end time information on the timeline.

[0073] S502: Determine multiple segments of the timeline according to the start point and the end point of each multimedia material track segment on the timeline.

[0074] The time axis can be segmented by using the start point and the end point of each multimedia material track segment on the time axis as segmentation points, wherein the time axis area between any two adjacent segmentation points corresponds to a segment of the time axis, thereby obtaining multiple segments on the time axis.

[0075] Please refer to Figure 6 , which shows a schematic diagram of an application scenario. Figure 6 As shown, the starting point of multimedia segment A on the time axis is t1, and the ending point is t5; the starting point of multimedia segment B on the time axis is t4, and the ending point is t7. The starting point of multimedia segment C on the time axis is t1, and the ending point is t2; the starting point of multimedia segment D on the time axis is t3, and the ending point is t6. The above t1, t2, t3, t4, t5, t6 and t7 are respectively a segmentation point for segmenting the time axis. The area on the time axis between any two adjacent segmentation points is a segment.

[0076] S503: Determine the segment view data of each segment according to the multimedia material track fragment view data included in each segment.

[0077] For each segment, the execution subject may determine the multimedia segment view data corresponding to the segment according to the time information of the segment, and then create a segment view of the segment based on the multimedia segment view data corresponding to the segment.

[0078] For each segment, the time information of the segment may be used to determine the multimedia video segment view data associated with the segment, and then the view data of the segment may be created using the associated multimedia video segment view data.

[0079] In some embodiments, the above step S501 includes the following steps:

[0080] First, arrange the multimedia material track segments according to their respective appearance times on the timeline;

[0081] Secondly, starting from the zero point of the timeline, multiple multimedia material track segments are traversed to establish a set of multiple multimedia material track segments arranged in an orderly manner:

[0082] The starting point and the ending point of the multimedia segment are respectively used as the set keywords; for each set keyword, the multimedia segment starting after the set keyword is used as the key value of the set keyword to determine the multimedia segment set of the set keyword, wherein if the starting points are the same, they are classified into the same multimedia segment set.

[0083] The above-mentioned plurality of orderly arranged multimedia material track fragment sets constitute multimedia material track fragment view data.

[0084] Combine the following Figure 6 To explain, Figure 6 As shown, there are video material track segments A and B being edited on track 0. There are video material track segments C and D being edited on track 1. Among them, the starting point of video material track segment A on the timeline is t1, and the ending point is t5; the starting point of video material track segment C on the timeline is t1, and the ending point is t2; the starting point of video material track segment B on the timeline is t4, and the ending point is t7; the starting point of video material track segment D on the timeline is t3, and the ending point is t6.

[0085] Starting from the zero point of the timeline, traverse the video material track segments A, B, C, and D, and the starting point and ending point of each video material track segment such as t1, t2, t3, t4, t5, t6, and t7 are divided into set keywords. For the set keyword t1, the video material track segments A and C starting with the set keyword t1 are the key values ​​of the set keyword t1, and the video generation track segment set (t1, {A, C}) of the set keyword t1 is determined. For the set keyword t2, there is no video segment starting with t2 in the video material track segment. Therefore, the set keyword t2 corresponds to the empty set, and the multimedia segment set corresponding to the set keyword t2 is expressed as (t2, {}). Similarly, the video material track segment set (t3, {D}) corresponding to the set keyword t3 can be determined; the video material track segment set (t4, {B}) corresponding to the set keyword t4; the multimedia segment set (t5, {}) corresponding to the set keyword t5; the multimedia segment set (t6, {}) corresponding to the set keyword t6; and the video material track segment set (t7, {}) corresponding to the set keyword t5. Thus, the ordered video material track segment sets (t1, {A, C}), (t2, {}), (t3, {D}), (t4, {B}), (t5, {}), (t6, {}), (t7, {}) arranged in time are obtained. The above ordered video segment sets constitute the video material track segment view data.

[0086] In some embodiments, the above step S503 includes the following steps:

[0087] First, for each segment, the starting point of the segment is used as the segment keyword; the multimedia material track segment set corresponding to the segment keyword is traversed, and if the starting point of a multimedia material track segment in the multimedia material track segment set is equal to the segment keyword, the multimedia material track segment view data is added to the segment view data of the segment.

[0088] Secondly, the historical segments are traversed, and if the end point of the multimedia material track fragment included in the historical segment is not equal to the end point of the segment, the view data of the multimedia material track fragment is added to the segment view data of the segment.

[0089] Still Figure 6 Take an example to illustrate, after determining a plurality of video material track segment sets arranged in an orderly manner, a set keyword is used as a segment start point, and a subsequent set keyword is used as a segment end point to establish a segment. Here, because the above set keyword is the start point and end point of the video material track segment, therefore, the time axis is segmented by using a set keyword as a segment start point and a subsequent set keyword as a segment end point, and the time axis is segmented by using the start point and end point of the video material track segment in essence, which is the same segmentation scheme.

[0090] For example, [t1, t2], [t2, t3], [t3, t4], [t4, t5], [t5, t6], and [t6, t7] are the segmentation results respectively.

[0091] Taking the segment [t3, t4] as an example, t3 is used as the segment keyword. Traverse the video material track segment set. If the video segment D in the video material track segment set has a starting point equal to t3, it is used as the video material track segment corresponding to the segment. In addition, the segment keyword of the historical segment [t2, t3] is t2, and t2 does not have a video material track segment starting with it. However, the end point of A in the video segment corresponding to its historical segment [t1, t2] is not t3, so the historical segment [t2, t3] includes the video material track segment A. For the segment [t3, t4], t4 is also not the end point of the video material track segment A, so the video material track segment A is also the video material track segment included in the segment [t3, t4]. Therefore, the video segments included in the segment [t3, t4] include D and A.

[0092] The segment view data of each segment determined according to the above method is as follows:

[0093] The segment view data of segment [t1, t2] includes view data of video material track segments A and C; the segment view data of segment [t2, t3] includes view data of video material track segment A; the view data of segment [t3, t4] includes view data of video material track segments D and A; the view data of segment [t4, t5] includes view data of video material track segments D, A and B; the view data of segment [t5, t6] includes view data of video material track segments D and B; the view data of segment [t6, t7] includes view data of video material track segment B.

[0094] In this way, each segment view data depends on the corresponding video material track fragment view data, thus breaking the relationship between the segment view data and the track. It is convenient to directly create view data through the multimedia data of the data layer, and improve the response speed of editing operations.

[0095] Corresponding to the above Figure 3 The multimedia editing method of the embodiment, Figure 7 This is a structural block diagram of a multimedia editing device provided by an embodiment of the present disclosure. For ease of explanation, only the parts related to the embodiment of the present disclosure are shown. Figure 7 The device 70 includes: a first determining unit 701, a second determining unit 702, an updating unit 703 and a generating unit 704. Among them:

[0096] A first determining unit 701 is used to receive multiple editing operations and determine a time period corresponding to each editing operation on the time axis, wherein the editing operation is used to edit a multimedia material track segment;

[0097] The second determining unit 702 is configured to determine, in response to receiving the editing operation end instruction, a target time period covering each editing operation according to the time periods corresponding to each editing operation;

[0098] An updating unit 703, configured to update the target segment view data of the target time period according to the editing effects respectively indicated by the editing operations;

[0099] The generating unit 704 is configured to generate multimedia content to be displayed according to the target segment view data and the historical segment view data of other time periods.

[0100] In some embodiments of the present disclosure, the device 70 further includes a historical segment data determination unit (not shown in the figure). The historical segment data determination unit is used to determine the historical segment view data based on the following steps:

[0101] Extract multimedia material track segment data from the data layer, and create multimedia material track segment view data according to each multimedia material track segment data;

[0102] Determine multiple segments of the timeline according to the start point and the end point of each multimedia material track segment on the timeline;

[0103] The segment view data of each segment is determined according to the multimedia material track fragment view data included in each segment.

[0104] In some embodiments of the present disclosure, the historical segment data determination unit is further configured to:

[0105] Arrange the multimedia material track segments according to their respective appearance times on the timeline;

[0106] Starting from the zero point of the timeline, traverse multiple multimedia material track segments to establish a set of multiple multimedia material track segments arranged in an orderly manner:

[0107] The starting point and the ending point of the multimedia segment are respectively used as the set keywords; for each set keyword, the multimedia segment starting after the set keyword is used as the key value of the set keyword to determine the multimedia segment set of the set keyword, wherein if the starting points are the same, they are classified into the same multimedia segment set.

[0108] In some embodiments of the present disclosure, the historical segment data determination unit is further configured to:

[0109] For each segment, the starting point of the segment is used as the segment keyword; the multimedia material track segment set corresponding to the segment keyword is traversed, and if the starting point of a multimedia material track segment in the multimedia material track segment set is equal to the segment keyword, the multimedia material track segment view data is added to the segment view data of the segment;

[0110] The historical segments are traversed, and if the end point of the multimedia material track fragment included in the historical segment is not equal to the end point of the segment, the view data of the multimedia material track fragment is added to the segment view data of the segment.

[0111] In some embodiments of the present disclosure, the second determining unit 702 is further configured to:

[0112] Determine the first starting point of the earliest time period among the time periods related to the editing operations;

[0113] The starting point of the previous segment earlier than the first starting point is used as the starting point of the target time period.

[0114] In some embodiments of the present disclosure, the updating unit 703 is further configured to: determine multimedia material track segment view data of a target time period;

[0115] Re-segmenting the target time period according to the starting point and the ending point of the updated multimedia material track segment data to which the editing effect is applied within the target time period to obtain at least one updated segment;

[0116] For each updated segment, the view data of the updated segment is determined according to the multimedia material track fragment view data corresponding to the updated segment.

[0117] In some embodiments of the present disclosure, the multimedia data includes:

[0118] Audio data and / or video data.

[0119] In some embodiments of the present disclosure, the editing operation includes one or more of the following:

[0120] Delete multimedia tracks, delete multimedia clips, move multimedia clips, change the playback speed of multimedia clips, adjust the order between multimedia clips, and add multimedia track clips.

[0121] In order to implement the above embodiment, the embodiment of the present disclosure also provides an electronic device.

[0122] refer to Figure 8 , which shows a schematic diagram of the structure of an electronic device 800 suitable for implementing an embodiment of the present disclosure, and the electronic device 800 may be various terminal devices and servers. Among them, the terminal device may include an extended reality device. The extended reality device may include, for example, various wearable devices, such as a head-mounted extended reality device, etc. Figure 8 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0123] like Figure 8 As shown, the electronic device 800 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage device 808 to a random access memory (RAM) 803. Various programs and data required for the operation of the electronic device 800 are also stored in the RAM 803. The processing device 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0124] Typically, the following devices may be connected to the I / O interface 805: input devices 806 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 807 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 808 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 809. The communication device 809 may allow the electronic device 800 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 8 The electronic device 800 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0125] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 809, or installed from a storage device 808, or installed from a ROM 802. When the computer program is executed by the processing device 801, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0126] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0127] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0128] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.

[0129] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0130] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0131] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of a unit does not limit the unit itself in some cases. For example, the first determination unit may also be described as "a unit that receives multiple editing operations and determines the time period corresponding to each editing operation on the time axis".

[0132] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0133] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0134] In a first aspect, according to one or more embodiments of the present disclosure, a multimedia editing method is provided, comprising:

[0135] Receiving multiple editing operations, determining a time period corresponding to each editing operation on a timeline, the editing operations being used to edit a multimedia material track segment;

[0136] In response to receiving the editing operation end instruction, determining a target time period covering each editing operation according to the time periods corresponding to each editing operation;

[0137] Update the target segment view data of the target time period according to the editing effects respectively indicated by each editing operation;

[0138] The multimedia content to be displayed is generated according to the target segment view data and the historical segment view data of other segments of the timeline.

[0139] According to one or more embodiments of the present disclosure, the historical segment view data corresponding to the time axis is obtained based on the following steps:

[0140] Extract multimedia material track segment data from the data layer, and create multimedia material track segment view data according to each multimedia material track segment data;

[0141] Determine multiple segments of the timeline according to the start point and the end point of each multimedia material track segment on the timeline;

[0142] The segment view data of each segment is determined according to the multimedia material track fragment view data included in each segment.

[0143] According to one or more embodiments of the present disclosure, extracting multimedia material track fragment data from a data layer, and creating multimedia material track fragment view data according to each multimedia material track fragment data includes:

[0144] Arrange the multimedia material track segments according to their respective appearance times on the timeline;

[0145] Starting from the zero point of the timeline, traverse multiple multimedia material track segments to establish a set of multiple multimedia material track segments arranged in an orderly manner:

[0146] The starting point and the ending point of the multimedia segment are respectively used as the set keywords; for each set keyword, the multimedia segment starting after the set keyword is used as the key value of the set keyword to determine the multimedia segment set of the set keyword, wherein if the starting points are the same, they are classified into the same multimedia segment set.

[0147] According to one or more embodiments of the present disclosure, determining the segment view data of each segment according to the multimedia material track fragment view data included in each segment includes:

[0148] For each segment, the starting point of the segment is used as the segment keyword; the multimedia material track segment set corresponding to the segment keyword is traversed, and if the starting point of a multimedia material track segment in the multimedia material track segment set is equal to the segment keyword, the multimedia material track segment view data is added to the segment view data of the segment;

[0149] The historical segments are traversed, and if the end point of the multimedia material track fragment included in the historical segment is not equal to the end point of the segment, the view data of the multimedia material track fragment is added to the segment view data of the segment.

[0150] According to one or more embodiments of the present disclosure, in response to receiving an editing operation end instruction, determining time periods related to each editing operation on a time axis, and determining a target time period according to the time periods respectively related to each editing operation, includes:

[0151] Determine the first starting point of the earliest time period among the time periods related to the editing operations;

[0152] The starting point of the previous segment earlier than the first starting point is used as the starting point of the target time period.

[0153] According to one or more embodiments of the present disclosure, updating target segment view data of a target time period according to the editing effects respectively indicated by each editing operation includes:

[0154] Determine multimedia material track fragment view data of a target time period;

[0155] Re-segmenting the target time period according to the starting point and the ending point of the updated multimedia material track segment data to which the editing effect is applied within the target time period to obtain at least one updated segment;

[0156] For each updated segment, the view data of the updated segment is determined according to the multimedia material track fragment view data corresponding to the updated segment.

[0157] According to one or more embodiments of the present disclosure, the multimedia data includes:

[0158] Audio data and / or video data.

[0159] According to one or more embodiments of the present disclosure, the editing operation includes one or more of the following:

[0160] Delete tracks, delete multimedia material track segments, move multimedia material track segments, change the playback speed of multimedia material track segments, adjust the order between multimedia material track segments, and add multimedia material track segments.

[0161] In a second aspect, according to one or more embodiments of the present disclosure, a multimedia editing device is provided, comprising:

[0162] A first determining unit, configured to receive a plurality of editing operations and determine a time period corresponding to each editing operation on a time axis, wherein the editing operation is used to edit a multimedia material track segment;

[0163] A second determining unit, configured to determine, in response to receiving an editing operation end instruction, a target time period covering each editing operation according to time periods corresponding to each editing operation;

[0164] An updating unit, configured to update the target segment view data of the target time period according to the editing effects respectively indicated by the editing operations;

[0165] The generating unit is used to generate multimedia content to be displayed according to the target segment view data and the historical segment view data of other time periods.

[0166] According to one or more embodiments of the present disclosure, the device further includes a historical segment data determination unit. The historical segment data determination unit is configured to determine the historical segment view data based on the following steps:

[0167] Extract multimedia material track segment data from the data layer, and create multimedia material track segment view data according to each multimedia material track segment data;

[0168] Determine multiple segments of the timeline according to the start point and the end point of each multimedia material track segment on the timeline;

[0169] The segment view data of each segment is determined according to the multimedia material track fragment view data included in each segment.

[0170] According to one or more embodiments of the present disclosure, the historical segment data determination unit is further configured to:

[0171] Arrange the multimedia material track segments according to their respective appearance times on the timeline;

[0172] Starting from the zero point of the timeline, traverse multiple multimedia material track segments to establish a set of multiple multimedia material track segments arranged in an orderly manner:

[0173] The starting point and the ending point of the multimedia segment are respectively used as the set keywords; for each set keyword, the multimedia segment starting after the set keyword is used as the key value of the set keyword to determine the multimedia segment set of the set keyword, wherein if the starting points are the same, they are classified into the same multimedia segment set.

[0174] According to one or more embodiments of the present disclosure, the historical segment data determination unit is further configured to:

[0175] For each segment, the starting point of the segment is used as the segment keyword; the multimedia material track segment set corresponding to the segment keyword is traversed, and if the starting point of a multimedia material track segment in the multimedia material track segment set is equal to the segment keyword, the multimedia material track segment view data is added to the segment view data of the segment;

[0176] The historical segments are traversed, and if the end point of the multimedia material track fragment included in the historical segment is not equal to the end point of the segment, the view data of the multimedia material track fragment is added to the segment view data of the segment.

[0177] According to one or more embodiments of the present disclosure, the second determining unit is further configured to:

[0178] Determine the first starting point of the earliest time period among the time periods related to the editing operations;

[0179] The starting point of the previous segment earlier than the first starting point is used as the starting point of the target time period.

[0180] According to one or more embodiments of the present disclosure, the updating unit is further configured to: determine multimedia material track segment view data of a target time period;

[0181] Re-segmenting the target time period according to the starting point and the ending point of the updated multimedia material track segment data to which the editing effect is applied within the target time period to obtain at least one updated segment;

[0182] For each updated segment, the view data of the updated segment is determined according to the multimedia material track fragment view data corresponding to the updated segment.

[0183] According to one or more embodiments of the present disclosure, the multimedia data includes:

[0184] Audio data and / or video data.

[0185] According to one or more embodiments of the present disclosure, the editing operation includes one or more of the following:

[0186] Delete tracks, delete multimedia material track segments, move multimedia material track segments, change the playback speed of multimedia material track segments, adjust the order between multimedia material track segments, and add multimedia material track segments.

[0187] In a third aspect, according to one or more embodiments of the present disclosure, there is provided an electronic device, comprising: at least one processor and a memory;

[0188] Memory stores computer-executable instructions;

[0189] At least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the first aspect as well as various multimedia editing methods that may be involved in the first aspect.

[0190] In a fourth aspect, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, in which computer execution instructions are stored. When a processor executes the computer execution instructions, the multimedia editing method as described in the first aspect and various possible aspects involved in the first aspect are implemented.

[0191] In a fifth aspect, according to one or more embodiments of the present disclosure, a computer program product is provided, including a computer program, which, when executed by a processor, implements the first aspect as well as various multimedia editing methods that may be involved in the first aspect.

[0192] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0193] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0194] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A multimedia editing method, comprising: Receiving multiple editing operations and determining a time period corresponding to each editing operation on a timeline, wherein the editing operations are used to edit multimedia material track segments; In response to receiving the editing operation end instruction, determining a target time period covering each editing operation according to the time periods corresponding to each editing operation; Update the target segment view data of the target time period according to the editing effects respectively indicated by the editing operations; The multimedia content to be displayed is generated according to the target segment view data and the historical segment view data of other segments of the timeline.

2. The method according to claim 1, characterized in that The historical segment view data corresponding to the time axis is obtained based on the following steps: Extract multimedia material track segment data from the data layer, and create multimedia material track segment view data according to each multimedia material track segment data; Determine multiple segments of the timeline according to the start point and the end point of each multimedia material track segment on the timeline; The segment view data of each segment is determined according to the multimedia material track fragment view data included in each segment.

3. The method according to claim 2, characterized in that The step of extracting multimedia material track segment data from the data layer and creating multimedia material track segment view data according to each multimedia material track segment data includes: Arrange the multimedia material track segments according to their respective appearance times on the timeline; Starting from the zero point of the timeline, traverse multiple multimedia material track segments to establish a set of multiple multimedia material track segments arranged in an orderly manner: The starting point and the ending point of the multimedia segment are respectively used as the set keywords; for each set keyword, the multimedia segment starting after the set keyword is used as the key value of the set keyword to determine the multimedia segment set of the set keyword, wherein if the starting points are the same, they are classified into the same multimedia segment set.

4. The method according to claim 3, characterized in that The determining of the segment view data of each segment according to the multimedia material track fragment view data included in each segment comprises: For each segment, the starting point of the segment is used as the segment keyword; the multimedia material track segment set corresponding to the segment keyword is traversed, and if the starting point of a multimedia material track segment in the multimedia material track segment set is equal to the segment keyword, the multimedia material track segment view data is added to the segment view data of the segment; The historical segments are traversed, and if the end point of the multimedia material track fragment included in the historical segment is not equal to the end point of the segment, the view data of the multimedia material track fragment is added to the segment view data of the segment.

5. The method according to claim 1, characterized in that In response to receiving the editing operation end instruction, determining time periods related to each editing operation on the time axis, and determining a target time period according to the time periods related to each editing operation, includes: Determine the first starting point of the earliest time period among the time periods related to the editing operations; The starting point of the previous segment earlier than the first starting point is used as the starting point of the target time period.

6. The method according to claim 1, characterized in that The updating of the target segment view data of the target time period according to the editing effects respectively indicated by the editing operations includes: Determine multimedia material track fragment view data of a target time period; Re-segmenting the target time period according to the starting point and the ending point of the updated multimedia material track segment data to which the editing effect is applied within the target time period to obtain at least one updated segment; For each updated segment, the view data of the updated segment is determined according to the multimedia material track fragment view data corresponding to the updated segment.

7. The method according to claim 1, characterized in that The multimedia data includes: Audio data and / or video data.

8. The method according to any one of claims 1 to 7, characterized in that The editing operation includes one or more of the following: Delete tracks, delete multimedia material track segments, move multimedia material track segments, change the playback speed of multimedia material track segments, adjust the order between multimedia material track segments, and add multimedia material track segments.

9. A multimedia editing device, comprising: A first determining unit, configured to receive a plurality of editing operations and determine a time period corresponding to each editing operation on a time axis, wherein the editing operation is used to edit a multimedia material track segment; A second determining unit, configured to determine, in response to receiving an editing operation end instruction, a target time period covering each editing operation according to time periods corresponding to each editing operation; An updating unit, configured to update the target segment view data of the target time period according to the editing effects respectively indicated by the editing operations; A generating unit is used to generate multimedia content to be displayed according to the target segment view data and the historical segment view data of other time periods.

10. An electronic device, comprising: Processor and memory; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions. When a processor executes the computer-executable instructions, the method according to any one of claims 1 to 8 is implemented.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images