Multimedia editing method, device and electronic equipment

Abstract

The embodiment of the present disclosure provides a multimedia editing method, device and electronic equipment, the method comprises the following steps: generating initial data layer data according to an editing operation on multimedia material, wherein the initial data layer data comprises multimedia track segment data obtained by the editing operation; determining an editing effect attribute indicated by the editing operation, associating the editing effect attribute with the initial data layer data, and taking the initial data layer data associated with the editing effect attribute as target data layer data; according to a received display instruction, acquiring multimedia track segment data and the editing effect attribute from the target data layer data by using a consumption view layer, and generating multimedia segment content after the editing operation based on the multimedia track segment data and the editing effect attribute. Data redundancy in the multimedia editing process is reduced.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a multimedia editing method, apparatus, and electronic device. Background Technology

[0002] Multimedia data can include audio and / or video data. Users can perform various edits on multimedia data, such as merging different audio and video clips, adding background music to video clips, and adding special effects. Through these audio and video edits, a work that achieves the user's intended result can be created. Summary of the Invention

[0003] This disclosure provides a multimedia editing method, apparatus, and electronic device.

[0004] In a first aspect, embodiments of this disclosure provide a multimedia editing method, the method comprising: generating initial data layer data based on an editing operation on multimedia material, the initial data layer data including multimedia material track fragment data obtained by the editing operation; determining an editing effect attribute indicated by the editing operation, associating the editing effect attribute with the initial data layer data, and using the initial data layer data associated with the editing effect attribute as target data layer data; and, based on a received display instruction, obtaining the multimedia material track fragment data and the editing effect attribute from the target data layer data using a consumer view layer, and generating multimedia fragment content after the editing operation based on the multimedia material track fragment data and the editing effect attribute.

[0005] Secondly, embodiments of this disclosure provide a multimedia editing apparatus, comprising: a first generation unit, configured to generate initial data layer data based on an editing operation on multimedia materials, the initial data layer data including multimedia material track fragment data obtained by the editing operation; a determining unit, configured to determine an editing effect attribute indicated by the editing operation, associate the editing effect attribute with the initial data layer data, and use the initial data layer data associated with the editing effect attribute as target data layer data; and a second generation unit, configured to, based on a received display instruction, obtain the multimedia material track fragment data and the editing effect attribute from the target data layer data using a consumer view layer, and generate multimedia fragment content after the editing operation based on the multimedia material track fragment data and the editing effect attribute.

[0006] Thirdly, embodiments of this disclosure provide an electronic device, including: a processor and a memory; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory, causing the at least one processor to perform the first aspect above and various multimedia editing methods that may be involved in the first aspect.

[0007] Fourthly, embodiments of this disclosure provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the first aspect above and various possible multimedia editing methods involved in the first aspect.

[0008] Fifthly, embodiments of this disclosure provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect above and various possible multimedia editing methods involved in the first aspect.

[0009] The multimedia editing method, apparatus, and electronic device provided in this embodiment generate initial data layer data based on editing operations on multimedia materials. This initial data layer data includes multimedia material track segment data obtained from the editing operations. The method determines the editing effect attribute indicated by the editing operation, associates the editing effect attribute with the initial data layer data, and uses the initial data layer data associated with the editing effect attribute as the target data layer data. Based on a received display instruction, the method uses a consumer view layer to obtain the multimedia material track segment data and the editing effect attribute from the target data layer data, and generates the multimedia segment content after the editing operation based on the multimedia material track segment data and the editing effect attribute. This reduces data redundancy during multimedia editing. Furthermore, it avoids inconsistencies between the displayed effect and the editing operation caused by inconsistencies in multi-layer data. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a schematic diagram illustrating an application scenario of multimedia editing provided by an embodiment of the present disclosure;

[0012] Figure 2 A schematic diagram of a multimedia editing process provided in this embodiment of the disclosure;

[0013] Figure 3 A flowchart illustrating a multimedia editing method provided in this embodiment of the disclosure. Figure 1 ;

[0014] Figure 4 A schematic diagram of the multimedia editing process provided in this embodiment of the disclosure;

[0015] Figure 5Flowchart of the multimedia editing method provided in this disclosure Figure 2 ;

[0016] Figure 6 This is a schematic diagram of an application scenario;

[0017] Figure 7 This is a schematic diagram of an application scenario;

[0018] Figure 8 A structural block diagram of a multimedia editing device provided in this disclosure embodiment;

[0019] Figure 9 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0021] Please refer to Figure 1 The diagram illustrates an application scenario of multimedia editing according to this disclosure. The multimedia editing interface includes a timeline. In this editing operation, partial segments of video material 1 and video material 2 are imported, forming two video material track segments 11 and 12. Users can add background music, special effects, etc., to the video material track segments in subsequent editing operations.

[0022] In an example of a multimedia editing method, the physical data structure presented by the business logic or user interface can be abstracted into a data layer by a data model layer. Then, multiple layers of preset model layers process the video and audio track data in the data layer, ultimately resulting in the consumer view layer. Please refer to [reference needed]. Figure 2 , Figure 2 The aforementioned preset model layers include a conversion layer 202 and an engine layer 203. The data in the data layer 201 can be video and audio track segment data obtained by the user editing audio and / or video materials in the multimedia editing interface. Video track segment data includes data for video track segments, image track segments, and sticker track segments, etc. Transition effects can also exist between video track segments and their consecutive image track segments. In the conversion layer, multiple video conversion track objects are generated for the video track segment data; at least one audio conversion track object is generated for the audio track data. For example... Figure 2As shown, in the conversion layer 202, two video conversion objects 2021 and 2022 are generated for the video track data in the data layer 201. One video conversion track object can include video, image, and sticker track segments extracted from audio and video material track segments, as well as transitions between videos and images; the other video conversion track object includes audio extracted from the audio and video material track segments. An audio conversion track object is generated for the audio track data in the data layer 201, which can include audio material track segment data, etc.

[0023] In the conversion layer, the audio and video material track segments are separated into audio and video, splitting the original video material track segment data into two video material conversion track data.

[0024] After obtaining the data from the conversion layer, further data processing yields the engine layer data. Each video conversion track object in the conversion layer can be further processed into multiple engine video track data, generating a data object for each engine video track. For example, video conversion track 2021 in the conversion layer is processed into two engine video tracks 2031 and 2032 in the engine layer. Engine video track 2031 can include video data, transition data, and data corresponding to sticker material track segments. Engine video track 2032 also includes transition data, image material data, and sticker data.

[0025] The audio data in video conversion track 2022 is also correspondingly split into multiple engine audio tracks 2033 and 2034 in the engine layer. Similarly, audio conversion track 2023 is also correspondingly split into multiple audio tracks in the engine layer, such as... Figure 2 The audio track data in the file is 2035 and 2036.

[0026] Finally, video clip views and video slice views can be generated based on the engine video track data 2031 and 2032 in engine layer 203. Audio clip views and audio slice views can be generated based on the engine audio track data 2033 and 2034 and audio track data 2035 and 2036 in engine layer.

[0027] However, as business complexity increases, more intermediate data layers may emerge, such as editing model layers and rendering model layers. With an increased number of intermediate data layers, much of the data in the process of reaching the engine layer from the data layer through these multiple intermediate layers is identical: such as file paths, original video information, and video speed. Therefore, a large amount of redundant data exists. Furthermore, with an increased number of layers, maintaining the mapping, updating, and consistency of data between data layers requires significant maintenance costs.

[0028] To address the aforementioned issues, the solution provided in this disclosure generates initial data layer data based on editing operations on multimedia materials. This initial data layer data includes multimedia material track segment data obtained from the editing operations. It then determines the editing effect attributes of the editing operation instructions, associates these attributes with the initial data layer data, and uses this associated initial data layer data as the target data layer. Based on received display instructions, it retrieves the multimedia material track segment data and editing effect attributes from the target data layer using the consumer view layer, and generates the edited multimedia segment content based on these attributes. Since this eliminates the need for multiple preset models, it avoids additional object construction and duplicate data, reducing data redundancy. Furthermore, it addresses the issue of inconsistent effect display and editing operations caused by inconsistencies in multi-layered data.

[0029] Please refer to Figure 3 , Figure 3 A flowchart illustrating a multimedia editing method provided in this embodiment of the disclosure. Figure 1 .like Figure 3 As shown, the method includes:

[0030] S301: Generate initial data layer data based on the editing operations on multimedia materials. The initial data layer data includes the multimedia material track segment data obtained from the editing operations.

[0031] The multimedia editing method disclosed herein can be executed by a terminal device, specifically a multimedia editing client running on the terminal device for editing multimedia materials. Alternatively, the multimedia editing method can also be executed by a server, such as a server that provides services to the multimedia editing client running on the terminal device.

[0032] The following explanation uses a multimedia editing client as an example of the execution entity mentioned above.

[0033] Users can perform editing operations on multimedia materials within the multimedia editing client interface. These editing operations include adding multimedia material track clips, cutting multimedia material track clips, moving multimedia material track clips within a track, adjusting the playback speed of multimedia material track clips, setting the hierarchy of multimedia material track clip data, and adding special effects.

[0034] For example, multimedia footage can be split into segments on a timeline, and the resulting segments can be placed on different timelines or the same timeline. Each of these segments constitutes a multimedia footage timeline segment. Transition effects can be added between two consecutive multimedia footage timeline segments on the same timeline.

[0035] The aforementioned multimedia material track segments include audio material track segments and / or video material track segments.

[0036] For video footage clips, the track can also include each sticker, effect, text, image fragment, etc. For example... Figure 1 The video footage track clip data can include data from tracks 11 and 12.

[0037] The audio clips here can be segments extracted from audio materials. These audio materials may be unrelated to the audio / video materials corresponding to the video clips, or they may be audio materials obtained after audio separation from the aforementioned audio / video materials. Editing operations on the audio clips can include fade-in, fade-out, etc.

[0038] The initial data layer of a multimedia material track segment includes data storage path, original multimedia material information, etc.

[0039] Please refer to Figure 4 , Figure 4 A schematic diagram illustrating the multimedia editing process provided in this embodiment of the disclosure. Figure 4 Data layer 401 is the target data layer. The video clips (audio, video, transitions, images, stickers, etc.) in the video track of the target data layer can be the initial video clip data from the initial data layer. The aforementioned editing effect attributes can be calculated from the initial video clip data through corresponding editing operations.

[0040] To illustrate, the layer information and speed settings of image A (which corresponds to a track in the data layer) in the video footage track clip are used as the editing effect attributes of the image A footage track clip.

[0041] To illustrate, the layer information and speed settings of video clip B (which corresponds to a track in the data layer) are used as editing effect attributes for video clip B.

[0042] S302: Determine the editing effect attribute of the editing operation instruction, associate the editing effect attribute with the initial data layer data, and use the initial data layer data associated with the editing effect attribute as the target data layer data.

[0043] The editing effect attributes of the above editing operations indicate the editing effect of the corresponding multimedia material track segment. These editing effect attributes include, but are not limited to, one or more of the following: playback time attribute, playback speed attribute, and layer attribute.

[0044] The aforementioned editing effect attributes can be associated with the initial data layer data. Specifically, the editing effect attributes indicated by the editing operation can be associated with the initial data layer data of the multimedia material track segment operated on by the editing operation. For example, if the editing operation sets a playback speed X1 for multimedia material track segment A, then the playback speed X1 is associated with the initial data layer data of multimedia material track segment A.

[0045] If the editing operation modifies the display time T of multimedia material track segment B on the timeline, then the aforementioned display time T will be associated with the initial data layer data of multimedia material track segment B.

[0046] After associating the editing effect attributes corresponding to each editing operation with the initial data layer data of the multimedia material track segment operated by each editing operation, the initial data layer data associated with the editing effect attributes is used as the target data layer data.

[0047] S303: Based on the received display instructions, use the consumer view layer to obtain multimedia material track segment data and editing effect attributes from the target data layer, and generate the multimedia segment content after the editing operation based on the multimedia material track segment data and editing effect attributes.

[0048] After a user performs an editing operation, they can send a display command to the aforementioned execution entity. This command displays the corresponding effect of the editing operation. Upon receiving the display command, the execution entity can forward it to the consumer view layer. The consumer view layer then retrieves multimedia material track segment data and editing effect data from the target data layer. Based on the multimedia material track segment data and editing effect attributes, the multimedia segment content after the editing operation can be generated.

[0049] The aforementioned multimedia material track segment content can be obtained by applying editing effects to the multimedia material track segment data and then merging the track segment data of different multimedia materials.

[0050] exist Figure 2In the multimedia editing method shown, within a multi-layered preset model, objects are constructed for all multimedia material track segments on all tracks at each layer, generating object data. In the conversion layer, multimedia material track segments, including audio and video data, are split into pure video track segments and audio / video track segments. Each video conversion track object in the conversion layer can be further processed into multiple engine video track data, generating a data object for each engine video track data. The objects at each of the above layers can be obtained from the initial multi-data in the data layer. For example, when modifying the playback speed of a multimedia material track segment, the playback speed corresponding to the modification operation can be obtained by multiplying the playback speed of the original data of the multimedia material track segment by the aforementioned speed-up indicator. When modifying the display time of a multimedia material track segment, the target display time corresponding to the modification operation can be obtained by moving the original multimedia material track segment along the timeline.

[0051] Based on this, the editing effect attributes corresponding to the editing operation can be determined. These editing effect attributes are then bound to the initial data of the multimedia material track segment in the initial data layer of the multimedia material track segment to obtain the target data layer data. The consumption view layer then directly extracts the initial data and editing effect attributes of the multimedia material track segment from the target data layer for consumption. The consumption effect can achieve... Figure 2 The effect of consuming data extracted from the engine layer in the multi-layer preset model layer.

[0052] The solution provided in this embodiment generates initial data layer data based on editing operations on multimedia materials. This initial data layer data includes multimedia material track fragment data obtained from the editing operations. It determines the editing effect attributes indicated by the editing operations, associates these attributes with the initial data layer data, and uses this associated initial data layer data as the target data layer data. Based on the received display instructions, it uses the consumer view layer to obtain the multimedia material track fragment data and editing effect attributes from the target data layer data, and generates the edited multimedia fragment content based on these attributes. Because it eliminates the need for multiple preset models, it avoids additional object construction and duplicate data, reducing data redundancy. Furthermore, it addresses the issue of inconsistent effect display and editing operations caused by inconsistencies in multi-layered data.

[0053] In some embodiments, step S303 above includes the following steps:

[0054] First, the consumer view layer is used to obtain the first video material track segment data containing audio information from the target data layer;

[0055] Secondly, extract the second video clip data (with audio information removed) from the first video clip track segment data;

[0056] Finally, the track segment data of the first video material is used as the corresponding audio track segment data.

[0057] For the first video clip track segment data containing audio information, image sequences are extracted from this video clip track segment. These image sequences can then be used to construct the second video clip track segment data. These image sequences are then used to build the video clip view.

[0058] The first video clip track segment data is directly used as the corresponding audio track segment data to construct the audio segment view.

[0059] In these embodiments, for video clip data that includes audio, using the image data in the video clip track segment to construct the video clip view, and using the video clip track segment to construct the audio clip view, can avoid data redundancy caused by splitting and saving the audio video clip track segment data separately in order to construct the audio clip view and the video clip view.

[0060] Please refer to Figure 5 It illustrates a schematic flow of the multimedia editing method provided in this disclosure. Figure 2 ,like Figure 5 As shown, the method includes the following steps:

[0061] S501: Generate initial data layer data based on the editing operations on multimedia materials. The initial data layer data includes the multimedia material track segment data obtained from the editing operations.

[0062] S502: Determine the editing effect attribute of the editing operation instruction, associate the editing effect attribute with the initial data layer data, and use the initial data layer data associated with the editing effect attribute as the target data layer data.

[0063] For detailed implementation of steps S501 to S502, please refer to Figure 3 The relevant parts of the illustrated embodiment will not be described in detail here.

[0064] S503: In the consumer view layer, apply editing effect attributes to the multimedia material track segment data to obtain the multimedia material track segment view data.

[0065] Please continue to refer to this. Figure 4 In these embodiments, the consumer view layer can extract multimedia material track segment data and editing effect attributes from the target data layer, and apply the editing effect attributes to the multimedia segment data to obtain multimedia material track segment view data.

[0066] For example, you can modify the display time of the multimedia content track clip data on the timeline to match the display time of the editing effect indicator. You can also modify the display level of the multimedia content track clip data to match the display level of the editing effect indicator. Finally, you can modify the playback speed of the multimedia content track clip data to match the playback speed of the editing effect indicator. This will give you the multimedia content track clip view data.

[0067] In some embodiments, step S503 includes:

[0068] Multimedia clip view data is constructed according to the order in which each multimedia material track clip appears on the timeline in the target data layer.

[0069] The timeline mentioned above can be found in the audio / video editor interface.

[0070] Taking multimedia material track segment data as an example, for multiple video material track segment data, the data is sorted according to the order in which the first frame of each video material track segment appears on the timeline, resulting in video segment view data. These multiple video material track segment data can be distributed across different tracks.

[0071] Please refer to Figure 6 , Figure 6 This is a diagram illustrating an application scenario. For example... Figure 6 As shown, track 0 includes video clips A and B, and track 1 includes video clips C and D. Each video clip can correspond to a view clip. Video clips A and B appear simultaneously at time 0 on timeline 601. D appears at time t1, and B appears at time t2. t1 is earlier than t2. Therefore, the video clip view data can include video clips arranged in chronological order (video clips containing only images), for example, the video clip view data could be Map1{0: {A, C}; t1: {D}; t2: {B}}.

[0072] Similarly, audio clip view data can be obtained.

[0073] S504: Generate multimedia clip content corresponding to the editing operation based on the multimedia material track clip view data.

[0074] Based on the multimedia material track segment view data generated in step S504, multimedia segment content corresponding to the editing operation can be generated. For example, the multimedia material track segment view data can be merged to obtain multimedia segment content.

[0075] In some embodiments, step S504 includes: determining the target time period corresponding to the editing operation, and updating the target slice view data corresponding to the target time period. Step S505 includes: generating multimedia segment content corresponding to the editing operation based on historical slice view data from other time periods and the target slice view data; wherein the historical slice view data is determined based on the following steps:

[0076] Taking video clip track segments as an example, the timeline can be divided according to preset rules to obtain multiple slices. Each slice can correspond to at least one video clip track segment and at least one audio clip track segment. For each slice, the view data of the slice is constructed based on the already constructed video clip track segment view data.

[0077] For example, by using the first frame appearance time and the last frame end time of each video clip track segment as a timeline segmentation point, multiple timeline segmentation points can be obtained.

[0078] The timeline is divided according to the above-mentioned dividing points to obtain multiple slices. Each slice can include at least one video clip track segment and at least one audio clip track segment.

[0079] Secondly, for each slice, the view data for that slice is constructed based on the already constructed video material track segment data.

[0080] For each slice, the view data corresponding to that slice includes the slice's identifier and the identifier of the video footage track segment corresponding to that slice.

[0081] The identifier for the aforementioned slice can include the start and end times of the slice (e.g., as a key), while the identifier for the aforementioned video footage track segment can be used as a value. For each slice, a key:value pair can be generated for that slice.

[0082] Please refer to Figure 7 , Figure 7 This is a schematic diagram of an application scenario. Figure 7 The timeline 701 is divided into segments T0, T1, T2, T3, and T4. The timeline is then divided into slices S1, S2, S3, and S4. The view data for each slice can be recorded as follows: S1: Key(T0, T1), Value(A, C); S2: Key(T1, T2), Value(A); Key(T2, T3), Value(A, D); Key(T3, T4), Value(D).

[0083] In these embodiments, the time period corresponding to the editing operation can be determined, and the editing effect can be applied only to the view data within the target time period to obtain the multimedia segment view data corresponding to the editing operation. This can reduce the amount of data generated and improve the efficiency of multimedia editing.

[0084] Corresponding to the multimedia editing method in the above embodiments, Figure 8 This is a structural block diagram of a multimedia editing apparatus provided in an embodiment of the present disclosure. For ease of explanation, only the parts relevant to the embodiments of the present disclosure are shown. (Refer to...) Figure 8 The device 80 includes: a first generation unit 801, a determination unit 802, and a second generation unit 803. Wherein:

[0085] The first generation unit 801 is used to generate initial data layer data based on the editing operation of multimedia materials. The initial data layer data includes multimedia material track segment data obtained from the editing operation.

[0086] The determining unit 802 is used to determine the editing effect attribute of the editing operation instruction, associate the editing effect attribute with the initial data layer data, and use the initial data layer data associated with the editing effect attribute as the target data layer data;

[0087] The second generation unit 803 is used to obtain multimedia material track segment data and editing effect attributes from the target data layer using the consumer view layer according to the received display instructions, and generate multimedia segment content after editing operation based on the multimedia material track segment data and editing effect attributes.

[0088] In some embodiments of this disclosure, the second generation unit 803 is further configured to:

[0089] In the consumer view layer, editing effect attributes are applied to the multimedia material track segment data to obtain the multimedia material track segment view data;

[0090] Generate multimedia clip content corresponding to the editing operation based on the multimedia material track clip view data.

[0091] In some embodiments of this disclosure, multimedia material track segment data includes audio material track segment data and / or video material track segment data.

[0092] In some embodiments of this disclosure, the second generation unit 803 is further configured to:

[0093] The consumer view layer is used to obtain the first video clip track data containing audio information from the target data layer.

[0094] Extract the second video clip data (with audio information removed) from the first video clip track segment data;

[0095] Use the track segment data of the first video material as the corresponding audio track segment data.

[0096] In some embodiments of this disclosure, the editing effect attributes of the editing operation indication include one or more of the following:

[0097] Playback time attribute, playback speed attribute, and layer attribute.

[0098] In some embodiments of this disclosure, the second generation unit 803 is further configured to:

[0099] Based on the order in which each multimedia material track segment appears on the timeline in the target data layer, construct the multimedia material track segment view data.

[0100] In some embodiments of this disclosure, the second generation unit 803 is further configured to:

[0101] Determine the target time period corresponding to the editing operation, and update the target slice view data corresponding to the target time period;

[0102] Based on historical slice view data from other time periods and target slice view data, generate multimedia clip content corresponding to the editing operation; wherein, the historical slice view data is determined based on the following steps:

[0103] The timeline is divided according to preset rules to obtain multiple slices. Each slice can correspond to at least one video material track segment and at least one audio material track segment.

[0104] For each slice, construct the view data for that slice based on the constructed video segment view data.

[0105] To implement the above embodiments, this disclosure also provides an electronic device.

[0106] refer to Figure 9 The diagram illustrates a structural schematic of an electronic device 900 suitable for implementing embodiments of the present disclosure. The electronic device 900 can be various terminal devices or servers. The terminal device may include an extended reality device. Extended reality devices may include, for example, various wearable devices, such as head-mounted extended reality devices. Figure 9 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0107] like Figure 9As shown, the electronic device 900 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage device 908 into a random access memory (RAM) 903. The RAM 903 also stores various programs and data required for the operation of the electronic device 900. The processing unit 901, ROM 902, and RAM 903 are interconnected via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0108] Typically, the following devices can be connected to I / O interface 905: input devices 906 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 907 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 908 including, for example, magnetic tapes, hard disks, etc.; and communication devices 909. Communication device 909 allows electronic device 900 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 9 An electronic device 900 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0109] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 909, or installed from a storage device 908, or installed from a ROM 902. When the computer program is executed by a processing device 901, it performs the functions defined in the methods of embodiments of this disclosure.

[0110] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0111] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0112] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods shown in the above embodiments.

[0113] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0114] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0115] The units described in the embodiments of this disclosure can be implemented in software or in hardware. The names of the units are not necessarily limiting in certain circumstances; for example, the first generation unit can also be described as "a unit that generates initial data layer data based on editing operations on multimedia materials".

[0116] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0117] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0118] In a first aspect, according to one or more embodiments of this disclosure, a multimedia editing method is provided, comprising:

[0119] Initial data layer data is generated based on the editing operations on multimedia materials. The initial data layer data includes the multimedia material track segment data obtained from the editing operations.

[0120] Determine the editing effect attributes of the editing operation instructions, associate the editing effect attributes with the initial data layer data, and use the initial data layer data associated with the editing effect attributes as the target data layer data;

[0121] Based on the received display instructions, the consumer view layer obtains multimedia material track segment data and editing effect attributes from the target data layer, and generates the multimedia segment content after editing based on the multimedia material track segment data and editing effect attributes.

[0122] According to one or more embodiments of this disclosure, multimedia material track segment data and editing effect attributes are obtained from the target data layer using the consumer view layer, and multimedia segment content after editing operations is generated based on the multimedia material track segment data and editing effect attributes, including:

[0123] In the consumer view layer, editing effect attributes are applied to the multimedia material track segment data to obtain the multimedia material track segment view data;

[0124] Generate multimedia clip content corresponding to the editing operation based on the multimedia material track clip view data.

[0125] According to one or more embodiments of this disclosure, multimedia material track segment data includes audio material track segment data and / or video material track segment data.

[0126] According to one or more embodiments of this disclosure, based on a received display instruction, multimedia material track segment data is obtained from the target data layer using the consumer view layer, including:

[0127] The consumer view layer is used to obtain the first video clip track data containing audio information from the target data layer.

[0128] Extract the second video clip data (with audio information removed) from the first video clip track segment data;

[0129] Use the track segment data of the first video material as the corresponding audio track segment data.

[0130] According to one or more embodiments of this disclosure, the editing effect attributes of the editing operation indication include one or more of the following:

[0131] Playback time attribute, playback speed attribute, and layer attribute.

[0132] According to one or more embodiments of this disclosure, in the consumer view layer, editing effect attributes are applied to multimedia material track segment data to obtain multimedia material track segment view data, including:

[0133] Based on the order in which each multimedia material track segment appears on the timeline in the target data layer, construct the multimedia material track segment view data.

[0134] According to one or more embodiments of this disclosure, in the consumer view layer, applying editing effect attributes to multimedia material track segment data to obtain multimedia material track segment view data further includes:

[0135] Determine the target time period corresponding to the editing operation, update the target slice view data corresponding to the target time period; and generate the multimedia clip content corresponding to the editing operation based on the multimedia material track clip view data, including:

[0136] Based on historical slice view data from other time periods and target slice view data, generate multimedia clip content corresponding to the editing operation; wherein, the historical slice view data is determined based on the following steps:

[0137] The timeline is divided according to preset rules to obtain multiple slices. Each slice can correspond to at least one video material track segment and at least one audio material track segment.

[0138] For each slice, construct the view data for that slice based on the constructed video segment view data.

[0139] Secondly, according to one or more embodiments of this disclosure, a multimedia editing apparatus is provided, comprising: a first generating unit, a determining unit, and a second generating unit. Wherein:

[0140] The first generation unit is used to generate initial data layer data based on the editing operation of multimedia materials. The initial data layer data includes multimedia material track segment data obtained from the editing operation.

[0141] The determining unit is used to determine the editing effect attributes of the editing operation instruction, associate the editing effect attributes with the initial data layer data, and use the initial data layer data associated with the editing effect attributes as the target data layer data.

[0142] The second generation unit is used to obtain multimedia material track segment data and editing effect attributes from the target data layer using the consumer view layer according to the received display instructions, and generate the multimedia segment content after the editing operation based on the multimedia material track segment data and editing effect attributes.

[0143] According to one or more embodiments of this disclosure, the second generating unit is further configured to:

[0144] In the consumer view layer, editing effect attributes are applied to the multimedia material track segment data to obtain the multimedia material track segment view data;

[0145] Generate multimedia clip content corresponding to the editing operation based on the multimedia material track clip view data.

[0146] According to one or more embodiments of this disclosure, multimedia material track segment data includes audio material track segment data and / or video material track segment data.

[0147] In some embodiments of this disclosure, the second generation unit is further configured to:

[0148] The consumer view layer is used to obtain the first video clip track data containing audio information from the target data layer.

[0149] Extract the second video clip data (with audio information removed) from the first video clip track segment data;

[0150] Use the track segment data of the first video material as the corresponding audio track segment data.

[0151] According to one or more embodiments of this disclosure, the editing effect attributes of the editing operation indication include one or more of the following:

[0152] Playback time attribute, playback speed attribute, and layer attribute.

[0153] According to one or more embodiments of this disclosure, the second generating unit is further configured to:

[0154] Based on the order in which each multimedia material track segment appears on the timeline in the target data layer, construct the multimedia material track segment view data.

[0155] According to one or more embodiments of this disclosure, the second generating unit is further configured to:

[0156] Determine the target time period corresponding to the editing operation, and update the target slice view data corresponding to the target time period;

[0157] Based on historical slice view data from other time periods and target slice view data, generate multimedia clip content corresponding to the editing operation; wherein, the historical slice view data is determined based on the following steps:

[0158] The timeline is divided according to preset rules to obtain multiple slices. Each slice can correspond to at least one video material track segment and at least one audio material track segment.

[0159] For each slice, construct the view data for that slice based on the constructed video segment view data.

[0160] Thirdly, according to one or more embodiments of the present disclosure, an electronic device is provided, comprising: at least one processor and a memory;

[0161] The memory stores the instructions that the computer executes;

[0162] At least one processor executes computer execution instructions stored in memory, causing at least one processor to perform the first aspect above and various multimedia editing methods that may be involved in the first aspect.

[0163] Fourthly, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, which stores computer-executable instructions that, when executed by a processor, implement the first aspect above and various multimedia editing methods that may be involved in the first aspect.

[0164] Fifthly, according to one or more embodiments of the present disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the first aspect as described above and various multimedia editing methods that may be involved in the first aspect.

[0165] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0166] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0167] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A multimedia editing method, comprising: Initial data layer data is generated based on the editing operations on multimedia materials, and the initial data layer data includes the multimedia material track segment data obtained from the editing operations; Determine the editing effect attribute of the editing operation instruction, associate the editing effect attribute with the initial data layer data, and use the initial data layer data associated with the editing effect attribute as the target data layer data; Based on the received display instructions, the consumer view layer obtains the first video material track segment data containing audio information from the target data layer; Extract the second video clip data (with audio information removed) from the first video clip data; Use the first video material track segment data as the corresponding audio track segment data to obtain multimedia material track segment data; Based on the received display instructions, the consumer view layer obtains the editing effect attributes from the target data layer, and generates the multimedia segment content after the editing operation based on the multimedia material track segment data and the editing effect attributes.

2. The method according to claim 1, characterized in that, The process of generating the edited multimedia segment content based on the multimedia material track segment data and the editing effect attributes includes: In the consumer view layer, the editing effect attributes are applied to the multimedia material track segment data to obtain multimedia material track segment view data; The multimedia clip content corresponding to the editing operation is generated based on the multimedia material track clip view data.

3. The method according to claim 1, characterized in that, The multimedia material track segment data includes audio material track segment data and / or video material track segment data.

4. The method according to claim 1, characterized in that, The editing effect attributes indicated by the editing operation include one or more of the following: Playback time attribute, playback speed attribute, and layer attribute.

5. The method according to claim 2, characterized in that, In the consumer view layer, applying the editing effect attributes to the multimedia material track segment data to obtain multimedia material track segment view data includes: Based on the order in which each multimedia material track segment appears on the timeline in the target data layer, construct the multimedia material track segment view data.

6. The method according to claim 5, characterized in that, The step of applying the editing effect attributes to the multimedia material track segment data in the consumer view layer to obtain multimedia material track segment view data further includes: Determine the target time period corresponding to the editing operation, update the target slice view data corresponding to the target time period; and generate the multimedia segment content corresponding to the editing operation based on the multimedia material track segment view data, including: Based on historical slice view data from other time periods and the target slice view data, the multimedia segment content corresponding to the editing operation is generated; wherein, the historical slice view data is determined based on the following steps: The timeline is divided according to preset rules to obtain multiple slices. Each slice can correspond to at least one video material track segment and at least one audio material track segment. For each slice, construct the view data for that slice based on the constructed video segment view data.

7. A multimedia editing device, comprising: The first generation unit is used to generate initial data layer data based on the editing operation of multimedia materials, wherein the initial data layer data includes multimedia material track segment data obtained by the editing operation; The determining unit is used to determine the editing effect attribute of the editing operation instruction, associate the editing effect attribute with the initial data layer data, and use the initial data layer data associated with the editing effect attribute as the target data layer data. The second generation unit is used to obtain multimedia material track fragment data and the editing effect attributes from the target data layer using the consumer view layer according to the received display instructions, and generate the multimedia fragment content after the editing operation based on the multimedia material track fragment data and the editing effect attributes; The second generation unit is further configured to: The consumer view layer is used to obtain first video material track segment data containing audio information from the target data layer; Extract the second video clip data (with audio information removed) from the first video clip data; Use the first video clip track data as the corresponding audio clip track data.

8. An electronic device, comprising: Processor and memory; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 6.

Images