Multimedia editing method and device and electronic equipment

Abstract

The embodiment of the invention provides a multimedia editing method and device and electronic equipment, and the method comprises the steps: generating initial data layer data according to the editing operation of a multimedia material, and the initial data layer data comprises multimedia material track fragment data obtained through the editing operation; determining an editing effect attribute indicated by the editing operation, associating the editing effect attribute with the initial data layer data, and taking the initial data layer data associated with the editing effect attribute as target data layer data; and according to a received display instruction, acquiring multimedia material track fragment data and the editing effect attribute from the target data layer data by utilizing a consumption view layer, and generating multimedia fragment content after editing operation based on the multimedia material track fragment data and the editing effect attribute. And data redundancy occurring in the multimedia editing process is reduced.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of computer technology, and more particularly to a multimedia editing method, device, and electronic device. Background Art

[0002] Multimedia data may include audio data and / or video data, and users may perform various edits on the multimedia data, such as merging different audio and video clips, adding background music to video clips, adding special effects, etc. By editing the audio and video as described above, a work that meets the user's intentions may be obtained. Summary of the invention

[0003] Embodiments of the present disclosure provide a multimedia editing method, device, and electronic device.

[0004] In a first aspect, an embodiment of the present disclosure provides a multimedia editing method, the method comprising: generating initial data layer data according to an editing operation on a multimedia material, the initial data layer data comprising multimedia material track fragment data obtained by the editing operation; determining editing effect attributes indicated by the editing operation, associating the editing effect attributes with the initial data layer data, and using the initial data layer data associated with the editing effect attributes as target data layer data; according to a received display instruction, obtaining the multimedia material track fragment data and the editing effect attributes from the target data layer data using a consumption view layer, and generating the multimedia fragment content after the editing operation based on the multimedia material track fragment data and the editing effect attributes.

[0005] In a second aspect, an embodiment of the present disclosure provides a multimedia editing device, comprising: a first generating unit, configured to generate initial data layer data according to an editing operation on a multimedia material, wherein the initial data layer data includes multimedia material track fragment data obtained by the editing operation; a determining unit, configured to determine the editing effect attributes indicated by the editing operation, associate the editing effect attributes with the initial data layer data, and use the initial data layer data associated with the editing effect attributes as target data layer data; and a second generating unit, configured to obtain the multimedia material track fragment data and the editing effect attributes from the target data layer data using a consumption view layer according to a received display instruction, and generate the multimedia fragment content after the editing operation based on the multimedia material track fragment data and the editing effect attributes.

[0006] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising: a processor and a memory; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory, so that the at least one processor executes the first aspect above and various multimedia editing methods that may be involved in the first aspect.

[0007] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, in which computer execution instructions are stored. When a processor executes the computer execution instructions, the multimedia editing method as described in the first aspect and various possible aspects of the first aspect are implemented.

[0008] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the above first aspect and various multimedia editing methods that may be involved in the first aspect.

[0009] The multimedia editing method, device and electronic device provided in this embodiment generate initial data layer data according to the editing operation of multimedia materials, wherein the initial data layer data includes multimedia material track segment data obtained by the editing operation; determine the editing effect attribute indicated by the editing operation, associate the editing effect attribute with the initial data layer data, and use the initial data layer data associated with the editing effect attribute as the target data layer data; according to the received display instruction, use the consumption view layer to obtain the multimedia material track segment data and the editing effect attribute from the target data layer data, and generate the multimedia segment content after the editing operation based on the multimedia material track segment data and the editing effect attribute, thereby reducing data redundancy in the multimedia editing process. In addition, the problem of inconsistency between effect display and editing operation caused by inconsistency of multiple layers of data can also be avoided. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0011] Figure 1 A schematic diagram of an application scenario of multimedia editing provided by an embodiment of the present disclosure;

[0012] Figure 2 A schematic diagram of a multimedia editing process provided by an embodiment of the present disclosure;

[0013] Figure 3 A flowchart of a multimedia editing method provided by an embodiment of the present disclosure Figure 1 ;

[0014] Figure 4 A schematic diagram of a multimedia editing process provided by an embodiment of the present disclosure;

[0015] Figure 5A schematic diagram of the multimedia editing method provided by the present invention Figure 2 ;

[0016] Figure 6 This is a schematic diagram of an application scenario;

[0017] Figure 7 This is a schematic diagram of an application scenario;

[0018] Figure 8 A structural block diagram of a multimedia editing device provided by an embodiment of the present disclosure;

[0019] Fig. 9 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0021] Please refer to Figure 1 , which shows a schematic diagram of an application scenario of multimedia editing of the present disclosure. The multimedia editing interface includes a timeline. In the editing operation, partial clips of video material 1 and video material 2 are imported respectively, forming two video material track clips 11 and 12. The user can add background music, special effects, etc. to the video material track clips in subsequent editing operations.

[0022] In an example of a multimedia editing method, the physical data structure presented by the business or user interface can be abstracted into a data layer by the data model layer, and then the video track data and audio track data in the data layer are processed by multiple layers of preset model layers to finally obtain the consumer view layer. Please refer to Figure 2 , Figure 2 The above-mentioned multiple preset model layers include a conversion layer 202 and an engine layer 203. The data in the above-mentioned data layer 201 may be video material track fragment data and audio material track fragment data obtained by the user editing the audio material and / or video material in the multimedia editing interface. The video material track fragment data includes data of video material track fragments, picture material track fragments, sticker track material fragments, etc. Among them, there may also be a transition effect between the video material track fragment and its continuous picture material track fragment. In the conversion layer, multiple video conversion track objects of the conversion layer are generated for the video material track fragment data; and at least one audio conversion track object is generated for the audio track data. As Figure 2As shown, in the conversion layer 202, two video conversion objects 2021 and 2022 are generated for the video track data in the data layer 201. One of the video conversion track objects may include the video, picture material track fragment, sticker material track fragment, and the transition between the video and the picture extracted from the audio and video material track fragment; the other video conversion track object includes the audio extracted from the audio and video material track fragment. The audio conversion track object generated for the audio track data in the data layer 201 may include the audio material track fragment data, etc.

[0023] In the conversion layer, the audio and video material track segments are separated into audio and video, and the original video material track segment data is split into two video material conversion track data.

[0024] After obtaining the data of the conversion layer, the engine layer data can be obtained through corresponding data processing. Each video conversion track object in the conversion layer can be further processed into multiple engine video track data, and a data object is generated for each engine video track data. For example, the video conversion track 2021 in the conversion layer is processed into two engine video tracks 2031 and 2032 in the engine layer. The engine video track 2031 can include video data, transition data, and data corresponding to the sticker material track fragment. The engine video track 2032 also includes transition data, picture material data, and sticker data.

[0025] The audio data in the video conversion track 2022 is also split into multiple engine audio tracks 2033 and 2034 in the engine layer. The audio conversion track 2023 is also split into multiple audio tracks in the engine layer, such as Figure 2 The audio track data 2035 and 2036 in.

[0026] Finally, the video segment view and the video slice view may be generated according to the engine video track data 2031, 2032 in the engine layer 203. The audio segment view and the audio slice view may be generated according to the engine audio track data 2033, 2034 and the audio track data 2035, 2036 in the engine layer.

[0027] However, as the business complexity increases, there may be more intermediate data layers, such as the editing model layer and the rendering model layer. When there are more intermediate data layers, most of the data is the same in the process of passing through the above multiple intermediate data layers from the data layer to the engine layer: such as file path, original video information, video speed, etc. Therefore, there is a large amount of redundant data. In addition, when there are more layers, the mapping, updating and consistency of the data between the data layers require a large cost to maintain.

[0028] In order to improve the above problems, the solution provided by the present disclosure generates initial data layer data according to the editing operation of multimedia materials, and the initial data layer data includes multimedia material track fragment data obtained by the editing operation; determines the editing effect attribute indicated by the editing operation, associates the editing effect attribute with the initial data layer data, and uses the initial data layer data associated with the editing effect attribute as the target data layer data; according to the received display instruction, uses the consumption view layer to obtain the multimedia material track fragment data and the editing effect attribute from the target data layer data, and generates the multimedia fragment content after the editing operation based on the multimedia material track fragment data and the editing effect attribute. Since the multi-layer preset model is omitted, it will not cause additional object construction and duplicate data, reducing data redundancy. In addition, the problem of inconsistency between the effect display and the editing operation caused by the inconsistency of multi-layer data is solved.

[0029] Please refer to Figure 3 , Figure 3 A flowchart of a multimedia editing method provided by an embodiment of the present disclosure Figure 1 .like Figure 3 As shown, the method includes:

[0030] S301: Generate initial data layer data according to an editing operation on multimedia materials, where the initial data layer data includes multimedia material track segment data obtained by the editing operation.

[0031] The execution subject of the multimedia editing method in the present disclosure may be a terminal device, specifically a multimedia editing client running in the terminal device for editing multimedia materials. In addition, the execution subject of the multimedia editing processing method may also be a server, for example, a server providing services for the multimedia editing client running on the terminal device.

[0032] The following description will be made by taking the multimedia editing client as an example.

[0033] The user can perform editing operations on multimedia materials in the interface of the multimedia editing client. The above editing operations include adding multimedia material track segments, cutting multimedia material track segments, moving multimedia material track segments in the track, adjusting the playback speed of multimedia material track segments, setting the level of multimedia material track segment data, adding special effects, etc.

[0034] For example, you can split a multimedia material on a track and put the resulting fragments into different tracks or into the same track. Each of the above fragments is a multimedia material track fragment. You can add transition effects, etc. between two consecutive multimedia material track fragments on the same track.

[0035] The multimedia material track fragments include audio material track fragments and / or video material track fragments.

[0036] For video material track clips, it can also include each sticker, special effect, text, image clip, etc. Figure 1 The video material track fragment data in may include data of tracks 11 and 12.

[0037] The audio material track segment here can be a segment extracted from the audio material. The above audio material can be unrelated to the audio and video material corresponding to the above video material track segment, or it can be the audio material obtained after the audio is separated from the above audio and video material. The editing operation of the audio material track segment can include fade-in, fade-out, etc.

[0038] The initial data layer data of the multimedia material track fragment includes data storage path, original multimedia material information, etc.

[0039] Please refer to Figure 4 , Figure 4 A schematic diagram of the multimedia editing process provided by an embodiment of the present disclosure. Figure 4 The data layer 401 in the target data layer is the target data layer. The video material track segments such as audio and video, transitions, pictures, stickers, etc. in the video track in the target data layer can be the initial video material track segment data in the initial data layer. The above-mentioned editing effect attributes can be obtained by calculating the initial video material track segment data corresponding to the editing operation.

[0040] Schematically, the layer information and set speed corresponding to the picture A (corresponding to a track in the data layer) in the video material track segment are used as the editing effect attributes of the picture A material track segment.

[0041] Illustratively, the layer information and set speed corresponding to the video segment B (corresponding to a track in the data layer) in the video material track segment are used as the editing effect attributes of the video segment B.

[0042] S302: Determine the editing effect attribute of the editing operation instruction, associate the editing effect attribute with the initial data layer data, and use the initial data layer data associated with the editing effect attribute as the target data layer data.

[0043] The editing effect attribute of the editing operation indicates the editing effect of the multimedia material track segment corresponding to the editing operation. The editing effect attribute includes but is not limited to one or more of the following: play time attribute, play speed attribute, and level attribute.

[0044] The above-mentioned editing effect attributes may be associated with the initial data layer data. Specifically, the editing effect attributes indicated by the editing operation may be associated with the initial data layer data of the multimedia material track segment operated by the editing operation. Schematically, if the editing operation sets a play speed X1 for the multimedia material track segment A, the above-mentioned play speed X1 is associated with the initial data layer data of the multimedia material track segment A.

[0045] If the editing operation modifies the display time T of the multimedia material track segment B on the timeline, the display time T is associated with the initial data layer data of the multimedia material track segment B.

[0046] After the editing effect attributes corresponding to each editing operation are associated with the initial data layer data of the multimedia material track segment operated by each editing operation, the initial data layer data associated with the editing effect attributes is used as the target data layer data.

[0047] S303: According to the received display instruction, the consumption view layer is used to obtain multimedia material track segment data and editing effect attributes from the target data layer data, and the multimedia segment content after the editing operation is generated based on the multimedia material track segment data and the editing effect attributes.

[0048] After the user performs the editing operation, a display instruction may be sent to the execution subject, and the display instruction is used to display the display effect corresponding to the editing operation. After receiving the display instruction, the execution subject may send the display instruction to the consumption view layer, and the consumption view layer obtains the multimedia material track segment data and the editing effect data from the target data layer. The multimedia segment content after the editing operation may be generated according to the multimedia material track segment data and the editing effect attributes.

[0049] The multimedia material track segment content may be content obtained by applying editing effects to the multimedia material track segment data and fusing different multimedia material track segment data.

[0050] exist Figure 2In the multimedia editing method shown, in the multi-layer preset model, for each layer of the model, objects are constructed for the multimedia material track segments on all tracks and object data is generated. In the conversion layer, the multimedia material track segments including audio and video data are split into pure video track segments and audio and video track segments. Each video conversion track object in the conversion layer can be further processed into multiple engine video track data, and a data object is generated for each engine video track data. The objects on the above layers can be obtained from the initial multi-data in the data layer. For example, when modifying the speed playback of a multimedia material track segment, the playback speed corresponding to the modification operation can be obtained by multiplying the playback speed of the original data of the multimedia material track segment by the multiple indicated by the above speed. When modifying the display time of a multimedia material track segment, the target display time corresponding to the modification operation can be obtained by moving it according to the display time of the original multimedia material track segment on the timeline.

[0051] Based on this, the editing effect attributes corresponding to the editing operation can be determined, and the above editing effect attributes can be bound to the initial data of the multimedia material track segment in the initial data layer of the multimedia material track segment to obtain the target data layer data, and then the consumption view layer directly extracts the initial data and editing effect attributes of the multimedia material track segment from the target data layer for consumption. The consumption effect can be achieved from Figure 2 The effect of consuming data extracted by the engine layer in the multi-layer preset model layer.

[0052] The solution provided by this embodiment generates initial data layer data according to the editing operation of multimedia materials, and the initial data layer data includes multimedia material track segment data obtained by the editing operation; determines the editing effect attribute indicated by the editing operation, associates the editing effect attribute with the initial data layer data, and uses the initial data layer data associated with the editing effect attribute as the target data layer data; according to the received display instruction, uses the consumption view layer to obtain the multimedia material track segment data and the editing effect attribute from the target data layer data, and generates the multimedia segment content after the editing operation based on the multimedia material track segment data and the editing effect attribute. Since the multi-layer preset model is omitted, it will not cause additional object construction and duplicate data, reducing data redundancy. In addition, the problem of inconsistency between the effect display and the editing operation caused by the inconsistency of multi-layer data is solved.

[0053] In some embodiments, the above step S303 includes the following steps:

[0054] First, the first video material track segment data containing audio information is obtained from the target data layer using the consumption view layer;

[0055] Secondly, extracting the second video material track segment data without audio information from the first video material track segment data;

[0056] Finally, the first video material track segment data is used as the corresponding audio track segment data.

[0057] For the first video material track fragment data of the audio information, image sequences are extracted from the video material track fragment, and these image sequences may be the second video material track fragment data. These image sequences are used to construct the video fragment view.

[0058] The first video material track segment data is directly used as the corresponding audio track segment data for constructing the audio segment view.

[0059] In these embodiments, for video material track segment data including audio, image data in the video material track segment is used to construct a video segment view, and the video material track segment is used to construct an audio segment view, thereby avoiding data redundancy caused by splitting the video material track segment data for the audio and saving them separately in order to construct an audio segment view and a video segment view.

[0060] Please refer to Figure 5 , which shows a schematic flow chart of the multimedia editing method provided by the present disclosure Figure 2 ,like Figure 5 As shown, the method comprises the following steps:

[0061] S501: Generate initial data layer data according to an editing operation on multimedia materials, where the initial data layer data includes multimedia material track segment data obtained by the editing operation.

[0062] S502: Determine the editing effect attribute of the editing operation instruction, associate the editing effect attribute with the initial data layer data, and use the initial data layer data associated with the editing effect attribute as the target data layer data.

[0063] The specific implementation of steps S501 to S502 can refer to Figure 3 The relevant parts of the illustrated embodiment are not described in detail here.

[0064] S503: In the consumption view layer, applying the editing effect attribute to the multimedia material track segment data to obtain the multimedia material track segment view data.

[0065] Please continue to refer to Figure 4 In these embodiments, the consumption view layer may extract the multimedia material track segment data and the editing effect attributes from the target data layer, and apply the editing effect attributes to the multimedia segment data to obtain the multimedia material track segment view data.

[0066] For example, the display time of the multimedia material track segment data on the time axis is modified to the display time indicated by the editing effect. The display level of the multimedia material track segment data is modified to the display level indicated by the editing effect. The playback speed of the multimedia material track segment data is modified to the playback speed indicated by the editing effect. In this way, the multimedia material track segment view data can be obtained.

[0067] In some embodiments, the above step S503 includes:

[0068] The multimedia segment view data is constructed according to the order in which each multimedia material track segment data in the target data layer data appears on the time axis.

[0069] The above timeline can be the timeline in the audio and video editor interface.

[0070] Taking the multimedia material track segment data as video material track segment data as an example, for the multiple video material track segment data, the video material track segment data are sorted according to the order in which the first frame image of each video material track segment data appears on the time axis to obtain the video segment view data. The multiple video material track segment data can be distributed in different tracks.

[0071] Please refer to Figure 6 , Figure 6 is a schematic diagram of an application scenario. Figure 6 As shown, track 0 includes video material track segments A and B, and track 1 includes video material track segments C and D. Each video material track segment may correspond to a view segment. Video material track segments A and B appear at the same time at time 0 in timeline 601. D appears at time t1, and B appears at time t2. t1 is earlier than t2. Therefore, the above-mentioned video material track segment view data may include video material track segments (including only video segments of images) arranged in chronological order. For example, the above-mentioned video material track segment view data may be Map1{0: {A, C}; t1: {D}; t2: {B}}.

[0072] Likewise, audio clip view data may be obtained.

[0073] S504: Generate multimedia segment content corresponding to the editing operation according to the multimedia material track segment view data.

[0074] The multimedia segment content corresponding to the editing operation may be generated based on the multimedia material track segment view data generated in step S504, for example, by merging the multimedia material track segment view data to obtain the multimedia segment content.

[0075] In some embodiments, the above step S504 includes: determining a target time period corresponding to the editing operation, and updating target slice view data corresponding to the target time period. The above step S505 includes: generating multimedia segment content corresponding to the editing operation according to historical slice view data and target slice view data of other time periods; wherein the historical slice view data is determined based on the following steps:

[0076] Taking the video material track fragment as an example, the timeline can be divided according to preset rules to obtain multiple slices, each slice can correspond to at least one video material track fragment and at least one audio material track fragment; for each slice, the view data of the slice is constructed according to the constructed video material track fragment view data.

[0077] For example, a time point at which the first frame appears and a time point at which the last frame ends of each video material track segment are used as a time axis segmentation point to obtain multiple time axis segmentation points.

[0078] The time axis is divided according to the above division points to obtain multiple slices, and each slice may include at least one video material track segment and at least one audio material track segment.

[0079] Secondly, for each slice, the view data of the slice is constructed according to the constructed video material track fragment data.

[0080] For each slice, the view data corresponding to the slice includes an identifier of the slice and an identifier of a video material track fragment corresponding to the slice.

[0081] The identification of the slice may include the start time and the end time of the slice (for example, as a Key), and the identification of the video material track fragment may be used as a Value. For each slice, a Key:Value pair of the slice may be generated.

[0082] Please refer to Figure 7 , Figure 7 This is a schematic diagram of an application scenario. Figure 7 The segmentation points of the time axis 701 include T0, T1, T2, T3 and T4. The slices obtained by segmenting the time axis with the above segmentation points include S1, S2, S3 and S4. The view data of each slice can be recorded as follows: S1: Key (T0, T1), Value (A, C); S2: Key (T1, T2), Value (A); Key (T2, T3), Value (A, D); Key (T3, T4), Value (D).

[0083] In these embodiments, the time period corresponding to the editing operation can be determined, and the editing effect can be applied only to the view data within the above target time period, thereby obtaining the multimedia segment view data corresponding to the editing operation. On the one hand, the amount of generated data can be reduced, and on the other hand, the efficiency of multimedia editing can be improved.

[0084] Corresponding to the multimedia editing method of the above embodiment, Figure 8 This is a structural block diagram of a multimedia editing device provided by an embodiment of the present disclosure. For ease of explanation, only the parts related to the embodiment of the present disclosure are shown. Figure 8 , the device 80 includes: a first generating unit 801, a determining unit 802, and a second generating unit 803. Wherein:

[0085] The first generating unit 801 is used to generate initial data layer data according to the editing operation on the multimedia material, and the initial data layer data includes multimedia material track segment data obtained by the editing operation;

[0086] The determining unit 802 is used to determine the editing effect attribute of the editing operation instruction, associate the editing effect attribute with the initial data layer data, and use the initial data layer data associated with the editing effect attribute as the target data layer data;

[0087] The second generating unit 803 is used to obtain multimedia material track segment data and editing effect attributes from the target data layer data by using the consumption view layer according to the received display instruction, and generate multimedia segment content after the editing operation based on the multimedia material track segment data and the editing effect attributes.

[0088] In some embodiments of the present disclosure, the second generating unit 803 is further configured to:

[0089] In the consumption view layer, applying the editing effect attribute to the multimedia material track segment data to obtain the multimedia material track segment view data;

[0090] Generate multimedia segment content corresponding to the editing operation according to the multimedia material track segment view data.

[0091] In some embodiments of the present disclosure, the multimedia material track fragment data includes audio material track fragment data and / or video material track fragment data.

[0092] In some embodiments of the present disclosure, the second generating unit 803 is further configured to:

[0093] Obtaining first video material track segment data including audio information from the target data layer using the consumer view layer;

[0094] Extracting second video material track segment data without audio information from the first video material track segment data;

[0095] The first video material track segment data is used as the corresponding audio track segment data.

[0096] In some embodiments of the present disclosure, the editing effect attributes of the editing operation indication include one or more of the following:

[0097] Playback time attributes, playback speed attributes, and level attributes.

[0098] In some embodiments of the present disclosure, the second generating unit 803 is further configured to:

[0099] The multimedia material track segment view data is constructed according to the order in which each multimedia material track segment data in the target data layer data appears on the time axis.

[0100] In some embodiments of the present disclosure, the second generating unit 803 is further configured to:

[0101] Determine a target time period corresponding to the editing operation, and update target slice view data corresponding to the target time period;

[0102] Generate multimedia segment content corresponding to the editing operation according to the historical slice view data of other time periods and the target slice view data; wherein the historical slice view data is determined based on the following steps:

[0103] The timeline is divided according to a preset rule to obtain a plurality of slices, each of which may correspond to at least one video material track segment and at least one audio material track segment;

[0104] For each slice, the view data of the slice is constructed according to the constructed view data of the video segment.

[0105] In order to implement the above embodiment, the embodiment of the present disclosure also provides an electronic device.

[0106] refer to Fig. 9 , which shows a schematic diagram of the structure of an electronic device 900 suitable for implementing an embodiment of the present disclosure, and the electronic device 900 may be various terminal devices and servers. Among them, the terminal device may include an extended reality device. The extended reality device may include, for example, various wearable devices, such as a head-mounted extended reality device, etc. Fig. 9 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0107] like Fig. 9As shown, the electronic device 900 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage device 908 to a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the electronic device 900 are also stored. The processing device 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0108] Typically, the following devices may be connected to the I / O interface 905: input devices 906 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 907 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 908 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 909. The communication device 909 may allow the electronic device 900 to communicate with other devices wirelessly or by wire to exchange data. Although Fig. 9 The electronic device 900 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0109] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 909, or installed from a storage device 908, or installed from a ROM 902. When the computer program is executed by the processing device 901, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0110] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0111] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0112] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.

[0113] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0114] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0115] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of a unit does not limit the unit itself in some cases. For example, the first generation unit may also be described as "a unit for generating initial data layer data according to the editing operation of multimedia materials".

[0116] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0117] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0118] In a first aspect, according to one or more embodiments of the present disclosure, a multimedia editing method is provided, comprising:

[0119] Generating initial data layer data according to the editing operation of the multimedia material, the initial data layer data including the multimedia material track segment data obtained by the editing operation;

[0120] Determine the editing effect attribute of the editing operation instruction, associate the editing effect attribute with the initial data layer data, and use the initial data layer data associated with the editing effect attribute as the target data layer data;

[0121] According to the received display instruction, the multimedia material track segment data and the editing effect attributes are obtained from the target data layer data using the consumption view layer, and the multimedia segment content after the editing operation is generated based on the multimedia material track segment data and the editing effect attributes.

[0122] According to one or more embodiments of the present disclosure, the multimedia material track segment data and the editing effect attribute are obtained from the target data layer data by using the consumption view layer, and the multimedia segment content after the editing operation is generated based on the multimedia material track segment data and the editing effect attribute, including:

[0123] In the consumption view layer, applying the editing effect attribute to the multimedia material track segment data to obtain the multimedia material track segment view data;

[0124] Generate multimedia segment content corresponding to the editing operation according to the multimedia material track segment view data.

[0125] According to one or more embodiments of the present disclosure, the multimedia material track fragment data includes audio material track fragment data and / or video material track fragment data.

[0126] According to one or more embodiments of the present disclosure, according to the received presentation instruction, obtaining multimedia material track fragment data from target data layer data using the consumption view layer includes:

[0127] Obtaining first video material track segment data including audio information from the target data layer using the consumer view layer;

[0128] Extracting second video material track segment data without audio information from the first video material track segment data;

[0129] The first video material track segment data is used as the corresponding audio track segment data.

[0130] According to one or more embodiments of the present disclosure, the editing effect attributes of the editing operation indication include one or more of the following:

[0131] Playback time attributes, playback speed attributes, and level attributes.

[0132] According to one or more embodiments of the present disclosure, in a consumption view layer, applying editing effect attributes to multimedia material track fragment data to obtain multimedia material track fragment view data includes:

[0133] The multimedia material track segment view data is constructed according to the order in which each multimedia material track segment data in the target data layer data appears on the time axis.

[0134] According to one or more embodiments of the present disclosure, in the consumption view layer, applying the editing effect attribute to the multimedia material track fragment data to obtain the multimedia material track fragment view data further includes:

[0135] Determining a target time period corresponding to the editing operation, updating target slice view data corresponding to the target time period; and generating multimedia segment content corresponding to the editing operation according to the multimedia material track segment view data, including:

[0136] Generate multimedia segment content corresponding to the editing operation according to the historical slice view data of other time periods and the target slice view data; wherein the historical slice view data is determined based on the following steps:

[0137] The timeline is divided according to a preset rule to obtain a plurality of slices, each of which may correspond to at least one video material track segment and at least one audio material track segment;

[0138] For each slice, the view data of the slice is constructed according to the constructed view data of the video segment.

[0139] In a second aspect, according to one or more embodiments of the present disclosure, a multimedia editing device is provided, comprising: a first generating unit, a determining unit, and a second generating unit.

[0140] A first generating unit, configured to generate initial data layer data according to an editing operation on the multimedia material, wherein the initial data layer data includes multimedia material track segment data obtained by the editing operation;

[0141] a determination unit, configured to determine an editing effect attribute of the editing operation instruction, associate the editing effect attribute with the initial data layer data, and use the initial data layer data associated with the editing effect attribute as the target data layer data;

[0142] The second generation unit is used to obtain multimedia material track segment data and editing effect attributes from the target data layer data using the consumption view layer according to the received display instruction, and generate the multimedia segment content after the editing operation based on the multimedia material track segment data and the editing effect attributes.

[0143] According to one or more embodiments of the present disclosure, the second generating unit is further configured to:

[0144] In the consumption view layer, applying the editing effect attribute to the multimedia material track segment data to obtain the multimedia material track segment view data;

[0145] Generate multimedia segment content corresponding to the editing operation according to the multimedia material track segment view data.

[0146] According to one or more embodiments of the present disclosure, the multimedia material track fragment data includes audio material track fragment data and / or video material track fragment data.

[0147] In some embodiments of the present disclosure, the second generating unit is further configured to:

[0148] Obtaining first video material track segment data including audio information from the target data layer using the consumer view layer;

[0149] Extracting second video material track segment data without audio information from the first video material track segment data;

[0150] The first video material track segment data is used as the corresponding audio track segment data.

[0151] According to one or more embodiments of the present disclosure, the editing effect attributes of the editing operation indication include one or more of the following:

[0152] Playback time attributes, playback speed attributes, and level attributes.

[0153] According to one or more embodiments of the present disclosure, the second generating unit is further configured to:

[0154] The multimedia material track segment view data is constructed according to the order in which each multimedia material track segment data in the target data layer data appears on the time axis.

[0155] According to one or more embodiments of the present disclosure, the second generating unit is further configured to:

[0156] Determine a target time period corresponding to the editing operation, and update target slice view data corresponding to the target time period;

[0157] Generate multimedia segment content corresponding to the editing operation according to the historical slice view data of other time periods and the target slice view data; wherein the historical slice view data is determined based on the following steps:

[0158] The timeline is divided according to a preset rule to obtain a plurality of slices, each of which may correspond to at least one video material track segment and at least one audio material track segment;

[0159] For each slice, the view data of the slice is constructed according to the constructed view data of the video segment.

[0160] In a third aspect, according to one or more embodiments of the present disclosure, there is provided an electronic device, comprising: at least one processor and a memory;

[0161] Memory stores computer-executable instructions;

[0162] At least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the first aspect as well as various multimedia editing methods that may be involved in the first aspect.

[0163] In a fourth aspect, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, in which computer execution instructions are stored. When a processor executes the computer execution instructions, the multimedia editing method as described in the first aspect and various possible aspects involved in the first aspect are implemented.

[0164] In a fifth aspect, according to one or more embodiments of the present disclosure, a computer program product is provided, including a computer program, which, when executed by a processor, implements the first aspect as well as various multimedia editing methods that may be involved in the first aspect.

[0165] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0166] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0167] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A multimedia editing method, comprising: Generating initial data layer data according to an editing operation on the multimedia material, the initial data layer data including multimedia material track segment data obtained by the editing operation; Determine the editing effect attribute of the editing operation indication, associate the editing effect attribute with the initial data layer data, and use the initial data layer data associated with the editing effect attribute as the target data layer data; According to the received display instruction, the multimedia material track segment data and the editing effect attributes are obtained from the target data layer data by using the consumption view layer, and the multimedia segment content after the editing operation is generated based on the multimedia material track segment data and the editing effect attributes.

2. The method according to claim 1, characterized in that The method of acquiring multimedia material track segment data and the editing effect attribute from the target data layer data by using the consumption view layer according to the received display instruction, and generating multimedia segment content after the editing operation based on the multimedia material track segment data and the editing effect attribute, includes: In the consumption view layer, applying the editing effect attribute to the multimedia material track segment data to obtain multimedia material track segment view data; The multimedia segment content corresponding to the editing operation is generated according to the multimedia material track segment view data.

3. The method according to claim 1, characterized in that The multimedia material track fragment data includes audio material track fragment data and / or video material track fragment data.

4. The method according to claim 1, characterized in that: The method of acquiring multimedia material track fragment data from the target data layer data by using the consumption view layer according to the received display instruction includes: Obtaining first video material track segment data including audio information from the target data layer using the consumption view layer; Extracting second video material track segment data without audio information from the first video material track segment data; The first video material track segment data is used as the corresponding audio track segment data.

5. The method according to claim 1, characterized in that The editing effect attributes indicated by the editing operation include one or more of the following: Playback time attributes, playback speed attributes, and level attributes.

6. The method according to claim 2, characterized in that The step of applying the editing effect attribute to the multimedia material track segment data in the consumption view layer to obtain the multimedia material track segment view data includes: The multimedia material track segment view data is constructed according to the order in which each multimedia material track segment data in the target data layer data appears on the time axis.

7. The method according to claim 6, characterized in that The step of applying the editing effect attribute to the multimedia material track segment data in the consumption view layer to obtain the multimedia material track segment view data further includes: Determining a target time period corresponding to the editing operation, and updating target slice view data corresponding to the target time period; and generating multimedia segment content corresponding to the editing operation according to the multimedia material track segment view data, comprising: Generate multimedia segment content corresponding to the editing operation according to historical slice view data of other time periods and the target slice view data; wherein the historical slice view data is determined based on the following steps: The timeline is divided according to a preset rule to obtain a plurality of slices, each of which may correspond to at least one video material track segment and at least one audio material track segment; For each slice, the view data of the slice is constructed according to the constructed view data of the video segment.

8. A multimedia editing device, comprising: A first generating unit, configured to generate initial data layer data according to an editing operation on the multimedia material, wherein the initial data layer data includes multimedia material track segment data obtained by the editing operation; a determining unit, configured to determine an editing effect attribute of the editing operation indication, associate the editing effect attribute with the initial data layer data, and use the initial data layer data associated with the editing effect attribute as the target data layer data; The second generation unit is used to obtain the multimedia material track segment data and the editing effect attributes from the target data layer data using the consumption view layer according to the received display instruction, and generate the multimedia segment content after the editing operation based on the multimedia material track segment data and the editing effect attributes.

9. An electronic device, comprising: Processor and memory; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when a processor executes the computer-executable instructions, the method according to any one of claims 1 to 7 is implemented.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images