Method, apparatus, and storage medium for processing audio and video data

Abstract

The embodiment of the present application provides a kind of audio and video data processing method and device, the method comprises: obtaining the original audio and video data to be edited;Initial state information is stored in operation queue based on original audio and video data, initial state information contains the track attribute information of all track data and the segment attribute information of all segment data;In response to each editing operation for target track data, according to the operation order of each editing operation, the editing state information corresponding to each editing operation is respectively stored in operation queue.The embodiment of the present application stores the editing state information of target track data after editing operation in operation queue, for non-target track data without editing operation, do not store any data in operation queue, reduce the amount of data stored in operation queue after each editing operation, reduce the resource occupation amount of operation queue to terminal.

Description

Technical Field

[0001] This invention relates to the field of audio and video technology, and in particular to a method and apparatus for processing audio and video data, as well as an electronic device and a computer-readable storage medium. Background Technology

[0002] When editing audio and video data on a terminal, a queue needs to be maintained in the terminal's local memory. For each editing operation, all tracks of the edited audio and video data need to be stored in the queue. The more operations performed on the audio and video data, the more data is stored in the queue, resulting in excessive consumption of terminal memory space and potentially causing terminal lag or crashes. Summary of the Invention

[0003] The purpose of this invention is to provide a method, apparatus, electronic device, and computer-readable storage medium for processing audio and video data, so as to reduce the memory usage of the terminal when editing audio and video data. The specific technical solution is as follows:

[0004] In a first aspect of this invention, a method for processing audio and video data is provided, applied to a terminal, the method comprising:

[0005] Obtain the original audio and video data to be edited, wherein the original audio and video data contains at least one track data, and at least one track data contains at least one segment data;

[0006] Based on the original audio and video data, initial state information is stored in the operation queue. The initial state information includes track attribute information of all track data and segment attribute information of all segment data.

[0007] In response to each edit operation on target orbit data in at least one of the orbit data, edit status information corresponding to each edit operation is stored in the operation queue in the order of each edit operation.

[0008] Optionally, in response to each edit operation on target orbit data in at least one of the orbit data, edit status information corresponding to each edit operation is stored in the operation queue according to the operation sequence of each edit operation, including:

[0009] In response to each edit operation on the target orbit data, the edit status information is stored in the operation queue according to the operation type of each edit operation and the orbit information of the target orbit data, and in the order of each edit operation.

[0010] Optionally, in response to each edit operation on the target orbit data, based on the operation type of each edit operation and the orbit information of the target orbit data, and in accordance with the operation order of each edit operation, the edit status information is stored in the operation queue, including:

[0011] In response to the addition operation for the target orbit data, according to the operation type of the addition operation and the orbit information of the target orbit data, the addition and editing status information corresponding to the addition operation is stored in the operation queue, and the corresponding index information is set for the addition and editing status information according to the operation order of the addition operation in each editing operation;

[0012] The added editing status information includes the track identifier and track type of the target track data, the segment identifier of the target segment data corresponding to the added operation, and the segment attribute information of the target segment data.

[0013] Optionally, in response to each edit operation on the target orbit data, based on the operation type of each edit operation and the orbit information of the target orbit data, and in accordance with the operation order of each edit operation, the edit status information is stored in the operation queue, including:

[0014] In response to the deletion operation on the target orbit data, according to the operation type of the deletion operation and the orbit information of the target orbit data, the deletion editing status information corresponding to the deletion operation is stored in the operation queue, and index information corresponding to the deletion editing status information is set according to the operation order of the deletion operation in each editing operation;

[0015] The deletion / editing status information includes the track identifier, track type, and segment identifier of the target segment data corresponding to the deletion operation.

[0016] Optionally, in response to each edit operation on the target orbit data, based on the operation type of each edit operation and the orbit information of the target orbit data, and in accordance with the operation order of each edit operation, the edit status information is stored in the operation queue, including:

[0017] In response to an update operation on the target orbit data, update editing status information corresponding to the update operation is stored in the operation queue according to the operation type of the update operation and the orbit information of the target orbit data, and index information corresponding to the update editing status information is set according to the operation order of the update operation in each of the editing operations;

[0018] The update editing status information includes the track identifier and track type of the target track data, the segment identifier and update content of the target segment data corresponding to the update operation.

[0019] Optionally, after storing the edit status information corresponding to each edit operation in the operation queue in accordance with the operation sequence of each edit operation in response to each edit operation on target orbit data in at least one of the orbit data, the method further includes:

[0020] In response to an undo operation for any of the edit operations, undo index information corresponding to the undo operation is identified from the operation queue. The undo index information is the index information of the edit status information corresponding to the edit operation adjacent to the undo operation.

[0021] The initial state information and the previous edit state information before the undo index information are merged to obtain the undo state information of the target track data.

[0022] Optionally, the initial state information and the edit state information preceding the undo index information are merged to obtain the undo state information of the target orbit data, including:

[0023] Based on the sequential relationship between the edit status information preceding the undo index information, the undo edit status information preceding the undo index information is sequentially accumulated to obtain undo edit status accumulation information, and the initial status information is superimposed with the undo edit status accumulation information to obtain the undo status information.

[0024] Optionally, after merging the initial state information and the edit state information prior to the undo index information to obtain the undo state information of the target orbit data, the method further includes:

[0025] The cancellation status information is stored in the operation queue, and the target orbit data is restored based on the cancellation status information.

[0026] Optionally, after merging the initial state information and the edit state information prior to the undo index information to obtain the undo state information of the target orbit data, the method further includes:

[0027] In response to a forward operation for the undo operation, forward index information corresponding to the forward operation is identified from the operation queue, wherein the forward index information is the index information following the undo index information;

[0028] The forward state information of the target trajectory data is obtained by merging the initial state information and the previous edit state information before the forward index information.

[0029] Optionally, the step of merging the initial state information and the previous edit state information before the forward index information to obtain the forward state information of the target orbit data includes:

[0030] According to the sequential relationship between the edit state information before the forward index information, the edit state information before the forward index information is accumulated sequentially to obtain the forward edit state accumulation information, and the initial state information is superimposed with the forward edit state accumulation information to obtain the forward state information.

[0031] Optionally, after merging the initial state information and the preceding edit state information to obtain the forward state information of the target orbit data, the method further includes:

[0032] The forward progress information is stored in the operation queue, and the target trajectory data is restored based on the forward progress information.

[0033] In a second aspect of the invention, an audio / video data editing device is also provided, applied to a terminal, the device comprising:

[0034] The data acquisition module is used to acquire the original audio and video data to be edited, wherein the original audio and video data includes at least one track data, and the at least one track data includes at least one segment data;

[0035] The information storage module is used to store initial state information into the operation queue based on the original audio and video data. The initial state information includes track attribute information of all track data and segment attribute information of all segment data.

[0036] The information storage module is further configured to, in response to each editing operation on target orbit data in at least one of the orbit data, store editing status information corresponding to each editing operation in the operation queue according to the operation sequence of each editing operation.

[0037] Optionally, the information storage module is configured to, in response to each editing operation on the target orbit data, store the editing status information in the operation queue according to the operation type of each editing operation and the orbit information of the target orbit data, and in the order of each editing operation.

[0038] Optionally, the information storage module is configured to, in response to an add operation on the target orbit data, store add / edit status information corresponding to the add operation in the operation queue according to the operation type of the add operation and the orbit information of the target orbit data, and set corresponding index information for the add / edit status information according to the operation order of the add operation in each edit operation;

[0039] The added editing status information includes the track identifier and track type of the target track data, the segment identifier of the target segment data corresponding to the added operation, and the segment attribute information of the target segment data.

[0040] Optionally, the information storage module is configured to, in response to a deletion operation on the target track data, store deletion and editing status information corresponding to the deletion operation in the operation queue according to the operation type of the deletion operation and the track information of the target track data, and set corresponding index information for the deletion and editing status information according to the operation order of the deletion operation in each editing operation;

[0041] The deletion / editing status information includes the track identifier, track type, and segment identifier of the target segment data corresponding to the deletion operation.

[0042] Optionally, the information storage module is configured to, in response to an update operation on the target orbit data, store update editing status information corresponding to the update operation in the operation queue according to the operation type of the update operation and the orbit information of the target orbit data, and set corresponding index information for the update editing status information according to the operation order of the update operation in each of the editing operations;

[0043] The update editing status information includes the track identifier and track type of the target track data, the segment identifier and update content of the target segment data corresponding to the update operation.

[0044] Optionally, the device further includes:

[0045] The identification module is configured to, after the information storage module stores the editing status information corresponding to each editing operation in the operation queue according to the operation order of each editing operation for each target track data in at least one of the track data, identify the undo index information corresponding to the undo operation from the operation queue in response to the undo operation for any editing operation, wherein the undo index information is the index information of the editing status information corresponding to the editing operation adjacent to the undo operation;

[0046] The information merging module is used to merge the initial state information and the edit state information before the undo index information to obtain the undo state information of the target track data.

[0047] Optionally, the information merging module is used to sequentially accumulate the edit status information before the undo index information according to the order relationship between the edit status information before the undo index information to obtain undo edit status accumulation information, and then superimpose the initial status information with the undo edit status accumulation information to obtain the undo status information.

[0048] Optionally, the device further includes:

[0049] The restoration processing module is used to store the cancellation status information of the target orbit data in the operation queue after the information merging module merges the initial status information and the cancellation index information to obtain the cancellation status information of the target orbit data, and restores the target orbit data according to the cancellation status information.

[0050] Optionally, the identification module is further configured to, in response to a forward operation for the cancellation operation, identify forward index information corresponding to the forward operation from the operation queue after the information merging module merges the initial state information and the cancellation index information before the cancellation index information to obtain the cancellation state information of the target track data, wherein the forward index information is the index information after the cancellation index information;

[0051] The information merging module is further configured to merge the initial state information and the edit state information preceding the forward index information to obtain the forward state information of the target trajectory data.

[0052] Optionally, the information merging module is used to sequentially accumulate each of the edit state information before the forward index information according to the order relationship between the edit state information before the forward index information to obtain forward edit state accumulation information, and to superimpose the initial state information with the forward edit state accumulation information to obtain the forward state information.

[0053] Optionally, the restoration processing module is further configured to, after the information merging module merges the initial state information and the preceding edit state information to obtain the forward state information of the target orbit data, store the forward state information in the operation queue, and perform restoration processing on the target orbit data according to the forward state information.

[0054] In another aspect of the present invention, a computer-readable storage medium is also provided, wherein instructions are stored therein, which, when executed on a computer, cause the computer to perform any of the above-described methods for processing audio and video data.

[0055] In another aspect of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the above-described methods for processing audio and video data.

[0056] The audio and video data processing method provided in this embodiment of the invention avoids storing all track data of the audio and video data in the operation queue after each editing operation on the target track data. This is achieved by storing the initial state information of the original audio and video data in the operation queue and storing the editing state information corresponding to each editing operation in the operation queue after each editing operation on the target track data. This embodiment of the invention stores the editing state information of the target track data that has undergone editing operations in the operation queue, while not storing any data for non-target track data that has not undergone editing operations in the operation queue. This reduces the amount of data stored in the operation queue after each editing operation and lowers the resource consumption of the operation queue on the terminal. Attached Figure Description

[0057] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0058] Figure 1 This is a flowchart illustrating the steps of an audio / video data processing method according to an embodiment of the present invention.

[0059] Figure 2 This is a schematic diagram of track data in the original audio and video data in an embodiment of the present invention;

[0060] Figure 3 This is a schematic diagram of the structure of an audio and video data editing device according to an embodiment of the present invention;

[0061] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0062] The technical solutions of the present invention will now be described with reference to the accompanying drawings in the embodiments of the present invention.

[0063] This invention provides an audio / video data editing scheme. This editing scheme can be applied to a terminal, which can serve as an audio / video data editing client, or provide technical support for an audio / video data editing client. This invention first stores the initial state information of the original audio / video data in the terminal's operation queue. Then, after each editing operation on the target track data, it stores the editing state information related to the target track data in the operation queue, but does not store editing state information related to non-target track data in the operation queue.

[0064] Reference Figure 1 This diagram illustrates a flowchart of a method for processing audio and video data according to an embodiment of the present invention. This processing method can be applied to a terminal. The terminal can be an audio and video data editing client, or it can provide technical support to an audio and video data editing client. Specifically, the processing method may include the following steps.

[0065] Step 101: Obtain the original audio and video data to be edited.

[0066] In embodiments of the present invention, the original audio and video data may include at least one track data. In practical applications, the track data of the original audio and video data may include audio track data, video track data, transition track data, subtitle track data, etc. Each track data may contain at least one segment data. Audio track data may contain one or more audio segment data. Video track data may contain one or more video segment data. For example, refer to... Figure 2 , Figure 2 A schematic diagram of the track data in the original audio and video data is shown. Figure 2 The original audio and video data includes Text Track 1, Sticker Track 1, and Video Track 1. Text Track 1 contains two text snippets: Text 1 and Text 2. Sticker Track 1 contains two sticker snippets: vlog-in business and vlog-lately two or three… Video Track 1 contains two video snippets: Transition 1 and Transition 2.

[0067] Step 102: Store initial state information in the operation queue based on the original audio and video data.

[0068] In embodiments of the present invention, an operation queue for raw audio and video data can be pre-set in the terminal, and this operation queue can be located in the terminal's memory. The memory space occupied by the operation queue can be dynamically changed. The memory space occupied by the operation queue increases as the amount of data stored in the operation queue increases, and decreases as the amount of data stored in the operation queue decreases.

[0069] In embodiments of the present invention, before performing editing operations on the original audio and video data, initial state information of the original audio and video data is stored in the operation queue. This initial state information may include track attribute information for all tracks of the original audio and video data and segment attribute information for all segment data. For example, the track attribute information for audio track data may include audio track identifiers, audio track time points (start and end times of the audio track data), and audio track Uniform Resource Locators (URLs). The track attribute information for video track data may include video track identifiers, video track time points (start and end times of the video track data), and video track URLs.

[0070] Step 103: In response to each edit operation on target orbit data in at least one of the orbit data, edit status information corresponding to each edit operation is stored in the operation queue according to the operation sequence of each edit operation.

[0071] In embodiments of the present invention, each editing operation can target one or more tracks from the entire track data of the original audio data. The track data targeted by the editing operation is the target track data. Each editing operation can target the same target track data or different target track data. After each editing operation, editing status information is stored in the operation queue. It should be noted that the editing status information stored in the operation queue after the editing operation can be the editing status information corresponding to the target track data, not the editing status information of all track data.

[0072] For example, the original audio / video data D01 includes audio track data a1 and video track data v1. Audio track data a1 contains audio segment data a1-1 and a1-2, and video track data v1 contains video segment data v1-1 and v1-2. First, the initial state information cs of the original audio data D01 is stored in the operation queue M. The initial state information cs includes track attribute information for audio track data a1 and video track data v1, as well as segment attribute information for audio segment data a1-1 and a1-2, and video segment data v1-1 and v1-2. After the user performs an editing operation c1 on the audio track data, specifically, on audio segment data a1-1, the editing state information bz1 corresponding to editing operation c1 is stored in the operation queue M. Next, after the user performs an editing operation c2 on the video track data, specifically, on video segment data v1-1, the editing state information bz2 corresponding to editing operation c2 is stored in the operation queue M. At this point, the operation queue M contains initial state information cs, edit state information bz1, and edit state information bz2. Furthermore, the initial state information cs, edit state information bz1, and edit state information bz2 are arranged sequentially. If processed according to the solution in the background art, in addition to storing the initial state information cs, edit state information bz1, and edit state information bz2, the operation queue M also needs to store the state information of audio segment data a1-2, video segment data v1-1, and v1-2 after editing operation c1, as well as the state information of video segment data v1-2, audio segment data a1-1, and a1-2 after editing operation c2.

[0073] The audio and video data processing method provided in this embodiment of the invention avoids storing all track data of the audio and video data in the operation queue after each editing operation on the target track data. This is achieved by storing the initial state information of the original audio and video data in the operation queue and storing the editing state information corresponding to each editing operation in the operation queue after each editing operation on the target track data. This embodiment of the invention stores the editing state information of the target track data that has undergone editing operations in the operation queue, while not storing any data for non-target track data that has not undergone editing operations in the operation queue. This reduces the amount of data stored in the operation queue after each editing operation and lowers the resource consumption of the operation queue on the terminal.

[0074] In a preferred embodiment of the present invention, in response to each editing operation on target orbit data in at least one of the orbit data, an implementation method is to store editing status information corresponding to each editing operation in the operation queue according to the operation order of each editing operation. In response to each editing operation on the target orbit data, the editing status information is stored in the operation queue according to the operation type of each editing operation and the orbit information of the target orbit data, and according to the operation order of each editing operation.

[0075] In practical applications, editing operations can be specifically divided into add, delete, and update operations. Add operations are categorized as add operations, delete operations as delete operations, and update operations as update operations. The orbital information of the target orbital data may include, but is not limited to, orbital attribute information and segment attribute information.

[0076] When the editing operation is an add operation, in response to the add operation on the target track data, according to the operation type of the add operation and the track information of the target track data, add editing status information corresponding to the add operation is stored in the operation queue, and index information is set for the add editing status information according to the operation order of the add operation in each edit operation. The add editing status information includes the track identifier and track type of the target track data, the segment identifier of the target segment data corresponding to the add operation, and the segment attribute information of the target segment data.

[0077] In practical applications, adding data to a target track can involve adding segment data to existing target track data. For example, if audio track data a1 already exists, audio segment data a1-3 can be added to audio track data a1. Alternatively, adding data to a target track can involve directly adding segment data if the target track data does not exist. For example, if video track data v1 does not exist, video segment data v1-3 can be added directly. Simultaneously with adding video segment data v1-3, video track data v1 containing that video segment data v1-3 is also added.

[0078] For example, a user performs an add operation on audio track data a1, adding audio segment data a1-3 to audio track data a1. Based on the operation type and track information of audio track data a1, the corresponding add / edit status information is stored in the operation queue. This add / edit status information may include, but is not limited to: the track identifier and track type of audio track data a1, the segment identifier and segment attribute information of the audio segment data a1-3 corresponding to the add operation. In practical applications, this add / edit status information can be represented as: type:'add', trackType:'audio', trackId:a1, clipId:a1-3, all detail information. Here, type represents the operation type, add indicates the operation type is add, trackType represents the track type of the target track data, audio indicates the track type is audio track data, trackId:a1 indicates the track identifier of the target track data is a1, clipId:a1-3 indicates the segment identifier of the added audio segment data is a1-3, and all detail information represents the segment attribute information of the added audio segment data.

[0079] If the add operation is the first operation in the order of each edit operation, then when the index information of the initial state information is 0, the index information of the add edit state information corresponding to the add operation is 1.

[0080] When the editing operation is a deletion operation, in response to the deletion operation on the target trajectory data, based on the operation type of the deletion operation and the trajectory information of the target trajectory data, deletion editing status information corresponding to the deletion operation is stored in the operation queue, and index information corresponding to the deletion editing status information is set according to the operation order of the deletion operation in each editing operation. The deletion editing status information includes the trajectory identifier, trajectory type, and segment identifier of the target segment data corresponding to the deletion operation.

[0081] For example, a user performs a delete operation on audio track data a1, deleting audio segment data a1-3 within audio track data a1. Based on the operation type and track information of audio track data a1, the corresponding deletion / editing status information is stored in the operation queue. This deletion / editing status information may include, but is not limited to: the track identifier and track type of audio track data a1, and the segment identifier of the audio segment data a1-3 corresponding to the deletion operation. In practical applications, this deletion / editing status information can be represented as: type:'delete', trackType:'audio', trackId:a1, clipId:a1-3. Here, type represents the operation type, delete indicates a delete operation, trackType represents the track type of the target track data, audio indicates the track type is audio track data, trackId:a1 indicates the track identifier of the target track data is a1, and clipId:a1-3 indicates the segment identifier of the deleted audio segment data is a1-3.

[0082] When an edit operation is an update operation, in response to the update operation on the target trajectory data, update edit status information corresponding to the update operation is stored in the operation queue according to the operation type of the update operation and the track information of the target track data. Index information corresponding to the update edit status information is then set according to the operation order of the update operation in each edit operation. The update edit status information includes the track identifier and track type of the target track data, the segment identifier of the target segment data corresponding to the update operation, and the update content.

[0083] For example, a user performs an update operation on audio track data a1, updating the duration of audio segment data a1-3 within audio track data a1. Based on the operation type and track information of audio track data a1, update editing status information corresponding to the update operation is stored in the operation queue. This update editing status information may include, but is not limited to: the track identifier, track type, update content, and the segment identifier of the audio segment data a1-3 corresponding to the update operation. In practical applications, this update editing status information can be represented as: type:'update', trackType:'audio', trackId:a1, clipId:a1-3, duration:200. Here, type indicates the operation type, update indicates an update operation, trackType indicates the track type of the target track data, audio indicates the track type is audio track data, trackId:a1 indicates the track identifier of the target track data is a1, clipId:a1-3 indicates the segment identifier of the updated audio segment data is a1-3, and duration:200 indicates the duration of the updated audio segment data is 200.

[0084] It should be noted that when setting the corresponding index information for deleting and updating edit status information, the same process as setting the corresponding index information for adding edit status information can be used, and will not be repeated here.

[0085] This invention combines the specific operation type of the editing operation with the specific track information of the target track data to store editing status information. For audio and video data with rich editing operation types and abundant track information, the corresponding editing status information can be stored in detail according to the editing operation type and track information, further refining the editing status information.

[0086] In a preferred embodiment of the present invention, after storing the edit status information corresponding to each edit operation in the operation queue according to the operation sequence of each edit operation in response to each edit operation of the target track data in at least one of the track data, if the user performs an undo operation for any edit operation, then in response to the undo operation, the undo index information corresponding to the undo operation is identified from the operation queue, and starting from the initial status information, the edit status information before the undo index information is merged to obtain the undo status information of the target track data. The undo index information is the index information of the edit status information corresponding to the edit operation adjacent to the undo operation.

[0087] For example, after a user sequentially performs add, update, and delete operations on target track data, they then perform an undo operation on the delete operation. It should be noted that the target track data used for each edit operation can be the same track data or different track data. In response to this undo operation, the undo index information corresponding to the undo operation is identified from the operation queue. This undo index information is the index information of the delete edit status information corresponding to the delete operation. If the initial status information index is s0, and the index information of the edit status information corresponding to the add, update, and delete operations are s1, s2, and s3 respectively, then the undo index information is s3. Starting from the initial status information, the edit status information with indexes s1 and s2 are merged to obtain the undo status information.

[0088] In a preferred embodiment of the present invention, one method for merging the initial state information and the edit state information preceding the undo index information to obtain the undo state information of the target track data is as follows: Following the sequential relationship between the edit state information preceding the undo index information, the edit state information preceding the undo index information is sequentially accumulated to obtain undo edit state accumulation information; then, the initial state information and the undo edit state accumulation information are superimposed to obtain the undo state information. Continuing the previous example, the edit state information with index information s1 and s2 are accumulated to obtain undo edit state accumulation information, and then the undo edit state accumulation information is superimposed to the initial state information to obtain the undo state information.

[0089] In a preferred embodiment of the present invention, after merging the initial state information and the edit state information prior to the undo index information to obtain the undo state information of the target track data, the undo state information can be stored in the operation queue and used as the current edit state information of the target track data. The target track data is then restored based on the undo state information. That is, the target track data is restored to the state corresponding to the undo state information. Continuing the previous example, the target track data is restored to the state after the aforementioned update operation.

[0090] In a preferred embodiment of the present invention, after merging the initial state information and the edited state information prior to the undo index information to obtain the undo state information of the target track data, a forward operation can be performed on the target track data according to the actual situation. This forward operation can be understood as a recovery operation for the undo operation. Similar to the processing flow of the undo operation, in response to the forward operation for the undo operation, the forward index information corresponding to the forward operation is identified from the operation queue. This forward index information can be the index information after the undo index information. Then, the initial state information and the edited state information prior to the forward index information are merged to obtain the forward state information of the target track data.

[0091] For example, after a user sequentially performs add, update, and delete operations on target track data, they then perform an undo operation on the delete operation. It should be noted that the target track data used for each edit operation can be the same track data or different track data. After performing the undo operation, a forward operation is performed. In response to this forward operation, the forward index information corresponding to the forward operation is identified from the operation queue. This forward index information is the index information after the undo index information. If the initial state information index information is s0, and the corresponding edit state information index information for the add, update, and delete operations is s1, s2, and s3 respectively, then the undo index information is s3, and the forward index information is s4. Starting from the initial state information, the edit state information with index information of s1, s2, and s3 are merged to obtain the forward state information.

[0092] In a preferred embodiment of the present invention, one method for merging the initial state information and the edited state information preceding the forward index information to obtain the forward state information of the target trajectory data is as follows: According to the sequential relationship between the edited state information preceding the forward index information, the edited state information preceding the forward index information is sequentially accumulated to obtain forward edited state accumulated information; then, the initial state information and the forward edited state accumulated information are superimposed to obtain the forward state information. Continuing the previous example, the edited state information with index information s1, s2, and s3 are accumulated to obtain forward edited state accumulated information, and then the forward edited state accumulated information is superimposed with the initial state information to obtain the forward state information.

[0093] In a preferred embodiment of the present invention, after merging the initial state information and the previous edit state information before the forward index information to obtain the forward state information of the target track data, the forward state information can be stored in the operation queue and used as the current edit state information of the target track data. The target track data is then restored based on the forward state information. That is, the target track data is restored to the state corresponding to the forward state information. Continuing the previous example, the target track data is restored to the state after the aforementioned deletion operation.

[0094] Regardless of the type of editing operation performed on audio and video data, such as copying or deleting, the operation queue stores the corresponding editing status information. When the terminal needs to render and display the audio and video data, it can read the current editing status information from the operation queue and render and display the audio and video data according to the current editing status information.

[0095] This invention not only stores corresponding editing status information in the operation queue when performing editing operations on audio and video data, but also stores corresponding undo or forward status information in the operation queue when performing undo or forward operations, so as to restore the audio and video data based on the undo or forward status information, thereby improving the convenience of audio and video data restoration processing.

[0096] Reference Figure 3 This diagram illustrates a structural schematic of an audio / video data editing device according to an embodiment of the present invention. This editing device can be applied to a terminal. The terminal can be an audio / video data editing client, or it can provide technical support to an audio / video data editing client. Specifically, the editing device may include the following modules.

[0097] The data acquisition module 31 is used to acquire the original audio and video data to be edited, wherein the original audio and video data includes at least one track data, and the at least one track data includes at least one segment data;

[0098] Information storage module 32 is used to store initial state information into the operation queue based on the original audio and video data. The initial state information includes track attribute information of all track data and segment attribute information of all segment data.

[0099] The information storage module 32 is also configured to, in response to each editing operation on target orbit data in at least one of the orbit data, store editing status information corresponding to each editing operation in the operation queue according to the operation sequence of each editing operation.

[0100] In a preferred embodiment of the present invention, the information storage module 32 is configured to, in response to each editing operation on the target orbit data, store the editing status information in the operation queue according to the operation type of each editing operation and the orbit information of the target orbit data, and in the order of each editing operation.

[0101] In a preferred embodiment of the present invention, the information storage module 32 is configured to, in response to an add operation on the target orbit data, store add / edit status information corresponding to the add operation in the operation queue according to the operation type of the add operation and the orbit information of the target orbit data, and set corresponding index information for the add / edit status information according to the operation order of the add operation in each edit operation;

[0102] The added editing status information includes the track identifier and track type of the target track data, the segment identifier of the target segment data corresponding to the added operation, and the segment attribute information of the target segment data.

[0103] In a preferred embodiment of the present invention, the information storage module 32 is configured to, in response to a deletion operation on the target orbit data, store deletion editing status information corresponding to the deletion operation in the operation queue according to the operation type of the deletion operation and the orbit information of the target orbit data, and set corresponding index information for the deletion editing status information according to the operation order of the deletion operation in each editing operation;

[0104] The deletion / editing status information includes the track identifier, track type, and segment identifier of the target segment data corresponding to the deletion operation.

[0105] In a preferred embodiment of the present invention, the information storage module 32 is configured to, in response to an update operation on the target orbit data, store update editing status information corresponding to the update operation in the operation queue according to the operation type of the update operation and the orbit information of the target orbit data, and set corresponding index information for the update editing status information according to the operation order of the update operation in each of the editing operations;

[0106] The update editing status information includes the track identifier and track type of the target track data, the segment identifier and update content of the target segment data corresponding to the update operation.

[0107] In a preferred embodiment of the present invention, the device further includes:

[0108] The identification module is configured to, after the information storage module 32 stores the editing status information corresponding to each editing operation in the operation queue according to the operation order of each editing operation for each target track data in at least one of the track data, identify the undo index information corresponding to the undo operation from the operation queue in response to the undo operation for any editing operation, wherein the undo index information is the index information of the editing status information corresponding to the editing operation adjacent to the undo operation;

[0109] The information merging module is used to merge the initial state information and the edit state information before the undo index information to obtain the undo state information of the target track data.

[0110] In a preferred embodiment of the present invention, the information merging module is used to sequentially accumulate each of the edit state information before the undo index information according to the order relationship between the edit state information before the undo index information to obtain undo edit state accumulation information, and superimpose the initial state information and the undo edit state accumulation information to obtain the undo state information.

[0111] In a preferred embodiment of the present invention, the device further includes:

[0112] The restoration processing module is used to store the cancellation status information of the target orbit data in the operation queue after the information merging module merges the initial status information and the cancellation index information to obtain the cancellation status information of the target orbit data, and restores the target orbit data according to the cancellation status information.

[0113] In a preferred embodiment of the present invention, the identification module is further configured to, in response to a forward operation for the cancellation operation, identify forward index information corresponding to the forward operation from the operation queue after the information merging module merges the initial state information and the edit state information before the cancellation index information to obtain the cancellation state information of the target track data, wherein the forward index information is the index information after the cancellation index information;

[0114] The information merging module is further configured to merge the initial state information and the edit state information preceding the forward index information to obtain the forward state information of the target trajectory data.

[0115] In a preferred embodiment of the present invention, the information merging module is used to sequentially accumulate each of the edit state information before the forward index information according to the order relationship between the edit state information before the forward index information to obtain forward edit state accumulation information, and to superimpose the initial state information and the forward edit state accumulation information to obtain the forward state information.

[0116] In a preferred embodiment of the present invention, the restoration processing module is further configured to store the forward state information in the operation queue after the information merging module merges the initial state information and the forward index information before each of the edit state information to obtain the forward state information of the target orbit data, and perform restoration processing on the target orbit data according to the forward state information.

[0117] This invention also provides an electronic device, such as... Figure 4As shown, it includes a processor 41, a communication interface 42, a memory 43, and a communication bus 44, wherein the processor 41, the communication interface 42, and the memory 43 communicate with each other through the communication bus 44.

[0118] Memory 43 is used to store computer programs;

[0119] When processor 41 executes the program stored in memory 43, it performs the following steps:

[0120] Obtain the original audio and video data to be edited, wherein the original audio and video data contains at least one track data, and at least one track data contains at least one segment data;

[0121] Based on the original audio and video data, initial state information is stored in the operation queue. The initial state information includes track attribute information of all track data and segment attribute information of all segment data.

[0122] In response to each edit operation on target orbit data in at least one of the orbit data, edit status information corresponding to each edit operation is stored in the operation queue in the order of each edit operation.

[0123] The above-described step of storing, in order of the operation sequence of each edit operation, editing status information corresponding to each edit operation in the operation queue for each edit operation of at least one target orbit data in the orbit data includes:

[0124] In response to each edit operation on the target orbit data, the edit status information is stored in the operation queue according to the operation type of each edit operation and the orbit information of the target orbit data, and in the order of each edit operation.

[0125] The above-described step of storing the edit status information into the operation queue according to the operation type of each edit operation and the orbit information of the target orbit data, and in accordance with the operation order of each edit operation, includes:

[0126] In response to the addition operation for the target orbit data, according to the operation type of the addition operation and the orbit information of the target orbit data, the addition and editing status information corresponding to the addition operation is stored in the operation queue, and the corresponding index information is set for the addition and editing status information according to the operation order of the addition operation in each editing operation;

[0127] The added editing status information includes the track identifier and track type of the target track data, the segment identifier of the target segment data corresponding to the added operation, and the segment attribute information of the target segment data.

[0128] The above-described step of storing the edit status information into the operation queue according to the operation type of each edit operation and the orbit information of the target orbit data, and in accordance with the operation order of each edit operation, includes:

[0129] In response to the deletion operation on the target orbit data, according to the operation type of the deletion operation and the orbit information of the target orbit data, the deletion editing status information corresponding to the deletion operation is stored in the operation queue, and index information corresponding to the deletion editing status information is set according to the operation order of the deletion operation in each editing operation;

[0130] The deletion / editing status information includes the track identifier, track type, and segment identifier of the target segment data corresponding to the deletion operation.

[0131] The above-described step of storing the edit status information into the operation queue according to the operation type of each edit operation and the orbit information of the target orbit data, and in accordance with the operation order of each edit operation, includes:

[0132] In response to an update operation on the target orbit data, update editing status information corresponding to the update operation is stored in the operation queue according to the operation type of the update operation and the orbit information of the target orbit data, and index information corresponding to the update editing status information is set according to the operation order of the update operation in each of the editing operations;

[0133] The update editing status information includes the track identifier and track type of the target track data, the segment identifier and update content of the target segment data corresponding to the update operation.

[0134] After the step of storing the edit status information corresponding to each edit operation in the operation queue in accordance with the operation sequence of each edit operation in response to each edit operation of target orbit data in at least one of the orbit data, the method further includes:

[0135] In response to an undo operation for any of the edit operations, undo index information corresponding to the undo operation is identified from the operation queue. The undo index information is the index information of the edit status information corresponding to the edit operation adjacent to the undo operation.

[0136] The initial state information and the previous edit state information before the undo index information are merged to obtain the undo state information of the target track data.

[0137] The step of merging the initial state information and the edit state information preceding the undo index information to obtain the undo state information of the target orbit data includes:

[0138] Based on the sequential relationship between the edit status information preceding the undo index information, the undo edit status information preceding the undo index information is sequentially accumulated to obtain undo edit status accumulation information, and the initial status information is superimposed with the undo edit status accumulation information to obtain the undo status information.

[0139] After the step of merging the initial state information and the edit state information prior to the undo index information to obtain the undo state information of the target orbit data, as described above, the method further includes:

[0140] The cancellation status information is stored in the operation queue, and the target orbit data is restored based on the cancellation status information.

[0141] After the step of merging the initial state information and the edit state information prior to the undo index information to obtain the undo state information of the target orbit data, as described above, the method further includes:

[0142] In response to a forward operation for the undo operation, forward index information corresponding to the forward operation is identified from the operation queue, wherein the forward index information is the index information following the undo index information;

[0143] The forward state information of the target trajectory data is obtained by merging the initial state information and the previous edit state information before the forward index information.

[0144] The step of merging the initial state information and the preceding edit state information to obtain the forward state information of the target orbit data, as described above, includes:

[0145] According to the sequential relationship between the edit state information before the forward index information, the edit state information before the forward index information is accumulated sequentially to obtain the forward edit state accumulation information, and the initial state information is superimposed with the forward edit state accumulation information to obtain the forward state information.

[0146] After the step of merging the initial state information and the preceding edit state information to obtain the forward state information of the target orbit data as described above, the method further includes:

[0147] The forward progress information is stored in the operation queue, and the target trajectory data is restored based on the forward progress information.

[0148] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0149] The communication interface is used for communication between the aforementioned terminal and other devices.

[0150] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0151] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0152] In another embodiment of the present invention, a computer-readable storage medium is also provided, which stores instructions that, when executed on a computer, cause the computer to perform any of the audio and video data processing methods described in the above embodiments.

[0153] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the audio and video data processing methods described in the above embodiments.

[0154] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0155] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0156] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0157] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A method for processing audio and video data, characterized in that, Applied to a terminal, the method includes: Obtain the original audio and video data to be edited, wherein the original audio and video data contains at least two track data, and at least one track data contains at least one segment data; Initial state information is stored in the operation queue based on the original audio and video data. The initial state information includes track attribute information of all track data and segment attribute information of all segment data. The track attribute information includes at least track identifier, track time point, and track unified resource locator. In response to each edit operation on target track data in at least one of the track data, based on the operation type of each edit operation and the track information of the target track data, and in the order of each edit operation, edit status information corresponding to each edit operation is stored in the operation queue; the edit status information is the edit status information corresponding to the target track data, and when the edit operation is a deletion operation, the edit status information includes the track identifier, track type, and segment identifier of the target segment data corresponding to the edit operation.

2. The method according to claim 1, characterized in that, In response to each edit operation on the target orbit data, based on the operation type of each edit operation and the orbit information of the target orbit data, and in accordance with the operation order of each edit operation, the edit status information is stored in the operation queue, including: In response to the addition operation for the target orbit data, according to the operation type of the addition operation and the orbit information of the target orbit data, the addition and editing status information corresponding to the addition operation is stored in the operation queue, and the corresponding index information is set for the addition and editing status information according to the operation order of the addition operation in each editing operation; The added editing status information includes the track identifier and track type of the target track data, the segment identifier of the target segment data corresponding to the added operation, and the segment attribute information of the target segment data.

3. The method according to claim 1, characterized in that, In response to each edit operation on the target orbit data, based on the operation type of each edit operation and the orbit information of the target orbit data, and in accordance with the operation order of each edit operation, the edit status information is stored in the operation queue, including: In response to the deletion operation on the target orbit data, according to the operation type of the deletion operation and the orbit information of the target orbit data, the deletion editing status information corresponding to the deletion operation is stored in the operation queue, and index information corresponding to the deletion editing status information is set according to the operation order of the deletion operation in each editing operation; The deletion / editing status information includes the track identifier, track type, and segment identifier of the target segment data corresponding to the deletion operation.

4. The method according to claim 1, characterized in that, In response to each edit operation on the target orbit data, based on the operation type of each edit operation and the orbit information of the target orbit data, and in accordance with the operation order of each edit operation, the edit status information is stored in the operation queue, including: In response to an update operation on the target orbit data, update editing status information corresponding to the update operation is stored in the operation queue according to the operation type of the update operation and the orbit information of the target orbit data, and index information corresponding to the update editing status information is set according to the operation order of the update operation in each of the editing operations; The update editing status information includes the track identifier and track type of the target track data, the segment identifier and update content of the target segment data corresponding to the update operation.

5. The method according to any one of claims 2-4, characterized in that, After storing edit status information corresponding to each edit operation in the operation queue in the order of each edit operation in response to each edit operation of target orbit data in at least one of the orbit data, the method further includes: In response to an undo operation for any of the edit operations, undo index information corresponding to the undo operation is identified from the operation queue. The undo index information is the index information of the edit status information corresponding to the edit operation adjacent to the undo operation. The initial state information and the previous edit state information before the undo index information are merged to obtain the undo state information of the target track data.

6. The method according to claim 5, characterized in that, The initial state information and the edit state information preceding the undo index information are merged to obtain the undo state information of the target track data, including: Based on the sequential relationship between the edit status information preceding the undo index information, the undo edit status information preceding the undo index information is sequentially accumulated to obtain undo edit status accumulation information, and the initial status information is superimposed with the undo edit status accumulation information to obtain the undo status information.

7. The method according to claim 5, characterized in that, After merging the initial state information and the edit state information prior to the undo index information to obtain the undo state information of the target orbit data, the method further includes: The cancellation status information is stored in the operation queue, and the target orbit data is restored based on the cancellation status information.

8. The method according to claim 5, characterized in that, After merging the initial state information and the edit state information prior to the undo index information to obtain the undo state information of the target orbit data, the method further includes: In response to a forward operation for the undo operation, forward index information corresponding to the forward operation is identified from the operation queue, wherein the forward index information is the index information following the undo index information; The forward state information of the target trajectory data is obtained by merging the initial state information and the previous edit state information before the forward index information.

9. The method according to claim 8, characterized in that, The step of merging the initial state information and the preceding edit state information with the forward index information to obtain the forward state information of the target trajectory data includes: According to the sequential relationship between the edit state information before the forward index information, the edit state information before the forward index information is accumulated sequentially to obtain the forward edit state accumulation information, and the initial state information is superimposed with the forward edit state accumulation information to obtain the forward state information.

10. The method according to claim 8, characterized in that, After merging the initial state information and the preceding edit state information with the forward index information to obtain the forward state information of the target orbit data, the method further includes: The forward progress information is stored in the operation queue, and the target trajectory data is restored based on the forward progress information.

11. An audio / video data editing device, characterized in that, Applied to a terminal, the device includes: The data acquisition module is used to acquire the original audio and video data to be edited, wherein the original audio and video data contains at least two track data, and at least one track data contains at least one segment data; The information storage module is used to store initial state information in the operation queue based on the original audio and video data. The initial state information includes track attribute information of all track data and segment attribute information of all segment data. The track attribute information includes at least track identifier, track time point, and track unified resource locator. The information storage module is further configured to, in response to each editing operation on target track data in at least one of the track data, store editing status information corresponding to each editing operation in the operation queue according to the operation type of each editing operation and the track information of the target track data, and in the order of each editing operation; the editing status information is the editing status information corresponding to the target track data, and when the editing operation is a deletion operation, the editing status information includes the track identifier, track type, and segment identifier of the target segment data corresponding to the editing operation.

12. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the steps of the method described in any one of claims 1-10.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-10.

Images