Audio and video editing method and device, storage medium and electronic equipment

Abstract

The invention discloses an audio and video editing method and device, a storage medium and electronic equipment. The method comprises the steps that the track type of a target media file is determined according to the file type of the added target media file, and the track type comprises a subtitle track, a video track and an audio track; the rendering number of the target media file and the track position of the target media file on a timeline are determined according to the track type, the rendering number is used for indicating the rendering sequence of the target media file, and the timeline is used for carrying out layered placement on the media files of different track types; and displaying the target media file on the timeline according to the track position, and rendering the reference media files and the target media file on the timeline according to the rendering numbers corresponding to the plurality of reference media files and the target media file on the timeline. The technical problem that a nonlinear audio and video editing system provided by the related technology is low in complex project operation efficiency is solved.

Description

Technical Field

[0001] This application relates to the technical field of audio - video editing. Specifically, it relates to an audio - video editing method, device, storage medium, and electronic device. Background Art

[0002] In a non - linear audio - video editing system, the timeline is the core of the editing workflow, which allows editors to layout various media materials along the timeline for precise cutting, stacking, and synthesis.

[0003] However, in complex editing projects, editors may need to process media materials on multiple tracks, and the materials on each track may require specific rendering parameters and stacking orders. If the track management is improper, even a minor change in the track position or a slight adjustment of the media material may cause deviation in the rendering effect of the final output audio - video. That is, the non - linear audio - video editing system provided by the related technology has the problem of low operation efficiency in complex projects. At present, no effective solution has been proposed for the above problems. Summary of the Invention

[0004] Embodiments of this application provide an audio - video editing method, device, storage medium, and electronic device to at least solve the technical problem of low operation efficiency in complex projects of the non - linear audio - video editing system provided by the related technology.

[0005] According to one aspect of the embodiments of this application, an audio - video editing method is provided, including: determining the track type of a target media file according to the file type of the added target media file, where the track type includes a subtitle track, a video track, and an audio track; determining the rendering number of the target media file and the track position of the target media file on the timeline according to the track type, where the rendering number is used to indicate the rendering order of the target media file, and the timeline is used to place media files of different track types in layers; displaying the target media file on the timeline according to the track position, and rendering the reference media files and the target media file on the timeline according to the respective rendering numbers of the multiple reference media files and the target media file on the timeline.

[0006] According to another aspect of the embodiments of the present application, there is also provided an audio-visual editing device, including: a track type determination unit, configured to determine the track type of a target media file according to the file type of the added target media file, where the track type includes a subtitle track, a video track, and an audio track; a track addition unit, configured to determine the rendering number of the target media file and the track position of the target media file on the timeline according to the track type, where the rendering number is used to indicate the rendering order of the target media file, and the timeline is used to hierarchically place media files of different track types; a rendering unit, configured to display the target media file on the timeline according to the track position, and render the reference media files and the target media file on the timeline according to the respective rendering numbers of the multiple reference media files and the target media file on the timeline.

[0007] Optionally, the above track addition unit includes: a rendering number determination module, configured to determine the rendering number of the target media file according to the track type; an index determination module, configured to determine the rendering number of the target media file as the index number in the track array corresponding to the target media file on the timeline, where the track array is used to manage the track data corresponding to the timeline; a track position determination module, configured to determine its track position in the track array corresponding to the timeline according to the track type.

[0008] Optionally, the above rendering number determination module is further configured to: in the case where the track type is a subtitle track, determine the total number of tracks corresponding to the current timeline as the rendering number of the target media file; in the case where the track type is a video track, determine the total number of tracks corresponding to the current timeline minus the number of media files of the subtitle track type as the rendering number of the target media file; in the case where the track type is an audio track, determine the rendering number of the target media file as a preset number.

[0009] Optionally, after determining the rendering number of the target media file according to the track type, it includes: in the case where the track type is a video track, incrementing by one the rendering number corresponding to the media file of the subtitle track type on the current timeline; in the case where the track type is an audio track, incrementing by one the rendering numbers corresponding to all media files on the current timeline.

[0010] Optionally, the above-mentioned track position determination module is further configured to: when the track type is a subtitle track, determine that the track position of the target media file is under the functional track of the current timeline, and determine that the position number of the target media file in the track array corresponding to the timeline is 1; when the track type is a video track, determine that the track position of the target media file is under the subtitle track of the current timeline, and determine that the position number of the target media file in the track array corresponding to the timeline is the sum of the numbers of the functional track and the subtitle track; when the track type is an audio track, determine that the track position of the target media file is under all tracks of the current timeline, and determine that the position number of the target media file in the track array corresponding to the timeline is the sum of the numbers of all tracks.

[0011] Optionally, the above-mentioned track type determination unit includes: a subtitle track recognition module, configured to add the target media file to the functional track of the timeline and obtain the file type of the target media file; when the file type of the target media file is the first file type and the second file type, determine that the track type is a subtitle track, where the first file type is used to store time stamps and text information, and the second file type is used to store time stamps, text information, and text styles; an audio track recognition module, configured to parse the resource stream in the target media file, and when only an audio stream is included in the resource stream, determine that the track type is an audio track; a video track recognition module, configured to determine that the track type is a video track when the track type is neither a subtitle track nor an audio track.

[0012] Optionally, the above-mentioned audio and video editing device further includes: a track swapping unit, configured to respond to a movement instruction on the timeline and swap the positions of the first track and the second track on the timeline, where the movement instruction is used to move the positions of the first track and the second track, and the first track and the second track are tracks of the same track type; in the track array corresponding to the timeline, swap the position numbers and index numbers corresponding to the first track and the second track; determine the rendering numbers of the first track and the second track as the updated index numbers.

[0013] According to another aspect of the embodiments of the present application, there is also provided a computer-readable storage medium, in which a computer program is stored, and the computer program is configured to execute the above-mentioned audio and video editing method when running.

[0014] According to another aspect of the embodiments of the present application, there is provided a computer program product, which includes computer programs / instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer programs / instructions from the computer-readable storage medium, and the processor executes the computer programs / instructions, so that the computer device executes the audio and video editing method as described above.

[0015] According to another aspect of the embodiments of the present application, an electronic device is further provided, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to execute the above audio and video editing method through the computer program.

[0016] In the embodiments of the present application, the track type of the target media file is determined according to the file type of the added target media file, where the track type includes a subtitle track, a video track, and an audio track; the rendering number of the target media file and the track position of the target media file on the timeline are determined according to the track type of the target media file, where the rendering number is used to indicate the rendering order of the target media file, and the timeline is used to layer and place media files of different track types; the target media file is displayed on the timeline according to the track position, and the reference media files and the target media file on the timeline are rendered according to the respective rendering numbers of the multiple reference media files and the target media file on the timeline. When the user adds different media files to the timeline of the editing system, the present application can automatically identify and classify the types of media files, place them on the corresponding business tracks, without manually selecting or creating specific types of tracks. And the hierarchical management of the tracks helps to clearly organize different types of media materials, determine the correct stacking between the tracks, and by accurately defining the rendering numbers, convert the track operations into operations on arrays, simplify the complexity of track management, and improve the efficiency and accuracy of the user when operating the timeline. Thus, the above method is used to solve the technical problem that the non-linear audio and video editing system provided by the related technology has low efficiency in complex project operations. Description of the Drawings

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation to the present application. In the drawings:

[0018] Figure 1 is a schematic diagram of the hardware environment of an optional audio and video editing method according to the embodiments of the present application;

[0019] Figure 2 is a flowchart of an optional audio and video editing method according to the embodiments of the present application; [[ID=IP19]]

[0020] Figure 3 is a schematic diagram of an optional audio and video editing method according to the embodiments of the present application;

[0021] Figure 4 is a schematic diagram of another optional audio and video editing method according to the embodiments of the present application;

[0022] Figure 5It is a schematic diagram of another optional audio - video editing method according to an embodiment of the present application;

[0023] Figure 6 It is a schematic diagram of another optional audio - video editing method according to an embodiment of the present application;

[0024] Figure 7 It is a schematic diagram of another optional audio - video editing method according to an embodiment of the present application;

[0025] Figure 8 It is a schematic diagram of another optional audio - video editing method according to an embodiment of the present application;

[0026] Figure 9 It is a schematic diagram of an optional audio - video editing device according to an embodiment of the present application;

[0027] Figure 10 It is a schematic diagram of an optional electronic device according to an embodiment of the present application. Detailed implementation manners

[0028] In order to enable those skilled in the art to better understand the solution of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above - mentioned drawings are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that such used data can be interchanged under appropriate circumstances so that the embodiments of the present application described herein can be implemented in an order different from those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non - exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units does not necessarily need to be limited to those clearly listed steps or units, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0030] According to one aspect of the embodiments of the present application, an audio - video editing method is provided. As an optional implementation manner, the above - mentioned audio - video editing method can be but is not limited to being applied to, for example Figure 1An audio-video editing system in the hardware environment shown. Among them, the audio-video editing system may include, but is not limited to, a terminal device 102, a network 110, a server 112, and a database 114. The terminal device 102 runs a target program (such as Figure 1 shown, taking the target program as an example of a program with non-linear audio-video editing functions). The above terminal device 102 includes a display 108, a processor 106, and a memory 104. The display 108 can be used to display different media tracks, etc., and is also used to provide a human-computer interaction interface to receive touch-based human-computer interaction operations on different information. The processor is used to generate interaction instructions in response to the above human-computer interaction operations and send the interaction instructions to the server to obtain the rendering numbers of each track on the current timeline, etc. The memory is used to store data such as the rendering numbers updated by the running client and the track position information.

[0031] In addition, the server 112 includes a processing engine 116, and the processing engine 116 is used to perform storage or reading operations on the database 114. Specifically, the processing engine 116 reads the rendering numbers and track positions corresponding to the default media files from the database 114.

[0032] Suppose Figure 1 a program for editing audio-video files is running in the terminal device 102 in . The specific process of this embodiment is as follows: As in step S102, the terminal device 102 can obtain the track type of the added target media file. The terminal device 102 may be one or more devices. Then step S104 is executed to determine the rendering number and track position of the target media file. Before S104, the rendering numbers and track positions of the existing tracks corresponding to the current timeline can be obtained from the server 112 through the network 110. After determining the rendering number and track position of the target media file, step S106-1 can be executed to render the media files including the target media file on the timeline, or step S106-2 can be executed asynchronously to send the rendering number and track position of the target media file to the server 112 so that in the case where other users edit the same project, the track information can be updated in a timely manner.

[0033] Optionally, in this embodiment, the above terminal device 102 may be a terminal device configured with a target program, and may include, but is not limited to, at least one of the following: mobile phone (such as Android mobile phone, iOS mobile phone, etc.), laptop computer, tablet computer, handheld computer, MID (Mobile Internet Devices), PAD, desktop computer, smart TV, etc. The client may be a video client, a browser client, or other clients that support providing audio and video editing tasks or can implement the rendering of media files. The above network may include, but is not limited to: wired network, wireless network, where the wired network includes: local area network, metropolitan area network, and wide area network, and the wireless network includes: Bluetooth, WIFI, and other networks that implement wireless communication. The above server may be a single server, or a server cluster composed of multiple servers, or a cloud server. The above is only an example, and this embodiment does not make any limitation thereto.

[0034] Optionally, in this embodiment, the above audio and video editing timeline track hierarchical management method may be, but is not limited to, applied in a video editing client. Especially when the user performs complex editing operations, such as adding, moving, deleting, or adjusting the positions of different types of media content on the timeline. When the system monitors the user's interaction operations, such as adding a new segment or adjusting the track order through a drag-and-drop operation, the system can dynamically update the classification and rendering order of the tracks according to the operation information (such as the media type of the operation target, the current state of the timeline, etc.). In the client, this mechanism can calculate and adjust the rendering number and track position of each segment on the timeline in real time to ensure the correct stacking and synthesis of all media content. The system can periodically update the visual presentation of the timeline to reflect the latest editing state and segment position changes, which is particularly important when the user frequently performs track operations, and can ensure the fluency and real-time responsiveness of the editing interface. For example, when adding a subtitle segment, the system automatically recognizes that the file type is SRT or ASS, and according to the predefined track hierarchical rules, places the new segment on the subtitle track layer of the timeline and updates its rendering number to ensure that the subtitle is correctly displayed above the video content. When moving a video segment, the system will automatically adjust the rendering order of all affected tracks to maintain the consistency of the timeline and the logical relationship between segments. During the operation, the system can also highlight the track or segment that the user is currently operating on, enhancing the user's spatial perception and operation accuracy through visual feedback. This dynamic and intuitive track management and segment rendering mechanism not only improves the editing efficiency but also enhances the user experience, making audio and video editing more efficient, intuitive, and user-friendly, and providing the user with an efficient and intelligent audio and video editing environment.

[0035] In the embodiments of the present application, the track type of the target media file is determined according to the file type of the added target media file, where the track types include subtitle tracks, video tracks, and audio tracks; the rendering number of the target media file and the track position of the target media file on the timeline are determined according to the track type of the target media file, where the rendering number is used to indicate the rendering order of the target media file, and the timeline is used to hierarchically place media files of different track types; the target media file is displayed on the timeline according to the track position, and the reference media files and the target media file on the timeline are rendered according to the respective rendering numbers of the multiple reference media files and the target media file on the timeline. When the user adds different media files to the timeline of the editing system, the present application can automatically identify and classify the types of media files, place them on the corresponding business tracks, without manually selecting or creating specific types of tracks. And hierarchical management of the tracks helps to clearly organize different types of media materials, determine the correct stacking between the tracks, and by accurately defining the rendering numbers, convert the track operations into operations on arrays, simplify the complexity of track management, and improve the efficiency and accuracy of the user when operating the timeline. Thus, the above method solves the technical problem that the non-linear audio and video editing system provided by the related art has low efficiency in complex project operations.

[0036] As an alternative embodiment, as Figure 2 shown, the above audio and video editing method includes the following steps:

[0037] S202, determining the track type of the target media file according to the file type of the added target media file, where the track types include subtitle tracks, video tracks, and audio tracks;

[0038] It should be noted that the target media file refers to the audio and video material that the editor hopes to add or import into the non-linear editing system. It can be a video file, an audio file, or a subtitle file. The file type refers to the specific format of the target media file, such as MP4 (video), MP3 (audio), SRT or ASS (subtitle), etc. The file type determines whether the media data contained in the file is video, audio, or subtitle.

[0039] In a non-linear editing system, the timeline consists of different tracks. The track type refers to the classification of these tracks based on the media type they will carry, mainly divided into subtitle tracks, video tracks, and audio tracks. The subtitle track is specifically used to store and edit subtitle content, usually for adding subtitle files in formats such as SRT and ASS to achieve the display of synchronized subtitles. The video track is used to store and edit video content, and can process various video files, including composite media files with embedded audio and subtitles. The audio track is specifically used to store and edit audio content, can process various audio files, and can also extract audio data from composite media files separately for editing.

[0040] In an alternative embodiment, the editing system needs to identify the file type of the target media file, which is usually done by the file extension or content recognition. Once the file type is determined, the system automatically decides which track on the timeline the file should be placed on according to the file type. For example, if the file type is SRT or ASS, it is classified as a subtitle track; if it is a video format such as MP4 or AVI, it is classified as a video track; if it is an audio format such as MP3 or WAV, it is classified as an audio track. The timeline in the non-linear editing system is designed with dedicated tracks to process different types of media files, ensuring the efficiency and accuracy of the editing operations.

[0041] S204, determine the rendering number of the target media file and the track position of the target media file on the timeline according to the track type, where the rendering number is used to indicate the rendering order of the target media file, and the timeline is used to layer and place media files of different track types;

[0042] It should be noted that the rendering number is a numerical identifier that determines the rendering order of media files in the final audio-visual synthesis output. A lower rendering number means that the media file will be rendered later or be at the bottom layer in the layered display. The timeline is a visual tool in the non-linear editing system used to organize and edit audio-visual materials. It arranges media segments in chronological order, allowing users to edit, mix, and synthesize. Layered placement means that on the timeline, media files are placed in specific layers according to their track types. For example, subtitles are placed above the video, and audio is placed below the video and subtitles to ensure the correct visual and auditory effects.

[0043] In an alternative embodiment, when the user drags a media file onto the timeline, the system automatically identifies the type of the file (video, audio, subtitle), and based on this information, determines the rendering number of the file and its placement on the timeline. The rendering number is a crucial attribute that determines the order of the media files in the final output, ensuring that the video, audio, and subtitles are correctly synthesized according to the preset priorities and effects. The design of the timeline allows different types of media files to be placed on different layers, and this hierarchical management can clearly distinguish the audio-visual and subtitle materials, simplify the editing process, and guarantee the quality of the final work.

[0044] Figure 3 is a schematic diagram of an alternative audio-visual editing method according to an embodiment of the present application; as Figure 3 shown, in a non-linear audio-visual editing system, the editing of media files and tracks can be operated on the timeline 308 interface. In the timeline 308 area, multiple media files can be managed, and they can be placed in layers according to the type of the track and the preset hierarchical method. For example, the subtitle track 302, the video track 304, and the audio track 306 are arranged in sequence.

[0045] In an alternative embodiment, in a non-linear audio-visual editing system, the timeline is the core editing environment, and it manages the audio-visual and subtitle materials through track layering. The system automatically calculates the rendering number of the media file according to the type of the track, and then determines its placement on the timeline. The setting of the rendering number ensures that the media files are rendered in the preset order, and the hierarchical placement allows the editor to process the video, audio, and subtitles in an intuitive way, keeping the structure of the audio-visual work clear and the continuity of the playback effect.

[0046] In an alternative embodiment, assume that in the Chuangying video editing client software, the user wants to add a background music to the video project being edited. The user adds the audio file (the target media file) to the timeline through a drag-and-drop operation. The system first identifies the file as an audio type, and then calculates the rendering number and track position of the audio file according to the current state of the timeline (such as the number and type of existing tracks). Since the audio track is usually placed under the video and subtitle tracks (i.e., the bottom layer of the stacking order), the system will find a suitable index position in the audio track level of the rendering list to insert the new audio track, and update the track position value accordingly to ensure that the rendering order of other tracks is not affected. Finally, the newly added audio file will be correctly synthesized in the final output according to the calculated rendering number, and the user can preview this effect in real time on the editing interface to ensure that the audio-visual synchronization and subtitle display of the work meet the expectations.

[0047] S206. Display the target media file on the timeline according to the track position, and render the reference media files and the target media file on the timeline according to the respective rendering numbers of the multiple reference media files and the target media file on the timeline.

[0048] It should be noted that when the user adds a target media file (such as a video clip, an audio file, or a subtitle) to the timeline, the system will place it at a specific position on the timeline according to the track type of the media (S, V, A represent subtitle, video, and audio tracks).

[0049] To prepare for rendering, the system will read the rendering numbers of all media files on the timeline (including the target media file and the existing reference media files). The rendering number determines the order in which the media files are stacked (overlaid) in the final output video. A lower rendering number means that the media file will be rendered on a lower layer, that is, below or in the background in the final video.

[0050] In an optional implementation, the non-linear audio-visual editing system mixes all audio files according to the order of the rendering numbers of the audio tracks to ensure that audio content such as background music, sound effects, and dialogues are correctly merged according to the preset priorities and volume levels. The files on the video track are stacked from the bottom layer to the top layer according to their rendering numbers. This means that video clips will be overlaid in the specified order. For example, video effects, stickers, or templates may be placed on top of the main video to form a visual hierarchical effect. The files on the subtitle track are placed on top of the video track to ensure that the subtitle content is clearly visible and not blocked by the video content. The rendering order of the subtitles also follows their rendering numbers to ensure the correct display hierarchy. Finally, the system will comprehensively render the media content (audio, video, subtitles) on all tracks according to the order of their rendering numbers to generate a complete audio-visual output. This process may also include post-production steps such as color correction, audio equalization, and subtitle style adjustment to optimize the audio-visual effect of the final work.

[0051] In an embodiment of the present application, the track type of the target media file is determined according to the file type of the added target media file, where the track types include subtitle tracks, video tracks, and audio tracks; the rendering number of the target media file and the track position of the target media file on the timeline are determined according to the track type of the target media file, where the rendering number is used to indicate the rendering order of the target media file, and the timeline is used to hierarchically place media files of different track types; the target media file is displayed on the timeline according to the track position, and the reference media files and the target media file on the timeline are rendered according to the respective rendering numbers of the multiple reference media files and the target media file on the timeline. When the user adds different media files to the timeline of the editing system, the present application can automatically identify and classify the types of media files, place them on the corresponding business tracks, without manually selecting or creating specific types of tracks. And hierarchical management of the tracks helps to clearly organize different types of media materials, determine the correct stacking between the tracks, and by accurately defining the rendering numbers, transform the track operations into operations on arrays, simplify the complexity of track management, and improve the efficiency and accuracy of the user when operating the timeline. Thus, the technical problem of the low efficiency of complex project operations existing in the non-linear audio and video editing system provided by the related art is solved by the above method.

[0052] In an alternative embodiment, determining the track type of the target media file according to the file type of the added target media file includes: adding the target media file to the function track of the timeline, and obtaining the file type of the target media file; in the case where the file type of the target media file is the first file type and the second file type, determining that the track type is a subtitle track, where the first file type is used to store timestamps and text information, and the second file type is used to store timestamps, text information, and text styles; parsing the resource stream in the target media file, and in the case where only an audio stream is included in the resource stream, determining that the track type is an audio track; in the case where the track type is neither a subtitle track nor an audio track, determining that the track type is a video track.

[0053] Figure 4 is a schematic diagram of another alternative audio and video editing method according to an embodiment of the present application; as Figure 4 shown, in addition to including a subtitle track 302, a video track 304, and an audio track 306 in the timeline 308, a function track 402 is also displayed at the top layer of the tracks displayed in the timeline 308 (i.e., above the subtitle track 304). The subtitle track, video track, and audio track can be collectively referred to as business tracks, that is, they can have an effect on the finally generated media file. The function track is responsible for automatically classifying the dragged-in media files.

[0054] It should be noted that the function track is a special track in the audio-video editing software used to automatically identify and classify media files added by users. It is located at the topmost layer of the timeline and serves as the "switch" or "entry point" for all other tracks (such as subtitle tracks, video tracks, and audio tracks). When a user drags a media file to the timeline, the system first places the file in the function track and then automatically classifies it into the corresponding business tracks (subtitle, video, audio) according to the type of the file.

[0055] The first file type can be the SRT type. SRT (SubRip Subtitle) is a simple subtitle file format mainly used to store timestamp and text information. Each line of subtitle in the SRT file contains the start and end times, as well as the corresponding text content. The system can easily parse this format, automatically identify the file type, and place it on the subtitle track of the timeline to achieve synchronous display of subtitles and video. The second file type can be the ASS type. ASS (Advanced SubStation Alpha) is a more powerful subtitle file format that can not only store timestamp and text information but also define the styles of the text, such as font, size, color, and position, etc. The parsing and processing of ASS files are more complex than those of SRT, but it provides more advanced subtitle editing capabilities. When the system recognizes a file in the ASS format, it will also classify the file into the subtitle track and at the same time allow users to edit more subtitle attributes, such as dynamic effects.

[0056] An audio stream refers to the audio data part contained in a media file, which can be background music, dialogue, narration, or sound effects, etc. In audio-video editing, the system will detect the resource streams in the media file. If the file only contains an audio stream and no video stream, then the system will determine its track type as an audio track and place it on the audio track layer of the timeline to ensure the correct mixing and output of the audio content.

[0057] The video track is the track used to place and edit video clips, which can cover various video contents, such as the main video, video templates, special effect animations, etc. When the track type of the target media file is neither a subtitle track nor an audio track, the system classifies it as a video track. The video track is located in the middle layer of the timeline, usually below the subtitle track and above the audio track, to ensure that the video content is the main presentation, while the subtitles and audio are correctly superimposed or mixed according to their track types and rendering numbers.

[0058] In an alternative embodiment, when the user drags a media file onto the timeline, the system first captures the file and then checks its extension or determines the file type (SRT, ASS, video, or audio format) by the file content. For SRT and ASS files, the system classifies them as subtitle tracks; for files containing only an audio stream, the system classifies them as audio tracks; and for other types, the system classifies them as video tracks. During this process, for video and audio files, the system further parses the stream information in the file to check whether it contains an audio stream, a video stream, or both. If the file contains only an audio stream, then even if it is a video file, the system will classify it into the audio track. The system maintains a timeline array to manage the tracks on the timeline. According to the classification of the target media file (subtitle, audio, video), the system inserts a new track into the corresponding business track layer. Update the rendering to reflect the correct rendering position of the target media file on the timeline, and at the same time, it may be necessary to adjust the rendering numbers of other tracks to maintain the coherence of the rendering order.

[0059] In the interface of the audio-visual editing system, the system will update the display of the timeline in real time, place the media file on the corresponding track, and highlight the newly added file to help the user intuitively understand the classification and position of the media file. Operation guides can also be provided in the system, such as the preview effect during drag-and-drop or the description of track classification, to enhance the user experience.

[0060] Through the above embodiments of the present application, the functional track can automatically and intelligently classify and manage the media files added by the user, which not only simplifies the editing process but also ensures that the audio-visual editing software can handle various media formats and content types, improving the flexibility and efficiency of editing.

[0061] In an alternative embodiment, for the tracks in the audio-visual editing system, their corresponding track arrays and rendering arrays can be defined to manage the tracks in an array manner. The track array can be m_trackList, and the rendering array can be m_tractor.

[0062] It should be noted that the track array m_trackList is an array used to store and manage the basic information of all tracks on the timeline. Each row of data represents a track and can contain the following fields: index: the index position (position number) of the track in the m_trackList array. type: the type of the track, represented by an initialism, such as F for the functional track, S for the subtitle track, V for the video track, and A for the audio track. number: the serial number of the track, indicating the order among the tracks of the same type. mgrt_index: the index number indicating the rendering priority of the track in the final audio-visual synthesis, and the smaller the value, the lower the rendering level.

[0063] In an alternative embodiment, the index number mgrt_index can represent the Multimedia Group Track Index. The present application does not limit the specific technical means. For example, the MLT framework can be used to edit and manage multimedia tracks. Components such as mlt_multitrack and mlt_playlist in MLT can be used to manage multiple tracks and the materials on the tracks. It can also be the widely used multimedia processing library FFmpeg that supports operations such as decoding, encoding, and transcoding of audio and video. Although FFmpeg itself does not directly provide multi-track editing functions, similar functions can be implemented through scripts or custom code. The modular multimedia processing framework GStreamer can also be used to support complex audio and video processing pipelines and can also be used to implement multi-track management and editing. Of course, if the existing framework cannot meet the requirements, custom development can be carried out based on MLT or other frameworks to implement specific multimedia track management functions. Here, it is only necessary to be able to manage audio and video tracks. The specific technical means and the numbering of the arrays are not limited. For example, in the case of using the MLT framework, mlt_index can be used to replace mgrt_index to represent the index number.

[0064] For the rendering level, in audio and video editing software, tracks or media files with a lower rendering level are rendered before those with a higher rendering level. This follows a basic principle in computer graphics and video synthesis, namely the "cascading" principle. During rendering, the system processes the tracks or segments in ascending order of their rendering levels (represented by the mgrt_index value). A lower mgrt_index value means a lower level, and such media content will be rendered first, while the higher levels are superimposed and rendered on top of the lower levels to form the final audio and video output.

[0065] Specifically regarding the rendering levels of subtitle, video, and audio tracks: Audio tracks usually have a lower rendering level because the synthesis of audio does not directly affect the visual effect but exists as a background or an audio track synchronized with the video. In most cases, the rendering of audio is independent of the video and subtitles and is carried out first to ensure the integrity and quality of the audio track. The rendering level of the video track is above that of the audio but usually below that of the subtitles. Video clips are the basis of audiovisual works. They are processed after the audio rendering is completed to form a sequence of video frames. The rendering of the video track is the main component of the visual content and needs to be completed before the subtitle overlay to ensure that the subtitles cover the video content. The subtitle track has a higher rendering level and is above the video track. Subtitles are provided to offer additional information such as dialogue, annotations, or translations. They are designed to cover the video content, so they must be overlaid after the video rendering is completed to ensure that the subtitles are clearly visible and not blocked by other elements in the video.

[0066] The rendering array m_tractor is a list of indices that records the specific names and rendering numbers of each track on the timeline. This list is directly associated with the mgrt_index field (index number) in m_trackList. Through the rendering numbers in m_tractor, the system can find the specific track names, facilitating rendering.

[0067] Figure 5 It is a schematic diagram of another optional audio-visual editing method according to an embodiment of the present application; as Figure 5 shown, the track array 502 (m_trackList) shows the following track structure: The functional track (F) is located at the position with index 0, and the mgrt_index (index number) is also 0, indicating that it is the topmost track and is used for classifying and identifying the added media files. There are two instances of the subtitle track (S), with numbers 1 and 0 respectively, located at the positions with mgrt_index values of 6 and 5, indicating that they are displayed in the final output in the rendering order of 6 and 5. There are two instances of the video track (V), with numbers 1 and 0 respectively, located at the positions with mgrt_index values of 4 and 3. There are two instances of the audio track (A), with numbers 0 and 1 respectively, located at the positions with mgrt_index values of 2 and 1. Through the track array m_trackList, the system can quickly locate and manage each track, including its type, serial number, and rendering number. These information are crucial for material classification and rendering order control during audio-visual editing.

[0068] The name corresponding to rendering number 0 in the rendering array 504 (m_tractor) is empty, indicating that this is the position of the functional track for classification rather than specific editing. Rendering numbers 1 to 6 correspond to the names of audio tracks A2, A1, video tracks V1, V2, and subtitle tracks S1, S2 respectively.

[0069] The combined use of the two arrays can effectively manage the tracks on the timeline, ensuring clear classification of tracks and accurate rendering order during audio-visual editing, thereby improving editing efficiency and work quality.

[0070] In an alternative embodiment, determining the rendering number of the target media file and the track position of the target media file on the timeline according to the track type includes: determining the rendering number of the target media file according to the track type; determining the rendering number of the target media file as the index number in the track array corresponding to the target media file on the timeline, where the track array is used to manage the track data corresponding to the timeline; determining its track position in the track array corresponding to the timeline according to the track type.

[0071] It should be noted that the track array can be the m_trackList described above, a data structure for managing all tracks on the timeline. Each track has a corresponding index position in the array, and this index also reflects its rendering order in audio-visual synthesis.

[0072] When the system obtains a new target media file, it will assign a specific rendering number according to whether the file is a video, audio, or subtitle. This number determines its display order in the final work. The rendering number can be the index data in the rendering array and can correspond to the index number in the m_trackList array, so that the track array not only identifies the rendering order of the media file but also reflects its position on the timeline. Then, the target media file is placed in the correct position on the timeline according to its type (such as video, audio, or subtitle), and the information in m_trackList is updated simultaneously to ensure that the rendering numbers and positions of all tracks are accurate.

[0073] In an alternative embodiment, assume that the user is editing a video project and adds a new video clip to the timeline. The system first identifies this media file as a video file and determines its track type as a video track (V). Since the video tracks in m_trackList are managed according to mgrt_index, if there are already two video tracks and one audio track in the current m_trackList, and their mgrt_index values are 3, 4, and 2 respectively. Therefore, the rendering number (mgrt_index) of the newly added video clip will be determined to be larger than the mgrt_index value of the current video tracks, for example, 5.

[0074] Next, the system will find an appropriate position on the timeline to insert this new video track. Considering that the video track should be above the audio track and below the subtitle track, the system will place the new video track at the position in m_trackList where mgrt_index is 5, that is, after the current video track and above the audio track. At the same time, due to the addition of the new video track, the system will automatically adjust the mgrt_index values of all subsequent tracks in m_trackList to maintain the coherence of the rendering order. For example, the video track with an original mgrt_index of 5 will now be adjusted to 6 to ensure the correct rendering order of all tracks.

[0075] Through the above implementation manner of the present application, the newly added video segment is not only correctly displayed and positioned on the timeline, but also correctly stacked during audio-visual synthesis according to its rendering number, ensuring that the video segment is correctly superimposed on other video and audio contents as the expected visual content, and at the same time reserving the correct stacking position for the subtitle track that may be added later. The editing process and the quality of the work are optimized.

[0076] In an alternative implementation manner, determining the rendering number of the target media file according to the track type includes: in the case where the track type is a subtitle track, determining the total number of tracks corresponding to the current timeline as the rendering number of the target media file; in the case where the track type is a video track, determining the total number of tracks corresponding to the current timeline minus the number of media files of the subtitle track type as the rendering number of the target media file; in the case where the track type is an audio track, determining the rendering number of the target media file as a preset number.

[0077] It should be noted that the rendering numbers are different for media files of different track types. In the case where the track type of the target media file is a subtitle track, determining the total number of tracks corresponding to the current timeline as the rendering number of the target media file means that the newly added subtitle track will be placed at the top layer of all tracks on the timeline for rendering, ensuring that the subtitle content is clearly visible above the video and audio.

[0078] In the case where the track type of the target media file is a video track, determining the total number of tracks corresponding to the current timeline minus the number of media files of the subtitle track type as the rendering number of the target media file. The rendering number of the video track is based on the total number of tracks on the current timeline minus the number of all subtitle tracks. In this way, the video content will be arranged to be rendered below all subtitle tracks but usually above the audio track to ensure that the video is the main visual content.

[0079] When the track type of the target media file is an audio track, determine that the rendering number of the target media file is a preset number. The rendering number of the audio track is usually set to a relatively low fixed value to ensure that the audio is synthesized before the video and subtitle tracks, maintaining audio-video synchronization.

[0080] The rendering process of the audio is equivalent to mixing, and its rendering order has less impact on the final video result. Therefore, in order to more simply determine the rendering number of the newly added audio track, the preset number can be set to 1, so that the newly added audio track can be directly determined without identifying the categories, quantities, etc. of the remaining tracks on the current timeline.

[0081] Through the above implementation manners of the present application, the subtitle track always obtains the highest rendering number to ensure that the subtitles cover all visual content; the video track is below the subtitle track and above the audio track, and obtains the rendering number calculated according to the total number of tracks and the number of subtitle tracks; while the rendering number of the audio track is usually fixed to give priority to audio synthesis. Ensure that the media content on the timeline is rendered in the expected stacking order, thereby achieving high-quality synthesis of audio-video works.

[0082] In an alternative embodiment, after determining the rendering number of the target media file according to the track type, it includes: when the track type is a video track, increment the rendering number corresponding to the media file of the subtitle track type on the current timeline by one; when the track type is an audio track, increment the rendering number corresponding to all media files on the current timeline by one.

[0083] It should be noted that when a new file is added, the system will specify its rendering number according to whether the file is a video, audio, or subtitle to ensure that the content is correctly stacked by type. When the track type of the target media file added by the user is a video track, the system will automatically increase the rendering number of the subtitle track to prevent the new video content from being blocked by the subtitles and ensure that the video can be correctly displayed below the subtitles. When adding an audio file, the system will adjust the rendering numbers of all existing media files (including videos and subtitles) to ensure that the audio track is at the lowest layer and is rendered or mixed before all visual content, maintaining audio-video synchronization and clarity.

[0084] In an alternative embodiment, assume that there are the following tracks and their rendering numbers on the timeline: video track (mgrt_index = 3), subtitle track (mgrt_index = 4), audio track (mgrt_index = 1). When the user adds a new video clip to the timeline: The system identifies that the target media file type corresponds to the video track, and the rendering number of the new video clip is set to the position after the current video track (mgrt_index = 3), for example, 4.

[0085] The system automatically increments the rendering numbers of all subtitle tracks on the timeline, that is, updates the mgrt_index of the original subtitle track from 4 to 5, to ensure that the new video clip is rendered below the existing subtitle track, achieving the correct display of the video and subtitles. The rendering numbers (mgrt_index = 1) of all audio tracks remain unchanged because the addition of the video does not affect the stacking order of the audio.

[0086] If the user subsequently adds a media file whose track type is recognized as an audio track, its rendering number is set to a preset number, such as 1. The system automatically increments the rendering numbers of all other media files on the timeline, that is, the mgrt_index values of all video tracks and subtitle tracks will be incremented by 1, to ensure that the audio content is rendered first and maintain audio-video synchronization.

[0087] Through the above embodiments of the present application, when the user adds a media file to the timeline, the system assigns it a rendering number according to the file type, and then adjusts the rendering numbers of other media files on the timeline according to the type of the newly added file, ensuring the coherence of the editing operation and the visual effect and audio quality of the final work. This mechanism can simplify the editing process, avoid the user manually adjusting the display level of each clip, and at the same time ensure that the audio, video, and subtitle content are correctly rendered in the expected order.

[0088] In an alternative embodiment, determining the track position of a target media file in the corresponding track array on the timeline according to the track type includes: in the case where the track type is a subtitle track, determining that the track position of the target media file is below the functional track of the current timeline, and determining that the position number of the target media file in the corresponding track array on the timeline is 1; in the case where the track type is a video track, determining that the track position of the target media file is below the subtitle track of the current timeline, and determining that the position number of the target media file in the corresponding track array on the timeline is the sum of the numbers of the functional track and the subtitle track; in the case where the track type is an audio track, determining that the track position of the target media file is below all tracks of the current timeline, and determining that the position number of the target media file in the corresponding track array on the timeline is the sum of the numbers of all tracks.

[0089] It should be noted that if the added file is a subtitle track, it will be placed in the second layer among all tracks, immediately following the functional track, and its position number (index) in m_trackList will be set to 1. The video track is positioned below all subtitle tracks but above the audio track, and its position number in m_trackList will be determined according to the total number of the current functional track and subtitle tracks. The audio track is placed at the bottom layer of the timeline, below all other tracks, to ensure the integrity and continuity of the audio track, and its position number in m_trackList will be the total number of all existing tracks.

[0090] Figure 6 It is a schematic diagram of another optional audio - video editing method according to an embodiment of the present application; as Figure 6 shown, when the target media file is identified as a subtitle track through the function track, the changes in the track array 602 and the rendering array 604 corresponding to the timeline. Each element in m_trackor (rendering array) represents a track on the timeline and contains information such as the name of the track. When a new subtitle track is added, the system adds a new Playlist entry at the end of m_trackor, indicating that this new subtitle track will be rendered last. In Figure 6 , it can be seen that after adding a new subtitle track (S3), a new element is added at the end of m_trackor, that is, the new subtitle track is added to the last position, and the rendering number is 7.

[0091] The function track on the timeline is used for automatically classifying and managing tracks, and it is located at the top layer. Below the function track, business tracks such as subtitle, video, and audio tracks can be added. When a new subtitle track is added, the track will be located below the function track and immediately follow the existing subtitle tracks. mgrt_index is a value used to determine the rendering order of the track during the final synthesis, and it is set to the rendering number of the newly added subtitle track in m_trackor. In Figure 6 , the new subtitle track S3 is added below the function track, and its mgrt_index is set to 7, that is, the rendering number in m_trackor, ensuring that it is located after the function track and above all video and audio tracks during synthesis.

[0092] In an optional implementation, when adding a new subtitle track S3, the system first adds a Playlist for S3 at the end of the m_trackor list, and the new m_trackor is as Figure 6 shown.

[0093] Then, a new subtitle track is added to m_trackList, and its mgrt_index is set to the rendering number in m_trackor, that is, 7. Its position number (index) will be determined to be 1 to display the track on the timeline.

[0094] Figure 7 It is a schematic diagram of another optional audio - video editing method according to an embodiment of the present application; as Figure 7 shown, when the target media file is identified as a video track through the function track, the changes in the track array 702 and the rendering array 704 corresponding to the timeline.

[0095] In an alternative embodiment, the system needs to count the number of all subtitle tracks (S) and functional tracks on the timeline. In Figure 7 , the existing tracks are: 1 functional track (F) and 2 subtitle tracks (S), so the value of tmpTrackCount (the sum of the number of functional tracks and subtitle tracks) is 3. The system needs to know the total number of tracks in the rendering sequence, and then subtract the value of the number of media files of the subtitle track type from this total to determine the position in the rendering sequence where the new video track should be inserted. In Figure 7 , the total number of the rendering sequence is 7 (including 1 functional track, 2 subtitle tracks, 2 video tracks, and 2 audio tracks). Subtracting 2 gives new_mgrt_index = 5.

[0096] Next, the system inserts a new Playlist, i.e., a new video track, at the position of new_mgrt_index = 5 in the rendering sequence. In Figure 7 , the new video track is labeled as V3. To maintain the continuity of the rendering order, for all tracks at the position of new_mgrt_index and later, their corresponding mgrt_index values are incremented by 1 to make room for the newly added video track. In Figure 7 , the mgrt_index values of the original S1 and S2 are incremented from 5 and 6 to 6 and 7 respectively, while the audio tracks remain unchanged.

[0097] tmpTrackCount now equals the position number (new_track_index) of the newly inserted video track, which is 3. This indicates that the new video track V3 will be the 4th track on the timeline (since the functional track with position number 0 is included), after all subtitle tracks but before the audio tracks. Finally, the system inserts the new video track V3 at the position of new_track_index = 3 on the timeline, ensuring that it is correctly displayed in the editing interface and is synchronized with the rendering sequence.

[0098] Figure 8 is a schematic diagram of another alternative audio - video editing method according to an embodiment of the present application; as Figure 8 shown, in the case where the target media file is identified as an audio track through the functional track, the changes in the corresponding track array 802 and rendering array 804 of the timeline.

[0099] In Figure 8In [the described situation], it can be seen that before adding a new audio track (A3), there are already multiple tracks in the rendering sequence. The mgrt_index values of audio tracks A1 and A2 are 1 and 2 respectively. When adding a new audio track, the system will insert a new Playlist at the rendering position numbered 1, that is, before the existing lowest-level audio track (in this case, A2, mgrt_index = 1). Inserting a new Playlist means that this new audio track will be processed first during audio-visual synthesis.

[0100] mgrt_index is a numerical value used internally by the system to identify the position of a track in the rendering sequence, and it determines the stacking order of each track in the final synthesis. Since the new audio track is inserted at position 1, the mgrt_index values of all tracks whose original mgrt_index values are greater than or equal to 1 (including the original audio tracks A1 and A2, as well as video and subtitle tracks) need to be incremented by 1. This is to prevent track position conflicts and ensure that all tracks can be correctly stacked in the new order during rendering.

[0101] Since in the rendering sequence, the new audio track has been placed at the bottom layer, in the visual representation of the timeline, it needs to be added after all other tracks, that is, at the bottom or end of the timeline. Its position number can be set to 1.

[0102] Through the above implementation manner of the present application, when adding a new track, corresponding updates will be made in the m_trackor and m_trackList data structures to ensure that the new track is correctly displayed on the timeline as expected and follows the correct rendering order during the final synthesis output.

[0103] In an alternative implementation manner, after displaying the target media file on the timeline according to the track position, it includes: in response to a movement instruction on the timeline, swapping the positions of the first track and the second track on the timeline, where the movement instruction is used to move the positions of the first track and the second track, and the first track and the second track are tracks of the same track type; in the track array corresponding to the timeline, swapping the position numbers and index numbers corresponding to the first track and the second track; and determining the rendering numbers of the first track and the second track as the updated index numbers.

[0104] In an alternative implementation manner, when the user selects to swap the positions of two tracks of the same type (such as video tracks V1 and V2) through a movement instruction, the system will immediately adjust the positions of these two tracks on the timeline to reflect the user's operation. In the track array, the position numbers (index) corresponding to the first track and the second track can be swapped to correctly display the two tracks on the timeline.

[0105] And in the rendering array, swap the rendering numbers corresponding to the first track and the second track to ensure that after moving the track, the correct rendering order can be guaranteed.

[0106] Through the above embodiments of the present application, the track position is intuitively adjusted on the timeline, and the management information of the track, including the position number and the rendering number, is synchronously updated inside the system to ensure the efficiency and accuracy of audio-visual editing.

[0107] It should be noted that for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the present application is not limited by the described action sequence, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the present application.

[0108] According to another aspect of the embodiments of the present application, there is also provided an audio-visual editing device for implementing the above audio-visual editing method. As Figure 9 shown, the device includes:

[0109] A track type determination unit 902, configured to determine the track type of the target media file according to the file type of the added target media file, where the track type includes a subtitle track, a video track, and an audio track;

[0110] A track addition unit 904, configured to determine the rendering number of the target media file and the track position of the target media file on the timeline according to the track type, where the rendering number is used to indicate the rendering order of the target media file, and the timeline is used to hierarchically place media files of different track types;

[0111] A rendering unit 906, configured to display the target media file on the timeline according to the track position, and render the reference media file and the target media file on the timeline according to the respective rendering numbers of the multiple reference media files and the target media file on the timeline.

[0112] Optionally, the above track addition unit 904 includes: a rendering number determination module, configured to determine the rendering number of the target media file according to the track type; an index determination module, configured to determine the rendering number of the target media file as the index number in the track array corresponding to the target media file on the timeline, where the track array is used to manage the track data corresponding to the timeline; a track position determination module, configured to determine its track position in the track array corresponding to the timeline according to the track type.

[0113] Optionally, the above-mentioned rendering number determination module is further configured to: in the case where the track type is a subtitle track, determine the total number of tracks corresponding to the current timeline as the rendering number of the target media file; in the case where the track type is a video track, subtract the number of media files of the subtitle track type from the total number of tracks corresponding to the current timeline, and determine the result as the rendering number of the target media file; in the case where the track type is an audio track, determine the rendering number of the target media file as a preset number.

[0114] Optionally, after determining the rendering number of the target media file according to the track type, the following steps are included: in the case where the track type is a video track, increment the rendering number corresponding to the media file of the subtitle track type on the current timeline by one; in the case where the track type is an audio track, increment the rendering number corresponding to all media files on the current timeline by one.

[0115] Optionally, the above-mentioned track position determination module is further configured to: in the case where the track type is a subtitle track, determine that the track position of the target media file is under the functional track of the current timeline, and determine that the position number of the target media file in the track array corresponding to the timeline is 1; in the case where the track type is a video track, determine that the track position of the target media file is under the subtitle track of the current timeline, and determine that the position number of the target media file in the track array corresponding to the timeline is the sum of the number of functional tracks and subtitle tracks; in the case where the track type is an audio track, determine that the track position of the target media file is under all tracks of the current timeline, and determine that the position number of the target media file in the track array corresponding to the timeline is the sum of the number of all tracks.

[0116] Optionally, the above-mentioned track type determination unit 902 includes: a subtitle track recognition module, configured to add the target media file to the functional track of the timeline and obtain the file type of the target media file; in the case where the file type of the target media file is the first file type and the second file type, determine that the track type is a subtitle track, where the first file type is used to store timestamps and text information, and the second file type is used to store timestamps, text information, and text styles; an audio track recognition module, configured to parse the resource stream in the target media file, and in the case where only an audio stream is included in the resource stream, determine that the track type is an audio track; a video track recognition module, configured to determine that the track type is a video track in the case where the track type is neither a subtitle track nor an audio track.

[0117] Optionally, the above audio-visual editing device further includes: a track swapping unit, configured to, in response to a moving instruction on the timeline, swap the positions of a first track and a second track on the timeline, where the moving instruction is used to move the positions of the first track and the second track, and the first track and the second track are tracks of the same track type; in the track array corresponding to the timeline, swap the position numbers, index numbers corresponding to the first track and the second track; and determine the rendering numbers of the first track and the second track as the updated index numbers.

[0118] Optionally, in this embodiment, for the embodiments to be implemented by the above respective unit modules, reference may be made to the above respective method embodiments, which will not be elaborated herein.

[0119] According to another aspect of the embodiments of the present application, there is also provided an electronic device for implementing the above audio-visual editing method, and the electronic device may be Figure 10 the terminal device or server shown in the figure. This embodiment takes the electronic device as the terminal device as an example for illustration. As Figure 10 shown in the figure, the electronic device includes a memory 1002 and a processor 1004. A computer program is stored in the memory 1002, and the processor 1004 is configured to execute the steps in any of the above method embodiments through the computer program.

[0120] Optionally, in this embodiment, the above electronic device may be at least one network device among multiple network devices in a computer network.

[0121] Optionally, in this embodiment, the above processor may be configured to execute the following steps through the computer program:

[0122] S1. Determine the track type of the target media file according to the file type of the added target media file, where the track type includes a subtitle track, a video track, and an audio track;

[0123] S2. Determine the rendering number of the target media file and the track position of the target media file on the timeline according to the track type, where the rendering number is used to indicate the rendering order of the target media file, and the timeline is used to hierarchically place media files of different track types;

[0124] S3. Display the target media file on the timeline according to the track position, and render the reference media file and the target media file on the timeline according to the respective rendering numbers of the multiple reference media files and the target media file on the timeline.

[0125] Optionally, those of ordinary skill in the art can understand that Figure 10The structure shown is only schematic, and the electronic device or electronic equipment may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a personal digital assistant, and terminal devices such as Mobile Internet Devices (MID) and PAD. Figure 10 It does not limit the structure of the above-mentioned electronic device or electronic equipment. For example, the electronic device or electronic equipment may further include more or fewer components (such as a network interface, etc.) than those shown Figure 10 in the figure, or have a different configuration from that Figure 10 shown.

[0126] Among them, the memory 1002 can be used to store software programs and modules, such as the program instructions / modules corresponding to the audio-video editing method and device in the embodiments of the present application. The processor 1004 executes various functional applications and data processing by running the software programs and modules stored in the memory 1002, that is, implements the above-mentioned audio-video editing method. The memory 1002 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some instances, the memory 1002 may further include a memory remotely set relative to the processor 1004, and these remote memories can be connected to the terminal through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof. Among them, the memory 1002 may specifically but not limitedly be used to store media files. As an example, as Figure 10 shown, the above-mentioned memory 1002 may but not limitedly include the track type determination unit 902, the track addition unit 904, and the rendering unit 906 in the above-mentioned audio-video editing device. In addition, it may also include but not limited to other module units in the above-mentioned audio-video editing device, which will not be elaborated in this example.

[0127] Optionally, the above-mentioned transmission device 1006 is used to receive or send data via a network. Specific examples of the above network may include a wired network and a wireless network. In one instance, the transmission device 1006 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices and routers through a network cable so as to communicate with the Internet or a local area network. In one instance, the transmission device 1006 is a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0128] In addition, the above-mentioned electronic equipment further includes: a display 1008, which is used to display the audio-video editing system; and a connection bus 1010, which is used to connect each module component in the above-mentioned electronic equipment.

[0129] In other embodiments, the above-mentioned terminal device or server may be a node in a distributed system. Among them, the distributed system may be a blockchain system, and the blockchain system may be a distributed system formed by connecting the multiple nodes in a network communication manner. Among them, the nodes may form a peer-to-peer (P2P) network, and any form of computing device, such as electronic devices like servers and terminals, can become a node in the blockchain system by joining the peer-to-peer network.

[0130] According to one aspect of the present application, there is provided a computer program product, which includes computer programs / instructions, and the computer programs / instructions contain program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part, and / or installed from a removable medium. When the computer program is executed by a central processing unit, various functions provided by the embodiments of the present application are executed.

[0131] The serial numbers of the embodiments of the present application above are only for description and do not represent the advantages or disadvantages of the embodiments.

[0132] According to one aspect of the present application, there is provided a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the above-mentioned audio-visual editing method.

[0133] Optionally, in this embodiment, the above-mentioned computer-readable storage medium may be set to store a computer program for executing the following steps:

[0134] S1. Determine the track type of the target media file according to the file type of the added target media file, where the track type includes a subtitle track, a video track, and an audio track;

[0135] S2. Determine the rendering number of the target media file and the track position of the target media file on the timeline according to the track type, where the rendering number is used to indicate the rendering order of the target media file, and the timeline is used to hierarchically place media files of different track types;

[0136] S3. Display the target media file on the timeline according to the track position, and render the reference media files and the target media file on the timeline according to the respective rendering numbers of the multiple reference media files and the target media file on the timeline.

[0137] Optionally, in this embodiment, those of ordinary skill in the art may understand that all or part of the steps in the various methods of the above embodiments can be completed by a program instructing the relevant hardware of the terminal device. The program can be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disc, etc.

[0138] If the integrated unit in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in the above computer-readable storage medium. Based on such an understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in the storage medium and includes several instructions for causing one or more computer devices (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in the various embodiments of this application.

[0139] In the above embodiments of this application, the descriptions of the various embodiments each have their own emphases. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0140] In the several embodiments provided by this application, it should be understood that the disclosed client can be implemented in other ways. Among them, the device embodiments described above are only illustrative. For example, the above division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection can be through some interfaces. The indirect coupling or communication connection of units or modules can be in an electrical or other form.

[0141] The units described as separate components above may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place, or they can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0142] In addition, the functional units in the various embodiments of this application can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0143] The above are only the preferred embodiments of the present application. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present application, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present application.

Claims

1. An audio-video editing method, characterized in that, Including: Determine the track type of the target media file according to the file type of the added target media file, where the track type includes a subtitle track, a video track, and an audio track; Determine the rendering number of the target media file and the track position of the target media file on the timeline according to the track type, where the rendering number is used to indicate the rendering order of the target media file, and the timeline is used to hierarchically place media files of different track types; Display the target media file on the timeline according to the track position, and render the reference media files and the target media file on the timeline according to the rendering numbers corresponding to the multiple reference media files and the target media file on the timeline.

2. The method according to claim 1, characterized in that, The determining the rendering number of the target media file and the track position of the target media file on the timeline according to the track type includes: Determine the rendering number of the target media file according to the track type; Determine the rendering number of the target media file as the index number in the track array corresponding to the target media file on the timeline, where the track array is used to manage the track data corresponding to the timeline; Determine its track position in the track array corresponding to the timeline according to the track type.

3. The method according to claim 2, wherein The determining the rendering number of the target media file according to the track type includes: In the case where the track type is a subtitle track, determine the total number of tracks corresponding to the current timeline as the rendering number of the target media file; In the case where the track type is a video track, determine the total number of tracks corresponding to the current timeline minus the number of media files of the subtitle track type as the rendering number of the target media file; In the case where the track type is an audio track, determine the rendering number of the target media file as a preset number.

4. The method according to claim 3, characterized in that, After determining the rendering number of the target media file according to the track type, it includes: In the case where the track type is a video track, increment by one the rendering number corresponding to the media file of the subtitle track type on the current timeline; In the case where the track type is an audio track, increment by one the rendering numbers corresponding to all media files on the current timeline.

5. The method according to claim 2, characterized in that, The determining its track position in the track array corresponding to the timeline according to the track type includes: In the case where the track type is a subtitle track, determine that the track position of the target media file is under the functional track of the current timeline, and determine that the position number of the target media file in the track array corresponding to the timeline is 1; In the case where the track type is a video track, determine that the track position of the target media file is under the subtitle track of the current timeline, and determine that the position number of the target media file in the track array corresponding to the timeline is the sum of the number of the functional track and the subtitle track; When the track type is an audio track, determine that the track position of the target media file is below all tracks on the current timeline, and determine that the position number of the target media file in the track array corresponding to the timeline is the sum of the numbers of all tracks.

6. The method according to any one of claims 1 to 5, characterized in that, The determining the track type of the target media file according to the file type of the added target media file includes: Add the target media file to the functional track of the timeline, and obtain the file type of the target media file; When the file type of the target media file is a first file type and a second file type, determine that the track type is a subtitle track, where the first file type is used to store timestamps and text information, and the second file type is used to store timestamps, text information, and text styles; Parse the resource stream in the target media file, and when only an audio stream is included in the resource stream, determine that the track type is an audio track; When the track type is neither a subtitle track nor an audio track, determine that the track type is a video track.

7. The method according to any one of claims 1 to 5, characterized in that After displaying the target media file on the timeline according to the track position, it includes: In response to a movement instruction on the timeline, exchange the positions of a first track and a second track on the timeline, where the movement instruction is used to move the positions of the first track and the second track, and the first track and the second track are tracks of the same track type; In the track array corresponding to the timeline, exchange the position numbers and index numbers corresponding to the first track and the second track; Determine the rendering numbers of the first track and the second track as the updated index number.

8. An audio-video editing device, characterized in that, It includes: A track type determination unit for determining the track type of the target media file according to the file type of the added target media file, where the track type includes a subtitle track, a video track, and an audio track; A track addition unit for determining the rendering number of the target media file and the track position of the target media file on the timeline according to the track type, where the rendering number is used to indicate the rendering order of the target media file, and the timeline is used to hierarchically place media files of different track types; A rendering unit for displaying the target media file on the timeline according to the track position, and rendering the reference media file and the target media file on the timeline according to the rendering numbers corresponding to the multiple reference media files and the target media file on the timeline.

9. A computer program product, comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. An electronic device, comprising a memory and a processor, characterized in that, A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 7 through the computer program.

Images