Audio and video processing method and device, equipment and storage medium

Abstract

The invention relates to the technical field of web media, and discloses an audio and video processing method, which comprises the following steps: in response to an editing operation acted on an editing interface by a user, acquiring edited data comprising a plurality of audio and video clips; and in response to a preview operation input by a user, obtaining the first audio and video clip through the first video element for playing, and pre-loading the second audio and video clip through the second video element. The second audio and video clip is the next audio and video clip of the first audio and video clip. And when the playing of the first audio and video clip is finished and the second audio and video clip is loaded by the second video element, acquiring the second audio and video clip through the first video element and playing the second audio and video clip until the playing of the plurality of audio and video clips is finished. According to the method, the problems of audio and picture asynchronization, jamming and the like during editing preview are reduced, and the experience of a user when the audio and the video are edited through a web application is improved. The invention also discloses an audio and video processing device and equipment, and a storage medium.

Description

Technical Field

[0001] This application relates to the field of web media technology, and for example to an audio and video processing method, apparatus, device, and storage medium. Background Technology

[0002] Currently, users frequently edit audio and video using audio and video editing software running on electronic devices. However, some audio and video editing software cannot be used across platforms and is inconvenient to operate.

[0003] To facilitate audio and video editing for users, related technologies provide web applications with audio and video editing functions. Since web applications can run through a browser, users can perform audio and video editing across platforms, which is convenient.

[0004] However, during audio and video editing, it is necessary to preview and play the edited audio and video. In related technologies, when users preview the edited audio and video using web applications, real-time rendering is typically used for playback preview. Real-time rendering is prone to problems such as audio-visual desynchronization and stuttering during playback, affecting the preview effect and thus impacting the user experience when editing audio and video using web applications. Summary of the Invention

[0005] To provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. This summary is not intended as a general commentary, nor is it intended to identify key / important components or describe the scope of protection of these embodiments, but rather as a prelude to the detailed description that follows.

[0006] This application provides an audio and video processing method, apparatus, device, and storage medium that can ensure the preview effect of the edited audio and video when editing audio and video through a web application, thereby improving the user's experience when editing audio and video through a web application.

[0007] In a first aspect, embodiments of this application provide an audio / video processing method applied to an electronic device. When the electronic device runs an audio / video editing application through a browser, the browser displays the editing interface of the audio / video editing application. The method includes:

[0008] In response to the user's editing operations on the editing interface, the editing data is obtained; the editing data includes multiple audio and video clips;

[0009] In response to the user's preview input, the first audio / video segment is obtained through the first video element; the first audio / video segment is any one of multiple audio / video segments.

[0010] Play the first audio and video segment, and while playing the first audio and video segment, preload the second audio and video segment through the second video element; the second audio and video segment is the next audio and video segment after the first audio and video segment among multiple audio and video segments;

[0011] Perform the first operation: when the first audio and video segment has finished playing and the second video element has finished loading the second audio and video segment, obtain the pre-loaded second audio and video segment through the first video element and play the audio and video segment currently obtained by the first video element; pre-load the next audio and video segment of the second audio and video segment through the second video element;

[0012] Repeat the first operation until all audio and video segments have finished playing.

[0013] Optionally, obtaining the pre-loaded second audio / video segment through the first video element includes: adjusting the first video element to point to the second audio / video segment.

[0014] Optionally, the editing interface includes a player component. Correspondingly, playing the first audio / video segment as described above includes: calling a first preset function of the first video element to control the player component to play the first audio / video segment.

[0015] Optionally, the processing method provided in this application embodiment further includes: when playing the first audio and video segment; calling a second preset function of the second video element to control the player component to hide the second audio and video segment.

[0016] Optionally, the processing method provided in this application embodiment further includes: pausing the playback of the second audio and video segment when the second video element has not finished loading the second audio and video segment, and waiting for the second video element to finish loading the second audio and video segment.

[0017] Optionally, the editing interface includes a track component; after acquiring the clip data, it also includes: rendering a view of the track component in the editing interface based on the clip data, so that the user can input preview operations based on the view of the track component.

[0018] Secondly, embodiments of this application provide an audio / video processing device integrated into an electronic device. When the electronic device runs an audio / video editing application through a browser, the browser displays the editing interface of the audio / video editing application. The device includes:

[0019] The data acquisition module is configured to acquire editing data in response to user editing operations on the editing interface; the editing data includes multiple audio and video clips.

[0020] The video acquisition module is configured to respond to the user's preview input and acquire the first audio and video segment through the first video element; the first audio and video segment is any one of multiple audio and video segments;

[0021] The first playback module is configured to play a first audio and video segment, and during the playback of the first audio and video segment, a second audio and video segment is preloaded through a second video element; the second audio and video segment is the next audio and video segment of the first audio and video segment among multiple audio and video segments.

[0022] The second playback module is configured to perform the first operation: when the first audio and video segment has finished playing and the second video element has finished loading the second audio and video segment, the module obtains the pre-loaded second audio and video segment through the first video element and plays the audio and video segment currently obtained by the first video element.

[0023] The third playback module is configured to repeatedly execute the first operation until all audio and video segments have finished playing.

[0024] Thirdly, embodiments of this application provide an electronic device, including a processor and a memory storing program instructions, wherein the processor is configured to execute the audio and video processing method as described in the first aspect when running the program instructions.

[0025] Fourthly, embodiments of this application provide a storage medium storing program instructions, wherein, when the program instructions are executed, the audio and video processing method as described in the first aspect is performed.

[0026] The audio and video processing methods, apparatus, devices, and storage media provided in the embodiments of this application can achieve the following technical effects:

[0027] When the editing interface displayed on the browser of an electronic device receives a user's editing operation, it indicates that the user is editing audio and video through an audio and video editing application running on the browser. The electronic device can acquire audio and video editing data, which includes multiple audio and video segments. To facilitate user preview of the edited audio and video, the electronic device responds to the user's preview operation by acquiring and playing the first audio and video segment from the multiple audio and video segments in the editing data through a first video element. During the playback of the first audio and video segment, a second audio and video segment is preloaded through a second video element. When the first audio and video segment finishes playing, the electronic device acquires and plays the preloaded second audio and video segment again through the first video element. During the playback of the second audio and video segment, the electronic device preloads the next audio and video segment through the second video element, and so on, until all multiple audio and video segments of the editing data have finished playing. In this way, when previewing the edited data through the audio and video editing application on the electronic device, the next audio and video segment (i.e., the second audio and video segment) can be preloaded through the second video element while the first audio and video segment acquired by the first video element is playing, so that the preloaded second audio and video segment can be played directly when the first audio and video segment finishes playing. Compared to real-time loading of audio and video clips, this method reduces issues such as audio-visual asynchrony and stuttering, thus ensuring the preview effect of the edited audio and video when editing audio and video through web applications and improving the user experience.

[0028] The above general description and the description below are exemplary and illustrative only and are not intended to limit this application. Attached Figure Description

[0029] One or more embodiments are illustrated by way of example with reference to the accompanying drawings. These illustrations and drawings do not constitute a limitation on the embodiments. Elements having the same reference numerals in the drawings are considered similar elements. The drawings do not constitute a limitation of scale, and wherein:

[0030] Figure 1 This is a schematic diagram of the editing interface of an audio / video editing application provided in an embodiment of this application;

[0031] Figure 2 This is a schematic diagram of a design pattern for an audio / video editing application provided in an embodiment of this application;

[0032] Figure 3 This is a schematic diagram of an audio and video processing method provided in an embodiment of this application;

[0033] Figure 4 This is a schematic diagram of an audio / video processing apparatus provided in an embodiment of this application;

[0034] Figure 5 This is a schematic diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0035] The terms "first," "second," etc., used in the specification, claims, and drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion.

[0036] Unless otherwise stated, the term "multiple" means two or more.

[0037] In this embodiment, the character " / " indicates that the objects before and after it are in an "or" relationship. For example, A / B means: A or B.

[0038] The term "and / or" describes an association between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or A and B.

[0039] The term "correspondence" can refer to an association or binding relationship. The correspondence between A and B means that there is an association or binding relationship between A and B.

[0040] To provide a more detailed understanding of the features and technical content of the embodiments of this application, the implementation of the embodiments of this application will be described in detail below with reference to the accompanying drawings. The accompanying drawings are for illustrative purposes only and are not intended to limit the embodiments of this application. In the following technical description, for ease of explanation, several details are used to provide a full understanding of the disclosed embodiments. However, one or more embodiments may still be implemented without these details. In other cases, well-known structures and devices may be simplified in their depiction to simplify the drawings.

[0041] Currently, with the development of internet technology, users frequently need to create videos. For example, with the popularity of short videos, short video enthusiasts often create videos. Another example is in meeting or teaching scenarios, where video is used to present meeting or teaching content. To ensure the effectiveness of the meeting or teaching, the video content should not be too long, and redundant content often needs to be trimmed. Therefore, users often use audio and video editing software running on electronic devices to edit audio and video. However, some audio and video editing software cannot be used across platforms, is inconvenient to operate, and also needs to be installed on the electronic device, occupying storage space.

[0042] To facilitate audio and video editing for users, related technologies provide web applications with audio and video editing capabilities. These web applications run solely through a browser, allowing users to perform audio and video editing across platforms with ease. Furthermore, web applications do not require installation, saving storage space on electronic devices.

[0043] To clearly understand whether the audio and video editing effect meets the requirements, users need to preview the edited audio and video.

[0044] In related technologies, when users preview their edited audio and video using web applications, the web applications typically use real-time rendering to play the user-edited audio and video. However, real-time rendering is prone to problems such as audio-visual asynchrony and stuttering during audio and video playback, affecting the preview effect and thus impacting the user experience when editing audio and video using web applications.

[0045] In view of this, embodiments of this application provide an audio and video processing method, apparatus, device, and storage medium. For audio and video editing data, when playing multiple audio and video segments in the editing data, the current audio and video segment to be played is obtained through a video element supported by the browser for playback. During this period, the next audio and video segment can also be preloaded through another video element so that the next audio and video segment can be played directly when the current audio and video segment finishes playing. This reduces problems such as audio and video desynchronization and stuttering when previewing the editing data, thereby ensuring the preview effect of the editing data.

[0046] In this embodiment, the device executing the audio and video data processing method is an electronic device. For example, the electronic device can be a tablet computer, mobile phone, laptop, desktop computer, or other electronic device. This electronic device can run a browser that supports HTML (HyperText Markup Language) 5.

[0047] When users need to edit audio and video, they can open the corresponding audio and video editing application in the browser running on their electronic device. This application is a web application with audio and video editing capabilities. Once the application is open, its editing interface will be displayed in the browser.

[0048] For example, an audio / video editing application can be opened by the user clicking a corresponding link or by the user entering a corresponding URL. This application does not specifically limit the way an audio / video editing application can be opened.

[0049] Combination Figure 1As shown, this application embodiment provides an editing interface for an audio / video editing application. The editing interface includes at least a player component 11, a track component 12, and a media component 13. The player component 11 is used to play editing data. After the user performs a media upload operation in the media component 13 area, the user can add the audio or video file to be edited to the media component 13, and the timeline information of the audio or video file to be edited will be displayed in the track component 12 area. In this way, the user can perform editing operations in the editing interface to achieve audio and video editing, and play the editing data through the player component 11 to achieve editing preview.

[0050] Combination Figure 2 As shown in this embodiment, the audio / video editing application can be designed using the MVVM (Model-View-ViewModel) pattern. The Model manages the editing data and can be an editing data management module. The View displays the editing interface and receives user editing operations. The ViewModel corresponds to the React component in the React framework.

[0051] Combination Figure 3 As shown, this application provides an audio and video processing method, which can be applied to the aforementioned electronic device, specifically to an audio and video editing application running in the browser of the electronic device. The method includes:

[0052] S31, in response to the user's editing operation on the editing interface, acquires editing data including multiple audio and video clips.

[0053] S32, in response to the user's preview input, obtain the first audio / video segment through the first video element. The first audio / video segment can be any one of multiple audio / video segments.

[0054] S33, play the first audio / video segment, and during the playback of the first audio / video segment, preload the second audio / video segment using the second video element. The second audio / video segment is the next audio / video segment after the first audio / video segment among multiple audio / video segments.

[0055] S34, when the first audio and video segment finishes playing, determine whether the second video element has finished loading the second audio and video segment.

[0056] S35, if the second video element has finished loading the second audio and video segment, obtain the pre-loaded second audio and video segment through the first video element.

[0057] S36, Play the audio and video segment currently obtained by the first video element.

[0058] S37, the next audio / video segment of the second audio / video segment is preloaded through the second video element and played.

[0059] In this embodiment of the application, steps S34 to S37 are repeated operations until the multiple audio and video segments in the edited data have finished playing and then the operation stops.

[0060] In this embodiment, when the editing interface displayed by the browser of the electronic device receives a user's editing operation, it indicates that the user is editing audio and video through an audio and video editing application running on the browser. The electronic device can obtain the audio and video editing data, which includes multiple audio and video segments. To facilitate the user's preview of the edited audio and video, the electronic device responds to the user's preview operation by acquiring and playing the first audio and video segment of the multiple audio and video segments in the editing data through a first video element. During the playback of the first audio and video segment, a second audio and video segment is preloaded through a second video element. When the first audio and video segment finishes playing, the electronic device acquires the preloaded second audio and video segment again through the first video element and plays the second audio and video segment. During the playback of the second audio and video segment, the electronic device preloads the next audio and video segment of the second audio and video segment through the second video element, and so on, until all multiple audio and video segments of the editing data have finished playing.

[0061] In this way, when previewing the edited data through the audio and video editing application in the electronic device, the next audio and video segment (i.e. the second audio and video segment) of the first audio and video segment can be preloaded through the second video element when the first audio and video segment is played, so that the preloaded second audio and video segment can be played directly when the first audio and video segment finishes playing.

[0062] Compared to real-time loading of audio and video segments, this embodiment of the application reduces the occurrence of problems such as audio-visual asynchrony and stuttering by pre-loading audio and video segments, thereby ensuring the preview effect of the edited audio and video when editing audio and video through web applications and improving the user's experience when editing audio and video through web applications.

[0063] The audio and video clips mentioned in the embodiments of this application can be audio files, video files, or silent image files. The embodiments of this application do not specifically limit them.

[0064] In step S31 above, the editing operations performed by the user on the editing interface can be, in the track component of the editing interface, performing operations such as inserting, trimming, and deleting audio and video clips on the audio and video data of a single track, or performing operations such as adding, inserting, and deleting track data on the track component of the editing interface.

[0065] It's understandable that adding new track data to the track component refers to users uploading new audio and video files to the media component in the editing interface and adding them to the track component for editing. In this way, users can edit the audio and video data in the track component in the editing interface to generate editing data. This editing data can correspond to the audio and video data of one track or multiple tracks, with each track's audio and video data including multiple audio and video clips.

[0066] It should be noted that in this embodiment, the description is based on multiple audio and video segments corresponding to one track in the editing data.

[0067] Of course, the embodiments of this application are also applicable to scenarios where the editing data corresponds to multiple tracks of audio and video data. The implementation method is the same as that of editing data corresponding to multiple audio and video segments of one track, so it will not be described again.

[0068] Following step S31 above, the electronic device can render the view of the track component in the editing interface based on the editing data. Specifically, when the audio / video editing application is designed using the MVVM pattern, the M layer updates the editing data to the VM layer and sends rendering instructions to drive the V layer to render the view of the track component. In this way, after the user performs an editing operation in the editing interface, the editing interface can quickly provide feedback on the view changes to the user, reducing the problem of delayed feedback on view changes and thus improving the user experience of audio / video editing using web applications.

[0069] In step S32 above, when a user plays the edited data using the player component, they typically click the start playback button on the player component, and the player component plays multiple audio and video segments sequentially according to their time order. However, considering that users may not need to play multiple audio and video segments sequentially during the preview of the edited data, but rather play a specific audio and video segment as needed, this is not the case.

[0070] In this way, when the first video element acquires the first audio / video segment, it can determine the first audio / video segment based on the user's preview input. The user's preview operation can control the preview playback progress; that is, the electronic device will start playing from the audio / video segment selected by the user among multiple audio / video segments in the clip data.

[0071] For example, in combination Figure 1As shown, the preview operation can be the user clicking the play button in the player component 11. In response to this operation, the electronic device can obtain the first audio / video segment from multiple audio / video segments through the first video element and play it. The preview operation can also be the user selecting the playback progress in the track component 12. In response to this operation, the electronic device can obtain the audio / video segment corresponding to the playback progress selected by the user through the first video element and play it as the first audio / video segment.

[0072] Using this implementation method, users can play corresponding audio and video clips according to their playback needs while previewing the edited data, allowing them to quickly preview the editing effect and thus improve their experience when editing audio and video using web applications.

[0073] In steps S32 and S33 above, for each track's editing data, two video elements are configured: a first video element and a second video element. The first video element is used to retrieve the first audio / video segment that needs to be played from the editing data and provide it to the player component for playback. The second video element is used to preload the next audio / video segment (i.e., the second audio / video segment) following the first audio / video segment. The process of playing the first audio / video segment will be explained below.

[0074] Optionally, playing the first audio / video segment includes: calling a first preset function of the first video element to control the player component to play the first audio / video segment, and calling a second preset function of the second video element to control the player component to hide the second audio / video segment.

[0075] In this embodiment, during the playback of the first audio / video segment, the first preset function of the first video element can include a display function and a playback function. After executing the display function and the playback function, the player component can be controlled to display and play the first audio / video segment. Both the display function and the playback function are user-preset encapsulated functions. The preset encapsulated display function can be the `activeVideo.show()` function, and the preset encapsulated playback function can be the `activeVideo.play()` function. `activeVideo` refers to the first video element.

[0076] To ensure the first audio / video segment plays correctly, the second audio / video segment needs to be hidden during its playback. This can be achieved by calling a second preset function of the second video element. This second preset function can include a hide function and a pause function. Executing these functions will control the player component to hide and pause the playback of the second audio / video segment. Both the hide and pause functions are user-defined pre-defined functions. The pre-defined hide function could be `freeVideo.hide()`, and the pre-defined playback function could be `activeVideo.pause()`. `freeVideo` refers to the second video element.

[0077] By adopting this implementation method, during the preview and playback of multiple audio and video segments of the edited data, when the first audio and video segment is played, the second audio and video segment is hidden, so that the second audio and video segment will not be displayed in the player component, thereby avoiding interference with the normal playback of the first audio and video segment, thus ensuring that multiple audio and video segments can be previewed and played normally.

[0078] In steps S34 and S35 above, in order to ensure that the next audio and video segment (i.e. the second audio and video segment) of the first audio and video segment can be played smoothly, when the first audio and video segment finishes playing, it can be determined whether the second video element has finished loading the second audio and video segment, and if the second video element has finished loading the second audio and video segment, the pre-loaded second audio and video segment can be obtained through the first audio and video element.

[0079] In this embodiment, if the second video element has finished loading the second audio / video segment, a third preset function will be called. Thus, by monitoring the calling process of the third preset function, it can be determined whether the second video element has finished loading the second audio / video segment. If the third preset function is detected to be called, it indicates that the second audio / video segment has been pre-loaded, and at this time, the second audio / video segment can be obtained and played through the first video element. The third preset function can be the `freeVideo.onReady()` function, where `freeVideo` refers to the second video element.

[0080] Furthermore, considering the possibility that the second audio / video segment might not have finished preloading when the first audio / video segment finishes playing, if the incompletely loaded second audio / video segment is played directly, it will be in a state of loading and playing simultaneously, further affecting the loading progress of the remaining audio / video frames of the second audio / video segment. Therefore, if the second audio / video segment is not fully loaded by the second video element, playback of the second audio / video segment can be paused, and playback can resume only after the second video element has finished loading, and the second audio / video segment can be obtained and played through the first video element.

[0081] In this implementation, if the first audio / video segment finishes playing but the second audio / video segment has not finished loading, it indicates that the electronic device's performance is poor. When electronic performance is poor, pausing the player component to continue playing the audio / video segment will allow the electronic device to allocate more resources to loading the second audio / video segment, thus improving its loading efficiency and enabling playback of the second audio / video segment in the edited data to resume as soon as possible.

[0082] By adopting this implementation method, when the first audio and video segment finishes playing, it is determined whether the second audio and video segment has been pre-loaded. This ensures that the second audio and video segment is played only when it has been pre-loaded, thus guaranteeing smooth playback of the second audio and video segment. This further ensures the preview effect of the edited audio and video and improves the user experience when editing audio and video through web applications.

[0083] In steps S35 to S37 above, when the first audio and video segment ends playing, the pre-loaded second audio and video segment is obtained through the first video element, including: adjusting the first video element to point to the second audio and video segment.

[0084] In this implementation, once the first audio / video segment finishes playing, it indicates that the first video element is in an idle state. At this point, the first video element needs to acquire the second audio / video segment preloaded by the second video element. Specifically, by adjusting the first video element's pointer to the second audio / video segment, the second video element can continue to acquire the second audio / video segment and provide it to the player component for playback, achieving continuous playback of audio / video segments. When the first video element points to the second audio / video segment, it indicates that the second audio / video segment has played successfully, and the second video element then enters an idle state. To ensure continuous playback of subsequent audio / video segments, the idle second video element needs to continue preloading the next audio / video segment. By adjusting the second video element's pointer to the next audio / video segment of the second audio / video segment, the preloading of audio / video segments can continue. In this implementation, the preloading of the next audio / video segment by the second video element can be controlled by calling a preset encapsulated function. For example, the preset encapsulated function could be `freeVideo.prepare()`.

[0085] For example, suppose the first video element is Video1, the second video element is Video2, and the editing data includes three audio and video clips, P1, P2, and P3, respectively. Video1 initially points to P1. Video1 obtains P1 and provides it to the player component for playback. During P1 playback, Video2 points to P2, preloads P2, and performs pre-playback decoding and other operations on P2. When P1 finishes playing, to ensure P2 can continue playing, Video1 is changed from pointing to P1 to pointing to P2. Video1 continues to obtain P2 and provide it to the player component for continued playback. At this point, Video2 is idle. To ensure P3 can play continuously, Video2 is changed from pointing to P2 to pointing to P3, causing Video2 to continue preloading P3. This cycle continues until P1, P2, and P3 have all finished playing. It should be noted that, for the edited data including the three audio and video segments P1, P2 and P3, when Video1 retrieves P3 for playback, Video2 can continue to preload P1 to achieve loop playback of multiple audio and video segments in the edited data.

[0086] By adopting this implementation method, by adjusting the pointer of the first video element, the first video element can continuously obtain preloaded audio and video segments from the second video element, thereby enabling continuous playback of multiple audio and video segments and avoiding pauses during playback. This improves the preview experience for users when playing edited data using web applications.

[0087] In this embodiment of the application, during the playback of the first audio and video segment, it is also necessary to monitor the playback progress of the first audio and video segment in order to determine whether the first audio and video segment has finished playing.

[0088] In this implementation, the timestamps of the audio and video frames of the first audio and video segment in the track component view of the editing interface can be monitored. When the last frame of the first audio and video segment finishes playing, it can be determined that the first audio and video segment has ended, indicating that the next audio and video segment (i.e., the second audio and video segment) is about to be played. In this way, by determining whether the first audio and video frame segment has finished playing, the pre-loaded second audio and video segment can be obtained and played in a timely manner when the first audio and video segment finishes playing, so as to ensure timely and continuous playback between audio and video segments and avoid the problem of pauses during the playback of multiple audio and video segments. This improves the preview experience for users when playing edited data using web applications.

[0089] Combination Figure 4As shown, this application embodiment provides an audio and video processing device integrated into the aforementioned electronic device for controlling the operation of audio and video editing applications within the electronic device. The device includes a data acquisition module 401, a video acquisition module 402, a first playback module 403, and a second playback module 404. Wherein:

[0090] The data acquisition module 401 is configured to acquire editing data in response to user editing operations on the editing interface. The editing data includes multiple audio and video clips.

[0091] The video acquisition module 402 is configured to respond to the preview operation input by the user and acquire the first audio and video segment through the first video element; the first audio and video segment is any one of a plurality of audio and video segments.

[0092] The first playback module 403 is configured to play a first audio / video segment, and during the playback of the first audio / video segment, a second audio / video segment is pre-loaded from multiple audio / video segments via a second video element. The second audio / video segment is the next audio / video segment after the first audio / video segment.

[0093] The second playback module 404 is configured to perform a first operation: when the first audio and video segment has finished playing and the second video element has finished loading the second audio and video segment, the pre-loaded second audio and video segment is obtained through the first video element, and the audio and video segment currently obtained by the first video element is played.

[0094] The third playback module 405 is configured to repeatedly execute the first operation until all audio and video segments have finished playing.

[0095] Optionally, the second playback module 404 is specifically configured to adjust the first video element to point to the second audio and video segment when the first video element obtains the pre-loaded second audio and video segment.

[0096] Optionally, the first playback module 403 is specifically configured to call a first preset function of the first video element to control the player component to play the first audio and video segment.

[0097] Optionally, the first playback module 403 is also used to call a second preset function of the second video element when playing the first audio and video segment, so as to control the player component to hide the second audio and video segment.

[0098] Optionally, the second playback module 404 is also configured to pause the playback of the second audio / video segment and wait for the second audio / video segment to finish loading if the second video element has not finished loading the second audio / video segment.

[0099] Optionally, the audio and video processing apparatus provided in this application embodiment further includes a view update module 406, which is configured to render the view of the track component in the editing interface according to the editing data, so that the user can input preview operations based on the view of the track component.

[0100] The audio and video processing apparatus provided in this application corresponds to the audio and video processing method in the above embodiments, and its implementation principle and technical effect are similar, so it will not be described again here.

[0101] Combination Figure 5 As shown, this application embodiment provides an electronic device 500, including a processor 100 and a memory 101. Optionally, the electronic device 500 may further include a communication interface 102 and a bus 103. The processor 100, communication interface 102, and memory 101 can communicate with each other via the bus 103. The communication interface 102 can be used for information transmission. The processor 100 can call logical instructions in the memory 101 to execute the audio and video processing methods described in the above embodiment.

[0102] Furthermore, the logic instructions in the aforementioned memory 101 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium.

[0103] The memory 101, as a computer-readable storage medium, can be used to store software programs and computer-executable programs, such as program instructions / modules corresponding to the methods in the embodiments of this application. The processor 100 executes functional applications and data processing by running the program instructions / modules stored in the memory 101, that is, it implements the audio and video processing methods in the above embodiments.

[0104] The memory 101 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the terminal device. Furthermore, the memory 101 may include high-speed random access memory and may also include non-volatile memory.

[0105] This application provides a storage medium storing computer-executable instructions configured to execute the audio and video processing methods described in the above embodiments.

[0106] The aforementioned storage medium can be a transient computer-readable storage medium or a non-transitory computer-readable storage medium.

[0107] The technical solutions of this application embodiment can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes one or more instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in this application embodiment. The aforementioned storage medium can be a non-transitory storage medium, including: USB flash drive, portable hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, and other media capable of storing program code; it can also be a transient storage medium.

[0108] The foregoing description and accompanying drawings fully illustrate embodiments of this disclosure to enable those skilled in the art to practice them. Other embodiments may include structural, logical, electrical, procedural, and other changes. The embodiments represent only possible variations. Individual components and functions are optional unless explicitly required, and the order of operation may vary. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. Moreover, the terminology used in this application is for describing embodiments only and is not intended to limit the claims. As used in the description of embodiments and claims, the singular forms “a,” “an,” and “the” are intended to equally include the plural forms unless the context clearly indicates otherwise. Similarly, the term “and / or” as used in this application means including one or more of the associated listed items and all possible combinations thereof. Additionally, when used in this application, the term "comprise" and its variations "comprises" and / or "comprising" refer to the presence of stated features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof. Without further limitations, an element defined by the phrase "comprises a..." does not exclude the presence of other identical elements in the process, method, or apparatus that includes said element. In this document, each embodiment may focus on the differences from other embodiments, and similar or identical parts between embodiments can be referred to mutually. For methods, products, etc., disclosed in the embodiments, if they correspond to the method section disclosed in the embodiments, the relevant parts can be referred to the description of the method section.

[0109] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0110] The methods and products (including but not limited to devices and equipment) disclosed in the embodiments herein can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units may be merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed units may be through some interfaces, and the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to implement this embodiment according to actual needs. Furthermore, the functional units in the embodiments of this application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0111] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than that shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. In the descriptions corresponding to the flowcharts and block diagrams in the accompanying drawings, the operations or steps corresponding to different blocks may also occur in a different order than disclosed in the description; sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. Each block in a block diagram and / or flowchart, and combinations of blocks in a block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

Claims

1. A method for processing audio and video, characterized in that, Applied to electronic devices, when the electronic device runs an audio / video editing application through a browser, the browser displays the editing interface of the audio / video editing application; the method includes: In response to a user's editing operation on the editing interface, editing data is acquired; the editing data includes multiple audio and video segments. In response to a preview operation input by the user, a first audio / video segment is obtained through a first video element; the first audio / video segment is any one of the plurality of audio / video segments. Play the first audio and video segment, and during the playback of the first audio and video segment, preload the second audio and video segment through the second video element; the second audio and video segment is the next audio and video segment of the first audio and video segment among the plurality of audio and video segments; Perform the first operation: when the first audio and video segment finishes playing and the second video element has finished loading the second audio and video segment, obtain the pre-loaded second audio and video segment through the first video element and play the audio and video segment currently obtained by the first video element; pre-load the next audio and video segment of the second audio and video segment through the second video element; Repeat the first operation until all the audio and video segments have finished playing.

2. The processing method according to claim 1, characterized in that, Using the first video element, obtain the pre-loaded second audio and video segment, including: Adjust the first video element to point to the second audio / video segment.

3. The processing method according to claim 1 or 2, characterized in that, The editing interface includes a player component; playing the first audio / video segment includes: The first preset function of the first video element is invoked to control the player component to play the first audio and video segment.

4. The processing method according to claim 1 or 2, characterized in that, The processing method further includes: When the first audio / video segment is playing, a second preset function of the second video element is invoked to control the player component to hide the second audio / video segment.

5. The processing method according to claim 1 or 2, characterized in that, The editing interface includes a track component; After acquiring the clip data, the process also includes: Based on the clipping data, render the view of the track component in the editing interface so that the user can input the preview operation based on the view of the track component.

6. An audio and video processing device, characterized in that, Integrated into the electronic device, when the electronic device runs an audio / video editing application through a browser, the browser displays the editing interface of the audio / video editing application; the device includes: The data acquisition module is configured to acquire editing data in response to a user's editing operation on the editing interface; the editing data includes multiple audio and video clips. The video acquisition module is configured to respond to a preview operation input by the user and acquire a first audio-video segment through a first video element; the first audio-video segment is any one of the plurality of audio-video segments. The first playback module is configured to play the first audio and video segment, and during the playback of the first audio and video segment, a second audio and video segment is preloaded through a second video element; the second audio and video segment is the next audio and video segment of the first audio and video segment among the plurality of audio and video segments. The second playback module is configured to perform a first operation: when the first audio and video segment has finished playing and the second video element has finished loading the second audio and video segment, the module obtains the pre-loaded second audio and video segment through the first video element and plays the audio and video segment currently obtained by the first video element. The third playback module is configured to repeatedly execute the first operation until all the multiple audio and video segments have finished playing.

7. An electronic device comprising a processor and a memory storing program instructions, characterized in that, The processor is configured to perform the audio and video processing method as described in any one of claims 1 to 5 when executing the program instructions.

8. A storage medium storing program instructions, characterized in that, When the program instructions are executed, they perform the audio and video processing method as described in any one of claims 1 to 5.

Images