Video processing methods, apparatus, devices and computer-readable storage media

Abstract

This application provides a video processing method, apparatus, device, and computer-readable storage medium, relating to the fields of artificial intelligence and cloud technology. The method includes: responding to a video editing request, performing video editing on a video to be processed to obtain an edited video; recording editing information during the video editing process; performing audio separation processing on the edited video based on the editing information to obtain audio information corresponding to the edited video; performing speech-to-text processing on the audio information to obtain subtitle information corresponding to the audio information and timestamp information corresponding to the subtitle information; and adding the subtitle information to the video to be processed according to the timestamp information to obtain a processed video. This application enables the acquisition and addition of subtitle information that precisely matches the edited video, ensuring a precise correspondence between the processed video and the displayed subtitle information.

Description

Technical Field

[0001] This application relates to the field of Internet technology, and includes, but is not limited to, a video processing method, apparatus, device, and computer-readable storage medium. Background Technology

[0002] Currently, when adding subtitles to videos in post-editing processes on mobile devices, the process typically involves decoding the video to obtain audio, sending the audio to a server for text conversion to obtain the corresponding text, and then the server returning the converted text to the mobile device to add the subtitles.

[0003] However, in some editing scenarios, after users have edited the video, such as speed adjustment, cropping, or splicing multiple video segments, it is difficult to obtain text that accurately matches the video. This can lead to problems such as the video and the displayed subtitles not matching, or the audio and the displayed subtitles not matching. Summary of the Invention

[0004] This application provides a video processing method, apparatus, device, and computer-readable storage medium, relating to the fields of artificial intelligence and cloud technology. While performing video editing on the video to be processed, the editing information during the editing process is recorded. Based on this editing information, the edited video is sequentially subjected to audio separation and speech-to-text processing to obtain subtitle information matching the edited video, thereby achieving precise subtitle addition to the video to be processed.

[0005] The technical solution of this application embodiment is implemented as follows:

[0006] This application provides a video processing method, the method comprising:

[0007] In response to a video editing request, the video to be processed is edited to obtain the edited video;

[0008] Record the editing information during the video editing process;

[0009] Based on the editing information, perform audio separation processing on the edited video to obtain audio information corresponding to the edited video;

[0010] The audio information is processed into speech-to-text to obtain subtitle information corresponding to the audio information and timestamp information corresponding to the subtitle information;

[0011] The subtitle information is added to the video to be processed according to the timestamp information to obtain the processed video.

[0012] This application provides a video processing apparatus, the apparatus comprising:

[0013] The video editing module is used to respond to video editing requests, perform video editing on the video to be processed, and obtain the edited video;

[0014] The recording module is used to record editing information during the video editing process;

[0015] An audio separation module is used to perform audio separation processing on the edited video according to the editing information to obtain audio information corresponding to the edited video;

[0016] The speech-to-text module is used to process the audio information into text, and obtain subtitle information corresponding to the audio information and timestamp information corresponding to the subtitle information;

[0017] An adding module is used to add the subtitle information to the video to be processed according to the timestamp information, so as to obtain the processed video.

[0018] This application provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions to implement the aforementioned video processing method.

[0019] This application provides a video processing device, including: a memory for storing executable instructions; and a processor for executing the executable instructions stored in the memory to implement the video processing method described above.

[0020] This application provides a computer-readable storage medium storing executable instructions, which are used to cause a processor to execute the executable instructions to implement the above-described video processing method.

[0021] The embodiments of this application have the following beneficial effects: While performing video editing on the video to be processed, the editing information during the video editing process is recorded; and based on the editing information, the edited video is sequentially subjected to audio separation processing and speech-to-text processing to obtain subtitle information and timestamp information corresponding to the subtitle information; the subtitle information is added to the video to be processed according to the timestamp information to obtain the processed video. Thus, because the editing information during the video editing process is recorded simultaneously with the video editing on the video to be processed, the edited video can be accurately converted based on the editing information to obtain and add subtitle information that precisely matches the edited video, ensuring that the processed video and the displayed subtitle information correspond precisely. Attached Figure Description

[0022] Figure 1 This is a schematic diagram illustrating the process of adding subtitle information to videos in related technologies;

[0023] Figure 2 This is an optional architecture diagram of the video processing system provided in the embodiments of this application;

[0024] Figure 3 This is a schematic diagram of the structure of the video processing device provided in the embodiments of this application;

[0025] Figure 4 This is an optional flowchart illustrating the video processing method provided in an embodiment of this application;

[0026] Figure 5A This is an optional flowchart illustrating the video processing method provided in an embodiment of this application;

[0027] Figure 5B This is a schematic diagram of the structure of the TAVKit video editing framework provided in the embodiments of this application;

[0028] Figure 6 This is an optional flowchart illustrating the video processing method provided in an embodiment of this application;

[0029] Figure 7 This is an optional flowchart illustrating the video processing method provided in an embodiment of this application;

[0030] Figure 8 This is an optional flowchart illustrating the video processing method provided in an embodiment of this application;

[0031] Figure 9 This is a schematic diagram illustrating an application scenario of the post-editing subtitle addition scheme provided in this application embodiment;

[0032] Figure 10 This is a structural diagram of the mobile video post-editing and subtitle addition scheme provided in the embodiments of this application;

[0033] Figure 11 This is a flowchart illustrating the mobile video post-editing and subtitle addition scheme provided in this application embodiment;

[0034] Figure 12 This is a schematic diagram of the audio transcoding process provided in the embodiments of this application. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0036] In the following description, references to "some embodiments" refer to a subset of all possible embodiments. However, it is understood that "some embodiments" may be the same or different subsets of all possible embodiments and may be combined with each other without conflict. Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art to which the embodiments of this application pertain. The terminology used in the embodiments of this application is for the purpose of describing the embodiments of this application only and is not intended to limit the application.

[0037] Before explaining the embodiments of this application, the nouns and key terms involved in this application will be explained first:

[0038] 1) Multimedia video processing tool (FFMpeg, Fast Forward Mpeg): It is a set of open-source computer programs that can be used to record and convert digital audio and video, and can convert digital audio and video into streams.

[0039] 2) PCM: Pulse Code Modulation is one of the encoding methods in digital communication. The main process is to sample analog signals such as voice and images at regular intervals to discretize the analog signals. At the same time, the sampled values ​​are rounded to the nearest integer according to the layer unit, and the sampled values ​​are represented by a set of binary codes to represent the amplitude of the sampled pulse.

[0040] 3) Advanced Audio Coding (AAC): This is a file compression format specifically designed for audio data. Unlike MP3, AAC uses a completely new encoding algorithm, making it more efficient and offering better value for money. Using AAC, file sizes can be significantly reduced without a noticeable decrease in sound quality.

[0041] 4) TAVKit: It is a video editing framework that uses a simple and flexible application programming interface (API) to build complex video structures, a powerful resource protocol to extend rich video sources (e.g., still images, GIFs, empty videos, etc.), real-time images, and a highly customizable audio rendering chain for visual and sound effects.

[0042] Before explaining the video processing method provided in the embodiments of this application, the video processing methods in related technologies will first be described:

[0043] Currently, most mobile editing apps have similar functions. In the editing scenario, the video is decoded to obtain audio, the audio is sent to the server for text conversion to obtain the corresponding text and timestamp, and then the text is returned to the mobile app to add subtitles, thereby reducing the video editing cost for users. Figure 1 This is a schematic diagram illustrating the implementation process of adding subtitle information to videos in related technologies, such as... Figure 1 As shown, the mobile terminal 101 performs audio separation on the video, obtains separated audio 102, and sends separated audio 102 to the server 103. The server 103 performs text conversion on the audio to obtain subtitle information 104, and returns the subtitle information 104 to the mobile terminal 101.

[0044] In related technologies, mobile devices use audio-video separation technology to obtain audio files (i.e., separated audio 102), typically achieved using the FFMpeg tool. The mobile device uploads the audio file to the server, where the server can process the audio file, performing speech recognition to obtain the speech recognition results. Finally, the server adds a timestamp to the speech recognition results to obtain the corresponding subtitle information and returns it to the mobile device.

[0045] However, in the current implementation, in some editing scenarios, such as when users change the speed of a video, crop it, or splice and merge multiple video segments, it is difficult to obtain the corresponding accurate text. There may be problems such as the video and the displayed subtitles not matching, or the audio and the displayed subtitles not matching. In addition, the current editing scenario usually obtains PCM data and sends the PCM data to the server for text conversion. Due to the large amount of PCM data, the video processing process takes a long time.

[0046] Based on the aforementioned problems in related technologies, this application provides a video processing method. This method is an improved solution based on TAVKIT video editing software development kit (SDK) + backend speech recognition, which adds TAVKIT's audio export and supports backend parsing of exported audio, thereby accurately obtaining the corresponding audio subtitle information.

[0047] The video processing method provided in this application firstly, in response to a video editing request, performs video editing on the video to be processed to obtain an edited video; and records the editing information during the video editing process; then, performs audio separation processing on the edited video based on the editing information to obtain audio information corresponding to the edited video; performs speech-to-text processing on the audio information to obtain subtitle information corresponding to the audio information and timestamp information corresponding to the subtitle information; finally, adds the subtitle information to the video to be processed according to the timestamp information to obtain the processed video. Thus, because the editing information during the video editing process is recorded simultaneously with the video editing of the video to be processed, the edited video can be accurately converted based on the editing information to obtain and add subtitle information that precisely matches the edited video, ensuring a precise correspondence between the processed video and the displayed subtitle information.

[0048] The following describes exemplary applications of the video processing device according to embodiments of this application. In one implementation, the video processing device provided in this application embodiment can be implemented as any terminal with data processing, video playback, and video subtitle addition functions, such as a laptop computer, tablet computer, desktop computer, mobile device (e.g., mobile phone, portable music player, personal digital assistant, dedicated messaging device, portable gaming device), intelligent robot, in-vehicle computer, wearable electronic device, smart home device, VR / AR device, etc. In another implementation, the video processing device provided in this application embodiment can also be implemented as a server. The following will describe exemplary applications when the video processing device is implemented as a terminal.

[0049] See Figure 2 , Figure 2This is an optional architecture diagram of the video processing system 10 provided in this application embodiment. To add subtitle information to the video to be processed, the video processing system 10 provided in this application embodiment includes at least a terminal 100, a network 200, and a server 300. The terminal 100 runs a video playback application capable of playing and editing the video to be processed; alternatively, the terminal has a video generation application capable of generating a complete video to be processed. In this embodiment, a user can operate on an application client on terminal 100 to generate a video editing request. This request is used to request video editing of the video to be processed. After receiving the request, terminal 100 performs video editing on the video to be processed, obtaining an edited video and recording the editing information. Based on the editing information, the edited video undergoes audio separation processing to obtain audio information corresponding to the edited video. After obtaining the audio information, terminal 100 can send the audio information to server 300 via network 200, allowing server 300 to perform speech-to-text processing on the audio information to obtain subtitle information and timestamp information corresponding to the audio information. Server 300 sends the subtitle information and timestamp information to terminal 100, and terminal 100 adds the subtitle information to the video to be processed according to the timestamp information, obtaining a processed video. The processed video with subtitle information is then displayed and played on the current interface 100-1 of terminal 100.

[0050] In some embodiments, after obtaining the audio information, the terminal 100 can also perform speech-to-text processing on its own through the speech-to-text module. That is, the terminal performs the process of speech-to-text processing on the audio information to obtain subtitle information corresponding to the audio information and timestamp information corresponding to the subtitle information. The terminal 100 adds the subtitle information to the video to be processed according to the timestamp information to obtain the processed video and play it.

[0051] In some embodiments, after the terminal receives a user's video editing request, it can also send the video editing request to the server, requesting the server to perform video editing on the video to be processed. While performing video editing on the video to be processed to obtain the edited video, the server records the editing information during the video editing process, and performs audio separation processing on the edited video according to the editing information to obtain audio information corresponding to the edited video; then, it performs speech-to-text processing on the audio information to obtain subtitle information corresponding to the audio information and timestamp information corresponding to the subtitle information. The subtitle information is added to the video to be processed according to the timestamp information, or the server sends the subtitle information and timestamp information to the terminal, and the terminal adds the subtitle information to the video to be processed according to the timestamp information to obtain the processed video.

[0052] The video processing method provided in this application embodiment can also be implemented based on a cloud platform and through cloud technology. For example, the aforementioned server can be a cloud server, which performs video editing on the video to be processed to obtain an edited video. Alternatively, the cloud server can perform audio separation processing on the edited video to obtain audio information corresponding to the edited video. Or, the cloud server can perform speech-to-text processing on the audio information to obtain subtitle information and timestamp information corresponding to the audio information. Alternatively, the cloud server can add subtitle information to the video to be processed according to the timestamp information to obtain a processed video. Of course, in other embodiments, a cloud storage device can also be included to store information such as the video to be processed, the edited video, editing information, audio information, subtitle information, timestamp information corresponding to the subtitle information, and the processed video. In this way, when it is necessary to add subtitle information to the video to be processed again, the information corresponding to the video to be processed can be directly retrieved from the cloud storage device, or the processed video corresponding to the video to be processed can be directly retrieved.

[0053] It's important to clarify that cloud technology refers to a hosting technology that unifies hardware, software, and network resources within a wide area network (WAN) or local area network (LAN) to achieve data computation, storage, processing, and sharing. Cloud technology is a collective term for network technologies, information technologies, integration technologies, management platform technologies, and application technologies applied in the cloud computing business model. It can form resource pools, providing flexible and convenient on-demand access. Cloud computing technology will become a crucial support. Backend services of technical network systems require substantial computing and storage resources, such as video websites, image websites, and many portal websites. With the rapid development and application of the internet industry, every item may have its own identification mark in the future, requiring transmission to backend systems for logical processing. Data at different levels will be processed separately, and various industry data will require robust system support, which can only be achieved through cloud computing.

[0054] The video processing method provided in this application also relates to the field of artificial intelligence (AI) technology. AI technology is used to determine the processed video corresponding to the video to be processed. Specifically, AI technology can be used to edit the video to obtain an edited video, or AI technology can be used to process audio information into speech-to-text to obtain subtitle information and timestamp information corresponding to the audio information; alternatively, AI technology can be used to add subtitle information to the video to be processed to obtain a processed video. In some embodiments, a video processing model can also be trained using AI technology, and this video processing model can be used to implement the video processing method of this application, i.e., the processed video can be automatically generated using this video processing model.

[0055] In this embodiment, it can be achieved at least through computer vision, speech technology, and natural language processing technologies within artificial intelligence. Computer vision (CV) is a science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes for target recognition and measurement, and further performs image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision researches related theories and technologies, attempting to establish artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), and common biometric recognition technologies such as facial recognition and fingerprint recognition. Key technologies in speech technology include Automatic Speech Recognition (ASR), Text-to-Speech (TTS), and Voiceprint Recognition. Speech technology enables computers to hear, see, speak, and feel, representing the future direction of human-computer interaction, with speech being one of the most promising methods. Natural Language Processing (NLP) is an important area in computer science and artificial intelligence. NLP research explores theories and methods for enabling effective communication between humans and computers using natural language. NLP is a science integrating linguistics, computer science, and mathematics. Therefore, research in this field involves natural language—the language people use in daily life—and is closely related to linguistic research. NLP technologies typically include text processing, semantic understanding, machine translation, question answering, and knowledge graphs.

[0056] Figure 3 This is a schematic diagram of the structure of the video processing device provided in the embodiments of this application. Figure 3 The video processing device shown includes at least one processor 310, a memory 350, at least one network interface 320, and a user interface 330. The various components in the video processing device are coupled together via a bus system 340. It is understood that the bus system 340 is used to implement communication between these components. In addition to a data bus, the bus system 340 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 3 The general labeled all buses as Bus System 340.

[0057] The processor 310 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0058] User interface 330 includes one or more output devices 331 that enable the presentation of media content, including one or more speakers and / or one or more visual displays. User interface 330 also includes one or more input devices 332, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0059] Memory 350 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. Memory 350 may optionally include one or more storage devices physically located remote from processor 310. Memory 350 may include volatile memory or non-volatile memory, or both. Non-volatile memory may be read-only memory (ROM), and volatile memory may be random access memory (RAM). The memory 350 described in this application embodiment is intended to include any suitable type of memory. In some embodiments, memory 350 is capable of storing data to support various operations, examples of which include programs, modules, and data structures, or subsets or supersets thereof, as illustrated below.

[0060] Operating system 351 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0061] The network communication module 352 is used to reach other computing devices via one or more (wired or wireless) network interfaces 320, exemplary network interfaces 320 including: Bluetooth, WiFi, and Universal Serial Bus (USB), etc.

[0062] The input processing module 353 is used to detect and translate one or more user inputs or interactions from one or more input devices 332.

[0063] In some embodiments, the apparatus provided in this application may be implemented in software. Figure 3 A video processing device 354 stored in memory 350 is shown. This video processing device 354 can be a video processing unit within a video processing apparatus, and can be software in the form of programs and plugins. It includes the following software modules: a video editing module 3541, a recording module 3542, an audio separation module 3543, a speech-to-text module 3544, and an adding module 3545. These modules are logically connected and can therefore be arbitrarily combined or further separated according to the functions they implement. The functions of each module will be described below.

[0064] In other embodiments, the apparatus provided in this application can be implemented in hardware. As an example, the apparatus provided in this application can be a processor in the form of a hardware decoding processor, which is programmed to execute the video processing method provided in this application. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0065] The following will describe the video processing method provided in this application embodiment with reference to the exemplary application and implementation of the video processing device provided in the embodiments of this application. The video processing device can be any terminal with data processing, video playback and video subtitle addition functions, or it can be a server. That is, the video processing method in this application embodiment can be executed by a terminal, or by a server, or it can be executed by interaction between a terminal and a server.

[0066] See Figure 4 , Figure 4 This is an optional flowchart illustrating the video processing method provided in the embodiments of this application. The following will be combined with... Figure 4 The steps shown will be explained. It should be noted that... Figure 4 The video processing method in this system is implemented through the terminal as the execution subject.

[0067] Step S401: In response to the video editing request, perform video editing on the video to be processed to obtain the edited video.

[0068] Here, video editing processing can include at least one of the following: multi-video splicing, video speed adjustment, video reverse playback, multi-video simultaneous display, background music addition, and extended video functionality. Users can trigger video editing operations on the video to be processed on the terminal to request video editing processing.

[0069] In this embodiment of the application, the video to be processed can be at least one video. When performing video editing on the video to be processed, multiple videos can be processed to obtain an edited video.

[0070] In this embodiment, the video editing process of the video to be processed can be implemented by the terminal. The terminal has a video playback application installed, and the user can trigger the video editing operation through the client of the video playback application, and then complete the video editing process of the video to be processed through the client of the video playback application.

[0071] In some embodiments, the video to be processed can also be edited based on artificial intelligence technology to obtain an edited video.

[0072] Step S402: Record the editing information during video editing.

[0073] Here, editing information refers to information corresponding to video editing processes. Editing information includes the type of video editing process, the editing location of the video editing process, the editing object of the video editing process, and the attribute information of the video after editing.

[0074] In this embodiment of the application, while performing video editing on the video to be processed, the editing information during the video editing process is recorded, so as to obtain editing information that matches the edited video, that is, to obtain editing information that accurately describes the edited video.

[0075] In some embodiments, recording editing information may involve storing real-time acquired editing information in a preset storage unit, or storing the editing information and the edited video in a preset storage unit, thereby enabling accurate video processing based on the editing information when generating and adding subtitle information for the edited video in the future.

[0076] Step S403: Perform audio separation processing on the edited video based on the editing information to obtain the audio information corresponding to the edited video.

[0077] Here, after obtaining the editing information, the audio separation process is performed on the edited video based on the editing information. This can be done according to the type of video editing, the editing position of the video editing, the editing object of the video editing, and the attribute information of the video after editing, etc. The original audio information corresponding to the video to be processed is then processed accordingly. For example, audio processing methods of the same type as the video editing type are performed, the audio position of the original audio information corresponding to the editing position of the video editing is processed, and the original audio information corresponding to the editing object is processed, etc.

[0078] For example, when performing multi-video splicing on videos A1 and A2 to obtain an edited video A1+A2 including both videos A1 and A2, the editing information obtained is as follows: the type of video editing is multi-video splicing, the editing position is the complete videos A1 and A2, the editing objects are videos A1 and A2, and the attribute information of the edited video is that the length of the edited video is the sum of A1 and A2. In this case, the corresponding audio processing of the original audio information of the videos to be processed can be to splice the original audio information A11 of video A1 and the original audio information A21 of video A2 to obtain spliced ​​audio A11+A21.

[0079] In some embodiments, one way to implement audio separation processing is to separate the audio corresponding to the edited video from the edited video; or, another way to implement audio separation processing is to use an audio processing method corresponding to the video editing processing method to process the original audio information corresponding to the video to be processed, obtain the processed audio, and extract the processed audio.

[0080] In the first method described above, while the video to be processed is being edited, the corresponding original audio information is also being edited. The audio is edited to match the edited video, resulting in a video that includes the edited audio. In this case, when separating the audio corresponding to the edited video from the edited video, any audio extraction method or any audio recognition method can be used to separate the audio information.

[0081] For the second method mentioned above, video editing processing can be performed on the video frame images of the video to be processed first. After the video editing processing is completed, the audio processing method corresponding to the video editing processing method is used to process the original audio information corresponding to the video to be processed, so as to obtain audio information that matches the edited video.

[0082] Step S404: Perform speech-to-text processing on the audio information to obtain subtitle information corresponding to the audio information and timestamp information corresponding to the subtitle information.

[0083] Here, the subtitle information includes at least one text segment, each text segment corresponding to a video segment of the edited video. There is a mapping relationship between each text segment and its corresponding video segment. Each video segment has a timestamp, and the timestamp of each video segment is the timestamp of its corresponding text segment. The timestamps of all text segments constitute the timestamp information. In this embodiment, since the timestamp of each video segment of the edited video is known, after obtaining the subtitle information, the timestamp of each video segment can be mapped to each text segment in the subtitle information.

[0084] In some embodiments, the audio information can also be processed into speech-to-text based on artificial intelligence technology to obtain subtitle information corresponding to the audio information and timestamp information corresponding to the subtitle information. Speech-to-text processing can involve performing speech recognition on the audio information to obtain text information corresponding to the audio information, and then performing text editing on the text information to obtain subtitle information corresponding to the audio information.

[0085] Step S405: Add subtitle information to the video to be processed according to the timestamp information to obtain the processed video.

[0086] Here, text segments can be added to each video segment of the edited video according to the order of their timestamps to obtain the processed video.

[0087] The video processing method provided in this application embodiment records editing information during the video editing process while performing video editing on the video to be processed. Based on the editing information, the edited video undergoes audio separation and speech-to-text processing sequentially to obtain subtitle information and timestamp information corresponding to the subtitle information. The subtitle information is then added to the video to be processed according to the timestamp information to obtain the processed video. Thus, because the editing information is recorded simultaneously with the video editing process, the edited video can be accurately converted based on the editing information to obtain and add subtitle information that precisely matches the edited video, ensuring a precise correspondence between the processed video and the displayed subtitle information.

[0088] In some embodiments, the video processing system includes a terminal and a server, and the terminal and the server interact to implement the above-described video processing method. Figure 5A This is an optional flowchart illustrating the video processing method provided in an embodiment of this application, such as... Figure 5A As shown, the method includes the following steps:

[0089] Step S501: The terminal receives the user's video editing operation to obtain the video editing request.

[0090] In step S502, the terminal responds to the video editing request and performs video editing on the video to be processed, thereby obtaining the edited video.

[0091] In step S503, the terminal records the editing information during the video editing process while performing video editing processing on the video to be processed through a preset video editing framework.

[0092] In some embodiments, the video editing framework implements video editing processing through an extensible interface; wherein, the video editing processing includes at least one of the following: multi-video splicing processing, video speed adjustment processing, video reverse playback processing, multi-video simultaneous display processing, background music addition processing, and extended video function processing.

[0093] In some embodiments, the video editing framework may be the TAVKit video editing framework. Figure 5B This is a schematic diagram of the structure of the TAVKit video editing framework provided in the embodiments of this application, as shown below. Figure 5B As shown, the TAV Kit video editing framework 50 is a post-editing solution for audio and video, consisting of a core component and other components.

[0094] TAVKit's core provides complete video post-editing capabilities. It greatly simplifies video editing development, allowing developers to easily implement complex video editing needs. It offers rich low-level audio and video editing capabilities, such as cropping, multi-video splicing, speed adjustment, reverse playback, multi-video simultaneous display, background music, and other video editing capabilities, as well as extensibility. While simplifying the API, it retains easily extensible interfaces, such as providing custom video resources, inserting image and audio filters as components, and custom transition effects. Figure 5B As shown, the TAVKit video editing framework 50 provides an Android media system 501, an AV foundation module 502 (AVFoundation module), and a core media module 503 (CoreMedia).

[0095] TAVKit's component section provides commonly used components for typical audio and video business scenarios. These include: a camera, video synthesizer, player, timeline UI component, audio waveform graph, and album selection component. With these components, functionality can be quickly built during business development, fulfilling requirements in a very short time. For example... Figure 5B As shown, an image effects plugin 504 implements image rendering, an audio effects plugin 505 implements audio rendering, and a transition effects plugin 506 implements transition effects. Furthermore, the rendering chain management module 507 manages the rendering chains of the aforementioned image effects plugin 504, audio effects plugin 505, and transition effects plugin 506. The timeline management module 508 includes resource materials 5081, editing configurations 5082, multitracks 5083, and time effects 5084.

[0096] The TAVKit video editing framework simplifies the API in both its core and component parts, while implementing interfaces in an extensible manner, making it easy for businesses to customize them according to their own needs during subsequent development.

[0097] In this embodiment, the use of the TAVKIT video editing framework makes it easier to record editing information such as cropping, multi-video splicing, speed adjustment, reverse playback, multi-video simultaneous display, and background music. When exporting audio information, the corresponding edited audio data can be quickly exported based on this editing information, making it convenient to obtain subtitles using this audio information.

[0098] In step S504, after obtaining the edited video, the terminal records the video parameter information of the edited video.

[0099] Here, a video editing framework can be used to record the video parameters of the edited video after processing the original video. These video parameters include, but are not limited to, at least one of the following: the length of the edited video, the content of the edited video, the playback order of the edited video, and the attribute information of the edited video.

[0100] In step S505, the terminal simultaneously performs audio separation processing on the edited video based on the editing information and video parameter information to obtain audio information corresponding to the edited video.

[0101] Here, the original audio information corresponding to the video to be processed can be processed according to the content recorded in the editing information and video parameter information to obtain the processed audio. Then, the processed audio can be separated from the edited video and processed to obtain the audio information corresponding to the edited video.

[0102] In step S506, the terminal sends the separated audio information to the server.

[0103] In step S507, the server performs speech-to-text processing on the audio information to obtain subtitle information corresponding to the audio information and timestamp information corresponding to the subtitle information.

[0104] In step S508, the server sends the subtitle information and the timestamp information corresponding to the subtitle information to the terminal.

[0105] In step S509, the terminal adds subtitle information to the video to be processed according to the timestamp information, and obtains the processed video.

[0106] Step S510: The terminal plays the processed video.

[0107] In some embodiments, editing information includes the editing method and editing location of the video editing process, based on Figure 4 , Figure 6 This is an optional flowchart illustrating the video processing method provided in an embodiment of this application, such as... Figure 6 As shown, step S403 can be achieved through the following steps:

[0108] Step S601: Extract the original audio information of the video to be processed.

[0109] Step S602: In the original audio information, determine the sub-audio information corresponding to the editing position.

[0110] In this embodiment, when editing the video to be processed, it is not necessarily the entire content of the video to be processed that is edited; it may only be a certain segment or a few segments of the video to be processed. Therefore, the editing position of the video to be processed and the editing method of the video editing at that editing position can be determined. Consequently, when processing the original audio information, only the sub-audio information corresponding to the editing position can be processed, thereby reducing the amount of data processed by the audio.

[0111] Step S603: Perform audio encoding processing on the sub-audio information according to the editing method to obtain the encoded audio of the corresponding sub-audio information.

[0112] Here, the audio encoding process is implemented using an audio encoding method corresponding to the editing method of the video to be processed. For example, when the editing method of the video to be processed is to delete video segments, the audio encoding method for the sub-audio information can also be to delete that sub-audio segment.

[0113] Step S604: The encoded audio from different editing positions is fused with the original audio information to obtain the audio information corresponding to the edited video.

[0114] Here, the encoded audio can be integrated into the original audio information according to the editing position corresponding to each encoded audio.

[0115] In some embodiments, step S604 can be implemented by the following steps:

[0116] Step S6041: Obtain other audio information in the original audio information besides the sub-audio information.

[0117] Step S6042: Perform format encoding processing on other audio information to obtain format-encoded audio.

[0118] Step S6043: According to the position of the editing position and other audio information in the original audio information, the encoded audio and the format-encoded audio are spliced ​​together to obtain the audio information.

[0119] Here, the encoded audio and format-encoded audio can be concatenated sequentially according to the order in which the editing position and other audio information are located in the original audio information to obtain the audio information.

[0120] In some embodiments, the editing position is at least one, and the above-mentioned format encoding processing can be AAC format encoding processing; correspondingly, step S603 can be implemented through the following steps:

[0121] Step S6031: Perform AAC format encoding processing on each of the sub-audio information according to the editing method to obtain AAC encoded audio corresponding to each sub-audio information.

[0122] Furthermore, step S6042 can also involve performing the AAC format encoding process on other audio information to obtain format-encoded audio.

[0123] In some embodiments, the audio information may be in AAC format. Since AAC format audio information can have a high compression rate while ensuring audio quality, it can save data transmission and upload time. Therefore, AAC format audio information can be sent to the server, and the backend server can process the audio information into speech-to-text, thereby reducing the amount of data processing on the terminal. This allows the terminal to complete more business processes simultaneously, avoiding the phenomenon of high terminal computation and lag caused by simply generating and adding subtitle information for the video to be processed on the terminal.

[0124] based on Figure 5A , Figure 7 This is an optional flowchart illustrating the video processing method provided in an embodiment of this application, such as... Figure 7 As shown, step S506 can be achieved through the following steps:

[0125] Step S701: Send the audio information in AAC format to the server.

[0126] Correspondingly, the method also includes the following steps:

[0127] In step S702, the server transcodes the audio information to obtain audio information in PCM format.

[0128] Correspondingly, step S507 can be achieved through the following steps:

[0129] In step S703, the server performs speech-to-text processing on the audio information in PCM format to obtain subtitle information corresponding to the audio information and timestamp information corresponding to the subtitle information.

[0130] based on Figure 4 , Figure 8 This is an optional flowchart illustrating the video processing method provided in an embodiment of this application, such as... Figure 8 As shown, step S404 can be achieved through the following steps:

[0131] Step S801: Obtain the video timestamp of each video segment in the edited video.

[0132] Here, each video segment corresponds to a video timestamp, which is used to indicate the chronological order of the video segment within the entire edited video.

[0133] Step S802: Determine the sub-audio segments corresponding to each video segment in the audio information. The sub-audio segments correspond to video segments, and the duration of the sub-audio segment is equal to the duration of the corresponding video segment.

[0134] Step S803: Perform speech-to-text processing on each sub-audio segment to obtain audio subtitle information corresponding to each sub-audio segment.

[0135] Step S804: Determine the video timestamp of each video segment as the audio timestamp of the corresponding sub-audio segment's audio subtitle information.

[0136] Here, since video segments correspond to sub-audio segments, the video timestamp of the video segment can be determined as the audio timestamp of the corresponding sub-audio segment.

[0137] Step S805: According to the order of audio timestamps, the audio subtitle information of all sub-audio segments is spliced ​​into subtitle information.

[0138] Step S806: Determine the audio timestamp of each audio subtitle information as the timestamp information corresponding to the subtitle information.

[0139] Please continue to refer to Figure 8 In some embodiments, step S405 can be implemented by the following steps:

[0140] Step S807: Determine the subtitle addition position in the video frame of each video segment of the edited video.

[0141] Step S808: According to the timestamp information, add the subtitle information to the subtitle addition position of each video segment of the edited video to obtain the processed video.

[0142] In this embodiment of the application, the user can pre-specify the subtitle addition position, or the subtitle addition position can be determined from the video frames of each video segment based on artificial intelligence technology. For example, blank positions in the video frame, positions where no substantial content is displayed, or display positions of non-core objects (such as positions outside of people images or buildings in the video frame) can be determined as subtitle addition positions.

[0143] The following will describe an exemplary application of the embodiments of this application in a real-world application scenario.

[0144] This application provides a video processing method, which is an improved solution based on TAVKit video editing SDK + backend speech recognition. It adds the audio information export function of TAVKit and supports backend parsing of the exported audio information to obtain the corresponding audio subtitle information.

[0145] The video processing method of this application embodiment can be widely applied to scenarios involving adding video subtitle information during post-editing of videos on mobile devices. Taking the post-editing scenario of any video editing application as an example, when a user adds several video clips for video editing, these video clips contain dialogue, requiring quick access to this dialogue information for editing, and achieving accurate matching with the spoken dialogue in the video. Using the technical solution of this application embodiment, it is only necessary to send the video clips to the TAVKit video editing SDK, complete the audio export through the TAVKit video editing SDK, and then send the audio to the server to obtain the subtitle description information (i.e., subtitle information). The client can then implement the corresponding business logic based on the subtitle description information.

[0146] Figure 9 This is a schematic diagram illustrating an application scenario of the post-editing subtitle addition scheme provided in this application embodiment, such as... Figure 9 As shown, for the edited video 901, after the user clicks the subtitle recognition button on the client, subtitle recognition can begin. During the subtitle recognition process, the subtitle recognition progress will be displayed 902. The user can also terminate the subtitle recognition process by clicking the cancel button 903. After the subtitle recognition of the edited video 901 is completed, the recognized subtitle information 904 can be displayed on the current interface. The subtitle information 904 corresponds to different time periods of the video. When the user confirms that the subtitle recognition process is accurate, they can click the checkmark 905 on the current interface to confirm and save the recognized subtitle information, obtaining a video containing the subtitle information. When the user wants to abandon the subtitle recognition result, they can click the cross 906 on the current interface to abandon the subtitle recognition result.

[0147] Figure 10 This is a structural diagram of the mobile video post-editing and subtitle addition scheme provided in the embodiments of this application, such as... Figure 10As shown, the system comprises three modules: a mobile video encoding / decoding module 1001, a server audio encoding / decoding module 1002, and a server speech-to-text module 1003. The video encoding / decoding module 1001 is implemented on the mobile device and can encode and decode videos edited by the user in real time. In this embodiment, the video encoding / decoding module 1001 can use the TAVKit video editing SDK to encode and decode videos edited by the user in real time, separating the video to obtain audio information in AAC format. The audio encoding / decoding module 1002 performs the following functions: when the server receives audio information (AAC format) uploaded by the client, it transcodes it using FFMPGE to obtain PCM format audio information, which is then used as input to the speech-to-text module 1003. The speech-to-text module 1003 is deployed on the server and can convert the audio information uploaded by the client into text using speech-to-text capabilities, while adding timestamp information to obtain subtitle information matching the video audio. After obtaining the subtitle information, the server's speech-to-text module 1003 returns the subtitle information to the mobile device for processing.

[0148] Figure 11 This is a flowchart illustrating the mobile video post-editing and subtitle addition scheme provided in this application embodiment, as shown below. Figure 11 As shown, this example uses any video editing application. In the video editing application, the user selects media files T1-Tn, which can contain both video and images. These media files are built into the business information using TAVKit, and at this point, they have not undergone speed adjustment or cropping. Then, after entering the editing page, the user may perform audio and video editing (e.g., through the editing panel on the editing page), such as removing T2 media, shrinking T4 media, removing T6 media, and stretching Tn media. The description information of these editing operations is recorded by TAVKit. When the user clicks... Figure 9 When the subtitle recognition button is pressed, the audio separation process is first triggered, resulting in a trimmed and speed-adjusted audio file 1102 (i.e., audio information in AAC format). The timeline of the audio file 1102 is matched with the timeline of the edited material 1101. The client transmits the audio file to the server, which converts the audio file 1102 into a PCM format audio file 1103 as needed. Finally, the server processes the PCM format audio file 1103 to obtain the subtitle information 1104 corresponding to the PCM data.

[0149] In this embodiment, TAVKit supports both iOS and Android platforms during the audio separation process, so all post-editing scenarios are implemented using TAVKit for audio export. The video serves as the input source for the TAVKit video editing SDK. Information such as cropping, speed adjustment, and sequence changes made during the user's editing process are applied when exporting audio information (AAC format, ensuring audio quality while achieving a high compression rate, saving upload time). The resulting audio information then matches the actual information after the user edited these video clips.

[0150] During the audio transcoding process, after the server receives the audio file (AAC format) uploaded by the client, it transcodes it using FFMPGE to obtain a PCM format audio file, which is then used as input for the speech-to-text module. Figure 12 This is a schematic diagram of the audio transcoding process provided in the embodiments of this application, such as... Figure 12 As shown, the AAC format audio file 1202 is transcoded into a PCM format audio file 1203 by FFMPGE processing 1201.

[0151] During the speech-to-text process, the PCM format audio file is passed to the speech-to-text module, which can convert the PCM data into text information (i.e., subtitle information). The text information is timestamped and then returned to the client, which can then edit and process it to obtain subtitles similar to those in movies.

[0152] The video processing method provided in this application embodiment, based on the TAVKit video editing framework, proposes a design scheme that supports the quick addition of subtitles to videos, while also meeting the user's post-editing needs. After the user performs operations such as cropping, speed adjustment, and changing the order of the video, the subtitles can be quickly added, thus improving the user experience in mobile video post-editing scenarios.

[0153] The following continues to describe the exemplary structure of the video processing apparatus 354 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 3 As shown, the video processing apparatus 354 includes:

[0154] The video editing module 3541 is used to respond to video editing requests, perform video editing on the video to be processed, and obtain the edited video;

[0155] Recording module 3542 is used to record editing information during the video editing process;

[0156] The audio separation module 3543 is used to perform audio separation processing on the edited video according to the editing information to obtain audio information corresponding to the edited video;

[0157] The speech-to-text module 3544 is used to process the audio information into text to obtain subtitle information corresponding to the audio information and timestamp information corresponding to the subtitle information;

[0158] The module 3545 is used to add the subtitle information to the video to be processed according to the timestamp information, so as to obtain the processed video.

[0159] In some embodiments, the recording module is further configured to: record the editing information while performing video editing processing on the video to be processed using a video editing framework preset by the terminal; and record the video parameter information of the edited video after obtaining the edited video; correspondingly, the audio separation module is further configured to: perform the audio separation processing on the edited video simultaneously based on the editing information and the video parameter information.

[0160] In some embodiments, the video editing framework implements the video editing process through an extensible interface; wherein the video editing process includes at least one of the following: multi-video splicing, video speed adjustment, video reverse playback, multi-video simultaneous display, background music addition, and extended video functionality.

[0161] In some embodiments, the editing information includes the editing method and editing position of the video editing process; the audio separation module is further configured to: extract the original audio information of the video to be processed; determine the sub-audio information corresponding to the editing position in the original audio information; perform audio encoding processing on the sub-audio information according to the editing method to obtain the encoded audio corresponding to the sub-audio information; and fuse the encoded audio of different editing positions with the original audio information to obtain the audio information corresponding to the edited video.

[0162] In some embodiments, the audio separation module is further configured to: acquire other audio information in the original audio information besides the sub-audio information; perform format encoding processing on the other audio information to obtain format-encoded audio; and perform concatenation processing on the encoded audio and the format-encoded audio according to the editing position and the position of the other audio information in the original audio information to obtain the audio information.

[0163] In some embodiments, the editing position is at least one; the format encoding process is AAC format encoding process; the audio separation module is further configured to: perform AAC format encoding process on each of the sub-audio information according to the editing method to obtain AAC encoded audio corresponding to each of the sub-audio information; correspondingly, perform the AAC format encoding process on the other audio information to obtain the format encoded audio.

[0164] In some embodiments, the audio information has an AAC format; the apparatus further includes: a sending module, configured to send the audio information having the AAC format to a server, so that the server can perform transcoding processing on the audio information to obtain audio information having a PCM format; correspondingly, the speech-to-text module is further configured to: perform the speech-to-text processing on the audio information having the PCM format to obtain subtitle information corresponding to the audio information and timestamp information corresponding to the subtitle information.

[0165] In some embodiments, the speech-to-text module is further configured to: obtain the video timestamp of each video segment in the edited video; determine the sub-audio segments corresponding to each video segment in the audio information; perform the speech-to-text processing on each sub-audio segment to obtain audio subtitle information corresponding to each sub-audio segment; determine the video timestamp of each video segment as the audio timestamp of the audio subtitle information of the corresponding sub-audio segment; concatenate the audio subtitle information of all sub-audio segments into the subtitle information according to the order of the audio timestamps; and determine the audio timestamp of each audio subtitle information as the timestamp information corresponding to the subtitle information.

[0166] In some embodiments, the adding module is further configured to: determine the subtitle addition position in the video frame of each video segment of the edited video; and add the subtitle information to the subtitle addition position of each video segment of the edited video according to the timestamp information, thereby obtaining the processed video.

[0167] It should be noted that the description of the apparatus in this application embodiment is similar to the description of the method embodiment described above, and has similar beneficial effects as the method embodiment; therefore, it will not be repeated. For technical details not disclosed in this apparatus embodiment, please refer to the description of the method embodiment of this application for understanding.

[0168] This application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method described in this application.

[0169] This application provides a storage medium storing executable instructions. When these executable instructions are executed by a processor, they cause the processor to perform the method provided in this application, for example... Figure 4 The method shown.

[0170] In some embodiments, the storage medium may be a computer-readable storage medium, such as ferromagnetic random access memory (FRAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, magnetic surface memory, optical disk, or compact disk-read-only memory (CD-ROM); or it may be a device that includes one or any combination of the above-mentioned memories.

[0171] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0172] As an example, executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file containing other programs or data, for example, in one or more scripts within a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., files storing one or more modules, subroutines, or code sections). As an example, executable instructions may be deployed to execute on a single computing device, or on multiple computing devices located in one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network.

[0173] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A video processing method, characterized in that, The method includes: In response to a video editing request, the video to be processed is edited to obtain the edited video; The terminal uses a preset video editing framework to perform video editing on the video to be processed and records the editing information during the video editing process. The video editing framework implements the video editing process through an extensible interface. The video editing process includes at least one of the following: multi-video splicing, video speed adjustment, video reverse playback, multi-video simultaneous display, background music addition, and extended video function processing. Record the video parameter information of the edited video; Simultaneously, based on the editing information and the video parameter information, the edited video undergoes audio separation processing to obtain audio information corresponding to the edited video; The audio information is processed into speech-to-text to obtain subtitle information corresponding to the audio information and timestamp information corresponding to the subtitle information; The subtitle information is added to the video to be processed according to the timestamp information to obtain the processed video.

2. The method according to claim 1, characterized in that, The editing information includes the editing method and editing location of the video editing process; The step of simultaneously performing audio separation processing on the edited video based on the editing information and the video parameter information to obtain audio information corresponding to the edited video includes: Extract the original audio information of the video to be processed; In the original audio information, the sub-audio information corresponding to the editing position is determined; The sub-audio information is processed by audio encoding according to the editing method to obtain the encoded audio corresponding to the sub-audio information; The encoded audio from different editing positions is fused with the original audio information to obtain audio information corresponding to the edited video.

3. The method according to claim 2, characterized in that, The process of fusing encoded audio from different editing positions with the original audio information to obtain audio information corresponding to the edited video includes: Obtain other audio information from the original audio information besides the sub-audio information; The other audio information is then processed using format encoding to obtain format-encoded audio. The encoded audio and the format-encoded audio are concatenated according to the edit position and the position of the other audio information in the original audio information to obtain the audio information.

4. The method according to claim 3, characterized in that, The editing location is at least one; the format encoding process is AAC format encoding process; The step of performing audio encoding processing on the sub-audio information according to the editing method to obtain the encoded audio corresponding to the sub-audio information includes: Each sub-audio information is encoded in AAC format according to the editing method to obtain AAC encoded audio corresponding to each sub-audio information; Correspondingly, the step of performing format encoding processing on the other audio information to obtain format-encoded audio includes: The other audio information is then processed using the AAC format encoding to obtain the format-encoded audio.

5. The method according to claim 1, characterized in that, The audio information is in AAC format; the method further includes: The audio information in the AAC format is sent to the server so that the server can transcode the audio information to obtain audio information in the PCM format. Correspondingly, the audio information in PCM format is subjected to speech-to-text processing to obtain subtitle information corresponding to the audio information and timestamp information corresponding to the subtitle information.

6. The method according to claim 1, characterized in that, The step of performing speech-to-text processing on the audio information to obtain subtitle information corresponding to the audio information and timestamp information corresponding to the subtitle information includes: Obtain the video timestamp of each video segment in the edited video; Determine the sub-audio segment in the audio information that corresponds to each of the video segments; The speech-to-text processing is performed on each of the sub-audio segments to obtain audio subtitle information corresponding to each of the sub-audio segments; The video timestamp of each video segment is determined as the audio timestamp of the audio subtitle information of the corresponding sub-audio segment; According to the order of the audio timestamps, the audio subtitle information of all sub-audio segments is spliced ​​together to form the subtitle information; The audio timestamp of each audio subtitle is determined as the timestamp information corresponding to the subtitle information.

7. The method according to claim 1, characterized in that, The process of adding the subtitle information to the video to be processed according to the timestamp information to obtain the processed video includes: Determine the location for adding subtitles in each video frame of the edited video segment; According to the timestamp information, the subtitle information is added to the subtitle addition position of each video segment of the edited video to obtain the processed video.

8. A video processing apparatus, characterized in that, The device includes: The video editing module is used to respond to video editing requests, perform video editing on the video to be processed, and obtain the edited video; The recording module is used to record editing information during the video editing process while performing video editing on the video to be processed through a preset video editing framework on the terminal; wherein, the video editing framework implements the video editing process through an extensible interface; the video editing process includes at least one of the following: multi-video splicing, video speed adjustment, video reverse playback, multi-video simultaneous display, background music addition, and extended video function processing; and records the video parameter information of the edited video; An audio separation module is used to simultaneously perform audio separation processing on the edited video based on the editing information and the video parameter information to obtain audio information corresponding to the edited video; The speech-to-text module is used to process the audio information into text, and obtain subtitle information corresponding to the audio information and timestamp information corresponding to the subtitle information; An adding module is used to add the subtitle information to the video to be processed according to the timestamp information, so as to obtain the processed video.

9. A video processing device, characterized in that, include: Memory, used to store executable instructions; A processor, when executing executable instructions stored in the memory, implements the video processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The device stores executable instructions for causing a processor to execute the executable instructions to implement the video processing method according to any one of claims 1 to 7.

11. A computer program product comprising executable instructions or a computer program, characterized in that, When the executable instructions or computer program are executed by a processor, they implement the video processing method according to any one of claims 1 to 7.

Images