Display device and media asset synthesis method
By configuring the controller in the display device and performing sub-region extraction, encoding, format conversion and packaging processing, the problems of slow video editing and synthesis speed and low richness are solved, and efficient media synthesis is achieved.
Patent Information
- Application Number
- CN202411245494.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-05
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-09-05
AI Technical Summary
Existing display devices are slow during video editing and synthesis, and the original material is limited, resulting in low video richness.
By configuring a controller in the display device, performing pixel-based sub-region extraction, encoding, format conversion, time stamping and packaging processing, the original picture, audio and text data are obtained and processed, and synthetic media data is generated.
It improves the speed and richness of media synthesis, can process multiple different types of original media data in real time, and generate high-quality synthetic media data.
Smart Images

Figure CN119094808A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of display devices, and in particular to a display device and a media synthesis method. Background Art
[0002] Display devices are intelligent devices that can present user interfaces and support user interaction. Taking smart TVs as an example, smart TVs are based on Internet application technology, have open operating systems and chips, have open application platforms, can realize two-way human-computer interaction functions, and are TV products that integrate multiple functions such as audio and video, entertainment, and data to meet the diverse and personalized needs of users. Display devices can play videos from a variety of different sources, including videos from online video platforms, videos from live broadcast software, and videos created by users.
[0003] Users can create videos through specific applications in the display device. For example, upload a video shot in advance to the display device, use the application on the display device to perform editing operations such as cropping and splicing on the video; or select existing videos on the display device for synthesis and other operations.
[0004] However, the method of editing the shot videos or synthesizing multiple videos is slow to produce videos, and the required original materials are limited, resulting in low richness of the produced videos. Summary of the invention
[0005] The present application provides a display device and a media resource synthesis method to solve the problems of slow speed and low richness of media resource data synthesis.
[0006] In a first aspect, some embodiments of the present application provide a display device, including: a display and a controller.
[0007] Wherein, the display is configured to display a user interface; and the controller is configured to:
[0008] In response to a media synthesis instruction input by a user, original media data is acquired, where the original media data includes original picture data, original audio data, and original text data;
[0009] Performing region extraction on the original image data in units of pixels to obtain video frame data;
[0010] Encoding the video frame data according to a preset encoding format to obtain video data;
[0011] Performing format conversion processing on the original audio data to obtain audio data in a preset encoding format;
[0012] Adding a time stamp to the original text data to obtain subtitle data;
[0013] The video data, the audio data and the subtitle data are packaged to obtain synthetic media data; wherein the playing time of the synthetic media data is equal to the playing time of the original audio data.
[0014] In some embodiments of the present application, before the controller executes the step of performing region extraction on the original image data in units of pixels to obtain video frame data, the controller is also configured to: perform decoding on the original image data to obtain decoded image data; if the data format of the decoded image data is different from a preset format, perform data format conversion on the decoded image data to obtain decoded image data in a preset format.
[0015] In some embodiments of the present application, the controller performs regional extraction on the original image data in units of pixels to obtain video frame data, and is specifically configured as follows: obtaining a first boundary point and a second boundary point of the decoded image data; the first boundary point and the second boundary point are two symmetrical vertices in the decoded image data; setting an extraction window according to the first boundary point and the second boundary point, and the width of the extraction window is equal to a preset proportion of the pixel points contained between the first boundary point and the second boundary point; taking the first boundary point as the starting point of the extraction window, taking a preset number as the translation step of the extraction window, and extracting pixel points located in the extraction window in the decoded image data in sequence along a first direction to obtain a plurality of the video frame data, the first direction being the direction from the first boundary point to the second boundary point; the video frame data includes extraction boundary points.
[0016] In some embodiments of the present application, the controller performs regional extraction on the original image data in units of pixels to obtain video frame data, and is also configured as: if the extraction boundary point of the video frame data coincides with the second boundary point, then the second boundary point is used as the starting point of the extraction window, the preset number is used as the translation step of the extraction window, and pixel points located in the extraction window are extracted from the decoded image data in sequence along the second direction to obtain a plurality of video frame data, wherein the second direction is the direction from the second boundary point to the first boundary point.
[0017] In some embodiments of the present application, the controller performs format conversion processing on the original audio data to obtain audio data in a preset encoding format, and is specifically configured to: obtain newly added audio data; decode the original audio data to obtain first decoded audio data; decode the newly added audio data to obtain second decoded audio data; synthesize the first decoded audio data and the second decoded audio data to obtain synthesized audio data; encode the synthesized audio data to obtain audio data in a preset encoding format.
[0018] In some embodiments of the present application, the controller executes adding a time mark to the original text data to obtain subtitle data, and is specifically configured to: obtain the sampling frequency and the number of sampling points of the original audio data; calculate the starting time point and the ending time point of the original audio data according to the sampling frequency and the number of sampling points; add the starting time point and the ending time point of the original audio data to the original text data to obtain the subtitle data.
[0019] In some embodiments of the present application, the controller performs encapsulation processing on the video data, audio data and subtitle data to obtain synthetic media data, and is specifically configured to: generate an initial media template, the initial media template including a preset media formula; input the video data, audio data and subtitle data into the initial media template; obtain an end identifier; and based on the end identifier, generate synthetic media data of the preset media formula according to the initial media template.
[0020] In some embodiments of the present application, after the controller executes the step of inputting the video data, audio data and subtitle data into the initial media template, it is also configured to: obtain the starting time mark of the subtitle data; take the time point indicated by the starting time mark as the starting point, and add the subtitle data to the initial media template at preset time intervals.
[0021] In some embodiments of the present application, after the controller executes the step of obtaining the original media data in response to the media synthesis instruction input by the user, it is also configured to: obtain the data length, sampling frequency, number of channels and bit depth of the original audio data; and calculate the playback time of the original audio data based on the data length, sampling frequency, number of channels and bit depth.
[0022] In a second aspect, some embodiments of the present application further provide a media synthesis method, which is applied to the display device provided in the first aspect, wherein the display device includes: a display and a controller. The method includes:
[0023] In response to a media synthesis instruction input by a user, original media data is acquired, where the original media data includes original picture data, original audio data, and original text data;
[0024] Performing region extraction on the original image data in units of pixels to obtain video frame data;
[0025] Encoding the video frame data according to a preset encoding format to obtain video data;
[0026] Performing format conversion processing on the original audio data to obtain audio data in a preset encoding format;
[0027] Adding a time stamp to the original text data to obtain subtitle data;
[0028] The video data, the audio data and the subtitle data are packaged to obtain synthetic media data; wherein the playing time of the synthetic media data is equal to the playing time of the original audio data.
[0029] It can be seen from the above technical solutions that some embodiments of the present application provide a display device and a media synthesis method. The method can obtain original media data in response to a media synthesis instruction input by a user; perform region extraction on the original image data in units of pixels to obtain video frame data; encode the video frame data according to a preset encoding format to obtain video data; perform format conversion processing on the original audio data to obtain audio data in a preset encoding format; add time tags to the original text data to obtain subtitle data; perform encapsulation processing on the video data, audio data and subtitle data to obtain synthesized media data. The method can obtain multiple different types of original media data, and perform corresponding processing on the obtained original media data in real time, and then encapsulate the processed media data to obtain synthesized media data, thereby improving the speed and richness of the synthesized media data. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.
[0031] Figure 1 A schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application;
[0032] Figure 2 A schematic diagram of the hardware configuration of a display device provided in some embodiments of the present application;
[0033] Figure 3 A schematic diagram of software configuration of a display device provided in some embodiments of the present application;
[0034] Figure 4 A schematic diagram of a method for a controller to control a media synthesis application to synthesize media data provided by some embodiments of the present application;
[0035] Figure 5 A schematic diagram of a media resource synthesis method provided in some embodiments of the present application;
[0036] Figure 6 A timing diagram of a media resource synthesis method provided in some embodiments of the present application;
[0037] Figure 7 A diagram showing the architecture of a media synthesis method provided in some embodiments of the present application;
[0038] Figure 8 A schematic diagram of a method for performing sub-region extraction by a controller provided in some embodiments of the present application;
[0039] Fig. 9 A schematic diagram of a first image acquired by a controller provided in some embodiments of the present application;
[0040] Fig.10 A schematic diagram of an extraction window determined by a controller provided in some embodiments of the present application;
[0041] Fig.11 A schematic diagram of the second stage provided for some embodiments of the present application;
[0042] Fig.12 A schematic diagram of a method for a controller to obtain video data provided by some embodiments of the present application;
[0043] Fig.13 A schematic diagram of a method for synthesizing audio data by a controller provided in some embodiments of the present application;
[0044] Fig.14 A schematic diagram of a method for a controller to generate subtitle data provided by some embodiments of the present application;
[0045] Fig.15 A schematic diagram of a method for a controller to generate synthetic media data provided in some embodiments of the present application. DETAILED DESCRIPTION
[0046] The following embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following embodiments do not represent all implementations consistent with the present application. They are only examples of systems and methods consistent with some aspects of the present application as detailed in the claims.
[0047] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.
[0048] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.
[0049] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.
[0050] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.
[0051] In the embodiment of the present application, the display device 200 generally refers to a device with image display and data processing capabilities. For example, the display device 200 includes but is not limited to a smart TV, a mobile terminal, a computer, a monitor, an advertising screen, a wearable device, a virtual reality device, an augmented reality device, etc.
[0052] Figure 1 This is a schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application. Figure 1 As shown in FIG. 1 , a user can operate the display device 200 through touch operation, the mobile terminal 300 and the control device 100. The control device 100 is used to receive operation instructions input by the user and convert the operation instructions into control instructions that can be recognized and responded to by the display device 200. For example, the control device 100 can be a remote controller, a stylus pen, a handle, etc.
[0053] The mobile terminal 300 can be used as a control device for performing human-computer interaction between the user and the display device 200. The mobile terminal 300 can also be used as a communication device for establishing a communication connection with the display device 200 and performing data interaction. In some embodiments, the mobile terminal 300 can install software applications with the display device 200, and achieve connection and communication through a network communication protocol to achieve the purpose of one-to-one control operation and data communication. The audio and video content displayed on the mobile terminal 300 can also be transmitted to the display device 200 to achieve a synchronous display function.
[0054] In some embodiments, the mobile terminal 300 or other electronic devices may also simulate the functions of the control device 100 by running an application program for controlling the display device 200 .
[0055] like Figure 1 As also shown in FIG. 4 , the display device 200 also communicates data with the server 400 through various communication methods. The display device 200 may be allowed to communicate and connect through a local area network (LAN), a wireless local area network (WLAN), and other networks.
[0056] The display device 200 may provide a broadcast receiving television function, and may also additionally provide an intelligent network television function with a computer support function, including but not limited to network television, smart television, Internet Protocol television (IPTV), and the like.
[0057] Figure 2 Some embodiments of the present application provide Figure 1 A schematic diagram of the hardware configuration of the display device 200 is shown in FIG.
[0058] In some embodiments, the display device 200 may include at least one of a tuner 210 , a communication device 220 , a detector 230 , a device interface 240 , a controller 250 , a display 260 , an audio output device 270 , a memory, a power supply, and a user input interface 280 .
[0059] In some embodiments, the detector 230 is used to collect signals of the external environment or external interaction. For example, the detector 230 includes a light receiver, a sensor for collecting the intensity of ambient light; or, the detector 230 includes an image collector, such as a camera, which can be used to collect external environment scenes, user attributes or user interaction gestures, or, the detector 230 includes a sound collector, such as a microphone, etc., for receiving external sounds.
[0060] In some embodiments, the display 260 includes a display function component for presenting a picture, and a driving component for driving an image display. The display 260 is used to receive an image signal output from the controller 250 for display. For example, the display 260 can be used to display video content, image content, and components of a menu control interface and a user control UI interface.
[0061] In some embodiments, the communication device 220 is a component for communicating with an external device or server 400 according to various communication protocol types. The display device 200 may be provided with a plurality of communication devices 220 according to different supported communication modes. For example, when the display device 200 supports wireless network communication, the display device 200 may be provided with a communication device 220 including a WiFi function. When the display device 200 supports Bluetooth connection communication, the display device 200 needs to be provided with a communication device 220 including a Bluetooth function.
[0062] The communication device 220 can enable the display device 200 to communicate with the external device or server 400 by wireless or wired connection. Among them, the wired connection can connect the display device 200 with the external device through components such as data cables and interfaces. The wireless connection can connect the display device 200 with the external device through wireless signals or wireless networks. The display device 200 can establish a connection relationship with the external device directly, or indirectly establish a connection relationship through a gateway, a router, a connection device, etc.
[0063] In some embodiments, the controller 250 may include at least one of a central processing unit, a video processor, an audio processor, a graphics processor, and a power processor, and a first interface to an nth interface for input / output. The controller 250 controls the operation of the display device and responds to the user's operation through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200.
[0064] In some embodiments, the controller 250 and the tuner-demodulator 210 may be located in different separate devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.
[0065] In some embodiments, the user may input a user command through a graphical user interface (GUI) displayed on the display 260 , and the user input interface receives the user input command through the graphical user interface (GUI).
[0066] In some embodiments, the audio output device 270 may be a local speaker of the display device 200, or may be an external audio output device of the display device 200. In particular, for the external audio output device of the display device 200, the display device 200 may also be provided with an external audio output terminal, and the audio output device may be connected to the display device 200 through the external audio output terminal to output the sound of the display device 200.
[0067] In some embodiments, the user input interface 280 may be used to receive instructions from a user.
[0068] In order to perform user interaction, in some embodiments, the display device 200 may run an operating system. The operating system is a computer program for managing and controlling hardware resources and software resources in the display device 200. The operating system may control the display device 200 to provide a user interface, for example, the operating system may directly control the display device 200 to provide a user interface, or may provide a user interface by running an application program. The operating system also allows the user to interact with the display device 200.
[0069] It should be noted that the operating system may be a native operating system based on a specific operating platform, or a third-party operating system deeply customized based on a specific operating platform, or an independent operating system specially developed for the display device 200 .
[0070] The operating system can be divided into different modules or layers according to the functions implemented, such as Figure 3 As shown, in some embodiments, the system is divided into four layers, from top to bottom, namely, the application layer (Applications) layer (referred to as "application layer"), the application framework layer (Application Framework) layer (referred to as "framework layer"), the system library layer and the kernel layer.
[0071] In some embodiments, the application layer is used to provide services and interfaces for applications so that the display device 200 can run applications and interact with users based on the applications. At least one application can be run in the application layer, and these applications can be window programs, system settings programs, clock programs, etc. that come with the operating system; they can also be applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the above examples.
[0072] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center that determines the actions that applications in the application layer take. Through the API interface, applications can access system resources and obtain system services during execution.
[0073] like Figure 3As shown, in the embodiment of the present application, the application framework layer includes a view system, managers, content providers, etc., wherein the view system can design and implement the interface and interaction of the application, and the view system includes lists, grids, text boxes, buttons, etc. The manager includes at least one of the following modules: an activity manager for interacting with all activities running in the system; a location manager for providing system services or applications with access to system location services; a package manager for retrieving various information related to the application package currently installed on the device; a notification manager for controlling the display and clearing of notification messages; and a window manager for managing icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.
[0074] In some embodiments, the activity manager is used to manage the life cycle of each application and the usual navigation back function, such as controlling the exit, opening, and back of the application. The window manager is used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, capturing the screen, and controlling the display window changes, for example, reducing the display window, shaking the display, distorting the display, etc.
[0075] In some embodiments, the system runtime layer can provide support for the framework layer. When the framework layer is used, the operating system will run the instruction library contained in the system runtime layer, such as the C / C++ instruction library, to implement the functions to be implemented by the framework layer.
[0076] In some embodiments, the kernel layer is a functional layer between the hardware and software of the display device 200. The kernel layer can implement functions such as hardware abstraction, multitasking, and memory management. Figure 3 As shown, the kernel layer may be configured with hardware drivers, and the drivers included in the kernel layer may be at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.
[0077] It should be noted that the above example is only a simple division of the operating system functions and does not constitute a limitation on the specific operating system form of the display device 200 in the embodiment of the present application. Depending on factors such as the function of the display device and the type of operating system, the number of levels and specific level types contained in the operating system may be expressed in other forms.
[0078] In some embodiments, the display device 200 may display a specific user interface in response to a specific control instruction input by the user. For example, the display device 200 may display a home page interface in response to a power-on instruction input by the user.
[0079] For another example, the display device 200 may display a media asset playback interface in response to a media asset playback instruction input by a user. In some embodiments, the media assets played by the display device 200 through the media asset playback interface may be media assets from different channels, including media assets in an online video platform, such as media assets pre-stored in "video platform A"; media assets in live broadcast software, such as live broadcast videos of "sports channels"; and media assets created by users, such as videos synthesized by users from two original videos.
[0080] In some embodiments, the display device 200 may control a media synthesis application installed in the display device 200 to synthesize media data in response to a media synthesis instruction input by a user. Figure 4 As shown, when the display device 200 synthesizes media data, step S401 may be performed first: obtaining the original media data to be synthesized. For example, the controller 250 in the display device 200 may shoot a video through a built-in camera to obtain the original media data. For another example, the controller 250 may also receive a video uploaded by a user as the original media data.
[0081] After acquiring the original media data, the controller 250 performs step S402: inputting the original media data to be synthesized into a media synthesis application. For example, the display device 200 acquires the first original media data and the second original media data, and then inputs the first original media data and the second original media data into the media synthesis application for processing.
[0082] Finally, the controller 250 executes step S403: controlling the media synthesis application to synthesize the original media data to be synthesized into new media data. For example, the display device 200 inputs the first original media data and the second original media data into the media synthesis application, and controls the media synthesis application to perform processing on the first original media data and the second original media data, including cutting, splicing, synthesis, etc., to generate new media data.
[0083] However, the original media data that the display device 200 can obtain has limitations, and it is difficult to accurately obtain the corresponding original media data according to user needs, and the richness of the synthesized media data is low. In addition, the process of synthesizing media data by the display device 200 through the media synthesis application first needs to obtain the original media data, and then process the original media data, and the synthesis speed is slow.
[0084] In some embodiments, the display device 200 may also perform a media synthesis method to synthesize the media to solve the problem of slow speed and low richness of the media synthesis of the display device 200. In order to meet the implementation of the media synthesis method, the display device 200 should at least include a display 260 and a controller 250. The display 260 is configured to display a user interface; Figure 5 As shown, the controller 250 is configured to execute program steps corresponding to the media resource synthesis method, including the following contents:
[0085] S100: Responding to a media asset synthesis instruction input by a user, obtaining original media asset data.
[0086] In some embodiments, Figure 6 As shown, after receiving the media synthesis instruction input by the user, the controller 250 first obtains the original media data, wherein the original media data includes data in various formats, such as original picture data, original audio data and original text data.
[0087] In some embodiments, the controller 250 may obtain an original image data, a piece of original audio data, and a piece of original text data as the same set of original media data for generating a synthetic media data. For example, the controller 250 obtains a "first image", "first audio", and "first text" in response to a media synthesis instruction input by a user, and generates synthetic media data using the "first image", "first audio", and "first text".
[0088] In some embodiments, the playback duration of the synthesized media data is equal to the playback duration of the original audio data. Therefore, by obtaining the playback duration of the original audio data, the playback duration of the synthesized media data can be obtained. When the controller 250 obtains the playback duration of the original audio, it can first obtain the data length, sampling frequency, number of channels and bit depth of the original audio data. For example, the data length of the "first audio" obtained by the controller 250 is 1048576 bytes; the sampling frequency is 44100 Hz; the number of channels is 2; and the bit depth is 16 bits.
[0089] Then calculate the playback time of the original audio data based on the data length, sampling frequency, number of channels and bit depth. For example, according to the formula: Calculate the total number of sampling points of "first audio" as Then according to the formula: Calculate the playing time of the "first audio" as
[0090] In some embodiments, Figure 7 As shown, the media synthesis architecture of the controller 250 includes a data cache module. If the raw media data continuously acquired by the display device 200 exceeds the maximum data volume that the controller 250 can process in real time, the unprocessed raw media data can be stored in the data cache module.
[0091] For example, the maximum amount of data that the controller 250 can process in real time is 60 megabytes (MB). If the original media data acquired by the controller 250 in response to the media synthesis instruction input by the user are: "first picture", "first audio", "first text", "second picture" and "second audio", among which the data amount of the "first picture" is 30MB, the data amount of the "first audio" is 20MB, the data amount of the "first text" is 10MB, the data amount of the "second picture" is 40MB, and the data amount of the "second audio" is 10MB, then the total amount of data of all the original media data acquired by the controller 250 is greater than the maximum amount of data that the controller 250 can process in real time. At this time, the controller 250 processes the "first picture", "first audio" and "first text" preferentially according to the acquisition order of the original media data, and inputs the "second picture" and "second audio" that cannot be processed in time into the data cache module for storage according to the acquisition order, and then the controller 250 processes them in sequence according to the storage order.
[0092] S200: performing region extraction on the original image data in units of pixels to obtain video frame data.
[0093] In some embodiments, Figure 7 As shown, the media synthesis architecture of the controller 250 also includes a picture conversion module. When the controller 250 obtains the original picture data in response to the media synthesis instruction input by the user, it first needs to decode the original picture data to obtain decoded picture data. For example, the original picture data obtained by the controller 250 is the "first picture", and the "first picture" is decoded to obtain decoded picture data in RGB format or decoded picture data in YUV format.
[0094] In some embodiments, if the data format of the decoded image data is different from the preset format, the data format conversion is performed on the decoded image data to obtain the decoded image data in the preset format. For example, the preset format is set to the YUV format, if the controller 250 decodes the "first image" and the decoded image data format obtained is in the RGB format, then the data format of the decoded image data needs to be converted to the YUV format.
[0095] In some embodiments, after acquiring the decoded picture data in a preset format, the controller 250 may perform region extraction on the decoded picture data to obtain video frame data. Figure 8 As shown, when the controller 250 performs region extraction on the decoded picture data, it can first perform step S801: obtaining the first boundary point and the second boundary point of the decoded picture data. The first boundary point and the second boundary point are two symmetrical vertices in the decoded picture data. For example, Fig. 9 is a schematic diagram of the “first picture” obtained by the controller 250, Fig. 9 It can be seen that the width of the "first picture" is 1920 pixels and the height is 1680 pixels. A can be used as the first boundary point and B can be used as the second boundary point.
[0096] Secondly, the controller 250 executes step S802: setting an extraction window according to the first boundary point and the second boundary point. The width of the extraction window is equal to the number of pixels contained between the first boundary point and the second boundary point in a preset ratio. In some embodiments, the extraction window is used to extract data from the decoded picture data in different regions to split the decoded picture data into multiple video frame data, and then splice the multiple video frame data into video data. For example, the preset ratio is set to 80%, the width of the "first picture" is 1920 pixels, and the width of the extraction window is 1920*80%=1536 pixels.
[0097] In some embodiments, to improve the integrity of video frame data extraction, the extraction window can be set to a rectangular shape, and the height of the extraction window is the same as the height of the decoded picture data. For example, if the height of the "first picture" is 1680 pixels, the height of the extraction window is set to 1680 pixels. Fig.10 As shown in the dashed box.
[0098] It should be noted that the extraction window can be set to a variety of shapes, such as rectangle, square, etc., as long as it can completely extract the video frames in the decoded image data, and no specific limitation is made in the embodiments of the present application.
[0099] In some embodiments, after the controller 250 has determined the first boundary point, the second boundary point, and the shape and size of the extraction window, step S803 may be started: taking the first boundary point as the starting point of the extraction window, taking a preset number as the translation step length of the extraction window, and sequentially extracting pixel points located in the extraction window from the decoded image data in the first direction to obtain a plurality of video frame data. Fig.10 As shown, the first boundary point A is used as the starting point of the extraction window, the preset number is set to 1 pixel point, and the first direction is set to the direction from the first boundary point A to the second boundary point B, that is, Fig.10In the direction indicated by the arrow in the figure, after the extraction window extracts the first video frame data of the first stage, it moves 1 pixel point in the direction of the second boundary point B, and then extracts the second video frame data of the first stage, and so on.
[0100] In some embodiments, each video frame data includes extracted boundary points, that is, boundary points determined by the extraction window. Fig.10 As shown, if the first boundary point of the decoded picture data is A and the second boundary point is B, the extracted boundary point of the video frame data in the first stage is the point closest to the second boundary point B.
[0101] In some embodiments, during the process of extracting video frame data, the extraction window is gradually moved toward the second boundary point with a preset number of pixel points as the translation step. If the extraction boundary point of the video frame data coincides with the second boundary point, the second stage is entered. In the second stage, the second boundary point is used as the starting point of the extraction window, the preset number of pixel points is used as the translation step of the extraction window, and the pixel points located in the extraction window are extracted from the decoded picture data in the second direction in sequence to obtain multiple video frame data. For example, Fig.11 As shown, the second boundary point B is used as the starting point of the extraction window, the preset number is set to 1 pixel point, and the second direction is set to the direction from the second boundary point B to the first boundary point A, that is, Fig.11 In the direction indicated by the arrow in the figure, after the extraction window extracts the first video frame data in the second stage, it moves one pixel in the direction of the first boundary point A, and then extracts the second video frame data in the second stage, and so on.
[0102] It should be noted that, since the moving direction of the extraction window in the second stage is opposite to that in the first stage, the extraction boundary points of the video frame data in the second stage are also opposite to those in the first stage, such as Fig.11 As shown, it is the point closest to the first boundary point A.
[0103] In some embodiments, in the second stage, when the extraction window moves to make the extraction boundary point of the video frame data coincide with the first boundary point A, the second stage of the controller 250 extracting the video frame data is completed, that is, the controller 250 has extracted the required multiple video frame data.
[0104] S300: Encoding the video frame data according to a preset encoding format to obtain video data.
[0105] In some embodiments, after extracting the video frame data, the controller 250 can calculate the time interval between the video frame data according to the set frame rate and send it to the encoder, which encodes the video frame data to obtain the video data required for the synthetic media data. For example, if the frame rate set by the controller 250 is 30 frames per second, the time interval between two video frame data is 0.1332 seconds. The video frame data and the time interval are sent to the encoder, and the video frame data is encoded into the H.264 format by the encoder. The H.264 format can provide high-quality image transmission at low bandwidth and has a high compression ratio, which can reduce the resources occupied by image storage and transmission while improving image quality.
[0106] It should be understood that the video frame data may also be encoded into other formats besides the H.264 format, as long as it is used for synthesizing media asset data, and is not specifically limited in the embodiments of the present application.
[0107] In some embodiments, the controller 250 may process all the original image data in the data cache module in sequence. Fig.12 As shown, when the controller 250 generates video data according to an original picture data in the data cache module, it first decodes the original picture data, and then determines whether the data format of the decoded picture data is a preset format. If the data format of the decoded picture data is not a preset format, it is converted to the preset format. Secondly, the first stage of regional extraction is performed on the decoded picture data of the preset format. In the process of regional extraction, after each video frame data is extracted by the controller 250, the video frame data is encoded, and then the extraction window is moved along the first direction to extract the next video frame data, and at the same time, it is determined whether the second boundary point coincides with the extraction boundary point. If the second boundary point coincides with the extraction boundary point, the second stage of regional extraction is performed, and the extraction window is moved along the second direction to extract the next video frame data. In the process of regional extraction in the second stage, the controller 250 will determine whether the first boundary point coincides with the extraction boundary point. If the first boundary point coincides with the extraction boundary point, the controller 250 preliminarily completes the processing of an original picture data. Finally, the controller 250 determines whether the regional extraction is completed according to the video frame rate of the regional extraction.
[0108] In some embodiments, when the controller 250 finishes processing an original picture data in the data cache module, it determines whether there is any unprocessed original picture data in the data cache module. If so, it continues to process the original picture data according to the above method.
[0109] As can be seen from the above embodiments, by performing sub-region extraction from the first boundary point A to the second boundary point B on the decoded picture data, video data with left and right movement effects can be generated. It should be understood that if the positions of the first boundary point and the second boundary point are changed, video data with different movement effects can be generated. Fig. 9 As shown, if A is set as the first boundary point and C is set as the second boundary point, video data with an up and down movement effect can be generated, which is not specifically limited in the embodiments of the present application.
[0110] S400: Performing format conversion processing on original audio data to obtain audio data in a preset encoding format.
[0111] In some embodiments, Figure 7 As shown, the media synthesis architecture of the controller 250 also includes an audio conversion module. When the controller 250 obtains the original audio data in response to the media synthesis instruction input by the user, the original audio data can be input into the audio conversion module for decoding, and then the decoded audio data can be encoded to obtain audio data in a preset encoding format.
[0112] For example, the original audio data acquired by the controller 250 is the "first audio". The "first audio" is first decoded. The decoder decompresses the compressed "first audio" into the original audio sampling points according to the encoding format and compression algorithm of the "first audio". Then, the audio sampling points are quantized, and the values of the audio sampling points are mapped to a discrete set of numerical values, which are represented by binary digital signal values. Finally, the quantized digital signal values are converted into a preset format suitable for storage or transmission, such as the AAC format, to obtain audio data in a preset encoding format.
[0113] It should be understood that the audio data may also be encoded into other formats besides the AAC format, as long as it is used to synthesize media asset data, and is not specifically limited in the embodiments of the present application.
[0114] In some embodiments, the controller 250 may also acquire newly added audio data, and combine the newly added audio data with the original audio data to obtain synthesized audio data, so as to improve the richness of the synthesized media asset data.
[0115] like Fig.13 As shown, when generating synthesized audio data, the controller 250 may also perform step S1301: acquiring newly added audio data. For example, when the controller 250 uses the "first audio" to synthesize media asset data, it may also acquire the "second audio" as newly added audio data.
[0116] After acquiring the original audio data and the newly added audio data, the controller 250 performs step S1302: decoding the original audio data to obtain first decoded audio data. And performs step S1303: decoding the newly added audio data to obtain second decoded audio data. For example, the controller 250 acquires the "first audio" as the original audio data and acquires the "second audio" as the newly added audio data. The "first audio" is decoded to obtain the first decoded audio data, and the "second audio" is decoded to obtain the second decoded audio data.
[0117] After acquiring the first decoded audio data and the second decoded audio data, the controller 250 performs step S1304: synthesizing the first decoded audio data and the second decoded audio data to obtain synthesized audio data. For example, the controller 250 synthesizes the first decoded audio data obtained by decoding the "first audio" and the second decoded audio data obtained by decoding the "second audio" into a unified stream of synthesized audio data.
[0118] Finally, the controller 250 performs step S1305: encoding the synthesized audio data to obtain audio data in a preset encoding format. For example, if the preset format is set to AAC format, the controller 250 encodes the synthesized audio data through an encoder to obtain audio data in AAC format. The audio data can also be encoded into other formats besides AAC format, which is not specifically limited in the embodiments of the present application.
[0119] S500: Add a time tag to the original text data to obtain subtitle data.
[0120] In some embodiments, Figure 7 As shown, the media synthesis architecture of the controller 250 also includes a text conversion module. When the controller 250 obtains the original text data in response to the media synthesis instruction input by the user, the original text data can be input into the text conversion module, and the text conversion module adds a time mark to the original text data to obtain subtitle data.
[0121] In some embodiments, Fig.14 As shown, when the controller 250 adds a time stamp to the original text data through the text conversion module, step S1401 can be performed first: the sampling frequency of the original audio data in the same group as the original text data and the number of sampling points of each frame of audio data are obtained. For example, the original text data obtained by the controller 250 is "first text", and the original audio data in the same group as the "first text" is "first audio", then the sampling frequency of the "first audio" obtained by the controller 250 is 44100 Hz; the number of sampling points of each frame of audio data is 1024.
[0122] Next, the controller 250 performs step S1402: calculating the starting time point and the ending time point of the original audio data according to the sampling frequency and the number of sampling points of each frame of audio data. For example, the sampling frequency of the "first audio" acquired by the controller 250 is 44100 Hz, and the number of sampling points of each frame of audio data is 1024, then according to the formula: Calculate the time interval of each frame of audio data in the "first audio" as Seconds, and then determine the starting time point and ending time point of each frame of audio data. For example, the starting time point of the first frame of audio data is 0 seconds, and the ending time point is 0.0232 seconds, the starting time point of the second frame of audio data is 0.0232 seconds, and the ending time point is 0.0464 seconds, and so on.
[0123] Finally, the controller 250 performs step S1403: adding the starting time point and the ending time point of the original audio data to the original text data to obtain the subtitle data. For example, the original audio data obtained by the controller 250 is the "first audio", and the original text data obtained is the "first text". According to the starting time point and the ending time point of each frame of audio data in the "first audio", the starting time point and the ending time point are added to each text content in the "first text", such as: the "first audio" includes the "first audio frame", the "second audio frame" and the "third audio frame". If the time interval of each frame of audio data in the "first audio" is 2.6 seconds, and the "first text" includes the "first sentence", the "second sentence" and the "third sentence", then the starting time point added to the "first sentence" is 0 seconds and the ending time point is 2.6 seconds, the starting time point added to the "second sentence" is 2.6 seconds and the ending time point is 5 seconds, and the starting time point added to the "third sentence" is 5 seconds and the ending time point is 7.5 seconds.
[0124] In some embodiments, after adding the start time point and the end time point to the original text data, subtitle data in a preset format can be generated. For example, after the controller 250 adds the start time point and the end time point to the "first sentence", "second sentence" and "third sentence" in the "first text", it first defines the style of the subtitle data, including font, size, color, border, etc. Secondly, the format of the subtitle data is set to ASS format. Thirdly, define the playback resolution that is suitable for the video data. Finally, edit the subtitle style to generate subtitle data that meets user needs. It should be understood that the subtitle data can also be in a format other than the ASS format, as long as it is used to synthesize media asset data, and no specific limitation is made in the embodiments of the present application.
[0125] In some embodiments, each subtitle data generated by the controller 250 cannot be fully displayed in the corresponding video frame data. Therefore, in order to improve the compatibility of the subtitle data with the video data, so that the subtitle data can be displayed normally when the user controls the controller 250 to jump to the specified video screen, the controller 250 needs to perform frame insertion processing on the subtitle data. First, obtain the starting time mark of the subtitle data. For example, the subtitle data generated by the controller 250 from the "first text" includes subtitle data corresponding to the "first sentence", subtitle data corresponding to the "second sentence", and subtitle data corresponding to the "third sentence". Taking the subtitle data corresponding to the "first sentence" as an example, the time mark is the starting time point 0 seconds and the ending time point 2.6 seconds.
[0126] Then, taking the time point indicated by the start time mark as the starting point, add subtitle data to the initial media asset template at every preset time interval. For example, the starting time point of the subtitle data corresponding to the "first sentence" is 0 seconds. If the preset time interval is 500 milliseconds, then taking 0 seconds as the starting point, insert the subtitle data corresponding to the "first sentence" every 0.5 seconds between 0 seconds and 2.6 seconds, and insert a total of 5 subtitle data corresponding to the "first sentence", namely "first inserted subtitles", "second inserted subtitles", "third inserted subtitles", "fourth inserted subtitles" and "fifth inserted subtitles". Among them, the starting time point of the "first inserted subtitles" is 0.5 seconds, the starting time point of the "second inserted subtitles" is 1 second, the starting time point of the "third inserted subtitles" is 1.5 seconds, the starting time point of the "fourth inserted subtitles" is 2 seconds, and the starting time point of the "fifth inserted subtitles" is 2.5 seconds.
[0127] S600: Encapsulate the video data, audio data and subtitle data to obtain composite media asset data.
[0128] In some embodiments, Figure 7 As shown, the media synthesis architecture of the controller 250 also includes a data encapsulation module. After the controller 250 converts the original picture data, the original audio data and the original text data into video data, audio data and subtitle data in a preset encoding format, the data encapsulation module can be used to perform encapsulation processing to obtain synthesized media data. For example, the controller 250 converts the "first picture" into video data in H.264 format, converts the "first audio" into audio data in AAC format, and converts the "first text" into subtitle data in ASS format, and inputs them into the data encapsulation module for encapsulation processing to generate synthesized media data.
[0129] In some embodiments, Fig.15As shown, when the controller 250 generates the synthetic media data, step S1501 may be performed first: generating an initial media template. For example, the controller 250 needs to generate the synthetic media data in MP4 format in response to the media synthesis instruction input by the user. Then, in the data encapsulation module, the initial media template is first generated, and the preset media format is set to MP4 format in the initial media template. It should be understood that the preset media format can also be other formats besides MP4 format, which is not specifically limited in the embodiments of the present application.
[0130] Next, the controller 250 executes step S1502: inputting the video data, audio data and subtitle data into the initial media asset template. Figure 7 As shown, the media synthesis architecture of the controller 250 also includes a video transmission pipeline, an audio transmission pipeline and a subtitle transmission pipeline. Among them, the video transmission pipeline, the audio transmission pipeline and the subtitle transmission pipeline correspond to a data transmission thread respectively. After the controller 250 converts the original media data into video data, audio data and subtitle data through the picture conversion module, the audio conversion module and the text conversion module, the video data can be transmitted through the data transmission thread corresponding to the video transmission pipeline, the audio data can be transmitted through the data transmission thread corresponding to the audio transmission pipeline, and the subtitle data can be transmitted through the data transmission thread corresponding to the subtitle transmission pipeline, and the converted video data, audio data and subtitle data can be input into the data encapsulation module in real time. It should be understood that by setting different transmission pipelines to transmit video data, audio data and subtitle data respectively, the data transmission speed can be increased, thereby increasing the speed at which the controller 250 generates synthetic media data.
[0131] Finally, the controller 250 executes steps S1503 and S1504: obtaining an end identifier, and based on the end identifier, generating synthetic media data of a preset media format according to the initial media template. In some embodiments, the controller 250 can perform data conversion on multiple groups of original media data, and encapsulate the converted data into the same synthetic media data. When the controller 250 obtains the end identifier in the data encapsulation module, it indicates that the encapsulation process is over, and the controller 250 no longer obtains new original media data. For example, the controller 250 obtains two groups of original media data, the first group of original media data includes "first picture", "first audio" and "first text", and the second group of original media data includes "second picture", "second audio" and "second text". According to the acquisition order, the conversion is performed successively through the picture conversion module, the audio conversion module and the text conversion module to obtain two groups of converted video data, audio data and subtitle data. When the controller 250 sends the two converted sets of video data, audio data and subtitle data to the data encapsulation module through the video transmission pipeline, the audio transmission pipeline and the subtitle transmission pipeline, the data encapsulation module detects the end identifier, and the data encapsulation module encapsulates the two sets of video data, audio data and subtitle data into the same synthetic media data in the order of input.
[0132] It should be noted that in the media synthesis architecture of the controller 250, the process of converting the original media data by the image conversion module, the audio conversion module and the text conversion module and the process of encapsulating the converted data by the data encapsulation module can be performed asynchronously, that is, when the image conversion module, the audio conversion module and the text conversion module perform conversion on the second group of original media data, the data encapsulation module can perform encapsulation on the converted data of the first group of original media data. Therefore, when the controller 250 uses the media synthesis architecture to process multiple groups of original media data, the speed of generating synthesized media data can be increased.
[0133] In some embodiments, the original media data acquired by the controller 250 in response to the media synthesis instruction input by the user also includes original video data. After the controller 250 acquires the original video data, the original video data is first parsed into original picture data, original audio data and original text data. Then, the conversion is performed through the picture conversion module, the audio conversion module and the text conversion module to obtain video data, audio data and subtitle data. Finally, the video data, audio data and subtitle data are transmitted to the data encapsulation module through the video transmission pipeline, the audio transmission pipeline and the subtitle transmission pipeline for encapsulation to obtain the synthesized media data.
[0134] Based on the above display device 200, some embodiments of the present application further provide a media synthesis method, including: in response to a media synthesis instruction input by a user, obtaining original media data, wherein the original media data includes original picture data, original audio data, and original text data;
[0135] Performing region extraction on the original image data in units of pixels to obtain video frame data;
[0136] Encoding the video frame data according to a preset encoding format to obtain video data;
[0137] Performing format conversion processing on the original audio data to obtain audio data in a preset encoding format;
[0138] Adding a time stamp to the original text data to obtain subtitle data;
[0139] The video data, the audio data and the subtitle data are packaged to obtain synthetic media data; wherein the playing time of the synthetic media data is equal to the playing time of the original audio data.
[0140] It can be seen from the above technical solutions that some embodiments of the present application provide a display device and a media synthesis method. The method can obtain original media data in response to a media synthesis instruction input by a user; perform region extraction on the original image data in units of pixels to obtain video frame data; encode the video frame data according to a preset encoding format to obtain video data; perform format conversion processing on the original audio data to obtain audio data in a preset encoding format; add time tags to the original text data to obtain subtitle data; perform encapsulation processing on the video data, audio data and subtitle data to obtain synthesized media data. The method can obtain multiple different types of original media data, and perform corresponding processing on the obtained original media data in real time, and then encapsulate the processed media data to obtain synthesized media data, thereby improving the speed and richness of the synthesized media data.
[0141] The same and similar parts between the various embodiments in this specification can be referenced to each other and will not be described again here.
[0142] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution in the embodiments of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of various embodiments of the present invention or certain parts of the embodiments.
[0143] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.
[0144] For the convenience of explanation, the above description has been made in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or limit the embodiments to the specific forms disclosed above. Based on the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various different variations of the embodiments suitable for specific use considerations.
Claims
1. A display device, characterized in that: include: a display configured to display a user interface; The controller is configured as: In response to a media synthesis instruction input by a user, original media data is acquired, where the original media data includes original picture data, original audio data, and original text data; Performing region extraction on the original image data in units of pixels to obtain video frame data; Encoding the video frame data according to a preset encoding format to obtain video data; Performing format conversion processing on the original audio data to obtain audio data in a preset encoding format; Adding a time stamp to the original text data to obtain subtitle data; The video data, the audio data and the subtitle data are packaged to obtain synthetic media data; wherein the playing time of the synthetic media data is equal to the playing time of the original audio data.
2. The display device according to claim 1, characterized in that Before the controller performs the step of performing region extraction on the original image data in units of pixels to obtain video frame data, the controller is further configured to: Decoding the original picture data to obtain decoded picture data; If the data format of the decoded picture data is different from the preset format, data format conversion is performed on the decoded picture data to obtain decoded picture data in the preset format.
3. The display device according to claim 2, characterized in that The controller performs region extraction on the original image data in units of pixels to obtain video frame data, and is specifically configured as follows: Acquire a first boundary point and a second boundary point of the decoded picture data; the first boundary point and the second boundary point are two symmetrical vertices in the decoded picture data; An extraction window is set according to the first boundary point and the second boundary point, wherein a width of the extraction window is equal to a preset ratio of pixel points included between the first boundary point and the second boundary point; Taking the first boundary point as the starting point of the extraction window and a preset number as the translation step of the extraction window, pixel points located within the extraction window are extracted from the decoded picture data in sequence along a first direction to obtain a plurality of video frame data, wherein the first direction is the direction from the first boundary point to the second boundary point; the video frame data includes the extraction boundary point.
4. The display device according to claim 3, characterized in that The controller performs region extraction on the original image data in units of pixels to obtain video frame data, and is further configured to: If the extraction boundary point of the video frame data coincides with the second boundary point, the second boundary point is used as the starting point of the extraction window, the preset number is used as the translation step of the extraction window, and the pixel points located in the extraction window are extracted in the decoded picture data in sequence along the second direction to obtain multiple video frame data, and the second direction is the direction from the second boundary point to the first boundary point.
5. The display device according to claim 1, characterized in that The controller performs format conversion processing on the original audio data to obtain audio data in a preset encoding format, and is specifically configured as follows: Get the newly added audio data; Decoding the original audio data to obtain first decoded audio data; Decoding the newly added audio data to obtain second decoded audio data; synthesizing the first decoded audio data and the second decoded audio data to obtain synthesized audio data; The synthesized audio data is encoded to obtain audio data in a preset encoding format.
6. The display device according to claim 1, characterized in that The controller adds a time mark to the original text data to obtain subtitle data, and is specifically configured as follows: Obtaining the sampling frequency and number of sampling points of the original audio data; Calculate the starting time point and the ending time point of the original audio data according to the sampling frequency and the number of sampling points; The starting time point and the ending time point of the original audio data are added to the original text data to obtain the subtitle data.
7. The display device according to claim 6, characterized in that The controller performs encapsulation processing on the video data, audio data and subtitle data to obtain synthetic media asset data, and is specifically configured as follows: Generate an initial media resource template, wherein the initial media resource template includes a preset media resource formula; Inputting the video data, audio data and subtitle data into the initial media asset template; Get the end identifier; Based on the end identifier, synthetic media asset data of the preset media asset formula is generated according to the initial media asset template.
8. The display device according to claim 7, characterized in that After the controller executes the step of inputting the video data, audio data and subtitle data into the initial media asset template, the controller is further configured to: Obtaining a start time mark of the subtitle data; Taking the time point indicated by the start time mark as the starting point, the subtitle data is added to the initial media asset template at every preset time interval.
9. The display device according to claim 1, characterized in that: After the controller executes the step of obtaining original media data in response to the media synthesis instruction input by the user, the controller is further configured to: Obtaining the data length, sampling frequency, number of channels and bit depth of the original audio data; The playback duration of the original audio data is calculated according to the data length, sampling frequency, number of channels and bit depth.
10. A media resource synthesis method, characterized in that: Applied to the display device according to any one of claims 1 to 9, the method comprising: In response to a media synthesis instruction input by a user, original media data is acquired, where the original media data includes original picture data, original audio data, and original text data; Performing region extraction on the original image data in units of pixels to obtain video frame data; Encoding the video frame data according to a preset encoding format to obtain video data; Performing format conversion processing on the original audio data to obtain audio data in a preset encoding format; Adding a time stamp to the original text data to obtain subtitle data; The video data, the audio data and the subtitle data are packaged to obtain synthetic media data; wherein the playing time of the synthetic media data is equal to the playing time of the original audio data.
Citation Information
Patent Citations
Video synthesis method and device, electronic equipment and computer readable storage medium
CN111083396A
Multi-material video synthesis method and device, electronic equipment and storage medium
CN112153463A
Method for determining lyric timestamp information and training method of acoustic model
CN112735429A
Video generation method and device, terminal equipment and readable storage medium
CN117440116A
Method and system for realizing caption and speech synchronization in video-audio frequency processing
CN1932976A
