Multimedia data processing method, device, equipment, and medium

By dividing multimedia data into editing tracks for text, video, and audio segments and aligning them in an editing interface, the method enhances editing flexibility and quality of multimedia data processing.

JP2024540793AActive Publication Date: 2024-11-06BEIJING ZITIAO NETWORK TECH CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
JP2024503680
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2022-10-21
Filing Date
2023-09-27
Publication Date
2024-11-06
Estimated Expiration
2043-09-27

AI Technical Summary

Technical Problem

Current multimedia data processing methods lack flexibility and fail to meet users' diverse editing needs, resulting in low-quality multimedia data generation.

Method used

A method and apparatus for processing multimedia data by dividing it into multiple editing tracks for text, video image, and audio segments, allowing for synchronized editing and alignment of these segments in a multimedia editing interface.

Benefits of technology

Enriches the editing capabilities and improves the quality of multimedia data by enabling flexible and diversified editing of text, video, and audio segments, meeting user-specific needs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024540793000001_ABST
    Figure 2024540793000001_ABST
Patent Text Reader

Abstract

A method, device, equipment and medium for processing multimedia data, the method including the steps of receiving text information input by a user, and in response to a processing instruction for the text information, generating multimedia data based on the text information and displaying a multimedia editing interface for editing and manipulating the multimedia data, the multimedia data including a plurality of multimedia segments, the multimedia editing interface including a first editing track, a second editing track and a third editing track, the first track segment, the second track segment and the third track segment with which the timeline is aligned in the editing tracks respectively identify corresponding text segments, video image segments and audio segments. In the embodiment of the present disclosure, the editing tracks corresponding to the multimedia data can be enriched to meet the diversified editing needs of the multimedia data, and the quality of the multimedia data can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] [CROSS REFERENCE TO RELATED APPLICATIONS] This application claims priority to Chinese Patent Application No. 202211295639.8, filed on October 21, 2022, the entire contents of which are incorporated by reference into this application.

[0002] [Technical field] The present disclosure relates to a method, apparatus, device and medium for processing multimedia data. [Background technology]

[0003] 2. Description of the Related Art With the development of computer technology, the methods of sharing knowledge and information are becoming more and more diverse. In addition to text and audio information carriers, video information carriers are now widely seen.

[0004] In the related art, based on the text content that a user wants to share, a related image video containing the text content is generated. However, the user's ideas change from time to time, and the current creation style is not flexible, and cannot meet the fine-grained needs of the user to process flexibly, and the quality of the multimedia data is not high. Summary of the Invention [Means for solving the problem]

[0005] To solve the above technical problems or at least solve part of the above technical problems, the present disclosure provides a method, an apparatus, a device, and a medium for processing multimedia data.

[0006] An embodiment of the present disclosure provides a method for processing multimedia data, the method including the steps of: receiving text information input by a user; and in response to a processing instruction for the text information, generating multimedia data based on the text information, and displaying a multimedia editing interface for editing and manipulating the multimedia data, the multimedia data including a plurality of multimedia segments, the plurality of multimedia segments respectively corresponding to a plurality of text segments divided from the text information, the plurality of multimedia segments including a plurality of audio segments generated by reciting the plurality of text segments respectively corresponding to the plurality of text segments, and a plurality of video image segments respectively matching the plurality of text segments; and the multimedia editing interface includes: The interface includes a first edit track, a second edit track, and a third edit track, the first edit track including a plurality of first track segments, each of the first track segments being used to identify the plurality of text segments, the second edit track including a plurality of second track segments, each of the second track segments being used to identify the plurality of video image segments, the third edit track including a plurality of third track segments, each of the third track segments being used to identify the plurality of audio segments, and the first track segment, the second track segment, and the third track segment, which are timeline aligned in the edit tracks, identify corresponding text segments, video image segments, and audio segments, respectively.

[0007] An embodiment of the present disclosure further provides an apparatus for processing multimedia data, the apparatus including: a receiving module for receiving text information input by a user; a generating module for generating multimedia data based on the text information in response to a processing instruction for the text information; and a display module for displaying a multimedia editing interface for editing and manipulating the multimedia data, the multimedia data including a plurality of multimedia segments, the plurality of multimedia segments respectively corresponding to a plurality of text segments divided from the text information, the plurality of multimedia segments including a plurality of audio segments generated by reciting the plurality of text segments respectively corresponding to the plurality of text segments, and a plurality of video image segments respectively matching the plurality of text segments. the multimedia editing interface includes a first editing track, a second editing track, and a third editing track, the first editing track includes a plurality of first track segments, each of the first track segments being used to identify the plurality of text segments, the second editing track includes a plurality of second track segments, each of the second track segments being used to identify the plurality of video image segments, the third editing track includes a plurality of third track segments, each of the third track segments being used to identify the plurality of audio segments, and the first track segment, the second track segment, and the third track segment, with which a timeline is aligned in the editing tracks, identify corresponding text segments, video image segments, and audio segments, respectively.

[0008] An embodiment of the present disclosure further provides an electronic device, the electronic device including a processor and a memory for storing instructions executable by the processor, the processor being used to realize the multimedia data processing method provided by the embodiment of the present disclosure by reading the executable instructions from the memory and executing the instructions.

[0009] An embodiment of the present disclosure further provides a computer-readable storage medium, in which a computer program is stored, and the computer program is used to perform the method for processing multimedia data provided by the embodiment of the present disclosure.

[0010] The embodiments of the present disclosure further provide a computer program product, the instructions of which, when executed by a processor, implement the method for processing multimedia data provided by the embodiments of the present disclosure. Effect of the Invention

[0011] The technical solutions provided by the embodiments of the present disclosure have the following advantages:

[0012] The multimedia data processing means provided in the embodiment of the present disclosure receives text information input by a user, and in response to a processing instruction for the text information, generates multimedia data based on the text information, and displays a multimedia editing interface for editing and manipulating the multimedia data, wherein the multimedia editing interface includes a plurality of multimedia segments, each of which corresponds to a plurality of text segments divided from the text information, and the plurality of multimedia segments includes a plurality of audio segments generated by recitation corresponding to the plurality of text segments respectively, and a plurality of video image segments respectively matching the plurality of text segments, and the multimedia editing interface includes a first editing track, a second editing track, and a third editing track, and the first track segment corresponding to the first editing track, the second track segment corresponding to the second editing track, and the third track segment corresponding to the third editing track, in which the timeline is aligned in the editing tracks, respectively identify the corresponding text segments, video image segments, and audio segments, and the embodiment of the present disclosure enriches the editing tracks corresponding to the multimedia data, can meet the diversified editing needs of multimedia data, and improve the quality of the multimedia data.

[0013] The above and other features, advantages and aspects of each embodiment of the present disclosure will become more apparent with reference to the following specific embodiments in combination with the drawings. Throughout the drawings, the same or similar reference numerals refer to the same or similar elements. It should be understood that the drawings are illustrative and that the components and elements are not necessarily drawn to scale. [Brief description of the drawings]

[0014] [Figure 1] FIG. 1 is a flowchart of a method for processing multimedia data provided by an embodiment of the present disclosure. [Diagram 2] FIG. 2 is a schematic diagram of a text input interface provided by an embodiment of the present disclosure. [Diagram 3] FIG. 3 is a schematic diagram of a configuration of a multimedia segment of multimedia data provided by an embodiment of the present disclosure. [Figure 4] FIG. 4 is a schematic diagram of a multimedia data processing scene provided by an embodiment of the present disclosure. [Diagram 5] FIG. 5 is a schematic diagram of a multimedia editing interface provided by an embodiment of the present disclosure. [Figure 6] FIG. 6 is a flowchart of another multimedia data processing method provided by an embodiment of the present disclosure. [Figure 7] FIG. 7 is a schematic diagram of another multimedia data processing scene provided by an embodiment of the present disclosure. [Figure 8] FIG. 8 is a schematic diagram of another multimedia data processing scene provided by an embodiment of the present disclosure. [Figure 9] FIG. 9 is a schematic diagram of another multimedia data processing scene provided by an embodiment of the present disclosure. [Figure 10] FIG. 10 is a flowchart of another multimedia data processing method provided by an embodiment of the present disclosure. [Figure 11] FIG. 11 is a schematic diagram of another multimedia data processing scene provided by an embodiment of the present disclosure. [Figure 12] FIG. 12 is a structural schematic diagram of a multimedia data processing device provided by an embodiment of the present disclosure. [Figure 13] FIG. 13 is a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] Hereinafter, the embodiments of the present disclosure will be described in more detail with reference to the drawings. Although the drawings show some embodiments of the present disclosure, it should be understood that the present disclosure can be realized in various forms and should not be construed as being limited to the embodiments described herein, but on the contrary, these embodiments are provided for a more detailed and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are merely illustrative and are not intended to limit the scope of protection of the present disclosure.

[0016] As should be understood, the steps recited in the method embodiments of the present disclosure may be performed in different orders and / or in parallel, and method embodiments may include additional steps and / or omit the performance of steps as shown, and the scope of the present disclosure is not limited in this respect.

[0017] As used herein, the term "comprises" and variations thereof are open inclusions, including, but not limited to. The term "based on" means "based at least in part on." The term "in one embodiment" refers to "at least one embodiment," the term "in another embodiment" refers to "at least one other embodiment," and the term "in some embodiments" refers to "at least some embodiments." Relevant definitions of other terms are provided in the description below.

[0018] It should be noted that the concepts of "first", "second", etc. referred to in this disclosure are merely intended to distinguish different devices, modules or units, and are not intended to limit the order or interdependence of functions performed by these devices, modules or units.

[0019] It should be noted that the modifications "one" and "multiple" referred to in the present disclosure are merely exemplary and not limiting, and as should be understood by one of ordinary skill in the art, unless the context expressly indicates otherwise, should be understood as "one or more."

[0020] The names of messages or information exchanged between devices in the embodiments of the present disclosure are used for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0021] To solve the above problems, an embodiment of the present disclosure provides a multimedia data processing method, in which multimedia data is divided into multiple editing tracks, such as a text editing track, a video image editing track and an audio editing track, and corresponding information is edited through editing operations of the editing tracks, thereby meeting the diversified editing needs of multimedia data and improving the quality of multimedia data.

[0022] The method for processing multimedia data will now be described with reference to specific embodiments.

[0023] 1 is a flowchart of a multimedia data processing method provided by an embodiment of the present disclosure, the method can be executed by a multimedia data processing device, the device can be realized by software and / or hardware, and may be generally integrated into electronic devices such as computers. As shown in FIG. 1, the method includes the following steps 101 to 102.

[0024] Step 101: receive text information input by a user.

[0025] In one embodiment of the present disclosure, as shown in FIG. 2, a text input interface for editing characters can be provided, the input interface including a text area and a link area. In this embodiment, the text information can be customized and edited according to the needs of video production, that is, the text information input by the user in the text area of ​​the text editing interface is received, or an authorized link is pasted, and the text information is extracted from the link, etc., that is, the link information input by the user in the link area is received, the link information is identified to obtain the text information of the corresponding text, and displayed in the text area for the user to edit, that is, the text information displayed in the text area can be edited multiple times.

[0026] When creating a video, there is usually a time limit, and correspondingly, in some possible embodiments, there is a certain limit on the number of characters of the text information, for example, not to exceed 2000 characters. Therefore, it is possible to check whether the number of characters of the text in the text area exceeds the limit, and if so, a pop-up window indicating that the number of characters has exceeded the limit can be displayed to warn the user, etc.

[0027] In one embodiment of the present disclosure, still referring to FIG. 2, the text input interface may include a video generation button and a tone selection entry control in addition to the text area and link area, and in response to a user triggering the tone selection entry control, a candidate tone menu can be displayed, the candidate tone menu including one or more candidate tones (the multiple candidate tones may include multiple tone types such as uncle, boy, girl, loli, etc.) and preview controls corresponding to the candidate tones, and when the user triggers the preview control, some or all of the text information entered by the user is played in the corresponding candidate tone.

[0028] In this embodiment, a user can select a tone from a variety of candidate tones when inputting text information, thereby determining a first target tone based on the user's selection operation on the candidate tone menu, and further obtaining a plurality of voice segments generated by reciting a plurality of text segments into which the text information is divided based on the first target tone, at this time, the tone of each voice segment is the first target tone, thereby improving the selection efficiency for the first target tone in satisfying the personalized selection for the first target tone.

[0029] When the text information is divided into multiple audio segments, the text information can be subjected to a segmentation process according to the reading habit of the first target tone to determine the text segments contained in each segment; alternatively, the text information can be divided into multiple audio segments based on semantic information of the text information, and the text information can be converted into multiple audio segments corresponding to the multiple text segments with the text segment as the conversion granularity.

[0030] Step 102: in response to the processing instruction for the text information, generate multimedia data based on the text information, and display a multimedia editing interface for editing and manipulating the multimedia data; The multimedia data includes a plurality of multimedia segments, each of which corresponds to a plurality of text segments segmented from the text information, the plurality of multimedia segments including a plurality of audio segments generated by reciting the plurality of text segments, each of which corresponds to the plurality of text segments, and a plurality of video image segments, each of which matches the plurality of text segments; The multimedia editing interface includes a first edit track, a second edit track, and a third edit track, the first edit track includes a plurality of first track segments, each of the plurality of first track segments being used to identify a plurality of text segments, the second edit track includes a plurality of second track segments, each of the plurality of second track segments being used to identify a plurality of video image segments, the third edit track includes a plurality of third track segments, each of the plurality of third track segments being used to identify a plurality of audio segments, and the first track segments, the second track segments, and the third track segments, which are timeline aligned in the edit tracks, identify corresponding text segments, video image segments, and audio segments, respectively.

[0031] In one embodiment of the present disclosure, an entry for editing and processing text information is provided, and a processing instruction for the text information is obtained through the entry. For example, the entry for editing and processing text information may be a "video generation" control displayed in a text editing interface, and when it is detected that a user has triggered the "video generation" control of the text editing interface, a processing instruction for the text information is obtained. Of course, in other possible embodiments, the entry for editing and processing text information may be a gesture action input entry, a voice information input entry, etc.

[0032] In one embodiment of the present disclosure, in response to processing instructions for text information, multimedia data is generated based on the text information, the multimedia data including a plurality of multimedia segments, each of which corresponds to a plurality of text segments separated from the text information, the plurality of multimedia segments including a plurality of audio segments generated by recitation corresponding respectively to the plurality of text segments, and a plurality of video image segments each matching the plurality of text segments.

[0033] That is, as shown in FIG. 3, in this embodiment, the generated multimedia data consists of a plurality of multimedia segments with a segment granularity, and each multimedia segment includes at least three information types, respectively, namely, a text segment, an audio segment (the initial tone of the audio segment may be the first target tone selected in the text editing interface in the above embodiment), a video image segment, etc. (the video image segment may include a video stream consisting of consecutive pictures, and the consecutive pictures may correspond to pictures in the video stream matched in a pre-defined video library, and may also correspond to one or more pictures matched in a pre-defined image material library).

[0034] For example, as shown in FIG. 4, if the text information input by the user is “In today’s society, there are various types of women’s clothing, such as general Hanfu, fashion wear, sportswear, etc.”, after processing the text information, four multimedia segments A, B, C, and D are generated, where multimedia segment A includes text segment A1 “In today’s society, there are various types of women’s clothing”, audio segment A3 corresponding to text segment A1, and video image segment A2 matching with text segment A1; multimedia segment B includes text segment B1 “For example, general Hanfu”, audio segment B3 corresponding to text segment B1, and video image segment B2 matching with text segment B1; multimedia segment C includes text segment C1 “fashion wear”, audio segment C3 corresponding to text segment C1, and video image segment C2 matching with text segment C1; and multimedia segment D includes text segment D1 “sportswear, etc.”, audio segment D3 corresponding to text segment D1, and video image segment D2 matching with text segment D1.

[0035] As described above, in this embodiment, the multimedia data includes at least three information types. In order to meet the diversified editing needs for multimedia data, in one embodiment of the present disclosure, a multimedia editing interface is displayed for editing and manipulating the multimedia data, and the multimedia editing interface includes a first editing track, a second editing track, and a third editing track, the first editing track includes a plurality of first track segments, each of the plurality of first track segments is used to identify a plurality of text segments, the second editing track includes a plurality of second track segments, each of the plurality of second track segments is used to identify a plurality of video image segments, the third editing track includes a plurality of third track segments, each of the plurality of third track segments is used to identify a plurality of audio segments, and in order to facilitate visual representation of the multiple types of information segments corresponding to each multimedia data, the segments, the second track segments, and the third track segments are displayed to align with a timeline in the editing tracks, and the first track segments, the second track segments, and the third track segments of the first track respectively identify corresponding text segments, video image segments, and audio segments.

[0036] In an embodiment of the present disclosure, multimedia data is divided into multiple multimedia segments, and then each multimedia segment is divided into editing tracks corresponding to multiple information types, so that a user can not only edit a single multimedia segment, but also edit an information segment corresponding to an editing track in a single multimedia segment, thereby meeting the diversifying editing needs of users and ensuring the quality of the generated multimedia data.

[0037] It should be noted that the display manner of the multimedia editing interface is different in different application scenarios. In a possible implementation, as shown in FIG. 5, the multimedia editing interface may include a video playback area, an editing area, and an editing track display area, in which a first track segment corresponding to the first editing track, a second track segment corresponding to the second editing track, and a third track segment corresponding to the third editing track are displayed in order to align the timeline.

[0038] The editing area is used to display editing function controls corresponding to the currently selected information segment (the specific editing function controls can be set according to the needs of the experimental scene), the video playback area is used to display images and text information, etc. at the current playback point of the multimedia data (the editing track display area can display a reference line corresponding to the current playback point in a direction perpendicular to the time axis, the reference line is used to indicate the current playback position of the multimedia data, and the reference line can also be dragged, and the video playback area synchronously displays images and text information, etc. of the multimedia segment at the real-time position corresponding to the reference line, thereby facilitating the user to realize frame-by-frame browsing of the multimedia data based on the forward and backward movement of the reference line, etc.), the video playback area can further play the video playback control, and when the video playback control is triggered, it displays the current multimedia data, so that the user can visually know the playback effect of the current multimedia data.

[0039] Continuing with the scene shown in Fig. 4 as an example, in Fig. 5, the editing track display area displays four multimedia segments A, B, C, D, and A1A2A3B1B2B3C1C2C3D1D2D3 corresponding to the four multimedia segments A, B, C, D, so as to align the timeline. When the text segment A1 is selected, the editing area displays an editing interface corresponding to the text segment A1, which includes the editable A1, and editing controls such as font and character size. At this time, the video playback area displays the multimedia data at the position corresponding to the reference line of the current multimedia data, etc.

[0040] In addition, the multimedia editing interface of this embodiment may further include other editing tracks in addition to the above three editing tracks, and the number of the other editing tracks is not displayed, and each other editing track can be used to display information segments of other dimensions corresponding to the multimedia data. For example, when the other editing tracks include an editing track for editing the background sound of the multimedia data, that is, the multimedia editing interface may include a fourth editing track for identifying background audio data, so that in response to a trigger operation on the fourth editing track, the current background sound used by the fourth editing track is displayed in a preset background sound editing area (for example, the editing area described in the above embodiment), and replaceable candidate background sounds are displayed. The replaceable candidate background sounds can be displayed in any style, such as a label, in the background sound editing area, and the target background sound is updated and identified in the fourth editing track based on the target background sound generated by the user by modifying the current background sound based on the candidate background sounds in the background sound editing area.

[0041] As described above, the multimedia data processing method of the embodiment of the present disclosure divides multimedia data into multiple multimedia segments, and each multimedia segment has an editing track corresponding to each information type based on the information type contained therein, and information segments of a certain information type contained in the multimedia segment can be edited and modified in the editing track, thereby enriching the editing track corresponding to the multimedia data, meeting the diversifying editing needs of multimedia data, and improving the quality of the multimedia data.

[0042] Hereinafter, with reference to a specific embodiment, how to edit information segments of different information types of multimedia information segments corresponding to multimedia data will be described.

[0043] In one embodiment of the present disclosure, the text segments corresponding to the multimedia information segments can be independently edited and modified.

[0044] In this embodiment, as shown in FIG. 6, the step of independently editing and modifying a text segment corresponding to a multimedia information segment includes the following steps 601 to 602.

[0045] Step 601, in response to a user selecting a first target track segment in a first edit track, displays the currently identified text segment in the first target track segment in a text edit area.

[0046] In one embodiment of the present disclosure, in response to a user selecting a first target track segment in a first edit track, which may be one or more first target track segments, the currently identified text segment in the first target edit track segment is displayed in a text editing area, which may be located in the editing area described in the above embodiment. In addition to the currently identified text segment in the editable first target track segment, the text editing area may include other functional editing controls for the text segment, such as font editing controls, character size editing controls, etc.

[0047] Step 602 updates and identifies a target text segment in the first target track segment based on a target text segment generated by a user modifying the text segment currently displayed in the text edit area.

[0048] In this embodiment, the target text segment is updated and identified in the first target track segment based on the target text segment generated by the user modifying the text segment currently displayed in the text edit area.

[0049] For example, as shown in FIG. 7, if the text segment currently identified in the first target track segment selected by the user is “In today's society, there are various types of women's clothing”, and the user modifies the text segment currently displayed in the text editing area to “In today's society, there are various types of women's clothing, and there are many types”, the text segment can be updated in the first target track to “In today's society, there are various types of women's clothing, and there are many types”, thereby realizing the modification to a single text segment and meeting the need for modification to a single text segment of multimedia data.

[0050] In an embodiment of the present disclosure, in order to further improve the quality of the multimedia data and ensure matching between the image and the text, the image in the video image segment can be further synchronously updated according to the modification to the text segment. In this embodiment, in response to a text update operation on the target text segment in the first target track segment, a second target track segment corresponding to the first target track segment is determined in the second edit track, a target video image segment matching the target text segment is obtained, and the target video image segment is updated and identified in the second target track segment.

[0051] A semantic matching may be performed between the target text and a picture in a preset picture material library to determine a corresponding target video image, and then a target video segment may be generated based on the target video image, or a video segment matching the target text segment from a preset video segment material library may be directly determined as a target video image segment, etc., and the present invention is not limited thereto.

[0052] In one embodiment of the present disclosure, to ensure synchronization between text and speech, the speech segment may further be synchronously modified based on modifications to the text segment.

[0053] That is, in this embodiment, in response to a text update operation on the target text segment in the first target track segment, a third target track segment corresponding to the first target track segment is determined in the third edit track, the third track segment includes a voice segment corresponding to the text segment in the first target track segment, and a target voice segment corresponding to the target text segment is obtained, for example, by reciting the target text segment to obtain the target voice segment, and the target voice segment is updated and identified in the third target track segment, thereby realizing synchronization correction between voice and text.

[0054] Continuing to refer to FIG. 7, if the text segment currently identified in the first target track segment selected by the user is “In today's society, there are many types of women's clothing”, and the user modifies the text segment currently displayed in the text editing area to “In today's society, there are many types of women's clothing, and there are many types”, the text segment can be updated in the first target track to “In today's society, there are many types of women's clothing, and there are many types”, and a third target track segment corresponding to the first target track segment is determined in the third editing track, and the voice segment in the third target track segment is updated from “In today's society, there are many types of women's clothing” to “In today's society, there are many types of women's clothing, and there are many types”.

[0055] As described above, in the process of editing and modifying a text segment, the corresponding time length on the time axis of the modified text segment is different from the corresponding time length on the time axis of the text segment before the modification, so that in different application scenarios, different display processing can be performed on the editing track in the editing track display area based on the change in this time length.

[0056] In one embodiment of the present disclosure, when it is necessary to define a video image segment corresponding to a multimedia segment according to a scene as a main information segment whose time length cannot be changed, in order to ensure that the time length of the video image segment cannot be changed, when it is detected that the first updated time length corresponding to the target text segment in the first editing track does not match the time length corresponding to the text segment before modification, the second editing track is kept unchanged, that is, the time length of the corresponding video image segment is ensured not to be changed, and the first updated track segment corresponding to the first updated time length is displayed in the preset first candidate area.

[0057] Identify the target text segment in the first updated track segment, and the first candidate region may be located in other regions, such as the region above the text segment before modification, so that even if it is known that the first updated time length corresponding to the target text segment in the first edit track does not match the time length corresponding to the text segment before modification, the target text segment can be displayed in a form such as "track up", and the time length of the video image segment corresponding to the second edit track is not correspondingly modified, and the time length corresponding to the text information segment of other multimedia segments is not visually affected.

[0058] When it is found by detection that the third updated time length corresponding to the target audio segment in the third editing track does not match the time length corresponding to the audio segment before modification, the second editing track is maintained unchanged, a third updated track segment corresponding to the third updated time length is displayed in a second preset candidate area, and the target audio segment is identified in the third updated track segment. The second candidate area may be located in another area, such as a lower area of ​​the audio segment before modification, so that even when it is found that the third updated time length corresponding to the target audio segment in the third editing track does not match the time length corresponding to the audio segment before modification, the target audio segment can be displayed in a form such as "track down", and the time length of the video image segment corresponding to the second editing track is not correspondingly modified, and the time length corresponding to the audio segment of other multimedia information segments is not visually affected.

[0059] For example, as shown in FIG. 8, taking the scene shown in FIG. 7 as an example, the modified text segment “In today’s society, there are many different types of women’s clothing” obviously has a longer corresponding time length than the pre-modified text segment “In today’s society, there are many different types of women’s clothing”, so the modified text segment “In today’s society, there are many different types of women’s clothing” can be displayed above the pre-modified text segment, and the time length of the corresponding video image segment can be kept unchanged.

[0060] In this embodiment, the corresponding time length of the modified speech segment "In today's society, there are many different types of women's clothing" is obviously longer than that of the pre-modified speech segment "In today's society, there are many different types of women's clothing", so the modified speech segment "In today's society, there are many different types of women's clothing" can be displayed below the pre-modified speech segment, and the time length of the corresponding video image segment can be kept unchanged, thereby meeting the need that the time length of the video image segment in the corresponding scene cannot be changed.

[0061] In one embodiment of the present disclosure, when a video image segment corresponding to a multimedia segment needs to be synchronized with other information segments on a timeline according to a scene, in order to ensure the synchronization of the video image segment on the timeline, when it is detected that a first update time length corresponding to a target text segment in the first editing track does not match a time length corresponding to the text segment before modification, the length of the first target track segment is adjusted based on the first update time length, i.e., the length of the first target track segment is scaled at the original display position; similarly, when it is detected that a third update time length corresponding to a target audio segment in the third editing track does not match a time length corresponding to the audio segment before modification, the length of the third target track segment is adjusted based on the third update time length.

[0062] Furthermore, the length of the second target track segment corresponding to the first target track segment and the third target track segment in the second edit track is correspondingly adjusted, and the time axes of the adjusted first target track segment, the adjusted second target track segment and the adjusted third target track segment are aligned, thereby realizing the alignment of all information segments included in the multimedia segment on the timeline.

[0063] For example, as shown in FIG. 9, taking the scene shown in FIG. 7 as an example, the modified text segment “In today’s society, there are many different types of women’s clothing, and there are many different types” obviously has a longer corresponding time length than the pre-modified text segment “In today’s society, there are many different types of women’s clothing”, so the length of the first target track segment of the pre-modified text segment can be increased and displayed, and the modified text segment “In today’s society, there are many different types of women’s clothing” is displayed in the adjusted first target track segment.

[0064] In this embodiment, since the corrected speech segment "In today's society, there are many different types of women's clothing" obviously has a longer corresponding time length compared to the pre-corrected speech segment "In today's society, there are many different types of women's clothing", the length of the second target track segment of the pre-corrected speech segment can be increased and displayed, and the corrected speech segment "In today's society, there are many different types of women's clothing" is displayed in the adjusted second target track segment.

[0065] In order to achieve synchronization between the video image segment and other information segments, in this embodiment, the lengths of the second target track segments corresponding to the first target track segment and the third target track segment in the second edit track are correspondingly adjusted, and the time axes of the adjusted first target track segment, the adjusted second target track segment and the adjusted third target track segment are aligned.

[0066] In one embodiment of the present disclosure, the audio segments corresponding to the multimedia information segments can be independently edited and modified.

[0067] In this embodiment, as shown in FIG. 10, the steps of independently editing and modifying an audio segment corresponding to a multimedia information segment include the following steps 1001 to 1003.

[0068] Step 1001, in response to a user selecting a third target track segment in the third edit track, the third target track segment correspondingly identifies an audio segment that corresponds to the text segment displayed in the first target track segment.

[0069] In one embodiment of the present disclosure, in response to a user selecting a third target track segment in the third editing track, there may be one or more third target track segments, and the third target track segment correspondingly identifies an audio segment that corresponds to the text segment displayed in the first target track segment, i.e., in this embodiment, the audio segment can be edited independently.

[0070] Step 1002: In the preset audio editing area, the current tone used in the voice segment in the third target track segment is displayed, and alternative candidate tone colors are displayed.

[0071] The preset audio editing area in this embodiment may be located in the editing area described in the above embodiment, and the preset audio editing area displays the current tone used in the voice segment in the third target track segment, and displays the possible alternative tones. As shown in FIG. 11, the candidate alternative tones can be displayed in any style, such as a label form, for example, the labels of the candidate alternative tones, such as "uncle", "girl", "old man", etc., can be displayed, and the user can select the candidate tones by triggering the corresponding label.

[0072] Step 1003: updating and identifying a target speech segment in a third target track segment based on a second target timbre generated by the user modifying a current timbre based on a candidate timbre in the audio editing area, the target speech segment being a speech segment generated by reciting the text segment identified in the first target track segment using the second target timbre.

[0073] In this embodiment, the user can modify the current tone by triggering a candidate tone, and the current tone is modified to the triggered candidate tone, i.e., the second target tone, thereby modifying the tone of the voice segment in the third track segment, and meeting the user's need to modify the tone of a certain voice segment, for example, the user can modify multiple voice segments corresponding to the third track segment to different tones, thereby realizing an interesting voice playback effect.

[0074] As described above, the multimedia data processing method of the embodiment of the present disclosure can flexibly and independently edit and modify text segments, audio segments, etc. corresponding to multimedia segments, further meeting the diversifying editing needs of multimedia data and improving the quality of multimedia data.

[0075] To achieve the above embodiment, the present disclosure further provides an apparatus for processing multimedia data.

[0076] 12 is a structural schematic diagram of a multimedia data processing device provided by an embodiment of the present disclosure, which can be realized by software and / or hardware, and can be generally integrated into an electronic device to process multimedia data. As shown in FIG. 12, the device includes: a receiving module 1210, a generating module 1220 and a display module 1230; The receiving module 1210 is used for receiving text information input by a user; The generating module 1220 is used to generate multimedia data based on the text information in response to the processing instruction for the text information; The display module 1230 is used for displaying a multimedia editing interface for editing and manipulating multimedia data; The multimedia data includes a plurality of multimedia segments, each of which corresponds to a plurality of text segments segmented from the text information, the plurality of multimedia segments including a plurality of audio segments generated by reciting the plurality of text segments, each of which corresponds to the plurality of text segments, and a plurality of video image segments, each of which matches the plurality of text segments; The multimedia editing interface includes a first edit track, a second edit track, and a third edit track, the first edit track includes a plurality of first track segments, each of the plurality of first track segments being used to identify a plurality of text segments, the second edit track includes a plurality of second track segments, each of the plurality of second track segments being used to identify a plurality of video image segments, the third edit track includes a plurality of third track segments, each of the plurality of third track segments being used to identify a plurality of audio segments, and the first track segments, the second track segments, and the third track segments, which are timeline aligned in the edit tracks, identify corresponding text segments, video image segments, and audio segments, respectively.

[0077] Optionally, the receiving module is specifically: Receiving text information entered by a user in a text area; and / or It is used to receive link information input by a user in the link area, identify the link information to obtain text information of the corresponding page, and display it in the text area for the user to edit.

[0078] Optionally, the method further includes a first display module, a second display module, a timbre determination module, and a voice segment acquisition module; the first display module is used for displaying the tone selection entry control; the second display module is adapted to display a candidate tone menu in response to a user's triggering of the tone selection entry control, the candidate tone menu including candidate tones and preview controls corresponding to the candidate tones; The tone color determination module is used for determining a first target tone color based on a selection operation of the user on the candidate tone color menu; The voice segment acquisition module is used for acquiring a plurality of voice segments generated by reciting a plurality of text segments obtained by dividing the text information according to the first target tone.

[0079] Optionally, further comprising a third display module and a text segment editing module; a third display module for displaying in the text editing area a currently identified text segment in the first target track segment in response to a user selecting a first target track segment in the first edit track; The text segment editing module is used to update and identify a target text segment in the first target track segment based on a target text segment generated by a user modifying a text segment currently displayed in the text editing area.

[0080] Optionally, the method further includes a track segment determination module and an audio segment acquisition module; the track segment determining module is used to determine a third target track segment in the third edit track corresponding to the first target track segment in response to a text update operation on the target text segment in the first target track segment; The voice segment obtaining module is used for obtaining a target voice segment corresponding to the target text segment, and updating and identifying the target voice segment with the third target track segment.

[0081] Selectively, when detecting that a first updated time length corresponding to the target text segment in the first edit track does not match a time length corresponding to the text segment before modification, maintaining the second edit track unchanged and displaying a first updated track segment corresponding to the first updated time length in a first preset candidate area, where the first updated track segment identifies the target text segment; When it is detected that the third updated time length corresponding to the target audio segment in the third edit track does not match the time length corresponding to the audio segment before the correction, the second edit track is kept unchanged, and a third updated track segment corresponding to the third updated time length is displayed in the preset second candidate area, and the first time length display processing module is used for identifying the target audio segment in the third updated track segment.

[0082] Selectively, adjusting a length of the first target track segment based on the first updated time length when detecting that the first updated time length corresponding to the target text segment in the first edit track does not match the time length corresponding to the unmodified text segment; adjusting a length of the third target track segment based on the third updated time length when detecting that the third updated time length corresponding to the target audio segment in the third edit track does not match the time length corresponding to the audio segment before the modification; and a second time length display processing module used for correspondingly adjusting lengths of second target track segments corresponding to the first target track segment and the third target track segment in the second edit track, and aligning time axes of the adjusted first target track segment, the adjusted second target track segment and the adjusted third target track segment.

[0083] Selectively, determining a second target track segment in the second edit track corresponding to the first target track segment in response to a text update operation on the target text segment in the first target track segment; The video image updating module is used for obtaining a target video image segment matching the target text segment, and updating and identifying the target video image segment with the second target track segment.

[0084] Selectively, responsive to a user selecting a third target track segment in the third edit track, the third target track segment correspondingly identifying an audio segment that corresponds to the text segment displayed in the first target track segment; In the preset audio editing area, displaying the current tone used in the voice segment in the third target track segment and displaying candidate tones that can be substituted; The audio editing system further includes a timbre updating module for updating and identifying a target speech segment in a third target track segment based on a second target timbre generated by a user by modifying a current timbre based on a candidate timbre in an audio editing area, the target speech segment being a speech segment generated by reciting the text segment identified in the first target track segment using the second target timbre.

[0085] Optionally, the multimedia editing interface includes: a fourth edit track used to identify background audio data; and a background sound display module for displaying a current background sound used by the fourth editing track in a preset background sound editing area in response to a trigger operation on the fourth editing track, and displaying a candidate background sound that can be substituted; The background sound update processing module is used for updating and identifying a target background sound in the fourth edit track based on a target background sound generated by a user by modifying the current background sound based on the candidate background sounds in the background sound edit area.

[0086] The multimedia data processing device provided by the embodiments of the present disclosure can execute the multimedia data processing method provided by any embodiment of the present disclosure, and has corresponding functional modules and beneficial effects for executing the method, which will not be described repeatedly here.

[0087] To realize the above embodiment, the present disclosure further provides a computer program product, including computer programs / instructions, which realize the multimedia data processing method of the above embodiment when executed by a processor.

[0088] FIG. 13 is a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure.

[0089] Hereinafter, specific reference is made to FIG. 13, which shows a structural schematic diagram of an electronic device 1300 suitable for implementing the embodiments of the present disclosure. The electronic device 1300 of the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet PCs), PMPs (portable multimedia players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desk computers. The electronic device shown in FIG. 13 is merely an example and should not impose any limitations on the functions and scope of use of the embodiments of the present disclosure.

[0090] 13, the electronic device 1300 may include a processor (e.g., a central processor, a graphic processor, etc.) 1301, which can execute various appropriate operations and processes based on a program stored in a read-only memory (ROM) 1302 or a program loaded from a memory 1308 into a random access memory (RAM) 1303. The RAM 1303 further stores various programs and data necessary for the operation of the electronic device 1300. The processor 1301, the ROM 1302, and the RAM 1303 are connected to each other via a bus 1304. An input / output (I / O) interface 1305 is also connected to the bus 1304.

[0091] Typically, input devices 1306 including, for example, a touch panel, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc., output devices 1307 including, for example, a liquid crystal display (LCD), speaker, oscillator, etc., memory 1308 including, for example, a tape, hard disk, etc., and communication devices 1309 may be connected to the I / O interface 1305. The communication devices 1309 may allow the electronic device 1300 to communicate wirelessly or wired with other devices to exchange data. Although the electronic device 1300 is shown in FIG. 13 with various devices, it should be understood that it is not required to implement or include all the devices shown. Alternatively, more or fewer devices may be implemented or included.

[0092] In particular, according to an embodiment of the present disclosure, the above process described with reference to the flowcharts may be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes program code for performing the method illustrated in the flowchart. In such an embodiment, the computer program may be downloaded and installed from a network by the communication device 1309, or installed from the memory 1308, or installed from the ROM 1302. When the computer program is executed by the processor 1301, it performs the above-mentioned functions limited to the multimedia data processing method of the embodiment of the present disclosure.

[0093] It should be noted that the above computer readable medium of the present disclosure may be a computer readable signal medium or a computer readable storage medium or any combination of the above. The computer readable storage medium may be, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer readable storage medium may include, but are not limited to, an electrical connection having one or more conductors, a portable computer magnetic disk, a hard disk, a random access memory (RAM), a read only memory (ROM), an erasable programmable read only memory (EPROM or flash memory), optical fiber, a portable compact disk read only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer readable storage medium may be any tangible medium that contains or stores a program, which may be used by or in combination with an instruction execution system, apparatus, or device. In the present disclosure, the computer readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which has computer readable program code. Such a propagated data signal may take a number of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may transmit, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained in the computer-readable medium may be transmitted by any suitable medium, including, but not limited to, wire, cable, RF (radio frequency), etc., or any suitable combination of the above.

[0094] In some embodiments, clients and servers may communicate using any network protocol now known or later developed, such as HTTP (HyperText Transfer Protocol), and may be connected to each other via any form or medium of digital data communication (e.g., a communications network). Examples of communications networks include a local area network ("LAN"), a wide area network ("WAN"), a network of networks (e.g., the Internet), and an end-to-end network (e.g., an ad hoc end-to-end network), and any network now known or later developed.

[0095] The computer-readable medium may be included in the electronic device, or may exist separately and not be assembled into the electronic device.

[0096] When one or more programs are loaded on the computer-readable medium and the one or more programs are executed by the electronic device, the electronic device A method for editing and manipulating the multimedia data includes receiving text information input by a user, and generating multimedia data based on the text information in response to a processing instruction for the text information, and displaying a multimedia editing interface for editing and manipulating the multimedia data, wherein the multimedia editing interface includes a plurality of multimedia segments, each of which corresponds to a plurality of text segments divided from the text information, and the plurality of multimedia segments includes a plurality of audio segments generated by recitation corresponding to the plurality of text segments respectively, and a plurality of video image segments respectively matching the plurality of text segments, and the multimedia editing interface includes a first editing track, a second editing track, and a third editing track, and a first track segment corresponding to the first editing track, a second track segment corresponding to the second editing track, and a third track segment corresponding to the third editing track, in which a timeline is aligned in the editing tracks, identify corresponding text segments, video image segments, and audio segments, respectively, and an embodiment of the present disclosure can enrich the editing tracks corresponding to the multimedia data, meet the diversifying editing needs of the multimedia data, and improve the quality of the multimedia data.

[0097] The electronic device can create computer program code for performing the operations of the present disclosure in one or more programming languages ​​or combinations thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, C++, and also including conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computer, partially on the user computer, as a separate software package, partially on the user computer and partially on a remote computer, or entirely on a remote computer or server. In the context of a remote computer, the remote computer may be connected to the user computer via any network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet Service Provider).

[0098] The flowcharts and block diagrams in the drawings illustrate possible organizational structures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowcharts or block diagrams may represent a module, program segment, or part of code, which includes one or more executable instructions for implementing a specified logical function. It should be noted that in some alternative implementations, the functions depicted in the boxes may occur in a different order than depicted in the drawings. For example, two consecutive boxes may actually be essentially executed in parallel, but may be executed in the opposite order in some cases, as determined by the functions involved. It should be noted that each box in the block diagrams and / or flowcharts, and combinations of boxes in the block diagrams and / or flowcharts, may be implemented in a dedicated hardware-based system that performs the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0099] The units related to the embodiments of the present disclosure may be implemented in a software manner or in a hardware manner, and the names of the units do not constitute limitations on the units themselves in certain circumstances.

[0100] The functions described herein above may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary hardware logic components that may be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), etc.

[0101] In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain or store a program for use by or in combination with an instruction execution system, device, or apparatus. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the above. More specific examples of machine-readable storage media include one or more wire-based electrical connections, portable computer disks, hard disks, random access memories (RAMs), read-only memories (ROMs), erasable programmable read-only memories (EPROMs or flash memories), optical fibers, portable compact disk read-only memories (CDs), optical storage devices, magnetic storage devices, or any suitable combination of the above.

[0102] The above description is merely a description of the preferred embodiments of the present disclosure and the technical principles used. As should be understood by those skilled in the art, the disclosure scope of the present disclosure is not limited to the technical solutions based on the specific combination of the above technical features, but should also include other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the idea of ​​the above disclosure. For example, the above features and the technical features having similar functions (but not limited to) disclosed in the present disclosure may be substituted for each other to form the technical solutions.

[0103] Also, although operations have been described in a particular order, this should not be understood as requiring that these operations be performed in the particular order shown, or in a sequential order. In some cases, multitasking and parallel processing may be advantageous. Similarly, although the above description includes some specific implementation details, these should not be construed as limitations on the scope of the disclosure. Certain features that are described in the context of a single embodiment may also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment may also be implemented in multiple embodiments alone or in any suitable subcombination.

[0104] Although the present subject matter has been described in particular language of structural features and / or method logic operations, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or operations described above. Rather, the specific features and operations described above are merely example forms for implementing the claims.

Claims

1. A method for processing multimedia data, comprising the steps of: receiving text information entered by a user; in response to a processing instruction for the text information, generating multimedia data based on the text information, and displaying a multimedia editing interface for editing and manipulating the multimedia data; the multimedia data includes a plurality of multimedia segments, each of which corresponds to a plurality of text segments divided from the text information, the plurality of multimedia segments including a plurality of audio segments generated by reciting the plurality of text segments, each of which corresponds to the plurality of text segments, and a plurality of video image segments, each of which matches the plurality of text segments; a first edit track including a first track segment, the first track segment including a second track segment, the first track segment including a third edit track, the first edit track including a first track segment, the first track segment including a second track segment, the second track segment including a third track segment, the second track segment including a third track segment, the third track segment including a third track segment, the third track segment including a third track segment, the third track segment including a third track segment, the third track segment including a third track segment, the third track segment including a third track segment, the third track segment including a third track segment, the third track segment

2. The step of receiving text information entered by a user includes: receiving text information entered by a user in the text area; and / or The method of claim 1 , comprising the steps of receiving link information input by a user in a link area, identifying the link information to obtain text information of a corresponding page, and displaying the text information in the text area for the user to edit.

3. displaying a tone selection entry control; displaying a candidate timbre menu in response to a user triggering the timbre selection entry control, the candidate timbre menu including candidate timbres and preview controls corresponding to the candidate timbres; determining a first target timbre based on a selection operation by the user on the candidate timbre menu; The method of claim 1 or 2, further comprising the step of obtaining a plurality of speech segments generated by reciting a plurality of text segments obtained by dividing the text information based on the first target tone.

4. displaying a currently identified text segment in the first target track segment in a text editing area in response to the user selecting a first target track segment in the first edit track; 4. The method of claim 1, further comprising: updating and identifying the target text segment in the first target track segment based on a target text segment generated by the user modifying the currently displayed text segment in the text editing area.

5. determining a third target track segment in the third edit track corresponding to the first target track segment in response to a text update operation on the target text segment in the first target track segment; 5. The method of claim 4, further comprising the steps of: obtaining a target speech segment corresponding to the target text segment; and updating and identifying the target speech segment with the third target track segment.

6. if detecting that a first updated time length corresponding to the target text segment in the first edit track does not match a time length corresponding to the text segment before modification, maintaining the second edit track unchanged and displaying a first updated track segment corresponding to the first updated time length in a first preset candidate area, identifying the target text segment in the first updated track segment; 6. The method of claim 5, further comprising the steps of: when detecting that a third updated time length in the third edit track corresponding to the target audio segment does not match a time length corresponding to the audio segment before modification, maintaining the second edit track unchanged and displaying a third updated track segment corresponding to the third updated time length in a second preset candidate area, and identifying the target audio segment with the third updated track segment.

7. adjusting a length of the first target track segment based on the first updated time length when detecting that a first updated time length corresponding to the target text segment in the first edit track does not match a time length corresponding to the unmodified text segment; adjusting a length of the third target track segment based on the third updated time length when detecting that a third updated time length corresponding to the target audio segment in the third edit track does not match a time length corresponding to the audio segment before the correction; 6. The method of claim 5, further comprising the steps of: correspondingly adjusting lengths of second target track segments corresponding to the first and third target track segments in the second edit track; and aligning time axes of the adjusted first target track segment, the adjusted second target track segment, and the adjusted third target track segment.

8. determining a second target track segment in the second edit track corresponding to the first target track segment in response to a text update operation on the target text segment in the first target track segment; 5. The method of claim 4, further comprising the steps of obtaining a target video image segment that matches the target text segment, and updating and identifying the target video image segment with the second target track segment.

9. responsive to the user selecting a third target track segment in the third edit track, the third target track segment correspondingly identifying an audio segment that corresponds to a text segment displayed in the first target track segment; displaying a current tone color used in the voice segment of the third target track segment and a candidate tone color that can be substituted in the preset audio editing area; The method according to any one of claims 1 to 8, further comprising: updating and identifying a target speech segment in the third target track segment based on a second target timbre generated by the user modifying the current timbre based on the candidate timbre in the audio editing area, the target speech segment being a speech segment generated by reciting a text segment identified in the first target track segment using the second target timbre.

10. The multimedia editing interface includes: a fourth edit track used to identify background audio data; In response to a trigger operation on the fourth editing track, a current background sound used by the fourth editing track is displayed in a preset background sound editing area, and a candidate background sound that can be substituted is displayed; The method according to any one of claims 1 to 9, further comprising updating and identifying the target background sound in the fourth edit track based on a target background sound generated by the user by modifying the current background sound based on the candidate background sound in the background sound edit area.

11. A multimedia data processing device, comprising: a receiving module for receiving text information input by a user; a generation module responsive to processing instructions for the text information for generating multimedia data based on the text information; a display module for displaying a multimedia editing interface for editing and manipulating the multimedia data; the multimedia data includes a plurality of multimedia segments, each of which corresponds to a plurality of text segments divided from the text information, the plurality of multimedia segments including a plurality of audio segments generated by reciting the plurality of text segments, each of which corresponds to the plurality of text segments, and a plurality of video image segments, each of which matches the plurality of text segments; The multimedia editing interface includes a first edit track, a second edit track, and a third edit track, the first edit track including a plurality of first track segments, each of the plurality of first track segments being used to identify the plurality of text segments, the second edit track including a plurality of second track segments, each of the plurality of second track segments being used to identify the plurality of video image segments, the third edit track including a plurality of third track segments, each of the plurality of third track segments being used to identify the plurality of audio segments, and the first track segment, the second track segment, and the third track segment, in which a timeline is aligned in the edit tracks, identify corresponding text segments, video image segments, and audio segments, respectively.

12. An electronic device, A processor; a memory configured to store executable instructions; 11. An electronic device, wherein the processor is arranged to implement the method for processing multimedia data according to any one of claims 1 to 10 by reading the executable instructions from the memory and executing the executable instructions.

13. A computer-readable storage medium having a computer program stored thereon, the computer program being capable of being used to execute the method for processing multimedia data according to any one of claims 1 to 10.

Citation Information

Patent Citations

  • Video generation method and device, computer equipment and storage medium

    CN114513706A

  • System, method, and program for content generation

    JP2005062420A

  • Voice-attached animation production / distribution service system

    JP2011082789A

  • Method and apparatus for locating video playing node, device and storage medium

    US20220044703A1