Video generation method and apparatus, electronic device and storage medium

Abstract

Embodiments of the disclosure provide a video generation method and apparatus, an electronic device and a storage medium. The video generation method comprises: in response to a first user operation, obtaining input information used for representing target semantics, wherein the input information is formed by at least one text statement; converting the input information into a dialogue text representing the target semantics, wherein the dialogue text is formed by at least two dialogic sentence-based text statements; obtaining a character video material corresponding to the text statements in the dialogue text, and on the basis of the character video material, generating a target video for displaying the target semantics. By converting the input information input by the user into the dialogue text formed by the dialogic sentence-based text statements, and then obtaining the corresponding character video material on the basis of the dialogue text and generating the target video representing the target semantics, rapid and automatic generation of a video product is achieved, the video production efficiency is improved, and the production difficulty and the production costs are reduced.

Description

Video generation method, device, electronic device and storage medium

[0001] This application claims priority to Chinese Patent Application No. 202311236891.6 filed on September 22, 2023, and the contents of the above-mentioned Chinese patent application disclosure are hereby incorporated by reference in their entirety as a part of this application. Technical Field

[0002] The embodiments of the present disclosure relate to a video generation method, device, electronic device, and storage medium. Background Art

[0003] Currently, various video platforms have attracted a large number of users with their rich and diverse video content, including a large number of original videos generated by users.

[0004] In the existing technology, the production process of the above original videos usually requires users to go through multiple production steps, including script design, content arrangement, and video production, before the video work is completed. The above video production solutions in the existing technology have problems such as low video production efficiency, high cost, and great difficulty.

[0005] Summary of the Invention

[0006] Embodiments of the present disclosure provide a video generation method, device, electronic device, and storage medium.

[0007] In a first aspect, an embodiment of the present disclosure provides a video generation method, comprising:

[0008] In response to a first user operation, input information for representing target semantics is obtained, the input information consisting of at least one text sentence; the input information is converted into a dialogue text representing the target semantics, the dialogue text consisting of at least two text sentences based on a dialogue sentence pattern; character video material corresponding to the text sentences in the dialogue text is obtained, and a target video for displaying the target semantics is generated based on the character video material.

[0009] In a second aspect, an embodiment of the present disclosure provides a video generation device, including:

[0010] a response module, configured to obtain input information for representing target semantics in response to a first user operation, wherein the input information consists of at least one text sentence;

[0011] a processing module, configured to convert the input information into a dialogue text representing the target semantics, wherein the dialogue text is composed of at least two text sentences based on a dialogue sentence pattern;

[0012] A generation module is used to obtain role video materials corresponding to text sentences in the dialogue text, and generate a target video for displaying the target semantics based on the role video materials.

[0013] In a third aspect, an embodiment of the present disclosure provides an electronic device, including: a processor and a memory;

[0014] The memory stores computer-executable instructions;

[0015] The processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the video generation method as described in the first aspect and various possible designs of the first aspect.

[0016] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the video generation method described in the first aspect and various possible designs of the first aspect is implemented.

[0017] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the video generation method described in the first aspect and various possible designs of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0019] FIG1 is a diagram of an application scenario of the video generation method provided by an embodiment of the present disclosure;

[0020] FIG2 is a flow chart of a video generation method according to an embodiment of the present disclosure;

[0021] FIG3 is a schematic diagram of a text structure of a first text provided by an embodiment of the present disclosure;

[0022] FIG4 is a flowchart of a specific implementation of step S103 in the embodiment shown in FIG2 ;

[0023] FIG5 is a schematic diagram of a process for configuring role video materials according to an embodiment of the present disclosure;

[0024] FIG6 is a second flow chart of a video generation method provided by an embodiment of the present disclosure;

[0025] FIG7 is a schematic diagram of a process for generating a target video according to an embodiment of the present disclosure;

[0026] FIG8 is a schematic diagram of a process for obtaining a character video material according to an embodiment of the present disclosure;

[0027] FIG9 is a structural block diagram of a video generating device provided by an embodiment of the present disclosure;

[0028] FIG10 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure;

[0029] FIG11 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0030] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure more clear, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present disclosure without making any creative efforts shall fall within the scope of protection of the present disclosure.

[0031] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0032] The following explains the application scenarios of the embodiments of the present disclosure:

[0033] FIG1 is a diagram of an application scenario of the video generation method provided by an embodiment of the present disclosure. The video generation method provided by an embodiment of the present disclosure can be applied to applications with video production and video generation functions, and more specifically, can be applied to application scenarios in which videos are generated using text input or voice input. The execution subject of this embodiment can be a terminal device running the above-mentioned application with video generation function, or a server running the server corresponding to the above-mentioned application, or other electronic devices that perform similar functions. Referring to FIG1 , taking a terminal device as an example, after running a video editing application, the terminal device receives the text input by the user (represented by "X" in the figure) and displays it in a text box in the video generation interface. The above text can be entered into the text box by the user through an input method, or the terminal device can generate the corresponding text by recognizing the user's voice and fill it into the text box, which can be set as needed. Afterwards, in response to a trigger operation, such as clicking the "Video Generation" control in the video generation interface, the terminal device generates a target video that matches the content described by the text based on the text in the text box, and displays it in a video preview window in the video generation interface. The user can then further preview, modify, or generate the final video work.

[0034] In existing technologies, the video production process typically requires users to go through multiple stages, including script design, content arrangement, and video production, before a video work can be created. This is because the content presented by the video, as a carrier, needs to be designed and arranged in advance. Therefore, the video production process includes at least two stages: designing the video content and converting the video content into video images. Existing technologies, however, can only automatically generate static images based on descriptive information. For example, if the descriptive term is "warm spring," a pre-trained generative model can generate the corresponding image. However, the video generation process is generated from multiple frames of coherent images, requiring continuity between the frames. Furthermore, the content expressed in a video is richer than that of a single frame. This makes it impossible to directly convert a typical monologue text into a video in existing technologies. This requires manual production of single-frame images and then merging them into a video, leading to a series of problems such as low video production efficiency, high cost, and difficulty.

[0035] The embodiments of the present disclosure provide a video generation method to solve the above problems.

[0036] The video generation method, device, electronic device, and storage medium provided in this embodiment obtain input information representing target semantics in response to a first user operation, the input information consisting of at least one text sentence; convert the input information into dialogue text representing the target semantics, the dialogue text consisting of at least two text sentences based on a dialogue sentence pattern; obtain character video material corresponding to the text sentences in the dialogue text, and generate a target video representing the target semantics based on the character video material. By converting the user input information into dialogue text consisting of text sentences based on a dialogue sentence pattern, character planning for the video content is achieved, and then corresponding character video material is obtained based on the dialogue text, and a target video representing the target semantics is generated, thereby achieving rapid and automated generation of video works, while ensuring that the generated target video accurately expresses the target semantics corresponding to the first text, thereby improving the efficiency of video production and reducing production difficulty and cost.

[0037] Referring to FIG2 , FIG2 is a flow chart of a video generation method provided by an embodiment of the present disclosure. The method of this embodiment can be applied in a terminal device, and the video generation method includes:

[0038] Step S101: In response to a first user operation, input information for representing target semantics is acquired, where the input information consists of at least one text sentence.

[0039] For example, referring to the application scenario diagram shown in FIG1 , the execution subject of this embodiment of the method is a terminal device, such as a smartphone. After running a video editing application used to implement the method of this embodiment, the terminal device obtains first text by detecting a first user operation applied to a video generation interface. The first user operation may be, for example, entering text into a text box in the video generation interface, or entering text into a text box in the video generation interface using a voice input method. The terminal device then obtains input information representing the target semantics by examining the text content in the text box.

[0040] Furthermore, the input information is a text composed of at least one text sentence, wherein, illustratively, the text sentence constituting the input information is a sentence based on a monologue sentence pattern. A monologue sentence pattern is a sentence spoken by the same character object, corresponding to a complete target semantics, and expressing a complete meaning. Corresponding to this is a dialogue sentence pattern, that is, a sentence spoken by two or more character objects, which will be described in detail later. The following is a more specific example of the input information, which will be introduced in conjunction with the following example:

[0041] "The roads in the city are becoming increasingly congested, especially during rush hour, which is simply a torture. In comparison, commuting by bicycle is becoming a smarter choice."

[0042] Based on the above example, the input information includes two (paragraphs of) text sentences, namely "Now the roads in the city are getting more and more congested, especially during rush hours, which is simply a torture" and "In comparison, riding a bicycle to and from get off work is becoming a smarter choice." The above two text sentences are both implemented based on monologue sentences, that is, they are stated by the same role object. The complete semantics jointly represented by the above two text sentences is the target semantics of the input information, where the target semantics can be represented by feature identifiers, feature arrays or feature matrices, and examples are not given here one by one. Optionally, in another possible implementation method, the text sentences constituting the input information are sentences based on dialogue sentences. The dialogue text generated by processing the input information is equivalent to the optimization of the text sentences based on dialogue sentences in the input information, and this situation will not be repeated.

[0043] Furthermore, in one possible implementation, the input information includes a first text, which includes a monologue and a prompt word. The monologue is used to describe the target semantics, and the prompt word is used to describe the video content format of the target video displaying the target semantics. Figure 3 is a schematic diagram of the text structure of a first text provided by an embodiment of the present disclosure. As shown in Figure 3, the complete text content of the first text is:

[0044] "The roads in the city are becoming increasingly congested, especially during rush hour, which is simply a torture. By comparison, commuting by bicycle is becoming a smarter option." % Convert the above text into a video of a conversation between two people, and add emoji descriptions.

[0045] Among them, "The city's roads are becoming increasingly congested, especially during rush hour, which is simply a torture. In comparison, commuting by bicycle is becoming a smarter choice." is a monologue sentence; and "Convert the above text into a video of a conversation between two people, and add facial expressions" is a prompt sentence. The monologue sentence and the prompt sentence are separated by a separator "%". Of course, it is understood that the above separator is only exemplary, and other methods and rules can also be used to distinguish monologue sentences from prompt sentences, such as using specific characters, punctuation marks, or paragraph separators. These can be set as needed and will not be detailed here.

[0046] Referring to the above example, the monologue sentence is used to carry and describe the target semantics, while the prompt word sentence is used to describe the video content format of the target video displaying the target semantics. In subsequent steps, for example, the terminal device can use a generative model to generate a corresponding target video according to the video format represented by the prompt word sentence, thereby achieving control over the presentation format of the target video content. The monologue sentence and prompt word sentence in the first text can be obtained, for example, by splitting and parsing the first text using separation markers. The specific implementation method is not repeated here.

[0047] Step S102: converting the input information into a dialogue text representing the target semantics, where the dialogue text is composed of at least two text sentences based on a dialogue sentence pattern.

[0048] After obtaining the first text, semantic extraction is performed on the first text to obtain the target semantics. A preset language model is then used to generate a dialogue text that also expresses the target semantics, i.e., the second text. The dialogue text is composed of at least two text sentences based on a dialogue sentence structure. The following is a more specific example of a dialogue text generated from the first text, which will be described in conjunction with the following example:

[0049] “A: Have you noticed that the roads in the city are getting more and more congested?

[0050] B: Indeed, I feel that too.

[0051] A: Especially during rush hour, it is simply a torture.

[0052] B: It took me a long time to get home from get off work yesterday.

[0053] A: Maybe riding a bicycle to get off work would be a smarter choice.

[0054] B: That sounds like a good idea.”

[0055] Based on the above example, the dialogue text consists of six (segmented) text sentences, each of which is stated by two character objects, A and B, i.e., they are implemented using text sentences based on dialogue sentences. The complete semantics collectively represented by these six text sentences is the same as the target semantics expressed by the first text, i.e., a discussion about traffic congestion. As shown in the above example, the dialogue text expresses the same target semantics as the first text, but the text structure has changed, converting the first text into a text structure more suitable for generating the target video. The above process can be implemented based on a pre-trained language model.

[0056] Among them, after generating the dialogue text, the terminal device can output it, for example, display it in the video generation interface to show it to the user; it can also not output it, but only use it to execute subsequent video generation steps. It can be set as needed without specific restrictions.

[0057] The step of this embodiment, i.e., the step of converting the sentence structure of the first text, converts the text sentence stated by the same role object into the text sentence stated by multiple role objects. The processing of the first text by this step is the process of "procedural" conversion of the target semantics represented by the first text, which is equivalent to organizing the video content of the target video to be generated, that is, dividing the video content into frames. Therefore, after the first text is converted into a dialogue text in "dialogue form", the dialogue text carrying the target semantics has the basis for generating a video, that is, the dialogue text is more likely to generate corresponding multiple content-coherent "video frames", thereby better achieving the purpose of generating the target video and improving the video quality of the target video.

[0058] Step S103: Acquire role video materials corresponding to text sentences in the dialogue text, and generate a target video for displaying target semantics based on the role video materials.

[0059] Exemplarily, after obtaining the dialogue text, the dialogue text is further processed to obtain character video material corresponding to the dialogue text. The character video material may include one or more of images, text, sequence frames, music, and video clips. In one possible implementation, each dialogue sentence in the dialogue text may be processed to generate a set of character video materials corresponding to each dialogue sentence. Each set of character video materials may include one or more of the character video materials listed above. Specifically, for example, a dialogue sentence may correspond to a background image, two sticker images, and a piece of music. In one possible implementation, feature extraction may be performed on the text sentences to obtain a semantic matrix representing their semantics. This matrix is ​​then mapped to pre-generated character video material to obtain the target character video material corresponding to the dialogue text. In another possible implementation, feature extraction may be performed on the text sentences to obtain a semantic matrix representing their semantics. Based on the semantic matrix, generation may be performed, such as generating corresponding speech, images, or music melodies, to obtain the character video material corresponding to the dialogue text.

[0060] Among them, since the dialogue text corresponds to at least two text sentences of the dialogue sentence pattern, at least two corresponding character video materials can be obtained. Then, the playback timing of the corresponding character video materials is set according to the sequence of the text sentences in the dialogue text, thereby generating the target video.

[0061] In a possible implementation, as shown in FIG4 , a specific implementation of step S103 includes:

[0062] Step S1031: Obtain layer information corresponding to the character video material.

[0063] Step S1032: Based on the layer information, the character video material is configured to the corresponding video editing track.

[0064] Step S1033: triggering video synthesis for each video editing track to obtain the target video.

[0065] Exemplarily, first, the layer information corresponding to each character video material is obtained. The layer information represents the level of the video editing track corresponding to the character video material, wherein, exemplarily, the image on the high-level video editing track will occlude the image on the low-level video editing track. In one possible implementation, the layer information includes a layer identifier, and layer identifier #1 represents that the level of the video editing track where the corresponding character video material is located is L1; and layer identifier #2 represents that the level of the video editing track where the corresponding character video material is located is L2. Afterwards, according to the layer information corresponding to each character video material, each character video material is configured on the corresponding video editing track. Furthermore, after the user triggers the corresponding generation control, the terminal device triggers the video synthesis for each video editing track, realizes the synthesis of the character video materials on multiple video editing tracks, and obtains the target video. Among them, exemplarily, the image information corresponding to the character video material can be the preset attribute information bound to the character video material. After determining the character video material, the layer information can be obtained by reading the material attributes of the character video material; in another implementation method, the material type of the character video material can also be judged, and based on the material type, the layer information corresponding to the character video material can be obtained. The specific implementation method can be set according to needs and is not limited here.

[0066] Furthermore, in a possible implementation method, the video generation interface includes a video editing track. Figure 5 is a schematic diagram of a process for configuring character video materials provided by an embodiment of the present disclosure. As shown in Figure 5, after the user clicks the first generation control in the video generation interface, the terminal device generates a corresponding dialogue text based on the first text entered in the text box, and obtains the corresponding character video material based on each text sentence in the dialogue text, such as the character video material P1, character video material P2, character video material P3, character video material P4 shown in the figure, and the corresponding playback timing of each character video material. Afterwards, based on the layer information and corresponding playback timing of the character video material P1, character video material P2, character video material P3, and character video material P4, the character video material P1, character video material P2, character video material P3, and character video material P4 are respectively configured to corresponding positions on the video editing track. For example, as shown in the figure, the character video material P1 and character video material P2 are configured to the video editing track Track_1 (shown as Track_1 in the figure), the character video material P3 is configured to the video editing track Track_2 (shown as Track_2 in the figure), and the character video material P4 is configured to the video editing track Track_3 (shown as Track_3 in the figure). Afterwards, for example, in response to the user clicking the second generation control (not shown in the figure), the character video materials on the above-mentioned video editing tracks are synthesized to generate the target video, which is displayed in the preview window of the video editing interface.

[0067] Alternatively, in another implementation, after the character video material is assigned to the corresponding video editing track based on the layer information, in response to a user operation, such as clicking a "Save Draft" control in the video generation interface, the terminal device generates a video draft corresponding to the target video based on each video editing track. Subsequently, upon receiving a second user operation from the user, the terminal device reloads the video draft, thereby restoring the character video material to the video editing track, achieving secondary editing of the character video material on the video editing track. Thereafter, the above-mentioned character video material synthesis steps are repeated to generate an updated target video.

[0068] In an embodiment of the present disclosure, input information for representing target semantics is obtained in response to a first user operation, the input information being composed of at least one text sentence; the input information is converted into a dialogue text representing the target semantics, the dialogue text being composed of at least two text sentences based on a dialogue sentence pattern; character video material corresponding to the text sentences in the dialogue text is obtained, and a target video for displaying the target semantics is generated based on the character video material. By converting the input information input by the user into a dialogue text composed of text sentences based on a dialogue sentence pattern, character planning for the video content is achieved, and then corresponding character video material is obtained based on the dialogue text, and a target video representing the target semantics is generated, thereby achieving rapid and automated generation of video works, while ensuring that the generated target video can accurately express the target semantics corresponding to the first text, thereby improving the efficiency of video production and reducing production difficulty and cost.

[0069] Referring to FIG6 , FIG6 is a second flow chart of the video generation method provided by the embodiment of the present disclosure. Based on the embodiment shown in FIG2 , this embodiment further refines step S103 , and the video generation method includes:

[0070] Step S201: In response to a first user operation, input information for representing target semantics is acquired, where the input information consists of at least one text sentence.

[0071] Step S202: Convert the input information into a dialogue text representing the target semantics, where the dialogue text consists of at least two text sentences based on a dialogue sentence pattern.

[0072] Optionally, after step S202, the method further includes:

[0073] Step S203: In response to the second user operation, modify the dialogue text to generate a third text.

[0074] Exemplarily, after generating the dialogue text, the terminal device displays the dialogue text and can receive user modifications thereto. Specifically, for example, the terminal device displays the generated dialogue text in a dialogue text box on the video generation interface. Afterwards, the user can further modify the dialogue text as needed. For example, the dialogue text includes the following content:

[0075] “A: Have you noticed that the roads in the city are getting more and more congested?

[0076] B: Indeed, I feel that too.”

[0077] For example, the user can then operate on the dialog text box as needed to modify the dialog text to the following content:

[0078] A: Do you feel that the roads in the city are getting more and more congested? (Excited)

[0079] B: Of course! The traffic is so bad, it’s driving me crazy!

[0080] As can be seen from the above example, the terminal device can further modify the dialogue text generated by the language model by responding to the second user operation input by the user, thereby changing the appearance visual state of the character object by modifying the text content or adding appearance description statements, so as to obtain different character video materials in subsequent steps and achieve the effect of changing the target video.

[0081] For another example, modify the dialogue text to the following (hereinafter referred to as content #2):

[0082] “A: Have you noticed that the roads in the city are getting more and more congested?

[0083] B: No, I think it’s okay.

[0084] As can be seen from the above example, the terminal device can further modify the dialogue text generated by the language model by responding to the second user operation input by the user, thereby directly changing the semantics of the character's sentence by modifying the text content, thereby correcting the problem of deviation in the target semantics understood by the language model and improving the matching degree between the target video and the target semantics.

[0085] Step S204: Generate corresponding structured data according to the third text, where the structured data includes at least one data field, and the data field is used to record a content dimension of the image content representing the target semantics.

[0086] Exemplarily, based on the introduction in the steps of the previous embodiment, after the first text is converted into dialogue text using a speech model, the dialogue text can be formatted to generate corresponding structured data. The structured data includes at least one data field that carries the information in the dialogue text. Specifically, based on the introduction in the previous step, the dialogue text is composed of at least two text sentences based on dialogue sentences. Therefore, the data fields in the structured data provided by this embodiment can be used to record relevant information about the text sentences based on the dialogue sentences, thereby achieving the purpose of representing the image content of the target semantics. That is, the data fields are used to record a content dimension of the image content that represents the target semantics. Specifically, for example, in the image content that represents the target semantics, the character object that states the text sentence, the tone and emotional characteristics of the character object when stating the text sentence, and the content of the text sentence. Furthermore, based on the above data fields, the process of subsequently generating image frames corresponding to the image content is implemented.

[0087] Step S205: Obtain corresponding role video material according to the structured data.

[0088] Step S206: Generate a target video for displaying target semantics based on the role video material.

[0089] Furthermore, after obtaining structured data through the steps of the above embodiment, the content of each data field in the structured data is mapped to the corresponding character video material, thereby obtaining the character video material corresponding to the structured data. Since the number of character video materials is normally greater than one, the character video material set corresponding to the structured data is obtained. The above process is introduced below with a more specific embodiment. Figure 7 is a schematic diagram of a process for generating a target video provided by an embodiment of the present disclosure. As shown in Figure 7, first, according to the dialogue text obtained in the above steps, after formatting, the corresponding structured data is obtained, wherein the structured data contains N data records (shown as records R_1 to R_N in the figure), and each data record is composed of M data fields. Afterwards, based on the structured data, starting from the first data record (record R_1), the character video material corresponding to each data record is obtained. This process can be implemented by accessing the data server and will not be described in detail. Specifically, as shown in the figure, based on record R_1, character video material set Media_1 (shown as Media_1 in the figure) is obtained; based on data record R_2, character video material set Media_2 (shown as Media_2 in the figure) is obtained, and so on. Furthermore, the structured data also includes a timestamp field, which records the time when each character video material in the character video material set appears and disappears in the target video. Finally, using the character video materials obtained from the structured data and the corresponding timestamps of each character video material, video synthesis is performed to obtain the corresponding target video.

[0090] Furthermore, in a possible implementation, the structured data includes at least a first data field and a second data field, wherein the first data field is used to record a role identifier representing a role object, and the second data field is used to record a role statement corresponding to the role object.

[0091] Specifically, for example, the field content in the first data field is used to represent the title of the character object that states the character sentence in the character dialogue scene corresponding to the dialogue sentence pattern, such as "passerby A", "passerby B", "Little C", etc. The field content in the first data field can be the character identifier that comes with the dialogue text, or it can be a more readable identifier generated after processing based on the character identifier that comes with the dialogue text and combined with the target semantics. For example, in the dialogue text, the character identifiers are "#1" and "#2", which represent two character objects respectively. In the process of generating structured data, the terminal device converts the character identifiers "#1" and "#2" into "classmate A" and "classmate B" respectively, and stores them in the first data field of the structured data. Therefore, in the subsequent process of generating the target video based on the structured data, the character objects are displayed in the target video in the (text) form of "classmate A" and "classmate B", thereby improving the accuracy of the video content in the target video and improving the matching of the video content with the target semantics.

[0092] The second data field is used to record the role statements stated by the role object corresponding to the role identifier, that is, the text statements based on the dialogue sentence pattern in the dialogue text. Similar to the processing method of the first data field, the field content of the second data field can be the role statements corresponding to each role object in the dialogue text, or it can be the statements obtained after processing the role statements corresponding to each role object based on the target semantics. It will not be repeated here.

[0093] Accordingly, when the structured data includes at least a first data field and a second data field, the specific implementation of step S205 includes: obtaining at least one target character video material corresponding to each character identifier based on the first data field and the second data field, where the target character video material corresponds to at least one character statement. For example, if the field content of the first data field in data record R_1 represents character object A, then the corresponding character video material Media_1, such as an image and a piece of music, is obtained based on the field content of the second data field in the same data record (data record R_1). Specifically, the character video material is determined based on the field content of the second data field in data record R_1, namely, the dialogue content of the character dialogue. Similarly, if the field content of the first data field in data record R_2 represents character object B, then the corresponding character video material Media_2, such as two images, is obtained based on the field content of the corresponding second data field.

[0094] Furthermore, the structured data also includes at least one of the following: a third data field, which is used to record the role status of the role object corresponding to the role identifier; and a fourth data field, which is used to record the material timestamp of the role video material, which is used to determine the playback timing of the role video material constituting the target video. Accordingly, when the structured data includes at least the first data field, the second data field, and at least one of the third and fourth data fields, the specific implementation method of step S205 includes: obtaining at least one target role video material corresponding to each role identifier based on the first data field, the second data field, and at least one of the third and fourth data fields.

[0095] Specifically, the third data field is used to represent the character state of the character object. Exemplarily, the character state includes visual appearance and / or motion state. Motion state, for example, represents the character object's torso motion posture, such as running, standing, or sitting. Visual appearance, for example, represents the character object's facial emotional expression, such as happiness, sadness, or fear. Through the character state, the character object's external and internal activities can be further displayed in the target video, thereby achieving more comprehensive information expression, allowing the target semantics represented by the dialogue text to be more fully reflected in the target video, and improving the match between the target video and the target semantics. The fourth data field is used to represent the timestamp corresponding to each character video material, which is also the timestamp field mentioned in the previous embodiment steps, thereby enabling the playback order of each character video material to be set. The specific implementation process is not repeated here. Furthermore, the structured data may also include a fifth data field and a sixth data field. The fifth data field is used to record scene description information; the sixth data field is used to record background music information.

[0096] The above process is described below using a more detailed embodiment. Figure 8 is a schematic diagram of a process for obtaining character video footage provided by an embodiment of the present disclosure. As shown in Figure 8, after obtaining structured data, the structured data includes at least data record R_1 and data record R_2, where each data record includes five corresponding data fields, as shown in the figure, namely field I, field II, field III, field IV, and field V. Field I corresponds to the first data field in this embodiment; field II corresponds to the second data field in this embodiment; field III and field IV both correspond to the third data field in this embodiment; and field V corresponds to the fourth data field in this embodiment. More specifically, taking data record R_1 as an example, the field content of field I of data record R_1 is "passerby A"; the field content of field II is "What a nice weather today!"; the field content of field III is "running"; the field content of field IV is "happy"; and the field content of field IV is "A = 0; B, C, D = 00:00-00:02." Based on the above structured data, first, the terminal device obtains media material A, which is a background image, from the server based on the target semantics. In another possible implementation, the terminal device can also obtain the background image based on field I, or based on an additional field VI (not shown in the figure, corresponding to the fifth data field in the embodiment of the present disclosure), without limitation. Then, based on field I, a human sequence frame set Med_1 is determined, and then based on field III, a sequence frame corresponding to the "running state" is selected from the human sequence frame set Med_1, that is, media material B. Based on field I, an expression sequence frame set Med_2 is determined, and then based on field IV, a sequence frame corresponding to the "happy state" is selected from the expression sequence frame set Med_2, that is, media material C. Then, based on field II, the corresponding text or text map is generated, that is, media material D. Finally, based on field V, the timestamps corresponding to the above media materials are obtained. For example, the display time of media materials B, C, and D corresponding to data record R_1 is "00:00-00:02", and the display time of character video material A is always displayed. The above media assets are then loaded into the video editing track based on the timestamp. If each media asset has corresponding layer information, the media asset is loaded into the video editing track corresponding to the image information, thereby completing the loading of the media asset corresponding to data record R_1 (shown as Media_1 in the figure). Next, for example, data record R_2 in the structured data is further processed, and the above process is repeated until the media asset corresponding to data record R_2 (shown as Media_2 in the figure) is loaded. This process will not be further described here. Finally, after all the media assets corresponding to the structured data are loaded, video synthesis is performed to generate the final target video.

[0097] Among them, in the above process of generating the target video, the order of obtaining each media material is only exemplary, that is, the order of obtaining the corresponding media material based on each data field can be set as needed or performed simultaneously, and no specific limitation is made here.

[0098] In this embodiment, by converting the conversation text into data fields within structured data, the different content dimensions of the target image are expressed, thereby fully utilizing the information in the conversation text to engrave the image content of the target image. This allows the generated target image to fully and accurately express the target semantics contained in the first text, thereby improving the generated target image quality.

[0099] In this embodiment, the implementation of step S201-step S02 is the same as the implementation of step S101-step S102 in the embodiment shown in FIG. 2 of the present disclosure, and will not be described in detail here.

[0100] Corresponding to the video generation method of the above embodiment, FIG9 is a structural block diagram of the video generation device provided by the embodiment of the present disclosure. For ease of explanation, only the parts related to the embodiment of the present disclosure are shown. Referring to FIG9, the video generation device 3 includes:

[0101] A response module 31 is configured to obtain input information for representing target semantics in response to a first user operation, where the input information consists of at least one text sentence;

[0102] A processing module 32 is configured to convert the input information into a dialogue text representing target semantics, wherein the dialogue text is composed of at least two text sentences based on a dialogue sentence pattern;

[0103] The generation module 33 is used to obtain the role video material corresponding to the text sentence in the dialogue text, and generate the target video for displaying the target semantics based on the role video material.

[0104] In one embodiment of the present disclosure, the input information includes a first text, which includes a monologue sentence and a prompt word sentence, wherein the monologue sentence is used to describe the target semantics, and the prompt word sentence is used to describe the video content form of the target video displaying the target semantics.

[0105] In one embodiment of the present disclosure, at least one of the following items is included: the dialogue text includes at least two role sentences and corresponding role identifiers, the role sentence is used to describe at least a part of the semantics of the target semantics, and the role identifier is used to characterize the role object corresponding to the role sentence; the dialogue text includes an appearance description sentence corresponding to the role sentence, and the appearance description sentence is used to describe at least one appearance visual state of the role object corresponding to the role sentence.

[0106] In one embodiment of the present disclosure, when the generation module 33 obtains the character video material corresponding to the text sentence in the dialogue text, it is specifically used to: generate corresponding structured data based on the dialogue text, the structured data including at least one data field, and the data field is used to record a content dimension of the image content representing the target semantics; and obtain the corresponding character video material based on the structured data.

[0107] In one embodiment of the present disclosure, the structured data includes at least a first data field and a second data field, wherein the first data field is used to record a role identifier representing a role object, and the second data field is used to record a role statement corresponding to the role object; when the generation module 33 obtains the corresponding role video material based on the structured data, it is specifically used to: obtain at least one target role video material corresponding to each role identifier based on the first data field and the second data field, and the target role video material corresponds to at least one role statement.

[0108] In one embodiment of the present disclosure, the structured data also includes at least one of the following: a third data field, the third data field is used to record the role status of the role object corresponding to the role identifier; a fourth data field, the fourth data field is used to record the material timestamp of the role video material, and the material timestamp is used to determine the playback timing of the role video material constituting the target video.

[0109] In one embodiment of the present disclosure, when the generation module 33 generates a target video for displaying target semantics based on the character video material, it is specifically used to: obtain the layer information corresponding to the character video material; based on the layer information, configure the character video material to the corresponding video editing track; trigger video synthesis for each video editing track to obtain the target video.

[0110] In one embodiment of the present disclosure, after configuring the character video material to the corresponding video editing track based on the layer information, the generation module 33 is also used to: generate a video draft corresponding to the target video based on each video editing track, and the video draft is used to perform secondary editing on the character video material on the video editing track after receiving a second user operation to generate an updated target video.

[0111] In one embodiment of the present disclosure, after converting the input information into a dialogue text representing the target semantics, the response module 31 is further used to: modify the dialogue text in response to a second user operation to generate a third text; when the generation module 33 obtains the character video material corresponding to the text sentence in the dialogue text, it is specifically used to: obtain the character video material corresponding to the third text.

[0112] The response module 31, the processing module 32 and the generation module 33 are connected in sequence. The video generation device 3 provided in this embodiment can implement the technical solution of the above method embodiment, and its implementation principle and technical effect are similar, which will not be described in detail in this embodiment.

[0113] FIG10 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. As shown in FIG10 , the electronic device 4 includes:

[0114] A processor 41, and a memory 42 communicatively connected to the processor 41;

[0115] Memory 42 stores computer-executable instructions;

[0116] The processor 41 executes the computer-executable instructions stored in the memory 42 to implement the video generation method in the embodiments shown in FIG. 2 to FIG. 8 .

[0117] Optionally, the processor 41 and the memory 42 are connected via a bus 43 .

[0118] The relevant explanations can be understood by referring to the relevant descriptions and effects corresponding to the steps in the embodiments corresponding to Figures 2 to 8, and no further details will be given here.

[0119] An embodiment of the present disclosure provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the video generation method provided in any one of the embodiments corresponding to Figures 2 to 8 of the present disclosure.

[0120] In order to implement the above embodiment, the embodiment of the present disclosure further provides an electronic device.

[0121] Referring to FIG11 , a schematic diagram of the structure of an electronic device 900 suitable for implementing an embodiment of the present disclosure is shown. The electronic device 900 may be a terminal device or a server. The terminal device may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (Portable Android Devices, PADs), portable multimedia players (PMPs), vehicle-mounted terminals (e.g., vehicle-mounted navigation terminals), and fixed terminals such as digital TVs and desktop computers. The electronic device shown in FIG11 is merely an example and should not limit the functionality and scope of use of the embodiments of the present disclosure.

[0122] As shown in FIG11 , the electronic device 900 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage device 908 into a random access memory (RAM) 903. Various programs and data required for the operation of the electronic device 900 are also stored in the RAM 903. The processing device 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0123] Typically, the following devices may be connected to the I / O interface 905: an input device 906 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 907 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 908 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 909. The communication device 909 may allow the electronic device 900 to communicate with other devices wirelessly or by wire to exchange data. Although FIG. 11 shows an electronic device 900 having various devices, it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

[0124] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network via the communication device 909, or installed from the storage device 908, or installed from the ROM 902. When the computer program is executed by the processing device 901, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0125] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0126] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0127] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.

[0128] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet Service Provider).

[0129] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0130] The units involved in the embodiments described in this disclosure may be implemented in software or hardware. In some cases, the name of a unit does not limit the unit itself. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses."

[0131] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0132] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0133] In a first aspect, according to one or more embodiments of the present disclosure, a video generation method is provided, comprising:

[0134] In response to a first user operation, input information for representing target semantics is obtained, the input information consisting of at least one text sentence; the input information is converted into a dialogue text representing the target semantics, the dialogue text consisting of at least two text sentences based on a dialogue sentence pattern; character video material corresponding to the text sentences in the dialogue text is obtained, and a target video for displaying the target semantics is generated based on the character video material.

[0135] According to one or more embodiments of the present disclosure, the input information includes a first text, which includes a monologue sentence and a prompt word sentence, wherein the monologue sentence is used to describe the target semantics, and the prompt word sentence is used to describe the video content form of the target video displaying the target semantics.

[0136] According to one or more embodiments of the present disclosure, at least one of the following is included: the dialogue text includes at least two role sentences and corresponding role identifiers, the role sentences are used to describe at least a part of the semantics of the target semantics, and the role identifiers are used to characterize the role objects corresponding to the role sentences; the dialogue text includes appearance description sentences corresponding to the role sentences, and the appearance description sentences are used to describe at least one appearance visual state of the character object corresponding to the role sentences.

[0137] According to one or more embodiments of the present disclosure, obtaining the character video material corresponding to the text sentence in the dialogue text includes: generating corresponding structured data based on the dialogue text, the structured data including at least one data field, and the data field is used to record a content dimension of the image content representing the target semantics; and obtaining the corresponding character video material based on the structured data.

[0138] According to one or more embodiments of the present disclosure, the structured data includes at least a first data field and a second data field, wherein the first data field is used to record a role identification representing a role object, and the second data field is used to record a role statement corresponding to the role object; obtaining the corresponding role video material based on the structured data includes: obtaining at least one target role video material corresponding to each role identification based on the first data field and the second data field, and the target role video material corresponds to at least one role statement.

[0139] According to one or more embodiments of the present disclosure, the structured data also includes at least one of the following items: a third data field, the third data field is used to record the role status of the role object corresponding to the role identifier; a fourth data field, the fourth data field is used to record the material timestamp of the role video material, and the material timestamp is used to determine the playback timing of the role video material constituting the target video.

[0140] According to one or more embodiments of the present disclosure, generating a target video for displaying the target semantics based on the character video material includes: obtaining layer information corresponding to the character video material; configuring the character video material to a corresponding video editing track based on the layer information; triggering video synthesis for each of the video editing tracks to obtain the target video.

[0141] According to one or more embodiments of the present disclosure, after configuring the character video material to the corresponding video editing track based on the layer information, it also includes: generating a video draft corresponding to the target video based on each of the video editing tracks, and the video draft is used to perform secondary editing on the character video material on the video editing track after receiving a second user operation to generate an updated target video.

[0142] According to one or more embodiments of the present disclosure, after converting the input information into a dialogue text representing the target semantics, it also includes: in response to a second user operation, modifying the dialogue text to generate a third text; obtaining the character video material corresponding to the text sentence in the dialogue text includes: obtaining the character video material corresponding to the third text.

[0143] In a second aspect, according to one or more embodiments of the present disclosure, a video generating apparatus is provided, including:

[0144] a response module, configured to obtain input information for representing target semantics in response to a first user operation, wherein the input information consists of at least one text sentence;

[0145] a processing module, configured to convert the input information into a dialogue text representing the target semantics, wherein the dialogue text is composed of at least two text sentences based on a dialogue sentence pattern;

[0146] A generation module is used to obtain role video materials corresponding to text sentences in the dialogue text, and generate a target video for displaying the target semantics based on the role video materials.

[0147] According to one or more embodiments of the present disclosure, the input information includes a first text, which includes a monologue sentence and a prompt word sentence, wherein the monologue sentence is used to describe the target semantics, and the prompt word sentence is used to describe the video content form of the target video displaying the target semantics.

[0148] According to one or more embodiments of the present disclosure, at least one of the following is included: the dialogue text includes at least two role sentences and corresponding role identifiers, the role sentences are used to describe at least a part of the semantics of the target semantics, and the role identifiers are used to characterize the role objects corresponding to the role sentences; the dialogue text includes appearance description sentences corresponding to the role sentences, and the appearance description sentences are used to describe at least one appearance visual state of the character object corresponding to the role sentences.

[0149] According to one or more embodiments of the present disclosure, when the generation module obtains the character video material corresponding to the text sentence in the dialogue text, it is specifically used to: generate corresponding structured data based on the dialogue text, the structured data including at least one data field, and the data field is used to record a content dimension of the image content representing the target semantics; and obtain the corresponding character video material based on the structured data.

[0150] According to one or more embodiments of the present disclosure, the structured data includes at least a first data field and a second data field, wherein the first data field is used to record a role identification representing a role object, and the second data field is used to record a role statement corresponding to the role object; when the generation module obtains the corresponding role video material based on the structured data, it is specifically used to: obtain at least one target role video material corresponding to each role identification based on the first data field and the second data field, and the target role video material corresponds to at least one role statement.

[0151] According to one or more embodiments of the present disclosure, the structured data also includes at least one of the following items: a third data field, the third data field is used to record the role status of the role object corresponding to the role identifier; a fourth data field, the fourth data field is used to record the material timestamp of the role video material, and the material timestamp is used to determine the playback timing of the role video material constituting the target video.

[0152] According to one or more embodiments of the present disclosure, when the generation module generates a target video for displaying the target semantics based on the character video material, it is specifically used to: obtain the layer information corresponding to the character video material; based on the layer information, configure the character video material to the corresponding video editing track; trigger video synthesis for each of the video editing tracks to obtain the target video.

[0153] According to one or more embodiments of the present disclosure, after configuring the character video material to the corresponding video editing track based on the layer information, the generation module is further used to: generate a video draft corresponding to the target video based on each of the video editing tracks, and the video draft is used to perform secondary editing on the character video material on the video editing track after receiving a second user operation to generate an updated target video.

[0154] According to one or more embodiments of the present disclosure, after the input information is converted into a dialogue text representing the target semantics, the response module is further used to: modify the dialogue text in response to a second user operation to generate a third text; when the generation module 33 obtains the character video material corresponding to the text sentence in the dialogue text, it is specifically used to: obtain the character video material corresponding to the third text.

[0155] In a third aspect, according to one or more embodiments of the present disclosure, there is provided an electronic device, comprising: at least one processor and a memory;

[0156] The memory stores computer-executable instructions;

[0157] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the video generation method described in the first aspect and various possible designs of the first aspect.

[0158] In a fourth aspect, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the video generation method described in the first aspect and various possible designs of the first aspect is implemented.

[0159] In a fifth aspect, according to one or more embodiments of the present disclosure, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the video generation method as described in the first aspect and various possible designs of the first aspect.

[0160] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also includes other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

[0161] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details have been included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0162] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims

1. A video generation method, comprising: In response to a first user operation, obtaining input information for representing target semantics, wherein the input information is composed of at least one text sentence; Converting the input information into a dialogue text representing the target semantics, wherein the dialogue text is composed of at least two text sentences based on a dialogue sentence pattern; Acquire role video material corresponding to the text sentence in the dialogue text, and generate a target video for displaying the target semantics based on the role video material.

2. The method according to claim 1, wherein: The input information includes a first text, wherein the first text includes a monologue sentence and a prompt word sentence, wherein the monologue sentence is used to describe the target semantics, and the prompt word sentence is used to describe the video content form of the target video displaying the target semantics.

3. The method according to claim 1, wherein: The input information is composed of at least one text sentence based on a monologue sentence pattern.

4. The method according to claim 1, wherein: The dialogue text includes at least one of the following two items: At least two role statements and corresponding role identifiers, wherein the role statement is used to describe at least a part of the semantics of the target semantics, and the role identifier is used to represent the role object corresponding to the role statement; An appearance description sentence corresponding to a role sentence, wherein the appearance description sentence is used to describe at least one appearance visual state of a role object corresponding to the role sentence.

5. The method according to any one of claims 1 to 4, wherein: The step of obtaining the role video material corresponding to the text sentence in the dialogue text includes: Generate corresponding structured data according to the conversation text, wherein the structured data includes at least one data field, and the data field is used to record a content dimension of the image content representing the target semantics; According to the structured data, corresponding role video material is obtained.

6. The method according to claim 5, wherein: The structured data at least includes a first data field and a second data field, the first data field is used to record a role identifier representing a role object, and the second data field is used to record a role statement corresponding to the role object; The step of obtaining corresponding role video material according to the structured data includes: At least one target role video material corresponding to each role identification is obtained according to the first data field and the second data field, and the target role video material corresponds to at least one role sentence.

7. The method according to claim 6, wherein: The structured data also includes at least one of the following: The third data field is used to record the role of the role object corresponding to the role identifier. state; A fourth data field, wherein the fourth data field is used to record a material timestamp of the character video material, and the material timestamp is used to determine a playback timing of the character video material constituting the target video; A fifth data field, wherein the fifth data field is used to record scene description information; The sixth data field is used to record background music information.

8. The method according to any one of claims 1 to 7, wherein: The generating a target video for displaying the target semantics based on the role video material includes: Obtaining layer information corresponding to the role video material; Based on the layer information, the character video material is configured to a corresponding video editing track; Trigger video synthesis for each of the video editing tracks to obtain the target video.

9. The method according to claim 8, wherein: After configuring the role video material to the corresponding video editing track based on the layer information, the method further includes: Based on each of the video editing tracks, a video draft corresponding to the target video is generated. The video draft is used to perform secondary editing on the character video material on the video editing track after receiving a second user operation to generate an updated target video.

10. The method according to any one of claims 1 to 9, characterized in that: After converting the input information into a dialog text representing the target semantics, the method further includes: In response to a second user operation, modify the dialogue text to generate a third text; The step of obtaining the role video material corresponding to the text sentence in the dialogue text includes: Obtain the role video material corresponding to the third text.

11. A video generating device, comprising: A response module, configured to obtain input information for representing target semantics in response to a first user operation, wherein the input information is composed of at least one text sentence; A processing module, configured to convert the input information into a dialogue text representing the target semantics, wherein the dialogue text is composed of at least two text sentences based on a dialogue sentence pattern; A generation module is used to obtain role video materials corresponding to text sentences in the dialogue text, and generate a target video for displaying the target semantics based on the role video materials.

12. An electronic device comprising: Processor and memory; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the video generating method according to any one of claims 1 to 10.

13. A computer-readable storage medium, wherein: The computer-readable storage medium stores computer-executable instructions, and when the processor executes the computer-executable instructions, the video generation method according to any one of claims 1 to 10 is implemented.

14. A computer program product comprising a computer program, wherein: When the computer program is executed by a processor, the video generating method according to any one of claims 1 to 10 is implemented.

Images