Video generation method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a video generation method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining input information used for representing target semantics through responding to a first user operation, and enabling the input information to be composed of at least one text statement; converting the input information into a dialogue text representing target semantics, wherein the dialogue text is composed of at least two text statements based on dialogue sentence patterns; and obtaining a role video material corresponding to the text statement in the dialogue text, and generating a target video for displaying the target semantics based on the role video material. The input information input by the user is converted into the dialogue text formed by the text statements based on the dialogue sentence pattern, then the corresponding role video material is obtained based on the dialogue text, the target video representing the target semantics is generated, rapid and automatic generation of the video works is achieved, and the user experience is improved. The video production efficiency is improved, and the production difficulty and the production cost are reduced.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the technical fields of natural language processing and image processing, and in particular to a video generation method, device, electronic device, and storage medium. Background Art

[0002] Currently, various video platforms have attracted a large number of users with their rich and diverse video content, including a large number of original videos created by users.

[0003] In the prior art, the production process of the above original video usually requires users to go through multiple production links such as script design, content arrangement and video production before a video work can be formed. The above video production solutions in the prior art have problems such as low video production efficiency, high cost and great difficulty. Summary of the invention

[0004] The embodiments of the present disclosure provide a video generation method, device, electronic device and storage medium to overcome the problems of low efficiency, high cost and great difficulty in video production.

[0005] In a first aspect, an embodiment of the present disclosure provides a video generation method, including:

[0006] In response to a first user operation, input information for representing target semantics is obtained, the input information being composed of at least one text sentence; the input information is converted into a dialogue text representing the target semantics, the dialogue text being composed of at least two text sentences based on a dialogue sentence pattern; character video material corresponding to the text sentences in the dialogue text is obtained, and a target video for displaying the target semantics is generated based on the character video material.

[0007] In a second aspect, an embodiment of the present disclosure provides a video generating device, including:

[0008] A response module, configured to obtain input information for representing target semantics in response to a first user operation, wherein the input information is composed of at least one text sentence;

[0009] A processing module, configured to convert the input information into a dialogue text representing the target semantics, wherein the dialogue text is composed of at least two text sentences based on a dialogue sentence pattern;

[0010] A generation module is used to obtain role video materials corresponding to text sentences in the dialogue text, and generate a target video for displaying the target semantics based on the role video materials.

[0011] In a third aspect, an embodiment of the present disclosure provides an electronic device, including: a processor and a memory;

[0012] The memory stores computer-executable instructions;

[0013] The processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the video generation method as described in the first aspect and various possible designs of the first aspect.

[0014] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, in which computer execution instructions are stored. When a processor executes the computer execution instructions, the video generation method described in the first aspect and various possible designs of the first aspect is implemented.

[0015] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the video generation method described in the first aspect and various possible designs of the first aspect.

[0016] The video generation method, device, electronic device and storage medium provided in this embodiment obtain input information for representing target semantics in response to a first user operation, the input information being composed of at least one text sentence; converting the input information into a dialogue text representing the target semantics, the dialogue text being composed of at least two text sentences based on dialogue sentences; obtaining a role video material corresponding to the text sentence in the dialogue text, and generating a target video for displaying the target semantics based on the role video material. By converting the input information input by the user into a dialogue text composed of text sentences based on dialogue sentences, role planning for the video content is achieved, and then the corresponding role video material is obtained based on the dialogue text, and a target video representing the target semantics is generated, thereby achieving rapid and automatic generation of video works, while ensuring that the generated target video can accurately express the target semantics corresponding to the first text, thereby improving the efficiency of video production and reducing production difficulty and production cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0018] Figure 1 An application scenario diagram of the video generation method provided in the embodiment of the present disclosure;

[0019] Figure 2 A schematic diagram of the process of the video generation method provided in the embodiment of the present disclosure Figure 1 ;

[0020] Figure 3 A schematic diagram of a text structure of a first text provided in an embodiment of the present disclosure;

[0021] Figure 4 for Figure 2 A flowchart of a specific implementation method of step S103 in the illustrated embodiment;

[0022] Figure 5 A schematic diagram of a process for configuring role video material provided by an embodiment of the present disclosure;

[0023] Figure 6 A schematic diagram of the process of the video generation method provided in the embodiment of the present disclosure Figure 2 ;

[0024] Figure 7 A schematic diagram of a process for generating a target video provided by an embodiment of the present disclosure;

[0025] Figure 8 A schematic diagram of a process for obtaining a role video material provided by an embodiment of the present disclosure;

[0026] Fig. 9 A structural block diagram of a video generation device provided in an embodiment of the present disclosure;

[0027] Fig.10 A schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure;

[0028] Fig.11 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0030] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0031] The application scenarios of the embodiments of the present disclosure are explained below:

[0032] Figure 1 This is an application scenario diagram of the video generation method provided in the embodiment of the present disclosure. The video generation method provided in the embodiment of the present disclosure can be applied to applications with video production and video generation functions. More specifically, it can be applied to application scenarios that generate videos by using text input or voice input. The execution subject of this embodiment can be a terminal device running the above-mentioned application with video generation function, or a server running the service end corresponding to the above-mentioned application, or other electronic devices that perform similar functions. Figure 1 As shown in , taking a terminal device as an example, after the terminal device runs a video editing application, it receives the text input by the user (indicated by "X" in the figure) and displays it in a text box in the video generation interface. The above text can be entered into the text box by the user through an input method, or the terminal device can generate the corresponding text by recognizing the user's voice and fill it into the text box, which can be set as needed. Afterwards, in response to a trigger operation, such as clicking on the "Video Generation" control in the video generation interface, the terminal device generates a target video that matches the content described by the text based on the text in the text box, and displays it in the video preview window in the video generation interface. Subsequent users can further preview, modify or generate the final video work for the video.

[0033] In the prior art, the production process of a video usually requires users to go through multiple production links, including script design, content arrangement, and video production, before a video work can be formed. This is because the content expressed by the video as a carrier needs to be designed and arranged in advance, so the above video production process includes at least two links, namely, the design of the video content and the conversion of the video content into a video screen. In the prior art, only static images can be automatically generated based on the description information. For example, if the description word is "warm spring", the corresponding image can be generated through a pre-trained generative model. However, in the process of generating a video, the video is generated by multiple frames of images that are coherent in content, which requires content continuity between video frames, and the content expressed by the video is richer than that of a single-frame image. This results in the inability to directly convert a general monologue sentence text into a video in the prior art. Therefore, it is necessary to manually produce single-frame images and then merge them into videos, resulting in a series of problems such as low efficiency, high cost, and difficulty in video production.

[0034] The embodiment of the present disclosure provides a video generation method to solve the above-mentioned problem.

[0035] refer to Figure 2 , Figure 2 A schematic diagram of the process of the video generation method provided in the embodiment of the present disclosure Figure 1The method of this embodiment can be applied in a terminal device. The video generation method includes:

[0036] Step S101: in response to a first user operation, obtaining input information for representing target semantics, where the input information consists of at least one text sentence.

[0037] For example, reference Figure 1 In the application scenario diagram shown, the execution subject of the embodiment of the method is a terminal device, such as a smart phone. After the terminal device runs a video editing application for implementing the method of the embodiment, it obtains a first text by detecting a first user operation applied by a user to a video generation interface, wherein the first user operation is, for example, an operation of entering text into a text box in the video generation interface, or an operation of entering text into a text box in the video generation interface through a voice input method. Afterwards, the terminal device obtains input information representing the target semantics by checking the text content in the text box.

[0038] Furthermore, the input information is a text composed of at least one text sentence, wherein, exemplarily, the text sentence constituting the input information is a sentence based on a monologue sentence pattern. A monologue sentence pattern is a sentence spoken by the same character object, corresponding to a complete target semantics, and expressing a complete meaning. Corresponding to this is a dialogue sentence pattern, that is, a sentence spoken by two or more character objects, which will be described in detail later. The following is a more specific example of the input information, and the input information is introduced in conjunction with the following example:

[0039] "The roads in the city are getting more and more congested, especially during rush hour, which is simply a torture. In comparison, riding a bicycle to get off work is becoming a smarter choice."

[0040] Based on the above example, the input information includes two (paragraphs of) text sentences, namely "Now the roads in the city are getting more and more congested, especially during rush hours, which is simply a torture" and "In comparison, riding a bicycle to get off work is becoming a smarter choice." The above two text sentences are both implemented based on monologue sentences, that is, they are stated by the same role object. The complete semantics jointly represented by the above two text sentences is the target semantics of the input information, where the target semantics can be represented by feature identifiers, feature arrays or feature matrices, which are not given one by one here. Optionally, in another possible implementation, the text sentences constituting the input information are sentences based on dialogue sentences. The dialogue text generated by processing the input information thereafter is equivalent to the optimization of the text sentences based on dialogue sentences in the input information, and this situation will not be repeated.

[0041] Furthermore, in a possible implementation, the input information includes a first text, which includes a monologue sentence and a prompt word sentence, wherein the monologue sentence is used to describe the target semantics, and the prompt word sentence is used to describe the video content form of the target video displaying the target semantics. Figure 3 A text structure diagram of a first text provided in an embodiment of the present disclosure, such as Figure 3 As shown, the complete text content of the first text is:

[0042] "Now the roads in the city are getting more and more congested, especially during rush hour, which is simply a torture. In contrast, riding a bicycle to get off work is becoming a smarter choice. % Convert the above text into a video of the conversation between two people and add emoticons."

[0043] Among them, "Now the roads in the city are getting more and more congested, especially during rush hour, which is simply a torture. In contrast, riding a bicycle to get off work is becoming a smarter choice." is a monologue sentence; and "Convert the above text into a video of a conversation between two people, and add facial expressions" is a prompt sentence. The monologue sentence and the prompt sentence are separated by a separator "%". Of course, it can be understood that the above separator is only exemplary, and other methods and rules can be used to distinguish monologue sentences and prompt sentences, such as through specific characters, punctuation or paragraph separators, etc., which can be set as needed, and will not be repeated here.

[0044] Referring to the above example, the monologue sentence is used to carry and describe the target semantics, while the prompt word sentence is used to describe the video content form of the target video displaying the target semantics. In the subsequent steps, exemplarily, the terminal device can use the generative model to generate, according to the video form represented by the prompt word sentence, to generate the corresponding target video, thereby achieving control of the presentation form of the video content of the target video. Among them, the monologue sentence and the prompt word sentence in the first text can be obtained, for example, by splitting and parsing the first text through a separation mark, and the specific implementation method will not be repeated.

[0045] Step S102: converting the input information into a dialogue text representing the target semantics, wherein the dialogue text is composed of at least two text sentences based on a dialogue sentence pattern.

[0046] Further, after obtaining the first text, semantic extraction is performed on the first text to obtain the target semantics, and then a preset language model is used to generate a dialogue text that can also express the target semantics, that is, the second text. The dialogue text is composed of at least two text sentences based on the dialogue sentence pattern. The following is a more specific example of the dialogue text generated based on the first text, and the dialogue text is introduced in combination with the following example:

[0047] “A: Do you feel that the roads in the city are getting more and more congested?

[0048] B: Yes, I feel that too.

[0049] A: Especially during rush hour, it is simply a torture.

[0050] B: It took me a long time to get home from get off work yesterday.

[0051] A: Maybe riding a bicycle to get off work would be a smarter choice.

[0052] B: That sounds like a good idea.”

[0053] Based on the above example, the dialogue text includes six (segment) text sentences, which are respectively stated by two character objects A and B, that is, they are implemented based on text sentences in dialogue sentences. The complete semantics represented by the above six text sentences is the same as the target semantics expressed by the first text, that is, the discussion about traffic congestion. As shown in the above example, the dialogue text has the same target semantics as the first text, but the text structure has changed, converting the first text into a text structure that is more suitable for generating the target video. The above process can be implemented based on a pre-trained language model.

[0054] Among them, after generating the dialogue text, the terminal device can output it, for example, display it in the video generation interface to show it to the user; it can also not output it, but only use it to execute subsequent video generation steps, which can be set as needed without specific restrictions.

[0055] The step of this embodiment, i.e., the step of converting the sentence structure of the first text, converts the text sentence stated by the same role object into the text sentence stated by multiple role objects. The processing of the first text by this step is the process of "procedural" conversion of the target semantics represented by the first text, which is equivalent to organizing the video content of the target video to be generated, that is, dividing the video content into frames. Therefore, after the first text is converted into a dialogue text in the "dialogue form", the dialogue text carrying the target semantics has the basis for generating a video, that is, the dialogue text is more likely to generate corresponding multiple content coherent "video frames", so as to better achieve the purpose of generating the target video and improve the video quality of the target video.

[0056] Step S103: Acquire the role video material corresponding to the text sentence in the dialogue text, and generate a target video for displaying the target semantics based on the role video material.

[0057] Exemplarily, after obtaining the dialogue text, the dialogue text is further processed to obtain the role video material corresponding to the dialogue text, wherein the role video material may include one or more of pictures, text, sequence frames, music, video clips, etc. In a possible implementation, the text sentence of each dialogue sentence in the dialogue text may be processed, and then a group of role video materials corresponding to the text sentence of each dialogue sentence may be obtained. Each group of role video materials may include one or more role video materials listed above. Specifically, for example, a text sentence of a dialogue sentence corresponds to a background picture, two sticker pictures, and a piece of music. In a possible implementation, a semantic matrix representing its semantics may be obtained by extracting features from the text sentence, and then mapped to the pre-generated role video material according to the semantic matrix to obtain the target role video material corresponding to the dialogue text. In another possible implementation, a semantic matrix representing its semantics may be obtained by extracting features from the text sentence, and then generated based on the semantic matrix, such as generating corresponding voice, image, music melody, etc., to obtain the role video material corresponding to the dialogue text.

[0058] Among them, since the dialogue text corresponds to at least two text sentences of the dialogue sentence pattern, at least two corresponding role video materials can be obtained. Afterwards, the playback timing of the corresponding role video materials is set according to the sequence of the text sentences in the dialogue text, thereby generating the target video.

[0059] In one possible implementation, Figure 4 As shown, the specific implementation of step S103 includes:

[0060] Step S1031: Obtain layer information corresponding to the role video material.

[0061] Step S1032: Based on the layer information, the character video material is configured to the corresponding video editing track.

[0062] Step S1033: triggering video synthesis for each video editing track to obtain a target video.

[0063] Exemplarily, first, the layer information corresponding to each character video material is obtained, and the layer information represents the level of the video editing track corresponding to the character video material, wherein, exemplarily, the image on the high-level video editing track will occlude the image on the low-level video editing track. In a possible implementation, the layer information includes a layer identifier, and layer identifier #1 represents that the level of the video editing track where the corresponding character video material is located is L1; and layer identifier #2 represents that the level of the video editing track where the corresponding character video material is located is L2. Afterwards, according to the layer information corresponding to each character video material, each character video material is configured on the corresponding video editing track. Furthermore, after the user triggers the corresponding generation control, the terminal device triggers the video synthesis for each video editing track, realizes the synthesis of the character video materials on multiple video editing tracks, and obtains the target video. Among them, exemplarily, the image information corresponding to the character video material can be preset attribute information bound to the character video material. After determining the character video material, the layer information can be obtained by reading the material attributes of the character video material. In another implementation method, the material type of the character video material can be judged, and based on the material type, the layer information corresponding to the character video material can be obtained. The specific implementation method can be set as needed and is not limited here.

[0064] Furthermore, in a possible implementation, the video generation interface includes a video editing track. Figure 5 A schematic diagram of a process for configuring a role video material provided by an embodiment of the present disclosure, such as Figure 5As shown, after the user clicks the first generation control in the video generation interface, the terminal device generates the corresponding dialogue text according to the first text input in the text box, and obtains the corresponding role video material according to each text sentence in the dialogue text, such as the role video material P1, role video material P2, role video material P3, role video material P4 shown in the figure, and the corresponding playback timing of each role video material. Afterwards, based on the layer information and corresponding playback timing of the role video material P1, role video material P2, role video material P3, role video material P4, the role video material P1, role video material P2, role video material P3, role video material P4 are respectively configured to the corresponding position on the video editing track, for example, as shown in the figure, the role video material P1, role video material P2 are configured to the video editing track Track_1 (shown as Track_1 in the figure), the role video material P3 is configured to the video editing track Track_2 (shown as Track_2 in the figure), and the role video material P4 is configured to the video editing track Track_3 (shown as Track_3 in the figure). Afterwards, for example in response to the user clicking on the second generation control (not shown in the figure), the role video materials on the above-mentioned video editing tracks are synthesized to generate the target video, and displayed in the preview window of the video editing interface.

[0065] Alternatively, in another implementation, after the character video material is configured to the corresponding video editing track based on the layer information, in response to a user operation, such as clicking a "save draft" control in the video generation interface, the terminal device generates a video draft corresponding to the target video based on each video editing track. Afterwards, after the terminal device receives a second user operation from the user, the video draft is reloaded, so that the above-mentioned character video material is restored to the video editing track, and the character video material on the video editing track is edited again. Afterwards, the above-mentioned steps of synthesizing the character video material are repeated to generate an updated target video.

[0066] In the disclosed embodiment, input information for representing the target semantics is obtained in response to a first user operation, the input information being composed of at least one text sentence; the input information is converted into a dialogue text representing the target semantics, the dialogue text being composed of at least two text sentences based on a dialogue sentence pattern; a role video material corresponding to the text sentence in the dialogue text is obtained, and a target video for displaying the target semantics is generated based on the role video material. By converting the input information input by the user into a dialogue text composed of text sentences based on a dialogue sentence pattern, role planning for the video content is achieved, and then the corresponding role video material is obtained based on the dialogue text, and a target video representing the target semantics is generated, so as to achieve rapid and automatic generation of video works, and at the same time ensure that the generated target video can accurately express the target semantics corresponding to the first text, thereby improving the efficiency of video production and reducing the difficulty and cost of production.

[0067] refer to Figure 6 , Figure 6 A schematic diagram of the process of the video generation method provided in the embodiment of the present disclosure Figure 2 In this embodiment Figure 2 Based on the embodiment shown, step S103 is further refined, and the video generation method includes:

[0068] Step S201: in response to a first user operation, obtaining input information for representing target semantics, where the input information consists of at least one text sentence.

[0069] Step S202: converting the input information into a dialogue text representing the target semantics, wherein the dialogue text is composed of at least two text sentences based on a dialogue sentence pattern.

[0070] Optionally, after step S202, the method further includes:

[0071] Step S203: In response to the second user operation, modify the dialogue text to generate a third text.

[0072] Exemplarily, after generating the dialogue text, the terminal device displays the dialogue text and can receive the user's modification thereof. Specifically, for example, the terminal device displays the generated dialogue text in the dialogue text box of the video generation interface. Afterwards, the user can further modify the dialogue text as needed. For example, the dialogue text includes the following contents:

[0073] “A: Do you feel that the roads in the city are getting more and more congested?

[0074] B: Indeed, I feel that too.

[0075] For example, the user can then operate the dialog text box as needed to modify the dialog text to the following content:

[0076] “A: Do you feel that the roads in the city are getting more and more congested? (Excited)

[0077] B: Of course! The traffic is so congested, it’s driving me crazy! ”

[0078] It can be seen from the above example that the terminal device can further modify the dialogue text generated by the language model by responding to the second user operation input by the user, thereby changing the appearance visual state of the character object by modifying the text content or adding an appearance description statement, so as to obtain different character video materials in the subsequent steps, thereby achieving the effect of changing the target video.

[0079] For another example, modify the dialogue text to the following content (hereinafter referred to as content #2):

[0080] “A: Do you feel that the roads in the city are getting more and more congested?

[0081] B: No, I think it’s OK.

[0082] It can be seen from the above example that the terminal device can further modify the dialogue text generated by the language model by responding to the second user operation input by the user, thereby directly changing the semantics of the character's sentence by modifying the text content, thereby correcting the problem of deviation in the target semantics understood by the language model and improving the matching degree between the target video finally generated and the target semantics.

[0083] Step S204: Generate corresponding structured data according to the third text, where the structured data includes at least one data field, and the data field is used to record a content dimension of the image content representing the target semantics.

[0084] Exemplarily, based on the introduction in the steps of the previous embodiment, after the first text is converted into a dialogue text through a speech model, the dialogue text can be formatted to generate corresponding structured data. Among them, the structured data contains at least one data field, and the information in the dialogue text is carried by the data field. Specifically, based on the introduction in the previous step, the dialogue text is composed of at least two text sentences based on dialogue sentences. Therefore, the data field in the structured data provided by this embodiment can be used to record the relevant information of the above-mentioned text sentences based on dialogue sentences, so as to achieve the purpose of representing the image content of the target semantics, that is, the data field is used to record a content dimension of the image content representing the target semantics. Specifically, for example, in the image content representing the target semantics, the character object stating the text sentence, the tone and emotional characteristics of the character object when stating the text sentence, and the content of the text sentence, etc. Further, based on the above-mentioned data field, the process of subsequently generating the image frame corresponding to the image content is realized.

[0085] Step S205: Obtain corresponding role video material according to the structured data.

[0086] Step S206: Generate a target video for displaying target semantics based on the role video material.

[0087] Furthermore, after the structured data is obtained through the steps of the above embodiment, the content of each data field in the structured data is mapped to the corresponding role video material, thereby obtaining the role video material corresponding to the structured data. Since the number of role video materials is normally greater than one, the role video material set corresponding to the structured data is obtained. The above process is introduced in a more specific embodiment below. Figure 7 A schematic diagram of a process for generating a target video provided by an embodiment of the present disclosure is shown in FIG. Figure 7 As shown, first, according to the dialogue text obtained in the above steps, after formatting, the corresponding structured data is obtained, wherein the structured data contains N data records (shown as records R_1 to R_N in the figure), and each data record is composed of M data fields. Afterwards, based on the structured data, starting from the first data record (record R_1), the role video material corresponding to each data record is obtained respectively. This process can be implemented by accessing the data server and will not be repeated. Specifically, for example, as shown in the figure, according to record R_1, the role video material set Media_1 (shown as Media_1 in the figure) is obtained; according to data record R_2, the role video material set Media_2 (shown as Media_2 in the figure) is obtained, and so on. At the same time, in the structured data, a timestamp field is also included, which is used to record the time when each role video material in the role video material set appears and disappears in the target video. Finally, the role video material obtained through the above structured data and the timestamps corresponding to each role video material are used to perform video synthesis to obtain the corresponding target video.

[0088] Furthermore, in a possible implementation, the structured data includes at least a first data field and a second data field, wherein the first data field is used to record a role identifier representing a role object, and the second data field is used to record a role statement corresponding to the role object.

[0089] Specifically, for example, the field content in the first data field is used to represent the title of the role object that states the role sentence in the character dialogue scene corresponding to the dialogue sentence pattern, such as "passerby A", "passerby B", "Little C", etc. The field content in the first data field can be the role identifier that comes with the dialogue text, or it can be a more readable identifier generated after processing based on the role identifier that comes with the dialogue text and combined with the target semantics. For example, in the dialogue text, the role identifiers are "#1" and "#2", which represent two role objects respectively. In the process of generating structured data, the terminal device converts the role identifiers "#1" and "#2" into "classmate A" and "classmate B" respectively in combination with the target semantics, and stores them in the first data field of the structured data. Therefore, in the subsequent process of generating the target video based on the structured method data, the role object is displayed in the target video in the form of (text) "classmate A" and "classmate B", which improves the accuracy of the video content in the target video and improves the matching of the video content with the target semantics.

[0090] The second data field is used to record the role statements made by the role object corresponding to the role identifier, that is, the text sentences based on the dialogue sentence pattern in the dialogue text. Similar to the processing method of the first data field, the field content of the second data field can be the role statements corresponding to each role object in the dialogue text, or it can be the sentences obtained after processing the role statements corresponding to each role object based on the target semantics. It will not be repeated here.

[0091] Correspondingly, when the structured data includes at least the first data field and the second data field, the specific implementation of step S205 includes: according to the first data field and the second data field, obtaining at least one target role video material corresponding to each role identifier, and the target role video material corresponds to at least one role sentence. For example, in the data record R_1, the field content of the first data field represents the role object A, and then according to the field content of the second data field in the same data record (data record R_1) corresponding thereto, the corresponding role video material Media_1 is obtained, such as an image and a piece of music. Specifically, the determination of the above-mentioned role video material is based on the field content of the second data field in the data record R_1, that is, the dialogue content of the role dialogue. Afterwards, similarly, for example, in the data record R_2, the field content of the first data field represents the role object B, and then according to the field content of the corresponding second data field, the corresponding role video material Media_2, such as two images, is obtained.

[0092] Furthermore, the structured data also includes at least one of the following: a third data field, the third data field is used to record the role state of the role object corresponding to the role identifier; a fourth data field, the fourth data field is used to record the material timestamp of the role video material, and the material timestamp is used to determine the playback timing of the role video material constituting the target video. Accordingly, when the structured data includes at least the first data field, the second data field, and at least one of the third data field and the fourth data field, the specific implementation of step S205 includes: according to the first data field, the second data field, and at least one of the third data field and the fourth data field, at least one target role video material corresponding to each role identifier is obtained.

[0093] Specifically, the third data field is used to characterize the character state of the character object. Exemplarily, the character state includes the appearance visual state and / or the motion state, wherein the motion state, for example, characterizes the torso motion posture of the character object, for example, running state, standing state, sitting state, etc. The appearance visual state, for example, characterizes the facial emotional expression of the character object, such as happy, sad, fear, etc. Through the character state, the external activities and inner activities of the character object can be further displayed in the target video, thereby achieving more sufficient information expression, so that the target semantics represented by the dialogue text is more fully reflected in the target video, and the matching of the target video and the target semantics is improved. The fourth data field is used to characterize the timestamp corresponding to each character video material, that is, the timestamp field mentioned in the previous embodiment step, so as to realize the playback order setting between each character video material, and the specific implementation process is not repeated. Further, the structured data can also include a fifth data field and a sixth data field, wherein the fifth data field is used to record scene description information; the sixth data field, the sixth data field is used to record background music information.

[0094] The above process is introduced below with a more detailed embodiment. Figure 8 A schematic diagram of a process for obtaining a role video material provided by an embodiment of the present disclosure, such as Figure 8As shown, exemplarily, after the structured data is obtained, the structured data includes at least data record R_1 and data record R_2, wherein each data record includes 5 corresponding data fields, as shown in the figure, namely field I, field II, field III, field IV, and field V, wherein field I corresponds to the first data field in this embodiment; field II corresponds to the second data field in this embodiment; field III and field IV both correspond to the third data field in this embodiment; and field V corresponds to the fourth data field in this embodiment. More specifically, taking data record R_1 as an example, the field content of field I of data record R_1 is "passerby A"; the field content of field II is "the weather is really good today!"; the field content of field III is "running"; the field content of field IV is "happy"; the field content of field IV is "A=0; B, C, D=00:00-00:02". Based on the above structured data, first, the terminal device obtains media material A, which is a background image, from the server based on the target semantics. In another possible implementation, the terminal device can also obtain the background image based on field I, or obtain the background image based on an additional field VI (not shown in the figure, corresponding to the fifth data field in the embodiment of the present disclosure), without limitation; then, based on field I, a human sequence frame set Med_1 is determined, and then based on field III, a sequence frame corresponding to the "running state" is selected from the human sequence frame set Med_1, that is, media material B; based on field I, an expression sequence frame set Med_2 is determined, and then based on field IV, a sequence frame corresponding to the "happy state" is selected from the expression sequence frame set Med_2, that is, media material C; then, based on field II, a corresponding text or text map is generated, that is, media material D; finally, based on field V, the timestamps corresponding to the above media materials are obtained, for example, the display time of media material B, media material C, and media material D corresponding to data record R_1 is "00:00-00:02", and the display time of character video material A is always displayed. Then, based on the timestamp, the above media materials are loaded into the video editing track. If each media material has layer information corresponding to it, the media material is loaded into the video editing track corresponding to the image information, so that the media material corresponding to the data record R_1 (shown as Media_1 in the figure) is loaded. After that, for example, the data record R_2 in the structured data is further processed, and the above process is repeated until the media material corresponding to the data record R_2 (shown as Media_2 in the figure) is loaded. It will not be repeated here. Finally, after all the media materials corresponding to the structured data are loaded, video synthesis is performed to generate the final target video.

[0095] Among them, in the above process of generating the target video, the order of obtaining each media material is only exemplary, that is, the order of obtaining the corresponding media material based on each data field can be set as needed or performed simultaneously, and no specific limitation is made here.

[0096] In the steps of this embodiment, the different content dimensions in the target image are expressed by converting the dialogue text into data fields in structured data, thereby making full use of the information in the dialogue text to engrave the image content of the target image. The generated target image can fully and accurately express the target semantics contained in the first text, thereby improving the generation quality of the target image.

[0097] In this embodiment, the implementation method of step S201-step S02 is the same as that of the present disclosure. Figure 2 The implementation methods of step S101 and step S102 in the illustrated embodiment are the same and will not be described in detail here.

[0098] Corresponding to the video generation method of the above embodiment, Fig. 9 This is a structural block diagram of a video generation device provided by an embodiment of the present disclosure. For ease of explanation, only the parts related to the embodiment of the present disclosure are shown.

[0099] Reference Fig. 9 , the video generating device 3 comprises:

[0100] A response module 31, configured to obtain input information for representing target semantics in response to a first user operation, wherein the input information is composed of at least one text sentence;

[0101] A processing module 32, configured to convert the input information into a dialogue text representing the target semantics, wherein the dialogue text is composed of at least two text sentences based on a dialogue sentence pattern;

[0102] The generation module 33 is used to obtain the role video material corresponding to the text sentence in the dialogue text, and generate the target video for displaying the target semantics based on the role video material.

[0103] In one embodiment of the present disclosure, the input information includes a first text, which includes a monologue sentence and a prompt word sentence, wherein the monologue sentence is used to describe the target semantics, and the prompt word sentence is used to describe the video content form of the target video displaying the target semantics.

[0104] In one embodiment of the present disclosure, at least one of the following items is included: the dialogue text includes at least two role sentences and corresponding role identifiers, the role sentence is used to describe at least a part of the target semantics, and the role identifier is used to characterize the role object corresponding to the role sentence; the dialogue text includes an appearance description sentence corresponding to the role sentence, and the appearance description sentence is used to describe at least one appearance visual state of the role object corresponding to the role sentence.

[0105] In one embodiment of the present disclosure, when the generation module 33 obtains the character video material corresponding to the text sentence in the dialogue text, it is specifically used to: generate corresponding structured data based on the dialogue text, the structured data includes at least one data field, and the data field is used to record a content dimension of the image content representing the target semantics; based on the structured data, obtain the corresponding character video material.

[0106] In one embodiment of the present disclosure, the structured data includes at least a first data field and a second data field, wherein the first data field is used to record a role identification representing a role object, and the second data field is used to record a role sentence corresponding to the role object; when the generation module 33 obtains the corresponding role video material based on the structured data, it is specifically used to: obtain at least one target role video material corresponding to each role identification based on the first data field and the second data field, and the target role video material corresponds to at least one role sentence.

[0107] In one embodiment of the present disclosure, the structured data also includes at least one of the following: a third data field, the third data field is used to record the role status of the role object corresponding to the role identifier; a fourth data field, the fourth data field is used to record the material timestamp of the role video material, and the material timestamp is used to determine the playback timing of the role video material constituting the target video.

[0108] In one embodiment of the present disclosure, when the generation module 33 generates a target video for displaying target semantics based on the character video material, it is specifically used to: obtain the layer information corresponding to the character video material; based on the layer information, configure the character video material to the corresponding video editing track; trigger video synthesis for each video editing track to obtain the target video.

[0109] In one embodiment of the present disclosure, after configuring the character video material to the corresponding video editing track based on the layer information, the generation module 33 is also used to: generate a video draft corresponding to the target video based on each video editing track, and the video draft is used to perform secondary editing on the character video material on the video editing track after receiving a second user operation to generate an updated target video.

[0110] In one embodiment of the present disclosure, after converting the input information into a dialogue text representing the target semantics, the response module 31 is further used to: modify the dialogue text in response to a second user operation to generate a third text; when the generation module 33 obtains the character video material corresponding to the text sentence in the dialogue text, it is specifically used to: obtain the character video material corresponding to the third text.

[0111] The response module 31, the processing module 32 and the generation module 33 are connected in sequence. The video generation device 3 provided in this embodiment can implement the technical solution of the above method embodiment, and its implementation principle and technical effect are similar, which will not be described in detail in this embodiment.

[0112] Fig.10 A schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure is shown in FIG. Fig.10 As shown, the electronic device 4 includes:

[0113] A processor 41, and a memory 42 communicatively connected to the processor 41;

[0114] The memory 42 stores computer executable instructions;

[0115] The processor 41 executes the computer execution instructions stored in the memory 42 to implement the following Figure 2-Figure 8 The video generation method in the illustrated embodiment.

[0116] Optionally, the processor 41 and the memory 42 are connected via a bus 43 .

[0117] For related instructions, please refer to Figure 2-Figure 8 The relevant descriptions and effects corresponding to the steps in the corresponding embodiments can be understood, and no further elaboration is made here.

[0118] The present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, which are used to implement the present invention when executed by a processor. Figure 2-Figure 8 The video generation method provided in any one of the corresponding embodiments.

[0119] In order to implement the above embodiment, the embodiment of the present disclosure also provides an electronic device.

[0120] refer to Fig.11, which shows a schematic diagram of the structure of an electronic device 900 suitable for implementing the embodiment of the present disclosure, and the electronic device 900 may be a terminal device or a server. The terminal device may include but is not limited to mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (Portable Android Devices, PADs), portable multimedia players (PMPs), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Fig.11 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0121] like Fig.11 As shown, the electronic device 900 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage device 908 to a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the electronic device 900 are also stored. The processing device 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0122] Typically, the following devices may be connected to the I / O interface 905: input devices 906 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 907 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 908 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 909. The communication device 909 may allow the electronic device 900 to communicate with other devices wirelessly or by wire to exchange data. Although Fig.11 The electronic device 900 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0123] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 909, or installed from a storage device 908, or installed from a ROM 902. When the computer program is executed by the processing device 901, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0124] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0125] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0126] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.

[0127] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0128] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0129] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of a unit does not limit the unit itself in some cases. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses".

[0130] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0131] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0132] In a first aspect, according to one or more embodiments of the present disclosure, a video generation method is provided, including:

[0133] In response to a first user operation, input information for representing target semantics is obtained, the input information being composed of at least one text sentence; the input information is converted into a dialogue text representing the target semantics, the dialogue text being composed of at least two text sentences based on a dialogue sentence pattern; character video material corresponding to the text sentences in the dialogue text is obtained, and a target video for displaying the target semantics is generated based on the character video material.

[0134] According to one or more embodiments of the present disclosure, the input information includes a first text, which includes a monologue sentence and a prompt word sentence, wherein the monologue sentence is used to describe the target semantics, and the prompt word sentence is used to describe the video content form of the target video displaying the target semantics.

[0135] According to one or more embodiments of the present disclosure, at least one of the following is included: the dialogue text includes at least two role sentences and corresponding role identifiers, the role sentence is used to describe at least a part of the semantics of the target semantics, and the role identifier is used to characterize the role object corresponding to the role sentence; the dialogue text includes an appearance description sentence corresponding to the role sentence, and the appearance description sentence is used to describe at least one appearance visual state of the character object corresponding to the role sentence.

[0136] According to one or more embodiments of the present disclosure, the step of obtaining the character video material corresponding to the text sentences in the dialogue text includes: generating corresponding structured data based on the dialogue text, wherein the structured data includes at least one data field, and the data field is used to record a content dimension of the image content representing the target semantics; and obtaining the corresponding character video material based on the structured data.

[0137] According to one or more embodiments of the present disclosure, the structured data includes at least a first data field and a second data field, wherein the first data field is used to record a role identification representing a role object, and the second data field is used to record a role sentence corresponding to the role object; obtaining the corresponding role video material based on the structured data includes: obtaining at least one target role video material corresponding to each role identification based on the first data field and the second data field, and the target role video material corresponds to at least one role sentence.

[0138] According to one or more embodiments of the present disclosure, the structured data also includes at least one of the following: a third data field, the third data field is used to record the role status of the role object corresponding to the role identifier; a fourth data field, the fourth data field is used to record the material timestamp of the role video material, and the material timestamp is used to determine the playback timing of the role video material constituting the target video.

[0139] According to one or more embodiments of the present disclosure, generating a target video for displaying the target semantics based on the character video material includes: obtaining layer information corresponding to the character video material; configuring the character video material to a corresponding video editing track based on the layer information; triggering video synthesis for each of the video editing tracks to obtain the target video.

[0140] According to one or more embodiments of the present disclosure, after configuring the character video material to the corresponding video editing track based on the layer information, it also includes: generating a video draft corresponding to the target video based on each of the video editing tracks, and the video draft is used to perform secondary editing on the character video material on the video editing track after receiving a second user operation to generate an updated target video.

[0141] According to one or more embodiments of the present disclosure, after converting the input information into a dialogue text representing the target semantics, it also includes: in response to a second user operation, modifying the dialogue text to generate a third text; obtaining the character video material corresponding to the text sentences in the dialogue text includes: obtaining the character video material corresponding to the third text.

[0142] In a second aspect, according to one or more embodiments of the present disclosure, a video generating device is provided, including:

[0143] A response module, configured to obtain input information for representing target semantics in response to a first user operation, wherein the input information is composed of at least one text sentence;

[0144] A processing module, configured to convert the input information into a dialogue text representing the target semantics, wherein the dialogue text is composed of at least two text sentences based on a dialogue sentence pattern;

[0145] A generation module is used to obtain role video materials corresponding to text sentences in the dialogue text, and generate a target video for displaying the target semantics based on the role video materials.

[0146] According to one or more embodiments of the present disclosure, the input information includes a first text, which includes a monologue sentence and a prompt word sentence, wherein the monologue sentence is used to describe the target semantics, and the prompt word sentence is used to describe the video content form of the target video displaying the target semantics.

[0147] According to one or more embodiments of the present disclosure, at least one of the following is included: the dialogue text includes at least two role sentences and corresponding role identifiers, the role sentence is used to describe at least a part of the semantics of the target semantics, and the role identifier is used to characterize the role object corresponding to the role sentence; the dialogue text includes an appearance description sentence corresponding to the role sentence, and the appearance description sentence is used to describe at least one appearance visual state of the character object corresponding to the role sentence.

[0148] According to one or more embodiments of the present disclosure, when the generation module obtains the character video material corresponding to the text sentence in the dialogue text, it is specifically used to: generate corresponding structured data according to the dialogue text, the structured data includes at least one data field, and the data field is used to record a content dimension of the image content representing the target semantics; according to the structured data, obtain the corresponding character video material.

[0149] According to one or more embodiments of the present disclosure, the structured data includes at least a first data field and a second data field, wherein the first data field is used to record a role identification representing a role object, and the second data field is used to record a role statement corresponding to the role object; when the generation module obtains the corresponding role video material according to the structured data, it is specifically used to: obtain at least one target role video material corresponding to each role identification according to the first data field and the second data field, and the target role video material corresponds to at least one role statement.

[0150] According to one or more embodiments of the present disclosure, the structured data also includes at least one of the following: a third data field, the third data field is used to record the role status of the role object corresponding to the role identifier; a fourth data field, the fourth data field is used to record the material timestamp of the role video material, and the material timestamp is used to determine the playback timing of the role video material constituting the target video.

[0151] According to one or more embodiments of the present disclosure, when the generation module generates a target video for displaying the target semantics based on the character video material, it is specifically used to: obtain layer information corresponding to the character video material; based on the layer information, configure the character video material to the corresponding video editing track; trigger video synthesis for each of the video editing tracks to obtain the target video.

[0152] According to one or more embodiments of the present disclosure, after configuring the character video material to the corresponding video editing track based on the layer information, the generation module is further used to: generate a video draft corresponding to the target video based on each of the video editing tracks, and the video draft is used to perform secondary editing on the character video material on the video editing track after receiving a second user operation to generate an updated target video.

[0153] According to one or more embodiments of the present disclosure, after the input information is converted into a dialogue text representing the target semantics, the response module is further used to: modify the dialogue text in response to a second user operation to generate a third text; when the generation module 33 obtains the character video material corresponding to the text sentence in the dialogue text, it is specifically used to: obtain the character video material corresponding to the third text.

[0154] In a third aspect, according to one or more embodiments of the present disclosure, there is provided an electronic device, comprising: at least one processor and a memory;

[0155] The memory stores computer-executable instructions;

[0156] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the video generation method described in the first aspect and various possible designs of the first aspect.

[0157] In a fourth aspect, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer execution instructions. When a processor executes the computer execution instructions, the video generation method described in the first aspect and various possible designs of the first aspect is implemented.

[0158] In a fifth aspect, according to one or more embodiments of the present disclosure, a computer program product is provided, including a computer program, which, when executed by a processor, implements the video generation method described in the first aspect and various possible designs of the first aspect.

[0159] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0160] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0161] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A video generation method, characterized in that: include: In response to a first user operation, obtaining input information for representing target semantics, wherein the input information consists of at least one text sentence; Converting the input information into a dialogue text representing the target semantics, wherein the dialogue text is composed of at least two text sentences based on a dialogue sentence pattern; Acquire role video material corresponding to the text sentence in the dialogue text, and generate a target video for displaying the target semantics based on the role video material.

2. The method according to claim 1, characterized in that The input information includes a first text, which includes a monologue sentence and a prompt word sentence, wherein the monologue sentence is used to describe the target semantics, and the prompt word sentence is used to describe the video content form of the target video displaying the target semantics.

3. The method according to claim 1, characterized in that Include at least one of the following: The input information is composed of at least one text sentence based on a monologue sentence pattern; The dialogue text includes at least two role sentences and corresponding role identifiers, wherein the role sentences are used to describe at least a part of the target semantics, and the role identifiers are used to represent the role objects corresponding to the role sentences; The dialogue text includes an appearance description sentence corresponding to the character sentence, and the appearance description sentence is used to describe at least one appearance visual state of the character object corresponding to the character sentence.

4. The method according to claim 1, characterized in that The step of obtaining the role video material corresponding to the text sentence in the dialogue text includes: Generate corresponding structured data according to the conversation text, wherein the structured data includes at least one data field, and the data field is used to record a content dimension of the image content representing the target semantics; According to the structured data, corresponding role video material is obtained.

5. The method according to claim 4, characterized in that The structured data at least includes a first data field and a second data field, wherein the first data field is used to record a role identifier representing a role object, and the second data field is used to record a role statement corresponding to the role object; The step of obtaining corresponding role video material according to the structured data includes: At least one target role video material corresponding to each role identification is obtained according to the first data field and the second data field, and the target role video material corresponds to at least one role sentence.

6. The method according to claim 5, characterized in that The structured data also includes at least one of the following: A third data field, the third data field is used to record the role state of the role object corresponding to the role identifier; A fourth data field, the fourth data field is used to record a material timestamp of the character video material, the material timestamp is used to determine a playback timing of the character video material constituting the target video; A fifth data field, the fifth data field is used to record scene description information; The sixth data field is used to record background music information.

7. The method according to claim 1, characterized in that The generating a target video for displaying the target semantics based on the role video material includes: Obtaining layer information corresponding to the role video material; Based on the layer information, the character video material is configured to a corresponding video editing track; Trigger video synthesis for each of the video editing tracks to obtain the target video.

8. The method according to claim 7, characterized in that After configuring the role video material to the corresponding video editing track based on the layer information, the method further includes: Based on each of the video editing tracks, a video draft corresponding to the target video is generated. The video draft is used to perform secondary editing on the character video material on the video editing track after receiving a second user operation to generate an updated target video.

9. The method according to claim 1, characterized in that: After converting the input information into a dialog text representing the target semantics, the method further includes: In response to a second user operation, modify the dialogue text to generate a third text; The step of obtaining the role video material corresponding to the text sentence in the dialogue text includes: Obtain the role video material corresponding to the third text.

10. A video generating device, characterized in that: include: A response module, configured to obtain input information for representing target semantics in response to a first user operation, wherein the input information is composed of at least one text sentence; A processing module, configured to convert the input information into a dialogue text representing the target semantics, wherein the dialogue text is composed of at least two text sentences based on a dialogue sentence pattern; A generation module is used to obtain role video materials corresponding to text sentences in the dialogue text, and generate a target video for displaying the target semantics based on the role video materials.

11. An electronic device, characterized in that: include: Processor and memory; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the video generating method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the processor executes the computer-executable instructions, the video generation method according to any one of claims 1 to 9 is implemented.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the video generating method according to any one of claims 1 to 9 is implemented.

Images