AI video generation method, product and electronic equipment

Abstract

The invention discloses an AI video generation method, a product and electronic equipment. The AI video generation method comprises the steps of theme determination, part-of-speech tagging, part-of-speech extension, script generation, static picture acquisition, dynamic picture rendering and video generation. The method comprises the following steps: receiving a theme set by a user and corresponding to a video to be generated by the user, and a text statement corresponding to the video to be generated by the user; performing part-of-speech tagging on the character statement, and performing keyword extraction according to tagged part-of-speech information; according to the theme, adding a corresponding extension word to the keyword; generating a text script according to the theme, the keyword and the expanded keyword; obtaining a static picture according to the keyword; rendering the static picture into a dynamic picture by using the expanded keyword; fusing and splicing the generated dynamic pictures according to a character script to form a complete video picture; according to the invention, the new target video can be produced without the aid of the existing original video, and the application range of the video production system can be greatly expanded.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to an AI video generation method, product, and electronic device. Background Art

[0002] AI technology synthesis is also called deep synthesis technology, which refers to the technology of using generative synthesis algorithms represented by deep learning and virtual reality to produce text, images, audio, video, virtual scenes and other information. AI technology can facilitate users to create various videos, greatly improving work efficiency.

[0003] In the prior art, there is an AI-based video generation method, device and electronic device disclosed in publication number CN115460459A, which involves artificial intelligence fields such as deep learning and natural language processing. The method includes extracting target main words from original material video; identifying target description elements of the target main words and target description information of the target description elements from the original material video according to the target main words; obtaining description paragraphs of the target main words according to the target description elements and the target description information of the target description elements; and editing the original material video based on the description paragraphs to generate a target video. In the present disclosure, the waste of resources on the original material video that has been recorded is avoided, the target subject words of the original material video are extracted based on AI, and the original material video is edited according to the extracted target subject words, which saves labor costs and improves the efficiency of video editing. However, the existing technology is based on elemental material videos for AI creation to produce target videos. Therefore, the use of existing technology requires obtaining relevant original videos first, which not only requires spending more time to search for the original videos before creation, but also causes a certain impact on AI creation due to the limitations of the original videos. Secondly, for some videos that need to be created, they are often video scenes sketched out in the user's mind, that is, there is no relevant original video. For this situation, the existing technology cannot achieve AI video generation, which leads to large limitations in the existing generation method and a narrow scope of application. Summary of the invention

[0004] The purpose of the present invention is to provide an AI video generation method, product, and electronic device in order to solve at least one of the above-mentioned technical problems.

[0005] The present invention achieves the above-mentioned purpose through the following technical solutions: an AI video generation method, comprising the following steps:

[0006] S1, determining a theme, receiving a theme set by a user corresponding to a video that the user wants to generate, and a text sentence corresponding to the video that the user wants to generate;

[0007] S2, part-of-speech tagging, tagging the text sentences with parts of speech, extracting keywords based on the marked part-of-speech information, and obtaining keywords;

[0008] S3, part of speech extension, according to the theme, add corresponding extension words to the keywords to get the expanded keywords;

[0009] S4, generate a script, generate a text script according to the theme, keywords, and expanded keywords;

[0010] S5, obtaining a static picture, obtaining a static picture according to a keyword;

[0011] S6, rendering dynamic pictures, using the expanded keywords to render static pictures into dynamic pictures;

[0012] S7, generating a video, and fusing and splicing the generated dynamic images according to the text script to form a complete video image.

[0013] As a further solution of the present invention: in step S1, when a text sentence is received, the text sentence is in the form of text or voice. When the form is voice, the content of the voice is converted into text through natural language processing technology.

[0014] As a further solution of the present invention: in step S2, the specific steps of keyword extraction are:

[0015] S21, dividing the adjacent single or multiple characters in the text sentence that can form a word into a group, and separating the groups divided by the text sentence to obtain multiple groups of words;

[0016] S22, identifying the part of speech of each word group, deleting the word groups belonging to pronouns, prepositions, auxiliary words and interjections, and retaining other word groups in the text sentence;

[0017] S23. Check the retained word groups for duplicate content. Only one of the word groups with repeated content is retained. The retained word group is the keyword that needs to be extracted.

[0018] As a further solution of the present invention: In step S3, according to the subject, corresponding extension words are added to the keywords to obtain the specific steps of the expanded keywords:

[0019] Step S31, add extension words before the keywords: according to the subject, add corresponding modified or restrictive descriptive words before the keywords to make the keywords have more specific meanings; the modified or restrictive descriptive words are extension words;

[0020] Step S32, add extension words after the keywords. According to the subject, add corresponding modifying or limiting descriptive words after the keywords to make the keywords have more specific meanings; the modifying or limiting descriptive words are the extension words.

[0021] As a further solution of the present invention: in step S4, the specific steps of generating a text script according to the theme, keywords, and expanded keywords are:

[0022] S41, taking the keywords as the core and combining with the theme, generating a story framework, wherein the story framework includes time, place, and characters;

[0023] S42, further processing the story frame according to the expanded keywords, so that the time, place and characters in the story frame are further specified;

[0024] S43, generating a storyline based on the characters and places in the story frame, and then adding time to the storyline;

[0025] S44. Further process the storyline based on the theme to make it fit the theme and finally obtain a text script.

[0026] As a further solution of the present invention: in step S5, the specific steps of obtaining a static picture according to the keyword are as follows:

[0027] Use keywords to search in the big data library to obtain static images.

[0028] As a further solution of the present invention: In step S6, the specific steps of using the expanded keywords to render the static picture into a dynamic picture are as follows:

[0029] S61. Filtering out static images that match the theme from the static images;

[0030] S62, performing color adjustment and cutting processing on the static image according to the extended word;

[0031] S63. Use a video editing processor to dynamically process the objects in the static image to obtain a dynamic image.

[0032] As a further solution of the present invention: In step S7, according to the text script, the generated dynamic pictures are merged and spliced ​​to form a complete video picture:

[0033] S71, segmenting the text script according to the keywords, that is, dividing the text script into multiple scenes;

[0034] S72, splicing the dynamic images corresponding to each scene in the text script together to form continuous dynamic image frames;

[0035] S73, adjusting the brightness and size of each dynamic picture to be consistent, and performing transition modification on the splicing parts of the dynamic pictures to obtain a complete video picture.

[0036] The present invention also provides a computer program product, comprising a computer program, and the computer program implements the above method when executed by a processor.

[0037] The present invention also provides an AI video generation electronic device, comprising:

[0038] Memory, used to store computer programs;

[0039] The processor is used to implement the above method steps when executing the program stored in the memory.

[0040] The beneficial effects of the present invention are:

[0041] 1. The present invention can edit the video content information to be generated into text sentences and input them into the system, that is, it can produce the corresponding video in the form of text description, without the need to use the existing original video to assist in producing the new target video, which can greatly improve the application scope of the video production system. In addition, due to the use of text transmission, the user only needs to input a paragraph of words, which is easy to operate and use;

[0042] 2. The present invention can also supplement through associative mining to expand the keywords, that is, it can supplement the text sentences input by the user, which can ensure that the produced video has a stronger story and a more reasonable plot. That is, there is no need for the user to input a more detailed text information description, which greatly improves the intelligence of the video production system and is also simpler to operate. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a schematic diagram of the framework of the generation method of the present invention;

[0044] Figure 2 A video flow diagram is generated for the computer program product of the present invention. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] Embodiment 1

[0047] like Figure 1As shown, an AI video generation method includes the following steps:

[0048] S1, determining a theme, receiving a theme set by a user corresponding to a video that the user wants to generate, and a text sentence corresponding to the video that the user wants to generate;

[0049] S2, part-of-speech tagging, tagging the text sentences with parts of speech, extracting keywords based on the marked part-of-speech information, and obtaining keywords;

[0050] S3, part of speech extension, according to the theme, add corresponding extension words to the keywords to get the expanded keywords;

[0051] S4, generate a script, generate a text script according to the theme, keywords, and expanded keywords;

[0052] S5, obtaining a static picture, obtaining a static picture according to a keyword;

[0053] S6, rendering dynamic pictures, using the expanded keywords to render static pictures into dynamic pictures;

[0054] S7, generating a video, and fusing and splicing the generated dynamic images according to the text script to form a complete video image.

[0055] Embodiment 2

[0056] In addition to all the technical features of the first embodiment, this embodiment also includes:

[0057] In step S1, when receiving a text sentence, the text sentence is in the form of text or voice. When the form is voice, the content of the voice is converted into text through natural language processing technology;

[0058] In step S2, the specific steps of keyword extraction are:

[0059] S21, dividing the adjacent single or multiple characters in the text sentence that can form a word into a group, and separating the groups divided by the text sentence to obtain multiple groups of words;

[0060] S22, identifying the part of speech of each word group, deleting the word groups belonging to pronouns, prepositions, auxiliary words and interjections, and retaining other word groups in the text sentence;

[0061] S23. Check the retained word groups for duplicate content. Only one of the word groups with repeated content is retained. The retained word group is the keyword that needs to be extracted.

[0062] In step S3, according to the topic, corresponding extension words are added to the keywords to obtain the specific steps of the expanded keywords:

[0063] S31. Add extension words before keywords: According to the subject, add corresponding modifying or limiting descriptive words before keywords to make the keywords have more specific meanings; the modifying or limiting descriptive words are extension words;

[0064] S32. Extended words are added after the keywords. According to the subject, corresponding modified or restrictive descriptive words are added after the keywords to make the keywords have more specific meanings; the modified or restrictive descriptive words are extended words.

[0065] In step S4, the specific steps of generating a text script according to the theme, keywords, and expanded keywords are as follows:

[0066] S41, taking the keywords as the core and combining with the theme, generating a story framework, wherein the story framework includes time, place, and characters;

[0067] S42, further processing the story frame according to the expanded keywords, so that the time, place and characters in the story frame are further specified;

[0068] S43 generates a storyline based on the characters and places in the story frame, and then adds time to the storyline;

[0069] S44, further processing the storyline based on the theme so that the storyline is consistent with the theme, and finally obtaining a text script;

[0070] In step S5, the specific steps of obtaining a static image according to the keyword are as follows:

[0071] Use keywords to search in the big data library to get static images;

[0072] In step S6, the specific steps of using the expanded keywords to render the static image into a dynamic image are as follows:

[0073] S61. Filtering out static images that match the theme from the static images;

[0074] S62, performing color adjustment and cutting processing on the static image according to the extended word;

[0075] S63, using a video editing processor to dynamically process the objects in the static picture to obtain a dynamic picture;

[0076] In step S7, according to the text script, the generated dynamic images are merged and spliced ​​to form a complete video image.

[0077] S71, segmenting the text script according to the keywords, that is, dividing the text script into multiple scenes;

[0078] S72, splicing the dynamic images corresponding to each scene in the text script together to form continuous dynamic image frames;

[0079] S73, adjusting the brightness and size of each dynamic picture to be consistent, and performing transition modification on the splicing parts of the dynamic pictures to obtain a complete video picture.

[0080] Embodiment 3

[0081] This embodiment is a computer program product, including a computer program, and the computer program implements the above method when executed by a processor.

[0082] Embodiment 4

[0083] like Figure 2 As shown, this embodiment describes a method for a user to use the computer program product in Embodiment 3, comprising the following steps:

[0084] Step 1: Determine the theme of the video. Based on the input text and the determined theme, determine whether the produced video is cheerful and lively, warm and touching, tense and suspenseful, sad and heavy, or pleasant and relaxing;

[0085] Step 2: Segment the video frame by frame and group it into segments. Segment the video according to the picture frames, and group the video frames in the same scene into video segments.

[0086] Step 3: segment and match the text. According to the divided scene segments, the text in the generated text script that matches each scene segment is segmented and matched to the corresponding scene segment. For the text that does not belong to the scene segment, it is matched at the transition of each scene segment with narration.

[0087] Step 4: segment matching background dubbing: based on the determined video theme, select the corresponding background music, segment the background music into different lengths according to the duration of each scene segment, and match it to the corresponding scene segment;

[0088] Step 5: Video segment fusion: splice and fuse the video segments matched with text and background music together again to match subtitles and dubbing for the generated AI video.

[0089] Working principle: Edit the video content information to be generated into text sentences, set themes for the text sentences, and then input the edited text sentences into the system; perform part-of-speech tagging on the input text sentences, and extract keywords based on the annotated part-of-speech information; use the extracted keywords as core words, based on the set theme, supplement them through associative mining, and expand the keywords on the basis of keywords; combine the input text sentences with the expanded keywords to generate a complete text script that conforms to the set theme; render the things described by the keywords into static pictures; based on the generated static pictures, combine the things further described by the extended words to render the static pictures into dynamic pictures; combined with the generated text script, the generated dynamic pictures are merged and spliced ​​according to the plot settings in the script to form a complete video picture.

[0090] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

[0091] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. An AI video generation method, characterized in that: The following steps are involved: S1, determining a theme, receiving a theme set by a user corresponding to a video that the user wants to generate, and a text sentence corresponding to the video that the user wants to generate; S2, part-of-speech tagging, tagging the text sentences with parts of speech, extracting keywords based on the marked part-of-speech information, and obtaining keywords; S3, part of speech extension, according to the theme, add corresponding extension words to the keywords to get the expanded keywords; S4, generate a script, generate a text script according to the theme, keywords, and expanded keywords; S5, obtaining a static picture, obtaining a static picture according to a keyword; S6, rendering dynamic pictures, using the expanded keywords to render static pictures into dynamic pictures; S7, generating a video, and fusing and splicing the generated dynamic images according to the text script to form a complete video image.

2. The AI ​​video generation method according to claim 1, characterized in that: In S1, when a text sentence is received, the text sentence is in the form of text or voice. When the form is voice, the content of the voice is converted into text through natural language processing technology.

3. The AI ​​video generation method according to claim 1, characterized in that: In S2, the specific steps of keyword extraction are: S21, dividing the adjacent single or multiple characters in the text sentence that can form a word into a group, and separating the groups divided by the text sentence to obtain multiple groups of words; S22, identifying the part of speech of each word group, deleting the word groups belonging to pronouns, prepositions, auxiliary words and interjections, and retaining other word groups in the text sentence; S23. Check the retained word groups for duplicate content. Only one of the word groups with repeated content is retained. The retained word group is the keyword that needs to be extracted.

4. The AI ​​video generation method according to claim 1, characterized in that: In S3, according to the topic, add corresponding extension words to the keywords to obtain the specific steps of the expanded keywords: Step S31, add extension words before the keywords: according to the subject, add corresponding modified or restrictive descriptive words before the keywords to make the keywords have more specific meanings; the modified or restrictive descriptive words are extension words; Step S32, add extension words after the keywords. According to the subject, add corresponding modifying or limiting descriptive words after the keywords to make the keywords have more specific meanings; the modifying or limiting descriptive words are the extension words.

5. The AI ​​video generation method according to claim 1, characterized in that: In S4, the specific steps for generating a text script based on the theme, keywords, and expanded keywords are as follows: S41, taking the keywords as the core and combining with the theme, generating a story framework, wherein the story framework includes time, place, and characters; S42, further processing the story frame according to the expanded keywords, so that the time, place and characters in the story frame are further specified; S43, generating a storyline based on the characters and places in the story frame, and then adding time to the storyline; S44. Further process the storyline based on the theme to make it fit the theme and finally obtain a text script.

6. The AI ​​video generation method according to claim 1, characterized in that: In S5, the specific steps of obtaining a static image according to the keyword are as follows: Use keywords to search in the big data library to obtain static images.

7. The AI ​​video generation method according to claim 1, characterized in that: In S6, the specific steps of using the expanded keywords to render static images into dynamic images are as follows: S61. Filtering out static images that match the theme from the static images; S62, performing color adjustment and cutting processing on the static image according to the extended word; S63. Use a video editing processor to dynamically process the objects in the static image to obtain a dynamic image.

8. The AI ​​video generation method according to claim 1, characterized in that: In S7, according to the text script, the generated dynamic images are merged and spliced ​​to form a complete video image: S71, segmenting the text script according to the keywords, that is, dividing the text script into multiple scenes; S72, splicing the dynamic images corresponding to each scene in the text script together to form continuous dynamic image frames; S73, adjusting the brightness and size of each dynamic picture to be consistent, and performing transition modification on the splicing parts of the dynamic pictures to obtain a complete video picture.

9. A computer program product, comprising a computer program, characterized in that: The computer program implements the method according to any one of claims 1 to 8 when executed by a processor.

10. An AI video generation electronic device, characterized in that: include: Memory, used to store computer programs; A processor, for implementing the method steps described in any one of claims 1 to 8 when executing a program stored in a memory.

Images