Animation generation method based on AIGC

Abstract

The invention relates to the technical field of animation generation, in particular to an animation generation method based on AIGC. The method comprises the following steps: inputting a text corresponding to a story outline and an optimal cue word into a script generation model, and outputting a script text; respectively constructing a semantic relation graph and a spatial relation graph based on the phrase features and the object region features; calculating the similarity of the text images according to an integration result, judging whether the similarity exceeds a preset similarity threshold, and if yes, outputting a pre-generated scene image; and inputting a pre-generated scene image into the animation generation model, and outputting an initial animation video: generating a sound effect and background music of the initial animation video by adopting the optimal cue word combination, and automatically editing the generated animation to export a final animation file. According to the method, the animation creation efficiency and quality can be remarkably improved, the manufacturing cost is reduced, high customizability and expandability are achieved, and personalized customization and expansion can be carried out according to the requirements and preferences of users.

Description

Technical Field

[0001] The present invention relates to the technical field of animation generation, and particularly to an animation generation method based on AIGC. Background Art

[0002] With the rapid development of artificial intelligence technology, AIGC (Artificial Intelligence Generated Content) technology has become an important tool in the field of animation creation. Through a variety of AI technologies such as image recognition, image synthesis, and natural language processing, this technology can automate or semi-automate multiple links in animation creation, such as character design, scene construction, motion capture, dubbing, and background music. The existing technologies have the following problems:

[0003] There are still difficulties in dealing with complex contexts, polysemous words, and context implications. This results in the generated animation dialogues and narrations may seem monotonous, lacking depth and personalization;

[0004] There is often a lack of sufficient controllability when generating content. Users may have difficulty precisely specifying details such as the style, emotion, and actions of the generated content.

[0005] Therefore, there is an urgent need for an animation generation method based on AIGC to solve the above problems. Summary of the Invention

[0006] The purpose of the present invention is to provide an animation generation method based on AIGC: to solve the technical problem that the existing animation generation lacks sufficient controllability when dealing with complex contexts, polysemous words, and context implications.

[0007] The purpose of the present invention can be achieved by the following technical solutions:

[0008] An animation generation method based on AIGC, the method includes:

[0009] Input the story outline and the text corresponding to the best prompt words into the script generation model, and output the script text;

[0010] Use an NLP tool to perform text dependency parsing on the script text to obtain phrase features, and perform target region detection on the pre-generated scene image to obtain object region features; respectively construct a semantic relationship graph and a spatial relationship graph based on the phrase features and the object region features; calculate the first product of the phrase features and the semantic relationship graph, calculate the second product of the object region features and the spatial relationship graph, input the first product and the second product into two GCN networks respectively, output the integration result, calculate the similarity between the text and the image according to the integration result, and determine whether the similarity exceeds a preset similarity threshold. If so, output the pre-generated scene image;

[0011] Input the pre-generated scene image into the animation generation model to output the initial animation video;

[0012] Generate the sound effects and background music of the initial animation video using the optimal combination of prompting words, and automatically edit and export the generated animation to obtain the final animation file.

[0013] Furthermore, input the story outline and the text corresponding to the best prompting words into the script generation model, and the output script text specifically includes the following process:

[0014] Step 1: Combine the story outline and the best prompting words into a dataset S r Input it into the script generation model, and randomly initialize the parameters θ of the generator and discriminator of the script generation model respectively.

[0015] Step 2: Use the machine learning engine to pre-train the generator G on the dataset S r ; θ

[0016] Step 3: Use the generator G to generate a text set S θ ; f

[0017] Step 4: Pre-train the discriminator on the text set S f and the dataset S r through the following objective function

[0018]

[0019] where is the objective function expression, π p is the prior ratio, is the misclassification risk of the data in the dataset S r ;

[0020] Step 5: Determine whether the iteration number is reached. If so, output the text set S f , and use the text set S f as the script text. If not, return to Step 3 and loop through Steps 3 to 4 until the iteration number is reached.

[0021] Furthermore, use NLP tools to perform text dependency parsing on the script text to obtain phrase features, which specifically includes the following process:

[0022] Extract the field names and field types from the script text based on NLP tools, determine the relevance between fields according to the association rules, and perform PCA analysis on the related fields to obtain the phrase features between data fields. ​​

[0023] Furthermore, the process of obtaining the object region features by performing object region detection on the pre-generated scene image specifically includes the following steps:

[0024] Perform object region detection using the YOLOv4 algorithm to obtain the object region. Among them, the pre-generated scene image is an image dataset related to the corresponding text content of the script, obtained from the animation database.

[0025] Furthermore, the process of constructing a semantic relationship graph and a spatial relationship graph based on the phrase features and object region features respectively specifically includes the following steps:

[0026] Construct a semantic relationship graph G based on the phrase features and object region features respectively μ and a spatial relationship graph G v , where G μ is a directed graph, and G μ represents the semantic relative position relationship of different phrase features. The edges of G μ exist between the nodes whose relative position relationship is mentioned in the input sentence of the script text. The weight of the edge in the graph is the compressed word encoding corresponding to the word representing the relative position relationship; G v is a directed graph, whose nodes represent the detected object regions. Use polar coordinates to model the relative spatial distance and relative direction between pairs of target regions in the image to obtain the edge weights of G v . The distance between two target regions represents their relative spatial distance, and the relative direction between two target regions represents the category of the spatial relationship.

[0027] Furthermore, input the first product and the second product into two GCN networks respectively, output the integration result, and calculate the similarity of the text image according to the integration result, which specifically includes the following steps:

[0028] Input the first product and the second product into two GCN networks respectively, and output the integration result:

[0029]

[0030] Among them, K - 1 is the number of kernels of the GCN network, σ represents the ReLU activation function, Niμ is the neighbor node of the i-th phrase feature in the semantic relationship graph G μ , is the neighbor node of the i-th object feature in the spatial relationship graph G v , p j is the j-th adjacent phrase feature in N i μ , f j is the j-th adjacent region feature in b μFor G μ The learning parameters of the corresponding GCN network b v For G v The learning parameters of the corresponding GCN network, p i And f i Are the context text features and context visual features output by the two GCN networks respectively. n is the number of neighbor nodes in the semantic relation graph G μ In, and n0 is the number of adjacent region features in the spatial relation graph G v In;

[0031] Record the context text features and context visual features as the integration result;

[0032] Calculate the Euclidean distance between the two feature vectors based on the context text features and context visual features, and record the distance value as the similarity of the text image.

[0033] Furthermore, input the pre-generated scene image into the animation generation model, and the output of the initial animation video specifically includes the following process:

[0034] Ensure that the formats and resolutions of the pre-generated scene images are consistent;

[0035] Input the pre-generated scene image into the Vid2Vid model;

[0036] Output the initial animation video.

[0037] Furthermore, using the optimal prompt word combination to generate the sound effects and background music of the initial animation video specifically includes the following process:

[0038] Generate the sound effects and background music of the initial animation video based on the optimal prompt word combination according to the Jukedeck model.

[0039] Furthermore, automatically editing and exporting the generated animation to export the final animation file specifically includes the following process:

[0040] Shot switching: Automatically determine the timing and method of shot switching according to the rhythm and plot development of the animation;

[0041] Synchronization of sound effects and music: Synchronize the generated sound effects and background music with the animation content to ensure that the rhythm of the sound effects and music matches the actions and emotions of the animation;

[0042] Special effect addition: Automatically add transition effects, subtitles, and title special effects as needed;

[0043] Use video editing software or tools that support the automatic editing function. The video editing software or tools include the automatic editing function of Adobe Premiere Pro and the automatic editing template of Final Cut Pro;

[0044] Select the appropriate format: According to the compatibility of the target platform and device, select the appropriate video format and encoding settings;

[0045] Resolution and frame rate: Set the resolution and frame rate according to the quality and smoothness requirements of the animation;

[0046] Export the animation file: Use the export function of the video editing software to export the edited animation file to the disk.

[0047] Compared with the existing solutions, the beneficial effects achieved by the present invention are:

[0048] In the present invention, the story outline and the text corresponding to the best prompt words are input into the script generation model to output the script text; the NLP tool is used to perform text dependency parsing on the script text to obtain phrase features, and object region features are obtained by performing target region detection on the pre-generated scene images; a semantic relationship graph and a spatial relationship graph are respectively constructed based on the phrase features and the object region features; the first product of the phrase features and the semantic relationship graph is calculated, the second product of the object region features and the spatial relationship graph is calculated, the first product and the second product are respectively input into two GCN networks, the integration result is output, the similarity of the text image is calculated according to the integration result, and it is judged whether the similarity exceeds the preset similarity threshold. If so, the pre-generated scene image is output; the pre-generated scene image is input into the animation generation model to output the initial animation video: the optimal prompt word combination is used to generate the sound effects and background music of the initial animation video, and the generated animation is automatically edited and exported to obtain the final animation file. The present invention can significantly improve the efficiency and quality of animation creation, reduce the production cost, and has high customizability and scalability, and can be customized and extended according to the needs and preferences of users.

[0049] Furthermore, through advanced AIGC technology, animation creators can produce high-quality animation works more efficiently and conveniently. Description of the Drawings

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required to be used in the embodiments. Obviously, the drawings described below are only some embodiments recorded in the present invention, and those of ordinary skill in the art can also obtain other drawings based on these drawings.

[0051] Figure 1It is the flowchart of the first AIGC-based animation generation method according to an embodiment of the present invention;

[0052] Figure 2 It is the flowchart of the second AIGC-based animation generation method according to an embodiment of the present invention;

[0053] Figure 3 It is the flowchart of the third AIGC-based animation generation method according to an embodiment of the present invention. Detailed implementation manners

[0054] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0055] In addition, the described features, structures or characteristics may be combined in any suitable manner in one or more example embodiments. In the following description, many specific details are provided to give a full understanding of the example embodiments of the present disclosure. However, those skilled in the art will realize that the technical solutions of the present disclosure may be practiced by omitting one or more of the specific details, or by using other methods, components, steps, etc. In other cases, well-known structures, methods, implementations or operations are not shown or described in detail to avoid obscuring various aspects of the present disclosure.

[0056] This embodiment provides an AIGC-based animation generation method, Figure 1 It is the flowchart of the first AIGC-based animation generation method according to an embodiment of the present invention, as Figure 1 shown. The method includes the following steps:

[0057] Step S101: Input the story outline and the text corresponding to the best prompt words into the script generation model, and output the script text;

[0058] Step S102: Use an NLP tool to perform text dependency parsing on the script text to obtain phrase features, and perform target region detection on the pre-generated scene images to obtain object region features;

[0059] Step S103: Build a semantic relationship graph and a spatial relationship graph based on the phrase features and the object region features respectively;

[0060] Step S104: Calculate the first product of the phrase features and the semantic relationship graph, calculate the second product of the object region features and the spatial relationship graph, input the first product and the second product into two GCN networks respectively, output the integration result, and calculate the similarity of the text image according to the integration result;

[0061] Step S105: Determine whether the similarity exceeds a preset similarity threshold. If so, proceed to Step S106; if not, proceed to Step S109;

[0062] Step S106: Output the pre-generated scene image;

[0063] Step S107: Input the pre-generated scene image into the animation generation model to output the initial animation video;

[0064] Step S108: Generate the sound effects and background music of the initial animation video using the optimal prompt word combination, and automatically edit and export the generated animation to obtain the final animation file;

[0065] Step S109: Discard the pre-generated scene image and regenerate the scene image based on the script text.

[0066] In summary, the present invention inputs the story outline and the text corresponding to the best prompt words into the script generation model to output the script text; uses the NLP tool to perform text dependency parsing on the script text to obtain phrase features, and performs target region detection on the pre-generated scene image to obtain object region features; constructs a semantic relationship graph and a spatial relationship graph based on the phrase features and the object region features respectively; calculates the first product of the phrase features and the semantic relationship graph, calculates the second product of the object region features and the spatial relationship graph, inputs the first product and the second product into two GCN networks respectively, outputs the integration result, calculates the similarity of the text image according to the integration result, determines whether the similarity exceeds a preset similarity threshold, and if so, outputs the pre-generated scene image; inputs the pre-generated scene image into the animation generation model to output the initial animation video: uses the optimal prompt word combination to generate the sound effects and background music of the initial animation video, and automatically edits and exports the generated animation to obtain the final animation file. The present invention can significantly improve the efficiency and quality of animation creation, reduce the production cost, has high customizability and scalability, and can be customized and extended according to the needs and preferences of users.

[0067] In some embodiments, inputting the story outline and the text corresponding to the best prompt words into the script generation model to output the script text specifically includes the following process:

[0068] Step 1: The data set S composed of the story outline and the best prompt words rIn the input script generation model, the parameters θ of the generator and discriminator of the script generation model are randomly initialized respectively.

[0069] Step 2: Use the machine learning engine to pre-train the generator G on the dataset S r ; θ ;

[0070] Step 3: Use the generator G to generate the text set S θ ; f ;

[0071] Step 4: Pre-train the discriminator on the text set S f and the dataset S r through the following objective function

[0072] where is the objective function expression, π p is the prior ratio, is the misclassification risk of the data in the dataset S r ;

[0073] Step 5: Determine whether the iteration number is reached. If so, output the text set S f , and use the text set S f as the script text. If not, return to Step 3 and loop through Steps 3 to 4 until the iteration number is reached.

[0074] In some embodiments, Figure 2 is the workflow diagram of the second AIGC-based animation generation method of the embodiments of the present invention. As Figure 2 shown, using the NLP tool to perform text dependency parsing on the script text to obtain phrase features specifically includes the following steps:

[0075] Step S201: Extract the field names and field types from the script text based on the NLP tool;

[0076] Specifically, based on NLP (Natural Language Processing), the data fields of the script text are identified: field names, Chinese field names, and field types.

[0077] Step S202: Determine the relevance between fields for the field names and field types according to association rule mining;

[0078] Use association rule mining algorithms such as Apriori and FP-Growth to identify frequent item sets and generate association rules, and determine the relevance between fields for the field names and field types according to association rule mining.

[0079] Step S203: Perform PCA analysis on the associated fields to obtain the phrase features between the data fields.

[0080] Specifically, (1) form a matrix X from the associated fields and calculate the covariance matrix of the samples of matrix X;

[0081] (2) Calculate the eigenvectors e1, e2, …, e of the covariance matrix m ;

[0082] (3) Project the data into the space of the eigenvectors, using the formula:

[0083]

[0084] where the BV value is the value of the corresponding dimension in the original sample, and m is the number of eigenvectors.

[0085] Extract the features of the fields where newBV is greater than the preset threshold to obtain the phrase features.

[0086] In some embodiments, the process of obtaining the object region features by performing object region detection on the pre-generated scene image specifically includes the following steps:

[0087] Use the YOLOv4 algorithm to perform object region detection to obtain the object region, where the pre-generated scene image is the relevant image dataset corresponding to the script text content and is obtained from the animation database.

[0088] In some embodiments, the process of constructing a semantic relationship graph and a spatial relationship graph based on the phrase features and the object region features respectively specifically includes the following steps:

[0089] Construct a semantic relationship graph G μ and a spatial relationship graph G v , where G μ is a directed graph, G μ represents the semantic relative position relationship of different phrase features, and the edges of G μ exist between the nodes whose relative position relationship is mentioned in the script text input sentence. The weight of the edge in the graph is the compressed word encoding corresponding to the word representing the relative position relationship; G v is a directed graph, whose nodes represent the detected object regions. Use polar coordinates to model the relative spatial distance and relative direction between pairs of target regions in the image to obtain the edge weights of G v . The distance between two target regions represents their relative spatial distance, and the relative direction between two target regions represents the category of the spatial relationship.

[0090] In some embodiments, the first product and the second product are respectively input into two GCNs (Graph Convolutional Networks), and an integration result is output. Calculating the similarity of the text image according to the integration result specifically includes the following process:

[0091] Input the first product and the second product into two GCN networks respectively, and output the integration result:

[0092]

[0093] where K - 1 is the number of kernels of the GCN network, σ represents the ReLU activation function, N i μ is the neighbor node of the i-th phrase feature in the semantic relationship graph G μ in, is the neighbor node of the i-th object feature in the spatial relationship graph G v in, p j is N i μ the j-th adjacent phrase feature in, f j is the j-th adjacent region feature in, b μ is the learning parameter of the GCN network corresponding to G μ ; b v is the learning parameter of the GCN network corresponding to G v ; p i and f i are respectively the context text feature and the context visual feature output by the two GCN networks. n is the number of neighbor nodes in the semantic relationship graph G μ ; n0 is the number of adjacent region features in the spatial relationship graph G v ;

[0094] Record the context text feature and the context visual feature as the integration result;

[0095] Calculate the Euclidean distance between the two feature vectors based on the context text feature and the context visual feature, and record the distance value as the similarity of the text image.

[0096] In some embodiments, Figure 3 is the flowchart of the third AIGC-based animation generation method according to the embodiments of the present invention. As Figure 3 shown, inputting the pre-generated scene image into the animation generation model and outputting the initial animation video specifically includes the following process:

[0097] Step S301: Ensure that the formats and resolutions of the pre-generated scene images are consistent;

[0098] Step S302: Input the pre-generated scene image into the Vid2Vid model;

[0099] Step S303: Output the initial animated video.

[0100] It should be noted that the Vid2Vid model is an image translation model that can generate videos based on the input semantic map. Generator design:

[0101] The generator of the Vid2Vid model adopts the Coarse-to-fine idea and designs two generators G1 and G2. G1 is used for global perception to generate low-resolution images and optical flow; G2 is used for local enhancement to generate high-resolution images. During the generation process, first, G1 generates a low-clarity image and optical flow estimation from the input semantic map and historical images. Then, a Warped distorted map is obtained using the historical images and optical flow estimation. Finally, the final high-definition image is obtained using the distorted map and the low-clarity image, and the construction of the next frame of the image is carried out.

[0102] Discriminator design:

[0103] The Vid2Vid model proposes two conditional discriminators, namely an image discriminator and a video discriminator. The image discriminator is used to ensure the authenticity of the generated images, belonging to the same image domain as the labels; the video discriminator is used to ensure from the perspective of optical flow, ensuring the similarity between the generated image and the real image under the same optical flow and focusing on coherence. In addition, to distinguish high-resolution images, the image discriminator also adopts the method of a multi-scale discriminator. The real and synthetic images are sampled at 1x, 2x, and 4x by discriminators of different sizes to obtain three-scale image pyramids and are trained.

[0104] Loss function:

[0105] The Vid2Vid model uses five loss functions to ensure the quality and coherence of the generated video. These loss functions include image distribution loss (used to ensure the authenticity of the generated images), video loss (used to ensure the coherence of the generated video), optical flow loss (used to ensure the correctness of the estimated optical flow), Feature matching loss, and Content loss (the latter two are to extract features from the generated samples and real samples using the discriminator and VGG 16 respectively, and calculate the Element-wise loss using the feature maps to ensure content consistency and improve the stability of training).

[0106] Foreground and background distinction modeling:

[0107] For tasks such as semantic map to street view map conversion, there are usually significant differences between the foreground and the background. Therefore, the Vid2Vid model adopts a method of separately modeling the foreground and the background, which is conducive to accelerating the convergence speed and conducting targeted training.

[0108] Diversified control of output:

[0109] To control the customization of objects when generating videos, the Vid2Vid model inputs an additional features map outside the original input semantic labels map. By changing the labels of the color patches in the map, the replacement of things can be controlled, thus achieving diversified control of output.

[0110] In some embodiments, generating the sound effects and background music of the initial animation video using the optimal combination of prompt words specifically includes the following process:

[0111] Generating the sound effects and background music of the initial animation video based on the Jukedeck model according to the optimal combination of prompt words.

[0112] Jukedeck is a machine learning-based music generation platform that can automatically generate customized music according to the prompts provided by users (such as style, emotion, instruments, etc.). Although Jukedeck does not directly provide specific functions for the sound effects and background music of animation videos, the generation of music that meets the requirements of animation videos can be guided by optimizing the combination of prompt words.

[0113] In some embodiments, automatically editing and exporting the generated animation to obtain the final animation file specifically includes the following process:

[0114] Shot transition: Automatically determine the timing and method of shot transitions according to the rhythm and plot development of the animation;

[0115] Synchronization of sound effects and music: Synchronize the generated sound effects and background music with the animation content to ensure that the rhythm of the sound effects and music matches the actions and emotions of the animation;

[0116] Special effect addition: Automatically add transition effects, subtitles, and title special effects as needed;

[0117] Use video editing software or tools that support automatic editing functions. The video editing software or tools include the automatic editing function of Adobe Premiere Pro and the automatic editing template of Final Cut Pro;

[0118] Select the appropriate format: Select the appropriate video format and encoding settings according to the compatibility of the target platform and device;

[0119] Resolution and frame rate: Set the resolution and frame rate according to the quality and smoothness requirements of the animation;

[0120] Export the animation file: Use the export function of the video editing software to export the edited animation file to the disk.

[0121] The above embodiments can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wired (such as infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more collections of available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state drive.

[0122] Those of ordinary skill in the art will realize that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or in a combination of computer software and electronic hardware. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of the present application.

[0123] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments and will not be repeated here.

[0124] In several embodiments provided in the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only for some logical function divisions. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection to each other can be through some interfaces. The indirect coupling or communication connection of the devices or units can be in electrical, mechanical, or other forms.

[0125] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0126] As mentioned above, the above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed in the present application can easily think of changes or substitutions, which should all be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An animation generation method based on AIGC, characterized in that: Methods include: Input the story outline and the text corresponding to the best prompt words into the script generation model, and output the script text; Use NLP tools to perform text dependency analysis on the script text to obtain phrase features, and perform target area detection on the pre-generated scene images to obtain object area features; Construct semantic relationship graph and spatial relationship graph based on phrase features and object region features respectively; Calculate the first product of the phrase feature and the semantic relationship graph, calculate the second product of the object region feature and the spatial relationship graph, input the first product and the second product into two GCN networks respectively, output the integration result, calculate the similarity of the text image according to the integration result, and determine whether the similarity exceeds the preset similarity threshold. If so, output the pre-generated scene image; Input the pre-generated scene image into the animation generation model and output the initial animation video; The optimal combination of prompt words is used to generate the sound effects and background music of the initial animation video, and the generated animation is automatically edited and exported to export the final animation file.

2. The animation generation method based on AIGC according to claim 1, characterized in that: The story outline and the text corresponding to the best prompt words are input into the script generation model. The specific process of outputting the script text includes the following: Step 1: Take the data set S consisting of the story outline and the best prompt words r In the input script generation model, the parameters θ, Step 2: Use the machine learning engine to train the dataset S r Pre-trained generator G θ ; Step 3: Use the generator G θ Generate text set S f ; Step 4: In the text set S f And dataset S r The discriminator is pre-trained with the following objective function in, is the objective function expression, π p is the prior proportion, For the dataset S r The risk of misclassification of data; Step 5: Determine whether the number of iterations has been reached. If so, output the text set S f , the text set S f As the script text, if not, return to step three and loop step three to step four until the number of iterations is reached.

3. The animation generation method based on AIGC according to claim 1, characterized in that: Use NLP tools to perform text dependency analysis on the script text to obtain phrase features The process includes: Based on NLP tools, field names and field types are extracted from the script text. The correlation between field names and field types is determined based on association rule mining. The associated fields are subjected to PCA analysis to obtain phrase features between data fields.

4. The AIGC-based animation generation method according to claim 1, characterized in that: Perform target area detection on the pre-generated scene image to obtain specific object area features The process includes: The YOLOv4 algorithm is used to detect the target area and obtain the object area, wherein the pre-generated scene image is a relevant image data set corresponding to the script text content, which is obtained from the animation database.

5. The AIGC-based animation generation method according to claim 1, characterized in that: Constructing semantic relationship graph and spatial relationship graph based on phrase features and object region features respectively includes the following processes: Semantic relationship graphs G are constructed based on phrase features and object region features. μ and spatial relationship graph G v , where G μ is a directed graph, G μ Indicates the semantic relative position relationship of different phrase features, G μ The edge exists between nodes mentioned in the script text input sentence in a relative position relationship. The weight of the edge in the graph is the compressed word code corresponding to the word representing the relative position relationship; G v is a directed graph whose nodes represent the detected object regions. Polar coordinates are used to model the relative spatial distance and relative direction between paired target regions in the image, and G is obtained. v The edge weight of , the distance between the two target regions, represents the relative spatial distance between them, and the relative direction between the two target regions represents the category of the spatial relationship.

6. The AIGC-based animation generation method according to claim 1, characterized in that: The first product and the second product are input into two GCN networks respectively, and the integration result is output. The similarity of the text image is calculated based on the integration result, which specifically includes the following process: Input the first product and the second product into two GCN networks respectively, and output the integrated result: Among them, K-1 is the number of cores of the GCN network, σ represents the ReLU activation function, and N i μ is the feature of the i-th phrase in the semantic relationship graph G μ The neighbor nodes in is the feature of the i-th object in the spatial relationship graph G v The neighbor nodes in p j YesN i μ The jth adjacent phrase feature in j YesN i v The jth adjacent region feature in b μ G μ The corresponding GCN network learning parameters, b v G v The corresponding GCN network learning parameters, p i and f i are the contextual text features and contextual visual features output by the two GCN networks respectively, and n is the semantic relationship graph G μ The number of neighbor nodes in the spatial relationship graph G v The number of adjacent region features in ; The context text features and the context visual features are recorded as the integration result; The Euclidean distance between two feature vectors is calculated based on the contextual text features and the contextual visual features, and the distance value is recorded as the similarity of the text image.

7. The AIGC-based animation generation method according to claim 1, characterized in that: Inputting the pre-generated scene image into the animation generation model and outputting the initial animation video specifically includes the following process: Ensure that the format and resolution of pre-generated scene images are consistent; Input the pre-generated scene image into the Vid2Vid model; Output the initial animation video.

8. The AIGC-based animation generation method according to claim 1, characterized in that: The process of using the optimal cue word combination to generate the sound effects and background music of the initial animation video specifically includes the following steps: Based on the Jukedeck model, the sound effect background music of the initial animation video is generated according to the optimal prompt word combination.

9. The AIGC-based animation generation method according to claim 1, characterized in that: Automatically clip and export the generated animation. Exporting the final animation file includes the following processes: Camera switching: Automatically determine the timing and method of camera switching according to the rhythm of the animation and the development of the plot; Synchronization of sound effects and music: Synchronize the generated sound effects and background music with the animation content to ensure that the rhythm of the sound effects and music matches the movements and emotions of the animation; Add special effects: automatically add transition effects, subtitles, and title special effects as needed; Use video editing software or tools that support automatic editing functions, including the automatic editing function of Adobe Premiere Pro and the automatic editing template of Final Cut Pro; Choose the right format: Choose the right video format and encoding settings based on the compatibility of the target platform and device; Resolution and frame rate: Set the resolution and frame rate according to the quality and smoothness requirements of the animation; Export animation files: Use the export function of the video editing software to export the edited animation files to disk.

Images