An artificial intelligence-based method and system for generating animations for children's books

Abstract

The application discloses a kind of children's book animation generation method and system based on artificial intelligence, it is related to book animation generation technical field, the method includes: receiving and children's book related text content and audio content;The received text content and audio content are classified and data cleaning, and animation production material is obtained;Animation production material is analyzed to identify story scene, story element, emotional tone and sound effect element;According to the story scene, story element, emotional tone and sound effect element identified, generate animation script;According to animation script, design character modeling, dynamic background and static background;Character modeling, dynamic background and static background are matched and assembled with the role and event in animation script, and continuous animation sequence is generated;The continuous animation sequence generated is output in the form of video or the form of interactive electronic book.

Description

Technical Field

[0001] This application relates to the field of book animation generation technology, and more specifically, to a method and system for generating children's book animations based on artificial intelligence. Background Technology

[0002] Young children often cannot read children's books, such as picture books, independently due to their limited reading ability. Therefore, they need to use animation and other forms of assistance to help them understand the content.

[0003] Traditional animation production requires a significant investment of time and effort from professionals such as character designers, set designers, animators, and animation directors. This presents a substantial barrier for children, limiting their opportunities to develop creativity and language skills through animation creation.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This application provides a method and system for generating animations for children's books based on artificial intelligence, in order to solve the above-mentioned technical problems.

[0006] This application provides an AI-based method for generating animations for children's books, comprising: receiving text and audio content related to children's books; classifying and cleaning the received text and audio content to obtain animation production materials; analyzing the animation production materials to identify story scenes, story elements, emotional tone, and sound effects; generating an animation script based on the identified story scenes, story elements, emotional tone, and sound effects; designing character designs, dynamic backgrounds, and static backgrounds based on the animation script; matching and assembling the character designs, dynamic backgrounds, and static backgrounds with the characters and events in the animation script to generate a continuous animation sequence; and outputting the generated continuous animation sequence in the form of a video or an interactive e-book.

[0007] This application provides an AI-based children's book animation generation system, comprising: a multimodal input module for receiving text and audio content related to children's books; a preprocessing module for classifying and cleaning the received text and audio content to obtain animation production materials; a material analysis module for analyzing the animation production materials to identify story scenes, story elements, emotional tone, and sound effects; a story generation module for generating an animation script based on the identified story scenes, story elements, emotional tone, and sound effects; a character and scene design module for designing character models, dynamic backgrounds, and static backgrounds based on the animation script; an animation assembly module for matching and assembling the character models, dynamic backgrounds, and static backgrounds designed by the character and scene design module with the characters and events in the animation script generated by the story generation module to generate a continuous animation sequence; and an animation output module for outputting the generated continuous animation sequence in the form of a video or an interactive e-book.

[0008] Based on the embodiments provided in this application, text and audio content related to children's books are received; the received text and audio content are classified and cleaned to obtain animation production materials; the animation production materials are analyzed to identify story scenes, story elements, emotional tone, and sound effects; an animation script is generated based on the identified story scenes, story elements, emotional tone, and sound effects; character designs, dynamic backgrounds, and static backgrounds are designed based on the animation script; the character designs, dynamic backgrounds, and static backgrounds are matched and assembled with the characters and events in the animation script to generate a continuous animation sequence; and the generated continuous animation sequence is output in the form of a video or an interactive e-book. By converting the text and audio content of children's books into animation, it is possible to effectively attract the attention of young children, enhance their reading interest and comprehension, and thus help them better understand and absorb the book content. Automation simplifies the animation production process, reduces reliance on professionals, and allows children to participate in animation creation, lowering the technical barrier to entry. Because children can directly participate in the animation creation process, their creativity and imagination are fully developed, helping to enhance their creativity and innovative thinking. Through interaction with the animated content, children can better understand and learn language, especially during the critical period of language learning; this interactive learning can… It can significantly improve children's language expression skills; personalized animation content can be customized according to children's reading level and interests, providing a learning experience that better meets individual needs; the automated animation generation process greatly shortens the animation production cycle, saves manpower and time costs, and makes animation production more efficient; by automatically recognizing emotional tone and sound effects, the animation content can be more vivid and expressive, enhancing children's emotional experience; the output animation sequence can be used as an interactive e-book, providing children with a brand-new interactive learning tool and increasing the fun of learning; combining the learning process of book content and animation production can promote children's learning in multiple disciplines such as language, art, and technology. Attached Figure Description

[0009] The accompanying drawings, which are included to provide a further understanding of embodiments of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0010] Figure 1 This is a flowchart of an optional artificial intelligence-based children's book animation generation method according to an embodiment of this application;

[0011] Figure 2 This is a structural diagram of an optional artificial intelligence-based children's book animation generation system according to an embodiment of this application.

[0012] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0013] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0014] Optionally, such as Figure 1 As shown, this application provides a method for generating animations for children's books based on artificial intelligence, including:

[0015] S101 receives text and audio content related to children's books;

[0016] S102, classify and clean the received text and audio content to obtain animation production materials;

[0017] S103, analyze animation production materials to identify story scenes, story elements, emotional tone and sound effects;

[0018] S104, Generate an animation script based on the identified story scenes, story elements, emotional tone, and sound effects elements;

[0019] S105, based on the animation script, design character designs, dynamic backgrounds, and static backgrounds;

[0020] S106, matching and assembling character designs, dynamic backgrounds, and static backgrounds with characters and events in the animation script to generate a continuous animation sequence;

[0021] S107 outputs the generated continuous animation sequence as a video or an interactive ebook.

[0022] Based on the embodiments provided in this application, text and audio content related to children's books are received; the received text and audio content are classified and cleaned to obtain animation production materials; the animation production materials are analyzed to identify story scenes, story elements, emotional tone, and sound effects; an animation script is generated based on the identified story scenes, story elements, emotional tone, and sound effects; character designs, dynamic backgrounds, and static backgrounds are designed based on the animation script; the character designs, dynamic backgrounds, and static backgrounds are matched and assembled with the characters and events in the animation script to generate a continuous animation sequence; and the generated continuous animation sequence is output in the form of a video or an interactive e-book. By converting the text and audio content of children's books into animation, it is possible to effectively attract the attention of young children, enhance their reading interest and comprehension, and thus help them better understand and absorb the book content. Automation simplifies the animation production process, reduces reliance on professionals, and allows children to participate in animation creation, lowering the technical barrier to entry. Because children can directly participate in the animation creation process, their creativity and imagination are fully developed, helping to enhance their creativity and innovative thinking. Through interaction with the animated content, children can better understand and learn language, especially during the critical period of language learning; this interactive learning can… It can significantly improve children's language expression skills; personalized animation content can be customized according to children's reading level and interests, providing a learning experience that better meets individual needs; the automated animation generation process greatly shortens the animation production cycle, saves manpower and time costs, and makes animation production more efficient; by automatically recognizing emotional tone and sound effects, the animation content can be more vivid and expressive, enhancing children's emotional experience; the output animation sequence can be used as an interactive e-book, providing children with a brand-new interactive learning tool and increasing the fun of learning; combining the learning process of book content and animation production can promote children's learning in multiple disciplines such as language, art, and technology.

[0023] Optionally, such as Figure 2 As shown, this application provides an artificial intelligence-based children's book animation generation system, including:

[0024] The multimodal input module 201 is used to receive text and audio content related to children's books;

[0025] The preprocessing module 202 is used to classify and clean the received text and audio content to obtain animation production materials;

[0026] The material analysis module 203 is used to analyze animation production materials to identify story scenes, story elements, emotional tone, and sound effects.

[0027] The story generation module 204 is used to generate an animation script based on the identified story scenes, story elements, emotional tone, and sound effects elements;

[0028] The character and scene design module 205 is used to design character appearances, dynamic backgrounds, and static backgrounds based on the animation script;

[0029] The animation assembly module 206 is used to match and assemble the character designs, dynamic backgrounds, and static backgrounds designed by the character and scene design module with the characters and events in the animation script generated by the story generation module to generate a continuous animation sequence.

[0030] The animation output module 207 is used to output the generated continuous animation sequence in the form of video or interactive e-book.

[0031] Furthermore, the audio content includes image content and sound content; the sound content includes narrative sound content and sound effects sound content; the animation production materials include reference text, reference images, reference narrative sound, and reference sound effects sound; the received text content and audio content are classified and cleaned to obtain animation production materials; the animation production materials are analyzed to identify story scenes, story elements, emotional tone, and sound effects elements, and configured as follows:

[0032] The audio content is separated into narrative audio and sound effects audio using a spectrum subtraction algorithm and a deep clustering algorithm. The separated narrative audio and sound effects audio are then denoised to obtain reference narrative audio and reference sound effects audio. The spectrum subtraction algorithm is called SpectralSubtraction, and the deep clustering algorithm is called DeepClustering.

[0033] The text corresponding to the audio of the reference content is cleaned, part-of-speech tagging is performed, context analysis is conducted, and semantic role tagging is performed to obtain the reference text.

[0034] The image content is denoised and its contrast is adjusted to obtain a reference image:

[0035] Identify story scenes, story elements, and emotional tone based on reference text, reference images, and narrated audio content.

[0036] One approach is to use sentiment analysis algorithms to identify emotions from the tone and intensity of the narration.

[0037] Identify ambient sounds and motion sounds from reference sound effects to determine sound effect elements.

[0038] Furthermore, the story generation module generates an animation script based on the identified story scenes, story elements, emotional tone, and sound effects elements, and is configured as follows:

[0039] A deep learning-based named entity recognition model is used to design the structure of the animation script based on the story scene and story elements; and to assign characters and events to each scene included in the structure of the animation script; where events include dialogue and actions;

[0040] Mapping emotional tone to the characters and events in various scenarios;

[0041] In events across various scenarios, mark the parts where sound effects are applied; these markings include the sound effect trigger time and the sound effect stop time.

[0042] The script content of the animation script is determined based on the characters and events in each scene, as well as the marked sound effects.

[0043] Furthermore, the deep learning-based named entity recognition model is a bidirectional long short-term memory network-conditional random field model; the bidirectional long short-term memory network-conditional random field model is also known as the BiLSTM-CRF model.

[0044] A deep learning-based named entity recognition model is used to design the structure of the animation script based on the story scenes and elements; and characters and events are assigned to each scene within the animation script structure, configured as follows:

[0045] Based on the story scenes and elements, BiLSTM networks are used to capture the structure of the animation script, including contextual information in the starting scene, contextual information in the development scene, and contextual information in the ending scene; BiLSTM network is a bidirectional long short-term memory network.

[0046] For each scenario This scene Convert the story scenes and story elements into word vectors Word vectors are extracted using a BiLSTM network. Features; This serves as an index marker for the scene.

[0047] The output of the BiLSTM network ; Indicates the first The hidden state sequence of a scene Including each word vector Forward context information and backward context information;

[0048] The output of the BiLSTM network is processed by a pre-defined linear layer. Mapped to the label space and input to The layer performs label sequence prediction and annotation; among which, ; It is the predicted label sequence; yes The transition probability matrix of the layer; Layer transition probability matrix Initialize based on the transition frequency between the character's tag and the event's tag; transition probability matrix. Used to indicate from the label The probability of transitioning to label k;

[0049] Define a function ; where, function The input includes Layer transition probability matrix and rule set ,function The output includes the adjusted transition probability matrix. ;

[0050]

[0051] For rule sets Each rule in In the rule set Calculate its influence and will Weighted merging into the transition probability matrix The transition probability; where, For rule set Index of rules in the middle;

[0052] Based on the predicted label sequence and the adjusted transition probability matrix The structure of an animation script includes various scenes. Assign roles and events.

[0053] This can be based on the allocation function. Assign roles and events;

[0054]

[0055] Allocation function It not only considers the predicted label sequence assignment function and the adjusted transition probability matrix It also considered the scenario. The specific context, such as the time, place, and existing characters in the scene, helps ensure that the assigned roles and events match the context of the scene.

[0056] Based on the following formula, Weighted merging into the transition probability matrix Regarding the transition probability:

[0057]

[0058] in, From the label The transition probability matrix for label k; From the label The adjusted transition probability matrix for label k; It is a rule The weights; For the first One scenario; It is a rule In the scene The influence function on the transition probability.

[0059] Furthermore, the animation assembly module matches and assembles the character designs, dynamic backgrounds, and static backgrounds designed by the character and scene design modules with the characters and events in the animation script generated by the story generation module, generating a continuous animation sequence, which is configured as follows:

[0060] For each scene in the animation script According to this scenario Includes the characters, events, and the scene. To set the emotional tone, select dynamic background elements from the dynamic background library and static elements from the static element library to enhance the scene atmosphere;

[0061] For each scene in the animation script This scene The system integrates characters and events, character designs, dynamic background elements, and static elements to generate a continuous animation sequence.

[0062] Furthermore, the story generation module, character and scene design module, and animation assembly module communicate with each other through a central coordinator; the central coordinator is used to synchronize the data between the story generation module, character and scene design module, and animation assembly module.

[0063] Furthermore, the character and scene design module includes a scene element library; the scene element library is used to match the corresponding scene elements according to the animation script generated by the story generation module.

[0064] Furthermore, the AI-based children's book animation generation system also includes:

[0065] The interactive feedback module is used to receive feedback information provided by users during the animation generation process, and to adjust the data of the story generation module, character and scene design module, and animation assembly module based on the feedback information.

[0066] It should be noted that the embodiments implemented in the AI-based children's book animation generation system in this application can be referenced in conjunction with the embodiments implemented in the AI-based children's book animation generation method, and will not be described in detail here.

[0067] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A children's book animation generation system based on artificial intelligence, characterized in that, include: A multimodal input module is used to receive text and audio content related to children's books; The preprocessing module is used to classify and clean the received text and audio content to obtain animation production materials; The material analysis module is used to analyze animation production materials to identify story scenes, story elements, emotional tone, and sound effects. The story generation module is used to generate animation scripts based on the identified story scenes, story elements, emotional tone, and sound effects. The character and scene design module is used to design character appearances, dynamic backgrounds, and static backgrounds based on the animation script; The animation assembly module is used to match and assemble the character designs, dynamic backgrounds, and static backgrounds designed by the character and scene design module with the characters and events in the animation script generated by the story generation module, thereby generating a continuous animation sequence. The animation output module is used to output the generated continuous animation sequence as a video or an interactive e-book. The story generation module is configured as follows: A deep learning-based named entity recognition model is used to design the structure of the animation script based on the story scene and story elements; and to assign characters and events to each scene included in the structure of the animation script; where events include dialogue and actions; and to map the emotional tone to the characters and events in each scene. A deep learning-based named entity recognition model is used to design the structure of the animation script based on the story scenes and elements; and characters and events are assigned to each scene within the animation script structure, configured as follows: Based on the story scenes and elements, a BiLSTM network is used to capture the structure of the animation script, including contextual information in the starting scene, contextual information in the development scene, and contextual information in the ending scene. For each scenario This scene Convert the story scenes and story elements into word vectors Word vectors are extracted using a BiLSTM network. Features; This serves as an index marker for the scene. The output of the BiLSTM network ; Indicates the first The hidden state sequence of a scene Including each word vector Forward context information and backward context information; The output of the BiLSTM network is processed by a pre-defined linear layer. Mapped to the label space and input to The layer performs label sequence prediction and annotation; among which, ; It is the predicted label sequence; yes The transition probability matrix of the layer; Layer transition probability matrix Initialize based on the transition frequency between the character's tag and the event's tag; transition probability matrix. Used to indicate from the label The probability of transitioning to label k; Define a function ; where, function The input includes Layer transition probability matrix and rule set ,function The output includes the adjusted transition probability matrix. ; For rule sets Each rule in In the rule set Calculate its influence and will Weighted merging into the transition probability matrix The transition probability; where, For rule set Index of rules in the middle; Based on the predicted label sequence and the adjusted transition probability matrix The structure of an animation script includes various scenes. Assign roles and events.

2. The children's book animation generation system based on artificial intelligence according to claim 1, characterized in that, Audio content includes image content and sound content; sound content includes narrative sound content and sound effects sound content; animation production materials include reference text, reference images, reference narrative sound, and reference sound effects sound; the received text content and audio content are classified and cleaned to obtain animation production materials; the animation production materials are analyzed to identify story scenes, story elements, emotional tone, and sound effects elements, and configured as follows: By using the spectrum subtraction algorithm and the deep clustering algorithm, the content description sound content and the sound effect sound content in the sound content are separated; The separated content narrative audio and sound effect audio are noise-reduced to obtain reference content narrative audio and reference sound effect audio. The text corresponding to the audio of the reference content is cleaned, part-of-speech tagging is performed, context analysis is conducted, and semantic role tagging is performed to obtain the reference text. The image content is denoised and its contrast is adjusted to obtain a reference image: Identify story scenes, story elements, and emotional tone based on reference text, reference images, and narrated audio content. Identify ambient sounds and motion sounds from reference sound effects to determine sound effect elements.

3. The artificial intelligence-based children's book animation generation system according to claim 1, characterized in that, Based on the following formula, Weighted merging into the transition probability matrix Regarding the transition probability: ； in, From the label The transition probability matrix for label k; From the label The adjusted transition probability matrix for label k; It is a rule The weights; For the first One scenario; It is a rule In the scene The influence function on the transition probability.

4. The artificial intelligence-based children's book animation generation system according to claim 1, characterized in that, The animation assembly module matches and assembles the character designs, dynamic backgrounds, and static backgrounds designed by the character and scene design modules with the characters and events in the animation script generated by the story generation module, producing a continuous animation sequence, which is configured as follows: For each scene in the animation script According to this scenario Includes the characters, events, and the scene. To set the emotional tone, select dynamic background elements from the dynamic background library and static elements from the static element library to enhance the scene atmosphere; For each scene in the animation script This scene The system integrates characters and events, character designs, dynamic background elements, and static elements to generate a continuous animation sequence.

5. The artificial intelligence-based children's book animation generation system according to claim 1, characterized in that, The story generation module, character and scene design module, and animation assembly module communicate with each other through a central coordinator; the central coordinator is used to synchronize data among the story generation module, character and scene design module, and animation assembly module.

6. The artificial intelligence-based children's book animation generation system according to claim 1, characterized in that, The character and scene design module includes a scene element library; the scene element library is used to match the corresponding scene elements to the animation script generated by the story generation module.

7. The artificial intelligence-based children's book animation generation system according to claim 1, characterized in that, The AI-based children's book animation generation system also includes: The interactive feedback module is used to receive feedback information provided by users during the animation generation process, and to adjust the data of the story generation module, character and scene design module, and animation assembly module based on the feedback information.

Images