Picture book content generation method, device, equipment and storage medium

Abstract

The present disclosure provides a method, device, equipment and storage medium for generating picture book content, and relates to the field of artificial intelligence technology. In some embodiments of the present disclosure, a picture book keyword description text and a picture book theme are obtained; the picture book keyword description text and the picture book theme are input into a large language model that has been trained to obtain a picture book text, and the picture book text can be quickly obtained using the large language model; keywords are extracted from the picture book text to obtain keywords of the picture book text; keywords of the picture book text are input into a picture generation model that has been trained to obtain picture book illustrations, and picture book illustrations are quickly generated through keywords of the picture book text; the picture book text and the picture book illustrations are synthesized to obtain target picture book content; compared with the method of manually making picture books, the present disclosure reduces labor costs and improves the generation efficiency of picture books.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to a method, device, equipment and storage medium for generating picture book content. Background Art

[0002] Picture books are picture books suitable for children to read. They usually use rich illustrations and easy-to-understand text to tell stories, convey knowledge or inspire thinking. Picture books are designed to provide children with an opportunity to get close to words and images, and cultivate their interest in reading and comprehension. Picture books can be different types such as story books, cognitive enlightenment books, and emotion books, suitable for children of all ages to read.

[0003] At present, picture book content is mainly edited manually, which results in high labor costs and low efficiency in picture book generation. Summary of the invention

[0004] The present disclosure provides a method, device, equipment and storage medium for generating picture book content, so as to at least solve the problems of high labor cost and low efficiency of generating picture books.

[0005] The technical solution of the present disclosure is as follows:

[0006] The present disclosure provides a method for generating picture book content, including:

[0007] Get the picture book keyword description text and picture book theme;

[0008] Inputting the picture book keyword description text and the picture book theme into the trained large language model to obtain the picture book text;

[0009] Extracting keywords from the picture book text to obtain keywords of the picture book text;

[0010] Input the keywords of the picture book text into the trained image generation model to obtain picture book illustrations;

[0011] The picture book text and the picture book illustration are synthesized to obtain target picture book content.

[0012] Optionally, after obtaining the picture book keyword description text, the method further includes:

[0013] Inputting the picture book keyword description text into the large language model to obtain an initial picture book text;

[0014] The picture book keyword description text is input into the image generation model to obtain an initial picture book illustration.

[0015] Optionally, the step of inputting the picture book keyword description text and the picture book theme into a trained large language model to obtain the picture book text includes:

[0016] Inputting the picture book keyword description text and the picture book theme into the trained large language model to obtain the original picture book text;

[0017] In response to the modification operation on the original picture book text, the picture book text is obtained.

[0018] Optionally, extracting keywords from the picture book text to obtain keywords of the picture book text includes:

[0019] Performing data cleaning on the picture book text to obtain a cleaned picture book text;

[0020] Performing word segmentation processing on the cleaned picture book text to obtain original word segmentations;

[0021] According to the preset filtering rules, the candidate participles are filtered out from the original participles;

[0022] According to the importance of the candidate participles, keywords of the picture book text are selected from the candidate participles.

[0023] Optionally, the keywords of the picture book text are input into a trained image generation model to obtain picture book illustrations, including:

[0024] Input the keywords corresponding to each picture book text segment into the trained image generation model to obtain the picture book illustration corresponding to each picture book text segment, wherein the picture book text includes a plurality of the picture book text segments;

[0025] The picture book text and the picture book illustration are synthesized to obtain target picture book content, including:

[0026] Each picture book text segment and the picture book illustration corresponding to each picture book text segment are synthesized separately to obtain the target picture book content.

[0027] Optionally, after synthesizing the picture book text and the picture book illustration to obtain target picture book content, the method further includes:

[0028] The target picture book content is sent to a user terminal so that the user terminal displays the target picture book content.

[0029] Optionally, before using the large language model, the method further includes:

[0030] Collect multiple original picture book contents from the Internet;

[0031] Preprocessing the multiple original picture book contents to obtain multiple sample picture book contents;

[0032] Extracting features of each sample picture book content to obtain key features corresponding to each sample picture book content;

[0033] Initial model training is performed based on key features corresponding to the multiple sample picture book contents and the multiple sample picture book contents to obtain the large language model.

[0034] Optionally, before using the picture to generate the model, the method further includes:

[0035] Obtaining multiple types of sample picture book images; and

[0036] Obtain sample keywords corresponding to each type of the sample picture book image;

[0037] The deep learning model is trained according to the multiple types of sample picture book images and the sample keywords corresponding to the multiple types of sample picture book images to obtain the image generation model.

[0038] The present disclosure also provides a picture book content generation device, including:

[0039] The acquisition module is used to obtain the picture book keyword description text and picture book theme;

[0040] A text generation module, used for inputting the picture book keyword description text and the picture book theme into the trained large language model to obtain the picture book text;

[0041] An extraction module, used to extract keywords from the picture book text to obtain keywords of the picture book text;

[0042] An image generation module, used for inputting keywords of the picture book text into a trained image generation model to obtain picture book illustrations;

[0043] A synthesis module is used to synthesize the picture book text and the picture book illustration to obtain the target picture book content.

[0044] Optionally, after obtaining the picture book keyword description text, the acquisition module may also be used to:

[0045] Inputting the picture book keyword description text into the large language model to obtain an initial picture book text;

[0046] The picture book keyword description text is input into the image generation model to obtain an initial picture book illustration.

[0047] Optionally, when the text generation module inputs the picture book keyword description text and the picture book theme into a trained large language model to obtain the picture book text, it is used to:

[0048] Inputting the picture book keyword description text and the picture book theme into the trained large language model to obtain the original picture book text;

[0049] In response to the modification operation on the original picture book text, the picture book text is obtained.

[0050] Optionally, when the extraction module extracts keywords from the picture book text to obtain keywords of the picture book text, it is used to:

[0051] Performing data cleaning on the picture book text to obtain a cleaned picture book text;

[0052] Performing word segmentation processing on the cleaned picture book text to obtain original word segmentations;

[0053] According to the preset filtering rules, the candidate participles are filtered out from the original participles;

[0054] According to the importance of the candidate participles, keywords of the picture book text are selected from the candidate participles.

[0055] Optionally, when the picture generation module inputs the keywords of the picture book text into the trained picture generation model to obtain the picture book illustration, it is used to:

[0056] Input the keywords corresponding to each picture book text segment into the trained image generation model to obtain the picture book illustration corresponding to each picture book text segment, wherein the picture book text includes a plurality of the picture book text segments;

[0057] The picture book text and the picture book illustration are synthesized to obtain target picture book content, including:

[0058] Each picture book text segment and the picture book illustration corresponding to each picture book text segment are synthesized separately to obtain the target picture book content.

[0059] Optionally, after synthesizing the picture book text and the picture book illustration to obtain the target picture book content, the synthesis module may also be used to:

[0060] The target picture book content is sent to a user terminal so that the user terminal displays the target picture book content.

[0061] Optionally, before using the large language model, the text generation module may also be used to:

[0062] Collect multiple original picture book contents from the Internet;

[0063] Preprocessing the multiple original picture book contents to obtain multiple sample picture book contents;

[0064] Extracting features of each sample picture book content to obtain key features corresponding to each sample picture book content;

[0065] Initial model training is performed based on key features corresponding to the multiple sample picture book contents and the multiple sample picture book contents to obtain the large language model.

[0066] Optionally, before using the image generation model, the image generation module may also be used to:

[0067] Obtaining multiple types of sample picture book images; and

[0068] Obtain sample keywords corresponding to each type of the sample picture book image;

[0069] The deep learning model is trained according to the multiple types of sample picture book images and the sample keywords corresponding to the multiple types of sample picture book images to obtain the image generation model.

[0070] The present disclosure also provides an electronic device, including:

[0071] processor;

[0072] a memory for storing instructions executable by the processor;

[0073] The processor is configured to execute the instructions to implement the steps in the above method.

[0074] The embodiment of the present disclosure further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements each step of the above method when executed by a processor.

[0075] The embodiment of the present disclosure further provides a computer program product, including a computer program / instruction, characterized in that the computer program / instruction implements each step of the above method when executed by a processor.

[0076] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0077] In some embodiments of the present disclosure, a picture book keyword description text and a picture book theme are obtained; the picture book keyword description text and the picture book theme are input into a large language model that has been trained to obtain a picture book text, and the picture book text can be quickly obtained using the large language model; keyword extraction is performed on the picture book text to obtain keywords of the picture book text; the keywords of the picture book text are input into a picture generation model that has been trained to obtain picture book illustrations, and picture book illustrations are quickly generated using the keywords of the picture book text; the picture book text and the picture book illustrations are synthesized to obtain target picture book content; compared with the method of manually making picture books, the present disclosure reduces labor costs and improves the efficiency of picture book generation.

[0078] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0080] Figure 1 A schematic diagram of a process of generating picture book contents provided by an exemplary embodiment of the present disclosure;

[0081] Figure 2 An application scenario diagram of a picture book content generation method provided by an exemplary embodiment of the present disclosure;

[0082] Figure 3 A flowchart of another picture book content generation method provided by an exemplary embodiment of the present disclosure;

[0083] Figure 4 A schematic structural diagram of a picture book content generation device provided by an exemplary embodiment of the present disclosure;

[0084] Figure 5 A schematic structural diagram of an electronic device provided by an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0085] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0086] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure.

[0087] It should be noted that the user information involved in this disclosure includes but is not limited to: user device information and user personal information; the collection, storage, use, processing, transmission, provision and disclosure of user information in this disclosure comply with the relevant laws and regulations and do not violate public order and good morals.

[0088] Picture books are picture books suitable for children to read. They usually use rich illustrations and easy-to-understand text to tell stories, convey knowledge or inspire thinking. Picture books are designed to provide children with an opportunity to get close to words and images, and cultivate their interest in reading and comprehension. Picture books can be different types such as story books, cognitive enlightenment books, and emotion books, suitable for children of all ages to read.

[0089] At present, picture book content is mainly edited manually, which results in high labor costs and low efficiency in picture book generation.

[0090] In response to the above technical problems, in some embodiments of the present disclosure, a method for generating a picture book may be as follows: obtain picture book keyword description text and picture book theme; input the picture book keyword description text and picture book theme into a large language model that has been trained to obtain picture book text, and the picture book text can be quickly obtained using the large language model; extract keywords from the picture book text to obtain keywords of the picture book text; input the keywords of the picture book text into a trained image generation model to obtain picture book illustrations, and quickly generate picture book illustrations using the keywords of the picture book text; synthesize the picture book text and the picture book illustrations to obtain the target picture book content; compared with the method of manually making picture books, the present disclosure reduces labor costs and improves the efficiency of picture book generation.

[0091] The technical solutions provided by various embodiments of the present disclosure are described in detail below in conjunction with the accompanying drawings.

[0092] Figure 1 The following is a flow chart of a picture book content generator provided by an exemplary embodiment of the present disclosure. Figure 1 As shown, the method includes:

[0093] S101: Obtain picture book keyword description text and picture book theme;

[0094] S102: Input the picture book keyword description text and the picture book theme into the trained large language model to obtain the picture book text;

[0095] S103: extracting keywords from the picture book text to obtain keywords of the picture book text;

[0096] S104: input keywords of the picture book text into the trained image generation model to obtain picture book illustrations;

[0097] S105: Synthesize the picture book text and the picture book illustration to obtain target picture book content.

[0098] In this embodiment, the execution subject of the above method may be a smart terminal or a server.

[0099] In this embodiment, there is no limitation on the type of smart terminals, including but not limited to the following: personal computers, laptop computers, mobile phones, smart bracelets, smart watches and smart speakers.

[0100] In this embodiment, the implementation form of the server is not limited, and the server can be a conventional server, a cloud server, a cloud host, a virtual center, etc. Among them, the composition of the server mainly includes a processor, a hard disk, a memory, a system bus, etc., and a general computer architecture type.

[0101] In order to make a more detailed and clear definition of the technical solution of the present disclosure, the following definitions are given:

[0102] Convolutional Neural Networks (CNN), CNN is a deep learning algorithm commonly used in image recognition and processing. In the field of AI painting, CNN can be used for tasks such as image feature extraction, image classification, and image generation.

[0103] Generative Adversarial Networks (GAN), GAN is a deep learning algorithm based on the principle of competition, which can be used for tasks such as image generation, image restoration, and image style transfer. GAN consists of two neural networks: the generator and the discriminator. The task of the generator is to generate images that are as realistic as possible, while the task of the discriminator is to distinguish between real images and generated images.

[0104] Variational Autoencoder (VAE), VAE is a deep generative model that can be used for tasks such as image generation, image compression, and image restoration. VAE consists of two neural networks: encoder and decoder. The encoder's task is to compress the image into a latent vector, while the decoder's task is to decode the latent vector into an image.

[0105] Recurrent Neural Networks (RNN), RNN is a deep learning algorithm that can be used for tasks such as natural language processing, speech recognition, and image description. In the field of AI painting, RNN can be used to generate text-to-image descriptions or to generate continuous animations.

[0106] Figure 2 This is an application scenario diagram of a picture book content generation method provided by an exemplary embodiment of the present disclosure. Figure 2 As shown, the present disclosure mainly has the following application scenarios:

[0107] Scenario 1: Family education. Parents can use our platform to customize picture books and create stories based on their children’s interests and needs, promoting family education and strengthening parent-child relationships.

[0108] Scenario 2: School teaching. Educational institutions and schools can use customized picture books to assist teaching and provide personalized teaching content according to students of different ages and learning abilities.

[0109] Scenario 3: Kindergarten education. Kindergarten teachers can create picture book stories for use in kindergarten teaching activities to stimulate children's imagination and interest in reading.

[0110] Scene 4: Online Picture Book Plaza: The platform provides an online picture book plaza for users to share their own picture books and enable communication and interaction between users.

[0111] Before using the large language model and the image generation model, the present disclosure needs to pre-train the large language model and the image generation model.

[0112] In some embodiments of the present disclosure, a plurality of original picture book contents are collected from the Internet; the plurality of original picture book contents are preprocessed to obtain a plurality of sample picture book contents; feature extraction is performed on each sample picture book content to obtain key features corresponding to each sample picture book content; initial model training is performed based on the key features corresponding to the plurality of sample picture book contents and the plurality of sample picture book contents to obtain a large language model.

[0113] In the above embodiment, the original picture book content may be a children's story data set, such as a children's story book, a picture book, a children's literature work, etc.

[0114] In the above embodiment, a plurality of original picture book contents are preprocessed to obtain a plurality of sample picture book contents. For example, operations such as text cleaning, word segmentation, deduplication, removal of special characters and punctuation marks, etc. are performed on the plurality of original picture book contents to provide clean and neat input for the model.

[0115] In the above embodiment, feature extraction is performed on the content of each sample picture book to obtain key features corresponding to the content of each sample picture book, such as keywords of the storyline, character relationships, emotional tendencies, etc.

[0116] In the above embodiment, initial model training is performed based on the key features corresponding to the contents of multiple sample picture books and the contents of multiple sample picture books to obtain a large language model. A natural language processing model that performs well in processing sequence data and generating text is selected, such as a recurrent neural network, a long short-term memory network, and a transformer. The selected model is trained using the prepared data set. During the training process, the model learns the patterns and language rules of children's stories to generate new story texts.

[0117] In the above and following embodiments, after the model is trained, the model can be evaluated. Specifically, based on the various indicators of the model, an evaluation standard is formulated, for example, the evaluation standard is formulated according to the readability of the text, the coherence of the storyline, and the applicability to the target age group. Collect a picture book data set for training the model, including picture book text and picture book illustrations; generate picture book samples; evaluate the various indicators of the picture book samples according to the evaluation criteria to obtain the model evaluation results. The model evaluation results can be combined with manual data analysis and collation to further optimize the model. In addition, the model can also be tested for stability and continuously monitored and improved.

[0118] In the above and following embodiments, the scope of the model's answer can be controlled. Including but not limited to the following methods: trimming the text to limit the length of the output text; using prefixes and starting words; specifying keywords and themes; using temperature parameters to control the diversity of the picture book text generated by the model and the scope of text generation; filtering and screening answers, using keyword matching and grammatical analysis, and applying natural language processing technology to ensure that the model's answers are within a specific range; manual review and monitoring, using a third-party text content exposure and detection service, and performing secondary monitoring and manual review to help capture answers that may be generated by the model that do not meet the requirements and handle them appropriately.

[0119] In some embodiments of the present disclosure, after training the model, the obtained large language model is evaluated, and the performance of the model is evaluated using a test data set cross-validation method. During the training process, the performance and loss function of the model are monitored, and the hyperparameters and architecture of the model are adjusted as needed to improve the quality and diversity of the generated stories. In addition, a variety of different training strategies can be tried, such as using a larger data set, adjusting the learning rate, etc., to improve the accuracy of the model.

[0120] In some embodiments of the present disclosure, a trained large language model is used to generate picture book stories. An opening or certain keywords are input, and the model continues to create the story to evaluate the generated story to ensure that it is grammatically correct, fluent, and suitable for children. By integrating the large language model, the picture book story content is generated, making the storyline and expression more vivid, interesting, rich and diverse, thereby providing a personalized and innovative picture book experience. In the process of generating picture book story content, a large amount of natural language processing and creative generation are involved, and problems such as semantic understanding, context grasp, and story coherence need to be solved to ensure that the generated story logic is reasonable and matches the keywords and descriptions entered by the user.

[0121] In some embodiments of the present disclosure, the model is continuously iterated and improved based on the quality of the generated stories and user feedback to obtain better results. The performance of the model is further improved by increasing the data set and adjusting the model architecture.

[0122] In some embodiments of the present disclosure, multiple types of sample picture book images are obtained; sample keywords corresponding to each type of sample picture book image are obtained; and a deep learning model is trained based on the multiple types of sample picture book images and the sample keywords corresponding to the multiple types of sample picture book images to obtain a picture generation model. The types of picture books include but are not limited to science fiction and the second dimension.

[0123] In the above embodiments, in order to achieve the generation of images with a cartoon style, the present disclosure uses a deep learning model to learn the characteristics of the cartoon style by training a large number of cartoon image data sets, and applies these characteristics to the generation of new images. The picture generation model performs feature analysis based on the user's keywords and is pre-divided into a variety of different cartoon image categories, such as science fiction, two-dimensional, etc. In addition, image processing techniques are also used, such as image segmentation, scaling, color adjustment, etc., to further optimize the quality of the generated image. The present disclosure can generate images with a cartoon style through picture generation models and natural language processing technology, and can provide more creative and diverse image generation services. Based on user input, the present disclosure uses natural language processing technology to extract keywords and phrases to generate cartoon images that meet user needs, thereby meeting the personalized needs of each user.

[0124] In some embodiments of the present disclosure, a picture book keyword description text and a picture book theme are obtained. One achievable method is to obtain the picture book keyword description text in response to a user's keyword text input operation; and to obtain the picture book theme in response to a user's picture book theme input operation. The user's input operation may be to enter text in an input item, or to enter voice content in voice mode, and then convert the voice into text content. The picture book keyword description text is a keyword that can reflect the object described in the picture book, for example, "a giraffe", and the picture book theme is a text that can reflect the event associated with the object described in the picture book, for example, "this giraffe is looking for new friends in the forest". The embodiments of the present disclosure customize the picture book content according to the keyword description text and picture book theme input by the user to meet the personalized needs of each user.

[0125] In an optional embodiment, after obtaining the picture book keyword description text, the picture book keyword description text is input into the large language model to obtain the initial picture book text; the picture book keyword description text is input into the image generation model to obtain the initial picture book illustration. For example, the keyword description text is "a giraffe", and the picture book keyword description text is input into the image generation model to generate a cartoon image of a giraffe.

[0126] In some embodiments of the present disclosure, the picture book keyword description text and the picture book theme are input into the trained large language model to obtain the picture book text. The present disclosure uses the large language model for natural language generation, constructs the storyline according to the picture book keyword description text and the picture book theme input by the user, and ensures that the story logic is coherent and reasonable.

[0127] In an optional embodiment, the picture book keyword description text and the picture book theme are input into the trained large language model to obtain the original picture book text; in response to the modification operation on the original picture book text, the picture book text is obtained. It should be noted that the original picture book text output by the large language model is further modified and optimized to improve the text quality and the interest of the picture book text.

[0128] In some embodiments of the present disclosure, keyword extraction is performed on the picture book text to obtain keywords of the picture book text. One achievable method is to perform data cleaning on the picture book text to obtain a cleaned picture book text; perform word segmentation processing on the cleaned picture book text to obtain original word segmentations; filter out candidate word segmentations from the original word segmentations according to preset filtering rules; and select keywords of the picture book text from the candidate word segmentations according to the importance of the candidate word segmentations. . The present disclosure performs keyword extraction on the picture book text to ensure that the important elements and sections therein are accurately expressed. The keywords of the picture book text can reflect the important elements of the picture book and the keywords describing the plot of the object event, for example, "giraffe", "riverside", "drinking water", etc.

[0129] For example, obtain picture book text; perform data cleaning on the picture book text, including removing punctuation marks, stop words, and special characters, where stop words are, for example, "de", "shi", etc.; use natural language processing technology to segment the picture book text to obtain original segments; filter the original segments using a stop word list to remove common words with low information value to reduce noise and obtain candidate segments. Among them, the importance of the candidate segments is characterized by the TF-IDF value of each candidate segment to measure its importance and uniqueness in the text. According to the TF-IDF value of each candidate segment, select the words with larger TF-IDF values as the keywords of the picture book text.

[0130] Among them, TF-IDF (Term Frequency-Inverse Document Frequency) is a method for calculating the importance of words in text. It takes into account the frequency of a word in a document and its rarity in the entire text collection. The higher the TF-IDF value, the more important the word is in the text.

[0131] TF represents the frequency of a word in a document, that is, the number of times the word appears in the document divided by the length of the document. IDF represents the rarity of a word in the text collection, that is, the number of documents in the text collection that contain the word divided by the total number of documents in the text collection.

[0132] The TF-IDF value can be calculated by the following formula:

[0133] TF-IDF = TF × IDF

[0134] Among them, TF is the frequency of a word in a document, and IDF is the rarity of a word in the text collection.

[0135] In some embodiments of the present disclosure, each picture book text segment and the corresponding picture book illustration of each picture book text segment are respectively synthesized to obtain the target picture book content. It should be noted that the picture book text can be split into multiple picture book text segments, and picture book illustrations can be generated for each picture book text segment respectively. For example, the picture book text can be split into multiple picture book text segments according to the number of pages, and a picture book illustration is generated for each picture book text segment of each page.

[0136] In an alternative embodiment of the present disclosure, the keywords corresponding to each picture book text segment are input into a pre-trained image generation model to obtain the picture book illustration corresponding to each picture book text segment, and each picture book text segment and the corresponding picture book illustration of each picture book text segment are respectively synthesized to obtain the target picture book content. Among them, each picture book text segment and its corresponding picture book illustration are respectively synthesized, and finally synthesized into a complete target picture book content.

[0137] In some embodiments of the present disclosure, after the target picture book content is generated, the target picture book content can also be displayed to the user for viewing. One achievable method is to send the target picture book content to the user terminal and display the target picture book content on the display screen of the user terminal for the user to view.

[0138] Combined with the description of the above embodiments, Figure 3 The following is a flow chart of another method for generating picture book contents provided by an exemplary embodiment of the present disclosure. Figure 3 As shown, the method includes:

[0139] S301: Obtaining picture book keyword description text and picture book theme;

[0140] S302: Input the picture book keyword description text and the picture book theme into the trained large language model to obtain the picture book text;

[0141] S303: extract keywords from the picture book text to obtain keywords of the picture book text;

[0142] S304: inputting keywords corresponding to each picture book text segment into the trained image generation model to obtain picture book illustrations corresponding to each picture book text segment, wherein the picture book text includes multiple picture book text segments;

[0143] S305: Each picture book text segment and the picture book illustration corresponding to each picture book text segment are synthesized respectively to obtain the target picture book content.

[0144] In this embodiment, the implementation method of each step of the above method can refer to the description of the aforementioned embodiments, which will not be repeated here. At the same time, this embodiment can also achieve the technical effects of the corresponding parts of the aforementioned embodiments.

[0145] In the above method embodiment of the present disclosure, a picture book keyword description text and a picture book theme are obtained; the picture book keyword description text and the picture book theme are input into a large language model that has been trained to obtain a picture book text, and the picture book text can be quickly obtained using the large language model; keyword extraction is performed on the picture book text to obtain keywords of the picture book text; the keywords of the picture book text are input into a picture generation model that has been trained to obtain picture book illustrations, and picture book illustrations are quickly generated using the keywords of the picture book text; the picture book text and the picture book illustrations are synthesized to obtain the target picture book content; compared with the method of manually making picture books, the present disclosure reduces labor costs and improves the efficiency of picture book generation.

[0146] Figure 4 FIG. 4 is a schematic diagram of a structure of a picture book content generating device 40 provided by an exemplary embodiment of the present disclosure. Figure 4As shown, the picture book content generating device 40 includes: an acquisition module 41, a text generating module 42, an extraction module 43, a picture generating module 44 and a synthesis module 45.

[0147] The acquisition module 41 is used to acquire the picture book keyword description text and the picture book theme;

[0148] A text generation module 42 is used to input the picture book keyword description text and the picture book theme into the trained large language model to obtain the picture book text;

[0149] An extraction module 43 is used to extract keywords from the picture book text to obtain keywords of the picture book text;

[0150] The picture generation module 44 is used to input the keywords of the picture book text into the trained picture generation model to obtain the picture book illustrations;

[0151] The synthesis module 45 is used to synthesize the picture book text and the picture book illustration to obtain the target picture book content.

[0152] Optionally, after obtaining the picture book keyword description text, the obtaining module 41 may also be used to:

[0153] Input the picture book keyword description text into the large language model to obtain the initial picture book text;

[0154] Input the picture book keyword description text into the image generation model to obtain the initial picture book illustration.

[0155] Optionally, when the text generation module 42 inputs the picture book keyword description text and the picture book theme into the trained large language model to obtain the picture book text, it is used to:

[0156] Input the picture book keyword description text and the picture book theme into the trained large language model to obtain the original picture book text;

[0157] In response to the modification operation on the original picture book text, the picture book text is obtained.

[0158] Optionally, when the extraction module 43 extracts keywords from the picture book text to obtain keywords of the picture book text, it is used to:

[0159] Perform data cleaning on the picture book text to obtain the cleaned picture book text;

[0160] Perform word segmentation on the cleaned picture book text to obtain the original word segmentation;

[0161] According to the preset filtering rules, filter out candidate segmentation words from the original segmentation words;

[0162] According to the importance of the candidate participles, the keywords of the picture book text are selected from the candidate participles.

[0163] Optionally, when the picture generation module 44 inputs the keywords of the picture book text into the trained picture generation model to obtain the picture book illustration, it is used to:

[0164] Input the keywords corresponding to each picture book text segment into the trained image generation model to obtain the picture book illustration corresponding to each picture book text segment, wherein the picture book text includes multiple picture book text segments;

[0165] The picture book text and picture book illustrations are synthesized to obtain the target picture book content, including:

[0166] Each picture book text segment and the picture book illustration corresponding to each picture book text segment are synthesized separately to obtain the target picture book content.

[0167] Optionally, after synthesizing the picture book text and the picture book illustration to obtain the target picture book content, the synthesis module 45 can also be used to:

[0168] The target picture book content is sent to the user terminal so that the user terminal can display the target picture book content.

[0169] Optionally, before using the large language model, the text generation module 42 may also be used to:

[0170] Collect multiple original picture book contents from the Internet;

[0171] Preprocessing a plurality of original picture book contents to obtain a plurality of sample picture book contents;

[0172] Extract features from the content of each sample picture book to obtain key features corresponding to the content of each sample picture book;

[0173] Initial model training is performed based on key features corresponding to multiple sample picture book contents and multiple sample picture book contents to obtain a large language model.

[0174] Optionally, before using the image generation model, the image generation module 44 may also be used to:

[0175] Obtaining multiple types of sample picture book images; and

[0176] Get sample keywords corresponding to each type of sample picture book image;

[0177] According to various types of sample picture book images and sample keywords corresponding to the various types of sample picture book images, a deep learning model is trained to obtain a picture generation model.

[0178] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0179] Figure 5 FIG. 1 is a schematic diagram of a structure of an electronic device provided by an exemplary embodiment of the present disclosure. Figure 5 As shown, the electronic device includes: a memory 51 and a processor 52. In addition, the electronic device also includes a power supply component 53 and a communication component 54.

[0180] The memory 51 is used to store computer programs and can be configured to store various other data to support operations on the electronic device. Examples of such data include instructions for any application program or method operating on the electronic device.

[0181] The memory 51 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0182] The communication component 54 is used for data transmission with other devices.

[0183] The processor 52 can execute computer instructions stored in the memory 51 to: obtain picture book keyword description text and picture book theme; input the picture book keyword description text and picture book theme into the trained large language model to obtain the picture book text; extract keywords from the picture book text to obtain keywords of the picture book text; input the keywords of the picture book text into the trained image generation model to obtain picture book illustrations; synthesize the picture book text and the picture book illustrations to obtain the target picture book content.

[0184] Optionally, after the processor 52 obtains the picture book keyword description text, the method further includes:

[0185] Input the picture book keyword description text into the large language model to obtain the initial picture book text;

[0186] Input the picture book keyword description text into the image generation model to obtain the initial picture book illustration.

[0187] Optionally, when the processor 52 inputs the picture book keyword description text and the picture book theme into the trained large language model to obtain the picture book text, it is used to:

[0188] Input the picture book keyword description text and the picture book theme into the trained large language model to obtain the original picture book text;

[0189] In response to the modification operation on the original picture book text, the picture book text is obtained.

[0190] Optionally, when the processor 52 extracts keywords from the picture book text to obtain keywords of the picture book text, it is used to:

[0191] Perform data cleaning on the picture book text to obtain the cleaned picture book text;

[0192] Perform word segmentation on the cleaned picture book text to obtain the original word segmentation;

[0193] According to the preset filtering rules, filter out candidate segmentation words from the original segmentation words;

[0194] According to the importance of the candidate participles, the keywords of the picture book text are selected from the candidate participles.

[0195] Optionally, when the processor 52 inputs the keywords of the picture book text into the trained picture generation model to obtain the picture book illustration, it is used to:

[0196] Input the keywords corresponding to each picture book text segment into the trained image generation model to obtain the picture book illustration corresponding to each picture book text segment, wherein the picture book text includes multiple picture book text segments;

[0197] The picture book text and picture book illustrations are synthesized to obtain the target picture book content, including:

[0198] Each picture book text segment and the picture book illustration corresponding to each picture book text segment are synthesized separately to obtain the target picture book content.

[0199] Optionally, after synthesizing the picture book text and the picture book illustration to obtain the target picture book content, the processor 52 may also be used to:

[0200] The target picture book content is sent to the user terminal so that the user terminal can display the target picture book content.

[0201] Optionally, before using the large language model, the processor 52 may also be configured to:

[0202] Collect multiple original picture book contents from the Internet;

[0203] Preprocessing a plurality of original picture book contents to obtain a plurality of sample picture book contents;

[0204] Extract features from the content of each sample picture book to obtain key features corresponding to the content of each sample picture book;

[0205] Initial model training is performed based on key features corresponding to multiple sample picture book contents and multiple sample picture book contents to obtain a large language model.

[0206] Optionally, before using the image to generate the model, the processor 52 may also be used to:

[0207] Obtaining multiple types of sample picture book images; and

[0208] Get sample keywords corresponding to each type of sample picture book image;

[0209] According to various types of sample picture book images and sample keywords corresponding to the various types of sample picture book images, a deep learning model is trained to obtain a picture generation model.

[0210] Accordingly, the present disclosure also provides a computer-readable storage medium storing a computer program. When the computer-readable storage medium stores the computer program and the computer program is executed by one or more processors, the one or more processors execute Figure 1 Each step in the method embodiment.

[0211] Accordingly, the present disclosure also provides a computer program product, the computer program product includes a computer program / instruction, the computer program / instruction is executed by a processor Figure 1 Each step in the method embodiment.

[0212] Above Figure 5 The communication component in is configured to facilitate wired or wireless communication between the device where the communication component is located and other devices. The device where the communication component is located can access a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G / LTE, 5G and other mobile communication networks, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0213] Above Figure 5 The power supply component in the device provides power to various components of the device where the power supply component is located. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device where the power supply component is located.

[0214] The electronic device also includes a display screen and an audio component.

[0215] The display screen includes a screen, and the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

[0216] The above audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC), and when the device where the audio component is located is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in a memory or sent via a communication component. In some embodiments, the audio component also includes a speaker for outputting an audio signal.

[0217] In the above-mentioned device, apparatus and computer program product embodiments of the present disclosure, a picture book keyword description text and a picture book theme are obtained; the picture book keyword description text and the picture book theme are input into a large language model that has been trained to obtain a picture book text, and the picture book text can be quickly obtained using the large language model; keywords are extracted from the picture book text to obtain keywords of the picture book text; the keywords of the picture book text are input into a picture generation model that has been trained to obtain picture book illustrations, and picture book illustrations are quickly generated using the keywords of the picture book text; the picture book text and the picture book illustrations are synthesized to obtain the target picture book content; compared with the method of manually making picture books, the present disclosure reduces labor costs and improves the efficiency of picture book generation.

[0218] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0219] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0220] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0221] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0222] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0223] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0224] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0225] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0226] The above are only specific embodiments of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to these embodiments herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for generating picture book content, characterized in that: include: Get the picture book keyword description text and picture book theme; Inputting the picture book keyword description text and the picture book theme into the trained large language model to obtain the picture book text; Extracting keywords from the picture book text to obtain keywords of the picture book text; Input the keywords of the picture book text into the trained image generation model to obtain picture book illustrations; The picture book text and the picture book illustration are synthesized to obtain target picture book content.

2. The method according to claim 1, characterized in that After obtaining the picture book keyword description text, the method further includes: Inputting the picture book keyword description text into the large language model to obtain an initial picture book text; The picture book keyword description text is input into the image generation model to obtain an initial picture book illustration.

3. The method according to claim 1, characterized in that: The step of inputting the picture book keyword description text and the picture book theme into the trained large language model to obtain the picture book text includes: Inputting the picture book keyword description text and the picture book theme into the trained large language model to obtain the original picture book text; In response to the modification operation on the original picture book text, the picture book text is obtained.

4. The method according to claim 1, characterized in that: The step of extracting keywords from the picture book text to obtain keywords of the picture book text includes: Performing data cleaning on the picture book text to obtain a cleaned picture book text; Performing word segmentation processing on the cleaned picture book text to obtain original word segmentations; According to the preset filtering rules, the candidate participles are filtered out from the original participles; According to the importance of the candidate participles, keywords of the picture book text are selected from the candidate participles.

5. The method according to claim 1, characterized in that Input the keywords of the picture book text into the trained image generation model to obtain picture book illustrations, including: Input the keywords corresponding to each picture book text segment into the trained image generation model to obtain the picture book illustration corresponding to each picture book text segment, wherein the picture book text includes a plurality of the picture book text segments; The picture book text and the picture book illustration are synthesized to obtain target picture book content, including: Each picture book text segment and the picture book illustration corresponding to each picture book text segment are synthesized separately to obtain the target picture book content.

6. The method according to claim 1, characterized in that After synthesizing the picture book text and the picture book illustration to obtain the target picture book content, the method further includes: The target picture book content is sent to a user terminal so that the user terminal displays the target picture book content.

7. The method according to claim 1, characterized in that Before using the large language model, the method further includes: Collect multiple original picture book contents from the Internet; Preprocessing the multiple original picture book contents to obtain multiple sample picture book contents; Extracting features of each sample picture book content to obtain key features corresponding to each sample picture book content; Initial model training is performed based on key features corresponding to the multiple sample picture book contents and the multiple sample picture book contents to obtain the large language model.

8. The method according to claim 1, characterized in that Before using the picture to generate the model, the method further includes: Obtaining multiple types of sample picture book images; and Obtain sample keywords corresponding to each type of the sample picture book image; The deep learning model is trained according to the multiple types of sample picture book images and the sample keywords corresponding to the multiple types of sample picture book images to obtain the image generation model.

9. A picture book content generating device, characterized in that: include: The acquisition module is used to obtain the picture book keyword description text and picture book theme; A text generation module, used for inputting the picture book keyword description text and the picture book theme into the trained large language model to obtain the picture book text; An extraction module, used to extract keywords from the picture book text to obtain keywords of the picture book text; An image generation module, used for inputting keywords of the picture book text into a trained image generation model to obtain picture book illustrations; A synthesis module is used to synthesize the picture book text and the picture book illustration to obtain the target picture book content.

10. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement each step in the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, each step of the method according to any one of claims 1 to 8 is implemented.

12. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images