Drawing book generation method and device, equipment and storage medium

Abstract

The present disclosure provides a sketchbook generation method and device, equipment and storage medium, relates to artificial intelligence, big data, image generation and other technical fields, and particularly relates to the fields of intelligent document editing, intelligent assistants, virtual assistants and the like. The specific implementation scheme is as follows: based on determined theme content, a story outline around the theme content is generated; wherein the story outline contains N paragraphs around the theme content; N is an integer greater than or equal to 2; each paragraph in the N paragraphs contained in the story outline is subjected to a shot understanding, so as to obtain a plurality of prompt information for each paragraph; wherein the plurality of prompt information of a paragraph can describe the scene picture to which the paragraph is directed; based on the plurality of prompt information of each paragraph in the N paragraphs, an image for each paragraph is obtained, so as to obtain a target sketchbook for the theme content.

Description

Technical Field

[0001] This disclosure relates to the fields of data processing and image processing technology, and in particular to the fields of artificial intelligence, big data, and image generation, and can be applied to application areas such as intelligent document editing, intelligent assistants, and virtual assistants. Background Technology

[0002] Currently, picture book production relies heavily on manual labor and involves certain technical requirements, such as mastering specific technical knowledge and skills. This makes it difficult for ordinary users from unrelated industries (such as art majors) to independently complete the production of picture books. Even with customized picture book needs, ordinary users cannot achieve this on their own. In addition, the traditional handmade production process, including drawing illustrations, writing text, layout, and audio-visual production, all require a significant amount of time and effort. Summary of the Invention

[0003] This disclosure provides a method, apparatus, device, and storage medium for generating picture books.

[0004] According to one aspect of this disclosure, a method for generating a picture book is provided, comprising:

[0005] Based on the defined theme, a story outline is generated around the theme; wherein the story outline contains N paragraphs around the theme; and N is an integer greater than or equal to 2.

[0006] The story outline contains N paragraphs, each of which is analyzed in a scene-by-scene manner to obtain multiple prompts for each paragraph; wherein, the multiple prompts for each paragraph can describe the scene to which the paragraph is located.

[0007] Based on multiple prompts for each of the N paragraphs, images for each paragraph are obtained to create a target picture book for the theme content.

[0008] According to another aspect of this disclosure, a picture book generating apparatus is provided, comprising:

[0009] A text processing unit is configured to generate a story outline around a defined topic based on that topic; wherein the story outline contains N paragraphs around the topic, and N is an integer greater than or equal to 2.

[0010] The multimedia generation unit is used to perform scene analysis on each of the N paragraphs contained in the story outline to obtain multiple prompts for each paragraph; wherein, the multiple prompts for each paragraph can describe the scene to which the paragraph is located; based on the multiple prompts for each of the N paragraphs, images for each paragraph are obtained to obtain a target picture book for the theme content.

[0011] According to another aspect of this disclosure, an electronic device is provided, comprising:

[0012] At least one processor; and

[0013] The memory is communicatively connected to the at least one processor; wherein,

[0014] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the methods of any embodiment of the present disclosure.

[0015] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform a method according to any embodiment of this disclosure.

[0016] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements a method according to any embodiment of this disclosure.

[0017] In this way, the disclosed solution can perform storyboarding and understanding on the generated story outline, and obtain multiple prompts to describe the scene to which each paragraph is directed. Then, based on the multiple prompts for each paragraph, a picture book (such as a target picture book) matching the theme content is generated, which effectively improves the production efficiency and quality of the picture book. Compared with the existing manual production methods, the disclosed solution effectively avoids the cumbersome process of traditional picture book production, lowers the threshold for picture book production, realizes the automated process of picture book generation, and can generate picture book content that meets user needs, satisfying users' personalized needs, thereby improving the user experience.

[0018] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0019] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0020] Figure 1 This is an illustrative flow diagram of a picture book generation method according to an embodiment of this application. Figure 1 ;

[0021] Figure 2 This is an illustrative flow diagram of a picture book generation method according to an embodiment of this application. Figure 2 ;

[0022] Figure 3This is a schematic diagram illustrating the effect of a target drawing according to an embodiment of this application;

[0023] Figures 4(a) to 4(d) This is a schematic diagram of the process for achieving a page-turning effect in a target picture book according to an embodiment of this application;

[0024] Figure 5 This is a schematic diagram illustrating the page-turning effect of a target picture book according to an embodiment of this application;

[0025] Figure 6 This is an illustrative flow diagram of a picture book generation method according to an embodiment of this application. Figure 3 ;

[0026] Figures 7(a) and 7(b) are schematic diagrams illustrating the application process of a drawing generation method according to an embodiment of this application in a specific scenario;

[0027] Figure 8 This is a schematic diagram of a drawing generation system according to an embodiment of this application;

[0028] Figure 9 This is a schematic diagram of the structure of a book generation device according to an embodiment of this application;

[0029] Figure 10 This is a block diagram of an electronic device used to implement the picture book generation method of the present disclosure embodiments. Detailed Implementation

[0030] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0031] In this document, the term "and / or" merely describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. The term "at least one" in this document indicates any combination of at least two of a plurality of elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C. The terms "first" and "second" in this document refer to and distinguish between multiple similar technical terms, not to restrict the order or to limit there to only two. For example, "first feature" and "second feature" refer to two categories / two features; the first feature can be one or more, and the second feature can also be one or more.

[0032] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can still be practiced even without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0033] The following describes the related technologies of the embodiments of this disclosure. The following related technologies are optional solutions and can be combined with the technical solutions of the embodiments of this disclosure in any way, and they all fall within the protection scope of the embodiments of this disclosure.

[0034] Currently, picture book production relies heavily on manual labor and involves certain technical requirements, such as mastering specific technical knowledge and skills. This makes it difficult for ordinary users from unrelated industries (such as art majors) to independently complete the production of picture books. Even with customized picture book needs, ordinary users cannot achieve this on their own. In addition, the traditional handmade production process, including drawing illustrations, writing text, layout, and audio-visual production, all require a significant amount of time and effort.

[0035] Based on this, this disclosure proposes a picture book generation method based on a large model to automatically and efficiently generate target picture books.

[0036] Specifically, Figure 1 This is an illustrative flow diagram of a picture book generation method according to an embodiment of this application. Figure 1 This method can be optionally applied to electronic devices, such as personal computers, servers, server clusters, and other electronic devices.

[0037] Furthermore, the method includes at least a portion of the following: For example... Figure 1 As shown, it includes:

[0038] Step S101: Based on the determined theme content, generate a story outline around the theme content.

[0039] Here, the story outline can be understood as a summary of the story, such as, in one example, including the main plot, characters, conflicts, and ending.

[0040] Furthermore, to facilitate the automatic generation of picture books, the story outline generated around the theme can contain N paragraphs related to the theme, where N is an integer greater than or equal to 2. For example, each paragraph contains a main plot point, thus facilitating the subsequent generation of picture books on a paragraph-by-paragraph basis.

[0041] For example, in one example, the subject content can be specifically the text content entered by the target object, such as one or more keywords; this disclosed solution does not impose specific restrictions on the output subject content.

[0042] Step S102: Perform scene analysis on each of the N paragraphs contained in the story outline to obtain multiple hints for each paragraph.

[0043] Here, the multiple prompts from the segment obtained after understanding the storyboard describe the scene the segment targets. This lays the foundation for accurate image generation later. At the same time, it effectively avoids generating images from the same angle, significantly improving the visualization effect of the generated target sketchbook.

[0044] It should be noted that "storyboard understanding" can also be called "shot understanding". Its purpose is to ensure that the generated images (also known as storyboard images) accurately match the scene of the paragraph, thereby ensuring consistency between the text and images, improving the visual effect, and thus enhancing the user experience.

[0045] Step S103: Based on multiple prompts for each of the N paragraphs, obtain images for each paragraph to obtain a target picture book for the theme content.

[0046] In one example, the images in each paragraph can also include captions (e.g., the text content of the paragraph), thus creating a picture book that combines images and text, effectively improving the readability of the picture book.

[0047] In this way, the disclosed solution can perform storyboarding and understanding on the generated story outline, and obtain multiple prompts to describe the scene to which each paragraph is directed. Then, based on the multiple prompts for each paragraph, a picture book (such as a target picture book) matching the theme content is generated, which effectively improves the production efficiency and quality of the picture book. Compared with the existing manual production methods, the disclosed solution effectively avoids the cumbersome process of traditional picture book production, lowers the threshold for picture book production, realizes the automated process of picture book generation, and can generate picture book content that meets user needs, satisfying users' personalized needs, thereby improving the user experience.

[0048] In a specific example, a story outline can be obtained in the following way: obtaining multiple theme prompts for a given theme, inputting the multiple theme prompts into the target content generation model to generate a story outline around the theme. In this way, a story outline around the theme can be quickly derived, laying the foundation for the accurate generation of picture book content in the future.

[0049] Here, the target content generation model can be obtained by training a preset content generation model based on prompts and a few learning samples using supervised fine-tuning (SFT) data pre-training technology. This provides strong support for efficiently and accurately generating story outlines around the input theme, and also provides strong support for the subsequent generation of picture book content to have better completeness and readability.

[0050] Furthermore, the content generation model described above can be specifically a large model, and even more specifically, a large language model. Alternatively, it can be any other text generation model, as long as it can implement the above method; this disclosure does not limit this approach.

[0051] It should be noted that the story outline mentioned above can be obtained by reasoning using a content generation model, or it can be obtained using other text generation methods. This disclosed solution does not impose any specific restrictions on this.

[0052] Furthermore, in a specific example, this disclosed solution also provides editing functions, such as editing functions for the theme content or editing functions for the generated images; for example, after obtaining a story outline around the theme content (e.g., after step S101 described above), the generated story outline can also be edited. For example, after displaying the generated story outline, an adjusted story outline can also be obtained. In this way, the target audience can personalize the generated content according to their own needs, thereby further improving the user experience and making the subsequently generated picture book more in line with user needs.

[0053] In a specific example of the disclosed solution, after obtaining the target picture book for the subject content (e.g., step S103 described above), the target multimedia picture book can also be obtained in the following manner, specifically including:

[0054] Step S104: Generate audio data for each paragraph based on the text content of each paragraph.

[0055] For example, in one instance, text-to-speech (TTS) technology can be used to convert the text content of each paragraph into audio data. It should be noted that this disclosure does not limit the text-to-audio synthesis technology.

[0056] Step S105: Assemble the audio data of each segment with the images in the target picture book to obtain a target multimedia picture book for the theme content.

[0057] In one example, the present disclosure also provides editing functionality for each page of the multimedia picture book, wherein the editable content of each page may specifically include at least one of the following: text, audio, image details, etc.; thus, the interactivity with the user is enhanced to further ensure that a picture book that meets the user's needs can be generated, thereby improving the user's experience.

[0058] Thus, this disclosed solution provides an instantaneous automated audio generation method that can effectively combine the audio data of each segment with the target picture book, thereby further enhancing the user's reading experience.

[0059] Figure 2 This is an illustrative flow diagram of a picture book generation method according to an embodiment of this application. Figure 2 This method can be optionally applied to electronic devices, such as personal computers, servers, and server clusters. It is understood that the above... Figure 1 The methods shown can also be applied to this example, and the related content will not be elaborated further in this example.

[0060] Furthermore, the method includes at least a portion of the following: For example... Figure 2 As shown, it includes:

[0061] Step S201: Based on the determined theme content, generate a story outline around the theme content.

[0062] Here, the story outline contains N paragraphs that revolve around the theme; where N is an integer greater than or equal to 2.

[0063] The method for obtaining the story outline can be found in the above instructions and will not be repeated here.

[0064] Step S202: Perform scene analysis on each of the N paragraphs contained in the story outline to obtain multiple prompts for each paragraph.

[0065] Here, multiple prompts within a paragraph can describe the scene or situation that the paragraph addresses.

[0066] Step S203: Based on the multiple prompts for each of the N paragraphs, obtain an image for each paragraph.

[0067] Step S204: According to the story development order of the N paragraphs contained in the story outline, the images of each paragraph are superimposed to obtain the target picture book in which the images of each paragraph are superimposed.

[0068] Here, the target picture book can dynamically display a page-turning effect around a preset axis of symmetry during playback, and each image of the target picture book can be displayed in full size on the display interface.

[0069] Here, "full size" can be understood as the maximum size that the display interface can show. In other words, each image in the target sketchbook can be displayed at the maximum size of the display interface.

[0070] For example, in one example, such as Figure 3 As shown, when the target picture book is played on the display interface, each image can be displayed in full size, thus enhancing the reading experience.

[0071] Furthermore, the target picture book can also dynamically display the page-turning effect. For example, the page-turning effect can be dynamically displayed around a preset axis of symmetry (for example, if the display interface is a rectangle, the preset axis of symmetry can be the vertical center line of the display interface of the rectangle, for example, denoted as AA'). This enhances the user's reading experience.

[0072] In this way, the disclosed solution can utilize the story development sequence of each paragraph to overlay the images, thus facilitating the playback of the target picture book with a page-turning effect, enhancing the realism, and further improving the user's reading experience.

[0073] In a specific example of the disclosed solution, the images of each paragraph can be superimposed in the following manner to obtain the target picture book in a superimposed state, thus providing support for achieving a book-turning effect and displaying each page in full size. Specifically, the above-described method of superimposing the images of each paragraph according to the story development order of the N paragraphs contained in the story outline to obtain the target picture book in a superimposed state (e.g., step S204 above) specifically includes:

[0074] Step S204-1: According to the story development order of the N paragraphs contained in the story outline, determine the i-th image and the (i+1)-th image with the order of being displayed.

[0075] Step S204-2: Following the following overlay processing steps, overlay the i-th image (which can be understood as the current page) and the (i+1)-th image (which can be understood as the next page of the current page) to obtain the target picture book in which the images of each paragraph are overlaid.

[0076] Here, in one example, the i-th image can be understood as the currently displayed page, i.e., the current page; the (i+1)-th image can be understood as the next page after the current page. In other words, after "turning the page" from the current page, the (i+1)-th page needs to be displayed in full size.

[0077] Furthermore, the above-described superposition processing method (such as step S204-2) specifically includes:

[0078] Step S204-2-1: Overlay a blank display area on top of the (i+1)th image (e.g., the next page of the current page). For example, overlay the blank display area on top of the (i+1)th image.

[0079] Here, the size of the blank display area is the same as the size of the i-th image. In other words, this blank display area can also be displayed in full size on the display interface.

[0080] Step S204-2-2: Place the sub-image located on one side of the preset axis of symmetry in the i-th image (also known as the first side of the first i-th image) on one side of the blank display area (also known as the first side of the blank display area).

[0081] In other words, after placing the blank display area on top of the (i+1)th image, in order to achieve the book-flipping effect, it is also necessary to place some images from the i-th image, such as sub-images located on one side of the preset axis of symmetry, on one side of the blank display area.

[0082] Step S204-2-3: Flip the sub-image located on one side of the preset axis of symmetry in the (i+1)th image (also known as the first side of the (i+1)th image), and then attach it to the back of the sub-image located on the other side of the preset axis of symmetry in the (i)th image (also known as the second side of the i)th image.

[0083] Step S204-2-4: Place the laminated image on the other side of the blank display area (also known as the second side of the blank display area) to obtain the overlaid image.

[0084] In other words, after placing a portion of the i-th image on one side of the blank display area, for the other side of the blank display area, it is also necessary to place the sub-image of the i-th image located on the other side of the preset axis of symmetry on the other side of the blank display area. At this point, to achieve the page-turning effect, before placing the sub-image of the i-th image located on the other side of the preset axis of symmetry on the other side of the blank display area, the back of the sub-image of the i-th image located on the other side of the preset axis of symmetry is aligned with the portion of the (i+1)-th image that needs to be displayed after the page is turned. This lays the foundation for achieving the page-turning effect.

[0085] For example, taking the book-flipping effect as flipping from the right side of the display interface to the left side, as shown in Figure 4(a), a blank display area, such as an empty Document Object Model (DOM), is overlaid on the (i+1)th image; secondly, as shown in Figure 4(b), the image located on the preset axis of symmetry BB in the i-th image is copied. ′The left sub-image (i.e., sub-image 1 of the i-th image) is copied and added to the left area of ​​the empty DOM (i.e., the preset axis of symmetry AA) using absolute positioning layout techniques. ′ The left region of the i-th image is used to obtain the DOM (which can be denoted as DOM-1) of the sub-image 1 containing the i-th image; again, as shown in Figure 4(c), the i-th image located on the preset axis of symmetry BB is copied. ′ The sub-image to the right of the i-th image (i.e., sub-image 2 of the i-th image) and the sub-image located on the preset axis of symmetry AA in the (i+1)-th image. ′ The left sub-image (i.e., sub-image 1 of the (i+1)th image), wherein sub-image 1 of the (i+1)th image is flipped, for example by calling the rotateY function to rotate sub-image 1 of the (i+1)th image around a preset axis of symmetry AA. ′ Rotate 180 degrees; using absolute positioning layout technology, attach the flipped sub-image to the back of sub-image 2 of the i-th image. At this time, the attached image is a double-sided image, with the front being sub-image 2 of the i-th image and the back being sub-image 1 of the (i+1)-th image; finally, as shown in Figure 4(d), the attached image can be placed in the right region of DOM-1 (i.e., the preset axis of symmetry AA). ′ The right-hand region of DOM-1 can be used to obtain the overlaid image. Furthermore, a transition animation technique can be applied to the overlaid image placed in the right-hand region of DOM-1. This provides strong support for achieving a flipping effect; thus, the overlay method can be used to obtain an image like... Figure 5 The image shown is a page-turning effect from the i-th image to the (i+1)-th image.

[0086] It should be noted that in this example, the rotateY function can rotate an element around a vertical axis (i.e., the y-axis, for example, by setting a symmetry axis AA). ′ The elements are rotated to achieve the book-turning effect of this disclosed solution. Furthermore, in practical applications, parameters such as `transform-style` and `perspective` can be used to enhance the book-turning effect. Here, the `transform-style` parameter is used to set whether the elements in the book are in 3D space. For example, setting the `transform-style` parameter to `preserve-3d` will place the elements in the book in 3D space. The `perspective` parameter is used to set the distance between the observer and the plane, such as the plane with z=0 (the z-axis is a coordinate axis perpendicular to the y-axis), so that the book elements with three-dimensional positional transformations produce a perspective effect. Thus, by calling the above parameters, the book-turning effect can be achieved while enhancing the 3D effect, thereby further improving the user experience.

[0087] Furthermore, it's worth noting that in practical applications, to achieve a vertical flip effect, the `rotateY` ​​function is usually set to 180 degrees. However, for certain system environments (such as iOS), the `rotateY` ​​function should be set to -180 degrees. Additionally, while using the `rotateY` ​​function for vertical flipping, the `translateX` function can be set to -0.5px to avoid gaps that might appear when displaying the sketchbook. Moreover, to prevent elements on the back of the sketchbook from being obscured, the `rotateY` ​​function can be used in conjunction with the `backfaceVisibility` parameter; for example, setting the `rotateY` ​​function to -180 degrees and setting the `backfaceVisibility` parameter to `hidden`.

[0088] It should be noted that the above is only an illustrative example. In actual applications, additional processing can be applied based on the specific needs of the page-turning effect. This disclosed solution does not impose any restrictions on this.

[0089] In this way, the disclosed solution uses an overlay processing method to overlay the obtained images to achieve a specific book-turning effect. This improves the quality of picture book production, provides users with a personalized picture book reading experience, and enhances the interaction between users and the picture book through interactive functions, thereby increasing users' reading interest and participation, and ultimately improving the user experience.

[0090] Furthermore, since this disclosed solution achieves the book-turning effect by passing in a complete DOM element, and each page is a full-size DOM element that fills the entire screen, compared to the method of passing in only half a page of DOM, this disclosed solution can ensure that the text, buttons, and other elements on each page of the picture book are well adapted to the book-turning animation. Moreover, the above process can achieve the book-turning effect without relying on jQuery plugins. Therefore, this disclosed solution is more versatile.

[0091] Figure 6 This is an illustrative flow diagram of a picture book generation method according to an embodiment of this application. Figure 3 This method can be optionally applied to electronic devices, such as personal computers, servers, and server clusters. It is understood that the above... Figures 1 to 5 The methods shown can also be applied to this example, and the related content will not be elaborated further in this example.

[0092] Furthermore, the method includes at least a portion of the following: For example... Figure 6 As shown, it includes:

[0093] Step S601: Based on the determined theme content, generate a story outline around the theme content.

[0094] Here, the story outline contains N paragraphs that revolve around the theme; where N is an integer greater than or equal to 2.

[0095] Step S602: Determine the current segment for which you need to understand the storyboard.

[0096] In other words, after obtaining N paragraphs related to the topic, the current paragraph that has not been analyzed in terms of storyboarding is identified from among the N paragraphs.

[0097] Step S603: Input the determined current segment and the preceding and following paragraphs of the current segment in the story outline into the target large language model, so as to use the target large language model to perform scene understanding of the current segment.

[0098] For example, in one instance, based on the story development order of each of the N paragraphs, the two paragraphs adjacent to the current paragraph (corresponding to the context paragraphs mentioned above) are identified. The current paragraph, along with the two adjacent paragraphs, are then input into the target large language model to perform scene segmentation and understanding. This allows for a comprehensive understanding of the current paragraph by considering the context, such as background, visual description, and the subject's expression. This yields multiple prompts for the current paragraph, providing strong support for improving the relevance of the generated images and text content. Simultaneously, it effectively avoids generating images from the same angle for consecutive paragraphs, thus laying the foundation for further enhancing the user experience.

[0099] Step S604: Obtain multiple prompt messages for the current segment.

[0100] Here, multiple prompts in the current segment can describe the scene from multiple dimensions. For example, multiple prompts in the current segment can describe the scene from multiple dimensions such as background, scene description, and character expressions. This makes it easier to control the rhythm and theme of each segment's story and lays the foundation for accurately generating images that match each segment in the future.

[0101] By using the above method to analyze the scenes of each of the N paragraphs, multiple hints for each paragraph can be obtained.

[0102] Step S605: Based on multiple prompts for each of the N paragraphs, obtain images for each paragraph to obtain a target picture book for the theme content.

[0103] In this way, the disclosed solution utilizes a large language model to perform scene understanding of each paragraph, so as to obtain prompt information that can accurately describe the scene of each paragraph. This facilitates the accurate generation of target picture books based on the prompt information after scene understanding. Thus, compared with traditional manual picture book production, the disclosed solution using a large language model can significantly improve the efficiency of picture book creation. Moreover, it can precisely control the theme, scene and rhythm of the story in the images, enhance the ability to generate images from text, and thus generate picture book content that is of higher quality, more diverse and meets the personalized creative needs of users, thereby further improving the user experience.

[0104] Furthermore, in a specific example, the present disclosure can also determine the target style of the topic content, thereby enabling the generated image to possess that target style, thus further improving the user experience. Specifically, the above-described method of obtaining images for each paragraph based on multiple prompts for each of the N paragraphs (e.g., step S605 above) specifically includes:

[0105] Step S605-1: Determine the target style of the topic content.

[0106] Step S605-2: Determine the image generation model corresponding to the target style.

[0107] In one example, an image generation model capable of outputting a preset style can be pre-trained. For instance, multiple preset styles can be identified, and a stable diffusion (SD) model can be pre-trained for each style. The trained SD model can then generate images in the preset style. Based on this, once the target style is determined, a model matching that target style can be selected from the pre-trained SD models. Using this matching model, images in the target style can be generated, ensuring stylistic consistency across different images in the target sketchbook and further enhancing the user experience.

[0108] In this example, several related instances of each preset style can be used to fine-tune and pre-train specific parameters of a preset large SD model to obtain a pre-trained large SD model corresponding to the preset style. Thus, the above method can be used to obtain the large SD model corresponding to each preset style.

[0109] Furthermore, the pre-trained SD large models can reside in the graphics card memory to improve the speed of drawing production.

[0110] Step S605-3: Input multiple prompts from each of the N paragraphs into the image generation model corresponding to the target style to obtain images for each paragraph, wherein the images of each paragraph have the target style.

[0111] For example, in one instance, after displaying a story outline with content around the theme, in response to a click action for creating a picture book, multiple preset styles can be displayed (e.g., keywords for preset styles). At this point, the target user can select the desired style. In this way, the user's customization needs are met through the above interaction, laying the foundation for obtaining the picture book the user needs in the future. At the same time, it also lays the foundation for further improving and enriching the user experience.

[0112] In this way, the disclosed solution generates a sketchbook in the style required by the target audience based on their needs and preferences. Moreover, the above process does not require drawing skills to create a personalized sketchbook that meets the user's needs, effectively lowering the threshold for sketchbook production, satisfying the user's customization needs, and thus further enhancing the user experience.

[0113] The following detailed explanation of the disclosed solution, with reference to specific examples, specifically as shown in Figures 7(a) and 7(b), includes the following:

[0114] Step S701: In response to a click operation on the application entry point, for example, the client responds to the click operation and displays the input interface for creating a sketchbook.

[0115] Here, the application entry points can be specifically the homepage, the floating ball, the editor, and the welcome card.

[0116] Step S702: In response to the confirmation operation of the input content (such as topic content, prompt words, etc.) for the target object, display the story outline for the input content.

[0117] For example, in one instance, the client responds to the confirmation operation of the content entered by the target object by sending an outline generation request to the server; further, the server sends the outline generation request to the task scheduling center, which creates a task based on the outline generation request and stores the created task in a queue to wait for the allocation of processing resources for the outline generation request.

[0118] Furthermore, based on the tasks in the queue, when the task needs to be executed, the task scheduling center sends an outline generation request to the generation side, so that the generation side can call the trained content generation model and use the called content generation model to generate a story outline for the content input by the target object, so as to send the generated story outline to the client through the server.

[0119] Here, the content generation model described above can be specifically a large model, or more specifically, a large language model. Alternatively, it can be any other text generation model, as long as it can implement the above method; this disclosure does not limit this.

[0120] Furthermore, when a story outline is displayed on the client side, an adjusted story outline can be obtained in response to editing operations on the displayed story outline.

[0121] Furthermore, in this example, the content input by the target object can be user instructions, text content (such as plain text), local documents, or documents selected by the target object from the database, etc. This disclosure does not impose specific restrictions on this.

[0122] Step S703: In response to a submission of a story outline (or a revised story outline), display a style list page containing various preset styles (such as colored pencil style, outline style, comic book style, etc.).

[0123] Step S704: In response to a click operation on the style list page, obtain the target style selected by the target object, generate a picture book production task based on the obtained story outline and the target style, and enter the picture book production waiting queue, while displaying the real-time progress of the picture book production task (e.g., displaying a generation waiting page).

[0124] For example, as shown in Figure 7(b), firstly, after obtaining the story outline and the target style selected by the target object, the client generates a picture book production request and sends it to the server. The server then sends the received picture book production request to the task scheduling center to create a picture book production task. Here, the task scheduling center can return task creation information to the server; for example, if the task is successfully created, it returns the task ID; otherwise, it returns the task status.

[0125] Secondly, upon successful task creation, the server saves the task ID returned by the task scheduling center. Simultaneously, the task scheduling center sends a picture book generation request to the generation side for picture book production. Furthermore, the generation side can call an image generation model that matches the target style to generate images. Further, after completing the picture book production, the picture book data is returned to the task scheduling center for data storage, so that the picture book data can be sent to the client via the server.

[0126] It should be noted that during the picture book generation process, the task scheduling center can also return the current task status of the picture book production task (such as completion percentage) to the server. At this time, the server generates a prompt message (such as "Picture book has been generated") based on the received task status and sends it to the client.

[0127] Furthermore, in this example, after receiving the picture book generation request from the task scheduling center, the generation side, on the one hand, uses a large language model to perform storyboard understanding on the story outline (or the adjusted story outline) to obtain multiple hints for each paragraph in the story outline; on the other hand, based on the target style selected by the target object, the generation side calls the image generation model corresponding to the target style to generate the target picture book required by the target object using the image generation model corresponding to the target style.

[0128] Furthermore, for details regarding large language models and image generation models, please refer to the examples above; they will not be repeated here.

[0129] Furthermore, in another example, when the task ID returned by the task scheduling center is saved on the server side, if the task scheduling center fails to send the picture book generation request to the generation side, it returns the task status (such as being in a queue) to the server side, and then the server side sends the task status and task ID to the client side to show the user that the current picture book production task is in a queue. At this time, the client will initiate a task polling request based on the task ID to update the current page in real time.

[0130] In addition, during the asynchronous generation of the picture book, the server sends the real-time progress of the picture book production task to the client in the form of a push notification through a message queue mechanism, so that the client can display it.

[0131] Step S705: If a page showing that the picture book has been successfully generated is displayed, in response to a click operation of the preview function for the picture book, a playback page for the picture book is displayed.

[0132] Here, on the playback page for the picture book, the target object can play the generated picture book with images and text, play audio, perform page-turning operations with special page-turning effects, and pause and drag the page.

[0133] For example, as shown in Figure 7(b), in response to a preview operation on a picture book, the client generates a preview request based on the task ID corresponding to the picture book and sends the preview request to the server. The server determines the task information associated with the task ID based on the task ID contained in the received preview request, and generates a data acquisition request for the picture book data based on the determined task information, and sends the request to the task scheduling center. The task scheduling center returns the stored picture book data to the server based on the data acquisition request. The server directly renders the picture book based on the picture book data and sends the picture book directly to the client for the user to preview.

[0134] Step S706: In response to the editing operation on the sketchbook, the edited sketchbook is obtained.

[0135] Here, the editing operation can be used to edit the title, each page image, and the text (or copy) and audio in the picture book to regenerate it. In this way, a picture book that satisfies the user can be obtained, which meets the user's personalized needs and improves the user experience.

[0136] Thus, this disclosed solution has the following advantages and characteristics in practical applications, specifically including:

[0137] (1) Full lifecycle automation. This disclosed solution automates all stages from user commands and style selection to content generation, audio and video synthesis and optimization, thereby effectively reducing labor and time costs.

[0138] (2) High availability. This publicly available solution ensures the stability and availability of the system by introducing key technologies such as distributed microservice architecture, cloud deployment and load balancing, so as to meet the needs of large-scale and high-efficiency creation.

[0139] (3) Scalability. This publicly available solution supports dynamic expansion and scaling, and can increase or decrease the number of nodes according to actual needs to adapt to different scales of creative needs, while controlling the production cost of the picture book.

[0140] (4) Automated audio and video synthesis. This disclosed solution also supports real-time generation of audio readings. After users freely modify the text, they can immediately generate the corresponding audio data. In this way, the real-time audio generation function greatly facilitates users to adjust and improve the content during use, thereby enhancing practicality and convenience.

[0141] (5) Innovation. Compared with existing technologies, the proposed solution supports the editing and replacement of single-page images and text. Moreover, it can generate higher-quality, more diverse, and more user-expected picture book content through prompt word optimization and retry mechanisms.

[0142] Furthermore, in practical applications, such as Figure 8 As shown, this disclosure provides a picture book generation system. The system adopts a layered architecture, specifically including a data collection layer, a model training layer, a content generation layer, a quality evaluation layer, and an optimization and feedback layer.

[0143] (1) Data collection layer: used to collect a large amount of painting materials, style samples and user needs, etc., to provide a rich data source for subsequent model (such as image generation model) training.

[0144] (2) Model Training Layer: This layer uses deep learning technology to process and analyze the collected data to train a high-quality picture book model (such as an image generation model). At the same time, this layer also has the ability to automatically update and optimize the model to adapt to constantly changing data and user needs.

[0145] (3) Content generation layer: This layer is used to generate diverse picture book content based on user needs using the trained picture book model. In addition, this layer supports multiple creative styles to meet different user needs.

[0146] (4) Quality Assessment Layer: This layer assesses the quality of the generated picture book content to ensure it meets expectations. Specific assessment metrics may include image clarity, stylistic consistency, and content innovation. This allows for real-time monitoring of the generated picture book content's quality, ensuring an unaffected user experience.

[0147] (5) Optimization and Feedback Layer: This layer optimizes and adjusts the picture book model based on quality assessment results (such as assessment indicators) to improve the accuracy and efficiency of picture book content generation. Simultaneously, it collects user feedback for further improvement and refinement. This enhances the accuracy of large-scale picture book generation, thereby ensuring the stability and usability of the entire picture book generation system.

[0148] In this way, the disclosed solution can quickly and automatically generate higher-quality picture book content, realize the automated process of picture book generation, lower the threshold for picture book production, and enable users to obtain the picture books they need without spending a lot of time and effort, thereby improving the user's creative experience.

[0149] In summary, compared with the existing technology of handmade drawing books, the disclosed solution has the following advantages, specifically including:

[0150] First, it automates the generation of picture book content. This publicly available solution can automatically identify and generate picture book content through large-scale modeling technology, effectively avoiding the tedious process of traditional handmade picture book production and improving the efficiency of picture book production.

[0151] Second, intelligent typesetting design. This publicly available solution can achieve intelligent typesetting design through large-scale model technology. For example, it can automatically and reasonably arrange and adjust the content of the picture book, making the picture book more beautiful and more readable.

[0152] Third, personalized customization. This publicly available solution generates picture book content that meets the user's requirements and preferences, thereby satisfying the user's personalized needs.

[0153] Fourth, enhanced interactivity. This publicly available solution can provide richer playback functions through large-scale model technology, such as text and image playback, and video playback, to improve the user's reading experience.

[0154] Fifth, cross-platform adaptability. This disclosed solution can also run on different operating systems and devices, thus achieving cross-platform adaptability and facilitating user use in different environments.

[0155] Furthermore, this disclosed solution can be specifically applied to the following scenarios, including:

[0156] (1) Education Field. This disclosed solution can be developed into a child-friendly picture book generation tool to create picture books suitable for children's reading. The generated picture books cleverly combine vivid and interesting illustrations with educational and entertaining text to help children understand and learn knowledge. In addition, this disclosed solution can also serve as a tool to assist teachers in instructional design, providing students with richer learning resources.

[0157] (2) Publishing Industry. This disclosed solution can provide publishing institutions with an automated and intelligent tool for creating picture books, thereby improving the quality and efficiency of picture books and reducing production costs. At the same time, it can also provide individual creators with a fast and convenient way to create picture books, facilitating the rapid completion of their works.

[0158] (3) Digital Media. This disclosed solution can be applied to digital media platforms to provide users with a personalized picture book reading experience. Moreover, through interactive functions and intelligent question answering, it enhances the interactivity between users and picture books, effectively increasing users' reading interest and participation.

[0159] (4) Personal entertainment. This disclosed solution can also be applied to the field of personal entertainment, providing users with an easy and fun tool for creating sketchbooks. For example, users can create their own personalized sketchbooks according to their own needs and preferences for reading, sharing and communication.

[0160] This disclosure also provides a picture book generation device, such as Figure 9 As shown, it includes:

[0161] The text processing unit 901 is configured to generate a story outline around a determined topic based on that topic; wherein the story outline contains N paragraphs around the topic; and N is an integer greater than or equal to 2.

[0162] The multimedia generation unit 902 is used to perform scene analysis on each of the N paragraphs contained in the story outline to obtain multiple prompts for each paragraph; wherein, the multiple prompts for each paragraph can describe the scene to which the paragraph is located; based on the multiple prompts for each of the N paragraphs, images for each paragraph are obtained to obtain a target picture book for the theme content.

[0163] In a specific example of the disclosed solution, the multimedia generation unit is further configured to:

[0164] Based on the text content of each paragraph, audio data is generated for each paragraph.

[0165] The audio data of each segment is assembled with the images in the target picture book to obtain a target multimedia picture book for the theme content.

[0166] In a specific example of the disclosed solution, the multimedia generation unit is specifically used for:

[0167] According to the story development order of the N paragraphs contained in the story outline, the images of each paragraph are superimposed to obtain the target picture book in which the images of each paragraph are superimposed. During playback, the target picture book can dynamically display the page-turning effect around a preset axis of symmetry, and each image of the target picture book can be displayed in full size on the display interface.

[0168] In a specific example of the disclosed solution, the multimedia generation unit is specifically used for:

[0169] Based on the story development order of the N paragraphs contained in the story outline, determine the i-th image and the (i+1)-th image with a sequential display order;

[0170] The i-th image and the (i+1)-th image are superimposed according to the following superposition steps to obtain the target picture book in which the images of each paragraph are superimposed:

[0171] A blank display area is superimposed on the layer above the (i+1)th image; the area of ​​the blank display area is the same size as the area of ​​the i-th image.

[0172] Place the sub-image located on one side of the preset axis of symmetry in the i-th image on one side of the blank display area;

[0173] Flip the sub-image located on one side of the preset axis of symmetry in the (i+1)th image, and then attach it to the back of the sub-image located on the other side of the preset axis of symmetry in the i-th image;

[0174] Place the pasted image on the other side of the blank display area to obtain the overlaid image.

[0175] In a specific example of the disclosed solution, the multimedia generation unit is specifically used for:

[0176] Determine the current segment that requires the understanding of the storyboard;

[0177] The determined current segment, as well as the preceding and following paragraphs in the story outline, are input into the target large language model to utilize the target large language model for segmentation and understanding of the current segment.

[0178] The system obtains multiple prompts for the current segment, which can describe the scene of the current segment from multiple dimensions.

[0179] In a specific example of the disclosed solution, the multimedia generation unit is further configured to:

[0180] Determine the target style for the stated topic content;

[0181] Determine the image generation model corresponding to the target style;

[0182] Multiple prompts from each of the N paragraphs are input into the image generation model corresponding to the target style to obtain images for each paragraph, wherein the images of each paragraph have the target style.

[0183] For a description of the specific functions and examples of each unit of the apparatus in this disclosure embodiment, please refer to the relevant descriptions of the corresponding steps in the above method embodiments, which will not be repeated here.

[0184] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0185] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0186] Figure 10 A schematic block diagram of an example electronic device 1000 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0187] like Figure 10 As shown, device 1000 includes a computing unit 1001, which can perform various appropriate actions and processes according to a computer program stored in read-only memory (ROM) 1002 or a computer program loaded from storage unit 1008 into random access memory (RAM) 1003. The RAM 1003 may also store various programs and data required for the operation of device 1000. The computing unit 1001, ROM 1002, and RAM 1003 are interconnected via bus 1004. Input / output (I / O) interface 1005 is also connected to bus 1004.

[0188] Multiple components in device 1000 are connected to I / O interface 1005, including: input unit 1006, such as keyboard, mouse, etc.; output unit 1007, such as various types of monitors, speakers, etc.; storage unit 1008, such as disk, optical disk, etc.; and communication unit 1009, such as network card, modem, wireless transceiver, etc. Communication unit 1009 allows device 1000 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0189] The computing unit 1001 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1001 performs the various methods and processes described above, such as the picture book generation method. For example, in some embodiments, the picture book generation method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 1008. In some embodiments, part or all of the computer program may be loaded and / or installed on device 1000 via ROM 1002 and / or communication unit 1009. When the computer program is loaded into RAM 1003 and executed by the computing unit 1001, one or more steps of the picture book generation method described above may be performed. Alternatively, in other embodiments, the computing unit 1001 may be configured to perform the picture book generation method by any other suitable means (e.g., by means of firmware).

[0190] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0191] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0192] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0193] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0194] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0195] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0196] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0197] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A picture book generation method, comprising: generating an outline generation request based on a topic content input by a target object, and sending the outline generation request to a task scheduling center; wherein the topic content comprises one or more keywords; creating a task according to the outline generation request through the task scheduling center to obtain a plurality of topic prompt words for the topic content; calling a target content generation model and generating a story outline around the topic content based on the plurality of topic prompt words of the topic content; wherein the story outline contains N paragraphs around the topic content; N is an integer greater than or equal to 2; calling a target large language model to perform shot understanding of the paragraph-level semantics of each paragraph in the N paragraphs contained in the story outline to obtain a plurality of prompt information for each paragraph; wherein the plurality of prompt information of a paragraph can describe the scene picture of the paragraph; calling an image generation model corresponding to a target style, and obtaining an image for each paragraph based on the plurality of prompt information of each paragraph in the N paragraphs; wherein the target style is determined based on one or more keywords in the topic content; stacking the images of each paragraph in the order of story development of each paragraph to obtain a target picture book for the topic content, to realize an automatic process from the topic content input by the target object to the generation of the target picture book; wherein calling a target large language model to perform shot understanding of the paragraph-level semantics of each paragraph in the N paragraphs contained in the story outline to obtain a plurality of prompt information for each paragraph comprises: determining a current paragraph for which shot understanding is required; inputting the determined current paragraph and the context paragraphs before and after the current paragraph in the story outline into the target large language model to perform shot understanding of the current paragraph using the target large language model, to avoid the same angle of the images generated by the previous and next paragraphs; obtaining a plurality of prompt information for the current paragraph, wherein the plurality of prompt information for the current paragraph can describe the scene picture of the current paragraph from multiple dimensions, to control the rhythm and theme of the paragraph in the story development order, to ensure that the images for each paragraph match the story development order.

2. The method of claim 1, further comprising: generating audio data for each paragraph based on the text content of each paragraph; assembling the audio data of each paragraph with the images in the target picture book to obtain a target multimedia picture book for the topic content.

3. The method of claim 1 or 2, wherein, stacking the images of each paragraph in the order of story development of each paragraph to obtain a target picture book for the topic content comprises: stacking the images of each paragraph in the order of story development of the N paragraphs contained in the story outline to obtain the target picture book in which the images of each paragraph are in a stacked state, wherein the target picture book can dynamically display a page turning effect around a preset symmetry axis during playback, and each image of the target picture book can be displayed in full size on a display interface.

4. The method of claim 3, wherein, The story development sequence of the N paragraphs contained in the story outline is used to superimpose the images of the paragraphs to obtain the target picture book in which the images of the paragraphs are in a superimposed state, including: The story development sequence of the N paragraphs contained in the story outline is used to determine the i-th image and the i+1-th image having a front and back display sequence; The i-th image and the i+1-th image are superimposed according to the following superimposition steps to obtain the target picture book in which the images of the paragraphs are in a superimposed state: A blank display area is superimposed on the i+1-th image; wherein the area of the blank display area is the same as the size of the i-th image; The sub-image located on one side of the preset symmetry axis in the i-th image is placed on one side of the blank display area; The sub-image located on one side of the preset symmetry axis in the i+1-th image is flipped and attached to the back of the sub-image located on the other side of the preset symmetry axis in the i-th image after flipping; The attached image is placed on the other side of the blank display area to obtain the superimposed image.

5. The method of claim 1, further comprising: determining a target style of the theme content; determining an image generation model corresponding to the target style; wherein the image generation model corresponding to the target style is called, and images for each paragraph are obtained based on the plurality of prompt information of each paragraph in the N paragraphs, including: inputting the plurality of prompt information of each paragraph in the N paragraphs into the image generation model corresponding to the target style to obtain images for each paragraph, wherein each paragraph image has the target style.

6. A picture book generation device, comprising: a text processing unit configured to generate an outline generation request based on theme content input by a target object, and send the outline generation request to a task scheduling center; wherein the theme content includes one or more keywords; a task is created through the task scheduling center and according to the outline generation request to obtain a plurality of theme prompts for the theme content; a target content generation model is called and a story outline around the theme content is generated based on the plurality of theme prompts for the theme content; wherein the story outline contains N paragraphs around the theme content; N is an integer greater than or equal to 2; a multimedia generation unit configured to call a target large language model, perform shot understanding of paragraph-level semantics on each paragraph in the N paragraphs contained in the story outline to obtain a plurality of prompt information for each paragraph; wherein the plurality of prompt information of a paragraph can describe a scene picture corresponding to the paragraph; call an image generation model corresponding to a target style, and obtain images for each paragraph based on the plurality of prompt information of each paragraph in the N paragraphs; wherein the target style is determined based on one or more keywords in the theme content; superimpose the images of the paragraphs according to the story development sequence of the paragraphs to obtain a target picture book for the theme content, to realize an automated process from input of the theme content by the target object to generation of the target picture book; wherein the multimedia generation unit is specifically configured to: determine a current segment required for the shot understanding; input the determined current segment and context paragraphs before and after the current segment in the story outline into a target large language model to perform shot understanding on the current segment by using the target large language model, so as to avoid the generated images of the before and after segments being the same in angle; obtain multiple prompt information of the current segment, wherein the multiple prompt information of the current segment can describe the scene picture of the current segment from multiple dimensions, so as to control the rhythm and theme of the paragraph in the story development sequence, and ensure that the images of each paragraph match the story development sequence.

7. The apparatus of claim 6, wherein, The multimedia generation unit is further configured to: generate audio data for each paragraph based on the text content of the paragraph; assemble the audio data of each paragraph with the images in the target picture book to obtain a target multimedia picture book for the theme content.

8. The apparatus of claim 6 or 7, wherein, The multimedia generation unit is specifically configured to: superimpose the images of each paragraph in the story development sequence of the N paragraphs contained in the story outline to obtain the target picture book in which the images of each paragraph are in a superimposed state, wherein the target picture book can dynamically display a page turning effect around a preset symmetry axis during playback, and each image of the target picture book can be displayed in full size on a display interface.

9. The apparatus of claim 8, wherein, The multimedia generation unit is specifically configured to: determine the i-th image and the i+1-th image with a before-after display sequence in the story development sequence of the N paragraphs contained in the story outline; superimpose the i-th image and the i+1-th image according to the following superimposition steps to obtain the target picture book in which the images of each paragraph are in a superimposed state: superimpose the i-th image and the i+1-th image according to the following superimposition steps to obtain the target picture book in which the images of each paragraph are in a superimposed state: superimpose the i-th image and the i+1-th image according to the following superimposition steps to obtain the target picture book in which the images of each paragraph are in a superimposed state: superimpose the i-th image and the i+1-th image according to the following superimposition steps to obtain the target picture book in which the images of each paragraph are in a superimposed state: superimpose the i-th image and the i+1-th image according to the following superimposition steps to obtain the target picture book in which the images of each paragraph are in a superimposed state:

10. The apparatus of claim 6, wherein, The multimedia generation unit is further configured to: determine a target style for the theme content; determine an image generation model corresponding to the target style; input the multiple prompt information of each segment in the N paragraphs into the image generation model corresponding to the target style to obtain images for each paragraph, wherein each image of each paragraph has the target style.

11. An electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-5.

12. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to enable the computer to perform the method of any one of claims 1-5.

13. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-5.

Images