Digital picture book generation method and system

Abstract

The invention relates to a digital picture book generation method and system. The digital picture book generation method comprises the steps of input receiving, input standardization, capability judgment, multi-modal analysis, page-by-page planning, page-level verification, execution generation and auditing disk falling. Compared with the prior art, the method has the advantages that whether the AI image generation model supports the reference image or not is judged in the reference image input stage, whether the AI image generation model meets the multi-modal execution condition or not is judged in the picture book generation stage, double-layer capacity judgment guarantee is achieved, and a reliable degradation strategy is set; when the AI image generation model does not support the reference image or the analysis of the reference image fails or the AI image generation model does not meet the multi-modal execution condition, degrading to generate a digital text picture book so as to keep stable output; and through a reproducible, verifiable page list generated page by page, the context is referenced across stages.

Description

A method and system for generating digital picture books Technical Field

[0001] This invention relates to the field of digital picture book generation, and in particular to a method and system for generating digital picture books based on AI image generation models. Background Technology

[0002] Currently, most digital picture book generation systems based on AI image generation models typically use reference images throughout the three stages of digital picture book generation: New Tale (creation), Plan (page-by-page planning), and Edit / Re-generate (local regeneration). Existing solutions generally suffer from the following problems:

[0003] 1. The input sources of reference images are heterogeneous and the mapping is not consistent: Uploaded reference images, historical page images, and asset library images often use different fields, and it is difficult to unify id / sourceType / label, which can easily lead to loss of context across stages.

[0004] 2. Coarse-grained judgment on whether the AI ​​image generation model used supports reference graphs: Only the front end or a single node determines whether "reference graphs are supported", lacking secondary verification on the server side, which is prone to capability mismatch after model switching.

[0005] 3. Insufficient page-by-page constraints: Most digital picture book generation systems can only output global AI model prompts, without structural constraints on the usage of each page (retained elements / changed elements), resulting in unstable anchor points between characters and scenes.

[0006] 4. Incomplete audit evidence chain: The reference diagram analysis results, page-level reference list (referenceContext), and actual execution prompts are stored in a scattered manner, making it difficult to trace the source and purpose of the references.

[0007] 5. The downgrade strategy is not verifiable: There is a lack of unified downgrade records for scenarios such as AI image generation model not supporting, parsing failure, and invalid references, making it difficult to assess the effects and risks.

[0008] Therefore, the existing digital picture book generation systems still need further improvement in how they generate digital picture books. Summary of the Invention

[0009] The primary technical problem addressed by this invention is to provide a digital picture book generation method that addresses the aforementioned prior art. This method can maintain stable output even when the AI ​​image generation model does not support the reference image or the reference image parsing fails. It can also provide a reproducible and verifiable list of pages during page-by-page generation, allowing for cross-stage context referencing.

[0010] The further technical problem to be solved by the present invention is to provide a digital picture book generation system that applies the above-mentioned digital picture book generation method to the prior art.

[0011] The technical solution adopted by this invention to solve the aforementioned primary technical problem is as follows: a digital picture book generation method, applicable to a digital picture book generation system based on an AI image generation model, characterized by comprising the following steps:

[0012] Step 1: Receive input: Receive story text, model parameters of the AI ​​image generation model, and reference image;

[0013] Step 2, Input Standardization: Standardize the received reference drawings and perform specification verification;

[0014] Step 3, Capability Determination: Determine whether the AI ​​image generation model used by the digital picture book generation system supports reference images. If the AI ​​image generation model used by the digital picture book generation system supports reference images, proceed to Step 4; if the AI ​​image generation model used by the digital picture book generation system does not support reference images, proceed to Step 8.

[0015] Step 4, Multimodal parsing: Perform multimodal parsing on the reference graph. If the reference graph is parsed successfully, generate a structured summary of the reference graph and proceed to Step 5; if the reference graph parsing fails, proceed to Step 8.

[0016] Step 5, Page-by-page planning: Generate a page list page by page based on the structured summary of the reference diagram, the story text received in Step 1, and the page order, and then proceed to Step 6;

[0017] Step 6, Page-level validation: Perform structure validation, cross-page validity validation, and usage constraint validation on the page list generated in Step 5. If the validation passes, proceed to Step 7; if the validation fails, proceed to Step 8.

[0018] Step 7: Execute Generation: Perform capability determination again to determine whether the AI ​​image generation model used by the digital picture book generation system meets the multimodal execution conditions. If yes, execute multimodal image fusion to generate a digital picture book; otherwise, proceed to step 8.

[0019] Step 8, Audit Log: The downgraded data is used to generate a digital text book and record the downgrade information to form a replayable audit trail.

[0020] As an improvement, in step 3, a whitelist of AI image generation models that support reference graphs is created. If the AI ​​image generation model used by the current digital picture book generation system is in the whitelist, it indicates that the AI ​​image generation model used by the current digital picture book generation system supports reference graphs. The reference graph support flag is set to false, and the reference graph payload continues to be sent to the AI ​​image generation model. If the AI ​​image generation model used by the current digital picture book generation system is not in the whitelist, it indicates that the AI ​​image generation model used by the current digital picture book generation system does not support reference graphs. The reference graph support flag is set to true, and the reference graph payload is no longer sent to the AI ​​image generation model in this process. In the capability determination process in step 7, if the reference graph support flag is false, and there is a page list generated in step 5, and there is a parsed reference graph, it indicates that the AI ​​image generation model used by the current digital picture book generation system meets the multimodal execution conditions. The parsed reference graph payload continues to be sent to the AI ​​image generation model. Otherwise, it indicates that the AI ​​image generation model used by the current digital picture book generation system does not meet the multimodal execution conditions.

[0021] Further improvements include a page list generated in step 5 that includes a reference identifier, a reference image source type, and a usage description. The usage description field describes both "specified elements to be retained" and "elements to be changed." The "specified elements to be retained" include anchor information used to maintain the character's identity, such as the character's face, clothing, and key props. The "elements to be changed" include mandatory difference information used to generate differences, such as scene, time, lighting, camera position, and actions. The digital picture book generation system also has a constraint on the maximum number of references per page to avoid overfitting or near-perfect copying.

[0022] In a further improvement, the structured summary of the reference diagram generated in step 4 uses unified structured data, the content of which includes the main body of the reference diagram, the style of the reference diagram, and the composition of the reference diagram, for reuse in subsequent generation and auditing.

[0023] Further improvements include allowing the generated page list in step 5 to support reference images from uploaded reference images, historical page reference images, or asset library reference images, and restricting cross-page forward references in the page list.

[0024] In a further improvement, step 8 includes degradation information such as degradation reasons, reference image data, and audit records. Degradation reasons include AI image generation model not supporting reference image, reference image parsing failure, page-level verification failure, and AI image generation model not meeting multimodal execution conditions.

[0025] In a further improvement, the page list generated in step 5, along with the page-by-page prompts and the actual execution prompts, are recorded together in the audit storage for backtracking.

[0026] The technical solution adopted by the present invention to solve the above-mentioned further technical problems is as follows: a digital picture book generation system based on an AI image generation model, characterized in that: the digital picture book generation system uses the above-mentioned digital picture book generation method to generate digital picture books.

[0027] Compared with the prior art, the advantages of the present invention are as follows: The present invention determines whether the AI ​​image generation model supports the reference image during the reference image input stage, and determines whether the AI ​​image generation model meets the multimodal execution conditions during the picture book generation stage. This dual-layer capability judgment ensures stability, and a reliable degradation strategy is set up to degrade the generation of digital text picture books when the AI ​​image generation model does not support the reference image, fails to parse the reference image, or does not meet the multimodal execution conditions, thereby maintaining stable output. Furthermore, the present invention uses a reproducible and verifiable page list generated page by page to facilitate cross-stage context referencing. Attached Figure Description

[0028] Figure 1 is a flowchart of the digital picture book generation method in an embodiment of the present invention. Detailed Implementation

[0029] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0030] This embodiment provides a digital picture book generation method, which is applicable to digital picture book generation systems based on AI image generation models. As shown in Figure 1, it includes the following steps:

[0031] Step 1: Receive Input: Receive story text, model parameters from the AI ​​image generation model, and reference images. Reference images can be sourced from three types: uploaded reference images, historical page reference images, or asset library reference images. The following fields are set to represent the characteristics of the reference images: {id, sourceType, label, use}. The id field sets the reference image's ID code; the sourceType field sets the source type of the reference image; the label field sets the reference label of the reference image; and the use field sets the purpose of the reference image.

[0032] Step 2, Input Standardization: Standardize the received reference images, specifically by cropping and removing null values ​​from id, label, and use, deleting id, and performing specification validation. This involves setting a maximum limit on the total number of reference images, a maximum limit on a single request, and validation of file size and format. In this embodiment, the default settings are: maximum number of reference images = 20, maximum number of single requests = 7, maximum file size = 10Mb, and supported formats: JPEG / PNG / WebP. Cross-page validity is also set, meaning the current page can only reference previously generated pages. If an invalid reference is found, it is removed and the reason is recorded for auditing and troubleshooting.

[0033] Step 3, Capability Determination: Determine whether the AI ​​image generation model used by the digital picture book generation system supports reference images. If the AI ​​image generation model used by the digital picture book generation system supports reference images, proceed to Step 4; if the AI ​​image generation model used by the digital picture book generation system does not support reference images, proceed to Step 8.

[0034] In this step, a whitelist of AI image generation models that support reference graphs needs to be created. If the AI ​​image generation model used by the current digital picture book generation system is in the whitelist, it means that the AI ​​image generation model used by the current digital picture book generation system supports reference graphs. The reference graph support flag is set to false, and the reference graph payload is sent to the AI ​​image generation model. If the AI ​​image generation model used by the current digital picture book generation system is not in the whitelist, it means that the AI ​​image generation model used by the current digital picture book generation system does not support reference graphs. The reference graph support flag is set to true, and the reference graph payload is no longer sent to the AI ​​image generation model in this process.

[0035] Step 4, Multimodal parsing: Perform multimodal parsing on the reference graph. If the reference graph is parsed successfully, generate a structured summary of the reference graph and proceed to Step 5; if the reference graph parsing fails, proceed to Step 8. In this step, the structured summary of the reference graph is generated using unified structured data, the content of which includes the main body of the reference graph, the style of the reference graph, and the composition of the reference graph, for reuse in subsequent generation and auditing.

[0036] Step 5, Page-by-page planning: Generate a page list page by page based on the structured summary of the reference diagram, the story text received in Step 1, and the page order, and then proceed to Step 6;

[0037] In this step, the generated page list includes a reference identifier, the source type of the reference image, and a usage description. The usage description field describes both "specified retained elements" and "elements to be changed." "Specified retained elements" include anchor information used to maintain character identity, such as character face shape, clothing, and key props. "Elements to be changed" include mandatory difference information used to generate differences, such as scene, time, lighting, camera position, and actions. The digital picture book generation system also has a constraint on the maximum number of references per page to avoid overfitting or near-perfect copying. Furthermore, the reference image sources supported by the page list generated in this step are uploaded reference images, historical page reference images, or asset library reference images. The page list restricts cross-page forward references. At the same time, the page list generated in this step, along with the page-by-page prompt artifacts and the actual execution prompt artifacts, are recorded in the audit storage for retrospective purposes.

[0038] Step 6, Page-level validation: Perform structure validation, cross-page validity validation, and usage constraint validation on the page list generated in Step 5. If the validation passes, proceed to Step 7; if the validation fails, proceed to Step 8.

[0039] Step 7, Execute Generation: Perform capability determination again to determine whether the AI ​​image generation model used by the digital picture book generation system meets the multimodal execution conditions. If yes, perform multimodal image fusion to generate a digital picture book; otherwise, proceed to Step 8. During this second capability determination process, if the reference image support is marked as false, and there is a page list generated in Step 5, and there is a parsed reference image, it indicates that the AI ​​image generation model used by the current digital picture book generation system meets the multimodal execution conditions, and the parsed reference image payload is sent to the AI ​​image generation model. Otherwise, it indicates that the AI ​​image generation model used by the current digital picture book generation system does not meet the multimodal execution conditions.

[0040] Step 8, Audit Log: Degrade the digital text book and record the degradation information to form a replayable audit link; the degradation information includes the degradation reason, reference image data, and audit records, among which the degradation reasons include the AI ​​image generation model not supporting the reference image, reference image parsing failure, page-level verification failure, and the AI ​​image generation model not meeting the multimodal execution conditions.

[0041] The aforementioned digital picture book generation method performs multimodal parsing on reference images when the sources of reference images are diverse, and generates a structured summary of unified structured data. This ensures that the page list generated page by page maintains a consistent page-by-page reference structure, reducing context loss and reference mismatch. The dual-gate capability determination (corresponding to steps 3 and 7) reduces the risk of "mistransmitted reference images" and "model capability mismatch". The description of the purpose field in the page list and page-level verification improve the stability of role and scene anchors and suppress approximate cloning. The end-to-end disk storage and degradation record improve troubleshooting efficiency and audit traceability.

[0042] This embodiment further provides a digital picture book generation system based on an AI image generation model. The system includes a front-end input module, an input verification module, a creation-stage capability gate module, a reference image parsing module, a page-by-page prompt generation module, a page-by-page planning module, a page-level verification module, an execution-stage capability gate module, a generation module, and an audit and disk loading module and a degradation control module. This digital picture book generation system uses the aforementioned digital picture book generation method to generate digital picture books. Specifically, step 1 of the digital picture book generation method is completed by the front-end input module; step 2 by the input verification module; step 3 by the creation-stage capability gate module; step 4 by the reference image parsing module; step 5 by the page-by-page prompt generation module; step 6 by the page-level verification module; step 7 by the execution-stage capability gate module and the generation module; and step 8 by the audit and disk loading module and the degradation control module.

[0043] In specific application scenarios, during the creation phase, the front-end input module receives the story text, model parameters of the AI ​​image generation model, and reference images. The input verification module then standardizes the input reference images. During the creation phase, the capability gate module determines whether the AI ​​image generation model used by the digital picture book generation system supports reference images. If the AI ​​image generation model does not support reference images, the digital text picture book is generated using the audit and disk loading module and the degradation control module. If the AI ​​image generation model supports reference images, the reference image parsing module performs multimodal parsing on the reference images. If the reference image parsing fails, the digital text picture book is generated using the audit and disk loading module and the degradation control module. The process involves several steps: If the reference image is successfully parsed, the reference image parsing module generates a structured summary of the reference image. Then, the page planning module generates a page list based on the structured summary of the reference image, the story text, and the page order. The page-level validation module performs structure validation, cross-page validity validation, and usage constraint validation on the page list. If the validation fails, the audit disking module and the degradation control module are used to downgrade the digital text picture book. If the validation passes, the execution phase capability gate module performs capability judgment again to determine whether the AI ​​image generation model used by the digital picture book generation system meets the multimodal execution conditions. If not, the audit disking module and the degradation control module are used to downgrade the digital text picture book. If so, multimodal image fusion is performed to generate a digital image picture book.

[0044] For example, in creating a scene instance: a user uploads 3 reference images (character, prop, and style image) to create a 12-page picture book. If the model supports reference images, the system first generates a structured summary of the reference images, and then outputs a page list for each page of the 12-page picture book. In planning a scene, if a user references page_2 and img1 on page 6, the system retains the page reference and verifies that page_2 < 6. If the user mistakenly references page_8, it is automatically removed and the audit reason is recorded. In regenerating a scene instance, if a user regenerates page 9, only the page list for that page is submitted. If the system switches to a model that does not support reference images, it is set to output only a numeric text picture book, but the reference metadata and degradation records are retained.

Claims

1. A method for generating digital picture books, applicable to digital picture book generation systems based on AI image generation models, characterized in that... The process includes the following steps: Step 1, Receiving Input: Receive the story text, model parameters of the AI ​​image generation model, and reference images; Step 2, Input Standardization: Standardize the received reference images and perform specification verification; Step 3, Capability Determination: Determine whether the AI ​​image generation model used by the digital picture book generation system supports reference images. If the AI ​​image generation model supports reference images, proceed to Step 4; if the AI ​​image generation model does not support reference images, proceed to Step 8; Step 4, Multimodal Parsing: Perform multimodal parsing on the reference images. If the parsing is successful, generate a structured summary of the reference images and proceed to Step 5; if the parsing fails... Execute step 8; Step 5, Page-by-page planning: Generate a page list page by page based on the structured summary of the reference diagram, the story text received in step 1, and the page order, and then execute step 6; Step 6, Page-level verification: Perform structure verification, cross-page legality verification, and usage constraint verification on the page list generated in step 5. If the verification passes, execute step 7; if the verification fails, execute step 8; Step 7, Execute generation: Perform capability judgment again to determine whether the AI ​​image generation model used by the digital picture book generation system meets the multimodal execution conditions. If yes, execute multimodal image fusion to generate a digital image picture book; if not, execute step 8; Step 8, Audit and disk: Degrade the generated digital text picture book and record the degradation information to form a replayable audit link.

2. The digital picture book generation method according to claim 1, characterized in that: In step 3, a whitelist of AI image generation models that support reference graphs is created. If the AI ​​image generation model used by the current digital picture book generation system is in the whitelist, it indicates that the AI ​​image generation model used by the current digital picture book generation system supports reference graphs. The reference graph support flag is set to false, and the reference graph payload continues to be sent to the AI ​​image generation model. If the AI ​​image generation model used by the current digital picture book generation system is not in the whitelist, it indicates that the AI ​​image generation model used by the current digital picture book generation system does not support reference graphs. The reference graph support flag is set to true, and the reference graph payload is no longer sent to the AI ​​image generation model in this process. In step 7, during the capability determination process again, if the reference graph support flag is false, and the page list generated in step 5 exists, and there is a parsed reference graph, it indicates that the AI ​​image generation model used by the current digital picture book generation system meets the multimodal execution conditions. The parsed reference graph payload continues to be sent to the AI ​​image generation model. Otherwise, it indicates that the AI ​​image generation model used by the current digital picture book generation system does not meet the multimodal execution conditions.

3. The digital picture book generation method according to claim 1, characterized in that: The page list generated in step 5 includes a reference identifier, the source type of the reference image, and a usage description. The usage description field describes both "specified elements to be retained" and "elements to be changed". The "specified elements to be retained" include anchor information used to maintain the character's identity, and the "elements to be changed" include mandatory difference information used to generate differences. The digital picture book generation system also has a constraint on the maximum number of references per page to avoid overfitting or near-copying.

4. The digital picture book generation method according to claim 1, characterized in that: In step 4, the structured summary of the reference diagram is generated using unified structured data, which includes the main body of the reference diagram, the style of the reference diagram, and the composition of the reference diagram, for reuse in subsequent generation and auditing.

5. The digital picture book generation method according to claim 1, characterized in that: The page list generated in step 5 supports reference images from uploaded reference images, historical page reference images, or asset library reference images. The page list restricts cross-page forward referencing.

6. The digital picture book generation method according to claim 1, characterized in that: In step 8, the degradation information includes degradation reasons, reference graph data, and audit records. The degradation reasons include the AI ​​image generation model not supporting the reference graph, reference graph parsing failure, page-level verification failure, and the AI ​​image generation model not meeting the multimodal execution conditions.

7. The digital picture book generation method according to claim 1, characterized in that: The page list generated in step 5, along with the page-by-page prompt artifacts and the actual execution prompt artifacts, are recorded together in the audit storage for backtracking.

8. A digital picture book generation system based on an AI image generation model, characterized in that: The digital picture book generation system uses any one of the digital picture book generation methods described in claims 1 to 7 to generate digital picture books.

Images