A system and method for intelligent automated generation of images

Abstract

The application discloses a kind of intelligent automation generation system and method of image, including natural language requirement analysis module, for receiving the natural language input of user, it is decomposed into structured demand by pre-training semantic analysis model;Generation scheduling and workflow engine, according to the structured demand parsed automatically arranges the workflow of one or more models;Intelligent constraint and control unit, for extracting accurate constraint from structured demand, accurate constraint is converted into condition vector in generation process, guide model to meet text prompt word while enforcing these accurate constraints;Result evaluation and feedback optimization module, the clarity of generated image, aesthetic quality, with the matching degree of demand is automatically scored.The application solves the technical problems that existing AI image generation technology is too high on user prompt word engineering ability, generation process fragmentation, and poor controllability of result.

Description

Technical Field

[0001] This invention relates to a system and method, particularly a system and method for intelligent and automated image generation, belonging to the field of intelligent and computer vision technology. Background Technology

[0002] Currently, deep learning-based generative models (such as Generative Adversarial Networks (GANs) and Diffusion Models) have made significant progress in the field of text-to-image generation. In existing technologies, users input short text prompts, and AI models can generate corresponding images.

[0003] Chinese patent application CN114998451A (publication date September 2, 2022), entitled "An Image Generation Method and Apparatus Based on a Diffusion Model," describes a technique that generates an image by parsing a user's text description and using a diffusion model to progressively denoise the text. Furthermore, commercial platforms such as Midjourney and Stable Diffusion represent current mainstream technologies.

[0004] However, the aforementioned existing technologies still have the following technical problems in practical applications:

[0005] 1. User-unfriendly and with high learning curve: Generating high-quality images requires users to possess "cue word engineering" skills, namely, the precise use of specific keywords, artist styles, lighting, composition, and other professional terms. Ordinary users find it difficult to translate their ideas into effective cue words, resulting in a significant deviation between the generated results and expectations.

[0006] 2. Lack of automated workflow integration: Current platforms typically only generate an initial image, while producing a commercial-grade finished product usually requires multiple iterations, modifications, partial redrawing, enlargement, and other post-processing steps. Users must manually intervene in these steps, which is cumbersome and fails to create an automated closed loop from idea to finished product.

[0007] 3. Poor controllability of results: AI-generated images are highly random, making it difficult for users to precisely control and dynamically modify specific elements (such as object pose, background style, and color matching). Often, it is necessary to "regenerate" dozens of times to obtain a satisfactory image, which is inefficient. Summary of the Invention

[0008] The technical problem to be solved by the present invention is to provide a system and method for intelligent and automated image generation, which solves the technical problems of existing AI image generation technology being too dependent on the engineering capabilities of user prompt words, fragmented generation process, and poor controllability of results.

[0009] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: A system for intelligent and automated image generation, comprising The natural language requirement parsing module is used to receive the user's natural language input and decompose it into structured requirements through a pre-trained semantic parsing model. The generation scheduling and workflow engine connects the natural language requirement parsing module and multiple underlying generation models, and automatically orchestrates the workflow of one or more models based on the parsed structured requirements; The intelligent constraint and control unit is used to extract precise constraints from structured requirements, transform precise constraints into condition vectors in the generation process, and guide the model to enforce these precise constraints while satisfying the text prompt words. The results evaluation and feedback optimization module has a built-in image quality evaluation network that automatically scores the clarity, aesthetic quality, and matching degree of the generated image to the requirements.

[0010] Furthermore, the semantic parsing model employs a variant of BERT or a large language model.

[0011] Furthermore, the structured requirements include the main object, behavior, environmental background, style type, image quality requirements, and emotional tone.

[0012] Furthermore, the underlying generation model includes a diffusion model, a super-resolution model, and a redraw model.

[0013] Furthermore, the image quality assessment network employs a CLIP-based scoring model.

[0014] A method for generating images based on an intelligent automated image generation system includes the following steps: S1. Receive and parse user input: Obtain the user's request text described in natural language, and convert it into structured image metadata through a semantic parsing model; S2. Generate automated workflow: The workflow engine matches and instantiates a multi-stage processing flow from a preset template library based on key tags in the metadata; S3. Perform constrained image generation: Input the structured metadata and precise constraints as conditions into the first image generation model specified in the workflow to generate the initial image; S4. Automated evaluation of the generated results: Use an image quality assessment network to quantitatively score the initial image from multiple dimensions; S5. Feedback optimization based on evaluation results: If the score does not reach the preset threshold, a feedback loop is started, the score result is sent to the optimizer, an adjustment strategy is generated, and the process returns to step S3 to regenerate under the optimized parameters or workflow. S6. Output and store the final image: When the score of a generated image reaches or exceeds the threshold, or the number of iterations reaches the upper limit, the current image is output and stored as the final result.

[0015] Compared with the prior art, the present invention has the following advantages and effects: 1. Significantly lowers the barrier to entry for users: Users do not need to master complex prompting technology; they can directly use everyday natural language (such as "Draw me a picture of a Shiba Inu in a suit sitting in an office drinking coffee, photorealistic") to drive the system. Compared to existing technologies, the average user input length is reduced by about 60%, and the instruction complexity is reduced by 70%.

[0016] 2. Achieved a fully automated closed-loop image generation system: Through a built-in workflow engine and feedback optimization module, the iterative process of "generation-evaluation-optimization" is automated without manual user intervention. Experimental data shows that generating an image of commercially acceptable quality (aesthetic score > 8.5 / 10) requires an average reduction in the number of manual interventions from 4.2 in existing technologies to 0.5, resulting in an overall generation efficiency improvement of approximately 300%.

[0017] 3. Significantly enhanced controllability of results: The intelligent constraint control unit allows users to precisely control image elements in a natural way (e.g., "there must be three red balloons in the picture"). Through reinforcement learning feedback, the system can autonomously adjust the generation parameters to meet the constraint, increasing the success rate of the first generation result meeting the user's precise constraint from about 30% to over 85%. Detailed Implementation

[0018] To illustrate in detail the technical solutions adopted by the present invention to achieve the intended technical objectives, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Furthermore, the technical means or technical features in the embodiments of the present invention can be replaced without creative effort. The present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0019] The present invention provides an intelligent automated image generation system, comprising: The natural language requirement parsing module receives natural language input from users and breaks it down into structured requirements using a pre-trained semantic parsing model. The semantic parsing model employs either a variant of BERT or a large language model. The structured requirements include dimensions such as subject / object, behavior / action, environmental background, style / type, image quality requirements, and emotional tone.

[0020] The generation scheduling and workflow engine connects the natural language requirement parsing module and multiple underlying generation models, automatically orchestrating the workflow of one or more models based on the parsed structured requirements. For example, it first calls a stylization model to generate an initial draft, then calls a fine-grained redrawing model to refine facial features, and finally calls a super-resolution model to improve resolution. The underlying generation models include different diffusion models, super-resolution models, and redrawing models.

[0021] The intelligent constraint and control unit is used to extract precise constraints from structured requirements, such as "the cat's color must be white" and "the number of trees is 3". It transforms the precise constraints into condition vectors in the generation process, guiding the model to enforce these precise constraints while satisfying the text prompts.

[0022] The results evaluation and feedback optimization module incorporates an image quality assessment network that automatically scores the clarity, aesthetic quality, and matching degree of the generated images to the requirements. If the score is below a threshold, the module uses the score as a reward signal, dynamically adjusting the parameters (such as sampling steps and guidance strength) of the parsing module, workflow engine, or generation model using reinforcement learning algorithms, and triggering a new generation iteration until the requirements are met.

[0023] The image quality assessment network uses a CLIP-based scoring model.

[0024] A method for generating images based on an intelligent automated image generation system includes the following steps: S1. Receive and parse user input: Obtain the user's request text described in natural language, and convert it into structured image generation metadata through a semantic parsing model.

[0025] S2. Generate Automated Workflows: The workflow engine matches and instantiates a multi-stage processing flow from a pre-set template library based on key tags in the metadata (such as "interior design" or "game concept art"). For example, for a "product poster" requirement, it automatically generates a workflow of "background generation -> product rendering -> lighting and shadow compositing -> text layout".

[0026] S3. Perform constrained image generation: Input structured metadata and precise constraints as conditions into the first image generation model specified in the workflow to generate the initial image.

[0027] S4. Automated evaluation of the generated results: Using an image quality evaluation network, the initial image is quantitatively scored from multiple dimensions (such as text matching degree, composition aesthetics, and detail richness).

[0028] S5. Feedback optimization based on evaluation results: If the score does not reach the preset threshold, a feedback loop is started, the score result is sent to the optimizer, an adjustment strategy is generated (e.g., increase the guiding weight of the "style" dimension, or replace the next generative model), and the process returns to step S3 to regenerate under the optimized parameters or workflow.

[0029] S6. Output and store the final image: When the score of a generated image reaches or exceeds the threshold, or the number of iterations reaches the upper limit, the current image is output and stored as the final result.

[0030] The structural relationship and data flow of the intelligent automated image generation system of the present invention are as follows: User terminal -> (Natural Language Text) -> Requirement Parsing Module -> (Structured JSON Data) -> Workflow Engine.

[0031] The workflow engine connects to the intelligent constraint control unit (to obtain precise constraints) and the underlying model pool (such as SDXL, ControlNet, ESRGAN).

[0032] The workflow engine passes structured data and constraints to each model layer by layer in the orchestrated order, and the intermediate image data generated by the model flows between the nodes.

[0033] After each generation, the image data simultaneously flows to the evaluation module, and the evaluation score is fed back to the workflow engine and the requirements analysis module for adjustment in the next iteration.

[0034] The present invention will be further described below through specific embodiments.

[0035] Example 1.

[0036] This embodiment describes a specific process of generating "a cat image for social media promotion" using the system described in this invention.

[0037] The user typed: "I want a cute orange cat wearing a blue Superman cape, flying against a backdrop of blue sky and white clouds, cartoon style, high resolution." System operation: Analysis: The requirements analysis module outputs structured data: {Subject: "Orange Cat", Clothing: "Blue Superman Cloak", Action: "Flying", Background: "Blue Sky and White Clouds", Style: "Cartoon", Image Quality: "High Resolution"}. The constraint control unit extracts precise constraints: {Must-exist elements: "Blue Cloak", "Flying Posture"}.

[0038] Workflow Orchestration: The workflow engine detects the "cartoon style" tag and automatically orchestrates the workflow: Stage 1: Generate a 512x512 sketch using a cartoon diffusion model -> Stage 2: Optimize hair and cape textures using a detail enhancement model -> Stage 3: Output a 4K image using an image super-resolution model.

[0039] Iteration and optimization: First generation: The evaluation module scores the sketches generated in the first stage and finds that the "flight attitude" score is low (the cape is not fluttering). The feedback signal triggers the optimizer, increasing the weight of "motion blur" guidance.

[0040] Second generation: Based on the optimized parameters, the evaluation module gives a comprehensive score of 9.2 / 10 (out of 10), which meets the preset threshold.

[0041] Output: The system outputs a high-quality cartoon orange cat flying image.

[0042] Example 2.

[0043] This embodiment describes a different parameter configuration than Embodiment 1. When the user inputs "modern minimalist style living room interior design rendering," the workflow engine automatically switches to another workflow: Stage 1: Layout generation -> Stage 2: Material and lighting rendering -> Stage 3: Adding furniture models. Its feedback optimization module focuses on evaluation dimensions such as "rationality of spatial layout" and "realism of materials," rather than "dynamism" or "cuteness." This demonstrates the invention's strong adaptability to different application scenarios and its automated configuration capabilities.

[0044] This invention significantly lowers the user barrier: users do not need to master complex prompting technology; they can drive the system directly using everyday natural language (such as "Draw me a Shiba Inu in a suit sitting in an office drinking coffee, photorealistic"). Compared to existing technologies, the average user input length is reduced by approximately 60%, and the complexity of the instructions is reduced by 70%. It achieves a fully automated image generation loop: through a built-in workflow engine and feedback optimization module, the iterative process of "generation-evaluation-optimization" is automated, requiring no manual user intervention. Experimental data shows that generating an image of commercially acceptable quality (aesthetic score > 8.5 / 10) requires an average reduction in the number of manual interventions from 4.2 in existing technologies to 0.5, improving overall generation efficiency by approximately 300%. It greatly enhances the controllability of the results: the intelligent constraint control unit allows users to precisely control image elements in a natural way (such as "There must be three red balloons in the picture"). Through reinforcement learning feedback, the system can autonomously adjust generation parameters to meet this constraint, increasing the success rate of the first generation meeting the user's precise constraints from approximately 30% to over 85%.

[0045] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent substitutions, and improvements made to the above embodiments without departing from the scope of the present invention, based on the technical essence of the present invention and within the spirit and principles of the present invention, shall still fall within the protection scope of the present invention.

Claims

1. A system for intelligently and automatically generating images, characterized in that: Include The natural language requirement parsing module is used to receive the user's natural language input and decompose it into structured requirements through a pre-trained semantic parsing model. The generation scheduling and workflow engine connects the natural language requirement parsing module and multiple underlying generation models, and automatically orchestrates the workflow of one or more models based on the parsed structured requirements; The intelligent constraint and control unit is used to extract precise constraints from structured requirements, transform precise constraints into condition vectors in the generation process, and guide the model to enforce these precise constraints while satisfying the text prompt words. The results evaluation and feedback optimization module has a built-in image quality evaluation network that automatically scores the clarity, aesthetic quality, and matching degree of the generated image to the requirements.

2. The intelligent automated image generation system according to claim 1, characterized in that: The semantic parsing model uses either a variant of BERT or a large language model.

3. The intelligent automated image generation system according to claim 1, characterized in that: The structured requirements include the main object, behavior, environment, style, image quality, and emotional tone.

4. The intelligent automated image generation system according to claim 1, characterized in that: The underlying generation model includes a diffusion model, a super-resolution model, and a redraw model.

5. The intelligent automated image generation system according to claim 1, characterized in that: The image quality assessment network uses a CLIP-based scoring model.

6. A method for generating images based on the intelligent automated image generation system according to any one of claims 1-5, characterized in that... Includes the following steps: S1. Receive and parse user input: Obtain the user's request text described in natural language, and convert it into structured image metadata through a semantic parsing model; S2. Generate automated workflow: The workflow engine matches and instantiates a multi-stage processing flow from a preset template library based on key tags in the metadata; S3. Perform constrained image generation: Input the structured metadata and precise constraints as conditions into the first image generation model specified in the workflow to generate the initial image; S4. Automated evaluation of the generated results: Use an image quality assessment network to quantitatively score the initial image from multiple dimensions; S5. Feedback optimization based on evaluation results: If the score does not reach the preset threshold, a feedback loop is started, the score result is sent to the optimizer, an adjustment strategy is generated, and the process returns to step S3 to regenerate under the optimized parameters or workflow. S6. Output and store the final image: When the score of a generated image reaches or exceeds the threshold, or the number of iterations reaches the upper limit, the current image is output and stored as the final result.