Method and system for generating controllable high-quality AI drawing picture description

Abstract

The invention discloses a generation method and system for controllable high-quality AI drawing picture description, and the method comprises the steps: carrying out the analysis and recognition of the drawing intention of a user through a natural language processing technology, and extracting key drawing elements from the drawing intention; the meanings of the elements and the relation between the elements are understood on the basis of a deep learning neural network model, and the drawing intention of the user is expanded and supplemented on the basis of a generative adversarial network, so that various drawing cue words in different directions are generated; and on the basis of the drawing cue word, generating an image according with the description of the drawing cue word by utilizing a diffusion model image generation algorithm, and obtaining various styles of images according with an expected quality standard by introducing an image quality evaluation system. The method has the advantages that user thinking divergence can be guided, creativity is stimulated, effective interaction is achieved, feedback optimization results are fully utilized, image quality is greatly improved, and generation diversity and flexibility are increased to adapt to various requirements and scenes.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method and system for generating controllable high-quality AI painting picture descriptions. Background Art

[0002] There are many problems with existing image generation technology. For example, when the user description is inaccurate, only patterned images can be generated, which makes it difficult to meet personalization and innovation needs, and it cannot guide users to think divergently. In addition, user feedback cannot be effectively used for subsequent generation results. Summary of the invention

[0003] The purpose of the present invention is to provide a method and system for generating controllable high-quality AI painting picture descriptions, thereby solving the aforementioned problems existing in the prior art.

[0004] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0005] A method for generating a controllable high-quality AI painting picture description comprises the following steps:

[0006] S1. Element extraction: Use natural language processing technology to analyze and identify the user's drawing intention and extract the key drawing elements;

[0007] S2, prompt enhancement optimization: Based on the deep learning neural network model, the meaning of the elements and the relationship between the elements are understood, and based on the generative adversarial network, the user's drawing intention is expanded and supplemented to generate a variety of drawing prompt words in different directions;

[0008] S3. Drawing generation and screening: Based on drawing prompt words, the diffusion model image generation algorithm is used to generate images that meet the description of the drawing prompt words, and by introducing an image quality evaluation system, images of various styles that meet the expected quality standards are obtained.

[0009] Preferably, step S1 specifically includes the following contents:

[0010] S11. Preprocess the text using the preprocessing technology in the natural language processing technology, and correct spelling errors in the text through the spelling check tool and dictionary matching tool in the natural language processing technology, so that the text is clearer and more accurate, and the preliminary processing of the user's drawing intention text is achieved;

[0011] S12, inputting the preliminarily processed text into a lexical analyzer, which divides the continuous text into independent words according to the rules and vocabulary of the language;

[0012] S13, Named Entity Recognition algorithm is based on pre-trained patterns and semantic knowledge, through pattern matching and semantic understanding, to identify nouns representing core objects in the text and determine the key entities in the text;

[0013] S14, inputting the text into a sentiment analysis model to determine the overall atmosphere and emotional expression of the image;

[0014] S15. Use text classification and keyword extraction algorithms to conduct in-depth mining of the text, extract detailed descriptions of the image, and determine key information elements by counting word frequency, part of speech, and context relevance.

[0015] Preferably, step S2 specifically includes the following contents:

[0016] S21, inputting the extracted elements into a neural network model based on deep learning, performing deep learning and understanding of the elements, and understanding the meaning of the elements and the relationship between the elements;

[0017] S22, inputting the extracted elements into a generative adversarial network to expand and supplement the intention, and generating multiple expanded and supplemented versions of the user's drawing intention;

[0018] S23, use the attention mechanism to focus on new objects, emotional expressions and key details;

[0019] S24. With the help of transfer learning, the experience gained from other similar tasks is applied to the current drawing prompt optimization;

[0020] S25. Use variational autoencoding technology to explore the generation of drawing prompts in various directions.

[0021] Preferably, step S21 is specifically as follows: the deep learning neural network performs mathematical operations and changes on the elements represented by numerical vectors through the connections and weights between neurons; the model uses the knowledge and patterns previously acquired in the learning of a large amount of drawing-related data to compare the input elements with similar patterns that have been learned, and tries to understand the potential associations between the elements; through multiple forward propagation and back propagation processes, the link weights between neurons are continuously adjusted to accurately understand the meaning of the elements and the relationship between the elements.

[0022] Preferably, step S22 is specifically as follows: the generator in the generative adversarial network preliminarily generates multiple extended and supplementary versions of the user's drawing intention according to the input elements and the learned knowledge; the discriminator in the generative adversarial network evaluates whether the generated results are true and reasonable, and through adversarial training of the two, continuously optimizes the generated content, and finally generates multiple extended and supplementary versions of the user's drawing intention.

[0023] Preferably, step S3 specifically includes the following contents:

[0024] S31, using a variety of drawing prompt words, calling the diffusion model image generation algorithm to gradually generate images that meet the description of the drawing prompt words;

[0025] S32. Introduce an image quality assessment system to score the visual quality of images that meet the description of the drawing prompt words from multiple dimensions;

[0026] S33, setting a threshold for the quality score, filtering out images with a quality score lower than the threshold, and retaining images with a quality score higher than the threshold;

[0027] S34, continuously repeating the above generation and screening process until images of various styles that meet the expected quality standards are obtained.

[0028] Preferably, step S21 specifically includes the following contents:

[0029] S311, the diffusion model creates a random noise image as the initial input, and iterates in the forward diffusion process to gradually add Gaussian noise to the initial noise image;

[0030] S312, the diffusion model learns to gradually restore a clear image from a noisy image. When a new image is formed, the given drawing prompt word is used as a conditional input. The diffusion model predicts how to remove the noise based on the prompt word and the current noisy image, and gradually restores the structure and content of the image.

[0031] S313, during the back diffusion process, the diffusion model gradually removes noise, and the model predicts the adjustments that should be made to the pixel values ​​so that the image gradually becomes clearer and more meaningful;

[0032] S314, after multiple back diffusion iterations, the image gradually becomes clear and forms an image with specific content and features, and finally generates an image that meets the description of the drawing prompt word.

[0033] Preferably, before step S1, the step further includes:

[0034] S0. Intention collection: Collect user drawing intentions through guidance or direct description by users.

[0035] Preferably, after step S3, the method further includes:

[0036] S4, image return and user guidance in re-generation: return the image that meets the expected quality standard to the user, and guide the user to further generate a new image on this basis; repeat steps S0 to S4 until the user receives a satisfactory image.

[0037] The present invention also aims to provide a system for generating controllable high-quality AI painting picture descriptions, the system can implement the above-mentioned method, the system includes:

[0038] Intention collection module: collects user drawing intentions through guidance or direct description by users;

[0039] Element extraction module: uses natural language processing technology to analyze and identify the user's drawing intention and extract key drawing elements;

[0040] Prompt enhancement and optimization module: Based on the deep learning neural network model, it understands the meaning of elements and the relationship between elements, and based on the generative adversarial network, it expands and supplements the user's drawing intention to generate a variety of drawing prompts in different directions;

[0041] Drawing generation and screening module: Based on drawing prompt words, the diffusion model image generation algorithm is used to generate images that meet the description of the drawing prompt words, and by introducing an image quality evaluation system, images of various styles that meet the expected quality standards are obtained;

[0042] Image return and user-guided regeneration module: Return images that meet the expected quality standards to the user, and guide the user to further generate new images based on this; repeat the above modules until the user receives a satisfactory image.

[0043] The beneficial effects of the present invention are as follows: 1. The present invention can accurately understand the user's drawing intention, even if it is not precise enough, so as to generate high-quality images that meet their personalized and innovative needs, and no longer be stereotyped. 2. The present invention can guide the user's divergent thinking, stimulate creativity, achieve effective interaction, make full use of feedback optimization results, greatly improve image quality, and increase the diversity and flexibility of generation to adapt to various needs and scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 is a flow chart of a generating method in an embodiment of the present invention;

[0045] Figure 2 is a flowchart of text element extraction in an embodiment of the present invention;

[0046] Figure 3 is a flowchart of prompting enhanced optimization in an embodiment of the present invention;

[0047] Figure 4 4 is a flowchart of drawing generation and screening in an embodiment of the present invention. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation methods described herein are only used to explain the present invention and are not used to limit the present invention.

[0049] like Figure 1 As shown, in this embodiment, a method for generating a controllable high-quality AI painting picture description is provided, which can guide the user's divergent thinking and effectively use user feedback to continuously improve the generated results, thereby solving the above-mentioned problems existing in the prior art. It includes the following parts:

[0050] 1. Intention Collection

[0051] Collect user drawing intentions through guidance or direct description by users.

[0052] Specifically, provide users with clear and friendly instructions on how to accurately express their drawing intentions. For example, you can give some example descriptions, such as "I want an image of a horse running in the sunset" or "draw a city night scene full of futuristic technology", so that users can better understand how to clearly express their needs.

[0053] 2. Feature Extraction

[0054] Natural language processing technology is used to analyze and identify users' drawing intentions and extract key drawing elements.

[0055] like Figure 2 As shown, it specifically includes the following contents:

[0056] 2.1. Use natural language processing technology to preliminarily process the user input of the drawing intention text:

[0057] The preprocessing technology in natural language processing technology is used to preprocess the text (remove special characters, useless punctuation marks and other noise in the text), and the spelling checking tools and dictionary matching tools in natural language processing technology are used to correct spelling errors in the text, making the text clearer and more accurate, and realizing the preliminary processing of the user's drawing intention text, preparing for subsequent analysis.

[0058] 2.2. Input the preliminarily processed text into the lexical analyzer. The lexical analyzer divides the continuous text into independent words according to the language rules and vocabulary library. This can provide a clearer understanding of the basic composition structure of the text, which is helpful for subsequent analysis and understanding.

[0059] 2.3. Use the named entity recognition algorithm to process the text after word segmentation:

[0060] The named entity recognition algorithm is based on pre-trained patterns and semantic knowledge. Through pattern matching and semantic understanding, it identifies nouns representing core objects in the text (such as people, animals, objects, etc.) and determines these key entities in the text.

[0061] 2.4. Input the text into the sentiment analysis model. The model determines whether the emotion expressed by the user is joy, sadness, tranquility or other specific emotions by analyzing vocabulary, sentence structure and context, so as to determine the overall atmosphere and emotional expression of the image.

[0062] 2.5. Use text classification and keyword extraction algorithms to conduct in-depth mining of the text, extract detailed descriptions of the image (such as color (green), texture, shape and other features), and determine key information elements by counting word frequency, part of speech and context relevance.

[0063] During the entire feature extraction process, grammatical and semantic analysis is continuously performed. Consider whether the collocation between words is reasonable, whether the structure of the sentence conforms to normal expression, and whether the logic of the context is coherent. By integrating this information, the user's intention is understood more accurately, ensuring that the extracted elements fully and accurately reflect the user's drawing needs.

[0064] 3. Tips Enhancement and Optimization

[0065] Based on the deep learning neural network model, the meaning of elements and the relationship between elements are understood, and based on the generative adversarial network, the user's drawing intention is expanded and supplemented to generate drawing prompts in multiple different directions.

[0066] like Figure 3 As shown, it specifically includes the following contents:

[0067] 3.1. Input the elements extracted from the second part into the neural network model based on deep learning to conduct deep learning and understanding of the elements, and understand the meaning of the elements and the relationship between the elements;

[0068] Elements are usually represented in the form of numerical vectors, and the model passes these vectors to the hidden layer; in the hidden layer, complex mathematical operations and changes are performed on the element vectors through the connections and weights between neurons; the model uses its previous knowledge and patterns acquired in learning a large amount of drawing-related data to compare the input elements with similar patterns that have been learned, trying to understand the potential associations between the elements; through multiple forward propagation and backpropagation processes, the connection weights between neurons are continuously adjusted to more accurately understand the meaning of the input elements and the relationship between them.

[0069] 3.2. The extracted elements are input into the generative adversarial network (GAN) to expand and supplement the intention and generate multiple extended and supplemented versions of the user's drawing intention.

[0070] The generator in the generative adversarial network preliminarily generates multiple extended and supplementary versions of the user's drawing intention based on the input elements and learned knowledge; the discriminator in the generative adversarial network evaluates whether the generated results are true and reasonable. Through adversarial training between the two, the generated content is continuously optimized, and finally multiple extended and supplementary versions of the user's drawing intention are generated to achieve divergent improvement.

[0071] 3.3. Use the attention mechanism to focus on new objects, emotional expressions, key details and other elements for targeted enhancement.

[0072] 3.4. With the help of transfer learning, the experience gained from other similar tasks can be applied to the current drawing prompt optimization to improve the improvement and enhancement effects.

[0073] 3.5. Use technologies such as variational autoencoder (VAE) to explore the generation of drawing prompts in various directions, providing rich and diverse possibilities for subsequent drawing.

[0074] 4. Drawing generation and filtering

[0075] Based on the drawing prompt words generated in the third part, the diffusion model image generation algorithm is used to generate images that meet the description of the drawing prompt words, and by introducing an image quality evaluation system, images of various styles that meet the expected quality standards are obtained.

[0076] like Figure 4 As shown, it specifically includes the following contents:

[0077] 4.1. Using a variety of drawing prompt words, call the diffusion model image generation algorithm to gradually generate images that meet the description of the drawing prompt words.

[0078] (1) The diffusion model creates a random noise image as the initial input, and continuously iterates during the forward diffusion process, gradually adding Gaussian noise to the initial noise image, making the image gradually more disordered and random.

[0079] (2) The diffusion model learns to gradually restore a clear image from a noisy image. When forming a new image, the given drawing prompt words are used as conditional input. The diffusion model predicts how to remove noise based on these prompt words and the current noisy image, and gradually restores the structure and content of the image.

[0080] (3) During the back diffusion process, the diffusion model gradually removes noise, and the model predicts the adjustments that should be made to the pixel values ​​so that the image gradually becomes clearer and more meaningful.

[0081] (4) After multiple iterations of back diffusion, the image gradually becomes clear and forms an image with specific content and features, and finally generates an image that meets the description of the drawing prompt word.

[0082] 4.2. Introduce an image quality assessment system to score the visual quality of images that meet the description of the drawing prompt words from multiple dimensions (such as clarity, composition rationality, color coordination, etc.).

[0083] 4.3. Set a threshold for the quality score, filter out images with a quality score lower than the threshold, and retain images with a quality score higher than the threshold.

[0084] 4.4. Repeat the above generation and screening process until images of various styles that meet the expected quality standards are obtained.

[0085] 5. Image Return and User Guidance in Generation

[0086] The image that meets the expected quality standard is returned to the user, and the user is guided to further generate a new image based on this; the steps are repeated until the user receives a satisfactory image.

[0087] Specifically, the images that meet the quality requirements selected in the fourth part are returned to the user, and these images are displayed to the user, and friendly guidance is provided, such as asking the user how they feel about the image, whether they have further modification requirements or new ideas. If the user wants to further generate a new image based on the existing image, the operation of collecting the user's drawing intention in the first part is returned again. This cycle continues until the user is satisfied with the received image, and the whole process ends.

[0088] In this embodiment, a system for generating a controllable high-quality AI painting picture description is provided. The system can implement the above-mentioned method, and the system includes:

[0089] (1) Intention collection module: collects user drawing intentions through guidance or direct description by users.

[0090] (2) Element extraction module: Use natural language processing technology to analyze and identify the user's drawing intention and extract key drawing elements.

[0091] (3) Prompt enhancement and optimization module: Based on the deep learning neural network model, it understands the meaning of elements and the relationship between elements, and based on the generative adversarial network, it expands and supplements the user's drawing intention to generate drawing prompts in various directions.

[0092] (4) Drawing generation and screening module: Based on drawing prompt words, the diffusion model image generation algorithm is used to generate images that meet the description of the drawing prompt words, and by introducing an image quality assessment system, images of various styles that meet the expected quality standards are obtained.

[0093] (5) Image return and user-guided regeneration module: Return images that meet the expected quality standards to the user, and guide the user to further generate new images based on this; repeat the above modules until the user receives a satisfactory image.

[0094] By adopting the above technical solution disclosed in the present invention, the following beneficial effects are obtained:

[0095] The present invention provides a method and system for generating controllable high-quality AI painting picture descriptions. The present invention can accurately understand the user's drawing intentions, even if they are not precise enough, so as to generate high-quality images that meet their personalized and innovative needs, and no longer be stereotyped. The present invention can guide users to think divergently, stimulate creativity, achieve effective interaction, make full use of feedback optimization results, greatly improve image quality, and increase the diversity and flexibility of generation to adapt to various needs and scenarios.

[0096] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be considered as the scope of protection of the present invention.

Claims

1. A method for generating a controllable high-quality AI painting picture description, characterized by: The following steps are included: S1. Element extraction: Use natural language processing technology to analyze and identify the user's drawing intention and extract the key drawing elements; S2, prompt enhancement optimization: Based on the deep learning neural network model, the meaning of the elements and the relationship between the elements are understood, and based on the generative adversarial network, the user's drawing intention is expanded and supplemented to generate a variety of drawing prompt words in different directions; S3. Drawing generation and screening: Based on drawing prompt words, the diffusion model image generation algorithm is used to generate images that meet the description of the drawing prompt words, and by introducing an image quality evaluation system, images of various styles that meet the expected quality standards are obtained.

2. The method for generating a controllable high-quality AI painting picture description according to claim 1, characterized in that: Step S1 specifically includes the following contents: S11. Preprocess the text using the preprocessing technology in the natural language processing technology, and correct spelling errors in the text through the spelling check tool and dictionary matching tool in the natural language processing technology, so that the text is clearer and more accurate, and the preliminary processing of the user's drawing intention text is achieved; S12, inputting the preliminarily processed text into a lexical analyzer, which divides the continuous text into independent words according to the rules and vocabulary of the language; S13, Named Entity Recognition algorithm is based on pre-trained patterns and semantic knowledge, through pattern matching and semantic understanding, to identify nouns representing core objects in the text and determine the key entities in the text; S14, inputting the text into a sentiment analysis model to determine the overall atmosphere and emotional expression of the image; S15. Use text classification and keyword extraction algorithms to conduct in-depth mining of the text, extract detailed descriptions of the image, and determine key information elements by counting word frequency, part of speech, and context relevance.

3. The method for generating a controllable high-quality AI painting picture description according to claim 1, characterized in that: Step S2 specifically includes the following contents: S21, inputting the extracted elements into a neural network model based on deep learning, performing deep learning and understanding of the elements, and understanding the meaning of the elements and the relationship between the elements; S22, inputting the extracted elements into a generative adversarial network to expand and supplement the intention, and generating multiple expanded and supplemented versions of the user's drawing intention; S23, use the attention mechanism to focus on new objects, emotional expressions and key details; S24. With the help of transfer learning, the experience gained from other similar tasks is applied to the current drawing prompt optimization; S25. Use variational autoencoding technology to explore the generation of drawing prompts in various directions.

4. The method for generating a controllable high-quality AI painting picture description according to claim 3, characterized in that: Specifically, step S21 is that the deep learning neural network performs mathematical operations and changes on the elements represented by numerical vectors through the connections and weights between neurons; the model uses the knowledge and patterns it has previously acquired from learning a large amount of drawing-related data to compare the input elements with similar patterns that have been learned, and tries to understand the potential associations between the elements; through multiple forward propagation and backpropagation processes, the link weights between neurons are continuously adjusted to accurately understand the meaning of the elements and the relationship between the elements.

5. The method for generating a controllable high-quality AI painting picture description according to claim 3, characterized in that: Specifically, step S22 is that the generator in the generative adversarial network preliminarily generates multiple extended and supplementary versions of the user's drawing intention based on the input elements and learned knowledge; the discriminator in the generative adversarial network evaluates whether the generated results are true and reasonable, and through adversarial training between the two, continuously optimizes the generated content, and finally generates multiple extended and supplementary versions of the user's drawing intention.

6. The method for generating a controllable high-quality AI painting picture description according to claim 1, characterized in that: Step S3 specifically includes the following contents: S31, using a variety of drawing prompt words, calling the diffusion model image generation algorithm to gradually generate images that meet the description of the drawing prompt words; S32. Introduce an image quality assessment system to score the visual quality of images that meet the description of the drawing prompt words from multiple dimensions; S33, setting a threshold for the quality score, filtering out images with a quality score lower than the threshold, and retaining images with a quality score higher than the threshold; S34, continuously repeating the above generation and screening process until images of various styles that meet the expected quality standards are obtained.

7. The method for generating a controllable high-quality AI painting picture description according to claim 6, characterized in that: Step S21 specifically includes the following contents: S311, the diffusion model creates a random noise image as the initial input, and iterates in the forward diffusion process to gradually add Gaussian noise to the initial noise image; S312, the diffusion model learns to gradually restore a clear image from a noisy image. When a new image is formed, the given drawing prompt word is used as a conditional input. The diffusion model predicts how to remove the noise based on the prompt word and the current noisy image, and gradually restores the structure and content of the image. S313, during the back diffusion process, the diffusion model gradually removes noise, and the model predicts the adjustments that should be made to the pixel values ​​so that the image gradually becomes clearer and more meaningful; S314, after multiple iterations of reverse diffusion, the image gradually becomes clear and forms an image with specific content and features, and finally generates an image that meets the description of the drawing prompt word.

8. The method for generating a controllable high-quality AI painting picture description according to claim 1, characterized in that: Before step S1, the method further includes: S0. Intention collection: Collect user drawing intentions through guidance or direct description by users.

9. The method for generating a controllable high-quality AI painting picture description according to claim 9, characterized in that: Step S3 also includes: S4, image return and user guidance in re-generation: return the image that meets the expected quality standard to the user, and guide the user to further generate a new image on this basis; repeat steps S0 to S4 until the user receives a satisfactory image.

10. A system for generating controllable high-quality AI painting picture descriptions, characterized by: The system can implement the method described in any one of claims 1 to 9, and the system includes: Intention collection module: collects user drawing intentions through guidance or direct description by users; Element extraction module: uses natural language processing technology to analyze and identify the user's drawing intention and extract key drawing elements; Prompt enhancement and optimization module: Based on the deep learning neural network model, it understands the meaning of elements and the relationship between elements, and based on the generative adversarial network, it expands and supplements the user's drawing intention to generate a variety of drawing prompts in different directions; Drawing generation and screening module: Based on drawing prompt words, the diffusion model image generation algorithm is used to generate images that meet the description of the drawing prompt words, and by introducing an image quality evaluation system, images of various styles that meet the expected quality standards are obtained; Image return and user-guided regeneration module: Return images that meet the expected quality standards to the user, and guide the user to further generate new images based on this; repeat the above modules until the user receives a satisfactory image.

Images