Image generation method and system based on AI identification

Abstract

The invention provides an image generation method and system based on AI identification. The method comprises the following steps: S1, obtaining text data input by a user; s2, slicing the text data content to obtain a plurality of slice contents; s3, performing identification and target prediction on the plurality of slice contents by using a Transform model; s4, constructing an image generation model and training; s5, converting the plurality of target prediction results in the step S3 into image feature vectors, and inputting the image feature vectors into an image generation model to generate a plurality of initial images; s6, fusing the plurality of initial images through an average value fusion mode; and S7, performing image restoration on the fused image to obtain a final image, converting the predicted target object and the attributes thereof into the image feature vectors, inputting the image feature vectors into the image generation model to generate a plurality of different initial images, and merging the plurality of initial images into a comprehensive image by adopting an average value fusion mode, so as to obtain the final image. The quality and consistency of images are improved, and the shielding phenomenon is avoided.

Description

Technical Field

[0001] The present invention relates to the field of image generation technology, and in particular to an image generation method and system based on AI recognition. Background Art

[0002] Image generation methods and systems based on AI recognition generally refer to the use of artificial intelligence technology to create new images or modify and enhance existing images. Such technologies and systems are used in many fields, such as artistic creation, design, entertainment, education, and scientific research.

[0003] Although current AI-generated image technology has made significant progress, it still has certain limitations when dealing with complex scenes containing multiple targets: when AI image generation technology is applied to image compression, the image compression algorithm is used to guide the learning process of the model so that the compressed image has a smaller file size while maintaining as high quality as possible. Traditional image compression algorithms use discrete cosine transform (DCT) to remove redundant information in the image. Although this method is effective, it is prone to visible artifacts at high compression ratios. In the process of generating images, even if the target can be accurately identified, when multiple targets appear in the same scene, occlusion may occur between them, and the positions and relationships of the targets cannot be correctly arranged. It is still a challenge to accurately place these targets in the generated image.

[0004] Therefore, it is necessary to provide a new image generation method and system based on AI recognition to solve the above technical problems. Summary of the invention

[0005] In order to solve the technical problem of how to generate a high-quality image from a given text description and ensure that multiple targets in the image are accurately identified, the present invention provides an image generation method based on AI recognition, comprising the following steps:

[0006] S1. Obtain user input text data;

[0007] S2, slicing the text data content to obtain multiple slice contents;

[0008] S3, using the Transformer model to identify and predict the contents of multiple slices respectively;

[0009] S4. Build an image generation model and train it: collect training data sets and test data sets, determine the encoding module of the model, which is used to convert the training data sets into a compressed bit stream, set the rate-distortion optimization function and optimize the model by adjusting the weight between the compression rate and the distortion. At the same time, add the GAN loss to the optimization model process and combine it with the rate-distortion optimization function to complete the entire model training process.

[0010] S5, converting the multiple target prediction results of step S3 into image feature vectors and inputting them into the image generation model to generate multiple initial images;

[0011] S6, fusing the multiple initial images by using an average value fusion method;

[0012] S7. Perform image restoration on the fused image to obtain a final image.

[0013] Furthermore, the step S3 of using the Transformer model for identification and target prediction includes the following steps:

[0014] Convert text into vectors using word embeddings;

[0015] Add position information to each position in the vector sequence;

[0016] Load the pre-trained model;

[0017] Encode and decode sequences via self-attention mechanism and feed-forward neural network;

[0018] Pass the encoded input to the pre-trained model and get the output. Find the predicted category from the pre-trained model output.

[0019] Furthermore, the slicing process may be performed based on a fixed length of characters or the number of words or clear separators in the text.

[0020] Furthermore, each image in the training data set is scaled to a corresponding ratio according to its own situation, and randomly cropped to a size of 256×256 pixels.

[0021] Furthermore, the text data is any combination of text, pictures, videos and files.

[0022] Furthermore, the encoding module encoding comprises the following steps:

[0023] Training dataset image I t First, we transform g through nonlinear analysis. a Get the latent feature y;

[0024] y=g a (x; φ)

[0025] The latent features are then compressed using arithmetic coding to form a binary code stream. In order to reconstruct the compressed image, the discrete representation is reused to obtain a continuous value vector using arithmetic decoding. Then input the nonlinear synthetic transformation model g s Transform it back to the data space to get the reconstructed image

[0026]

[0027] Then, through the perceptual transformation The training dataset images and reconstructed images are converted to the perceptual space, and the loss generated during the encoding and decoding process is calculated in the perceptual space.

[0028] Furthermore, the loss includes adversarial loss, reconstruction loss and classification loss.

[0029] Furthermore, the image restoration includes image color correction and sharpening processing.

[0030] Another aspect of the present invention provides an AI recognition-based image generation system, which is used to execute the above-mentioned AI recognition-based image generation method, including: a data receiving module, which is used to receive text data input by a user;

[0031] A slicing module, used for slicing the text data content received by the data receiving module to obtain multiple slice contents;

[0032] The recognition module is used to recognize and predict the target of the slice content output by the slice module;

[0033] An image generation module, used to generate an initial image according to the predicted target of the recognition module;

[0034] Multi-image fusion module, used to fuse multiple initial images;

[0035] The image optimization module is used to perform image restoration on the fused image to obtain the final image.

[0036] Compared with the related art, the image generation method and system based on AI recognition provided by the present invention have the following beneficial effects:

[0037] 1. The present invention converts the predicted target object and its attributes into image feature vectors and inputs them into a trained image generation model to generate multiple different initial images. The pixel values ​​of the multiple initial images are averaged and merged into a comprehensive image using an average value fusion method to improve the quality and consistency of the image and avoid occlusion.

[0038] 2. The present invention utilizes GAN loss combined with rate-distortion optimization function to find an optimal balance between compression rate and image distortion. The degree of compression can be adjusted according to specific needs, rather than just relying on a fixed compression ratio to improve the quality of the compressed image. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A structural block diagram of the image generation system based on AI recognition provided by the present invention;

[0040] Figure 2 A flowchart of the image generation method based on AI recognition provided by the present invention;

[0041] Figure 3 A flowchart of target prediction provided by the present invention. DETAILED DESCRIPTION

[0042] The present invention will be further described below in conjunction with the accompanying drawings and implementation modes.

[0043] Please refer to Figure 1 , Figure 2 , Figure 3 ,in, Figure 1 A structural block diagram of the image generation system based on AI recognition provided by the present invention; Figure 2 A flowchart of the image generation method based on AI recognition provided by the present invention; Figure 3 A flowchart of target prediction provided by the present invention.

[0044] Embodiment 1

[0045] In the specific implementation process, refer to Figure 1 As shown, an image generation system based on AI recognition provided by the present invention includes: a data receiving module, which is used to receive text data input by a user, and the receiving method can be divided into web pages, mobile applications and desktop applications:

[0046] Web pages: Use HTML forms and JavaScript to capture user input and process this data through backend services (such as Node.js, Django, Flask, etc.);

[0047] Mobile apps: Use Swift, Objective-C (iOS) or Java, Kotlin (Android) to write interfaces and logic to handle user input on the mobile iOS or Android platforms;

[0048] Desktop Applications: Use frameworks such as Qt, Tkinter (Python), and Swing (Java) to create graphical user interfaces (GUIs) to receive text input from users.

[0049] A slicing module, used for slicing the text data content received by the data receiving module to obtain multiple slice contents;

[0050] The recognition module is used to recognize and predict the target of the slice content output by the slice module;

[0051] An image generation module, used to generate an initial image according to the predicted target of the recognition module;

[0052] Multi-image fusion module, used to fuse multiple initial images;

[0053] The image optimization module is used to perform image restoration on the fused image to obtain the final image.

[0054] Embodiment 2

[0055] In the specific implementation process, refer to Figure 2-Figure 3 As shown, the image generation method based on AI recognition provided by the present invention includes the following steps:

[0056] S1. Obtain user input text data;

[0057] S2, slicing the text data content to obtain a plurality of slice contents, wherein the slicing process can be performed according to a fixed length of characters or the number of words or a clear separator in the text to divide and slice;

[0058] Among them, fixed-length slicing: use Python's slicing operation or loop through the text, intercepting characters of a fixed length each time;

[0059] Word-based slicing: Use regular expressions or a natural language processing library such as NLTK to identify word boundaries and slice based on the number of words;

[0060] Slicing based on delimiters: If there are clear delimiters in the text (such as commas, periods, newlines, etc.), you can use these delimiters to split the text.

[0061] S3, using the Transformer model to identify and predict the contents of multiple slices respectively;

[0062] S4. Build an image generation model and train it: collect training data sets and test data sets, determine the encoding module of the model, which is used to convert the training data sets into a compressed bit stream, set the rate-distortion optimization function and optimize the model by adjusting the weight between the compression rate and the distortion. At the same time, add the GAN loss to the optimization model process and combine it with the rate-distortion optimization function to complete the entire model training process.

[0063] It should be noted that the training dataset is the COCO training dataset, which contains 118,282 images of different resolutions. In order to reduce the impact of the input image size on model training, each image will be scaled to the corresponding ratio and randomly cropped to 256×256 pixels before training begins. In addition, each sample will be normalized before entering the network:

[0064] The pixel value is normalized to the range [-1,1] by (x-127.5) / 127.5.

[0065] The test training set uses the Kodak dataset, which consists of 24 uncompressed PNG color images with a resolution of 768×512. These images contain a variety of different scenes and are suitable for evaluating the performance of the model under different image characteristics.

[0066] S5, converting the multiple target prediction results of step S3 into image feature vectors and inputting them into the image generation model to generate multiple initial images;

[0067] S6, fusing the multiple initial images by using an average value fusion method, that is, taking the average value of the pixel values ​​at corresponding positions of two or more images;

[0068] S7. Perform image restoration on the fused image to obtain a final image. The image restoration includes image color correction and sharpening processing.

[0069] It should be noted that the recognition and target prediction using the Transformer model in step S3 includes the following steps:

[0070] Convert text into vectors using word embeddings;

[0071] Add position information to each position in the vector sequence;

[0072] Load the pre-trained model;

[0073] Encode and decode sequences via self-attention mechanism and feed-forward neural network;

[0074] Pass the encoded input to the pre-trained model and get the output. Find the predicted category from the pre-trained model output.

[0075] Furthermore, the text data is any combination of text, pictures, videos and files.

[0076] Furthermore, the encoding module encoding includes the following steps:

[0077] Training dataset image I t First, we transform g through nonlinear analysis. a Get the latent feature y;

[0078] y=g a (x; φ)

[0079] The latent features are then compressed using arithmetic coding to form a binary code stream. In order to reconstruct the compressed image, the discrete representation is reused to obtain a continuous value vector using arithmetic decoding. Then input the nonlinear synthetic transformation model g s Transform it back to the data space to get the reconstructed image

[0080]

[0081] Then, through the perceptual transformation The training dataset images and reconstructed images are converted to the perceptual space, and the loss generated during the encoding and decoding process is calculated in the perceptual space.

[0082] Among them, the rate distortion optimization function

[0083] The losses include adversarial loss, reconstruction loss and classification loss:

[0084] 1. Fight against losses:

[0085]

[0086] Among them, G and D represent the generator and discriminator respectively, E is the mathematical expectation, and I s is the image of the training data set. When the image of the training data set is a positive sample, When the images in the training dataset are negative samples, P t The target posture.

[0087] 2. Reconstruction loss: Calculate the distance between the generated image and the training dataset image

[0088]

[0089] Among them, L rec To reconstruct the loss, I g To generate an image, I t is the training dataset image.

[0090] 3. Classification loss. Adding classification loss to the discriminator encourages the generator to reconstruct images of the same identity with fewer artifacts, using cross entropy loss:

[0091]

[0092] Among them, L id is the classification loss, p(I g |I t ) is used to generate the same prediction probability of an image as the image in the training dataset. According to the above total loss optimization model, an optimal balance between compression rate and image distortion can be found. The degree of compression can be adjusted according to specific needs, rather than just relying on a fixed compression ratio to improve the quality of the compressed image.

[0093] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0094] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and the present invention can be implemented in other specific forms without departing from the spirit or basic features of the present invention. Therefore, no matter from which point of view, the embodiments should be regarded as exemplary and non-restrictive. The scope of the present invention is defined by the attached claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention, and any figure mark in the claims should not be regarded as limiting the claims involved.

[0095] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. An image generation method based on AI recognition, characterized in that: The steps include: S1. Obtain user input text data; S2, slicing the text data content to obtain multiple slice contents; S3, using the Transformer model to identify and predict the contents of multiple slices respectively; S4. Build an image generation model and train it: collect training data sets and test data sets, determine the encoding module of the model, which is used to convert the training data sets into a compressed bit stream, set the rate-distortion optimization function and optimize the model by adjusting the weight between the compression rate and the distortion. At the same time, add the GAN loss to the optimization model process and combine it with the rate-distortion optimization function to complete the entire model training process. S5, converting the multiple target prediction results of step S3 into image feature vectors and inputting them into the image generation model to generate multiple initial images; S6, fusing the multiple initial images by using an average value fusion method; S7. Perform image restoration on the fused image to obtain a final image.

2. The image generation method based on AI recognition according to claim 1, characterized in that: The step S3 of using the Transformer model for identification and target prediction includes the following steps: Convert text into vectors using word embeddings; Add position information to each position in the vector sequence; Load the pre-trained model; Encode and decode sequences via self-attention mechanism and feed-forward neural network; Pass the encoded input to the pre-trained model and get the output. Find the predicted category from the pre-trained model output.

3. The image generation method based on AI recognition according to claim 1, characterized in that: The slicing process can be performed based on a fixed length of characters or the number of words or clear delimiters in the text to divide the text into slices.

4. The image generation method based on AI recognition according to claim 1, characterized in that: Each image in the training data set is scaled to a corresponding ratio according to its own situation and randomly cropped to a size of 256×256 pixels.

5. The image generation method based on AI recognition according to claim 1, characterized in that: The text data is any combination of text, pictures, videos and files.

6. The image generation method based on AI recognition according to claim 1, characterized in that: The encoding module encoding comprises the following steps: The training dataset images are first transformed through nonlinear analysis to obtain latent features; The latent features are then compressed using arithmetic coding to form a binary code stream. In order to reconstruct the compressed image, the discrete representation is re-decoded using arithmetic decoding to obtain a continuous value vector, which is then input into a nonlinear synthetic transformation model to transform it back to the data space to obtain the reconstructed image. Then, through perceptual transformation, the training dataset images and reconstructed images are converted to the perceptual space, and the loss generated during the encoding and decoding process is calculated in the perceptual space.

7. The image generation method based on AI recognition according to claim 6, characterized in that: The losses include adversarial loss, reconstruction loss and classification loss.

8. The image generation method based on AI recognition according to claim 1, characterized in that: The image restoration includes image color correction and sharpening processing.

9. An image generation system based on AI recognition, used to execute the image generation method based on AI recognition according to any one of claims 1 to 8, characterized in that: include: A data receiving module, used for receiving text data input by a user; A slicing module, used for slicing the text data content received by the data receiving module to obtain multiple slice contents; The recognition module is used to recognize and predict the target of the slice content output by the slice module; An image generation module, used to generate an initial image according to the predicted target of the recognition module; Multi-image fusion module, used to fuse multiple initial images; The image optimization module is used to perform image restoration on the fused image to obtain the final image.

Images