Commercial shooting-oriented AI image generation method

Abstract

The invention discloses a commercial shooting-oriented AI image generation method, and belongs to the field of commercial application of an artificial intelligence information technology. The method mainly comprises the following steps: S1, acquiring a model portrait picture; s2, portrait data preprocessing; s3, carrying out fine adjustment by using lora based on a table diffusion pre-training model; s4, generating prompt words required by the commercial shooting model; s5, generating a commercial image by using the fine-tuned model; s6, local redrawing is carried out on the commercial image generated in the step S5, and the clothes and accessories of the model are regenerated by using the table diffusion; and S7, repeatedly executing the step S6 to obtain a plurality of commercial pictures. According to the AI image generation method for commercial shooting, commercial shooting images required in different scenes can be provided according to commercial shooting requirements. According to the invention, a new way is provided for commercial shooting, a large number of commercial shooting images can be generated, the labor cost is effectively reduced, and the intelligence of commercial shooting is promoted.

Description

Technical Field

[0001] The present invention relates to the fields of computer technology and artificial intelligence, and in particular to the commercial application of artificial intelligence information technology, and specifically to an AI image generation method for commercial photography. Background Art

[0002] With the rapid development of the commercial world, visual communication plays an increasingly important role in business activities. Commercial photography, as a key means of visual communication, has therefore attracted widespread attention and application. Through creative and engaging imagery, commercial photography can effectively convey commercial information, increasing consumer awareness and purchasing intent. However, current commercial photography technology still has some challenges. For example, pre-shooting scenes require time-consuming outfit changes and poses for models, resulting in poor results, a lack of creativity, and high costs. This has hindered the application and development of commercial photography technology. Therefore, there is an urgent need for a low-cost, efficient, creative, and high-quality intelligent commercial photography solution. Summary of the Invention

[0003] To overcome the shortcomings of the existing technology, the present invention aims to provide an AI image generation method for commercial photography. Combining the latest Wensheng image technology, fine-tuning techniques are applied to generate a model face in a specified style. Local redrawing is used to replace the model's clothing and accessories. Image super-resolution technology is used to output high-quality commercial photography images. To achieve the above objectives, the present invention provides the following technical solution, which includes the following steps:

[0004] Step S1: Collecting model portrait photos. Use a camera or obtain portrait photos from the Internet.

[0005] Step S2: Portrait data preprocessing. Apply central cropping to the portrait images from step S1 and select the cropped images as training data. After image selection, use the DeepBooru algorithm to generate text labels for each image.

[0006] Step S3: Based on the StableDiffusion pre-trained model, apply Lora for fine-tuning. Add a parallel bypass to the original StableDiffusion model, with the original StableDiffusion model on the left and the introduced dimension reduction matrix A and dimension increase matrix B on the right. Input the image data and labels from step 2 for fine-tuning. After fine-tuning is completed, save the Lora weight file.

[0007] Step S4: Generate prompt words required for commercial model photography. The prompt words include positive prompt words and reverse prompt words. The NLLB-200 model is used to convert the input Chinese prompt words into English.

[0008] Step S5: Generate commercial images using the fine-tuned model. Use stable diffusion, load the fine-tuned weights, and input the prompt words to initially generate commercial images.

[0009] Step S6: Redrawing Partially the Commercial Image. The commercial image generated in step S5 is selected and partially redrawn to generate images of different accessories and clothing. Finally, image super-resolution is performed to output a high-quality commercial image.

[0010] Step S7: Repeat step S6 to obtain multiple commercial photos.

[0011] Preferably, the lora fine-tuning in step S3 is specifically to add a parallel bypass to the original stable diffusion model, that is, the left side is the original stable diffusion model, and the right side is the introduced dimensionality reduction matrix A and dimensionality increase matrix B. Select chilloutmix-Ni.safetensors or majicmixRealistic_v7.safetensors as the pre-trained weights of the left model. The weight file can be obtained from the huggingface official website. Use the pre-trained weights to initialize stable diffusion and freeze the weights, that is, these parameters are not updated during the fine-tuning process. The right bypass initializes the dimensionality reduction matrix A with a random Gaussian distribution and initializes the dimensionality increase matrix B with a 0 matrix to ensure that the output of the model is consistent with the original output at the beginning of training. Assume that the weight of the original stable diffusion model is W0, and this value does not change during training. ΔW is the parameter that needs to be updated during fine-tuning. Only the parameters in A and B are trained, which is expressed by the formula:

[0012] W0+ΔW=W0+BA

[0013] Input the training data and labels in step S2 and use Lora to fine-tune. After fine-tuning, the output of the model (the fine-tuning process is called forward propagation) can be expressed as:

[0014] h=W0x+ΔWx=W0x+BAx

[0015] Where x is the model input and h is the model output.

[0016] Preferably, the steps for generating the positive prompt words in step S4 are as follows:

[0017] S4-1: The user provides Chinese prompts based on the needs of the actual commercial auction. The Chinese prompts are described in phrases, and different phrases are separated by commas.

[0018] S4-2: Convert Chinese phrases into English phrases. The NLLB-200 model trained by Meta is used for the conversion.

[0019] S4-3: Organize the generated English prompt words and input them into the fine-tuned stable diffusion model to generate images.

[0020] S4-4: Use the DeepBooru algorithm to generate the corresponding image description for the image generated in the previous step, that is, to reversely infer the prompt word from the image.

[0021] S4-5: Supplement and filter the above prompt words, sort them according to character, clothing, face, background, style, and image quality, and obtain the final positive prompt words.

[0022] Reverse prompt words can be described in order from picture quality, the whole person to the details, and from head to toe, and are obtained by converting Chinese into English using the NLLB-200 model.

[0023] Preferably, the steps for generating the commercial image in step S5 are as follows:

[0024] S5-1: Initialize the stablediffusion model, load the majicmixRealistic_v7.safetensors weights and the fine-tuned weights.

[0025] S5-2: Enter the forward prompt word and reverse prompt word, and adjust the relevant configuration parameters of stablediffsuion.

[0026] S5-3: Apply the ADetailer plug-in to perform face and hand repair on the image during the generation process. After the repair is completed, the image is output.

[0027] Preferably, the specific steps of step S6 are as follows:

[0028] S6-1: Repeat step S5 to select pictures that meet the requirements and have good quality.

[0029] S6-2: Input the selected image into the stable diffusion network and select the parts of the image that need to be redrawn, such as the model's clothes, accessories, etc.

[0030] S6-3: Input related prompt words for clothing and accessories, and use the NLLB-200 model to convert Chinese and English. Once completed, input the prompt words into the model to change the clothing and accessories of the model in the original image.

[0031] S6-4: Repeat step S6-3, select the images that meet the requirements and send them to the R-ESRGAN 4x+ network for image super-resolution to generate high-quality, high-resolution, and high-definition images.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] This invention provides an AI-powered image generation method for commercial photography, building a system for intelligently generating product images. Based on user-entered prompts, this system automatically generates efficient, creative, and high-quality images, effectively reducing labor and photography costs. Furthermore, there are currently no publicly available methods or systems for automatically generating commercial photography images using AI. This invention provides a novel approach for commercial photography. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 Flowchart for the implementation of the present invention;

[0035] Figure 2 This is a schematic diagram of the LoRa fine-tuning principle of the present invention. DETAILED DESCRIPTION

[0036] Some knowledge is not explained in this specification and belongs to the prior art known to those skilled in the art and is easy to understand. The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings in the embodiments of the present invention. The present invention relates to an AI image generation method for commercial photography, which specifically includes the following steps:

[0037] Step S1: Collecting portrait photos of the model. Use a camera or obtain portrait photos from the Internet. The face in the photo must be clear and unobstructed.

[0038] Step S2: Portrait Data Preprocessing. Crop the portrait image from step S1 to ensure the face fills the entire image. Select approximately 20-30 cropped images with a minimum side of 512 pixels to serve as training data. After selecting the images, use the DeepBooru algorithm to generate text labels (.txt files) for each image. These serve as annotations for the training data images. Users can modify the content of the annotation files as appropriate.

[0039] Step S3: Based on the StableDiffusion pre-trained model, apply Lora for fine-tuning. Lora is a low-resource method for fine-tuning large models. It freezes the model's original weights and introduces a dimensionality reduction matrix and a dimensionality increase matrix in parallel. This reduces the number of parameters required for model fine-tuning and extracts features useful for downstream tasks.

[0040] Add a parallel bypass to the original stable diffusion model, such as Figure 2 As shown, the left side is the original stablediffusion model, and the right side is the introduced dimensionality reduction matrix A and dimensionality increase matrix B. Select majicmixRealistic_v7.safetensors as the pre-trained weights of the left model. The weight file can be obtained from the huggingface official website. Use the pre-trained weights to initialize stablediffusion and freeze the weights, that is, these parameters are not updated during fine-tuning. The right bypass initializes the dimensionality reduction matrix A with a random Gaussian distribution and the dimensionality increase matrix B with a 0 matrix to ensure that the output of the model is consistent with the original output at the beginning of training. Assume that the weight of the original stablediffusion model is W0, and this value does not change during training. ΔW is the parameter that needs to be updated during fine-tuning. Only the parameters in A and B are trained, which can be expressed as follows:

[0041] W0+ΔW=W0+BA

[0042] Input the training data and labels in step S2 and use Lora to fine-tune. After fine-tuning, the output of the model (the fine-tuning process is called forward propagation) can be expressed as:

[0043] h=W0x+ΔWx=W0x+BAx

[0044] Where x is the model input and h is the model output.

[0045] After finishing fine-tuning, save the lora weight file.

[0046] Step S4: Generate prompt words required for commercial model photography. Prompt words include positive prompt words and negative prompt words. Positive prompt words are generated using the following steps:

[0047] S4-1: The user provides Chinese prompts based on the needs of the actual commercial auction. The Chinese prompts are described in phrases, and different phrases are separated by commas.

[0048] S4-2: Convert Chinese phrases into English phrases. The NLLB-200 model trained by Meta is used for the conversion.

[0049] S4-3: Organize the generated English prompt words and input them into the fine-tuned stable diffusion model to generate images.

[0050] S4-4: Use the DeepBooru algorithm to generate the corresponding image description for the image generated in the previous step, that is, to reversely infer the prompt word from the image.

[0051] S4-5: Supplement and filter the above prompt words, sort them according to character, clothing, face, background, style, and image quality, and obtain the final positive prompt words.

[0052] Reverse prompt words can be described in order from picture quality, the whole person to the details, and from head to toe, and are obtained by converting Chinese into English using the NLLB-200 model.

[0053] Step S5: Generate commercial images using the fine-tuned model.

[0054] S5-1: Initialize the stablediffusion model, load the majicmixRealistic_v7.safetensors weights and the fine-tuned weights.

[0055] S5-2: Enter the forward and reverse prompt words and adjust the relevant configuration parameters of stablediffsuion.

[0056] S5-3: Apply the ADetailer plug-in to perform face and hand repair on the image during the generation process. After the repair is completed, the image is output.

[0057] Step S6: Redraw the commercial image partially. The process is as follows:

[0058] S6-1: Repeat step S5 to select pictures that meet the requirements and have good quality.

[0059] S6-2: Input the selected image into the stable diffusion network and select the parts of the image that need to be redrawn, such as the model's clothes, accessories, etc.

[0060] S6-3: Input related prompt words for clothing and accessories, and use the NLLB-200 model to convert Chinese and English. Once completed, input the prompt words into the model to change the clothing and accessories of the model in the original image.

[0061] S6-4: Repeat step S6-3, select the images that meet the requirements and send them to the R-ESRGAN 4x+ network for image super-resolution to generate high-quality, high-resolution, and high-definition images.

[0062] Step S7: Repeat step S6 to obtain multiple commercial photos.

Claims

1. An AI image generation method for commercial photography, characterized by The method comprises the following steps: Step S1: Use a camera to take a portrait picture or obtain a portrait picture from the Internet; Step S2: Apply center cropping to the portrait images in step S1, select the cropped images as the training data, and then use the DeepBooru algorithm to generate text labels for each image; Step S3: Based on the StableDiffusion pre-trained model, fine-tune it with Lora. Add a parallel bypass to the original StableDiffusion model. The left side is the original StableDiffusion model, and the right side is the introduced dimensionality reduction matrix A and dimensionality increase matrix B. Input the data to be trained and the text label in step 2 for fine-tuning. After fine-tuning is completed, save the Lora weight file. Step S4: Generate the positive prompt words and negative prompt words required for the commercial photo; Step S5: Generate commercial images using the fine-tuned model; Step S6: Select the commercial image generated in step S5, redraw the local content of the image generated in step S5, generate pictures of different accessories and clothing, and finally perform image super-resolution to output high-quality commercial pictures. Step S7: Repeat step S6 to obtain multiple commercial photos.

2. The AI image generation method for commercial photography according to claim 1, characterized in that: The lora fine-tuning in step S3 is specifically as follows: Add a parallel bypass to the original stable diffusion model, that is, the original stable diffusion model is on the left, and the introduced dimensionality reduction matrix A and dimensionality increase matrix B are on the right. Select chilloutmix-Ni.safetensors or majicmixRealistic_v7.safetensors as the pre-trained weights of the left model, use the pre-trained weights to initialize stable diffusion, freeze the weights, and use the random Gaussian distribution to initialize the dimensionality reduction matrix A and the dimensionality increase matrix B with a 0 matrix in the right bypass. The weight of the original stable diffusion model is W0, which does not change during training. ΔW is the parameter that needs to be updated during fine-tuning. Only the parameters in A and B are trained, which can be expressed as follows: W0+ΔW=W0+BA Input the training data and labels in step S2, use Lora to fine-tune, and after fine-tuning, the output of the model (the fine-tuning process is called forward propagation) can be expressed as follows: h=W0x+ΔWx=W0x+BAx Where x is the model input and h is the model output.

3. The AI image generation method for commercial photography according to claim 1, characterized in that: The steps for generating positive prompt words in step S4 are as follows: S4-1: The user provides Chinese prompt words based on the needs of commercial auctions in actual scenarios. The Chinese prompt words are described in phrases, and different phrases are separated by commas; S4-2: Convert Chinese phrases into English phrases using the NLLB-200 model trained with Meta. S4-3: Organize the generated English prompt words and input them into the fine-tuned stable diffusion model to generate images; S4-4: Use the DeepBooru algorithm to generate the corresponding image description for the image generated in S4-3, that is, to reversely infer the prompt word from the image; S4-5: Supplement and filter the above prompt words, sort them according to character, clothing, face, background, style, and image quality, and obtain the final positive prompt words; The reverse prompt words can be described in order of picture quality, the whole character to the parts, and from head to toe, and are obtained by converting Chinese into English using the NLLB-200 model.

4. The AI image generation method for commercial photography according to claim 1, characterized in that The steps for generating the commercial image in step S5 are as follows: S5-1: Initialize the stablediffusion model, load the majicmixRealistic_v7.safetensors weights and the fine-tuned weights; S5-2: Enter the forward prompt word and the reverse prompt word, and adjust the relevant configuration parameters of stablediffsuion; S5-3: Apply the ADetailer plug-in to perform face and hand repair on the image during the generation process. After the repair is completed, the image is output.

5. The AI image generation method for commercial photography according to claim 1, characterized in that: The specific steps of step S6 are as follows: S6-1: Repeat step S5 to select images that meet the requirements and have good quality; S6-2: Input the selected image into the stable diffusion network and select the parts of the image that need to be redrawn, such as the model's clothes and accessories; S6-3: Input prompt words related to clothing and accessories, and use the above-mentioned NLLB-200 model to convert Chinese and English. After completion, input the prompt words into the model to change the clothing and accessories of the model in the original image; S6-4: Repeat step S6-3, select the images that meet the requirements and send them to the R-ESRGAN 4x+ network for image super-resolution to generate high-quality, high-resolution, and high-definition images.

Images