Mural figure image generation method and system, computer equipment and storage medium

Abstract

The invention discloses a mural figure image generation method and system, computer equipment and a storage medium, and the method comprises the steps: constructing a mural image data set and a corpus, the corpus comprising a color feature corpus and a detail element corpus; preprocessing the wall painting image data set to obtain a plurality of image data sets for different aspects of the image; designing a mural figure image generation model based on multi-constraint LoRA; according to one type of image data set and a corresponding corpus, training a wall painting figure image generation model to obtain a trained model; and inputting the cue word and the real face image into the trained model to generate a mural figure image. According to the method, the mural figure image generation model based on the multi-constraint LoRA is introduced to guide the network model to fully learn the color, the line and the background pattern of the mural, so that the figure image with a better effect and higher quality in the mural style is generated.

Description

Technical Field

[0001] The invention belongs to the fields of image processing, computer vision and image conversion, and specifically relates to a mural character image generation method, system, computer equipment and storage medium. Background Art

[0002] At present, deep learning-based methods have been widely used to automatically generate images with specific artistic styles. These technologies are mainly applied in two directions: 1) text-to-image generation, such as generating images through text description input. 2) image-to-image generation, such as converting natural scenery images into various artistic images of different styles. The emergence of these technologies makes it possible to automatically generate mural-style character images.

[0003] Since the Generative Adversarial Networks (GAN) model was proposed, it has been widely used in many fields. Among them, the CycleGAN (from the paper "Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks") algorithm is the main application of GAN in the direction of style transfer. Compared with general convolutional neural networks, CycleGAN can achieve style transfer between different domain sets, and the generated images are more natural and realistic. However, the CycleGAN method has problems such as low image stability and difficulty in changing the shape of objects. At the same time, due to the structural characteristics of the CycleGAN model that is not a single network, the model training process is very difficult, and it is not easy to guarantee the generation of a specific image result with a one-to-one mapping.

[0004] The Diffusion model (from the paper "Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems") uses block-by-block contrast loss to generate images using the contrast loss between the sample and the original image, which is the basis of the Stable Diffusion model (from the paper "High-resolution image synthesis with latent diffusion models"). Compared with GAN, the stable diffusion model can not only generate images with corresponding styles through text control, but also has higher image quality and can retain more style features and details. At the same time, based on the overall structure of the image generated by gradually diffusing noise, its training process is more transparent. However, in further research on the generated images, it was found that the diffusion model is not sufficient to control the transfer of specific styles, is not sensitive to prompts, and the controllability of the image is greatly reduced. This impact is also fully reflected in the generation of portraits, with many errors that go against common sense, such as facial confusion and background overlap.

[0005] With the development of image generation, Low-Rank Adaptation (LoRA) (from the paper "Low-rank adaptation of large language models"), as a fine-tuning model flexibly constructed for different tasks, has achieved good results in combination with pre-trained large models. It freezes the parameters of the pre-trained model, then adds a trainable bypass matrix ΔW, fine-tunes a specific style based on the diffusion model, and generates higher-quality stylized images. It is worth mentioning that compared with a single LoRA model, a multi-LoRA model has a stronger control ability over a large model, and its ability to handle the corresponding style details is stronger than a single LoRA model. In order to generate higher-quality images, the training method of superimposing multiple LoRAs has been used for image generation tasks of different styles. However, this type of model is not optimized for the characteristics of murals. When merging multiple LoRA models, the merging process of different LoRA models is unstable, which easily leads to inconsistent picture elements and low-quality generated images. Summary of the invention

[0006] In order to address the shortcomings of the above-mentioned prior art, the present invention provides a mural character image generation method, system, computer device and storage medium, which introduces a mural character image generation model based on multi-constraint LoRA to guide the network model to fully learn the color, lines and background pattern of the mural, thereby generating a mural-style character image with better effect and higher quality.

[0007] The first object of the present invention is to provide a method for generating a mural character image.

[0008] The second object of the present invention is to provide a mural character image generation system.

[0009] A third object of the present invention is to provide a computer device.

[0010] A fourth object of the present invention is to provide a storage medium.

[0011] The first object of the present invention can be achieved by adopting the following technical solutions:

[0012] A method for generating a mural character image, the method comprising:

[0013] Constructing a mural image dataset and a corpus, wherein the corpus is related to the mural image dataset and includes a color feature corpus and a detail element corpus;

[0014] Preprocess the mural image dataset to obtain multiple image datasets targeting different aspects of the image;

[0015] Design a mural character image generation model based on multi-constrained LoRA;

[0016] According to multiple image datasets and corpora for different aspects of images, the mural character image generation model is trained to obtain a trained model;

[0017] Input the prompt words and real face images into the trained model to generate images of the characters in the mural.

[0018] Furthermore, the mural character image generation model includes a Stable Diffusion network structure and a multi-LoRA model;

[0019] The Stable Diffusion network structure includes a variational autoencoder based on an encoder-decoder architecture, a U-Net model based on a convolutional neural network, and a multimodal model based on contrastive learning. The multi-LoRA model includes a heavy color tone constrained LoRA model, a portrait detail and line constrained LoRA model, and a background constrained LoRA model.

[0020] The heavy color tone constraint LoRA model is obtained by contrastive learning between a multimodal model and a cross-attention layer in a U-Net model; the portrait detail and line constraint LoRA model is obtained by contrastive learning between a line drawing generator, a multimodal model and a cross-attention layer in a U-Net model; the background constraint LoRA model is obtained by contrastive learning between a multimodal model and a cross-attention layer in a U-Net model.

[0021] Furthermore, the heavy color tone constrained LoRA model is obtained by comparative learning between the multimodal model and the cross attention layer in the U-Net model, specifically:

[0022] The prompt words in the color feature corpus are used as the input of the text encoder in the multimodal model, and the embedding vector is output. The output embedding vector is compared with the image feature vector passed through the image encoder in the multimodal model for learning, and the relationship between the text and the image is established to constrain the color of the corresponding parts.

[0023] Furthermore, the portrait details and line constraint LoRA model are obtained by comparative learning of the line drawing generator, the multimodal model and the cross attention layer in the U-Net model, specifically:

[0024] A line drawing generator is used to obtain important facial lines and details from character images, and a line drawing portrait dataset is constructed as the input of the image encoder. The prompt words in the detail element corpus are used as the input of the text encoder in the multimodal model to establish the connection between text and image, and constrain the portrait details and line features of the generated image.

[0025] Furthermore, the background-constrained LoRA model is obtained by comparative learning between the multimodal model and the cross-attention layer in the U-Net model, specifically:

[0026] The complete background image and the person image with the background are used as the input of the image encoder in the multimodal model, and the prompt words in the detail element corpus are used as the input of the text encoder in the multimodal model to establish the association between the prompt words and the background and constrain the generation of the background.

[0027] Furthermore, the mural character image generation model is trained according to a plurality of image data sets and corpora for different aspects of the image to obtain a trained model, specifically including:

[0028] Select one of the image datasets and the corresponding corpus, and initialize the parameters of the mural character image generation model; randomly select a character image from the selected image dataset and input it into the image encoder in the multimodal model to output the image feature vector; use the prompt words in the selected corpus as the input of the text encoder in the multimodal model to output the embedding vector; compare and learn the embedding vector with the image feature vector, and iterate continuously to obtain a trained model.

[0029] Furthermore, the mural image dataset is preprocessed to obtain multiple image datasets for different aspects of the image, specifically including:

[0030] Unify the image file type and image size of the mural image dataset;

[0031] The mural image dataset with unified format is divided into color image dataset, line image dataset and background image dataset.

[0032] The second object of the present invention can be achieved by adopting the following technical solutions:

[0033] A mural character image generation system, the system comprising:

[0034] A construction module is used to construct a mural image dataset and a corpus, wherein the corpus is related to the mural image dataset and includes a color feature corpus and a detail element corpus;

[0035] A preprocessing module is used to preprocess the mural image dataset to obtain multiple image datasets for different aspects of the image;

[0036] Design module, used to design a mural character image generation model based on multi-constrained LoRA;

[0037] A training module, used to train a mural character image generation model based on a plurality of image data sets and corpora for different aspects of the image, to obtain a trained model;

[0038] The generation module is used to input the prompt words and real face images into the trained model to generate mural character images.

[0039] The third object of the present invention can be achieved by adopting the following technical solutions:

[0040] A computer device comprises a processor and a memory for storing a program executable by the processor. When the processor executes the program stored in the memory, the above-mentioned method for generating a mural character image is implemented.

[0041] The fourth object of the present invention can be achieved by adopting the following technical solutions:

[0042] A storage medium stores a program, and when the program is executed by a processor, the above-mentioned mural character image generation method is implemented.

[0043] The present invention has the following beneficial effects compared with the prior art:

[0044] 1. Aiming at the special style of murals, the present invention adopts a mural character image generation model based on multi-constraint LoRA to realize the generation of mural character images, which can more effectively learn the style characteristics of character images and generate higher-quality mural art character images with a smaller data set. The image generated by this method has natural colors, complete and clear details of the characters and background, and the overall image has a distinct style, and the effect is far better than the existing methods.

[0045] 2. The present invention adopts a separation training method for mural figure images, designs heavy color tone constraints and portrait detail and line constraints, separates the color and lines of the portrait, and trains the LoRA model with relevant professional corpora respectively. The generated images faithfully retain the color features and clear lines of the style, overcoming the limitations of the direct training method with confused colors and lines.

[0046] 3. The heavy color toning constrained LoRA model proposed in the present invention is aimed at the problem of color distortion in the mural character image generation model when realizing the generation of mural style images of characters. It is used to improve the efficiency of model training, while improving the color quality of the generated images, and promoting the model to generate more realistic and mural-style images.

[0047] 4. The portrait detail and line constraint LoRA model proposed in the present invention is aimed at the problem of blurred lines in the mural character image generation model when realizing the generation of character mural style images, and is used to improve the line quality of mural style generated images.

[0048] 5. The background-constrained LoRA model proposed in the present invention is aimed at the problem of background missing in the mural character image generation model when realizing the generation of character mural style images, and is used to improve the richness of the generated image.

[0049] 6. The present invention can realize the image generation of figure murals and the mural style transfer of various real figure images, which greatly simplifies and innovates the way of obtaining and creating mural art, and is conducive to the protection, inheritance and development of mural art. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0051] Figure 1 The figure is a simplified flowchart of the method for generating a mural character image according to the first embodiment of the present invention.

[0052] Figure 2 This is a detailed flow chart of the mural character image generation method of embodiment 1 of the present invention.

[0053] Figure 3 This is a structural diagram of a mural character image generation model according to Example 1 of the present invention.

[0054] Figure 4 This is a structural diagram of the heavy color tone constrained LoRA model of Example 1 of the present invention.

[0055] Figure 5 This is a structural diagram of the portrait details and line constraints LoRA model of Example 1 of the present invention.

[0056] Figure 6 This is a structural diagram of the background-constrained LoRA model of Example 1 of the present invention.

[0057] Figure 7 This is a comparison diagram of a real person image and a corresponding mural style transfer image according to Example 1 of the present invention.

[0058] Figure 8 This is a structural block diagram of a mural character image generation system according to Embodiment 2 of the present invention.

[0059] Fig. 9 This is a structural block diagram of a computer device according to Embodiment 3 of the present invention. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0061] Embodiment 1:

[0062] like Figure 1 and Figure 2As shown, this embodiment provides a method for generating a mural character image, the method comprising the following steps:

[0063] S201. Build a mural image dataset and corpus.

[0064] This embodiment collects mural character images from various image websites and museums, and screens and classifies the acquired mural character images according to the content and quality of the murals. In order to ensure the consistency of the style of the dataset and the clarity of the images, it is mainly composed of mural copies painted by the famous artist Zhang Daqian, thereby constructing a mural image dataset; mainly selected words from Chinese and foreign books, in addition, it also includes 50 articles on the detailed features of mural characters, 20 articles on the detailed features of mural backgrounds, and 30 articles on the color features of murals, thereby constructing a corpus related to the mural image dataset, which includes a color feature corpus and a detail element corpus.

[0065] S202: pre-process the mural image dataset to obtain a plurality of image datasets for different aspects of the image.

[0066] In this embodiment, the image file type and image size of the mural image dataset are unified, and the image file type is unified as a "png" file and the image size is unified as "768×768". In order to ensure the independence of each LoRA training, the mural image dataset with unified format is divided into a color image dataset, a line image dataset and a background image dataset by a separation training method. Each training is performed on one image dataset, and finally three training folders are obtained, namely color, edge and background. Finally, a test folder is set and named Test.

[0067] S203. Design a mural character image generation model based on multi-constraint LoRA.

[0068] The mural character image generation model of this embodiment is as follows: Figure 3As shown in the figure, it includes a Stable Diffusion network structure and a multi-LoRA model. The Stable Diffusion network structure is the main part of the mural character image generation model. It uses the Stable Diffusion 1.5 model, including a variational autoencoder (VAE) based on an encoder-decoder architecture, a U-Net model based on a convolutional neural network, and a multimodal model (CLIP) based on contrastive learning. The input of the multimodal model is text information and image information, and the output is a matrix of digital expression. It is essentially an encoder. Then the digital information is iteratively processed layer by layer through the U-Net model, and finally passed to the next module, namely the variational autoencoder; the multi-LoRA model includes a heavy color tone constrained LoRA model, a portrait detail and line constrained LoRA model, and a background constrained LoRA model.

[0069] The structure of the heavy color hue constrained LoRA model is as follows Figure 4 As shown in the figure, it is obtained by comparative learning between the cross-attention layer in the multimodal model and the U-Net model, specifically: the prompt word in the color feature corpus is used as the input of the text encoder in the multimodal model, and the embedding vector is output. The output embedding vector is compared with the image feature vector of the image encoder in the multimodal model, and the relationship between the text and the image is established. In this process, the heavy color tone constrained LoRA model will retain the parameter adjustment ΔW generated by this related update, guide the process of generating images by the Stable Diffusion network structure, and constrain the color of the corresponding part; the heavy color tone constrained LoRA model is mainly used to improve the color quality of the generated image, so that the color has specific heavy color characteristics.

[0070] Portrait details and line constraints LoRA model such as Figure 5 As shown in the figure, it is obtained by comparative learning of the line drawing generator (from the paper "Apdrawinggan: Generating artistic portrait drawings from face photos withhierarchicalgans"), the multimodal model and the cross-attention layer in the U-Net model. Specifically, the line drawing generator is used to obtain important facial lines and details from the character image, and a line drawing portrait dataset is constructed as the input of the image encoder. The prompt words in the detail element corpus are used as the input of the text encoder in the multimodal model to establish the connection between the text and the image, and constrain the portrait details and line features of the generated image; the portrait detail and line constraint LoRA model is mainly used to improve the line quality of the generated image, reduce the blurred area of ​​the overlapping color blocks, and produce clear lines.

[0071] Background constraint LoRA model such as Figure 6 As shown in FIG, it is obtained by comparative learning of the cross-attention layer in the multimodal model and the U-Net model. Specifically, the complete background image and the person image with the background are used as the input of the image encoder in the multimodal model, and the prompt words in the detail element corpus are used as the input of the text encoder in the multimodal model to establish the association between the prompt words and the background and constrain the generation of the background. The background-constrained LoRA model can overcome the problem of mixing of foreground characters and background patterns generated in the image generation process and improve the richness and completeness of the generated background.

[0072] For the matrix W0 of the cross attention layer, the three constrained LoRA models above will update the matrix W0 using a linear weighted method, and the updated matrix is:

[0073]

[0074] Among them, i represents the i-th LoRA model, w i It represents the weight of a certain aspect of the image, and ΔW represents the product of the weight and the change of the corresponding parameter. In other words, ΔW i =w i B i A i ; By adjusting the weights, the constraint ability of each LoRA model on features at different levels in the entire image generation process can be adjusted to adjust its impact on the results. This method can ensure that each LoRA can contribute effectively at each stage of the denoising process, thereby solving the problem of detail loss that may occur in a single LoRA. Since this embodiment uses three constraints, n=3, i=1, i=2, and i=3 correspond to ΔW1, ΔW2, and ΔW3, respectively.

[0075] S204: training a mural character image generation model based on a plurality of image data sets and corpora for different aspects of the image to obtain a trained model.

[0076] Load the basic model Stable Diffusion 1.5 into the network, select one of the image datasets and the corresponding corpus. If you select a color image dataset, the corresponding corpus is the color feature corpus. If you select a line image dataset or a background image dataset, the corresponding corpus is the detail element corpus.

[0077] After selection, the parameters of the mural character image generation model are initialized and fine-tuned through the corresponding LoRA model (the color image dataset corresponds to the heavy color tone constraint LoRA model, the line image dataset corresponds to the portrait detail and line constraint LoRA model, and the background image dataset corresponds to the background constraint LoRA model). The minimum batch processing data is 4; the learning factor is set to 0.0001; the number of training rounds Epoch is set to 2, and each image is trained 60 times; the floating point precision control used in the calculation selects fp16; and the weight parameters of the corresponding LoRA model are set.

[0078] A person image randomly selected from the selected image dataset is input into the image encoder in the multimodal model, and the image feature vector is output; the prompt word in the selected corpus is used as the input of the text encoder in the multimodal model, and the embedding vector is output; the embedding vector is compared with the image feature vector for learning, and the corresponding LoRA model will retain the updated parameter ΔW, and iterate continuously to obtain a trained model.

[0079] S205: Input the prompt words and the real face image into the trained model to generate a mural character image.

[0080] This embodiment reads the test data in the test folder Test and preprocesses the data. In the test process, the preprocessing only transforms the size of the real face image to 768×768, and does not process the prompt word; the prompt word and the real face image are input into the trained model. In the test process, the corresponding output image is obtained through the generator (a generator that converts the input text and real person image into a mural style image), thereby realizing the generation of mural person images and the transfer of mural style to real people. Figure 8 shown.

[0081] Those skilled in the art will appreciate that all or part of the steps in the method for implementing the above embodiments may be completed by instructing related hardware through a program, and the corresponding program may be stored in a computer-readable storage medium.

[0082] It should be noted that although the method operations of the above embodiments are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. On the contrary, the steps depicted can be performed in a different order. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step, and / or one step can be decomposed into multiple steps.

[0083] Embodiment 2:

[0084] like Figure 8As shown, this embodiment provides a mural character image generation system, which includes a construction module 801, a preprocessing module 802, a design module 803, a training module 804 and a generation module 805. The specific functions of each module are as follows:

[0085] A construction module 801 is used to construct a mural image dataset and a corpus, wherein the corpus is related to the mural image dataset and includes a color feature corpus and a detail element corpus;

[0086] A preprocessing module 802 is used to preprocess the mural image dataset to obtain multiple image datasets for different aspects of the image;

[0087] Design module 803, used to design a mural character image generation model based on multi-constraint LoRA;

[0088] A training module 804 is used to train a mural character image generation model based on a plurality of image data sets and corpora for different aspects of the image to obtain a trained model;

[0089] The generation module 805 is used to input the prompt words and the real face image into the trained model to generate the mural character image.

[0090] The specific implementation of each module in this embodiment can refer to the above-mentioned embodiment 1, which will not be described one by one here; it should be noted that the system provided in this embodiment is only illustrated by the division of the above-mentioned functional modules. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure can be divided into different functional modules to complete all or part of the functions described above.

[0091] Embodiment 3:

[0092] This embodiment provides a computer device, which may be a computer, such as Fig. 9 As shown, a processor 902, a memory, an input device 903, a display device 904 and a network interface 905 connected via a system bus 901, the processor is used to provide computing and control capabilities, the memory includes a non-volatile storage medium 906 and an internal memory 907, the non-volatile storage medium 908 stores an operating system, a computer program and a database, the internal memory 907 provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium, when the processor 902 executes the computer program stored in the memory, the mural character image generation method of the above-mentioned embodiment 1 is implemented as follows:

[0093] A mural image dataset and a corpus are constructed, wherein the corpus is related to the mural image dataset and includes a color feature corpus and a detail element corpus; the mural image dataset is preprocessed to obtain a plurality of image datasets for different aspects of the image; a mural character image generation model based on multi-constraint LoRA is designed; the mural character image generation model is trained according to the plurality of image datasets and corpora for different aspects of the image to obtain a trained model; prompt words and real face images are input into the trained model to generate mural character images.

[0094] Embodiment 4:

[0095] This embodiment provides a storage medium, which is a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the method for generating a mural character image in the above embodiment 1 is implemented as follows:

[0096] A mural image dataset and a corpus are constructed, wherein the corpus is related to the mural image dataset and includes a color feature corpus and a detail element corpus; the mural image dataset is preprocessed to obtain a plurality of image datasets for different aspects of the image; a mural character image generation model based on multi-constraint LoRA is designed; the mural character image generation model is trained according to the plurality of image datasets and corpora for different aspects of the image to obtain a trained model; prompt words and real face images are input into the trained model to generate mural character images.

[0097] It should be noted that the computer-readable storage medium of the present embodiment may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0098] To sum up, the present invention aims at the special style of murals and adopts a mural character image generation model based on multi-constraint LoRA to realize the generation of mural character images. The style characteristics of character images can be learned more effectively, and higher quality mural art character images can be generated with a smaller data set. The images generated by this method have natural colors, complete and clear details of the characters and background, and the overall image has a distinct style, and the effect is far better than the existing methods.

[0099] The above is only a preferred embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical solution and inventive concept of the present invention within the scope disclosed by the present invention, which shall fall within the protection scope of the present invention.

Claims

1. A method for generating a mural character image, characterized in that: The method comprises: Constructing a mural image dataset and a corpus, wherein the corpus is related to the mural image dataset and includes a color feature corpus and a detail element corpus; Preprocess the mural image dataset to obtain multiple image datasets targeting different aspects of the image; Design a mural character image generation model based on multi-constrained LoRA; According to multiple image datasets and corpora for different aspects of images, the mural character image generation model is trained to obtain a trained model; Input the prompt words and real face images into the trained model to generate images of the characters in the mural.

2. The method for generating a mural character image according to claim 1, characterized in that: The mural character image generation model includes a Stable Diffusion network structure and a multi-LoRA model; The Stable Diffusion network structure includes a variational autoencoder based on an encoder-decoder architecture, a U-Net model based on a convolutional neural network, and a multimodal model based on contrastive learning. The multi-LoRA model includes a heavy color tone constrained LoRA model, a portrait detail and line constrained LoRA model, and a background constrained LoRA model. The heavy color tone constraint LoRA model is obtained by contrastive learning between a multimodal model and a cross-attention layer in a U-Net model; the portrait detail and line constraint LoRA model is obtained by contrastive learning between a line drawing generator, a multimodal model and a cross-attention layer in a U-Net model; the background constraint LoRA model is obtained by contrastive learning between a multimodal model and a cross-attention layer in a U-Net model.

3. The method for generating a mural character image according to claim 2, characterized in that: The heavy color tone constrained LoRA model is obtained by contrasting the cross attention layer in the multimodal model and the U-Net model, specifically: The prompt words in the color feature corpus are used as the input of the text encoder in the multimodal model, and the embedding vector is output. The output embedding vector is compared with the image feature vector passed through the image encoder in the multimodal model for learning, and the relationship between the text and the image is established to constrain the color of the corresponding parts.

4. The method for generating a mural character image according to claim 1, characterized in that: The portrait detail and line constraint LoRA model is obtained by comparative learning of the line drawing generator, the multimodal model and the cross attention layer in the U-Net model, specifically: A line drawing generator is used to obtain important facial lines and details from character images, and a line drawing portrait dataset is constructed as the input of the image encoder. The prompt words in the detail element corpus are used as the input of the text encoder in the multimodal model to establish the connection between text and image, and constrain the portrait details and line features of the generated image.

5. The method for generating a mural character image according to claim 2, characterized in that: The background-constrained LoRA model is obtained by comparing the cross-attention layer in the multimodal model and the U-Net model, specifically: The complete background image and the person image with the background are used as the input of the image encoder in the multimodal model, and the prompt words in the detail element corpus are used as the input of the text encoder in the multimodal model to establish the association between the prompt words and the background and constrain the generation of the background.

6. The method for generating a mural character image according to claim 2, characterized in that: The method of training a mural character image generation model according to a plurality of image data sets and corpora for different aspects of the image to obtain a trained model specifically includes: Select one of the image datasets and the corresponding corpus, and initialize the parameters of the mural character image generation model; randomly select a character image from the selected image dataset and input it into the image encoder in the multimodal model to output the image feature vector; use the prompt words in the selected corpus as the input of the text encoder in the multimodal model to output the embedding vector; compare and learn the embedding vector with the image feature vector, and iterate continuously to obtain a trained model.

7. The method for generating a mural character image according to any one of claims 1 to 6, characterized in that: The mural image dataset is preprocessed to obtain a plurality of image datasets for different aspects of the image, specifically including: Unify the image file type and image size of the mural image dataset; The mural image dataset with unified format is divided into color image dataset, line image dataset and background image dataset.

8. A mural character image generation system, characterized in that: The system comprises: A construction module is used to construct a mural image dataset and a corpus, wherein the corpus is related to the mural image dataset and includes a color feature corpus and a detail element corpus; A preprocessing module is used to preprocess the mural image dataset to obtain multiple image datasets for different aspects of the image; Design module, used to design a mural character image generation model based on multi-constrained LoRA; A training module, used to train a mural character image generation model based on a plurality of image data sets and corpora for different aspects of the image, to obtain a trained model; The generation module is used to input the prompt words and real face images into the trained model to generate mural character images.

9. A computer device comprising a processor and a memory for storing a program executable by the processor, characterized in that: When the processor executes the program stored in the memory, the mural character image generation method described in any one of claims 1 to 7 is implemented.

10. A storage medium storing a program, characterized in that: When the program is executed by a processor, the mural character image generation method described in any one of claims 1 to 7 is implemented.

Images