Method and system for generating mural image of character, computer device and storage medium

Abstract

The application discloses a mural figure image generation method and system, computer equipment and a storage medium, and the method comprises the steps of: constructing a mural image dataset and a corpus, wherein the corpus comprises a color feature corpus and a detail element corpus; preprocessing the mural image dataset to obtain a plurality of image datasets for different aspects of the image; designing a mural figure image generation model based on multi-constraint LoRA; training the mural figure image generation model according to one type of image dataset and the corresponding corpus to obtain a trained model; and inputting a prompt word and a real face image into the trained model to generate a mural figure image. The mural figure image generation model based on multi-constraint LoRA is introduced to guide the network model to fully learn the color, lines and background patterns of the mural, so that a mural-style figure image with better effect and higher quality is generated.

Description

Technical Field

[0001] The present invention belongs to the fields of image processing, computer vision and image conversion, and specifically relates to a method, system, computer equipment and storage medium for generating mural character images. Background Art

[0002] Currently, deep learning-based methods are widely used to automatically generate images with specific artistic styles. These technologies have two main application areas: 1) text-to-image generation, such as generating images from textual descriptions. 2) image-to-image generation, such as converting natural landscapes into various artistic styles. The emergence of these technologies has made it possible to automatically generate images of people in the style of murals.

[0003] Since its introduction, the Generative Adversarial Network (GAN) model has been widely used in many fields. The CycleGAN algorithm (from the paper "Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks") is a major application of GAN in style transfer. Compared to conventional convolutional neural networks, CycleGAN can achieve style transfer between different domain sets, generating more natural and realistic images. However, the CycleGAN method suffers from issues such as low image stability and difficulty in changing object shapes. Furthermore, due to the CycleGAN model's structural characteristics of not being a single network, the model training process is very difficult, and it is not easy to guarantee the generation of a one-to-one mapping of specific images.

[0004] The Diffusion model (from the paper "Denoising Diffusion Probabilistic Models. Advances in Neural Information Processing Systems") uses a block-by-block contrast loss to generate images using the contrast loss between the sample and the original image. This serves as the basis for the Stable Diffusion model (from the paper "High-resolution Image Synthesis with Latent Diffusion Models"). Compared to GANs, the Stable Diffusion model not only allows text-controlled generation of images with a corresponding style, but also achieves higher image quality, preserving more stylistic features and details. Furthermore, its overall structure of generating images by gradually diffusing noise makes its training process more transparent. However, further research on generated images revealed that the diffusion model is insufficient to control the transfer of specific styles and is insensitive to cues, significantly reducing the controllability of the images. This effect is also evident in portrait generation, where many counterintuitive errors occur, such as face blurring and background overlap.

[0005] With the development of image generation, Low-Rank Adaptation (LoRA) (from the paper "Low-rank adaptation of large language models"), as a fine-tuning model flexibly constructed for different tasks, has achieved excellent results when combined with pre-trained large models. It freezes the parameters of the pre-trained model and then adds a trainable bypass matrix ΔW. Based on the diffusion model, it fine-tunes the specific style to generate higher-quality stylized images. It is worth mentioning that compared with a single LoRA model, a multi-LoRA model has stronger control over the large model and is more capable of processing the details of the corresponding style. To generate higher-quality images, the training method of stacking multiple LoRAs has been used for image generation tasks in different styles. However, this type of model is not optimized for the characteristics of murals. When merging multiple LoRA models, the merging process of different LoRA models is unstable, which can easily lead to inconsistent image elements and low-quality generated images. Summary of the Invention

[0006] In order to address the shortcomings of the above-mentioned prior art, the present invention provides a mural character image generation method, system, computer device and storage medium, which introduces a mural character image generation model based on multi-constraint LoRA to guide the network model to fully learn the color, lines and background pattern of the mural, thereby generating better and higher-quality mural-style character images.

[0007] The first object of the present invention is to provide a method for generating a mural character image.

[0008] The second object of the present invention is to provide a mural character image generation system.

[0009] A third object of the present invention is to provide a computer device.

[0010] A fourth object of the present invention is to provide a storage medium.

[0011] The first object of the present invention can be achieved by adopting the following technical solutions:

[0012] A method for generating a mural character image, the method comprising:

[0013] Constructing a mural image dataset and a corpus, wherein the corpus is related to the mural image dataset and includes a color feature corpus and a detail element corpus;

[0014] Preprocess the mural image dataset to obtain multiple image datasets targeting different aspects of the image;

[0015] Design a mural character image generation model based on multi-constrained LoRA;

[0016] Based on multiple image datasets and corpora targeting different aspects of images, a mural character image generation model is trained to obtain a trained model;

[0017] Input the prompt words and real face images into the trained model to generate mural character images.

[0018] Furthermore, the mural character image generation model includes a Stable Diffusion network structure and a multi-LoRA model;

[0019] The Stable Diffusion network structure includes a variational autoencoder based on an encoder-decoder architecture, a U-Net model based on a convolutional neural network, and a multimodal model based on contrastive learning. The multi-LoRA model includes a heavy color tone constrained LoRA model, a portrait detail and line constrained LoRA model, and a background constrained LoRA model.

[0020] The heavy color hue constraint LoRA model is obtained by comparative learning between the multimodal model and the cross-attention layer in the U-Net model; the portrait detail and line constraint LoRA model is obtained by comparative learning between the line drawing generator, the multimodal model and the cross-attention layer in the U-Net model; the background constraint LoRA model is obtained by comparative learning between the multimodal model and the cross-attention layer in the U-Net model.

[0021] Furthermore, the heavy color hue constrained LoRA model is obtained by comparative learning between the multimodal model and the cross attention layer in the U-Net model, specifically:

[0022] The prompt words in the color feature corpus are used as the input of the text encoder in the multimodal model, and the embedding vector is output. The output embedding vector is compared with the image feature vector passed through the image encoder in the multimodal model for learning, and the relationship between the text and the image is established to constrain the color of the corresponding parts.

[0023] Furthermore, the portrait details and line constraint LoRA model are obtained by comparative learning through the line drawing generator, multimodal model and cross attention layer in the U-Net model, specifically:

[0024] A line drawing generator is used to obtain important facial lines and details from character images, and a line drawing portrait dataset is constructed as the input of the image encoder. The prompt words in the detail element corpus are used as the input of the text encoder in the multimodal model to establish a connection between text and image, and constrain the portrait details and line features of the generated image.

[0025] Furthermore, the background-constrained LoRA model is obtained by comparative learning between the multimodal model and the cross-attention layer in the U-Net model, specifically:

[0026] The complete background image and the person image with background are used as the input of the image encoder in the multimodal model, and the prompt words in the detail element corpus are used as the input of the text encoder in the multimodal model to establish the association between the prompt words and the background and constrain the generation of the background.

[0027] Furthermore, the mural figure image generation model is trained based on a plurality of image datasets and corpora targeting different aspects of the image to obtain a trained model, specifically including:

[0028] Select one of the image datasets and the corresponding corpus to initialize the parameters of the mural figure image generation model; randomly select a figure image from the selected image dataset and input it into the image encoder in the multimodal model to output the image feature vector; use the prompt words in the selected corpus as the input of the text encoder in the multimodal model to output the embedding vector; compare and learn the embedding vector with the image feature vector, and iterate continuously to obtain a trained model.

[0029] Furthermore, the mural image dataset is preprocessed to obtain multiple image datasets targeting different aspects of the image, specifically including:

[0030] Unify the image file type and image size of the mural image dataset;

[0031] The mural image dataset with unified format is divided into color image dataset, line image dataset and background image dataset.

[0032] The second object of the present invention can be achieved by adopting the following technical solutions:

[0033] A mural character image generation system, the system comprising:

[0034] A construction module is used to construct a mural image dataset and a corpus, wherein the corpus is related to the mural image dataset and includes a color feature corpus and a detail element corpus;

[0035] A preprocessing module is used to preprocess the mural image dataset to obtain multiple image datasets targeting different aspects of the image;

[0036] Design module, used to design a mural character image generation model based on multi-constraint LoRA;

[0037] A training module is used to train a mural character image generation model based on multiple image datasets and corpora targeting different aspects of the image to obtain a trained model;

[0038] The generation module is used to input the prompt words and real face images into the trained model to generate mural character images.

[0039] The third object of the present invention can be achieved by adopting the following technical solutions:

[0040] A computer device comprises a processor and a memory for storing a program executable by the processor. When the processor executes the program stored in the memory, the above-mentioned method for generating a mural character image is implemented.

[0041] The fourth object of the present invention can be achieved by adopting the following technical solutions:

[0042] A storage medium stores a program, which, when executed by a processor, implements the above-mentioned method for generating a mural character image.

[0043] The present invention has the following beneficial effects compared to the prior art:

[0044] 1. Aiming at the special style of murals, the present invention adopts a mural character image generation model based on multi-constraint LoRA to realize the generation of mural character images. It can more effectively learn the style characteristics of character images and generate higher-quality mural art character images with a smaller data set. The images generated by this method have natural colors, complete and clear details of characters and backgrounds, and the overall image has a distinct style, which is far superior to existing methods.

[0045] 2. The present invention adopts a separation training method for mural figure images, designs heavy color tone constraints as well as portrait detail and line constraints, separates the color and lines of the portrait, and trains the LoRA model with relevant professional corpora. The generated images faithfully retain the rich color features and clear lines of the style, overcoming the limitation of the direct training method of color and line confusion.

[0046] 3. The heavy color toning constrained LoRA model proposed in this invention is aimed at the problem of color distortion in the mural figure image generation model when realizing the generation of mural-style image of the figure. It is used to improve the efficiency of model training, while improving the color quality of the generated image, and promoting the model to generate more realistic and mural-style images.

[0047] 4. The portrait detail and line constraint LoRA model proposed in this invention is aimed at the problem of blurred lines in the mural character image generation model when realizing the generation of mural-style character images, and is used to improve the line quality of mural-style generated images.

[0048] 5. The background-constrained LoRA model proposed in this invention addresses the problem of background loss in the mural figure image generation model when generating mural-style images of figures, and is used to improve the richness of the generated images.

[0049] 6. The present invention can realize the image generation of figure murals and the mural style transfer of various real figure images, greatly simplifying and innovating the methods of obtaining and creating mural art, which is conducive to the protection, inheritance and development of mural art. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0051] Figure 1 This is a simplified flowchart of the method for generating a mural character image according to Example 1 of the present invention.

[0052] Figure 2 This is a detailed flow chart of the method for generating a mural character image according to Example 1 of the present invention.

[0053] Figure 3 This is a structural diagram of a mural character image generation model according to Example 1 of the present invention.

[0054] Figure 4 This is a structural diagram of the heavy color tone constrained LoRA model of Example 1 of the present invention.

[0055] Figure 5 This is a structural diagram of the portrait details and line constraints LoRA model of Example 1 of the present invention.

[0056] Figure 6 This is a structural diagram of the background-constrained LoRA model of Example 1 of the present invention.

[0057] Figure 7 This is a comparison diagram of a real person image and the corresponding mural style transfer image according to Example 1 of the present invention.

[0058] Figure 8 This is a structural block diagram of a mural character image generation system according to embodiment 2 of the present invention.

[0059] Figure 9 This is a structural block diagram of a computer device according to embodiment 3 of the present invention. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0061] Example 1:

[0062] like Figure 1 and Figure 2As shown, this embodiment provides a method for generating a mural character image, which includes the following steps:

[0063] S201. Build a mural image dataset and corpus.

[0064] This example collects mural character images from various image websites and museums, and screens and classifies the acquired mural character images based on the content and quality of the murals. To ensure the consistency of the dataset style and the clarity of the images, the dataset mainly consists of mural copies painted by the famous artist Zhang Daqian, thereby constructing a mural image dataset. The dataset mainly selects words from Chinese and foreign books. In addition, it also includes 50 articles on the detailed features of mural characters, 20 articles on the detailed features of mural backgrounds, and 30 articles on the color features of murals. In this way, a corpus related to the mural image dataset is constructed, which includes a color feature corpus and a detail element corpus.

[0065] S202 : Preprocess the mural image dataset to obtain multiple image datasets targeting different aspects of the image.

[0066] In this embodiment, the image file type and image size of the mural image dataset are unified, and the unified image file type is "png" file and the unified image size is "768×768". In order to ensure the independence of each LoRA training, the separation training method is used to divide the mural image dataset with unified format into color image dataset, line image dataset and background image dataset. Each training is performed on one image dataset, and finally three training folders are obtained, namely color, edge and background. Finally, a test folder is set and named Test.

[0067] S203. Design a mural character image generation model based on multi-constraint LoRA.

[0068] The mural character image generation model of this embodiment is as follows Figure 3As shown in the figure, it includes a Stable Diffusion network structure and a multi-LoRA model. The Stable Diffusion network structure is the main part of the mural character image generation model. It uses the Stable Diffusion 1.5 model, including a variational autoencoder (VAE) based on an encoder-decoder architecture, a U-Net model based on a convolutional neural network, and a multimodal model (CLIP) based on contrastive learning. The input of the multimodal model is text information and image information, and the output is a matrix of digital expression. It is essentially an encoder. The digital information is then iteratively processed layer by layer through the U-Net model, and finally passed to the next module, namely the variational autoencoder; the multi-LoRA model includes a heavy color tone constrained LoRA model, a portrait detail and line constrained LoRA model, and a background constrained LoRA model.

[0069] The structure of the heavy color hue constrained LoRA model is as follows Figure 4 As shown in the figure, it is obtained by comparative learning between the cross-attention layer in the multimodal model and the U-Net model. Specifically, the prompt word in the color feature corpus is used as the input of the text encoder in the multimodal model, and the embedding vector is output. The output embedding vector is compared with the image feature vector of the image encoder in the multimodal model, and the relationship between the text and the image is established. In this process, the heavy color hue constrained LoRA model will retain the parameter adjustment ΔW generated by this related update, guide the process of generating images by the Stable Diffusion network structure, and constrain the color of the corresponding part; the heavy color hue constrained LoRA model is mainly used to improve the color quality of the generated image and make the color have specific heavy color characteristics.

[0070] Portrait details and line constraints LoRA model such as Figure 5 As shown in the figure, it is obtained by comparative learning of the line drawing generator (from the paper "Apdrawinggan: Generating artistic portrait drawings from face photos withhierarchicalgans"), the multimodal model and the cross-attention layer in the U-Net model. Specifically, the line drawing generator is used to obtain important facial lines and details from the character image, and a line drawing portrait dataset is constructed as the input of the image encoder. The prompt words in the detail element corpus are used as the input of the text encoder in the multimodal model to establish a connection between the text and the image, and constrain the portrait details and line features of the generated image; the portrait detail and line constraint LoRA model is mainly used to improve the line quality of the generated image, reduce the blurred area of ​​the overlapping color blocks, and produce clear lines.

[0071] Background constraint LoRA model such as Figure 6 As shown in the figure, it is obtained by comparative learning between the cross-attention layer in the multimodal model and the U-Net model. Specifically, the complete background image and the person image with background are used as the input of the image encoder in the multimodal model, and the prompt words in the detail element corpus are used as the input of the text encoder in the multimodal model to establish the association between the prompt words and the background and constrain the generation of the background. The background-constrained LoRA model can overcome the problem of mixing of foreground people and background patterns generated during the image generation process, and improve the richness and completeness of the generated background.

[0072] For the matrix W0 of the cross attention layer, the three constrained LoRA models above will update the matrix W0 using a linear weighted method. The updated matrix is:

[0073]

[0074] Among them, i represents the i-th LoRA model, w i It represents the weight of a certain aspect of the image, and ΔW represents the product of the weight and the change of the corresponding parameter, that is, ΔW i =w i B i A i ; By adjusting the weights, the constraint ability of each LoRA model on features at different levels in the entire image generation process can be adjusted to adjust its impact on the results. This method can ensure that each LoRA can effectively contribute at each stage of the denoising process, thereby solving the problem of detail loss that may occur in a single LoRA. Since this embodiment uses three constraints, n=3, i=1, i=2, and i=3 correspond to ΔW1, ΔW2, and ΔW3, respectively.

[0075] S204: Training a mural character image generation model based on a plurality of image data sets and corpora targeting different aspects of the image to obtain a trained model.

[0076] Load the basic model Stable Diffusion 1.5 into the network and select one of the image datasets and the corresponding corpus. If you select a color image dataset, the corresponding corpus is the color feature corpus. If you select a line image dataset or a background image dataset, the corresponding corpus is the detail element corpus.

[0077] After selection, the parameters of the mural figure image generation model are initialized and fine-tuned using the corresponding LoRA model (the color image dataset corresponds to the heavy color tone constrained LoRA model, the line image dataset corresponds to the portrait detail and line constrained LoRA model, and the background image dataset corresponds to the background constrained LoRA model). The minimum batch size is 4; the learning factor is set to 0.0001; the number of training epochs is set to 2, and each image is trained 60 times; fp16 is selected as the floating-point precision control used in the calculation; and the weight parameters of the corresponding LoRA model are set.

[0078] A randomly selected person image from the selected image dataset is input into the image encoder in the multimodal model to output an image feature vector; the prompt word in the selected corpus is used as the input of the text encoder in the multimodal model to output an embedding vector; the embedding vector is compared with the image feature vector for learning, and the corresponding LoRA model will retain the updated parameter ΔW, and continuously iterate to obtain a trained model.

[0079] S205: Input the prompt words and the real face image into the trained model to generate a mural character image.

[0080] This embodiment reads the test data in the test folder Test and preprocesses the data. During the test process, the preprocessing only transforms the size of the real face image to 768×768, and does not process the prompt word; the prompt word and the real face image are input into the trained model. During the test process, the generator (a generator that converts the input text and real person image into a mural style image) obtains the corresponding output image, realizing the generation of mural person images and the transfer of mural style to real people, such as Figure 8 shown.

[0081] Those skilled in the art will appreciate that all or part of the steps in the method for implementing the above embodiments may be completed by instructing related hardware through a program, and the corresponding program may be stored in a computer-readable storage medium.

[0082] It should be noted that although the method operations of the above embodiments are described in a particular order in the accompanying drawings, this does not require or imply that the operations must be performed in this particular order, or that all of the illustrated operations must be performed to achieve the desired results. Rather, the depicted steps may be performed in a different order. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into a single step, and / or a single step may be broken down into multiple steps.

[0083] Example 2:

[0084] like Figure 8As shown, this embodiment provides a mural character image generation system, which includes a construction module 801, a preprocessing module 802, a design module 803, a training module 804 and a generation module 805. The specific functions of each module are as follows:

[0085] A construction module 801 is used to construct a mural image dataset and a corpus, wherein the corpus is related to the mural image dataset and includes a color feature corpus and a detail element corpus;

[0086] A preprocessing module 802 is used to preprocess the mural image dataset to obtain multiple image datasets targeting different aspects of the image;

[0087] Design module 803, for designing a mural character image generation model based on multi-constraint LoRA;

[0088] A training module 804 is configured to train a mural character image generation model based on a plurality of image datasets and corpora targeting different aspects of the image to obtain a trained model;

[0089] The generation module 805 is used to input the prompt words and the real face image into the trained model to generate the mural character image.

[0090] The specific implementation of each module in this embodiment can be found in the above-mentioned embodiment 1, and will not be described one by one here; it should be noted that the system provided in this embodiment is only illustrated by the division of the above-mentioned functional modules. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure can be divided into different functional modules to complete all or part of the functions described above.

[0091] Example 3:

[0092] This embodiment provides a computer device, which can be a computer, such as Figure 9 As shown, it includes a processor 902, a memory, an input device 903, a display device 904, and a network interface 905 connected via a system bus 901. The processor is used to provide computing and control capabilities. The memory includes a non-volatile storage medium 906 and an internal memory 907. The non-volatile storage medium 908 stores an operating system, a computer program, and a database. The internal memory 907 provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. When the processor 902 executes the computer program stored in the memory, the method for generating a mural character image of the above-mentioned embodiment 1 is implemented as follows:

[0093] A mural image dataset and a corpus are constructed, wherein the corpus is related to the mural image dataset and includes a color feature corpus and a detail element corpus; the mural image dataset is preprocessed to obtain multiple image datasets targeting different aspects of the image; a mural character image generation model based on multi-constrained LoRA is designed; the mural character image generation model is trained based on the multiple image datasets and corpora targeting different aspects of the image to obtain a trained model; and prompt words and real human face images are input into the trained model to generate mural character images.

[0094] Example 4:

[0095] This embodiment provides a storage medium, which is a computer-readable storage medium and stores a computer program. When the computer program is executed by a processor, the method for generating a mural character image of the above embodiment 1 is implemented as follows:

[0096] A mural image dataset and a corpus are constructed, wherein the corpus is related to the mural image dataset and includes a color feature corpus and a detail element corpus; the mural image dataset is preprocessed to obtain multiple image datasets targeting different aspects of the image; a mural character image generation model based on multi-constrained LoRA is designed; the mural character image generation model is trained based on the multiple image datasets and corpora targeting different aspects of the image to obtain a trained model; and prompt words and real human face images are input into the trained model to generate mural character images.

[0097] It should be noted that the computer-readable storage medium of the present embodiment may be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0098] In summary, the present invention targets the special style of murals and adopts a mural character image generation model based on multi-constraint LoRA to realize the generation of mural character images. It can more effectively learn the style characteristics of character images and generate higher-quality mural art character images with a smaller data set. The images generated by this method have natural colors, complete and clear details of the characters and background, and the overall image has a distinct style, and the effect is far better than the existing methods.

[0099] The above is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes based on the technical solution and inventive concept of the present invention within the scope disclosed by the present invention, which falls within the scope of protection of the present invention.

Claims

1. A method for generating a mural character image, characterized in that: The method comprises: Constructing a mural image dataset and a corpus, wherein the corpus is related to the mural image dataset and includes a color feature corpus and a detail element corpus; Preprocess the mural image dataset to obtain multiple image datasets targeting different aspects of the image; Design a mural character image generation model based on multi-constrained LoRA; Based on multiple image datasets and corpora targeting different aspects of images, a mural character image generation model is trained to obtain a trained model; Input the prompt words and real face images into the trained model to generate mural character images; The mural character image generation model includes a Stable Diffusion network structure and a multi-LoRA model; The Stable Diffusion network structure includes a variational autoencoder based on an encoder-decoder architecture, a U-Net model based on a convolutional neural network, and a multimodal model based on contrastive learning. The multi-LoRA model includes a heavy color tone constrained LoRA model, a portrait detail and line constrained LoRA model, and a background constrained LoRA model. The heavy hue constraint LoRA model is obtained by comparative learning between the multimodal model and the cross-attention layer in the U-Net model; the portrait detail and line constraint LoRA model is obtained by comparative learning between the line drawing generator, the multimodal model and the cross-attention layer in the U-Net model; the background constraint LoRA model is obtained by comparative learning between the multimodal model and the cross-attention layer in the U-Net model; The heavy hue constrained LoRA model is obtained by comparative learning between the multimodal model and the cross-attention layer in the U-Net model, specifically: The prompt words in the color feature corpus are used as input to the text encoder in the multimodal model, and the output embedding vector is compared with the image feature vector passed through the image encoder in the multimodal model for learning. The relationship between the text and the image is established, and the colors of the corresponding parts are constrained. The portrait detail and line constraint LoRA model is obtained by comparative learning between the line drawing generator, the multimodal model and the cross-attention layer in the U-Net model, specifically: A line drawing generator is used to extract important facial lines and details from human images. A line drawing portrait dataset is constructed as input to the image encoder. The prompt words in the detail element corpus are used as input to the text encoder in the multimodal model to establish a connection between text and image, constraining the generated image's portrait details and line features. The background-constrained LoRA model is obtained by comparative learning between the multimodal model and the cross-attention layer in the U-Net model, specifically: The complete background image and the person image with background are used as the input of the image encoder in the multimodal model, and the prompt words in the detail element corpus are used as the input of the text encoder in the multimodal model to establish the association between the prompt words and the background and constrain the generation of the background.

2. The method for generating a mural character image according to claim 1, wherein: The method of training a mural character image generation model based on a plurality of image data sets and corpora for different aspects of the image to obtain a trained model specifically includes: Select one of the image datasets and the corresponding corpus to initialize the parameters of the mural figure image generation model; randomly select a figure image from the selected image dataset and input it into the image encoder in the multimodal model to output the image feature vector; use the prompt words in the selected corpus as the input of the text encoder in the multimodal model to output the embedding vector; compare and learn the embedding vector with the image feature vector, and iterate continuously to obtain a trained model.

3. The method for generating a mural character image according to any one of claims 1 to 2, characterized in that: The mural image dataset is preprocessed to obtain multiple image datasets targeting different aspects of the image, specifically including: Unify the image file type and image size of the mural image dataset; The mural image dataset with unified format is divided into color image dataset, line image dataset and background image dataset.

4. A mural character image generation system, characterized in that: The system comprises: A construction module is used to construct a mural image dataset and a corpus, wherein the corpus is related to the mural image dataset and includes a color feature corpus and a detail element corpus; A preprocessing module is used to preprocess the mural image dataset to obtain multiple image datasets targeting different aspects of the image; Design module, used to design a mural character image generation model based on multi-constraint LoRA; A training module is used to train a mural character image generation model based on multiple image datasets and corpora targeting different aspects of the image to obtain a trained model; The generation module is used to input the prompt words and real face images into the trained model to generate mural character images; The mural character image generation model includes a Stable Diffusion network structure and a multi-LoRA model; The Stable Diffusion network structure includes a variational autoencoder based on an encoder-decoder architecture, a U-Net model based on a convolutional neural network, and a multimodal model based on contrastive learning. The multi-LoRA model includes a heavy color tone constrained LoRA model, a portrait detail and line constrained LoRA model, and a background constrained LoRA model. The heavy hue constraint LoRA model is obtained by comparative learning between the multimodal model and the cross-attention layer in the U-Net model; the portrait detail and line constraint LoRA model is obtained by comparative learning between the line drawing generator, the multimodal model and the cross-attention layer in the U-Net model; the background constraint LoRA model is obtained by comparative learning between the multimodal model and the cross-attention layer in the U-Net model; The heavy hue constrained LoRA model is obtained by comparative learning between the multimodal model and the cross-attention layer in the U-Net model, specifically: The prompt words in the color feature corpus are used as input to the text encoder in the multimodal model, and the output embedding vector is compared with the image feature vector passed through the image encoder in the multimodal model for learning. The relationship between the text and the image is established, and the colors of the corresponding parts are constrained. The portrait detail and line constraint LoRA model is obtained by comparative learning between the line drawing generator, the multimodal model and the cross-attention layer in the U-Net model, specifically: A line drawing generator is used to extract important facial lines and details from human images. A line drawing portrait dataset is constructed as input to the image encoder. The prompt words in the detail element corpus are used as input to the text encoder in the multimodal model to establish a connection between text and image, constraining the generated image's portrait details and line features. The background-constrained LoRA model is obtained by comparative learning between the multimodal model and the cross-attention layer in the U-Net model, specifically: The complete background image and the person image with background are used as the input of the image encoder in the multimodal model, and the prompt words in the detail element corpus are used as the input of the text encoder in the multimodal model to establish the association between the prompt words and the background and constrain the generation of the background.

5. A computer device comprising a processor and a memory for storing a program executable by the processor, characterized in that: When the processor executes the program stored in the memory, the method for generating a mural character image according to any one of claims 1 to 3 is implemented.

6. A storage medium storing a program, characterized in that: When the program is executed by a processor, the method for generating a mural character image according to any one of claims 1 to 3 is implemented.

Images