Character graffiti graph generation method and device, equipment and storage medium

Abstract

The embodiment of the invention provides a character graffiti map generation method and device, equipment and a storage medium, and is applied to the technical field of character graffiti. The method comprises the following steps: receiving a to-be-scrawled text graph and a scrawling style reference graph uploaded by a user; performing picture coding processing on the graffiti style reference picture to obtain a picture coding vector; and inputting the to-be-scrawled character graph and the picture coding vector into a pre-trained character scrawling graph generation model, and outputting a character scrawling graph conforming to the scrawling style of the scrawling style reference graph. In this way, only a simple character graph with black and white colors needs to be introduced, which is used as a background color and which is used as a foreground color can be adaptively selected, and finally, the conditions that the painting part suitable for the reference graph is dark and the background is light and the painting part suitable for the reference graph is light are covered at the same time. And the character graffiti painting function can be realized under the two conditions that the background is dark.

Description

Technical Field

[0001] The present disclosure relates to the technical field of text graffiti, and in particular to a method, device, equipment and storage medium for generating a text graffiti image. Background Art

[0002] Graffiti is a kind of visual font design art. Graffiti includes a lot of content, mainly deformed fonts, followed by 3D realism, character realism, various scene realism, cartoon characters, etc., with bright colors to create a strong visual effect and publicity effect.

[0003] There are two main ways to generate text graffiti images. Solution 1: The designer uses AI to generate a desired style image, and then manually overlays words or graffiti paintings to let AI regenerate the image, and repeat until the designer is satisfied; Solution 2: Use the style image designed by the designer, or the image generated by AI as a reference image, and let the input text be used as a guide for generation. There are two ways of guidance: 1) Color guidance, manually extract the color and background color that can be used for painting, and use the extracted color to color the input text or graffiti painting; 2) Use brightness contrast guidance, use the ControlNet brightness contrast model to color the content used for painting in black and white.

[0004] The disadvantages of the above schemes are: Scheme 1 requires designers to repeat the operation many times, a large number of manual repetitive operations, and the quality of the generated paintings is completely determined by the designer's aesthetics. There will be a situation where the generated pictures have no logic, but just superimpose the text with colors; when the designer finishes making the first picture, if he makes other pictures of the same style of text later, he needs to start from scratch, and there is no simpler operation. Scheme 2, if color guidance is used, it is necessary to pre-extract the colors suitable for the drawing part and the background part of the reference picture; if brightness contrast guidance is used, the drawing part of the reference picture must be dark and the background must be light, and it cannot be extended to other reference pictures with rich styles. For example, it cannot meet the requirements of the drawing part of the reference picture being light and the background being dark; it cannot even cover both situations at the same time, that is, it can be applied to the drawing part of the reference picture being dark and the background being light, and it can also be applied to the drawing part of the reference picture being light and the background being dark. Summary of the invention

[0005] The present invention provides a method, device, equipment and storage medium for generating a text graffiti image.

[0006] According to a first aspect of the present disclosure, a method for generating a text graffiti image is provided. The method comprises:

[0007] Receive text images to be graffitied and graffiti style reference images uploaded by users;

[0008] Performing image coding processing on the graffiti style reference image to obtain an image coding vector;

[0009] The graffiti text image and the image encoding vector are input into a pre-trained text graffiti image generation model, and a text graffiti image with a graffiti style that conforms to the graffiti style reference image is output.

[0010] According to the above aspects and any possible implementation, an implementation is further provided, wherein the pre-trained text graffiti image generation model includes: a trained ControlNet model and a UNet model.

[0011] According to the above aspects and any possible implementation, an implementation is further provided, wherein the ControlNet model is trained by the following steps:

[0012] A first training sample and a second training sample are obtained as a training sample set; the first training sample is composed of a first text image generated from a designer reference image and the text to be drawn; the second training sample is composed of a second text image generated from a designer reference image and the text to be drawn;

[0013] The ControlNet model is trained using the training sample set. When the number of model training times reaches a preset number, the training is completed, and the trained model is used as the ControlNet model in the text graffiti image generation model.

[0014] According to the above aspects and any possible implementation, a further implementation is provided, in which a process of a designer generating a first character portrait image by referring to a drawing and characters to be drawn includes:

[0015] Obtaining a designer reference image, and extracting foreground and background colors and text content descriptions from the designer reference image;

[0016] Input the text to be drawn into the text-to-image model, and generate a text image from the text to be drawn according to the extracted foreground and background colors;

[0017] Inputting the text image into a picture coding model to obtain an initialized latent space code;

[0018] Inputting the extracted text content description into a text description encoding model to obtain a first feature vector;

[0019] Inputting the designer reference image into an image description coding model to obtain a second feature vector;

[0020] Inputting the initialized latent space code, the first feature vector and the second feature vector into a picture decoding model to generate a first text image;

[0021] The first character image is used as a first training sample.

[0022] According to the above aspects and any possible implementation, an implementation is further provided, wherein before inputting the initialized latent space code, the first feature vector and the second feature vector into the picture decoding model to generate the first text image, the method further includes:

[0023] The initialized latent space code, the first eigenvector and the second eigenvector are input into a denoising model for denoising.

[0024] According to the above aspects and any possible implementation, a further implementation is provided, wherein a process of generating a second text image by a designer using a reference image and a text image to be drawn includes:

[0025] Get designer reference drawings;

[0026] Input the text to be drawn into the text-to-image model to obtain an initial text image;

[0027] The brightness of the initial text image is adjusted according to the brightness of the designer's reference image to obtain the text image to be drawn;

[0028] Performing image encoding processing on the designer reference image to obtain an encoding vector;

[0029] Inputting the to-be-graffitied text image and the encoding vector into a pre-constructed text graffiti image generation model, and outputting a second text image; the pre-constructed text graffiti image generation model includes an initial ControlNet model and a UNet model;

[0030] The second character image is used as a second training sample.

[0031] According to the above aspects and any possible implementation, an implementation is further provided.

[0032] Perform image encoding processing on the designer reference image to obtain an encoding vector, including:

[0033] Use the IP-Adapter algorithm to encode the designer's reference image and obtain the encoding vector.

[0034] According to a second aspect of the present disclosure, a device for generating a text graffiti image is provided. The device comprises:

[0035] A data receiving module, used for receiving a text image to be graffitied and a graffiti style reference image uploaded by a user;

[0036] A coding processing module, used for performing image coding processing on the graffiti style reference image to obtain an image coding vector;

[0037] The text graffiti module is used to input the text image to be graffitied and the picture encoding vector into a pre-trained text graffiti image generation model, and output a text graffiti image in a graffiti style that conforms to the graffiti style reference image.

[0038] According to a third aspect of the present disclosure, an electronic device is provided, which includes a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the program, the method described above is implemented.

[0039] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the method according to the first aspect of the present disclosure is implemented.

[0040] The embodiments of the present invention provide a method, device, equipment and storage medium for generating text graffiti images, which extract the foreground / background colors of a designer reference image and adjust the brightness of the designer reference image to generate a guide image, guide the text to be drawn to generate a corresponding text image, and use it as a data sample for the ControlNet part of the training. The trained ControlNet is added to the UNet as a trained text graffiti image generation model, so that only a simple black and white text image (it can be a text image with black text and white background, or a text image with white text and black background) needs to be input to adaptively select which part is used as the background color and which part is used as the foreground color. Ultimately, it covers the text graffiti drawing function that can be applied to both the case where the drawing part of the reference image is dark and the background is light, and the case where the drawing part of the reference image is light and the background is dark.

[0041] It should be understood that the contents described in the summary of the invention are not intended to limit the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, among which:

[0043] Figure 1 A flow chart showing a method for generating a text graffiti image according to an embodiment of the present disclosure is shown;

[0044] Figure 2 A schematic diagram of a specific example process in an embodiment of the present disclosure is shown;

[0045] Figure 3 A block diagram of a device for generating a text graffiti image according to an embodiment of the present disclosure is shown;

[0046] Figure 4 A schematic block diagram of an exemplary electronic device capable of implementing embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0048] In addition, the term "and / or" in this article is only a description of the association relationship between the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0049] The method for generating text graffiti images proposed in the present disclosure relies on: (1) collecting template data, extracting foreground / background colors, generating guide maps, and using algorithms to generate a large number of text images; (2) collecting template data, adjusting the illumination contrast to generate guide maps, and using algorithms to generate a large number of text images; (3) fusing the generated data of (1) and (2) to train a new ControlNet, which relies on the input of a black-and-white image, in which white is the background part and black is the graffiti part to be drawn. The template data is a reference image, and the reference image encoding is input into the network in combination with the IP-Adapter algorithm. After the new ControlNet is trained, the algorithm only needs to input a simple black-and-white guide map according to the template image to adaptively select which part is used as the background color and which part is used as the foreground color. Finally, it covers both the function of text graffiti drawing that can be applied to the reference image in which the drawing part is dark and the background is light, and the function of text graffiti drawing that can be applied to the reference image in which the drawing part is light and the background is dark, rather than the existing case where the drawing part of the reference image is only dark and the background is light.

[0050] The following is combined with Figure 1-2 , a specific embodiment of the method for generating a text graffiti image provided by the present invention is described.

[0051] Figure 1 The flowchart of the method 100 for generating a text graffiti image according to an embodiment of the present disclosure is shown. The method 100 includes:

[0052] Step 110, receiving a text image to be graffitied and a graffiti style reference image uploaded by a user.

[0053] In some embodiments, Figure 2 As shown, the user wants to generate a style reference image of a little girl and lotus (zoom in to see that some lotus leaves form a word Zhao), and a graffiti image of the word "Li".

[0054] Step 120, performing image coding processing on the graffiti style reference image to obtain an image coding vector.

[0055] In some embodiments, the IP-Adapter algorithm is used to encode the graffiti style reference image to obtain an image encoding vector. The IP-Adapter algorithm (image prompt adapter) is a method that enables the text-to-image diffusion model to have the ability to generate image prompts. It works in a simple and practical way, avoiding the need for traditional model fine-tuning with a lot of time and computing resources. The core of IP-Adapter lies in its image prompt adapter, which enables the model to generate corresponding images based on the provided image prompts. The working principle of IP-Adapter mainly includes several key parts: 1) Image encoder: The image encoder of CLIP (Contrastive Language-Image Pre-training Model) is used to extract the image features of the image prompt, and the image embedding is projected into a feature sequence through a small linear layer. Decoupled attention mechanism: In the original SD model, the text features are extracted by the text encoder of CLIP and injected into the cross-attention mechanism layer to complete the semantic alignment of the text prompt and the generated image. The decoupled attention mechanism proposed by IP-Adapter regards the image prompt as text, calculates the cross-attention mechanism of the image prompt and the query feature according to the formula principle, and finally adds the cross-attention mechanism calculated by the text prompt and the cross-attention mechanism calculated by the image prompt. Through these mechanisms, IP-Adapter is able to achieve rapid adaptation and efficient generation of text-to-image diffusion models without extensive computation and model fine-tuning. This approach greatly reduces resource consumption and time costs while maintaining model generation capabilities, making it easier and more efficient to generate images of a specific style or theme.

[0056] Step 130: input the to-be-graffitied text image and the picture encoding vector into a pre-trained text graffiti image generation model, and output a text graffiti image in a graffiti style that conforms to the graffiti style reference image.

[0057] In some embodiments, the pre-trained text graffiti image generation model includes: a trained ControlNet model and a UNet model. Figure 2The part marked by the virtual box in the figure is the composition of the text graffiti image generation model. The UNet model here does not need to be trained in this solution, and can be used with the help of the existing UNet model that has been trained by others. In this solution, it is the ControlNet model that needs to be retrained. The ControlNet model is trained by the following steps: obtaining a first training sample and a second training sample as a training sample set; the first training sample is composed of a first text portrait image generated by a designer's reference picture and the text to be painted; the second training sample is composed of a second text portrait image generated by a designer's reference picture and the text to be painted; the ControlNet model is trained using the training sample set, and when the model training times reaches the preset times, the training is completed, and the trained model is used as the ControlNet model in the text graffiti image generation model.

[0058] In some embodiments, the process of generating a first text portrait image from a designer reference image and text to be drawn includes: obtaining a designer reference image, and extracting foreground and background colors and text content descriptions from the designer reference image; inputting the text to be drawn into a text-to-image model, and generating a text image from the text to be drawn according to the extracted foreground and background colors; inputting the text image into an image encoding model to obtain an initialized latent space code; inputting the extracted text content description into a text description encoding model to obtain a first feature vector; inputting the designer reference image into an image description encoding model to obtain a second feature vector; inputting the initialized latent space code, the first feature vector, and the second feature vector into an image decoding model to generate a first text portrait image; and using the first text portrait image as a first training sample. Before inputting the initialized latent space code, the first feature vector, and the second feature vector into the image decoding model to generate the first text portrait image, the method further includes: inputting the initialized latent space code, the first feature vector, and the second feature vector into a denoising model for denoising. Extracting foreground and background colors from the designer reference image includes: finding the mode of the center color cluster and the mode of the background area color cluster of the designer reference image through clustering, and using the colors corresponding to the mode of the center color cluster and the mode of the background area color cluster as the foreground and background colors. Extracting text content description from the designer reference image includes: identifying the constituent elements of the designer reference image, adding corresponding element tags to the elements respectively, and outputting the element tags as text content descriptions. The image description encoding model is the Clip image model, and the text description encoding model is the Clip text model.

[0059] In some embodiments, the process of generating a second text portrait image from a designer reference image and a text image to be painted includes: obtaining a designer reference image; inputting the text to be painted into a text-to-image model to obtain an initial text image; adjusting the brightness of the initial text image according to the brightness of the designer reference image to obtain a text image to be painted; performing image encoding processing on the designer reference image to obtain an encoding vector; inputting the text image to be graffitied and the encoding vector into a pre-built text graffiti image generation model to output a second text portrait image; the pre-built text graffiti image generation model includes an initial ControlNet model and a UNet model; and using the second text portrait image as a second training sample. The initial ControlNet model and UNet model here can use the existing ControlNet model and UNet model that have been trained by others as the basic model of the second training sample generation model. Among them, performing image encoding processing on the designer reference image to obtain an encoding vector includes: using the IP-Adapter algorithm to perform image encoding on the designer reference image to obtain an encoding vector. Among them, the process of adjusting the brightness of the initial text image according to the brightness of the designer's reference image can use the existing brightness adjustment tool, but whether the final adjusted result meets the needs can be determined by showing the image with adjusted brightness to the staff for confirmation, or by pre-setting brightness judgment conditions. The setting of the brightness judgment conditions can be set by the staff based on work experience.

[0060] That is, the first training sample and the second training sample are combined as new data inputs for the ControlNet model to train it. After the new ControlNet is trained, the algorithm can adaptively select which part is used as the background color and which part is used as the foreground color according to the template image by only inputting a simple black and white guide image. In other words, the training data (second training sample) is prepared in advance according to the color or brightness contrast, and then the white background and black graffiti are used as the guide signal (first training sample) for training. The model trained in this way can completely remove the guidance of color and contrast, automatically identify which elements are used as the foreground and which elements are used as the background in the reference image, and draw according to the given guide shape.

[0061] In some embodiments, the text image to be graffitied and the image encoding vector are input into the pre-trained text graffiti image generation model as described above, and the corresponding text graffiti image that meets the graffiti style required by the user can be obtained. Figure 2Only a schematic diagram of the process of scribbling on the input black "Li" character on a white background is given. It should be noted that the "Li" character input in this solution is not limited to a white background with black characters or a black background with white characters, that is, it can automatically recognize and match to generate a text scribbling diagram that conforms to the style given in the scribbling style diagram.

[0062] It should be noted that for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the present disclosure is not limited by the described action sequence, because according to the present disclosure, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to the present disclosure.

[0063] The above is the introduction of the method embodiments. The following further illustrates the solution of the present disclosure through device embodiments.

[0064] Figure 3 A block diagram of a text scribbling diagram generation device 300 according to an embodiment of the present disclosure is shown. As Figure 3 shown, the device 300 includes:

[0065] A data receiving module 310, configured to receive a text diagram to be scribbled and a scribbling style reference diagram uploaded by a user;

[0066] An encoding processing module 320, configured to perform picture encoding processing on the scribbling style reference diagram to obtain a picture encoding vector;

[0067] A text scribbling module 330, configured to input the text diagram to be scribbled and the picture encoding vector into a pre-trained text scribbling diagram generation model, and output a text scribbling diagram that conforms to the scribbling style of the scribbling style reference diagram.

[0068] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the described modules can refer to the corresponding processes in the foregoing method embodiments, and will not be elaborated herein.

[0069] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device and a readable storage medium.

[0070] Figure 4A schematic block diagram of an electronic device 400 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0071] The electronic device 400 includes a computing unit 401, which can perform various appropriate actions and processes according to a computer program stored in the ROM 402 or a computer program loaded from the storage unit 408 into the RAM 403. In the RAM 403, various programs and data required for the operation of the electronic device 400 can also be stored. The computing unit 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An I / O interface 405 is also connected to the bus 404.

[0072] Multiple components in the electronic device 400 are connected to the I / O interface 405, including: an input unit 406, such as a keyboard, a mouse, etc.; an output unit 407, such as various types of displays, speakers, etc.; a storage unit 408, such as a disk, an optical disk, etc.; and a communication unit 409, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 409 allows the electronic device 400 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0073] The computing unit 401 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 401 performs the various methods and processes described above, such as method 100. For example, in some embodiments, the method 100 may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 408. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 400 via ROM 402 and / or communication unit 409. When the computer program is loaded into RAM 403 and executed by the computing unit 401, one or more steps of the method 100 described above may be performed. Alternatively, in other embodiments, the computing unit 401 may be configured to perform the method 100 in any other appropriate manner (e.g., by means of firmware).

[0074] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0075] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0076] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0077] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0078] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0079] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0080] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0081] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for generating a text graffiti image, characterized in that: include: Receive text images to be graffitied and graffiti style reference images uploaded by users; Performing image coding processing on the graffiti style reference image to obtain an image coding vector; The graffiti text image and the image encoding vector are input into a pre-trained text graffiti image generation model, and a text graffiti image with a graffiti style that conforms to the graffiti style reference image is output.

2. The method according to claim 1, characterized in that The pre-trained text graffiti image generation model includes: a trained ControlNet model and a UNet model.

3. The method according to claim 2, characterized in that The ControlNet model is trained by the following steps: A first training sample and a second training sample are obtained as a training sample set; the first training sample is composed of a first text image generated from a designer reference image and the text to be drawn; the second training sample is composed of a second text image generated from a designer reference image and the text to be drawn; The ControlNet model is trained using the training sample set. When the number of model training times reaches a preset number, the training is completed, and the trained model is used as the ControlNet model in the text graffiti image generation model.

4. The method according to claim 3, characterized in that The process of generating the first text image from the designer's reference image and the text to be drawn includes: Obtaining a designer reference image, and extracting foreground and background colors and text content descriptions from the designer reference image; Input the text to be drawn into the text-to-image model, and generate a text image from the text to be drawn according to the extracted foreground and background colors; Inputting the text image into a picture coding model to obtain an initialized latent space code; Inputting the extracted text content description into a text description encoding model to obtain a first feature vector; Inputting the designer reference image into an image description coding model to obtain a second feature vector; Inputting the initialized latent space code, the first feature vector and the second feature vector into a picture decoding model to generate a first text image; The first character image is used as a first training sample.

5. The method according to claim 4, characterized in that Before inputting the initialized latent space code, the first feature vector and the second feature vector into the picture decoding model to generate the first text image, the method further includes: The initialized latent space code, the first eigenvector and the second eigenvector are input into a denoising model for denoising.

6. The method according to claim 3, characterized in that The process of generating the second text image from the designer's reference image and the text image to be drawn includes: Get designer reference drawings; Input the text to be drawn into the text-to-image model to obtain an initial text image; The brightness of the initial text image is adjusted according to the brightness of the designer's reference image to obtain the text image to be drawn; Performing image encoding processing on the designer reference image to obtain an encoding vector; Inputting the to-be-graffitied text image and the encoding vector into a pre-constructed text graffiti image generation model, and outputting a second text image; the pre-constructed text graffiti image generation model includes an initial ControlNet model and a UNet model; The second character image is used as a second training sample.

7. The method according to claim 6, characterized in that Perform image encoding processing on the designer reference image to obtain an encoding vector, including: Use the IP-Adapter algorithm to encode the designer's reference image and obtain the encoding vector.

8. A device for generating a text graffiti image, characterized in that: include: A data receiving module, used for receiving a text image to be graffitied and a graffiti style reference image uploaded by a user; A coding processing module, used for performing image coding processing on the graffiti style reference image to obtain an image coding vector; The text graffiti module is used to input the text image to be graffitied and the picture encoding vector into a pre-trained text graffiti image generation model, and output a text graffiti image in a graffiti style that conforms to the graffiti style reference image.

9. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-7.

Images