Method for generating clear image with periodicity

Abstract

The invention discloses a method for generating a clear image with periodicity. Modifying the attention mechanism of the diffusion model to be used into a focusing attention mechanism, and inputting a text cue word; generating a central region of the image in a feature space of the diffusion model, and restricting attention to the central region; sampling the central area, and repeatedly copying and splicing at the periphery of the central area until a set step number to obtain an initial pattern; the resolution of the image is improved by using an up-sampling technology; and repairing the image and adding details under the guidance of the text cue word by using an image editing technology, and generating the clear image meeting the target resolution requirement. The usability of the image generation model in the field of printing and dyeing pattern design is improved, so that high-quality stylized patterns can be generated, and the production cost is remarkably reduced.

Description

Technical Field

[0001] The invention belongs to the field of image generation and relates to a method for generating clear images with periodicity. Background Art

[0002] In the printing and dyeing industry, pattern design requires high periodicity and high definition to ensure seamless splicing on large areas of fabric. However, existing pattern design usually relies on manual hand-drawing or digital drawing tools, which consumes a lot of time and human resources, and it is difficult to make flexible and diverse modifications to the image content according to user needs.

[0003] In recent years, generative models, especially image generation technology based on text-to-pattern diffusion models (such as Midjourney and StableDiffusion), have made significant progress. Image generation models have also been widely used due to their diverse generated content and efficient generation process.

[0004] However, existing image generation methods cannot directly generate periodic patterns through text constraints. At the same time, due to limited computing resources, the model cannot directly generate high-resolution patterns. It is necessary to rely on existing super-resolution technology to improve the clarity of the generated patterns. However, this method will amplify artifacts and distortions in the generation process in printing and dyeing applications, resulting in quality degradation.

[0005] Therefore, there is an urgent need for a generation framework that can generate strict periodicity and automatically optimize image resolution to improve the generation quality and reduce the time cost of printing and dyeing design. Summary of the invention

[0006] In view of the above problems, the present invention proposes a method for generating clear images with periodicity. The method includes two core components: a splicing and focusing mechanism and a super-resolution process. A strict periodic pattern is generated by splicing and focusing the feature space during the image generation process, and then a super-resolution process that integrates image editing and super-resolution technology is used to repair the image and improve the image resolution.

[0007] The present invention provides a method for generating a periodic clear image, the steps of which include:

[0008] Modify the attention mechanism of the diffusion model to be used to a focused attention mechanism and input the text prompt word;

[0009] Generate a central region of the image in the feature space of the diffusion model and restrict attention to the central region;

[0010] Sampling the central area, and repeatedly copying and splicing the central area around it until a set number of steps is reached to obtain an initial pattern;

[0011] Use upsampling techniques to increase the resolution of images;

[0012] Image editing technology is used to repair the image and add details under the guidance of text prompt words to generate the clear image that meets the target resolution requirement.

[0013] Furthermore, the modification method is: trimming the corresponding dimensions of the K and V vectors in the self-attention process, retaining only the central area to smooth the splicing of the edge areas.

[0014] Furthermore, in the sampling process, an extended feature vector of each time step is generated by a replacement function and concatenated with the original feature vector.

[0015] Furthermore, the initial pattern is enlarged to a target resolution using a pre-trained diffusion model upsampling technique and segmented into sub-images with overlapping areas, wherein the resolution of the sub-images is within the processing capability of the diffusion model.

[0016] Furthermore, consistency constraints are added to the corresponding parts of the original area and the expanded area of ​​the segmented sub-image.

[0017] Furthermore, it is characterized in that, after repairing the image and adding details, all sub-images are re-joined and consistency constraints are imposed on the overlapping parts of adjacent regions.

[0018] Further, the image resolution is improved and the image is repaired and details are added alternately multiple times until the set image resolution is reached.

[0019] Beneficial effects of the present invention:

[0020] Compared with the prior art, the present invention provides a plug-and-play framework for generating periodic patterns that does not require training, and also provides a matching image processing flow that can simultaneously complete image restoration, generate details, and improve resolution, thereby improving the usability of image generation models in the field of printing and dyeing pattern design, enabling the generation of high-quality stylized patterns and significantly reducing production costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a schematic diagram of a process of an implementation mode of the present invention;

[0022] Figure 2 Schematic diagram of the splicing and focusing mechanism principle in an embodiment of the present invention;

[0023] Figure 3 Schematic diagram of a super-resolution module in an embodiment of the present invention. DETAILED DESCRIPTION

[0024] The embodiment of the present invention provides a method for generating a clear image with periodicity, and the process is shown as follows: Figure 1 shown.

[0025] Step 1: Periodic pattern generation. This step receives the text prompt word input by the user and generates a low-resolution initial pattern using a diffusion model. In this embodiment, the central area is 512 2 Pixels, after expansion is 640 2 To ensure that the generated pattern is periodic, the diffusion model is extended with the following splicing-focusing mechanism.

[0026] Feature space splicing: In this embodiment, the StableDiffusion1.5 model is used as the basic generation model. First, a random Gaussian distribution feature x is sampled in the feature space. T ~N(0,1), the resolution corresponds to the ratio of the size of the initial pattern image to the feature space scaling factor. Define the replacement function S(x t )=x t ′, that is, x t，e1′ ′=x t，e1 , x t,c1′ ′=x t，c1 , and so on, such as Figure 2 As shown, the feature vector x at each time step t in the above generative model sampling process is t The corresponding part of the central area is copied to the outer expansion area to achieve strict consistency. Sampling and splicing are repeated throughout the sampling process until 50 steps of sampling are completed.

[0027] Focused attention mechanism: Modify the standard self-attention mechanism of the diffusion model, and limit the attention range to the effective area in the center of the image to ensure the smoothness of edge splicing. The specific operation is to trim the corresponding dimensions of the K and V vectors in the self-attention process, and only keep the central area (the original area before the feature space is expanded above), and define the focused attention mechanism FocAttention, the calculation formula is Where K′ and V′ are the pruned K and V vectors, and their dimensions correspond to the portion of the center area in the expanded image. The self-attention mechanism is replaced with the above mechanism throughout the sampling process until the sampling completes the specified number of steps. This mechanism does not require additional training and improves the naturalness of pattern generation through lightweight modifications.

[0028] Step 2: Super-resolution and detail reconstruction. To meet the printing and dyeing industry’s demand for pattern resolution, the super-resolution process is used to perform super-resolution and detail restoration on the low-resolution periodic pattern generated in the above steps. The super-resolution module is divided into two stages: the super-resolution stage and the image editing stage. The pattern clarity is enhanced and distortion is restored by alternating. Figure 3 In this example, the original area is 4096 2 Pixels, after expansion is 5120 2 Pixel.

[0029] In the super-resolution stage, upsampling technology is used to enlarge the initial pattern to the target resolution. In this example, the upsampling technology based on the pre-trained diffusion model is used to segment the enlarged high-resolution image into sub-images with overlapping areas, so that the sub-image resolution is within the processing capacity of the diffusion model to facilitate subsequent detail restoration. At the same time, consistency constraints are added to the corresponding parts of the original area and the expanded area of ​​the segmented sub-image, that is, the feature vectors of the corresponding parts remain consistent to ensure that the smoothness and periodicity of the re-stitching remain unchanged.

[0030] In the image editing stage, image editing methods are used to reconstruct the details of super-resolution sub-images under the guidance of text prompts. This method first adds 0.3 intensity noise to the original image, and then uses a diffusion model to gradually sample and denoise during the denoising process to generate structures and details that meet the target resolution requirements. Finally, all sub-images are reassembled, and consistency constraints are imposed on the overlapping parts of adjacent areas (to keep the feature vectors of the corresponding parts consistent) to ensure seamless connection and overall consistency of the pattern.

[0031] The above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. A person skilled in the art may modify or make equivalent substitutions for the technical solutions of the present invention without departing from the spirit and scope of the present invention. The protection scope of the present invention shall be subject to the claims.

Claims

1. A method for generating clear images with periodicity: The attention mechanism of the diffusion model to be used is modified to a focused attention mechanism and a text prompt word is input; a central area of ​​the image is generated in the feature space of the diffusion model, and attention is limited to the central area; Sampling the central area, and repeatedly copying and splicing the central area around it until a set number of steps is reached to obtain an initial pattern; Use upsampling techniques to increase the resolution of images; Image editing technology is used to repair the image and add details under the guidance of text prompt words to generate the clear image that meets the target resolution requirement.

2. The method according to claim 1, characterized in that The modification method is: trimming the corresponding dimensions of the K and V vectors in the self-attention process, retaining only the central area to smooth the splicing of the edge areas.

3. The method according to claim 1, characterized in that In the sampling process, an extended feature vector of each time step is generated by a replacement function and concatenated with the original feature vector.

4. The method according to claim 1, characterized in that: The initial pattern is enlarged to a target resolution using a pre-trained diffusion model upsampling technique and segmented into sub-images with overlapping areas, where the resolution of the sub-images is within the processing capability of the diffusion model.

5. The method according to claim 4, characterized in that: Add consistency constraints to the corresponding parts of the original area and the expanded area of ​​the segmented sub-image.

6. The method according to claim 4 or 5, characterized in that: After inpainting the image and adding details, all sub-images are reassembled and consistency constraints are enforced on the overlapped parts of adjacent regions.

7. The method according to claim 1, characterized in that: The image resolution is increased and the image is repaired and details are added alternately several times until the set image resolution is reached.

8. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the computer program is executed by the processor to implement the steps of the method according to claims 1 to 7.

9. A storage medium storing a computer program, wherein the stored computer program implements the steps of the method according to claims 1 to 7 when executed.

Images