A paper-cut design pattern reconstruction generation method and system based on deep learning

Abstract

The application relates to the technical field of paper-cut design, and discloses a paper-cut design pattern line reconstruction generation method and system based on deep learning, which comprises the following steps: scribble preprocessing is performed on an original paper-cut design drawing to draw a scribble picture, paper-cut patterns, geometric features and texture features are collected to construct a prior knowledge base; preprocessing features, prior knowledge and latent feature representation are input into a ControlNet-GAN model; a generator of a GAN network analyzes a conditional tensor through a Transformer layer, a Control-UNet layer is integrated into a convolution module and an attention module of the generator, and a reconstructed paper-cut pattern design drawing is generated; a discriminator extracts pattern features in the reconstructed paper-cut pattern design drawing, compares the pattern features with the prior knowledge base, and punishes a generated result that does not conform to paper-cut rules; the application optimizes a ControlNet algorithm, integrates prior knowledge of paper-cut patterns, makes the generated image conform to paper-cut pattern rules, improves the generation efficiency and quality of a paper-cut design drawing, and meets the digital creation needs of a traditional paper-cut process.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of paper-cut design, and particularly relates to a paper-cut design pattern line reconstruction generation method and system based on deep learning. BACKGROUND

[0002] As a traditional folk craft, paper-cut art has unique pattern rules and style patterns, such as crescent moon patterns, zigzag patterns, geometric patterns, etc. Traditional manual paper-cut pattern design is low in efficiency and high in skill requirement for designers. With the application of deep learning in the field of image generation, although text-to-image models such as Stable Diffusion can generate creative images, they are difficult to accurately match the fixed rules of paper-cut patterns, resulting in deficiencies in pattern standardization and style consistency of the generated paper-cut images. The existing ControlNet technology lacks targeted optimization of paper-cut pattern rules when processing prior knowledge in a specific field, and cannot efficiently utilize prior information of paper-cut patterns for image generation and reconstruction. SUMMARY

[0003] The present application aims to solve the above problems, and provides a paper-cut design pattern line reconstruction generation method and system based on deep learning.

[0004] The first aspect of the present application provides a paper-cut design pattern line reconstruction generation method based on deep learning, which comprises the following steps:

[0005] scribble preprocessing is performed on the original paper-cut design to draw a scribble picture, and paper-cut patterns, geometric features and texture features are collected to construct a prior knowledge base;

[0006] The preprocessed features, prior knowledge and latent feature representation are input into a ControlNet-GAN model;

[0007] The generator of the GAN network parses the conditional tensor through a Transformer layer, integrates a Control-UNet layer into the convolution module and the attention module of the generator, and generates a reconstructed paper-cut pattern design picture;

[0008] The discriminator extracts the pattern features in the reconstructed paper-cut pattern design picture, compares them with the prior knowledge base, and punishes the generated results that do not comply with the paper-cut rules.

[0009] Optionally, in the first implementation manner of the first aspect of the present application, the scribble preprocessing is performed on the original paper-cut design to draw a scribble picture, and paper-cut patterns, geometric features and texture features are collected to construct a prior knowledge base, which comprises:

[0010] Draw a sketch on the original paper-cut design, mark the area to be modified, perform morphological closing operation and Gaussian blur on the sketch, and smooth the edges;

[0011] Collect typical paper-cut patterns to build an indexable pattern template library, and label geometric parameters for each pattern;

[0012] Extract the symmetry mode and repeated unit of the paper-cut, model them as a transformation matrix and a texture feature library, extract texture statistical features through a Gabor filter and a local binary pattern, and build a priori knowledge base.

[0013] Optionally, in the second implementation manner of the first aspect of the present application, the input of the pre-processing features, the priori knowledge embedding and the latent feature representation into the ControlNet-GAN model comprises:

[0014] The sketch is converted into a semantic mask, different color regions correspond to different modification instructions, feature extraction is performed on the semantic mask, and the semantic mask is converted into a high-dimensional feature vector to obtain pre-processing features;

[0015] The pattern templates in the priori knowledge base are first converted into learnable embedding vectors, and then the geometric features and the texture features are encoded into embedding vectors, which are fused with the pre-processing features to obtain the priori knowledge embedding, wherein each pattern corresponds to a unique vector representation;

[0016] The low-dimensional vector obtained after the original paper-cut design is encoded by the Encoder component of the VAE is the latent feature representation.

[0017] Optionally, in the third implementation manner of the first aspect of the present application, the generator of the ControlNet-GAN model is integrated with a ControlNet structure, the ControlNet structure comprises a UNet backbone, a trainable copy branch and a zero-initialized convolutional layer, wherein the UNet backbone is obtained using a pre-trained T2I model, and the trainable copy branch comprises a Transformer layer and a Control-UNet layer.

[0018] Optionally, in the fourth implementation manner of the first aspect of the present application, the generator of the GAN network obtains a conditional tensor through the Transformer layer, integrates the Control-UNet layer into the convolutional module and the attention module of the generator, and generates a modified paper-cut pattern design, comprising:

[0019] The basic features of the original paper-cut design are extracted through the UNet backbone;

[0020] The Transformer layer receives the pre-processed features, priori knowledge embedding and latent vector representation, analyzes the relationship through a self-attention mechanism, generates a conditional tensor, and captures the correlation between the scribble area and the target pattern and geometric rules.

[0021] The Control-UNet layer performs multi-scale processing on the conditional tensor to generate control features at different resolutions.

[0022] The control features generated by the Control-UNet layer are processed through a zero-initialized convolution layer, and the control features are added to the base features.

[0023] In areas requiring specific patterns, the influence of the corresponding pattern template is enhanced through attention weights, and the feature generation of the symmetric area is forced to maintain symmetry. After color adjustment and post-processing, the modified paper-cut pattern design graph is output.

[0024] Optionally, in the fifth implementation manner of the first aspect of the present application, the Control-UNet layer performs multi-scale processing on the conditional tensor to generate control features at different resolutions, including:

[0025] The symmetry center and the main layout structure are determined in the low-resolution layer in combination with priori geometric knowledge, and the local pattern is refined in the high-resolution layer based on the pattern knowledge.

[0026] Optionally, in the sixth implementation manner of the first aspect of the present application, the feature of the modified paper-cut pattern design graph is extracted by the discriminator, compared with the priori knowledge base, and the generated result that does not conform to the paper-cut rule is punished, including:

[0027] Edge and contour features are extracted from the modified paper-cut pattern design graph, compared with the priori knowledge base, and it is checked whether the geometric rules of specific patterns are met;

[0028] The symmetry degree of the generated image and the regularity of the repeating unit are detected, and the density, direction consistency of the generated texture are analyzed;

[0029] For the area that does not conform to the paper-cut rule, a high penalty is given through a loss function to guide the generator to improve.

[0030] The second aspect of the present application provides a paper-cut design graph pattern modification and generation system based on deep learning, which comprises:

[0031] A pre-processing module is configured to perform scribble pre-processing on an original paper-cut design graph to draw a scribble graph, and collect paper-cut patterns, geometric features and texture features to construct a priori knowledge base.

[0032] An input module is configured to input the pre-processed features, priori knowledge embedding and latent feature representation into the ControlNet-GAN model.

[0033] The generation module is configured to parse the conditional tensor by a generator of the GAN network through a Transformer layer, integrate a Control-UNet layer into a convolution module and an attention module of the generator, and generate a modified paper-cut pattern design diagram.

[0034] The comparison module is configured to extract pattern features in the modified paper-cut pattern design diagram by a discriminator, compare the pattern features with the prior knowledge base, and punish the generated result that does not conform to the paper-cut rule.

[0035] The third aspect of the present application provides a deep learning-based paper-cut design diagram pattern modification generation device, which comprises a memory and at least one processor, and the memory stores instructions; the at least one processor invokes the instructions in the memory to enable the deep learning-based paper-cut design diagram pattern modification generation device to perform the steps of the deep learning-based paper-cut design diagram pattern modification generation method according to any one of the above.

[0036] The fourth aspect of the present application provides a computer-readable storage medium, which stores instructions, and the instructions are executed by a processor to implement the steps of the deep learning-based paper-cut design diagram pattern modification generation method according to any one of the above.

[0037] In the technical solution provided by the present application, scribble preprocessing is performed on an original paper-cut design diagram to draw a scribble diagram, and a prior knowledge base is constructed by collecting paper-cut patterns, geometric features and texture features; the preprocessed features, the prior knowledge and the latent feature representation are input into a ControlNet-GAN model; a generator of the GAN network parses a conditional tensor through a Transformer layer, integrates a Control-UNet layer into a convolution module and an attention module of the generator, and generates a modified paper-cut pattern design diagram; pattern features in the modified paper-cut pattern design diagram are extracted by a discriminator, and the pattern features are compared with the prior knowledge base to punish the generated result that does not conform to the paper-cut rule; the present application integrates the prior knowledge of paper-cut patterns by optimizing the ControlNet algorithm, so that the generated image conforms to the paper-cut pattern rule, improves the generation efficiency and quality of the paper-cut design diagram, and meets the digital creation needs of the traditional paper-cut process; the generated image strictly follows the paper-cut pattern rule through the prior knowledge base and the optimized ControlNet module, and the standardization of the paper-cut design is improved; the automatic generation process replaces manual design, greatly shortens the paper-cut design cycle, supports batch generation of paper-cut design diagrams of different styles, can generate multiple paper-cut styles, meets diversified creation needs, and the ControlNet optimization module can be flexibly integrated into different T2I models, adapts to multiple deep learning frameworks, and is convenient for technical iteration and cross-field application. Attached Figure Description

[0038] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention.

[0039] Figure 1 A flowchart illustrating a method for generating paper-cutting design patterns based on deep learning, provided in an embodiment of the present invention.

[0040] Figure 2 This is a schematic diagram of the structure of the ControlNet-GAN model provided in an embodiment of the present invention;

[0041] Figure 3 This is a schematic diagram of the structure of the deep learning-based paper-cutting design pattern modification and generation device provided in an embodiment of the present invention. Detailed Implementation

[0042] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” or “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, apparatus, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0043] For ease of understanding, the specific process of the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 1 The flowchart of the deep learning-based paper-cutting design pattern modification and generation method provided in this embodiment of the invention includes the following steps:

[0044] Step 101: Perform Scribble preprocessing on the original paper-cutting design to create a doodle, and collect paper-cutting patterns, geometric features, and texture features to build a prior knowledge base;

[0045] In this embodiment, in the traditional paper-cut design, the manual drawing of the line is not only low in efficiency, but also needs to rely on experienced artists to draw doodles on the original paper-cut picture, mark the areas to be transformed, such as adding sawtooth lines by outlining the edges with red, and then perform morphological closing operation and smoothing processing on the doodle picture to eliminate the discontinuous points of hand-drawing, generate a machine-recognizable semantic mask, at the same time, collect 1000+ typical paper-cut patterns such as sawtooth lines and cloud patterns, construct a pattern template library, and label the geometric parameters such as sawtooth angle and density; for the symmetry mode specific to paper-cut such as left-right symmetry and the repeating unit such as continuous floral pattern, the transformation rule such as the translation vector is extracted to form a geometric library, in addition, the texture features such as density and direction are quantified through Gabor filter and local binary pattern to construct a texture library.

[0046] In this embodiment, doodles are drawn on the original paper-cut design picture to mark the areas to be transformed, morphological closing operation and Gaussian blur are performed on the doodle picture to smooth the edges; typical paper-cut patterns are collected to construct an indexable pattern template library, and geometric parameters are labeled for each pattern; the symmetry mode and repeating unit of the paper-cut are extracted to model as a transformation matrix and a texture feature library; texture statistical features are extracted through Gabor filter and local binary pattern to construct a priori knowledge base.

[0047] In the process of constructing the paper-cut texture library, the Gabor filter and the local binary pattern are combined to accurately quantify the texture features of the paper-cut. First, a Gabor filter bank is used, which includes 8 directions and 4 scales, to perform convolution operation on the paper-cut image. Each filter can capture texture information of a specific direction, such as horizontal, vertical, 45° angle, and frequency, such as coarse texture and fine texture, to generate a multi-channel feature map. For example, when processing sawtooth lines, the Gabor filter of a specific direction will enhance the response of the sawtooth edge, forming obvious bright and dark stripes. Then, the mean and variance of the Gabor feature map are calculated to represent the average intensity and variation amplitude of the texture, respectively. For example, dense sawtooth lines will show higher mean and smaller variance.

[0048] At the same time, the local binary pattern (LBP) is applied to further extract the microstructure features of the texture. The LBP operator compares the center pixel with its neighborhood pixels to generate a binary code, describing the local texture pattern (such as smooth, edge, and corner). For paper-cut images, LBP can effectively distinguish the detailed features of different patterns, such as the sharp edges of sawtooth lines and the smooth curves of cloud patterns. The LBP code is histogram-stated to extract the distribution features of the texture. For example, regular repeating patterns will show a concentrated histogram distribution, while irregular textures have a more dispersed histogram,

[0049] Finally, the Gabor features and LBP histogram features are concatenated to form high-dimensional texture vectors. These vectors are classified by a clustering algorithm to construct an indexable texture library. For example, the texture vectors are clustered into categories such as "dense zigzag texture" and "sparse cloud texture". Typical parameters such as average direction and density value are labeled for each category. During the generation phase, the model can accurately control the texture generation of specific areas based on these texture feature vectors, ensuring that the generated paper-cut patterns meet the characteristics of traditional techniques.

[0050] Step 102, input the pre-processing features, priori knowledge embedding, and latent feature representation into the ControlNet-GAN model;

[0051] In this embodiment, the graffiti is converted into a semantic mask, with different color regions corresponding to different modification instructions. Feature extraction is performed on the semantic mask, and a high-dimensional feature vector is obtained as the pre-processing feature. First, the pattern templates in the priori knowledge base are converted into learnable embedding vectors. Then, the geometric features and texture features are encoded into embedding vectors and fused with the pre-processing features to obtain the priori knowledge embedding. Each pattern corresponds to a unique vector representation. The low-dimensional vector obtained after encoding the original paper-cut design drawing by the Encoder component of the VAE is the latent feature representation.

[0052] In this embodiment, the low-dimensional vector obtained after encoding the original paper-cut design drawing by the Encoder component of the VAE is the latent feature representation. This latent vector, together with the pre-processing feature and priori knowledge embedding, constitutes the input of the ControlNet-GAN framework. By decoupling control and creation, it ensures that global structures such as symmetry and geometric rules are strictly constrained by the semantic mask and knowledge base, while allowing local details such as zigzag texture density and floral pattern arrangement to explore diversity within the rules. Unlike the traditional latent vector-dominated generation direction in the text-to-image model, the ControlNet structure deeply binds randomness and priori knowledge: the latent vector is only responsible for the randomness of details such as local texture density changes, while the global design relies on the graffiti and knowledge base templates, thereby achieving controllable generation while meeting the rules of paper-cut techniques.

[0053] In this embodiment, the generator of the ControlNet-GAN model is composed of the following components:

[0054] UNet backbone: based on a pre-trained T2I model (such as Stable Diffusion), responsible for extracting basic features of the original paper-cut design drawing (such as edges and contours);

[0055] ControlNet module: including a trainable replica branch (containing a Transformer layer and a UNet branch) and a zero-initialized convolution layer;

[0056] Generation logic: the generator generates control features through the ControlNet module and fuses them with the base features of the UNet backbone, finally outputting the modified paper-cut design.

[0057] In this embodiment, the digital processing of the graffiti map consists of two key steps: first, semantic mask generation, that is, different colors are used to mark different modification requirements, for example, the red channel marks the sawtooth modification area, the green channel marks the flower pattern filling area, and the blue channel marks the symmetry axis; this multi-channel mask not only preserves the user's modification intention, but also realizes the hierarchical management of requirements through color coding. Subsequently, the mask is extracted through a convolutional network. The convolution operation can capture the spatial position of the graffiti area, such as the information that the sawtooth should appear on the right edge of the image, the shape outline, such as the circular flower pattern area, and finally output a preprocessed feature vector containing spatial coordinates and semantic labels, providing accurate positional references for the model to locate the modification area.

[0058] In this embodiment, the UNet backbone directly uses the pre-trained Stable Diffusion model, retaining its powerful ability to extract basic features such as image edges, contours, and colors, avoiding repeated training and improving efficiency. For example, it can accurately identify the human contours and flower shapes in the original paper-cut;

[0059] The trainable copy branch contains two core components:

[0060] Transformer layer: responsible for fusing user graffiti, prior knowledge, and latent vectors to generate modification instruction tensors, such as adding a 30° sawtooth pattern to the edge of a human contour with medium density;

[0061] Control-UNet layer: multi-scale processing of instruction tensors, planning global layout at low resolution and refining local patterns at high resolution, and fusing through zero-initialized convolution layers with the base features of the UNet backbone;

[0062] This design allows the model to only adjust branch parameters during training, retaining the image generation capabilities of Stable Diffusion while allowing targeted learning of professional paper-cut modification knowledge, ensuring that the generated results not only conform to general image rules but also meet the special rules of paper-cut art.

[0063] The UNet backbone uses pre-trained T2I model weights, and the frozen UNet backbone is used to extract the visual base features of the original paper-cut design, such as edges, contours, color distribution, and global structure. These features are the bottom-level semantic representation of image generation, and the general visual rules learned by the T2I model in natural images can be reused without retraining;

[0064] The UNet backbone parameters are kept unchanged to avoid destroying the learned image generation capability, the control features generated by the ControlNet trainable auxiliary branch, such as the geometric rules of the paper-cut pattern and the texture features, are fused with the basic features of the UNet backbone through zero-initialized convolution layers, and the design allows the model to learn how to adjust the generation process according to the prior knowledge in the paper-cut field, such as the symmetry mode and the pattern template, without changing the backbone.

[0065] In step 103, the generator of the GAN network parses the conditional tensor through the Transformer layer, integrates the Control-UNet layer into the convolution module and the attention module of the generator, and generates the modified paper-cut pattern design diagram.

[0066] In this embodiment, the UNet backbone extracts the basic features of the original paper-cut design diagram; the Transformer layer receives the preprocessed features, prior knowledge embedding and latent vector representation, analyzes the relationship through the self-attention mechanism, generates the conditional tensor, and captures the relationship between the doodle area and the target pattern and geometric rules; the Control-UNet layer performs multi-scale processing on the conditional tensor and generates control features at different resolutions; the control features generated by the Control-UNet layer are processed through zero-initialized convolution layers, and the control features and the basic features are added; in the area where a specific pattern is needed, the influence of the corresponding pattern template is enhanced through attention weight, and the feature generation of the symmetric area is forced to maintain symmetry, and after color adjustment and post-processing, the modified paper-cut pattern design diagram is output; the Transformer layer receives image preprocessing features, prior knowledge embedding and latent feature representation, the preprocessing features such as the semantic mask of the doodle image, the prior knowledge embedding such as the geometric / texture vector of the pattern template, and the latent feature representation analyzes the relationship between these image features through the self-attention mechanism to generate a conditional tensor for guiding the generator to apply target patterns and geometric rules in specific areas. The input of the Transformer is image-related features such as doodle images and pattern template embeddings, rather than text descriptions, and its core function is to associate the user's modification requirements such as zigzag patterns and symmetry with the rules in the prior knowledge base to generate control signals that meet the paper-cut rules.

[0067] In this embodiment, the symmetric center and the main layout structure are determined in the low-resolution layer combined with prior geometric knowledge, and the local pattern is refined in the high-resolution layer based on the pattern knowledge.

[0068] In this embodiment, the Control-UNet layer workflow is divided into two stages:

[0069] Low-resolution global planning, i.e., 64x64 and below: combined with the symmetry patterns in the geometry library and the repeating unit rules, the macrostructure of the paper-cut is determined, for example, if the condition tensor indicates that "the whole needs to be left-right symmetric", Control-UNet will label the symmetry axis position at 64x64 resolution, and generate a symmetric basic layout in the left and right areas, such as the left side is the main pattern of flowers, and the right side synchronously generates the mirror image of flowers;

[0070] High-resolution local refinement, i.e., 128x128 and above:

[0071] As the resolution increases to 512x512, Control-UNet switches to the detail carving mode, and refines the local pattern based on the specific pattern template in the pattern library. For example, in the planned right edge area, according to the dense zigzag pattern instruction in the condition tensor, the zigzag pattern template with a pitch of 2 pixels, a height of 5 pixels, and an angle of 45° is retrieved from the pattern library, and a zigzag edge conforming to the parameters is generated pixel by pixel, ensuring that the angle and spacing of each tooth tip strictly follow the prior rules;

[0072] During generation, the attention mechanism enhances the generation weight of specific patterns, such as strengthening the zigzag pattern template in the marked area:

[0073] Pattern weight enhancement: when the generator processes the area marked by the user's doodle, the attention mechanism automatically enhances the weight of the corresponding pattern template, for example, in the feature map of the border area, the model calculates the similarity between each pixel and the zigzag pattern template vector, and the area with high similarity will get higher generation priority, ensuring that the zigzag pattern is strictly limited within the target area;

[0074] Symmetry area constraint: for the symmetric areas defined in the geometry library, the attention mechanism will force the establishment of cross-area dependency, when generating the flower pattern in the left half, the corresponding position in the right half will directly copy the features of the left half, ensuring that the number of petals and arrangement angles are completely symmetric, this mechanism avoids the common symmetry breaking problem in traditional generation models;

[0075] Control features are fused with UNet main features through zero-initialized convolution layers, ensuring that the impact is small in the early stage of training, and gradually learning control ability, the zero-initialized convolution layer is initialized to zero in the early stage of training, and the control features generated by Control-UNet have almost no impact on the UNet main features, the model mainly relies on the pre-training ability of StableDiffusion to generate basic images, avoiding the generation collapse caused by forcibly introducing paper-cut rules. As the training progresses, the weights of the zero-initialized convolution layer are gradually learned and adjusted, and the influence of the control features gradually increases, at the 10th epoch, the control features only affect 10% of the generation process; at the 100th epoch, the influence increases to 90%.

[0076] Finally, after color adjustment and post-processing, the output is a high-quality paper-cut design that meets user needs,

[0077] Color adjustment: For the saturated color system commonly used in paper-cut art, such as red, yellow, and green, the model automatically matches the traditional color matching scheme through the color mapping module, for example, mapping the generated gray sawtooth edge to red and the central floral pattern to gold.

[0078] Edge smoothing: Using non-sharpening masks and morphological filtering, the generated process can eliminate the jagged edges that may appear, making the lines smoother and more natural.

[0079] Detail enhancement: By adjusting the local contrast, the details of key patterns such as the tips of sawtooth patterns and the curvature of crescent patterns are highlighted.

[0080] In this embodiment, the advantage of the ControlNet algorithm is that it can alleviate overfitting and catastrophic forgetting by limiting the number or level of trainable parameters, while enhancing the robustness of small and large datasets.

[0081] Step 104: Extract the pattern features from the modified paper-cut pattern design through the discriminator, compare them with the prior knowledge base, and punish the generated results that do not conform to the paper-cut rules.

[0082] In this embodiment, the edge and contour features are extracted from the modified paper-cut pattern design, compared with the prior knowledge base, and checked for compliance with the geometric rules of specific patterns; the symmetry and regularity of the generated image are detected, and the density, direction consistency of the generated texture are analyzed; for areas that do not conform to the paper-cut rules, a high penalty is given through the loss function to guide the generator to improve.

[0083] In this embodiment, the Canny operator or morphological edge detection algorithm is used to extract the contour lines of the generated image, such as the connecting lines of the tips of sawtooth patterns and the petal edges of floral patterns, and convert them into binary edge maps; for sawtooth patterns, the distance between adjacent teeth and the angle of the teeth are calculated; for crescent patterns, the radius of curvature and the smoothness of the curve are measured. These parameters are directly related to the pre-set geometric rules in the pattern library, such as "standard sawtooth pattern tooth spacing is 3-5 pixels, angle 45°±10°";

[0084] The edge features extracted from the template are matched with the templates in the pattern library, such as detecting whether the generated sawtooth pattern has inconsistent tooth spacing or chaotic tooth tip angles that do not conform to the template parameters; if the tooth spacing in a certain area exceeds the pre-set range, it is determined that the pattern has been deformed, triggering the penalty mechanism; at the same time, the symmetry and regularity of the repeating units of the image are detected.

[0085] For the region marked as left-right symmetry, the image is divided into left and right parts along the symmetry axis. The matching degree of left and right edges is detected by pixel-by-pixel comparison and structural similarity calculation. For example, if there are 5 petals on the left side of the floral pattern, the corresponding position on the right side should also have 5 petals, and the angle deviation of the petals should not exceed 5°, otherwise it is determined that the symmetry is broken;

[0086] For continuous patterns such as the back-to-back frame, a sliding window is used to extract the repeating unit, and the spacing error between adjacent units is calculated. For example, if the standard spacing is 10 pixels with a fluctuation of ±1 pixel, if the spacing of the back-to-back frame in a certain section suddenly expands to 15 pixels, it is determined that the regularity is destroyed, prompting the generator to adjust the repeating pattern in this region.

[0087] In this embodiment, the discriminator generates a corresponding mask when it detects a violation region and calculates the loss weight of the region. The more serious the violation, the higher the weight. Based on the traditional adversarial loss, three special loss terms are added, such as pattern loss, geometric loss, and texture loss. For example, the pattern loss is the Hausdorff distance between the generated edge and the template, the geometric loss is the total pixel difference of the symmetric region, and the texture loss is the deviation value of density and direction. High penalty signals are propagated to the generator through gradient backpropagation, forcing it to adjust the generation parameters in the corresponding region. For example, when the sawtooth pattern tooth spacing is detected to be too large, the generator's convolution kernel weight in that region will be dynamically adjusted to reduce the tooth spacing value until it meets the pattern library standard. For example, if the generated sawtooth pattern tooth spacing exceeds the reasonable range, the discriminator will intensify the modification signal in this region.

[0088] In this embodiment, the generator gradually learns to create under the rules of the game after hundreds of rounds of training. For example, when the user requires a symmetric sawtooth frame, the generator will automatically select three tooth spacing parameters from the pattern library and combine them with random perturbations of the latent vector to generate three symmetric frames with different densities, satisfying the core rules of symmetry and sawtooth pattern while providing designers with a variety of choices.

[0089] Figure 2 The structure diagram of the ControlNet-GAN model provided by the embodiment of the present application is shown. The model is composed of a generator and a discriminator. The generator can integrate prior knowledge into the original image that does not meet the specifications for modification, and the generated result image is input to the discriminator. The discriminator compares the generated result image with the standard paper-cut sample image, identifies the part that does not meet the standard, and transmits it to the generator in the form of a penalty signal to guide the generator to generate a new image in the correct direction.

[0090] Generator process: The UNet backbone is responsible for extracting the basic features of the original paper-cut design, such as edges, contours, etc. The Transformer layer analyzes the relationship between the pre-processed features, prior knowledge embedding, and latent feature representation through self-attention mechanisms, generates a conditional tensor, and captures the association between the doodle area and the target pattern. The Control-UNet layer processes the conditional tensor at multiple scales, determines the symmetry center and layout based on low-resolution and geometric knowledge, and refines the local pattern based on high-resolution and pattern knowledge. After the control features are processed through zero-initialized convolution layers, they are fused with the basic features, and then the attention mechanism is used to enhance the weight of specific patterns and constrain the symmetry area. Finally, the modified image is output after post-processing.

[0091] Discriminator process: The modified paper-cut image is compared with the standard paper-cut image in the sample library, and the pattern features of the generated image are extracted through feature extraction modules such as edge detection and texture analysis. The geometric parameters, symmetry, and regularity of the repeating units are checked. For areas that do not meet the rules, the loss function is used to calculate the penalty weight, and the penalty signal is fed back to the generator.

[0092] In this embodiment, the discriminator extracts the pattern features of the generated image, such as symmetry and texture density, and compares them with the prior knowledge base. The loss function includes pattern loss, geometric loss, and texture loss, and the penalty signal is propagated back to the generator. The generator adjusts the generation strategy of the control features based on the feedback from the discriminator, such as enhancing the attention weight of specific patterns, to gradually improve the generation results. The generator and discriminator interact through the adversarial loss function, such as the loss of Wasserstein GAN: the goal of the generator is to generate images that meet the paper-cut rules, while the goal of the discriminator is to distinguish between generated images and real paper-cut images. Through dynamic game, both sides optimize together, and finally achieve the generation of high-quality and rule-compliant paper-cut design images.

[0093] Figure 3is a structural schematic view of a deep learning-based paper-cut design pattern reconstruction generation device provided by an embodiment of the present application. The deep learning-based paper-cut design pattern reconstruction generation device 300 can have relatively large differences due to different configurations or performances, and can include one or more processors (central processing units, CPUs) 310 (for example, one or more processors) and a memory 320, one or more storage media 330 (for example, one or more mass storage devices) storing application programs 333 or data 332. The memory 320 and the storage media 330 can be temporary storage or persistent storage. The programs stored in the storage media 330 can include one or more modules (not shown in the figure), and each module can include a series of instruction operations in the deep learning-based paper-cut design pattern reconstruction generation device 300. Furthermore, the processor 310 can be configured to communicate with the storage media 330 and execute a series of instruction operations in the storage media 330 on the deep learning-based paper-cut design pattern reconstruction generation device 300 to implement the method provided by the above embodiment.

[0094] The deep learning-based paper-cut design pattern reconstruction generation device 300 can also include one or more power supplies 340, one or more wired or wireless network interfaces 350, one or more input and output interfaces 360, and / or one or more operating devices 331, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, and the like. Those skilled in the art can understand that, Figure 3 The deep learning-based paper-cut design pattern reconstruction generation device structure shown does not constitute a limitation on the computer device provided by the present application, and can include more or fewer components than shown, or combine certain components, or different component arrangements.

[0095] The present application also provides a computer readable storage medium, which can be a non-volatile computer readable storage medium or a volatile computer readable storage medium. The computer readable storage medium has instructions stored therein, and when the instructions are run on a computer, the computer executes the steps of the deep learning-based paper-cut design pattern reconstruction generation method provided by the above embodiments.

[0096] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described device or apparatus, unit can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0097] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or say the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0098] The basic principles, main features and advantages of the present application are shown and described above. Those skilled in the art should understand that the present application is not limited by the above embodiments, and the above embodiments and descriptions in the specification are only preferred examples of the present application and are not intended to limit the present application. Without departing from the spirit and scope of the present application, various changes and improvements can be made to the present application, and these changes and improvements all fall within the scope of the claimed present application. The scope of protection of the present application is defined by the appended claims and their equivalents.

Claims

1. A deep learning-based paper-cut design pattern line reconstruction generation method, characterized in that, The method comprises the following steps: scribble preprocessing is performed on the original paper-cut design to draw a scribble diagram, and paper-cut patterns, geometric features and texture features are collected to construct a priori knowledge base; The preprocessed features, priori knowledge embedding and latent feature representation are input into a ControlNet-GAN model; The generator of the GAN network parses the conditional tensor through a Transformer layer, integrates a Control-UNet layer into the convolution module and the attention module of the generator, and generates a reformed paper-cut pattern design; The discriminator extracts the pattern features in the reformed paper-cut pattern design, compares them with the priori knowledge base, and punishes the generated results that do not conform to the paper-cut rules; The preprocessed features, priori knowledge embedding and latent feature representation are input into a ControlNet-GAN model, comprising: The scribble diagram is converted into a semantic mask, different color regions correspond to different transformation instructions, the semantic mask is subjected to feature extraction, and is converted into a high-dimensional feature vector to obtain preprocessed features; The pattern templates in the priori knowledge base are first converted into learnable embedding vectors, and then the geometric features and texture features are encoded into embedding vectors, which are fused with the preprocessed features to obtain priori knowledge embedding, wherein each pattern corresponds to a unique vector representation; The low-dimensional vector obtained after the original paper-cut design is encoded by the Encoder component of the VAE is a latent feature representation; The generator in the ControlNet-GAN model integrates a ControlNet structure, the ControlNet structure comprises a UNet backbone, a trainable copy branch and a zero-initialized convolution layer, wherein the UNet backbone is obtained using a pre-trained T2I model, and the trainable copy branch comprises a Transformer layer and a Control-UNet layer; The generator of the GAN network parses the conditional tensor through a Transformer layer, integrates a Control-UNet layer into the convolution module and the attention module of the generator, and generates a reformed paper-cut pattern design, comprising: The UNet backbone extracts the basic features of the original paper-cut design; The Transformer layer receives the preprocessed features, priori knowledge embedding and latent vector representation, parses the relationship through a self-attention mechanism, generates a conditional tensor, and captures the association between the scribble area and the target pattern and geometric rules; The Control-UNet layer performs multi-scale processing on the conditional tensor to generate control features at different resolutions; The control features generated by the Control-UNet layer are processed through the zero-initialized convolution layer, and the control features are added to the basic features; In the area where a specific pattern is needed, the influence of the corresponding pattern template is enhanced through attention weight, the symmetry of the feature generation in the symmetric area is forced to remain symmetric, and after color adjustment and post-processing, a reformed paper-cut pattern design is output.

2. The paper-cut design pattern reconstruction generation method based on deep learning according to claim 1, wherein, The scribble preprocessing is performed on the original paper-cut design to draw a scribble diagram, and paper-cut patterns, geometric features and texture features are collected to construct a priori knowledge base, comprising: Draw scribble on the original paper-cut design, mark the area to be modified, perform morphological closing operation and Gaussian blur on the scribble, and smooth the edges; Collect typical paper-cut patterns to build an indexable pattern template library, and label geometric parameters for each pattern; Extract the symmetry mode and repeating unit of the paper-cut, model them as transformation matrices and texture feature library, extract texture statistical features through Gabor filter and local binary pattern, and build a priori knowledge base. 3.The paper-cut design pattern reconstruction generation method based on deep learning of claim 1, wherein, The Control-UNet layer performs multi-scale processing on the conditional tensor to generate control features at different resolutions, including: At the low-resolution layer, determine the symmetry center and main layout structure in combination with prior geometric knowledge, and at the high-resolution layer, refine the local pattern based on the pattern knowledge.

4. The method for generating paper-cutting design patterns based on deep learning as described in claim 1, characterized in that, The features of the modified paper-cut pattern design are extracted through the discriminator, compared with the priori knowledge base, and the generated results that do not conform to the paper-cut rules are punished, including: Extract the edge and contour features from the modified paper-cut pattern design, compare them with the priori knowledge base, and check whether they conform to the geometric rules of the specific pattern; Detect the symmetry and regularity of the repeating unit of the generated image, and analyze the density and direction consistency of the generated texture; For areas that do not conform to the paper-cut rules, give high penalty through the loss function to guide the generator to improve.

5. A deep learning-based paper-cut design pattern line reconstruction generation system, characterized in that, The system comprises: A preprocessing module for scribble preprocessing of the original paper-cut design to draw a scribble, and collecting paper-cut patterns, geometric features and texture features to build a priori knowledge base; An input module for embedding the preprocessing features, priori knowledge and latent feature representation into the ControlNet-GAN model: converting the scribble into a semantic mask, different color areas corresponding to different modification instructions, extracting features from the semantic mask and converting them into a high-dimensional feature vector to obtain the preprocessing features; first convert the pattern templates in the priori knowledge base into learnable embedding vectors, then encode the geometric features and texture features into embedding vectors, and fuse them with the preprocessing features to obtain the priori knowledge embedding, where each pattern corresponds to a unique vector representation; the low-dimensional vector obtained after encoding the original paper-cut design through the Encoder component of the VAE is the latent feature representation; The generation module is configured to parse the conditional tensor by a Transformer layer of a generator of a GAN network, integrate a Control-UNet layer into a convolution module and an attention module of the generator, and generate a modified paper-cut pattern design: basic features of an original paper-cut design are extracted by a UNet backbone; the Transformer layer receives preprocessed features, prior knowledge embedding, and latent vector representation, analyzes relationships by a self-attention mechanism, generates a conditional tensor, and captures correlations between a scribble area and a target pattern and geometric rules; the Control-UNet layer performs multi-scale processing on the conditional tensor to generate control features at different resolutions; the control features generated by the Control-UNet layer are processed by a zero-initialized convolution layer, and the control features are added to the basic features; in areas requiring specific patterns, the influence of a corresponding pattern template is enhanced by attention weights, and the feature generation of a symmetric area is forced to maintain symmetry; after color adjustment and post-processing, a modified paper-cut pattern design is output; the generator of the ControlNet-GAN model integrates a ControlNet structure, and the ControlNet structure includes a UNet backbone, a trainable replica branch, and a zero-initialized convolution layer, wherein the UNet backbone is obtained by using a pre-trained T2I model, and the trainable replica branch includes a Transformer layer and a Control-UNet layer. The comparison module is configured to extract pattern features in the modified paper-cut pattern design by a discriminator, compare the pattern features with a prior knowledge base, and punish a generated result that does not comply with paper-cut rules.

6. A device for generating and modifying the patterns of paper-cutting designs based on deep learning, characterized in that, The deep learning-based paper-cut design pattern modification generation device includes a memory and at least one processor, and the memory stores instructions; the at least one processor calls the instructions in the memory to enable the deep learning-based paper-cut design pattern modification generation device to perform each step of the deep learning-based paper-cut design pattern modification generation method according to any one of claims 1-4.

7. A computer-readable storage medium having stored thereon instructions, the computer-readable storage medium comprising: The instructions are executed by the processor to implement each step of the deep learning-based paper-cut design pattern modification generation method according to any one of claims 1-4.

Images