A method for automatically generating specific artistic style images driven by learning from a small number of samples

Abstract

A method for automatically generating specific art style images driven by small sample learning is provided. [Solution] A method for generating a specific artistic style image that meets requirements based on small sample learning by introducing a style intensity adjustment gate into a style decoupling encoder, includes the steps of selecting a specific artistic style; constructing and training a three-stage mixture model, which includes an input layer, a style decoupling encoder, a dynamic feature fusion module, a multi-scale classifier and an output layer, the three-stage mixture model being sequentially connected, wherein a style intensity adjustment gate is introduced into the style decoupling encoder, the dynamic feature fusion module adopts a feature decoupling mechanism and a cross-domain distance constraint, and the multi-scale classifier adopts a three-stage classifier structure; and generating a target image using the trained three-stage mixture model and the selected specific artistic style.

Description

[Technical Field]

[0001] The present invention relates to the technical field of image generation, and more particularly to a method for automatically generating specific artistic style images driven by small sample learning. [Background technology]

[0002] In the fields of artistic creation and digital media, automatic generation of specific artistic style images has important application value, but traditional methods face many challenges.

[0003] Traditional art style transfer techniques, such as classical neural style transfer, rely heavily on a large number of style samples for training. If the number of samples is insufficient, the model is prone to overfitting, resulting in a significant decline in the quality of the generated images. For example, when using a traditional generative adversarial network (GAN) model to generate an image in the style of Vincent van Gogh's "Starry Night," providing only 10 samples will result in obvious defects such as broken brushstrokes and distorted colors, making the generated image unable to meet the needs of practical applications.

[0004] Current mainstream image generation techniques with a small number of samples also have significant shortcomings. Their feature decoupling capabilities are limited. For example, MC-GAN can only process 26 English characters, making it difficult to decouple the complex relationship between Chinese character stroke structure and style features. Classifiers are prone to overfitting when training with a small number of samples. When the dataset contains fewer than 500 samples, the FID value is generally higher than 80, with the ideal value being less than 50. Poor cross-domain adaptability is also a major issue, with significant loss of brushstroke continuity when transitioning from oil painting to ink painting.

[0005] Therefore, developing a method that can realize the automatic generation of high-quality images of specific artistic styles in the case of a small sample size has important theoretical significance and practical application value. Summary of the Invention [Problem to be solved by the invention]

[0006] SUMMARY OF THE INVENTION An object of the present invention is to provide a method for automatically generating specific artistic style images driven by small sample learning, which aims to solve or improve at least one of the above technical problems. [Means for solving the problem]

[0007] To achieve the above object, the present invention provides the following technical solution. A method for automatically generating specific artistic style images driven by small sample learning is selecting a particular art style; A step of constructing and training a three-stage mixture model, the three-stage mixture model including an input layer, a style decoupling encoder, a dynamic feature fusion module, a multi-scale classifier, and an output layer connected in series, wherein a style intensity adjustment gate is introduced in the style decoupling encoder, and the dynamic feature fusion module is Feature separation and attention weight mapping and cross-domain distance constraints are adopted, and the multi-scale classifier adopts a three-stage classification structure; and generating a target image using the trained three-stage mixture model and the selected specific artistic style.

[0008] Optionally, the style decoupling encoder adopts a double branching structure including a content branching and a style branching, and inserts a style intensity adjusting gate in the style branching to dynamically control the thickness of strokes.

[0009] Optionally, the content branch employs a ResNet-50 backbone network and is used to input standard black character images.

[0010] Optionally, the style branch employs an improved Squeeze-and-Excitation module to extract style features, and the improved manner of the Squeeze-and-Excitation module is to insert a style intensity adjustment gate between convolution layers.

[0011] Optionally, the dynamic feature fusion module specifically: Separating content features and style features through orthogonalization using Gram matrix, and introducing attention weight mapping to solve feature confusion caused by complex structures, where the content features include stroke structure and the style features include color texture; and The feature space Euclidean distance between the source domain and the target domain is calculated, and the relative distance relationship is maintained by a trigram loss function, where the source domain is used to adopt training data and the target domain is used to adopt generated data.

[0012] Optionally, the multi-scale classifier includes a global classifier, a local classifier and an edge classifier, wherein the local classifier introduces Gaussian anti-blur training so that the generated image establishes a link between sharp and blur samples.

[0013] Optionally, the global classifier employs a 64x64 resolution and is used to judge the consistency of the overall style, the local classifier employs a 32x32 random cut block and is used to evaluate the details of the brush strokes, and the edge classifier uses a Canny operator to extract the contour and obtain the accurate structure.

[0014] Optionally, the training process of the three-stage mixture model comprises: obtaining a training dataset comprising training data and corresponding generated tags; constructing a preset training network; The training data is input into the preset training network, and attribute elimination is first performed, followed by feature fusion based on cross-domain distance constraints, with the goal of minimizing the loss between the network output and the corresponding generated tag. Training is performed based on a two-stage optimization policy, and training is determined to be completed when a set accuracy condition is met. A trained three-stage mixture model is output, where the two-stage optimization policy includes a preset training stage and a fine-tuning stage, and the fine-tuning stage includes a step of adopting a learning rate warm restart policy. [Effects of the Invention]

[0015] Based on the specific embodiments provided by the present invention, the present invention discloses the following technical effects: The present invention discloses a method for automatically generating specific artistic style images driven by small sample learning, the method comprising the steps of selecting a specific artistic style and constructing and training a three-stage mixture model, the three-stage mixture model including an input layer, a style decoupling encoder, a dynamic feature fusion module, a multi-scale classifier and an output layer, which are connected in sequence, wherein a style intensity adjustment gate is introduced in the style decoupling encoder, and the dynamic feature fusion module is Feature separation and attention weight mapping and span distance constraints, and the multi-scale classifier employs a three-stage discriminant structure, and generates a target image using a trained three-stage mixture model and a selected specific artistic style. The present invention realizes generating a specific artistic style image that meets requirements based on small sample learning by introducing a style strength adjustment gate into the style decoupling encoder. [Brief explanation of the drawings]

[0016] In order to more clearly explain the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings that are required for use in the embodiments. It should be apparent that the drawings described below are merely a part of the embodiments of the present invention, and those skilled in the art can derive other drawings based on these drawings without any creative efforts. [Figure 1] 1 is a flowchart of a method for automatically generating specific artistic style images driven by small sample learning according to the present invention; DETAILED DESCRIPTION OF THE INVENTION

[0017] Hereinafter, the technical solutions in the embodiments of the present invention will be described clearly and completely with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, not all of the embodiments, and all other embodiments obtained by those skilled in the art based on the embodiments of the present invention without any creative work fall within the protection scope of the present invention.

[0018] SUMMARY OF THE INVENTION An object of the present invention is to provide a method for automatically generating specific artistic style images driven by small sample learning, which aims to solve or improve at least one of the above technical problems.

[0019] In order to make the above-mentioned objects, features and advantages of the embodiments of the present invention more readily comprehensible, the present invention will now be described in more detail with reference to the accompanying drawings and specific embodiments.

[0020] As shown in FIG. 1, the method for automatically generating specific artistic style images based on small sample learning provided by the present invention includes: a step 100 of selecting a particular art style; Step 200 of constructing and training a three-stage mixture model, the three-stage mixture model includes an input layer, a style decoupling encoder, a dynamic feature fusion module, a multi-scale classifier, and an output layer, which are connected in series, where a style intensity adjustment gate is introduced in the style decoupling encoder, and the dynamic feature fusion module is Feature separation and attention weight mappingand cross-domain distance constraints are adopted, and the multi-scale classifier adopts a three-stage classification structure; and generating 300 a target image using the trained three-stage mixture model and the selected specific artistic style.

[0021] In one specific embodiment, the style decoupling encoder employs a double-branch structure including a content branch and a style branch, and inserts a style intensity adjustment gate in the style branch to dynamically control stroke thickness. Here, the content branch employs a ResNet-50 backbone network and is used to input standard black body character images. The style branch employs an improved Squeeze-and-Excitation module to extract style features, and the improved Squeeze-and-Excitation module is inserted a style intensity adjustment gate between convolution layers.

[0022] In one specific embodiment, the dynamic feature fusion module specifically comprises: The Gram matrix is ​​used to separate the content features and style features through orthogonalization processing, and attention weight mapping is introduced to solve feature confusion caused by complex structures, where the content features include stroke structure, and the style features include color texture. The feature space Euclidean distance between the source region and the target region is calculated, and the relative distance relationship is maintained by the trigram loss function, where the source region is used to adopt training data, and the target region is used to adopt generated data.

[0023] In one specific embodiment, the multi-scale classifier includes a global classifier, a local classifier, and an edge classifier, where the local classifier implements Gaussian anti-blur training to establish a link between sharp and blurred samples in the generated image, where the global classifier employs a 64x64 resolution and is used to judge the consistency of the overall style, the local classifier employs a 32x32 random cut block and is used to evaluate the details of the brushstrokes, and the edge classifier uses a Canny operator to extract the contour and obtain the accurate structure.

[0024] In one specific embodiment, the training process of the three-stage mixture model includes: obtaining a training dataset including training data and corresponding generated tags and constructing a preset training network; The training data is input into the preset training network, and attribute elimination is first performed, followed by feature fusion based on cross-domain distance constraints, with the goal of minimizing the loss between the network output and the corresponding generated tag. Training is performed based on a two-stage optimization policy, and training is determined to be completed when a set accuracy condition is met. A trained three-stage mixture model is output, where the two-stage optimization policy includes a preset training stage and a fine-tuning stage, and the fine-tuning stage includes a step of adopting a learning rate warm restart policy.

[0025] Based on the above technical solutions, the following embodiments are provided.

[0026] In step 100, a particular art style is selected.

[0027] In this embodiment, Chinese ink painting style is selected as the specific artistic style for image generation. Chinese ink painting has distinctive brushstrokes, ink color variations, and contextual expression, and has distinct artistic characteristics, including variations in brushstroke thickness, ink color gradations, and whiteout.

[0028] In step 200, a three-stage mixture model is constructed and trained.

[0029] 1. Overall model architecture: The three-stage mixture model includes an input layer, a style decoupling encoder, a dynamic feature fusion module, a multi-scale discriminator and an output layer, which are connected sequentially.

[0030] 2, the style decoupling encoder includes: Dual branching structure: A dual branching structure is adopted, including content branching and style branching. Content Branching: A ResNet-50 backbone network is used to input standard black-and-white character images. ResNet-50 has strong feature extraction capabilities and can accurately capture content features such as stroke structure of the input image. Style Branching: An improved squeeze-and-excitation module is used to extract style features. A style strength adjustment gate is inserted between convolution layers, which can dynamically control the thickness of brush strokes. By adjusting the gate parameters, the thickness of brush strokes in the generated image can be controlled, thereby achieving flexible simulation of brush stroke features in ink painting style.

[0031] 3. Dynamic feature fusion module: Feature separation and attention weight mapping: Gram matrices are used to separate content features (including stroke structure) and style features (including color texture) through orthogonalization. Since ink paintings are primarily composed of black and white and gray, the style features here focus more on the variations in ink shade and the texture of brushstrokes. Attention weight mapping is also introduced to solve the feature confusion problem caused by complex structures. For example, when processing complex structures such as Chinese characters, attention weight mapping can direct the model to pay more attention to keystrokes and structural parts, ensuring effective integration of content features and style features. Cross-domain distance constraint: Calculate the feature space Euclidean distance between the source domain (using training data) and the target domain (using generated data), and maintain the relative distance relationship using the trigram loss function. This helps the model better learn the feature differences between the real ink painting data and the generated data during training, making the style of the generated image closer to the real ink painting.

[0032] 4. Multi-scale classifier: Global classifier: This classifier uses a resolution of 64x64 and is used to judge the overall style consistency of the generated image. For example, it judges whether the generated ink painting image has the elegance and ethereal style of the ink painting as a whole. Local classifier: A 32x32 randomly cut block is used to evaluate the details of brushstrokes. By introducing Gaussian anti-blur training, the generated image establishes a link between sharp and blurred samples. In ink painting, brushstrokes not only have clear contours, but also have a blurring effect due to the ink color fading. This local classifier design can better simulate these features. Edge detector: Canny operator is used to extract the contours and obtain accurate structure, which helps ensure the stroke structure of the generated image is accurate and meets the requirements of Chinese character writing and ink painting.

[0033] 5. Model training process: Obtaining training datasets and building a preset training network: A training dataset containing Chinese ink painting style images and corresponding standard black character images is collected, and corresponding generated tags are set for each training data. A preset training network is built to pre-learn the feature representation of the images. Training process: The training data is input into a preset training network, which first performs attribute elimination to remove interfering features unrelated to style, then performs feature fusion based on cross-domain distance constraints, and trains based on a two-stage optimization policy with the goal of minimizing the loss between the network output and the corresponding generated tags. Two-stage optimization policy: Includes a preset training stage and a fine-tuning stage. The preset training stage uses a relatively large learning rate to allow the model to quickly learn the basic features and style modes of the image. The fine-tuning stage uses a warm-restart learning rate policy to gradually adjust the learning rate, allowing the model to perform finer optimization within a local range and improve the quality of the generated images. Training is determined to be complete when the set accuracy criteria are met, and the trained three-stage mixture model is output.

[0034] Step 300: Generate a target image using the trained three-stage mixture model and the selected specific artistic style.

[0035] A standard black character image is input into a pre-trained three-stage mixture model, and a style decoupling encoder extracts content and style features. The style intensity adjustment gate parameters are adjusted according to demand during style branching to control the thickness of the brushstrokes. A dynamic feature fusion module effectively combines content and style features, and a multi-scale classifier evaluates and optimizes the overall style, brushstroke details, and structural accuracy of the generated image, ultimately outputting a Chinese ink painting-style Chinese character image.

[0036] To summarize, the method of this embodiment can generate high-quality images in a specific artistic style using a small amount of sample data in the style of Chinese ink painting. The generated images not only have the characteristics of the light elegance and emptiness of Chinese ink painting in their overall style, but also have excellent brushstroke detail and structural accuracy. Furthermore, the style intensity adjustment gate can flexibly control the thickness of the brushstrokes in the generated images, meeting the needs of different users for different ink painting styles.

[0037] Each embodiment in this specification is described in an incremental manner, with each embodiment focusing on the differences from other embodiments, and references being made to the same or similar parts between embodiments.

[0038] This specification describes the principles and embodiments of the present invention by applying specific examples, and the explanation of the above examples is intended to help understand the core idea of ​​the present invention, and those skilled in the art will be able to modify the specific embodiments and application scope based on the idea of ​​the present invention. As such, the contents of this specification should not be construed as limiting the present invention.

Claims

1. selecting a particular art style; constructing and training a three-stage mixture model, the three-stage mixture model including an input layer, a style decoupling encoder, a dynamic feature fusion module, a multi-scale classifier and an output layer connected in series, wherein a style intensity adjustment gate is introduced in the style decoupling encoder, the dynamic feature fusion module employs feature separation and attention weight mapping and cross-domain distance constraint, and the multi-scale classifier employs a three-stage classifier structure; generating a target image using the trained three-stage mixture model and the selected specific artistic style; The dynamic feature fusion module specifically includes: Separating content features and style features through orthogonalization using Gram matrix, and introducing attention weight mapping to solve feature confusion caused by complex structures, where the content features include stroke structure, and the style features include color texture; and Calculate the feature space Euclidean distance between the source domain and the target domain, and maintain the relative distance relationship by a trigram loss function, where the source domain is used to adopt the training data, and the target domain is used to adopt the generation data; A method for automatically generating specific artistic style images driven by small sample learning, characterized by:

2. 2. The method for automatically generating specific artistic style images driven by small sample learning as claimed in claim 1, wherein the style decoupling encoder adopts a double branching structure including a content branching and a style branching, and inserts a style intensity adjustment gate in the style branching to dynamically control the thickness of brush strokes.

3. The method for automatically generating specific artistic style images driven by small sample learning as claimed in claim 2, characterized in that the content branching adopts a ResNet-50 backbone network and is used to input standard black body character images.

4. The method for automatically generating specific artistic style images driven by small sample learning according to claim 2, wherein the style branching employs an improved Squeeze-and-Excitation module to extract style features, and the improved manner of the Squeeze-and-Excitation module is to insert a style intensity adjustment gate between convolution layers.

5. 2. The method for automatically generating specific artistic style images driven by small sample learning as claimed in claim 1, wherein the multi-scale classifier includes a global classifier, a local classifier and an edge classifier, wherein the local classifier introduces Gaussian anti-blur training to establish a link between sharp and blurred samples in the generated image.

6. 6. The method for automatically generating specific artistic style images driven by small sample learning as claimed in claim 5, wherein the global classifier adopts a 64x64 resolution and is used to judge the consistency of the overall style, the local classifier adopts a 32x32 random cut block and is used to evaluate the details of the brush strokes, and the edge classifier uses a Canny operator to extract contours and obtain accurate structures.

7. The training process of the three-stage mixture model is as follows: obtaining a training dataset comprising training data and corresponding generated tags; constructing a preset training network; 2. The method of claim 1, further comprising: inputting the training data into the preset training network; first performing attribute elimination; and then performing feature fusion based on cross-domain distance constraints, with the goal of minimizing the loss between the network output and the corresponding generated tags; training based on a two-stage optimization policy; determining that the training is finished when a set accuracy condition is met; and outputting a trained three-stage mixture model; wherein the two-stage optimization policy includes a preset training stage and a fine-tuning stage, and the fine-tuning stage adopts a learning rate warm restart policy.

Images