A method for constructing a Miao embroidery pattern generation model

Abstract

The present invention discloses a method for constructing a Miao embroidery pattern generation model, including: collecting Miao embroidery original pictures, generating Miao embroidery picture sets and pattern picture sets; defining a relationship model, determining data rules, generating Miao embroidery feature sets and relationship model parameters; after determining the data rules, processing the Miao embroidery picture sets and pattern picture sets to generate a Miao embroidery feature set; the Miao embroidery feature set includes: Miao embroidery noise pictures, text embedding vectors, and style feature matrices; training the relationship model to construct a Miao embroidery pattern generation model; the training process includes forward diffusion and reverse diffusion; forward diffusion is used to enable the relationship model to predict the noise tensor according to the text embedding vector and the style feature matrix; reverse diffusion is used to enable the relationship model to restore the Miao embroidery picture according to the noise tensor and the Miao embroidery noise picture. According to the above technical solution, embroidery patterns that conform to the Miao culture and aesthetics can be understood and generated, and high-quality Miao embroidery patterns can be generated without professional skills.

Description

Technical Field

[0001] The invention relates to the field of image processing, and in particular to a method for constructing a Miao embroidery pattern generation model. Background Art

[0002] Miao embroidery is an important part of China's intangible cultural heritage. In the design of Miao embroidery, the designer first needs to design the Miao embroidery pattern by hand or computer, generate the pattern pattern that the designer is satisfied with and in line with the cultural background, and then make the Miao embroidery. The traditional Miao embroidery art pattern design mainly depends on the designer's operation, selection of materials, research background and artistic processing. This method is time-consuming and labor-intensive, and has high requirements on the designer's painting skills. At the same time, the designer has an in-depth understanding of the Miao cultural background. With the development of computer technology, although there are some software tools that can assist in designing patterns, if there is a lack of understanding of the specific cultural background, even if these tools are used, it is impossible to accurately generate embroidery patterns with Miao characteristics. In addition, the existing pattern generation technology is mostly based on simple geometric shapes or repeated elements, lacking creativity and cultural depth, and it is difficult to meet the market's demand for personalization and diversification. Summary of the invention

[0003] To solve the above problems, the present application provides a method for constructing a Miao embroidery pattern generation model, comprising the following steps:

[0004] Collect the original Miao embroidery pictures, pre-process the original Miao embroidery pictures, generate Miao embroidery pictures, define the text description of the Miao embroidery pictures, and the Miao embroidery pictures and text descriptions constitute the Miao embroidery picture set; extract Miao embroidery pattern pictures from the Miao embroidery picture set, define the Miao embroidery pattern pictures and style types, and generate the pattern picture set; among which the style types include: line style and texture style;

[0005] Defining a relational model includes: determining the relational model, determining data rules, generating a Miao embroidery feature set, and determining relational model parameters; wherein the relational model is composed of a bert-base-chinese pre-trained language model and a Stable Diffusion model; after determining the data rules, processing the Miao embroidery picture set and the pattern picture set to generate a Miao embroidery feature set; the pattern data constituting the Miao embroidery feature set includes: Miao embroidery noise pictures, text embedding vectors, and style feature matrices;

[0006] The relational model is trained to build a Miao embroidery pattern generation model; the training process includes forward diffusion and backward diffusion; forward diffusion is used to enable the relational model to predict the noise tensor based on the text embedding vector and style feature matrix; backward diffusion is used to enable the relational model to gradually denoise the noise tensor and Miao embroidery noise picture according to the time step from large to small, and restore the Miao embroidery picture.

[0007] The preprocessing of the original Miao embroidery picture includes: using a tool to uniformly scale the original Miao embroidery picture to a standard size to generate a Miao embroidery picture; the standard size is: a height of 256 pixels, a width of 256 pixels, and a color channel of 3;

[0008] The contents of the text description of Miao embroidery pictures include: the patterns, pattern layout and color description of Miao embroidery;

[0009] Defining Miao embroidery pattern pictures and style types means: marking the style types of Miao embroidery pattern pictures according to the classification of Miao embroidery pictures.

[0010] Further, data rules include standard size and description rules:

[0011] The description rule refers to: determining the content of the Miao embroidery feature set based on the features of the pattern picture set and the Miao embroidery picture set. The content of the Miao embroidery feature set includes: Miao embroidery noise picture, style feature matrix and text embedding vector.

[0012] Among them, the processing of Miao embroidery picture collection and pattern picture collection includes Miao embroidery picture processing, text processing and style type processing;

[0013] The Miao embroidery picture processing is implemented by a Stable Diffusion model to generate the Miao embroidery noise picture;

[0014] The text processing and style type processing are implemented through the BERT-base-chinese pre-trained language model to generate text embedding vectors and style feature matrices.

[0015] Furthermore, the relational model parameters include: Miao embroidery noise images with added noise, text embedding vectors, style feature matrix, and the time step for adding noise;

[0016] Among them, the text embedding vector includes: the pattern description embedding vector of Miao embroidery, the pattern layout and color description embedding vector; the style feature matrix is ​​the line style and texture feature matrix of Miao embroidery.

[0017] Among them, Miao embroidery noise pictures refer to: adding noise to Miao embroidery pictures in the Miao embroidery picture set to generate Miao embroidery pictures with noise;

[0018] The time step of adding noise is used to mark the current Miao embroidery noise image parameters and the step time of the added noise;

[0019] The style feature matrix is ​​generated by extracting style type features from the VGG-16 network and generating the corresponding style feature tensor calculation;

[0020] The text embedding vector is generated by converting the text description information of the Miao embroidery image, namely the pattern description, pattern layout and color description.

[0021] Furthermore, when the VGG-16 network extracts style type features, the shallow convolutional layer in VGG-16 captures the line and edge information in the image to obtain the texture style features of Miao embroidery, and the middle convolutional layer obtains the line style features of Miao embroidery.

[0022] Among them, forward diffusion includes:

[0023] Convert the text embedding vector and style feature matrix into Key and Value feature tensor groups, convert the feature tensor of Miao embroidery noise image into Query tensor, and map Key and Query to the same spatial dimension through linear mapping;

[0024] The dot product calculation is performed using Key and Query to calculate the degree of association between Key and Query. The dot product value of the calculation result is proportional to the degree of association. The calculation method is: Among them, Cond feat is the conditional feature, Q is the Query tensor, K T is the transposed tensor of the Key tensor, is the dimension of the Key tensor, V is the Value tensor, and Softmax is the probability distribution function used to convert Converted into probability distribution; the probability distribution is the probability distribution of the line style tensor and the texture style tensor in the Miao embroidery noise picture feature matrix;

[0025] Get the conditional features with the best dot product value and correlation.

[0026] The conditional features for obtaining the optimal dot product value and correlation are: feat As a conditional feature, it is integrated into each convolution layer of u-net to guide and adjust the calculation of the feature tensor of the Miao embroidery noise image. Finally, u-net outputs a predicted noise tensor with the same size as the input Miao embroidery noise image. This tensor contains the Miao embroidery image features obtained through convolution calculation, and is also associated with the text description content and style type of Miao embroidery.

[0027] Among them, reverse diffusion includes:

[0028] Randomly extract Miao embroidery noise images;

[0029] Get the text embedding vector and style corresponding feature matrix of Miao embroidery pictures;

[0030] Iteratively run the noise removal calculation to output a pattern that conforms to the Miao embroidery description;

[0031] The noise removal calculation includes: setting a time step, predicting a third noise tensor, and removing noise from the noisy picture according to the third noise tensor.

[0032] According to the present invention, by learning a large number of real Miao embroidery samples, embroidery patterns that conform to Miao culture and aesthetics are automatically understood and generated, so that designers or ordinary users can quickly generate high-quality Miao embroidery patterns without deep professional skills, and support people from different backgrounds to easily use the model for creation, lowering the threshold for artistic creation; the Miao embroidery pattern generation model constructed by the present invention generates Miao embroidery patterns, which not only ensures that the generated patterns are not only beautiful, but also rich in the characteristics and traditional elements of Miao culture, which helps to protect and inherit intangible cultural heritage; at the same time, the innovative ability of the AI ​​model is used to generate design solutions that retain traditional styles and are innovative, meeting the needs of modern consumers for personalized products. In short, the technical means provided by the present invention provide new possibilities for the modernization of Miao embroidery art, and also open up new paths for the digital protection of intangible cultural heritage. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a schematic diagram of the steps of a method for constructing a Miao embroidery pattern generation model provided by an embodiment of the present invention;

[0034] Figure 2 The figure is a schematic diagram of a pattern feature set generation process of a Miao embroidery pattern generation model provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0035] The present invention proposes a model for generating Miao embroidery patterns, which processes the Miao embroidery according to its description content, texture style, line type and Miao embroidery pictures themselves, extracts features, and performs training and generation in combination with a relational model.

[0036] The specific implementation of the present invention is described in detail below with reference to the accompanying drawings.

[0037] Figure 1 The steps of constructing a method for generating a Miao embroidery pattern model are provided, as shown in the figure, including the following steps:

[0038] Step S100: collecting Miao embroidery original pictures, extracting Miao embroidery pictures and pattern pictures, generating Miao embroidery picture sets and pattern picture sets, and defining text descriptions of Miao embroidery pictures and style types of pattern pictures;

[0039] In this step, the collected original Miao embroidery pictures are preprocessed, and the preprocessing content specifically includes text preprocessing and picture preprocessing of the pictures.

[0040] When preprocessing the images, use tools (such as openCV) to uniformly scale all images (including those in the pattern data set) to a standard size to standardize the input format. At the same time, all scaled image data can be center cropped, with the width and height of the cropped part both being 256. The scaled images can be randomly rotated within a certain angle range (for example, between plus and minus 10 degrees) to generate new image data, which are saved as new Miao embroidery images. The cropped Miao embroidery images and the new Miao embroidery images constitute a Miao embroidery image set. In this way, the number of samples is increased, the diversity of image data is improved, and the Gaussian filtering technology of the tool is further used to eliminate high-frequency noise in the image, improve image quality, and reduce unnecessary interference information.

[0041] The content of text preprocessing is mainly to supplement the patterns, pattern layout and color description of Miao embroidery. For example, the detailed description of the patterns is focused, such as the specific content corresponding to the pattern types, such as diamond patterns, phoenix patterns, peasant patterns, wheat ear patterns, etc.; while the pattern layout and color mainly describe the layout of the patterns and the main colors of the Miao embroidery pictures, as well as the color rendering of the patterns. Therefore, the content of text preprocessing includes two parts: embroidery pattern description, pattern layout and color description.

[0042] After image preprocessing, the Miao embroidery images in the Miao embroidery image collection are screened according to the classification of Miao embroidery methods, and the patterns in the images are segmented through tools (such as openCV's pixel segmentation tool) to generate pattern images. The style types are marked according to texture styles (such as flat embroidery, braided embroidery, wrinkled embroidery, etc.) and stitch styles (such as picking up threads, long-foot stitches, etc.); the pattern images extracted from the Miao embroidery image collection and the corresponding style types constitute a pattern image collection.

[0043] like Figure 2 As shown, after collecting the original Miao embroidery pictures, the Miao embroidery pictures are generated after processing the original pictures, and the text descriptions of the Miao embroidery pictures are supplemented to form a Miao embroidery picture set; further, specific pattern pictures and style types of the pattern pictures are extracted to form a pattern picture set.

[0044] Step S110: defining a relationship model, including: determining a relationship model, determining data rules, generating a Miao embroidery feature set, and defining relationship model parameters;

[0045] 1) Determine the relationship model:

[0046] In the present invention, the bert-base-chinese pre-trained language model is used to extract features from the text descriptions in the Miao embroidery picture set; and the Stable Diffusion model is used as the basic model for generating images from text.

[0047] bert-base-chinese is a pre-trained BERT model designed specifically for Chinese. It generates context-related word embeddings through a bidirectional Transformer and has powerful language understanding and multi-task adaptability. StableDiffusion can generate high-quality images while maintaining image diversity and creativity, and supports conditional generation.

[0048] In order to better adapt to the characteristics of Miao embroidery images, the language processing module of Stable Diffusion was modified to support Chinese for data augmentation, conditional generation, and loss function optimization to improve the diversity and accuracy of generated images, which may improve the quality and relevance of generated Miao embroidery patterns.

[0049] 2) Data rules include standard size and description rules:

[0050] The standard size of Miao embroidery images is: 256 pixels in height, 256 pixels in width, and 3 color channels;

[0051] The description rule refers to: according to the features of the pattern picture set and the Miao embroidery picture set, the content of the Miao embroidery feature set is determined, including: Miao embroidery noise pictures, style feature matrices and text embedding vectors.

[0052] After determining the data rules, the relational model processes the Miao embroidery picture set and the pattern picture set to generate the Miao embroidery feature set;

[0053] The process of generating Miao embroidery feature set includes the following:

[0054] 1) Add noise to the Miao embroidery pictures in the Miao embroidery picture set to generate Miao embroidery pictures with noise. These Miao embroidery pictures with noise are used for comparison and model training with the noise pictures generated by model prediction in the subsequent steps. Since these noises are generated based on Miao embroidery pictures, they can not only retain all the information of the Miao embroidery pictures, but also conform to the semantic information of the corresponding Chinese descriptions;

[0055] The specific operation of adding noise is:

[0056] First, define a time step for adding noise and set it to 700;

[0057] Secondly, initialize a noise tensor through normal distribution. The formula of normal distribution is:

[0058]

[0059] Where e is the base of the natural logarithm (approximately 2.71828), and x is a random real number between -3 and 3. In this step, since the mean of the normal distribution is 0 and the standard deviation is 1, the method of generating the noise tensor is easier to standardize;

[0060] Third, traverse the Miao embroidery pictures in the Miao embroidery picture set, and add noise through the noise tensor and time step; each addition continues to add new noise based on the noise picture generated in the previous step until 700 noise additions are completed, generating 700 Miao embroidery noise pictures with layer-by-layer accumulated noise. The formula for adding noise is:

[0061] Among them, a t It is a hyperparameter that controls the degree of noise addition. The value of this parameter is T is the total number of time steps, t is the current time step, e θ is the noise tensor, where θ is a trainable parameter of the model; p t-1 Yes t Ethnic folk art picture with noise added in the previous step.

[0062] Specifically, when the step size is 1, according to the formula Get a t and is a value very close to 1, and Close to 0, then That is, the noisy image p 1 Approximately equal to the previous step p 0 The information of the picture, and p 0 This is the original pattern picture. 1 There is little noise content, but as the time step t increases, a t and Slowly tend to 0, when these two values ​​are 0, Close to 1, at this time That is, the noisy image p t Mainly contains noise e θ (p t ,t) information, the original image p 0 A lot of noise will be added.

[0063] By continuously increasing the time step, noise is added to the Miao embroidery image until 700 steps of noise addition are completed to form a picture with many noise points. t It is p t-1 Calculated, each calculated noise image p t In addition to adding noise information, it also contains the previous step noise image p t-1 Information, therefore, Miao embroidery pictures p 0 The features and information will be inherited to all noise images. The features can be compared with the predicted images in subsequent model predictions, and the noise images can be used to train and obtain the features of Miao embroidery images.

[0064] 2) Record the time step of adding noise. This time step is mainly used to mark the current Miao embroidery noise image parameters and the step time of the added noise.

[0065] 3) The text description information of Miao embroidery pictures, namely pattern description, pattern layout and color description, is converted into text embedding vectors. The text embedding vectors can be used as one of the prediction conditions to guide the model to generate noise pictures with specific semantic information.

[0066] The specific operation of converting the text description information of Miao embroidery pictures into embedded vectors is as follows:

[0067] Use BertTokenizer to segment the text description information containing the patterns, pattern layout and color description of Miao embroidery into meaningful words or phrases, while removing irrelevant words, punctuation marks and numbers; find the tokenId of each word and phrase and the position of each tokenId in the sentence in the predefined vocabulary matrix and sequence matrix of bert-base-chinese; the text embedding vectors corresponding to the relevant words and phrases are obtained through the vocabulary matrix and sequence matrix, expressed as: in is the embedding vector of the i-th token, W token and W position are the vocabulary matrix and the order matrix, p i Is the position code of the token. i is tokenId; after the text description is split, multiple words and phrases are generated, each word and phrase corresponds to a different text embedding vector. After integrating the multiple vectors into a text embedding matrix, it is passed into the multiple Transformer encoder layers of bert-base-chinese for processing, and the output is an integrated text embedding vector. The vector text embedding vector contains the semantics of the pattern features, pattern layout, and main colors of Miao embroidery images.

[0068] 4) Use the VGG-16 network to classify by style type, extract the style type features of the pattern images, and generate the corresponding style feature tensors. These style feature tensors are also used in the model prediction of Miao embroidery noise stage, as additional conditions to help the model generate noise with the unique line and texture characteristics of Miao embroidery.

[0069] The specific operation of obtaining the line style and texture style of Miao embroidery patterns is to first use VGG-16 to obtain the convolutional layer feature map of the pattern image, and then extract some shallow convolutional layer feature tensors and middle convolutional layer feature tensors from these convolutional layer feature tensors to extract the line features and texture features of the pattern respectively. In VGG-16, the shallow convolutional layer can efficiently capture the line and edge information in the image, which can be used to obtain the characteristics of the texture style of the pattern, while the middle convolutional layer is good at capturing more complex texture patterns, such as the texture of the fabric and the density of the lines, which can be used to obtain the characteristics of the line style of Miao embroidery. In order to associate the selected convolutional layer feature tensors to form the style corresponding to the pattern image, the Gram matrix is ​​used to calculate the inner product between different channels in these convolutional layer feature tensors, so as to reflect the relationship between the feature maps of different convolutional layers. This relationship is mainly reflected in the line style of the Miao embroidery pattern between the feature tensors of the shallow convolution layer, and is mainly reflected in the texture style of the Miao embroidery pattern between the feature tensors of the middle convolution layer. The Gram matrix formula is:

[0070] Among them, M l It is a new matrix formed by rearranging the matrices in each channel of the Gram matrix texture style map or the trace style map into arrays and then combining these arrays; It is M l The transposed matrix of .

[0071] Finally, the style feature matrix of the Miao embroidery feature set includes multiple categories of texture style data and line style data. The style matrices of all pictures in each category are calculated separately, and then the style matrices of each category are integrated by means of Gram matrix and weighted averaging to reflect their line features and texture features. These line features and texture features will be used as input options for the final model to generate pictures.

[0072] At the same time, in order to verify that the feature tensor obtained by VGG-16 represents the line features and texture features of the pattern, the feature tensor is visualized. The so-called feature tensor visualization is to average all channels of the aggregated feature tensor to obtain a global feature map. If the feature map clearly presents the line, edge or texture information, it means that effective features are extracted. If the feature map is fuzzy or cannot highlight the line or texture content of the Miao embroidery pattern, it is necessary to adjust the convolution layer of the VGG-16 extraction and optimize the pattern data set.

[0073] At this point, in this step, the relational model parameters are determined: Miao embroidery pictures with added noise, text embedding vectors (pattern description embedding vectors, pattern layout and color description embedding vectors), pattern line style and texture feature matrix, and the time step for adding noise.

[0074] Step S120: training the relationship model to construct a Miao embroidery pattern generation model;

[0075] The training process includes step S121 forward diffusion and step S122 reverse diffusion;

[0076] like Figure 1 As shown, the forward diffusion of step S121 is the key link of model training: the four types of parameters generated in step S110 are input into the model for repeated training, and the input text embedding vector and style feature matrix are used as conditions to make the model predict a suitable noise tensor. In the relational model, the input text embedding vector is used as conditional information, and the cross-modal attention mechanism is used to provide guidance for the model to generate images or predict noise. The cross-modal attention mechanism maps the text embedding vector, the line style tensor of the Miao embroidery pattern, the texture style tensor, and the feature tensor of the Miao embroidery noise image to the same spatial dimension through linear mapping for calculation, so that they can establish correlation.

[0077] Specifically, forward diffusion includes the following:

[0078] The text embedding vector and style feature matrix are converted into Key and Value feature tensor groups, the feature tensor of Miao embroidery noise image is converted into Query tensor, and Key and Query are mapped to the same spatial dimension through linear mapping. The tensors Key and Value are the relationship between index and value, and the Key tensor is the index of the Value tensor. Use MLP to convert the Key of the text embedding vector word With style type Key style , the Value of the text embedding vector word Value with style type style Matrix fusion is performed to form Key and Value matrices with Miao embroidery description semantics and pattern style. The corresponding value in the Value matrix can be found through the Key; the Key matrix is ​​used to calculate the association with the Query.

[0079] 1) In the specific implementation, the text embedding vector is converted into Key through normal distribution. word and Value word Two tensors, the line style tensor and the texture style tensor are expanded and fused to form a new style tensor, and then the tensor is converted into a key through normal distribution. style and Value style The two tensors are transformed into a Query tensor by normal distribution. The Key and Query tensors have the same size after transformation, that is, they are in the same spatial dimension.

[0080] Among them, the feature tensor of the Miao embroidery noise picture is generated by the u-net network convolution calculation of the diffusion model. This tensor contains the overall characteristics of the Miao embroidery noise picture. During the convolution calculation process, each layer of convolution calculation outputs a feature tensor of the Miao embroidery noise picture.

[0081] 2) Use Key and Query to perform dot product calculation to calculate the degree of association between Key and Query. The dot product value of the calculation result is proportional to the degree of association.

[0082] The matrix Value represents the actual information corresponding to each Key. If the two matrices are related in describing semantics and pattern styles, their dot product will be larger and the correlation will be stronger. Otherwise, the correlation will be smaller. During the training process, the parameters are continuously adjusted to make them as mathematically correlated as possible. If the descriptive information and style features corresponding to the Key matrix are highly similar to the Query feature matrix of the image, then its corresponding Value will be used to predict Miao embroidery noise.

[0083] The specific calculation method is expressed as: Among them, Cond feat is the conditional feature, Q is the Query tensor, K T is the transposed tensor of the Key tensor, is the dimension of the Key tensor, V is the Value tensor, and Softmax is the probability distribution function used to convert Converted into a probability distribution; the probability distribution is the probability distribution of the Miao embroidery description text embedding tensor and the style tensor in the Miao embroidery noise picture feature matrix.

[0084] 3) Obtain the optimal conditional features of dot product value and correlation degree:

[0085] Cond calculated in the previous step feat As a conditional feature, it is integrated into each convolution layer of u-net to guide and adjust the calculation of the Miao embroidery noise picture feature tensor. Finally, u-net outputs a predicted noise tensor of the same size as the input Miao embroidery noise picture. This tensor contains the Miao embroidery picture features obtained through convolution calculation, and is also associated with the Miao embroidery text description content and pattern style type. During the training process, the predicted noise tensor needs to be as similar as possible to the noise tensor in the input Miao embroidery noise picture. The two noise tensors are compared in each training prediction, which is expressed as a loss function:

[0086] Where N is the number of batch training data samples, e i refers to the input noise tensor, Refers to the predicted noise tensor, and θ is the set of trainable parameters of the model. During training, when using the cross-modal attention mechanism and u-net, linear trainable parameters are defined, such as: t represents the step size, represents the noise image of the national folk art picture generated by this step. i and If the difference is small, the value of the loss function will tend to 0 infinitely. The purpose of training the model is to make the loss function converge and minimize the value of the loss function as much as possible, so that the conditional tensor Cond feat The guidance and influence of predict a tensor similar to the input noisy tensor.

[0087] Step S122 is the process of back diffusion, i.e., generating an image, which is used to enable the relational model to gradually denoise the noise tensor and the Miao embroidery noise image according to the time step from large to small, and restore the Miao embroidery pattern image.

[0088] According to the principle of forward diffusion, if the noise tensor is removed from the Miao embroidery noise picture at the 700th time step, the 699th noise picture will be obtained. If it is pushed back to the 1st step, the original Miao embroidery picture without noise can be obtained. Based on this principle, the picture generation process is to use the noise tensor predicted by the model to gradually remove noise from a picture full of noise according to the time step from large to small, and finally restore a Miao embroidery picture with content. The implementation process is as follows:

[0089] 1) Randomly extract noise images;

[0090] 2) Obtain the feature matrix corresponding to the text embedding vector of the Miao embroidery picture and the style type of the pattern picture;

[0091] 3) Iteratively running the noise removal calculation to output a pattern that conforms to the Miao embroidery description;

[0092] The noise removal calculation includes: setting a time step, predicting a third noise tensor, and removing noise from the noisy picture according to the third noise tensor.

[0093] In the implementation, the noise image is first initialized by means of standard normal distribution. The size of the noise image is consistent with the final image processed in step S100, which is a tensor of 256*256*3. Each pixel value in this tensor comes from a standard normal distribution with a mean of 0 and a variance of 1. The formula of normal distribution is:

[0094]

[0095] At the same time, the input Miao embroidery description text is converted into a text embedding vector, the feature matrix corresponding to the style type is selected, the time step is set to 700, and the noise removal calculation is performed, that is, a new third noise tensor is predicted by the model, and the third noise tensor is calculated according to the forward diffusion formula. The forward diffusion formula is expressed as: The noise tensor is Therefore, it can be deduced that:

[0096]

[0097] Continue to iterate the noise removal calculation and set P t-1 The noise image, style feature matrix, and text embedding vector are used as parameters to input the model to predict the new noise tensor again, and then P can be obtained by calculating the formula. t-2 By repeating the prediction and calculation steps, the noise can be gradually removed from the random image that is full of noise, and finally a pattern image that meets the conditions is formed.

[0098] Through the above steps, the construction of the Miao embroidery pattern generation model is completed.

[0099] The Miao embroidery pattern generation model is saved locally and ensured to be loaded normally. The samples of the validation set are used for prediction, the output of each sample is calculated, and the prediction results are compared with the true labels of the validation set to evaluate the prediction accuracy, precision, and recall of the model. After completing the model evaluation, the model is iteratively optimized according to the evaluation results to improve the performance of the model, including adjusting the parameters of the model, changing the model structure, using different feature engineering methods, or trying different training strategies. Through iterative optimization, the accuracy and generalization ability of the model can be gradually improved, and finally a satisfactory performance level can be achieved.

[0100] After the Miao embroidery pattern generation model is built, the model is deployed and applied; in the relevant interface, the description of the pattern, layout and color of the Miao embroidery picture is input, and a Miao embroidery pattern style type is selected to generate a Miao embroidery picture that meets the requirements. If the generated picture meets the user's needs, the classification and description of the Miao embroidery picture can be further defined to supplement the Miao embroidery picture data warehouse. In this way, new data can be continuously added to the data warehouse, and can also be used to train new models. At the same time, these data will also become the basic data to support our expansion of cultural and creative industries, tourism souvenir development industries, and digital protection industries for ethnic minority works.

[0101] Through the present invention, a Miao embroidery pattern generation model is constructed. The model can not only identify and combine the basic elements of Miao embroidery patterns, but also deeply combine the line routing and texture characteristics of Miao embroidery to understand the cultural semantics behind the patterns and generate unique pattern designs according to the personalized needs of users. It provides strong technical support for the digital protection and inheritance of national folk art, and also provides designers and artists with new creative tools and sources of inspiration.

[0102] The above disclosures are only several specific embodiments of the present invention; however, the present invention is not limited thereto, and any changes that can be conceived by those skilled in the art should fall within the protection scope of the present invention.

Claims

1. A method for constructing a Miao embroidery pattern generation model, characterized in that: The following steps are involved: Collecting original Miao embroidery pictures, preprocessing the original Miao embroidery pictures to generate Miao embroidery pictures, defining text descriptions of the Miao embroidery pictures, wherein the Miao embroidery pictures and the text descriptions constitute a Miao embroidery picture set; Extracting Miao embroidery pattern pictures from the Miao embroidery picture set, defining the Miao embroidery pattern pictures and style types, and generating a pattern picture set; wherein the style types include: line style and texture style; Defining a relational model includes: determining the relational model, determining data rules, generating a Miao embroidery feature set, and determining relational model parameters; wherein the relational model is composed of a bert-base-chinese pre-trained language model and a Stable Diffusion model; after determining the data rules, processing the Miao embroidery picture set and the pattern picture set to generate a Miao embroidery feature set; the pattern data constituting the Miao embroidery feature set includes: Miao embroidery noise pictures, text embedding vectors, and style feature matrices; The relationship model is trained to construct a Miao embroidery pattern generation model; the training process includes forward diffusion and backward diffusion; the forward diffusion is used to enable the relationship model to predict the noise tensor according to the text embedding vector and the style feature matrix; the backward diffusion is used to enable the relationship model to gradually denoise the Miao embroidery noise picture according to the noise tensor and the Miao embroidery noise picture according to the time step from large to small, so as to restore the Miao embroidery picture.

2. The construction method according to claim 1, characterized in that: The preprocessing of the original Miao embroidery picture includes: using a tool to uniformly scale the original Miao embroidery picture to a standard size to generate a Miao embroidery picture; the standard size is: a height of 256 pixels, a width of 256 pixels, and a color channel of 3; The text description of the Miao embroidery picture includes: the pattern, pattern layout and color description of the Miao embroidery; The definition of the Miao embroidery pattern pictures and style types refers to: marking the style types of the Miao embroidery pattern pictures according to the classification of the Miao embroidery pictures.

3. The construction method according to claim 2, characterized in that: The data rules include standard size and description rules: The description rule refers to: determining the content of the Miao embroidery feature set based on the features of the pattern picture set and the Miao embroidery picture set, and the content of the Miao embroidery feature set includes: Miao embroidery noise picture, style feature matrix and text embedding vector.

4. The construction method according to claim 1, characterized in that: The processing of the Miao embroidery picture set and the pattern picture set includes Miao embroidery picture processing, text processing and style type processing; The Miao embroidery picture processing is implemented by a Stable Diffusion model to generate the Miao embroidery noise picture; The text processing and style type processing are implemented through the BERT-base-chinese pre-trained language model to generate text embedding vectors and style feature matrices.

5. The construction method according to claim 1, characterized in that: The relationship model parameters include: a Miao embroidery noise picture with added noise, a text embedding vector, a style feature matrix, and a time step for adding noise; Among them, the text embedding vector includes: the pattern description embedding vector of Miao embroidery, the pattern layout and color description embedding vector; the style feature matrix is ​​the line style and texture feature matrix of Miao embroidery.

6. The construction method according to claim 5, characterized in that: The Miao embroidery noise picture refers to: adding noise to the Miao embroidery picture in the Miao embroidery picture set to generate a Miao embroidery picture with noise; The time step of adding noise is used to mark the current Miao embroidery noise picture parameters and the step time of the added noise; The style feature matrix is ​​generated by extracting style type features from the VGG-16 network and generating corresponding style feature tensors; The text embedding vector is generated by transforming the text description information of the Miao embroidery picture, namely, the pattern description, pattern layout and color description.

7. The construction method according to claim 6, characterized in that: When the VGG-16 network extracts style type features, the shallow convolution layer in VGG-16 captures the line and edge information in the image to obtain the texture style features of Miao embroidery, and the middle convolution layer obtains the line style features of Miao embroidery.

8. The construction method according to claim 1, characterized in that: The forward diffusion includes: Convert the text embedding vector and style feature matrix into Key and Value feature tensor groups, convert the feature tensor of Miao embroidery noise image into Query tensor, and map Key and Query to the same spatial dimension through linear mapping; The dot product calculation is performed using Key and Query to calculate the degree of association between Key and Query. The dot product value of the calculation result is proportional to the degree of association. The calculation method is: Among them, Cond feat is the conditional feature, Q is the Query tensor, K T is the transposed tensor of the Key tensor, is the dimension of the Key tensor, V is the Value tensor, and Softmax is the probability distribution function used to convert Converted into probability distribution; the probability distribution is the probability distribution of the line style tensor and the texture style tensor in the Miao embroidery noise picture feature matrix; Get the conditional features with the best dot product value and correlation.

9. The construction method according to claim 8, characterized in that: The conditional features for obtaining the optimal dot product value and correlation degree are: feat As a conditional feature, it is integrated into each convolution layer of u-net to guide and adjust the calculation of the feature tensor of the Miao embroidery noise image. Finally, u-net outputs a predicted noise tensor with the same size as the input Miao embroidery noise image. This tensor contains the Miao embroidery image features obtained through convolution calculation, and is also associated with the text description content and style type of Miao embroidery.

10. The construction method according to claim 1, characterized in that: The reverse diffusion comprises: Randomly extract Miao embroidery noise images; Get the text embedding vector and style corresponding feature matrix of Miao embroidery pictures; Iteratively run the noise removal calculation to output a pattern that conforms to the Miao embroidery description; The noise removal calculation includes: setting a time step, predicting a third noise tensor, and removing noise from the noisy picture according to the third noise tensor.

Images