Deep Learning-Based Method for Generating Evaluation Feedback on the Embroidery Style of Printed Patterns

Abstract

The present invention discloses a method for generating an evaluation feedback of the embroidery style of a printed pattern based on deep learning, belonging to the technical field of digital printing. The method includes: inputting a printed design pattern into a four-path information fusion embroidery printing generation model to output an embroidery-style printed pattern; inputting the embroidery-style printed pattern into a digital printing machine to produce an embroidered-effect printed fabric; using a printed image acquisition device to collect an image of the embroidered-effect printed fabric and inputting it into a printed style realism evaluation model to output a realism evaluation and feedback it to the four-path information fusion embroidery printing generation model to further generate an embroidery-style printed pattern with a stronger sense of style.

Description

Technical Field

[0001] The present invention relates to a method for generating an evaluation feedback of an embroidery style of a printed pattern based on deep learning, and belongs to the technical field of digital printing. Background Art

[0002] Embroidery is a fabric style with distinct characteristics. Traditional embroidery techniques mainly rely on manual labor, which requires high requirements for designers, consumes a large amount of human and material resources, time costs, and material costs, and is also restricted by traditional styles. Using digital printing can enable fabrics to quickly have an embroidery style. Therefore, generating images with an embroidery style to guide digital printing to generate printed design patterns with an embroidery style can effectively promote resource savings.

[0003] Chinese Patent with publication number CN118015127A discloses a method and device for synthesizing a design pattern with an embroidery texture and a sense of quality, and an electronic device, mainly including a generator and a discriminator, which perform style transfer on an embroidery design draft to achieve the generation of an embroidery texture and a sense of quality. However, the style is single and feature loss will occur. Chinese Patent with publication number CN117094882A discloses a method, system, device, and medium for lossless digital embroidery image style transfer, which realizes the generation of an embroidery-style printed pattern image through a reversible residual module to achieve the style transfer of embroidery. However, this method is limited to the generation of embroidery images and does not generate fabrics through digital printing.

[0004] Due to limitations in the printing process, there are often significant losses in the embroidery style when an embroidery-style printed pattern image guides digital printing to generate fabrics. Many high-dimensional features and semantic features will be lost, resulting in a weak embroidery style. Moreover, the bleeding effect of printing ink is also likely to cause the loss of the embroidery style on the fabric. Therefore, Chinese Patent with publication number CN117290902A discloses a method for parametric modeling of a hand-embroidered pattern for digital printing technology. By extracting key features of the embroidery texture, the realism of the embroidery image is improved. In this way, the clarity of the embroidery fabric style can be effectively improved, but a large amount of manual intervention and experience are still required, and the operation is complicated.

[0005] A Chinese patent with the publication number CN101333740A discloses a computer embroidery process using simulation image technology. By performing image processing on the collected pictures and outputting images, this method further reduces manual intervention, but the embroidery style of the fabric is damaged to a certain extent. A Chinese patent with the publication number CN117156070A discloses an intelligent parameter regulation method and system for an embroidery machine. By performing process decomposition and label encryption on the embroidery image, an intelligent parameter adjustment scheme is further obtained. It can realize the intelligent parameter regulation of the embroidery machine, thereby weaving fabrics with embroidery style effects; however, the system load is cumbersome and there are problems with the adaptation of the embroidery machine.

[0006] In summary, the above methods have not yet maturely given the fabric an embroidered style through digital printing. Summary of the Invention

[0007] To solve the problems existing in the above-mentioned prior art, the present invention provides a method for generating an evaluation feedback of an embroidered style for a printed pattern based on deep learning. An embroidered style printed pattern is generated by a four-path information fusion embroidery printing generation model and input into a digital printing machine, and the image is collected by an image acquisition system and input into a printing style realism evaluation model. The authenticity evaluation is output and fed back, and the fabric is given an embroidered style through digital printing. The above process realizes the digital printing generation of embroidered style printed fabrics, and improves the authenticity through a feedback mechanism, and realizes the controllability and pertinence of elements through editability; this method realizes the digital printing generation of embroidered style printed fabrics by constructing two deep learning models, namely a four-path information fusion embroidery printing generation model and a printing style realism evaluation model.

[0008] The purpose of the present invention is to provide a method for generating an evaluation feedback of an embroidered style for a printed pattern based on deep learning. This method realizes the generation of an embroidered style printed pattern by constructing a four-path information fusion embroidery printing generation model and a printing style realism evaluation model.

[0009] The specific steps of the method proposed by the present invention are as follows:

[0010] Step 1: Collect printed design patterns and corresponding embroidered style printed patterns to construct Dataset 1, and divide Dataset 1 into a training set, a validation set, and a test set;

[0011] Step 2: Use the training set obtained in Step 1 for training to obtain a four-path information fusion embroidery printing generation model;

[0012] Step 3: Collect embroidered style printed patterns and their subjective evaluation grades as Dataset 2, and divide Dataset 2 into a training set, a validation set, and a test set;

[0013] Step 4: Use the training set obtained in Step 3 to train and obtain a real - sense evaluation model for printing styles;

[0014] Step 5: Input the printed design pattern into the four - path information fusion embroidery printing generation model obtained in Step 2 to generate an embroidery - style printed pattern;

[0015] Step 6: Input the embroidery - style printed pattern obtained in Step 5 into the real - sense evaluation model for printing styles obtained in Step 4, obtain the corresponding evaluation, and feedback it back to the four - path information fusion embroidery printing generation model;

[0016] Step 7: Input the printed design pattern into the four - path information fusion embroidery printing generation model that has been evaluated and feedback in Step 6 to generate an embroidery - style printed pattern, realizing the embroidery of the print;

[0017] In one implementation, the ratio of the training set, validation set, and test set in the data set one of Step 1 is 8:1:1;

[0018] In one implementation, the four - path information fusion embroidery printing generation model in Step 2 includes four information - processing paths, one information fusion operator, and one decoder, where:

[0019] The first path includes a knitting needle process editing software and a knitting needle information encoder. The input of the knitting needle information encoder is the knitting needle process feature map, and the output is the knitting needle trajectory feature;

[0020] The second path includes an edge information extraction module and an edge information encoder. The input of the edge information extraction module is the printed design pattern, and the output is the edge information feature map. The input of the edge information encoder is the edge information feature map, and the output is the edge information feature;

[0021] The third path includes a color block information extraction module and a color block positioning encoder. The input of the color block information extraction module is the printed design pattern, and the output is the color block information feature map. The input of the color block positioning encoder is the color block information feature map, and the output is the color clustering feature;

[0022] The fourth path includes an original information encoder, whose input is the printed design pattern, and the output is the original feature of the printed design pattern;

[0023] The input of the information fusion operator is the knitting needle trajectory feature, edge information feature, color clustering feature, and the original feature of the printed design pattern, and the output is the high - dimensional vector of embroidery printing.

[0024] The input of the decoder is the high - dimensional vector of embroidery printing, and the output is the embroidery - style printed pattern. Its calculation method is:

[0025] (1.1)

[0026] Among them, is the high-dimensional vector of embroidery printing, is the embroidery-style printing pattern, is the non-linear transformation function, where represents the non-linear calculation of the th layer, is a fully connected layer, where represents the calculation of the th fully connected layer, is the non-linear variation function.

[0027] In one implementation, in the first path, the knitting needle process diagram is derived from the knitting needle process editing software, and its calculation method is:

[0028] (1.2)

[0029] Among them, is the input printing design pattern, is the output knitting needle process feature diagram, is the processing process of the knitting needle process editing software, and the knitting process feature diagram is obtained by custom editing in the process feature vector matrix.

[0030] The knitting needle information encoder in the first path has the knitting needle process feature diagram as the input and the knitting needle trajectory feature as the output, and its calculation method is:

[0031] (1.3)

[0032] Among them, is the knitting needle process feature diagram, is the knitting needle trajectory feature, is stacked by three convolutional layers, and the convolutional kernel size of each convolutional layer is 3x3, is the pooling calculation method, is the high-dimensional mapping, and its calculation method is:

[0033] (1.4)

[0034] Among them, is the weight learned by the knitting needle information encoder, is the bias learned by the knitting needle information encoder.

[0035] In one implementation, in the second path, the input of the edge information extraction module is the printing design pattern, and the output is the edge information feature diagram, and its calculation method is:

[0036] (1.5)

[0037] Among them, is the input printed pattern design are the coordinates of the printed pattern design is the edge information feature map is the Laplacian filter for supplementing high frequencies is the weight for using the Laplacian filter indicates at the gradient in the direction represents the filter the weight for its use represents the filter the direction of the edge or change of interest represents the direction set, the filter is calculated as follows:

[0038] (1.6)

[0039] wherein, is the input printed pattern design are the coordinates of the input printed pattern design is the non - linear activation function

[0040] The edge information encoder in the second path, with the input being the edge information feature map and the output being the edge information feature, is calculated as follows:

[0041] (1.7)

[0042] wherein, is the edge information feature map is the edge information feature is stacked by three convolutional layers, and the convolutional kernel size of each convolutional layer is 3x3 is the pooling method is the high - dimensional mapping, and its calculation is as follows:

[0043] (1.8)

[0044] wherein, is the weight learned by the edge information encoder is the bias learned by the edge information encoder

[0045] In one implementation, in the third path, the input of the color block information extraction module is the printed pattern design, and the output is the color block information feature map, and the calculation method is:

[0046] (1.9)

[0047] wherein, are the coordinates of the input printed pattern design, is the color block information feature map, represents the number of color blocks, is the center position of the color block, is the type of color block, is a pre-trained neural network, represents the number of scales of convolution, represents the total number of scales of convolution, represents the morphological gradient for extracting the boundaries of color blocks, is a color clustering method, represents the convolution calculation with a convolution kernel size of 5x5.

[0048] In the third path, the input of the color block localization encoder is the color block information feature map, and the output is the color clustering feature. The calculation method is as follows:

[0049] (1.10)

[0050] where, is the color block information feature map, is the color clustering feature, is stacked by three convolutional layers, and the convolution kernel size of each convolutional layer is 3x3, is the pooling method, is the high-dimensional mapping, and its calculation method is as follows:

[0051] (1.10)

[0052] where, is the weight learned by the color block localization encoder, is the bias learned by the color block localization encoder.

[0053] In one implementation, the input of the information fusion operator is the knitting needle trajectory feature, the edge information feature, the color clustering feature, and the original feature of the printed pattern design, and the output is the high-dimensional vector of the embroidered print. The calculation method is as follows:

[0054] (1.11)

[0055] where, is the high-dimensional vector of the embroidered print, is the knitting needle trajectory feature, Edge information feature one, is the edge information feature two, is the color clustering feature, is the original feature of the printed pattern design, is the non-linear activation function one, Non - linear activation function two Represents the dot product Is a weighted merging operation, and its calculation process is as follows: The input edge information features are divided into edge information feature one and edge information feature two along the channel dimension; Edge information feature one is added to the knitting needle trajectory feature, and edge information feature two is multiplied by the color clustering feature. Finally, the two are subjected to a weighted merging operation with the original printing design pattern feature to obtain the high - dimensional vector of embroidered printing, where the weighted average operation calculation method is as follows:

[0056] (1.12)

[0057] Wherein Is the high - dimensional vector of embroidered printing , , Respectively represent different learnable weights, used to adjust the proportion of different information in the generation of embroidered style printing patterns , , The calculation method is as follows:

[0058] (1.13)

[0059] (1.14)

[0060] (1.15)

[0061] In one implementation, the total loss function of the model The calculation method is as follows:

[0062] (1.16)

[0063] The total loss function Is respectively composed of the first loss function And the second loss function The first loss function Is used to judge the color difference, and the second loss function Evaluates the authenticity of the stitch Is used to adjust the proportion of the first loss function In the total loss function , and the set range is (0.35 - 0.45) Is used to adjust the proportion of the second loss function In the total loss function , and the set range is (0.55 - 0.65)

[0064] Wherein, the calculation method of the first loss function Is as follows:

[0065] (1.17)

[0066] Among them, represents the color value of the th pixel of the label embroidery-style printed pattern, represents the color value of the th pixel of the generated embroidery-style printed pattern, represents the total number of pixel values;

[0067] The second loss function is calculated as follows:

[0068] (1.18)

[0069] Among them, is the total number of pixel points in the image, represents the Hessian matrix of the label embroidery-style printed pattern at the position, represents the Hessian matrix of the generated embroidery-style printed pattern at the position;

[0070] The second loss function The calculation process is to extract the high-frequency texture features of the image using a Gaussian high-pass filter and perform Fourier transform to obtain a spectrogram, and use the Hessian matrix to describe the second-order derivative information of the local area of the spectrogram to capture the local changes of the texture. The specific calculation method is as follows:

[0071] (1.19)

[0072] Among them, , is the second-order derivative of the spectrogram in the and directions, , is the mixed second-order derivative of the spectrogram; this matrix can describe the local curvature and directionality of the texture.

[0073] When the loss function reaches the minimum, the model training is completed.

[0074] In one implementation, the ratio of the training set, validation set, and test set in the second dataset of step 3 is 8:1:1;

[0075] In one embodiment, the evaluation model for the realism of the printing style in step 4 is a convolutional neural network stacked with multiple convolutional blocks. The input is an image of embroidered-style printed fabric, and the output is an evaluation index of realism. Its main structure is as follows: The first two layers are a convolutional layer with a convolutional kernel size of 7x7 and a max pooling layer of 3x3. Subsequently, it passes through six convolutional blocks. Each convolutional block contains 2-3 convolutional layers, and residual connections are used between the convolutional blocks. Different evaluation components are connected after every two convolutional blocks. The specific calculation method of the convolutional block is as follows:

[0076] (2.1)

[0077] where is the input of the convolutional block, is the output of the convolutional block, is a non-linear activation method, represents the calculation of the th convolutional layer in the convolutional block for , and the range of is (2, 3). The specific calculation method is as follows:

[0078] (2.2)

[0079] GY is the calculation method of normalization, is the rd convolutional layer in the convolutional block for the 3×3 convolutional operation.

[0080] In one embodiment, the loss function is used to constrain the evaluation model for the realism of the printing style. The expression of the loss function is as follows:

[0081] (2.3)

[0082] where, is the loss function of the evaluation model for the realism of the printing style, is the number of images in dataset two, represents the evaluation index of realism, represents the evaluation index of realism of the model prediction result. When the value of the loss function is the smallest, the model training is completed.

[0083] In one embodiment, the evaluation index of realism is constructed as follows:

[0084] Step one: Evaluation of the embroidered style; The overall embroidered style evaluation of the printed design pattern and the image feature map of the embroidered-style printed fabric is obtained through a pre-trained neural network model. The specific calculation method is as follows: ​

[0085] (2.4)

[0086] Among them, is the embroidery style evaluation, is the printed design pattern, is the image of the embroidered style printed fabric, is a pre-trained neural network model that can extract the main style features of the embroidery style for comparison and scoring.

[0087] Step 2: Texture feature evaluation; The feature maps of the printed design pattern and the image of the embroidered style printed fabric are used to extract features through Gabor filters and compared to obtain the texture feature evaluation. The specific calculation method is as follows:

[0088] (2.5)

[0089] Among them, is the texture feature evaluation, is the printed design pattern, is the image of the embroidered style printed fabric, is the th Gabor filter, is the cosine similarity, is the number of Gabor filters; Through the texture feature evaluation, the local detailed texture of the image of the embroidered style printed fabric can be fed back and optimized.

[0090] Step 3: Color transfer evaluation; The feature maps of the printed design pattern and the image of the embroidered style printed fabric are converted to a unified Lab color space, and the main color tones, contrast, and saturation distributions of the two are analyzed through a color clustering algorithm. Finally, the color histogram matching degree is used to quantify the retention degree of the color distribution. The specific calculation method is as follows:

[0091] (2.6)

[0092] Among them, is the color transfer evaluation, represents the weight of the contrast and saturation evaluation, represents the weight of the color histogram matching degree, is the printed design pattern, is the image of the embroidered style printed fabric, is the color space conversion, is the color clustering algorithm, is the contrast and saturation comparison of the two images, is the color histogram of the printed design pattern, is the color histogram of the image of the embroidered style printed fabric.

[0093] In one embodiment, the specific process of feeding the realistic evaluation index back to the four-path information fusion embroidery printing generation model is as follows:

[0094] Step (1): Embroidery style supplement; when the embroidery style evaluation does not meet the expectation, increase the weight of adding the edge information feature one and the knitting needle trajectory feature to enhance the supplement of the embroidery style.

[0095] Step (2): Texture feature supplement; add multi-channel embroidery stitches to the knitting needle process feature map through the knitting needle process editing software to enhance the supplement of the stitch texture, and change the weight learned by the edge information encoder .

[0096] Step (3): Color transfer alignment; when the color transfer evaluation does not meet the expectation, increase the weight of multiplying the edge information feature two and the color clustering feature to ensure the alignment of the image colors of the printed design pattern and the embroidered style printed fabric.

[0097] Advantages of the present invention:

[0098] High-quality embroidery stylization: The feedback mechanism promotes the high-quality generation of embroidery style images and the imparting of the embroidery style of the corresponding fabric. The information fusion operator enables the model to accept the input of various feature information and fuse it, thus ensuring the minimum loss of features.

[0099] Editable printing generation: The implementation methods include pattern editing of the four-path information fusion embroidery printing generation model and process parameter editing on a digital printing machine, which can provide a participation channel for experienced designers and enhance the embroidery style effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0100] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0101] Figure 1 is the technical roadmap of the method for generating and evaluating feedback on the embroidery style of printed patterns based on deep learning provided by the present invention;

[0102] Figure 2 is the structural diagram of the four-path information fusion embroidery printing generation model of the method for generating and evaluating feedback on the embroidery style of printed patterns based on deep learning provided by the present invention;

[0103] Figure 3It is the structural diagram of the realistic evaluation model of the printing style of the printing pattern embroidery style generation and evaluation feedback method provided by the present invention;

[0104] Figure 4 It is the structural diagram of the information fusion operator of the printing pattern embroidery style generation and evaluation feedback method provided by the present invention. Specific embodiments

[0105] To make the objectives, technical solutions and advantages of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0106] Embodiment 1

[0107] This embodiment provides a method for generating and evaluating feedback on the embroidery style of printing patterns based on deep learning. This method realizes the generation of embroidery-style printing patterns by constructing a four-path information fusion embroidery printing generation model and a realistic evaluation model of the printing style. The technical roadmap of this method is as follows Figure 1 shown. Input the printing design pattern into the four-path information fusion embroidery printing generation model. Through the four-path information fusion embroidery printing generation model, generate the embroidery-style printing pattern and input it into the digital printing machine. The digital printing machine outputs the embroidered printing cloth. Collect the generated image through the image acquisition system and input it into the realistic evaluation model of the printing style. Evaluate the authenticity of the image through the realistic evaluation model of the printing style and feedback the evaluation result to the four-path information fusion embroidery printing generation model to further constrain the generation of the image. The method steps of the present invention are as follows:

[0108] Step 1: Collect the printing design patterns and the corresponding embroidery-style printing patterns to construct Dataset 1, and divide Dataset 1 into a training set, a validation set, and a test set;

[0109] Step 2: Use the training set obtained in Step 1 to train to obtain a four-path information fusion embroidery printing generation model;

[0110] Step 3: Collect the embroidery-style printing patterns and their subjective evaluation levels as Dataset 2, and divide Dataset 2 into a training set, a validation set, and a test set;

[0111] Step 4: Use the training set obtained in Step 3 to train to obtain a realistic evaluation model of the printing style;

[0112] Step 5: Input the printing design pattern into the four-path information fusion embroidery printing generation model obtained in Step 2 to generate an embroidery-style printing pattern;

[0113] Step 6: Input the embroidered-style printed pattern obtained in Step 5 into the printed-style realism evaluation model obtained in Step 4, obtain the corresponding evaluation, and feedback it back to the four-path information fusion embroidery printing generation model;

[0114] Step 7: Input the printed design pattern into the four-path information fusion embroidery printing generation model that has been evaluated and feedback in Step 6 to generate an embroidered-style printed pattern, realizing the embroidery of the print;

[0115] The steps to construct the four-path information fusion embroidery printing generation model are as follows:

[0116] 1. Collect the printed design patterns and the corresponding embroidered-style printed patterns to construct Dataset 1, and divide Dataset 1 into a training set, a validation set, and a test set;

[0117] 2. Use Dataset 1 obtained in 1 to train to obtain the four-path information fusion embroidery printing generation model;

[0118] 3. Input the printed design pattern into the four-path information fusion embroidery printing generation model in 2 to generate an embroidered-style printed pattern, realizing the embroidery of the print;

[0119] The ratio of the training set, the validation set, and the test set in Dataset 1 of 1 is 8:1:1;

[0120] As Figure 2 shown, the four-path information fusion embroidery printing generation model includes four information processing paths, an information fusion operator, and a decoder, where:

[0121] The first path includes a knitting needle process editing software and a knitting needle information encoder. The input of the knitting needle information encoder is the knitting needle process feature map, and the output is the knitting needle trajectory feature;

[0122] The second path includes an edge information extraction module and an edge information encoder. The input of the edge information extraction module is the printed design pattern, and the output is the edge information feature map. The input of the edge information encoder is the edge information feature map, and the output is the edge information feature;

[0123] The third path includes a color block information extraction module and a color block positioning encoder. The input of the color block information extraction module is the printed design pattern, and the output is the color block information feature map. The input of the color block positioning encoder is the color block information feature map, and the output is the color clustering feature;

[0124] The fourth path includes an original information encoder, whose input is the printed design pattern, and the output is the original feature of the printed design pattern;

[0125] As Figure 4As shown, the inputs of the information fusion operator are the knitting needle trajectory features, edge information features, color clustering features, and the original features of the printed pattern, and the output is the high-dimensional vector of embroidered prints.

[0126] The input of the decoder is the high-dimensional vector of embroidered prints, and the output is the embroidered style printed pattern. Its calculation method is as follows:

[0127] (1.1)

[0128] Among them, is the high-dimensional vector of embroidered prints, is the embroidered style printed pattern, is a non-linear transformation function, where represents the non-linear calculation of the th layer, is a fully connected layer, where represents the calculation of the th fully connected layer, is a non-linear variation function.

[0129] The calculation method of

[0130] (1.20)

[0131] Among them, is a fully connected layer, where represents the calculation of the th fully connected layer;

[0132] is a non-linear variation function, and the specific calculation method is as follows:

[0133] (1.21)

[0134] In the first path, the knitting needle process diagram is from the knitting needle process editing software Vilcom Embroidery Studio, version number e4.2h. The main functions used are digitized flat stitch and display of stitches, etc.

[0135] The input of the knitting needle information encoder is the knitting needle process feature diagram, and the output is the knitting needle trajectory feature. The calculation method is as follows:

[0136] (1.3)

[0137] Among them, is the knitting needle process feature diagram, is the knitting needle trajectory feature, is stacked by three convolutional layers, and the convolutional kernel size of each convolutional layer is 3x3, It is a pooling calculation method, and the calculation method is as follows:

[0138] (1.22)

[0139] Among them, represents the pixels of a certain regional block in the feature map ;

[0140] is a high-dimensional mapping, and its calculation method is:

[0141] (1.4)

[0142] Among them, is the weight learned by the knitting needle information encoder, is the bias learned by the knitting needle information encoder.

[0143] In the second path, the input of the edge information extraction module is the printed pattern design, and the output is the edge information feature map. Its calculation method is:

[0144] (1.5)

[0145] Among them, is the input printed pattern design, is the coordinate of the printed pattern design, is the edge information feature map, is the Laplacian filter for supplementing high frequencies, is the weight for using the Laplacian filter, indicates the gradient in the direction, represents the weight for using the filter ; represents the filter the direction of the edge or change of concern, represents the direction set, and the calculation method of the filter is:

[0146] (1.6)

[0147] Among them, is the input printed pattern design, is the coordinate of the input printed pattern design, is a non-linear activation function,

[0148] In the second path, the edge information encoder takes the edge information feature map as the input and outputs the edge information feature. Its calculation method is:

[0149] (1.7)

[0150] Among them, is the edge information feature map, is the edge information feature, which is stacked by three convolutional layers, and the convolutional kernel size of each convolutional layer is 3x3. is the pooling method, and the calculation method is as follows:

[0151] (1.22)

[0152] represents the feature map the pixel points of a certain regional block;

[0153] is the high-dimensional mapping, and its calculation method is as follows:

[0154] (1.8)

[0155] Among them, is the weight learned by the edge information encoder, is the bias learned by the edge information encoder.

[0156] In the third path, the input of the color block information extraction module is the printed pattern design, and the output is the color block information feature map. The calculation method is:

[0157] (1.9)

[0158] Among them, is the coordinate of the input printed pattern design, is the color block information feature map, is the center position of the color block, is the type of the color block, is a pre-trained neural network, represents the scale number of the convolution, represents the total scale number of the convolution, represents the morphological gradient used to extract the color block boundary, is a color clustering method, represents the convolution calculation with a convolutional kernel size of 5x5.

[0159] The input of the color block positioning encoder is the color block information feature map, and the output is the color clustering feature. The calculation method is:

[0160] (1.10)

[0161] Among them, is the color block information feature map, is the color clustering feature, It is stacked by three convolutional layers, and the convolutional kernel size of each convolutional layer is 3x3. It is the pooling method, and the calculation method is as follows:

[0162] (1.23)

[0163] Among them, represents the feature map ;

[0164] is a high-dimensional mapping, and its calculation method is as follows:

[0165] (1.10)

[0166] Among them, is the weight learned by the color block positioning encoder, is the bias learned by the color block positioning encoder.

[0167] The input of the information fusion operator is the knitting needle trajectory feature, the edge information feature, the color clustering feature, and the original feature of the printed pattern design, and the output is the high-dimensional vector of embroidery printing. The calculation method is as follows:

[0168] (1.11)

[0169] Among them is the high-dimensional vector of embroidery printing, is the knitting needle trajectory feature, Edge information feature one, is the edge information feature two, is the color clustering feature, is the original feature of the printed pattern design, is the non-linear activation function one, is the non-linear activation function two, and the specific method is as follows:

[0170] (1.24)

[0171] (1.25)

[0172] Among them, is the weighted merging operation, and its calculation process is as follows: the input edge information feature is divided into edge information feature one and edge information feature two along the channel dimension; edge information feature one is added to the knitting needle trajectory feature, edge information feature two is multiplied by the color clustering feature, and finally the two are weighted and merged with the original feature of the printed pattern design to obtain the high-dimensional vector of embroidery printing; among them, the weighted average operation calculation method is as follows:

[0173] (1.12)

[0174] Among them is the high-dimensional vector of embroidery printing, , , respectively represent different learnable weights, which are used to adjust the proportion of different information in the generation of embroidery-style printing patterns, , , The calculation method is as follows:

[0175] (1.13)

[0176] (1.14)

[0177] (1.15)

[0178] The calculation method of the total loss function of the model is as follows:

[0179] (1.16)

[0180] Total loss function is composed of the first loss function and the second loss function respectively. The first loss function is used to judge the color difference, and the second loss function evaluates the authenticity of the stitch; is used to adjust the proportion of the first loss function in the total loss function , and the set range is (0.35 - 0.45), is used to adjust the proportion of the second loss function in the total loss function , and the set range is (0.55 - 0.65); among them, the first loss function The calculation method is:

[0181] (1.17)

[0182] Among them, represents the color value of the th pixel of the labeled embroidery-style printing pattern, represents the color value of the th pixel of the generated embroidery-style printing pattern, represents the total number of pixel values;

[0183] The second loss function The calculation method is:

[0184] (1.18)

[0185] Among them, is the total number of pixel points in the image, represents the Hessian matrix of the label embroidery style printed pattern at the position, represents the Hessian matrix of the generated embroidery style printed pattern at the position;

[0186] The second loss function is calculated by using a Gaussian high-pass filter to extract the high-frequency texture features of the image and performing a Fourier transform to obtain a spectrogram, and using the Hessian matrix to describe the second-order derivative information of the local area of the spectrogram to capture the local changes of the texture. The specific calculation method is as follows:

[0187] (1.19)

[0188] Among them, , is the second-order derivative of the spectrogram in the and directions, , is the mixed second-order derivative of the spectrogram; this matrix can describe the local curvature and directionality of the texture.

[0189] When the total loss function is minimized and no longer decreases with training, the four-path information fusion embroidery printing generation model training is completed.

[0190] The specific steps to construct a realistic evaluation model for the printed pattern style are as follows:

[0191] (1)Collect embroidery style printed patterns and their subjective evaluation levels as dataset two, and divide dataset two into a training set, a validation set, and a test set;

[0192] (2)Use dataset two obtained in (1) for training to obtain a realistic evaluation model for the printed pattern style;

[0193] (3)Input the embroidery style printed patterns collected by the image acquisition system into the realistic evaluation model for the printed pattern style in (2) to obtain the corresponding evaluation and feedback it back to the four-path information fusion embroidery printing generation model;

[0194] The ratio of the training set, the validation set, and the test set in dataset two of (1) is 8∶1∶1;

[0195] Such as Figure 3As shown, the realistic evaluation model of the printed pattern style is a convolutional neural network stacked with multiple convolutional blocks. The input is an image of an embroidered-style printed fabric, and the output is a realistic evaluation index. Its main structure is as follows: The first two layers are a convolutional layer with a convolutional kernel size of 7x7 and a max pooling layer of 3x3. Subsequently, it passes through six convolutional blocks, each convolutional block contains 2 - 3 convolutional layers. Different evaluation components are connected after every two convolutional blocks, and residual connections are used between the convolutional blocks. The specific calculation method of the convolutional block is:

[0196] (2.1)

[0197] Among them is the input of the convolutional block, is the output of the convolutional block, is a non-linear activation method, and the specific calculation method is:

[0198] (2.7)

[0199] represents the calculation of the convolutional layer in the rd layer of the convolutional block for , The range of

[0200] is (2, 3), and the specific calculation method is:

[0201] is the calculation method of normalization, and the specific calculation method is:

[0202] (2.8)

[0203] is the 3×3 convolutional operation in the convolutional layer of the th layer.

[0204] In one implementation, the loss function is used to constrain the realistic evaluation model of the printed pattern style. The expression of the loss function is:

[0205] (2.3)

[0206] Among them, is the loss function of the realistic evaluation model of the printed pattern style, is the number of pictures in the second dataset, represents the realistic evaluation index, represents the realistic evaluation index of the model prediction result. When the loss function When it is the smallest and does not decrease during the training process, the training of the printing style realism evaluation model is completed.

[0207] The realism evaluation index is constructed as follows:

[0208] (1) Embroidery style evaluation: The image feature maps of the printed design pattern and the embroidered style printed cloth are passed through a pre-trained neural network model to obtain the overall embroidery style evaluation of the embroidery style. The specific calculation method is as follows:

[0209] (2.4)

[0210] Among them, is the embroidery style evaluation, is the printed design pattern, is the image of the embroidered style printed cloth, is a pre-trained neural network model that can extract the main style features of the embroidery style for comparison and scoring.

[0211] (2) Texture feature evaluation: The image feature maps of the printed design pattern and the embroidered style printed cloth are used to extract features through a Gabor filter and compare them to obtain the texture feature evaluation. The specific calculation method is as follows:

[0212] (2.5)

[0213] Among them, is the texture feature evaluation, is the printed design pattern, is the image of the embroidered style printed cloth, is the th Gabor filter, is the cosine similarity, is the number of Gabor filters; through the texture feature evaluation, the local detail texture of the image of the embroidered style printed cloth can be fed back and optimized.

[0214] (3) Color transfer evaluation: The image feature maps of the printed design pattern and the embroidered style printed cloth are converted to a unified Lab color space, and the main color tone, contrast, and saturation distribution of the two are analyzed through a color clustering algorithm. Finally, the color histogram matching degree is used to quantify the retention degree of the color distribution. The specific calculation method is as follows:

[0215] (2.6)

[0216] Among them, is the color transfer evaluation, represents the weight of the contrast and saturation evaluation, represents the weight of the color histogram matching degree, It is a printed design pattern, It is an image of an embroidered-style printed fabric, It is color space conversion, It is a color clustering algorithm, It is the contrast and saturation comparison of two images, It is the color histogram of the printed design pattern, It is the color histogram of the image of the embroidered-style printed fabric.

[0217] The specific process of feedback of the realistic evaluation index to the four-path information fusion embroidery printing generation model is as follows:

[0218] (1) Embroidered style supplement: When the evaluation of the embroidered style does not meet the expectation, increase the weight of the addition of the edge information feature one and the knitting needle trajectory feature , and enhance the supplement of the embroidered style.

[0219] (2) Texture feature supplement: Add multi-channel embroidery stitches to the knitting needle process feature map through the knitting needle process editing software to enhance the supplement of the stitch texture, and change the weight learned by the edge information encoder .

[0220] (3) Color transfer alignment: When the evaluation of color transfer does not meet the expectation, increase the weight of the multiplication of the edge information feature two and the color clustering feature , and ensure the color alignment of the printed design pattern and the image of the embroidered-style printed fabric.

[0221] Some steps in the embodiments of the present invention can be implemented by software, and the corresponding software program can be stored in a readable storage medium, such as a CD or a hard disk, etc.

[0222] The above are only the preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for generating evaluation feedback of printed pattern embroidery style based on deep learning, characterized in that: The method comprises: S1: Collect print design patterns and corresponding embroidery style print patterns to construct data set 1, which is divided into training set, validation set and test set; S2: training the training set obtained in S1 to obtain a four-path information fusion embroidery print generation model; S3: collect embroidery style print patterns and their corresponding subjective evaluation levels as data set 2, and divide them into training set, validation set and test set; S4: training the training set obtained in S3 to obtain a printing style realism evaluation model; S5: Input the print design pattern into the four-path information fusion embroidery print generation model obtained in S2 to generate an embroidery style print pattern; S6: input the embroidery style print pattern obtained in S5 into the print style realism evaluation model obtained in S4, obtain the corresponding evaluation and feed it back to the four-path information fusion embroidery print generation model; S7: Generate embroidery style print patterns through the embroidery print generation model by fusing the four-path information after feedback; The four-path information fusion embroidery print generation model in S2 includes four information processing paths, an information fusion operator and a decoder, wherein: The first path includes a knitting needle information encoder; The second path includes an edge information extraction module and an edge information encoder; The third path includes a color block information extraction module and a color block positioning encoder; The fourth path includes a raw information encoder; The main structure of the print style realism evaluation model in S4 is as follows: the first two layers are a convolution layer with a convolution kernel size of 7x7 and a 3x3 maximum pooling layer, followed by six convolution blocks, each of which contains 2-3 convolution layers, and residual connections are used between the convolution blocks, and different evaluation components are connected after every two convolution blocks.

2. The method according to claim 1, characterized in that In the first path, the knitting needle process diagram is derived from the knitting needle process editing software, and the calculation method is: To input the print design pattern, To output the knitting needle process feature map, It is the process of knitting needle process editing software, which obtains the knitting process feature map by custom editing in the process feature vector matrix; The knitting needle information encoder in the first path has a knitting needle process feature map as input and a knitting needle trajectory feature as output, and the calculation method is: in, This is a characteristic diagram of knitting needle technology. is the needle trajectory feature, It is composed of three stacked convolutional layers, and the convolution kernel size of each convolutional layer is 3x3. It is a pooling calculation method. is a high-dimensional mapping, which is calculated as: in, are the weights learned by the knitting needle information encoder, Bias learned for the needle information encoder; In the second path, the input of the edge information extraction module is the printing design pattern, and the output is the edge information feature map, which is calculated by: in, To input the print design pattern, Design the coordinates for the print. is the edge information feature map, is a Laplace filter used to supplement high frequencies. is the weight used for Laplace filtering, Indicated in The gradient of the direction, Representative filter The usage weight, Representative filter The direction of the edge or change of interest, Representation direction A collection of filters The calculation method is: in, To input the print design pattern, is to input the coordinates of the printed design pattern, is a nonlinear activation function; The edge information encoder in the second path has an edge information feature map as input and an edge information feature as output, which is calculated as follows: in, is the edge information feature map, is the edge information feature, It is composed of three stacked convolutional layers, and the convolution kernel size of each convolutional layer is 3x3. It is a pooling method. is a high-dimensional mapping, which is calculated as follows: in, are the weights learned by the edge information encoder, Bias learned for edge information encoder; In the third path, the input of the color block information extraction module is the printing design pattern, and the output is the color block information feature map. The calculation method is: is to input the coordinates of the printed design pattern, is the color block information feature map, is the center of the color block. It is the type of color block. is a pre-trained neural network. represents the number of convolution scales, represents the total number of convolution scales, Represents the morphological gradient, which is used to extract the color block boundary. It is a color clustering method; In the third path, the input of the color block location encoder is the color block information feature map, and the output is the color clustering feature. The calculation method is: is the color block information feature map, is the color clustering feature, It is composed of three stacked convolutional layers, and the convolution kernel size of each convolutional layer is 3x3. It is a pooling method. is a high-dimensional mapping, which is calculated as follows: in, The weights learned by the encoder for patch localization, Bias learned by the encoder for patch localization; In the fourth path, the input of the original information encoder is the original information feature map, and the output is the original features of the printing design map.

3. The method according to claim 1, characterized in that The input of the information fusion operator is the needle trajectory feature, edge information feature, color clustering feature, and original feature of the printed design pattern, and the output is the embroidery print high-dimensional vector. The calculation method is: in O is a high-dimensional vector for embroidery printing, is the needle trajectory feature, Edge information feature 1, is the edge information feature 2, is the color clustering feature, Design the original features of the print pattern. is a nonlinear activation function, is the nonlinear activation function 2, It is a weighted merging operation, and its calculation process is as follows: the input edge information features are divided into edge information feature 1 and edge information feature 2 along the channel dimension; edge information feature 1 is added to the needle trajectory feature, and edge information feature 2 is multiplied by the color clustering feature. Finally, the two are weighted merged with the original features of the printing design pattern to obtain the embroidery printing high-dimensional vector. The weighted average operation calculation method is as follows: in is a high-dimensional vector for embroidery printing, , , They represent different learnable weights, which are used to adjust the proportion of different information in the generation of embroidery style print patterns.

4. The method according to claim 1, characterized in that: The input of the decoder is an embroidery print high-dimensional vector, and the output is an embroidery style print pattern, which is calculated as follows: in, O is a high-dimensional vector for embroidery printing, Embroidery style printed pattern. Indicates Layer nonlinear transformation function calculation, Indicates A fully connected layer is calculated, and sig is a nonlinear transformation function.

5. The method according to claim 1, characterized in that The total loss function calculation method of the four-path information fusion embroidery printing generation model is as follows: Total loss function The first loss function and the second loss function Composition, the first loss function Used to judge the color difference, the second loss function Evaluate the authenticity of the needle technique; Used to adjust the first loss function Total loss function The specific gravity is set in the range of 0.35-0.45; Used to adjust the second loss function Total loss function The specific gravity is set in the range of 0.55-0.65; Among them, the first loss function The calculation method is: in, Representative embroidery style print pattern The color value of a pixel, Represents the generated embroidery style print pattern The color value of a pixel, N Represents the total number of pixel values; The second loss function The calculation method is: in, N is the total number of pixels in the image, Representative embroidery style printed patterns The Hessian matrix of the position, Representative generated embroidery style print pattern in Hessian matrix of the position; The second loss function The calculation process is to use a Gaussian high-pass filter to extract the high-frequency texture features of the image and perform Fourier transform to obtain a spectrum map. The Hessian matrix is ​​used to describe the second-order derivative information of the local area of ​​the spectrum map to capture the local changes of the texture. The specific calculation method is: in, , The spectrum is and The second derivative of the direction, , is the mixed second-order derivative of the spectrum map; this matrix describes the local curvature and directionality of the texture; When the total loss function When it is minimum and does not decrease with the training process, the training of the four-path information fusion embroidery print generation model is completed.

6. The method according to claim 1, characterized in that The specific calculation method of the convolution block in the printing style realism evaluation model is: in is the convolutional block input, is the output of the convolutional block, It is a nonlinear activation method. Represents the convolutional block The convolutional layer pair of layers Calculation of The range is (2,3), and the specific calculation method is: GY is the normalized calculation method, It is The 3×3 convolution operation in the convolutional layer of the layer; A loss function is used to constrain the print style realism evaluation model, and the expression of the loss function is: in, is the loss function of the print style realism evaluation model, is the number of images in dataset 2, represents the realism evaluation index, An evaluation index representing the realism of the model prediction results; When the loss function When it is minimum and does not decrease during the training process, the training of the print style realism evaluation model is completed.

7. The method according to claim 1, characterized in that The purpose of the evaluation component is to construct a realism evaluation index, and there are three evaluation indexes as follows: Indicator 1: Embroidery style evaluation: The image feature map of the printed design pattern and the embroidery style printed cloth is used through the pre-trained neural network model to obtain the overall embroidery style evaluation of the embroidery style. The specific calculation method is as follows: For embroidery style evaluation, Design patterns for printing, For embroidery style printed cloth images, It is a pre-trained neural network model that can extract the main style features of embroidery styles for comparison and scoring; Indicator 2: Texture feature evaluation: The image feature maps of the printed design pattern and the embroidery style printed cloth are extracted with Gabor filters and compared to obtain texture feature evaluation. The specific calculation method is as follows: For texture feature evaluation, Design patterns for printing, For embroidery style printed cloth images, It is Gabor filters, is the cosine similarity, N is the number of Gabor filters; the local detail texture of the image of the embroidery style printed cloth is optimized through texture feature evaluation feedback; Indicator 3: Color migration evaluation: convert the image feature maps of the printed design pattern and the embroidery style printed cloth into a unified Lab color space, and analyze the main color tone, contrast and saturation distribution of the two through the color clustering algorithm; finally, use the color histogram matching degree to quantify the degree of color distribution retention. The specific calculation method is as follows: in, For color migration evaluation, Represents the weight of contrast and saturation evaluation, Represents the weight of the color histogram matching degree, Design patterns for printing, For embroidery style printed cloth images, For color space conversion, is the color clustering algorithm, is the contrast and saturation comparison of the two images, Color histogram for designing patterns for prints, Color histogram for an image of embroidery-style printed fabric.

8. The method according to claim 7, characterized in that The specific process of feeding back the realism evaluation index to the four-path information fusion embroidery print generation model is as follows: Indicator 1 feedback: embroidery style supplement; when the embroidery style evaluation does not meet expectations, the weight of the edge information feature 1 plus the needle trajectory feature is increased , to achieve the complement of embroidery style; Feedback for indicator 2: Texture feature supplementation; add multi-channel embroidery stitches to the knitting needle process feature map through the knitting needle process editing software to supplement the stitch texture, and change the weights learned by the edge information encoder ; Indicator 3 feedback: color migration alignment; When the color migration evaluation does not meet expectations, the weight of the edge information feature 2 multiplied by the color clustering feature is increased. , ensuring the alignment of the image colors of the printed design pattern and embroidery style printed fabric.

Images