Automatic embroidery pattern making method based on neural network

Abstract

The present invention relates to the technical field of embroidery design, and in particular to an automatic embroidery pattern making method based on a neural network. The method comprises inputting a style image and a Gaussian noise image into a trained VGG19 network, and outputting an embroidery finalized pattern image. The style loss function of the sampled image is weighted by dividing the historical embroidery images in the embroidery data set, and determining the window weights of the divided windows after the division, to obtain the final style loss function. The present invention improves the recognition ability of the style transfer network for the texture layering of embroidery, and improves the authenticity of the new embroidery pattern generated by the adversarial network in the trained VGG19 network.

Description

Technical Field

[0001] The invention relates to the technical field of embroidery design, and in particular to an automatic embroidery pattern making method based on a neural network. Background Art

[0002] The automatic embroidery pattern making method is an innovative method that combines traditional embroidery technology with modern artificial intelligence technology. At present, the automatic embroidery pattern making mainly uses deep learning and convolutional neural networks (CNN) to simulate and generate images with embroidery art style, thereby realizing automatic embroidery pattern production. Among them, it is necessary to extract the content features of the content image and the style texture features of the embroidery art style image through the VGG-NET network, and perform style transfer in the target area.

[0003] The existing VGG-NET network has two loss functions: content loss function and style loss function. The style loss function is based on the Gram matrix to measure the difference in style features between the generated image and the style image. For embroidery patterns, texture layering is a feature that can greatly reflect the difference between different embroidery patterns. However, due to the complexity of the image content, this texture layering is easily blurred during the neural network recognition process, which in turn affects the authenticity and layering of the new embroidery pattern generated by the generative adversarial network. Summary of the invention

[0004] In order to solve the technical problem that the authenticity and hierarchy of the new embroidery pattern generated by the generative adversarial network are low, the purpose of the present invention is to provide an automatic embroidery pattern making method based on a neural network, and the technical scheme adopted is as follows:

[0005] In a first aspect, an embodiment of the present invention provides an automatic embroidery pattern making method based on a neural network, the method comprising:

[0006] Input the style image and Gaussian noise image into the trained VGG19 network and output the embroidery version image;

[0007] Among them, the construction method of the style loss function in the VGG19 network is:

[0008] According to the partitioning window, each historical embroidery image in the embroidery data set is partitioned to determine the energy value of each partitioning window, wherein the size of the partitioning window is adjusted in an ascending order;

[0009] Segment the energy value sequence formed by the energy values ​​of the divided windows of the same size to obtain the divided segments; analyze the data frequency of the divided segments corresponding to the adjacent divided windows and the consistency of the textures in the divided windows to determine the combined texture and the single texture;

[0010] hierarchical clustering is performed on the same single texture and the same combined texture respectively to obtain hierarchical coordinate trees of the single texture and the combined texture; the hierarchical coordinate trees of the single texture and the combined texture are matched to obtain a single texture scale and a combined texture scale;

[0011] Based on the single texture scale and the combined texture scale, the window weight of the divided window is determined; for the Gaussian noise image and the style image, the corresponding sampled image is obtained by dividing the window, and the style loss function of the sampled image is weighted according to the window weight of each divided window to obtain the final style loss function.

[0012] Furthermore, the hierarchical clustering is performed on the same single texture and the same combined texture respectively to obtain the hierarchical coordinate trees of the single texture and the combined texture, including:

[0013] Calculate the ratio of the number of energy values ​​in each segment to the number of all energy values ​​to obtain the occurrence frequency of each segment;

[0014] The ratio of the number of segmented segments to the number of partitioned windows is used as a metric;

[0015] When the occurrence frequency is greater than the criterion, the edge in the partition window of the corresponding segment is taken as a texture edge;

[0016] Compare the frequencies of energy values ​​in the segmentation segment of the current segmentation window and the segmentation segment of the previous segmentation window, determine the reduction rate of the segmentation window, and judge whether the texture edge in the obtained segmentation window is a combined texture or a single texture;

[0017] The texture descriptor of the texture edge in each divided window is calculated, and single textures with the same texture descriptor are divided into the same category, and combined textures with the same texture descriptor are divided into the same category, thereby obtaining multiple single textures and multiple combined textures.

[0018] Furthermore, the comparing the frequencies of the energy values ​​in the segment to which the current segment window belongs and the segment to which the previous segment window belongs to determine the reduction rate of the segment window includes:

[0019] Calculate the frequency of the energy value with the largest proportion in the segment to which the partition window belongs, and record it as the maximum frequency; use the difference between the maximum frequencies of the segment to which the previous partition window belongs and the segment to which the current partition window belongs as the numerator, and the maximum frequency of the segment to which the previous partition window belongs as the denominator; the ratio formed by the numerator and the denominator is taken as the reduction rate of the partition window.

[0020] Furthermore, the determining whether the texture edge in the divided window is a combined texture or a single texture includes:

[0021] When the reduction rate of the divided window is greater than a preset texture threshold, the texture edge in the divided window is determined to be a combined texture;

[0022] When the reduction rate of the divided window is less than or equal to the preset texture threshold, the texture edge in the divided window is determined to be a single texture.

[0023] Furthermore, the hierarchical clustering is performed on the same single texture and the same combined texture respectively to obtain the hierarchical coordinate trees of the single texture and the combined texture, including:

[0024] The center point coordinates of each divided window are used as the window coordinates;

[0025] For the partitioned windows corresponding to the same single texture, the inverse of the Euclidean distance between any two partitioned window coordinates is used as the similarity between the windows. Through the bottom-up clustering method, a hierarchical clustering tree of the same single texture is obtained, which is recorded as the hierarchical coordinate tree of the single texture.

[0026] For the divided windows corresponding to the same combined texture, the inverse of the Euclidean distance between any two divided window coordinates is taken as the similarity between the windows. Through the bottom-up clustering method, a hierarchical clustering tree of the same combined texture is obtained, which is recorded as the hierarchical coordinate tree of the combined texture.

[0027] Furthermore, matching the hierarchical coordinate trees of the single texture and the combined texture to obtain the single texture scale and the combined texture scale includes:

[0028] The hierarchical coordinate tree of a single texture is used as the right node, and the hierarchical coordinate tree of a combined texture is used as the left node; the mean coordinate of all coordinates in each node is used as the representative coordinate of the node;

[0029] Match the nodes on the left and right sides to obtain the matching edge value between the nodes on the two sides; when the matching edge value is less than the preset matching threshold, determine that the single texture corresponding to the right node constitutes the combined texture corresponding to the left node, and use the window scale of the right node as the single texture scale and the window scale of the left node as the combined texture scale;

[0030] The method for obtaining the matching edge value is as follows: calculating the Euclidean distance of the representative coordinates of any two nodes in the same layer in the hierarchical coordinate tree corresponding to the left and right nodes, and using the variance of all Euclidean distances corresponding to the hierarchical coordinate tree of the left and right nodes as the matching edge value.

[0031] Furthermore, the determining the window weight of the divided window based on the single texture scale and the combined texture scale includes:

[0032] The sum of the matching edge values ​​corresponding to all single textures belonging to the divided window corresponding to the single texture scale is used as the window weight of the divided window corresponding to the single texture scale;

[0033] The matching edge value of each combined texture in the left node is used as the window weight of the divided window corresponding to the combined texture scale; wherein the window weight is a normalized value.

[0034] Furthermore, the method of dividing each historical embroidery image in the embroidery data set according to the divided window and determining the energy value of each divided window includes:

[0035] Grayscale quantization is performed on each historical embroidery image in the embroidery dataset, and the grayscale level in each historical embroidery image is quantized into N levels.

[0036] Furthermore, the method of dividing each historical embroidery image in the embroidery data set according to the divided window and determining the energy value of each divided window includes:

[0037] Obtaining gray level co-occurrence matrices corresponding to different directions of the divided windows, and determining the energy value of each gray level co-occurrence matrix;

[0038] The maximum energy value is taken as the energy value of the corresponding divided window.

[0039] Furthermore, the energy value sequence formed by dividing the energy values ​​of the divided windows of the same size to obtain the divided segments includes:

[0040] In order from small to large, an energy value sequence is constructed from the energy values ​​corresponding to the divided windows of the same size;

[0041] The energy value sequence is segmented using the Otsu multi-threshold segmentation algorithm to obtain a plurality of segmentation segments.

[0042] In a second aspect, an automatic embroidery pattern making system based on a neural network is provided, the system comprising the following modules:

[0043] The generation module is used to input the style image and Gaussian noise image into the trained VGG19 network and output the embroidery finalized image;

[0044] Training module, used to construct the style loss function in the VGG19 network:

[0045] According to the partitioning window, each historical embroidery image in the embroidery data set is partitioned to determine the energy value of each partitioning window, wherein the size of the partitioning window is adjusted in an ascending order;

[0046] Segment the energy value sequence formed by the energy values ​​of the divided windows of the same size to obtain the divided segments; analyze the data frequency of the divided segments corresponding to the adjacent divided windows and the consistency of the textures in the divided windows to determine the combined texture and the single texture;

[0047] hierarchical clustering is performed on the same single texture and the same combined texture respectively to obtain hierarchical coordinate trees of the single texture and the combined texture; the hierarchical coordinate trees of the single texture and the combined texture are matched to obtain a single texture scale and a combined texture scale;

[0048] Based on the single texture scale and the combined texture scale, the window weight of the divided window is determined; for the Gaussian noise image and the style image, the corresponding sampled image is obtained by dividing the window, and the style loss function of the sampled image is weighted according to the window weight of each divided window to obtain the final style loss function.

[0049] In a third aspect, an embodiment of the present invention provides an electronic device, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, various possible embodiments of the first aspect are implemented.

[0050] In a fourth aspect, an embodiment of the present invention provides a computer program product, which includes: a computer program code, which, when executed on a computer, enables the computer to execute the method in the first aspect or any possible implementation of the first aspect.

[0051] In a fifth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, enables the computer to execute various possible implementations of the first aspect.

[0052] The embodiments of the present invention have at least the following beneficial effects:

[0053] The present invention relates to the technical field of embroidery design. Since embroidery itself is composed of different textures, and then different single textures form a combined texture, the combined texture is regarded as a basic individual, and then combined to form an overall embroidery texture. The present invention focuses on the data scale that can reflect the texture hierarchy of embroidery, first determines the combined texture and the single texture, and matches the combined texture and the single texture to determine the appropriate single texture scale and the combined texture scale, so as to determine the window weights of different divided windows, and finally achieve the weighting of the style loss function, improve the recognition ability of the style transfer network for the texture hierarchy of embroidery, and improve the authenticity of the adversarial network in the trained VGG19 network to generate a new embroidery pattern. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0055] Figure 1 A flowchart of the steps of a method for constructing a style loss function in a VGG19 network provided by an embodiment of the present invention;

[0056] Figure 2 A schematic diagram of a hierarchical coordinate tree provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0057] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined purpose of the invention, the specific implementation method, structure, characteristics and effects of the automatic embroidery pattern making method based on neural network proposed by the present invention are described in detail as follows in combination with the accompanying drawings and preferred embodiments.

[0058] In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0059] Among them, in the description of the embodiments of the present invention, unless otherwise specified, " / " means or, for example, A / B can mean A or B: "and / or" in the text is only a way to describe the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present invention, "multiple" refers to two or more than two.

[0060] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features.

[0061] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0062] The embodiments of the present invention are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided by the embodiments of the present invention are also applicable to similar technical problems.

[0063] The embodiment of the present invention provides a specific implementation method of an automatic embroidery pattern making method based on a neural network, and the method is applicable to an embroidery pattern making scenario. In this scenario, the database contains an embroidery data set consisting of historical embroidery images.

[0064] The specific scheme of the automatic embroidery pattern making method based on neural network provided by the present invention is described in detail below with reference to the accompanying drawings.

[0065] Firstly, the embodiment of the present invention collects historical embroidery images from a database and constructs an embroidery dataset from the historical embroidery images.

[0066] In an embodiment of the present invention, historical embroidery images are collected by selecting embroidery-related websites, platforms or social media as data sources to collect embroidery images, patterns and descriptions.

[0067] After collecting historical embroidery images, use crawlers to capture data: use crawler tools such as Scrapy and BeautifulSoup to crawl embroidery patterns and related text information, such as materials and techniques, from web pages.

[0068] Furthermore, data cleaning was performed to remove irrelevant or useless images and texts to ensure uniform image formats and accurate text content of historical embroidery images.

[0069] Secondly, perform data storage, store images and text in appropriate formats, such as JPEG, PNG, CSV, JSON, etc., and record relevant information of each image, such as name, description, etc.

[0070] Finally, the data is stored in blocks to facilitate subsequent use and loading, ensuring easy management.

[0071] In the process of style transfer using the VGG19 network, the network input, output, and loss function are key components, as follows:

[0072] 1. Input:

[0073] Content Image: This is the image whose content you want to preserve after style transfer.

[0074] Style Image: This is an image that provides style features, and its artistic style will be transferred to the content image.

[0075] Generated Image: It is usually a Gaussian noise image at the beginning. It is gradually adjusted through the optimization process to combine the content of the content image and the style of the style image. Therefore, the input generated image can also be directly recorded as a Gaussian noise image.

[0076] 2. Output:

[0077] The output of the network is a new generated image after style transfer processing. It should retain the main structure and details of the content image while showing the artistic style of the style image. The output new generated image is different from the initial Gaussian noise image.

[0078] 3. Loss function (Content Loss):

[0079] Content loss measures the difference in content features between the generated image and the content image. It is usually calculated on a specific layer of the VGG19 network, such as "block5_conv2", to ensure that the generated image retains the structure of the content image.

[0080] Style Loss: The style loss function measures the difference in style features between the generated image and the style image based on the Gram matrix. The Gram matrix is ​​a tool for capturing the style texture of an image. By comparing the Gram matrices of the generated image and the style image, it can ensure that the style of the generated image is similar to that of the style image.

[0081] Total Variation Loss: This is a regularization term used to maintain the smoothness of the generated image and prevent overfitting. It is achieved by calculating the differences between adjacent pixels of the image.

[0082] Total Loss: The total loss is the weighted sum of the content loss function, the style loss function, and the total variational loss function, which is used to guide the optimization process of generating images. The choice of weights has an important impact on the final style transfer effect.

[0083] The embodiment of the present invention takes into account that when the traditional method calculates the style loss function, the image is calculated as a whole. For embroidery, texture layering is a feature that can greatly reflect the difference between different embroidery patterns. Therefore, by calculating the loss of textures at different scales, the style loss function is obtained by a weighted method.

[0084] The style image and Gaussian noise image are input into the trained VGG19 network, and the embroidery finalized image is output.

[0085] Among them, the construction method of the style loss function in the VGG19 network can be found in Figure 1 , which shows a flowchart of a method for constructing a style loss function in a VGG19 network provided by an embodiment of the present invention, the method comprising the following steps:

[0086] Step S100, dividing each historical embroidery image in the embroidery data set according to the dividing window, and determining the energy value of each dividing window, wherein the size of the dividing window is adjusted in order from small to large.

[0087] There may be single textures and combined textures in embroidery images. Single texture refers to a single type of texture, and combined texture refers to multiple single textures. This forms a bigger problem. Single texture and combined texture usually appear multiple times in embroidery images. Corresponding to embroidery, they can also be understood as basic textures and combined textures. Embroidery itself is formed by different textures, and different single textures form combined textures. Combined textures are regarded as basic individuals, and then combined to form an overall embroidery texture.

[0088] For each embroidery image, gray quantization is first performed, that is, each historical embroidery image in the embroidery dataset is divided according to the division window, and before the energy value of each division window is determined, gray quantization is performed on each embroidery image.

[0089] Grayscale quantization is performed on each historical embroidery image in the embroidery data set, and the grayscale in each historical embroidery image is quantized into N levels. More specifically, the grayscale value of the pixel is divided by N and rounded. In the embodiment of the present invention, the value of N is 32, and the data is rounded down. For example, when the pixel value of a certain pixel is 4, after the pixel value of the pixel is quantized, the pixel value of the pixel is 0.

[0090] Each historical embroidery image in the embroidery dataset is divided according to a fixed-size division window, and the energy value of each division window is determined, wherein the size of the division window is adjusted in order from small to large.

[0091] More specifically, 3×3, 5×5, …, (n / 20)×(n / 20) are used as the size of the gray-level co-occurrence matrix, that is, the size of the divided window. Here, n is the smaller value of the number of rows and columns of the embroidery image.

[0092] Taking a 3×3 size divided window as an example, four directions can be obtained. In each direction, the distance between the center point and the edge point in the window is used as the step size of the grayscale point pair to obtain the grayscale co-occurrence matrix in each direction, and then the energy value of the grayscale co-occurrence matrix in each direction can be obtained. If the element values ​​of the grayscale co-occurrence matrix are similar, the energy is small, indicating a fine texture; if some values ​​are large and others are small, the energy value is large. The larger the energy value, the more uniform the texture pattern, and the more regular the changes.

[0093] The method for obtaining the energy value of the gray level co-occurrence matrix is ​​a well-known technique to those skilled in the art and will not be described in detail here.

[0094] For the gray level co-occurrence matrix of each direction in the 5×5 size partition window, the energy value of the 5×5 size partition window is obtained. It should be noted that, for example, a 3×3 partition window can only have 4 directions, and a 5×5 partition window can have 8 directions. The gray level co-occurrence matrix of each direction that can be calculated under the partition window size is calculated.

[0095] Since each divided window corresponds to multiple grayscale co-occurrence matrices in different directions, the energy value of each grayscale co-occurrence matrix can be obtained. Furthermore, in an embodiment of the present invention, the maximum energy value in the grayscale co-occurrence matrices in multiple directions corresponding to the divided window is used as the energy value of the corresponding divided window.

[0096] Step S200, segmenting the energy value sequence formed by the energy values ​​of the partitioned windows of the same size to obtain the partitioned segments; analyzing the data frequency of the partitioned segments corresponding to the adjacent partitioned windows and the consistency of the textures within the partitioned windows to determine the combined texture and the single texture.

[0097] For divided windows of the same size, an energy value sequence is constructed from the energy values ​​corresponding to the divided windows of the same size in ascending order from small to large.

[0098] The energy value sequence is segmented by Otsu's multi-threshold segmentation algorithm to obtain multiple segments. The energy values ​​in the same segment are similar, and if a certain energy value represents a texture, the frequency of occurrence of this energy value is relatively high.

[0099] After the energy values ​​with similar values ​​are divided into the same segment, the segment to which the segment window and the segment window at the same position of the previous size belong is analyzed to determine whether the segment window corresponds to the combined texture or the single texture. Specifically:

[0100] Calculate the ratio of the number of energy values ​​in each segment to the number of all energy values ​​to obtain the frequency of occurrence of each segment; use the ratio of the number of segments to the number of partition windows as a criterion; when the frequency of occurrence is greater than the criterion, use the edge in the partition window of the corresponding segment as a texture edge; compare the frequency of the energy values ​​in the segment to which the current partition window belongs and the segment to which the previous partition window belongs to determine the reduction rate of the partition window, and judge whether the texture edge in the partition window is a combined texture or a single texture. That is, if the texture edge in the partition window is a combined texture, the partition window is a partition window corresponding to the combined texture; if the texture edge in the partition window is a single texture, the partition window is a partition window corresponding to the single texture.

[0101] Among them, the method for determining the reduction rate of the partition window is: calculate the frequency of occurrence of the energy value with the largest proportion in the segment to which the partition window belongs, recorded as the maximum frequency; use the difference between the maximum frequencies of the segment to which the previous partition window belongs and the segment to which the current partition window belongs as the numerator, and the maximum frequency of the segment to which the previous partition window belongs as the denominator, and the ratio formed by the numerator and the denominator is used as the reduction rate of the partition window.

[0102] The method for determining whether the texture edge in the divided window is a combined texture or a single texture is as follows: when the reduction rate of the divided window is greater than a preset texture threshold, the texture edge corresponding to the divided window is determined to be a combined texture; when the reduction rate of the divided window is less than or equal to the preset texture threshold, the texture edge corresponding to the divided window is determined to be a single texture. In the embodiment of the present invention, the value of the preset texture threshold is 0.3, and in other embodiments, the implementer adjusts it according to actual conditions.

[0103] More specifically, after determining the partitioning windows corresponding to the single texture and the combined texture, the single texture and the combined texture are partitioned, the texture descriptors of the texture edges in each partitioning window are calculated, the single textures with the same texture descriptors are partitioned into the same category, and the combined textures with the same texture descriptors are partitioned into the same category, thereby obtaining multiple single textures and multiple combined textures. The texture descriptor is obtained by following the following method: along the counterclockwise direction of the edge of the partitioning window, taking any edge point of the edge of the partitioning window as the target edge point, taking the target edge point as the starting point, and taking the previous edge point of the target edge point as the end point, a descriptor sequence can be obtained; changing the starting point, obtaining multiple descriptor sequences corresponding to the partitioning window, and then taking the smallest number of the numbers formed by all descriptor sequences as the texture descriptor, for example, if the descriptor sequence is {5,4,8,7}, then the number formed by the descriptor sequence is 5487. The texture descriptor is also the minimum chain code, and even if the edge of the minimum chain code rotates, the corresponding minimum chain code remains unchanged.

[0104] Step S300 , hierarchical clustering is performed on the same single texture and the same combined texture respectively to obtain hierarchical coordinate trees of the single texture and the combined texture; and the hierarchical coordinate trees of the single texture and the combined texture are matched to obtain a single texture scale and a combined texture scale.

[0105] After obtaining which textures in the divided windows are single textures and which textures in the divided windows are combined textures. Hierarchical clustering is performed on the combined textures and single textures respectively. If the clustering trees corresponding to a combined texture and a single texture are similar, and the coordinate points of different categories in each layer of the clustering tree are also similar, it means that the combined texture is indeed obtained by combining single textures, rather than an incomplete combined texture. For example, the window scale of a single texture is 7×7, the window scale of another single texture is 9×9, and the window scale of the combined texture of the two textures is 16×16. However, there are also combined textures in windows smaller than 16×16, but the combined textures are incomplete. The embodiment of the present invention wants a window that reflects the texture scale. By assigning a larger loss weight to the information on these scales, the recognition ability of the neural network is improved. Therefore, it is necessary to obtain the scale of the complete combined texture.

[0106] Therefore, in the embodiment of the present invention, the single texture and the combined texture are clustered respectively. For example, for a single texture, each single texture corresponds to multiple divided windows. For the divided windows corresponding to the same single texture, the center point coordinates of each divided window are used as the window coordinates; the reciprocal of the Euclidean distance between any two window coordinates is used as the similarity between the corresponding two windows. Through the bottom-up clustering method, a hierarchical clustering tree of the same single texture is obtained, which is recorded as the hierarchical coordinate tree of the single texture.

[0107] The same calculation method is used to obtain the hierarchical coordinate tree of each combined texture. Specifically, for the partitioned windows corresponding to the same combined texture, the inverse of the Euclidean distance between any two partitioned window coordinates is used as the similarity between the windows, and the hierarchical clustering tree of the same combined texture is obtained through the bottom-up clustering method, which is recorded as the hierarchical coordinate tree of the combined texture.

[0108] See also Figure 2 , Figure 2 A schematic diagram of a hierarchical coordinate tree. Figure 2 In the figure, since the distances between the two coordinates (1, 2) and (5, 6) are relatively close, they are first clustered into a new category, and then the new category and (13, 14) are clustered into another new category.

[0109] For each node in the layers higher than the first layer of the hierarchical coordinate tree, the mean coordinate of all coordinates in each node is used as the representative coordinate of the node.

[0110] The hierarchical coordinate trees of the single texture and the combined texture are matched to obtain the single texture scale and the combined texture scale. In the embodiment of the present invention, the hierarchical coordinate trees of the single texture and the combined texture are matched by KM matching.

[0111] KM matching is a method for calculating a one-to-one match between left and right nodes, where each node on the left is connected to the right by an edge. In the present invention, the hierarchical coordinate tree of each combined texture is used as the left node, and the hierarchical coordinate tree of each single texture is used as the right node, that is, the hierarchical coordinate tree of the single texture is used as the right node, and the hierarchical coordinate tree of the combined texture is used as the left node.

[0112] The left and right nodes are matched to obtain the matching edge value between the nodes on both sides; when the matching edge value is less than the preset matching threshold, it is determined that the single texture corresponding to the right node constitutes the combined texture corresponding to the left node, and the window scale of the right node is used as the single texture scale, and the window scale of the left node is used as the combined texture scale. In the embodiment of the present invention, the preset matching threshold value is 0.3, and in other embodiments, it can be adjusted by the implementer according to actual conditions.

[0113] The method for obtaining the matching edge value is as follows: calculating the Euclidean distance of the representative coordinates of any two nodes in the same layer in the hierarchical coordinate tree corresponding to the left and right nodes, and using the variance of all Euclidean distances corresponding to the hierarchical coordinate tree of the left and right nodes as the matching edge value.

[0114] More specifically, the similarity between the hierarchical coordinate trees is calculated as follows, that is, the matching edge value of the nodes on both sides is calculated as follows: the difference between the representative coordinates of the nodes in the same layer in the hierarchical coordinate tree is calculated, and the difference between the representative coordinates is characterized by the Euclidean distance in the embodiment of the present invention. When calculating the difference in the coordinates of the first layer, firstly, a one-to-one match of the first-layer nodes of the two hierarchical coordinate trees is obtained by KM matching, and the difference value of each match is obtained, that is, the difference of each matching pair of the first-layer nodes is obtained, wherein the first layer is also the bottom layer of the tree structure; then, when calculating the difference in the coordinates of the second layer, firstly, a KM matching of the coordinates of the second layer is performed to obtain a one-to-one match, that is, the difference of each matching pair of the second-layer nodes is obtained. For the connected nodes on the left and right sides, the difference values ​​of all matching pairs of the two hierarchical coordinate trees are formed into a sequence, and the sequence variance is used as the matching edge value between the two hierarchical coordinate trees; through KM matching, a one-to-one matching relationship between the left node and the right node is obtained. If the distribution of the left node and the right node is the same, that is, there is a situation where a single texture on the right side forms a combined texture on the left side, then the difference values ​​of all matching pairs are similar.

[0115] Step S400, based on the single texture scale and the combined texture scale, determine the window weight of each divided window; for the Gaussian noise image and the style image, obtain the corresponding sampled image by dividing the window, and weight the style loss function of the sampled image according to the window weight of each divided window to obtain the final style loss function.

[0116] Since there can be multiple single textures under each partition window corresponding to a single texture scale, there are multiple single textures on the right side of the KM matching that belong to the same partition window. The sum of the matching edge values ​​corresponding to all single textures belonging to the partition window corresponding to a single texture scale is used as the window weight of the partition window corresponding to the single texture scale.

[0117] The matching edge value of each combined texture on the left is used as the window weight of the divided window corresponding to the combined texture scale, thereby obtaining the window weight of the divided window corresponding to each combined texture scale. The window weight of each divided window is normalized, and the normalized result value is used as the final window weight of each divided window. In the embodiment of the present invention, the divided window under any single texture scale and combined texture scale is used as the target divided window, the window weight of the target divided window is used as the numerator, the sum of all divided windows is used as the denominator, and the ratio formed by the numerator and the denominator is used as the final window weight of the target divided window, thereby realizing the normalization operation of the initially obtained window weight.

[0118] In the present invention, the generated image and the style image are respectively obtained by dividing the window at each single texture scale and the combined texture scale to obtain the sampled image, and then the style loss of the same sampled image is calculated, and then the final style loss is obtained by the weighted loss method. More specifically, the sampled images corresponding to the divided windows have their own corresponding loss functions, and the window weights of the divided windows are used as the weights of the loss functions. The loss functions of all sampled images are weighted and summed to obtain the final style loss. It should be noted that the generated image here is the initial generated image, that is, the Gaussian noise image, and the style image and the Gaussian noise image can be directly obtained.

[0119] The training of the VGG19 network is completed by combining the style loss and the content loss. It should be noted that the present invention is to adjust the style loss function, but not the content loss function.

[0120] Initialize the generator: In a GAN network, the generator is responsible for generating new images. The generator network can be initialized, using Gaussian noise as input, and gradually adjusted through an optimization process to combine the content of the content image and the style of the style image.

[0121] Training the Generator and Discriminator: In the GAN network, the Discriminator is responsible for distinguishing between generated images and real images. During training, the Generator tries to deceive the Discriminator, while the Discriminator continuously learns to recognize the generated images. This process is achieved through adversarial training until the Generator can produce sufficiently realistic images.

[0122] Style transfer: Use the trained VGG19 network and GAN network for style transfer. By optimizing the style loss function, the generator network is controlled to generate a new generated image, which is similar to the content image in content and consistent with the style image in style.

[0123] Optimization and adjustment: During the training process, you may need to adjust the weight parameters in the style loss function to balance the effects of content preservation and style transfer. Through multiple iterations, the quality of the generated images is gradually optimized.

[0124] Generate embroidery patterns: After sufficient training, the generator network is able to generate images with embroidery styles. These images can be used as embroidery patterns for actual embroidery production.

[0125] Post-processing: The generated embroidery images may require further post-processing, such as color correction, clarity enhancement, etc., to meet the needs of actual embroidery.

[0126] Evaluation and iteration: Evaluate the generated embroidery patterns to check their visual effects and artistic expression. If necessary, adjust the network parameters or fusion strategy to optimize the final result.

[0127] The embodiment of the present invention provides an embroidery pattern automatic plate making system based on a neural network, the system comprising:

[0128] The generation module is used to input the style image and Gaussian noise image into the trained VGG19 network and output the embroidery finalized image;

[0129] Training module, used to construct the style loss function in the VGG19 network:

[0130] According to the partitioning window, each historical embroidery image in the embroidery data set is partitioned to determine the energy value of each partitioning window, wherein the size of the partitioning window is adjusted in an ascending order;

[0131] Segment the energy value sequence formed by the energy values ​​of the divided windows of the same size to obtain the divided segments; analyze the data frequency of the divided segments corresponding to the adjacent divided windows and the consistency of the textures in the divided windows to determine the combined texture and the single texture;

[0132] hierarchical clustering is performed on the same single texture and the same combined texture respectively to obtain hierarchical coordinate trees of the single texture and the combined texture; the hierarchical coordinate trees of the single texture and the combined texture are matched to obtain a single texture scale and a combined texture scale;

[0133] Based on the single texture scale and the combined texture scale, the window weight of the divided window is determined; for the Gaussian noise image and the style image, the corresponding sampled image is obtained by dividing the window, and the style loss function of the sampled image is weighted according to the window weight of each divided window to obtain the final style loss function.

[0134] Optionally, the transmission medium may be a wired link, such as but not limited to coaxial cable, optical fiber, and digital subscriber line, or a wireless link, such as but not limited to Wireless Fidelity (WIFI), Bluetooth, and mobile device network.

[0135] It should be noted that the device provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device can be divided into different functional modules to complete all or part of the functions described above.

[0136] Exemplarily, the computer device includes: a memory, a processor, and a computer program stored in the memory and running on the processor, wherein when the processor executes the computer program, the computer device can execute any of the neural network-based automatic embroidery pattern making methods introduced above.

[0137] In addition, an embodiment of the present invention also protects a device, which may include a memory and a processor, wherein the memory stores an executable program code, and the processor is used to call and execute the executable program code to execute the neural network-based embroidery pattern automatic plate making method provided by an embodiment of the present invention.

[0138] The embodiment of the present invention can divide the functional modules of the device according to the above method example. For example, each functional module can be corresponded, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware. It should be noted that the division of modules in this embodiment is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0139] In the case of dividing each module according to each function, the device may also include a signal uploading module, a determining module, an adjusting module, etc. It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module, which will not be repeated here.

[0140] It should be understood that the device provided in the embodiment of the present invention is used to execute the above-mentioned neural network-based automatic embroidery pattern making method, and thus can achieve the same effect as the above-mentioned implementation method.

[0141] In the case of an integrated unit, the device may include a processing module and a storage module. When the device is applied to a device, the processing module may be used to control and manage the actions of the device. The storage module may be used to support the device to execute mutual program codes, etc. The processing module may be a processor or a controller, which may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of the present invention. The processor may also be a combination that implements a computing function, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc. The storage module may be a memory.

[0142] In addition, the device provided in the embodiment of the present invention can specifically be a chip, a component or a module. The chip may include a connected processor and a memory; wherein the memory is used to store instructions. When the processor calls and executes the instructions, the chip can execute the automatic embroidery pattern making method based on neural network provided in the above embodiment.

[0143] An embodiment of the present invention also provides a computer-readable storage medium, which stores a computer program code. When the computer program code is run on a computer, the computer executes the above-mentioned related method steps to implement the neural network-based automatic embroidery pattern making method provided in the above embodiment.

[0144] The embodiment of the present invention further provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the above-mentioned related steps to implement the automatic embroidery pattern making method based on neural network provided in the above embodiment.

[0145] Among them, the device, computer-readable storage medium, computer program product or chip provided in the embodiments of the present invention are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here. Through the description of the above implementation methods, technicians in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways.

[0146] The device embodiments described above are merely illustrative. For example, the division of modules or units is only a logical function division. There may be other divisions in actual implementation. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection of devices or units, which may be electrical, mechanical or other forms.

[0147] It should also be noted that, in this article, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0148] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0149] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0150] The above contents are only specific implementation modes of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be covered within the protection scope of the present invention.

Claims

1. A method for automatic embroidery pattern making based on neural network, characterized in that: The method comprises the following steps: Input the style image and Gaussian noise image into the trained VGG19 network and output the embroidery version image; Among them, the construction method of the style loss function in the VGG19 network is: According to the partitioning window, each historical embroidery image in the embroidery data set is partitioned to determine the energy value of each partitioning window, wherein the size of the partitioning window is adjusted in an ascending order; Segment the energy value sequence formed by the energy values ​​of the divided windows of the same size to obtain the divided segments; analyze the data frequency of the divided segments corresponding to the adjacent divided windows and the consistency of the textures in the divided windows to determine the combined texture and the single texture; hierarchical clustering is performed on the same single texture and the same combined texture respectively to obtain hierarchical coordinate trees of the single texture and the combined texture; the hierarchical coordinate trees of the single texture and the combined texture are matched to obtain a single texture scale and a combined texture scale; Based on the single texture scale and the combined texture scale, the window weight of the divided window is determined; for the Gaussian noise image and the style image, the corresponding sampled image is obtained by dividing the window, and the style loss function of the sampled image is weighted according to the window weight of each divided window to obtain the final style loss function.

2. The method for automatic embroidery pattern making based on neural network according to claim 1, characterized in that: The step of performing hierarchical clustering on the same single texture and the same combined texture to obtain hierarchical coordinate trees of the single texture and the combined texture comprises: Calculate the ratio of the number of energy values ​​in each segment to the number of all energy values ​​to obtain the occurrence frequency of each segment; The ratio of the number of segmented segments to the number of partitioned windows is used as a metric; When the occurrence frequency is greater than the criterion, the edge in the partition window of the corresponding segment is regarded as a texture edge; Compare the frequencies of energy values ​​in the segmentation segment of the current segmentation window and the segmentation segment of the previous segmentation window, determine the reduction rate of the segmentation window, and judge whether the texture edge in the obtained segmentation window is a combined texture or a single texture; The texture descriptor of the texture edge in each divided window is calculated, and single textures with the same texture descriptor are divided into the same category, and combined textures with the same texture descriptor are divided into the same category, thereby obtaining multiple single textures and multiple combined textures.

3. The method for automatic embroidery pattern making based on neural network according to claim 2, characterized in that: The step of comparing the frequency of energy values ​​in the segment to which the current segment window belongs and the segment to which the previous segment window belongs to determine the reduction rate of the segment window includes: Calculate the frequency of the energy value with the largest proportion in the segment to which the partition window belongs, and record it as the maximum frequency; use the difference between the maximum frequencies of the segment to which the previous partition window belongs and the segment to which the current partition window belongs as the numerator, and the maximum frequency of the segment to which the previous partition window belongs as the denominator; the ratio formed by the numerator and the denominator is taken as the reduction rate of the partition window.

4. The method for automatic embroidery pattern making based on neural network according to claim 2, characterized in that: The determining whether the texture edge in the divided window is a combined texture or a single texture includes: When the reduction rate of the divided window is greater than a preset texture threshold, the texture edge in the divided window is determined to be a combined texture; When the reduction rate of the divided window is less than or equal to the preset texture threshold, the texture edge in the divided window is determined to be a single texture.

5. The method for automatic embroidery pattern making based on neural network according to claim 1, characterized in that: The step of performing hierarchical clustering on the same single texture and the same combined texture to obtain hierarchical coordinate trees of the single texture and the combined texture comprises: The center point coordinates of each divided window are used as the window coordinates; For the partitioned windows corresponding to the same single texture, the inverse of the Euclidean distance between any two partitioned window coordinates is used as the similarity between the windows. Through the bottom-up clustering method, a hierarchical clustering tree of the same single texture is obtained, which is recorded as the hierarchical coordinate tree of the single texture. For the divided windows corresponding to the same combined texture, the inverse of the Euclidean distance between any two divided window coordinates is taken as the similarity between the windows. Through the bottom-up clustering method, a hierarchical clustering tree of the same combined texture is obtained, which is recorded as the hierarchical coordinate tree of the combined texture.

6. The method for automatic embroidery pattern making based on neural network according to claim 1, characterized in that: The step of matching the hierarchical coordinate trees of the single texture and the combined texture to obtain a single texture scale and a combined texture scale includes: The hierarchical coordinate tree of a single texture is used as the right node, and the hierarchical coordinate tree of a combined texture is used as the left node; the mean coordinate of all coordinates in each node is used as the representative coordinate of the node; Match the nodes on the left and right sides to obtain the matching edge value between the nodes on the two sides; when the matching edge value is less than the preset matching threshold, determine that the single texture corresponding to the right node constitutes the combined texture corresponding to the left node, and use the window scale of the right node as the single texture scale and the window scale of the left node as the combined texture scale; The method for obtaining the matching edge value is as follows: calculating the Euclidean distance of the representative coordinates of any two nodes in the same layer in the hierarchical coordinate tree corresponding to the left and right nodes, and using the variance of all Euclidean distances corresponding to the hierarchical coordinate tree of the left and right nodes as the matching edge value.

7. The method for automatic embroidery pattern making based on neural network according to claim 6, characterized in that: The step of determining the window weights of the divided windows based on the single texture scale and the combined texture scale comprises: The sum of the matching edge values ​​corresponding to all single textures belonging to the divided window corresponding to the single texture scale is used as the window weight of the divided window corresponding to the single texture scale; The matching edge value of each combined texture in the left node is used as the window weight of the divided window corresponding to the combined texture scale; wherein the window weight is a normalized value.

8. The method for automatic embroidery pattern making based on neural network according to claim 1, characterized in that: The method of dividing each historical embroidery image in the embroidery data set according to the divided window and determining the energy value of each divided window includes: Grayscale quantization is performed on each historical embroidery image in the embroidery dataset, and the grayscale level in each historical embroidery image is quantized into N levels.

9. The method for automatic embroidery pattern making based on neural network according to claim 1, characterized in that: The method of dividing each historical embroidery image in the embroidery data set according to the divided window and determining the energy value of each divided window includes: Obtaining gray level co-occurrence matrices corresponding to different directions of the divided windows, and determining the energy value of each gray level co-occurrence matrix; The maximum energy value is taken as the energy value of the corresponding partition window.

10. The method for automatic embroidery pattern making based on neural network according to claim 1, characterized in that: The energy value sequence formed by dividing the energy values ​​of the divided windows of the same size to obtain the divided segments includes: In order from small to large, an energy value sequence is constructed from the energy values ​​corresponding to the divided windows of the same size; The energy value sequence is segmented using the Otsu multi-threshold segmentation algorithm to obtain a plurality of segmentation segments.

Images