A method and system for generating traditional sachet patterns based on deep learning

Abstract

The present invention discloses a method and system for generating traditional sachet patterns based on deep learning, the method comprising: obtaining a pattern sample set; preprocessing the pattern sample set to obtain an image set after contour detection; classifying the pattern sample set according to the image set after contour detection; then amplifying the pattern sample images of each category using affine transformation and making the number of images of each category consistent after amplification; unifying the image size of each category after amplification to obtain a pattern training sample set; inputting the pattern training sample set into a deep learning network for training to obtain a pattern training model. The present invention makes up for the lack of the number of existing pattern sample sets to a certain extent; and greatly alleviates the problem of insufficient novelty and low output of current pattern designs.

Description

Technical Field

[0001] The invention relates to a method and system for generating traditional sachet patterns based on deep learning, and belongs to the field of artificial intelligence. Background Art

[0002] With the continuous improvement of national cultural literacy and socio-economic level, people pay more and more attention to the inheritance of traditional Chinese culture, and "Chinese style" has started a new round of fashion "revolution". Scenes containing Chinese style elements are also more and more widely used. Sachet culture has a long history and tradition, and the patterns on the sachets carry people's good wishes for auspiciousness and good harvests every year. The enrichment of daily life culture has led to more and more people beginning to like the production of traditional Chinese sachets, so the generation of sachet patterns has great commercial value and entertainment value. Traditional artificial design patterns are not only time-consuming, not novel enough, but also costly, and cannot meet the needs of modernization, so the idea of ​​innovatively generating sachet patterns is born. Because it is necessary to study the method of generating traditional sachet patterns. Summary of the invention

[0003] The present invention provides a traditional sachet pattern generation method based on deep learning, which is used to generate sachet patterns; and is further applicable to the situation where existing sachet pattern samples are insufficient.

[0004] The technical solution of the present invention is: a method for generating traditional sachet patterns based on deep learning, comprising:

[0005] S1. Obtain a pattern sample set;

[0006] S2, preprocessing the pattern sample set to obtain an image set after contour detection;

[0007] S3, classifying the pattern sample set according to the image set after contour detection; then amplifying the pattern sample images of each category by using affine transformation, and the number of images of each category after amplification is consistent; unifying the size of the images of each category after amplification to obtain the pattern training sample set;

[0008] S4. Input the pattern training sample set into the deep learning network for training to obtain a pattern training model.

[0009] The preprocessing includes: performing grayscale conversion processing, smoothing processing, calculating pixel gradient, setting high and low threshold processing on each pattern sample image in the pattern sample set in sequence to obtain an image set after contour detection.

[0010] The grayscale conversion process is: grayscale conversion is achieved by weighting the three channels R, G, and B of the pattern sample image to obtain a grayscale pattern sample image.

[0011] The smoothing process is as follows: for the grayscale pattern sample image, a kernel of size a*a is moved by setting a step size, pixel weighting is performed on each kernel and the weighted pixel size is used as the pixel size of the center position of the current kernel, and the pixel at the weighted center position is used for the pixel weighting process of the next kernel.

[0012] The weight formula used in the weighted processing is: Where x and y represent the coordinates of the pixel points in the image in the x direction and y direction respectively.

[0013] The setting of high and low threshold processing is specifically as follows: high and low threshold processing is performed on each pixel value in the image after pixel gradient calculation, if the pixel value is less than th1 or greater than th2, it is assigned a value of 0; otherwise, no processing is performed; wherein th1 represents the low threshold, th2 represents the high threshold, and th1=0.4th2.

[0014] The affine transformation is processed in at least one of the following ways: horizontal mirroring, rotation, and color saturation change.

[0015] The color saturation change adopts a linear transformation function; wherein the linear scale factor in the linear transformation function is 0.3, 1.2, or 1.6.

[0016] The deep learning network includes a generative neural network and a discriminative neural network; the generative neural network includes four convolutional layers, the convolution kernel size of each layer is 4*4, the step size is 2, and the edge pixels of the image are filled with "0" when the convolution kernel is sampled. The activation function of each convolution layer adopts the Relu function, the number of input channels is 100, and the number of output channels is 3; the discriminative neural network includes four convolutional layers, the convolution kernel size of each layer is 4*4, the step size is 2, and the edge pixels of the image are filled with "0" when the convolution kernel is sampled. The activation function of each convolution layer adopts the leakyRelu function, the number of input channels is 3, and the number of output channels is 1.

[0017] According to another aspect of an embodiment of the present invention, a traditional sachet pattern generation system based on deep learning is provided, comprising:

[0018] An acquisition module, used for acquiring a pattern sample set;

[0019] The first acquisition module is used to pre-process the pattern sample set to obtain an image set after contour detection;

[0020] The second acquisition module is used to classify the pattern sample set according to the image set after contour detection; then use affine transformation to amplify the pattern sample images of each category and make the number of images of each category consistent after amplification; unify the size of the images of each category after amplification to obtain the pattern training sample set;

[0021] The third acquisition module is used to input the pattern training sample set into the deep learning network for training to obtain the pattern training model.

[0022] The beneficial effects of the present invention are:

[0023] The present invention firstly forms pattern samples based on photographed real pattern; completes the contour detection of pattern and realizes the classification of pattern sample set by adopting grayscale conversion processing, smoothing processing, pixel gradient calculation, and setting high and low threshold processing; realizes image amplification by means of horizontal mirroring, rotation, color saturation change, etc., which makes up for the shortage of existing pattern sample sets in quantity to a certain extent.

[0024] In view of the fact that traditional patterns have many delicate features, the clarity of the sachet pattern is mainly affected by the acquired detail features. The present invention uses a sample set that retains the original features of the image as much as possible and inputs it into the deep learning network, and the convolution layer in the generated neural network can achieve image feature dimensionality reduction, so that the model can quickly and fully learn the detail features of the input image and imitate similar pattern features, and the convolution layer of the generated neural network can directly connect random noise with the feature input of the convolution layer. This method not only ensures the learning effect of the model but also ensures the stability of the model training. Obviously, the method of the present invention can be better applied to the generation of sachet patterns, thereby greatly alleviating the current problems of insufficient novelty of pattern design and low output. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 is a flow chart of the method of the present invention;

[0026] Figure 2 This is a grayscale display of the actual photographed Phoenix pattern;

[0027] Figure 3 The image after contour detection is obtained by preprocessing the actual phoenix pattern color picture;

[0028] Figure 4 For affine transformation effect Figure 1 ;

[0029] Figure 5 For affine transformation effect Figure 2 ;

[0030] Figure 6It is the model probability distribution change diagram;

[0031] Figure 7 is the network layer parameter diagram;

[0032] Figure 8 Generate an image with 10,000 iterations;

[0033] Fig. 9 Generate an image with 20,000 iterations;

[0034] Fig.10 Generate the image after 35,000 iterations. DETAILED DESCRIPTION

[0035] The invention is further described below in conjunction with the accompanying drawings and embodiments, but the content of the invention is not limited to the scope of the embodiments.

[0036] Example 1: Figure 1-7 As shown, a method for generating traditional sachet patterns based on deep learning includes:

[0037] S1. Obtain a pattern sample set;

[0038] Specifically, the initial pattern sample set is formed by photographing the physical samples owned by the intangible cultural heritage inheritors and the physical objects exhibited in the sachet museum; the pattern sample set is constructed by acquiring 2000 pattern sample images, such as Figure 2 As shown in the figure, it is a grayscale display of a sample image of a phoenix pattern that was actually photographed. The color image is actually used in the grayscale conversion process. Figure 2 , 4 , 5, 8, 9, and 10 are actually color pictures, and this application uses grayscale display.

[0039] S2, preprocessing the pattern sample set to obtain an image set after contour detection;

[0040] S3, classifying the pattern sample set according to the image set after contour detection; then amplifying the pattern sample images of each category by using affine transformation, and the number of images of each category after amplification is consistent; unifying the size of the images of each category after amplification to obtain the pattern training sample set;

[0041] S4. Input the pattern training sample set into the deep learning network for training to obtain a pattern training model.

[0042] Furthermore, the preprocessing may include: sequentially performing grayscale conversion, smoothing, calculating pixel gradients, and setting high and low thresholds on each pattern sample image in the pattern sample set to obtain an image set after contour detection. Figure 3As shown, Figure 2 Image after contour detection after preprocessing.

[0043] Further, the grayscale conversion process can be set as follows: grayscale conversion is achieved by weighting the three channels R, G, and B of the pattern sample image to obtain a grayscale pattern sample image; the formula is 0.299*R+0.587*G+0.114*B, and this coefficient setting takes into account the physiological characteristics of the human eye and is consistent with the human eye's perception of grayscale images;

[0044] Furthermore, the smoothing process can be set as follows: for the grayscale pattern sample image, a kernel of size a*a is moved by setting a step size, pixel weighting is performed on each kernel and the weighted pixel size is used as the pixel size of the center position of the current kernel, and the weighted center position pixel is used for the pixel weighting process of the next kernel.

[0045] Furthermore, the weight formula used in the weighted processing can be set as Among them, x and y represent the coordinates of the pixel points in the x direction and y direction of the image respectively.

[0046] Specifically, a kernel of size 3*3 can be set to move with a step size of 1. The amount of calculation can be reduced by performing smoothing after grayscale conversion processing, and by setting a kernel of size 3*3 to move with a step size of 1, pixel weighting processing is performed on each kernel, thereby filtering out high-frequency noise and improving the accuracy of contour detection. The updated value of the current kernel center position is used to participate in the calculation of the next kernel center position, which makes the calculation more convenient to a certain extent and can further improve the effect of filtering out high-frequency noise. The following are the weights used for the kernel of size 3*3.

[0047]

[0048] Furthermore, the calculation pixel gradient can be set, and its theoretical derivation is to determine the zero-crossing point of the second-order derivative of the maximum value of the first-order derivative of each pixel, thereby obtaining the precise position of the contour; and the image pixels are discrete values, there is no continuity and therefore no derivative, so the first-order finite difference is used for approximation, and the approximate gradient operator of the partial derivatives of the image in the x and y directions is:

[0049]

[0050] Among them, s x and y are the gradient operators of pixel s in the x and y directions, K is a neighborhood point labeling matrix, and [i, j] is the pixel to be calculated. This operator can identify as many actual contours in the image as possible, which can improve the efficiency and accuracy of subsequent pattern image classification.

[0051] Furthermore, the high and low threshold processing can be set, specifically: for each pixel value in the image after pixel gradient calculation, high and low threshold processing is performed, if the pixel value is less than th1 or greater than th2, it is assigned a value of 0; otherwise, no processing is performed; wherein th1 represents the low threshold, th2 represents the high threshold, and th1=0.4th2. Based on the fact that the patterns of sachets are mostly represented by figures, animals or geometric patterns, and the contours of the patterns always have sharp or obvious line features, and in this application, the image subjected to pixel gradient processing is subjected to high and low threshold processing again, which can exclude non-contour points. Obviously, the method of this application can make the classification results more accurate.

[0052] Regarding S3, specifically, the pattern sample set is classified according to the image set after contour detection, including five categories: dragon pattern, bird pattern, fish pattern, phoenix pattern, and ornament pattern. The five categories basically cover the patterns of existing traditional sachets; then, the classified categories are amplified by affine transformation, and the number of images in each category after amplification is consistent (for example, after classifying 2000 pattern sample images, each category is amplified to 1200 images for each category, for a total of 6000 images), so that each category can be fully learned in subsequent model training; the images of each category after amplification are amplified. The image size is unified to 128*128; based on the processing in this step, it is convenient for the subsequent model to fully learn the image to realize feature extraction, and the training cycle of the model training is shorter than that of other specifications; after the unified specifications, the images are named according to "category_serial number", and the 6000 images are randomly divided into 5 Train-batches and 1 Test-batch. Train-batch represents the pattern training sample set for model training, and Test-batch represents the pattern verification sample set for model verification.

[0053] Furthermore, the affine transformation may be set to be processed in at least one of the following ways: horizontal mirroring, rotation, and color saturation change. Figure 4 As shown, from left to right are the original image, horizontal mirror image, and the result of 60-degree rotation; Figure 5 Shown from left to right are the results of alpha being 0.3, 1.2, and 1.6.

[0054] Furthermore, the color saturation change may be set to adopt a linear transformation function; wherein the linear scale factor alpha in the linear transformation function is set to 0.3, 1.2, or 1.6.

[0055] Specifically: the mirror function Flip in the cv module can be used to achieve horizontal mirroring; the Rotation function can be used to rotate the image around the central axis by the angel angle, whose value is [0°, 60°]; the linear transformation function Uint8 can be used to change the color saturation of the image, where the linear scale factor alpha in the linear function can be taken as 0.3, 1.2, or 1.6.

[0056] The affine transformation used in this application is translation invariant and can retain the original features of the image as much as possible; at the same time, the image after contour detection of each category is amplified from multiple angles during the transformation process, thereby enriching the images in the data set.

[0057] Regarding S4, specifically, the pattern training sample set and the random noise vector are input into the generating neural network to generate an output image; the output image of the generating neural network is sent to the discriminant neural network to obtain the output probability; the generating neural network adjusts the network weight parameters according to the output probability, thereby repeatedly iterating to obtain the pattern training model. The pattern training model is to update the model network weight parameters through iterative training to achieve feature extraction and learning of the input sample image, thereby generating the desired pattern.

[0058] Furthermore, the deep learning network can be set to include a generative neural network and a discriminative neural network; the generative neural network includes four convolutional layers, the convolution kernel size of each layer is 4*4, the step size is 2, and the image edge pixels are filled with "0" when the convolution kernel is sampled, so as to ensure that the image edge information is learned and improve the effect of generating pattern patterns. The activation function of each convolution layer adopts the Relu function, the number of input channels is 100, and the number of output channels is 3; the discriminative neural network includes four convolutional layers, the convolution kernel size of each layer is 4*4, the step size is 2, and the image edge pixels are filled with "0" when the convolution kernel is sampled. The activation function of each convolution layer adopts the leakyRelu function, the number of input channels is 3, and the number of output channels is 1.

[0059] Specifically, if Figure 7As shown, the generative neural network includes four convolutional layers, in_channels represents the number of input channels, the number of input channels in this application is 100, the dimension of the random vector z represented is 100, out_channels is the output channel, which changes with the number of convolutional layers. The number of output channels from the first convolutional layer to the fourth convolutional layer is 1024, 512, 256, 3 respectively. After learning the convolutional layer, it becomes an image of size 64*64*3 (64*64 represents the image specification, and 3 represents the number of channels). The activation function uses the Relu function, which can speed up the convergence of the model. The size of each convolution kernel is 4*4, stride is the convolution kernel moving step, and this application uses 2, and padding is the pixel value at the missing position of the filling image; through this specific generative neural network, while taking into account the amount of calculation, the convolutional layer can fully learn the detailed features of the input image during feature learning, and it is more likely to make the generated image retain the integrity of the pattern content. The discriminant neural network activation function adopts leakyRelu, which is a variant of the Relu function. It retains some negative axis learning, which can make the learning more comprehensive; the number of output channels from the first convolution layer to the fourth convolution layer is 256, 512, 1024, and 1 respectively.

[0060] The discriminant neural network regards the generated image and the real image as two specific distributions, and makes similarity judgments through relative entropy, i.e., Gibbs inequality, whose formula is:

[0061]

[0062] Among them, p(x) and q(x) represent the probability density functions of the generated image and the real image respectively;

[0063] The above formula is non-negative and asymmetric, which leads to different losses due to different observation methods, thus falling into the local optimum, so the formula for optimization is:

[0064]

[0065] The objective function of the model is:

[0066]

[0067] Where A represents the objective function, They represent the loss functions of the discriminant neural network and the generative neural network respectively, S represents the generative neural network, and J represents the discriminant neural network. The convergence of the model is judged by the transformation of the objective function A; P data (x) is a probability distribution function of the pattern training sample set, P zis the output probability distribution of the generative neural network, E represents the expected mean; the input of the generative neural network is the image x in the pattern training sample set and the random noise vector z of the generated pattern, and the output is the generated image S(z); the probability of the discriminant neural network output being (0, 1), its output is 0, which means that the generated image quality is poor, and when the probability is 0.5, the model training effect is the best; the probability distribution changes and probability judgments of the two networks are as follows Figure 6 , where z represents noise, which is initially uniformly distributed, and x represents that z becomes non-uniformly distributed after the generative neural network feature learning, and has the distribution characteristics of the input image. The thick black dotted line represents the probability distribution function of the real sample, the thin solid line represents the probability distribution function of the generated sample, and the thin black dotted line represents the judgment probability of the discriminant neural network. When it is 0.5, it means that the model converges and the generated image effect is good.

[0068] During the training process, in order to weaken the instability of the training, a penalty term is added to the objective function, and the penalty term is:

[0069] Among them, λ is the penalty intensity, It is a Laplace operator. In this application, the penalty term is adjusted according to the fluctuation degree of the network weight parameter.

[0070] The optimized objective function is:

[0071]

[0072] When using the model, the trained model parameters are saved and can be directly used by loading the weight parameters when applying the model, so that the model engineering application can be realized.

[0073] The following are the instructions for some steps:

[0074] 1)、 / home / DATAMc / Dataset / Pattern / 1_603.png

[0075] The above is the path for storing the image storage of the captured pattern sample set, such as 1_603.png is the storage path operation of a captured pattern sample image, that is, Figure 2 As shown;

[0076] 2) init implements the initialization of the corresponding category label (image_labels), and its function is:

[0077] images.append((words[0],int(words[1])))

[0078] This function is used to label the image set after contour detection according to "category_serial number";

[0079] 3) getitem reads data according to the tag document index and further uses it for specification processing. Its function is:

[0080] image = self.transform(image)

[0081] 4) len returns the length of the pattern training sample set / pattern verification sample set according to the list of index values;

[0082] 5) Load the image set through Dataloader and return the image and its label data to the main function. The function is:

[0083] train_dataloader=DataLoader(dataset=train_data,batch_size=args.batch_size,shuffle=True)

[0084] Optionally, the present invention provides an optional specific implementation method in combination with experimental data as follows:

[0085] In the process of training the model in this embodiment, 5000 augmented physical image sets are used, and the initial learning rate for weight update during model training is 2e -4 , the Adam algorithm with momentum of 0.5, the generated effect diagram is shown in the attached figure. Figure 7 The model was iterated 10,000 times, and the outline of the pattern can be seen. Figure 8 As shown in the figure, the pattern becomes clearer and clearer, but there are still shadows in the pattern background. When the iteration reaches 35,000 times, Fig. 9 As shown, there is no shadow on the background and the shape of the pattern can be distinguished.

[0086] Embodiment 2: A traditional sachet pattern generation system based on deep learning, comprising:

[0087] An acquisition module, used for acquiring a pattern sample set;

[0088] The first acquisition module is used to pre-process the pattern sample set to obtain an image set after contour detection;

[0089] The second acquisition module is used to classify the pattern sample set according to the image set after contour detection; then use affine transformation to amplify the pattern sample images of each category and make the number of images of each category consistent after amplification; unify the size of the images of each category after amplification to obtain the pattern training sample set;

[0090] The third acquisition module is used to input the pattern training sample set into the deep learning network for training to obtain the pattern training model.

[0091] The terms "first", "second", etc. in the present invention are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein.

[0092] According to another aspect of an embodiment of the present invention, a processor is provided, wherein the processor is used to run a program, wherein when the program is run, any one of the above-mentioned methods for generating traditional sachet patterns based on deep learning is executed.

[0093] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned methods for generating traditional sachet patterns based on deep learning.

[0094] The specific implementation modes of the present invention are described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above implementation modes, and various changes can be made within the knowledge scope of ordinary technicians in this field without departing from the purpose of the present invention.

Claims

1. A method for generating traditional sachet patterns based on deep learning, characterized in that: include: S1. Obtain a pattern sample set; S2, preprocessing the pattern sample set to obtain an image set after contour detection; S3, classifying the pattern sample set according to the image set after contour detection; then amplifying the pattern sample images of each category using affine transformation, and the number of images in each category after amplification is consistent; The size of each category of the enlarged images is unified to obtain a pattern training sample set; S4, inputting the pattern training sample set into the deep learning network for training to obtain a pattern training model; The preprocessing includes: performing grayscale conversion processing, smoothing processing, calculating pixel gradient, setting high and low threshold processing on each pattern sample image in the pattern sample set in turn to obtain an image set after contour detection; For each pixel value in the image after pixel gradient calculation, high and low threshold processing is performed. If the pixel value is less than th1 or greater than th2, it is assigned a value of 0; otherwise, no processing is performed; where th1 represents the low threshold, th2 represents the high threshold, and th1 = 0.4th2; The affine transformation is processed in the following ways: horizontal mirroring, rotation, and color saturation change; The color saturation change adopts a linear transformation function; wherein the linear scale factor in the linear transformation function is 0.3, 1.2, 1.6; The deep learning network includes a generative neural network and a discriminative neural network; the generative neural network includes four convolutional layers, the convolution kernel size of each layer is 4*4, the step size is 2, and the edge pixels of the image are filled with "0" when the convolution kernel is sampled. The activation function of each convolution layer adopts the Relu function, the number of input channels is 100, and the number of output channels is 3; the discriminative neural network includes four convolutional layers, the convolution kernel size of each layer is 4*4, the step size is 2, and the edge pixels of the image are also filled with "0" when the convolution kernel is sampled. The activation function of each convolution layer adopts the leakyRelu function, the number of input channels is 3, and the number of output channels is 1.

2. The method for generating traditional sachet patterns based on deep learning according to claim 1, characterized in that: The grayscale conversion process is: grayscale conversion is achieved by weighting the three channels R, G, and B of the pattern sample image to obtain a grayscale pattern sample image.

3. The method for generating traditional sachet patterns based on deep learning according to claim 1, characterized in that: The smoothing process is as follows: for the grayscale pattern sample image, a kernel of size a*a is moved by setting a step size, pixel weighting is performed on each kernel and the weighted pixel size is used as the pixel size of the center position of the current kernel, and the pixel at the weighted center position is used for the pixel weighting process of the next kernel.

4. The method for generating traditional sachet patterns based on deep learning according to claim 3 is characterized in that: The weight formula used in the weighted processing is: Among them, x and y represent the coordinates of the pixel points in the x direction and y direction of the image respectively.

5. A traditional sachet pattern generation system based on deep learning, which implements the traditional sachet pattern generation method based on deep learning according to claim 1, characterized in that: include: An acquisition module, used for acquiring a pattern sample set; The first acquisition module is used to pre-process the pattern sample set to obtain an image set after contour detection; The second acquisition module is used to classify the pattern sample set according to the image set after contour detection; then the pattern sample images of each category are amplified by affine transformation, and the number of images in each category is consistent after amplification; The size of each category of the enlarged images is unified to obtain a pattern training sample set; The third acquisition module is used to input the pattern training sample set into the deep learning network for training to obtain the pattern training model.

Images