A method and system for automatic alignment of garment button patterns based on visual detection

Abstract

This invention discloses a method and system for automatic alignment of garment button patterns based on visual detection. First, an image acquisition module acquires multiple original button images of patterned buttons. These original images are then preprocessed to obtain corresponding preprocessed button images. Next, these preprocessed button images are input into a trained deep neural network model for quality detection and recognition. If the pattern quality is good, post-processing is performed on the preprocessed button images corresponding to the multiple patterned buttons to obtain the corresponding button rotation angle. The corresponding patterned buttons are then rotated according to the rotation angle, aligning the patterns on the multiple patterned buttons. This invention expands the sample images with different rotation angles and qualities, solving the problem of insufficient training samples and positioning accuracy for button patterns in garment production lines. LSTM and GoogLeNet networks are used to improve the accuracy of button pattern quality detection and recognition.

Description

Technical Field

[0001] This invention relates to an automatic button alignment method in the field of button pattern quality inspection and pattern alignment, and more particularly to an automatic button pattern alignment method and system for clothing based on visual inspection. Background Technology

[0002] Regarding the quality inspection and alignment of button patterns, current garment production lines rely on manual inspection and alignment, resulting in high costs, low efficiency, and low automation. While some technologies utilize machine vision and deep learning for button quality inspection, these primarily focus on detecting specific quality issues such as color differences, lacking comprehensive quality inspection and automated alignment techniques. Summary of the Invention

[0003] In order to address the problems and needs existing in the background art, the present invention aims to provide a method and system for automatic alignment of clothing button patterns based on visual detection.

[0004] The technical solution adopted in this invention is as follows:

[0005] I. A method for automatic alignment of clothing button patterns based on visual detection

[0006] 1) Collect original button images of multiple patterned buttons, preprocess the original button images respectively, and obtain corresponding preprocessed button images respectively;

[0007] 2) Input multiple pre-processed button images into a deep neural network model for quality detection and recognition of button patterns, and output the quality detection results corresponding to multiple pattern buttons. If the pattern quality is good in the quality detection results, proceed to the next step.

[0008] 3) Perform image post-processing on the pre-processed button images corresponding to multiple patterned buttons to obtain the corresponding button rotation angles. Rotate the corresponding patterned buttons according to the button rotation angles to align the patterns on multiple patterned buttons.

[0009] In step 1), after grayscale processing and region of interest extraction of the original button image, a preprocessed button image is obtained.

[0010] In step 2), the deep neural network model for detecting the quality of button patterns is obtained through the following steps:

[0011] The original training set, consisting of multiple preprocessed button images, is input into a least-squares generative adversarial neural network for sample augmentation to obtain augmented button images.

[0012] The final training set consists of the original training set and augmented button images. The original training set is then input into the transfer learning network GoogLeNet for pre-training to obtain the pre-trained transfer learning network.

[0013] The network feature parameters of the pre-trained transfer learning network are then transferred to the long short-term memory deep neural network and used as the initial network parameters. The long short-term memory deep neural network is then trained using the final training set to obtain a trained long short-term memory deep neural network. The trained long short-term memory deep neural network is then used as a deep neural network model to detect the quality of button patterns.

[0014] In step 3), the rotation angle of each pattern button with good pattern quality is calculated using the feature corner point extraction method and template matching algorithm.

[0015] In step 3), the Harris corner extraction algorithm is used to extract the pixel coordinates of each feature corner point in the preprocessed button image corresponding to each pattern button with good pattern quality. The angle difference between the pixel coordinates of each feature corner point corresponding to each pattern button and the pixel coordinates of each feature corner point in the template is calculated using the following formula:

[0016]

[0017] Where, α ij x represents the angle difference of the i-th feature corner point corresponding to each patterned button. i ,y i This represents the two pixel coordinates of the i-th feature corner point corresponding to each patterned button, x j ,y j This represents the two pixel coordinates of the j-th feature corner point in the template, x o ,y o This represents the two pixel coordinates of the origin in the template;

[0018] An angle difference set is formed by all the angle differences corresponding to each patterned button. The angle intervals of the angle difference sets corresponding to each patterned button are divided according to a preset resolution. The average angle of the angle interval with the highest frequency of angle occurrence is calculated and used as the original angle of each angle difference set. The original angles of each set of angle differences The angle obtained by adding the compensation angle β to the button rotation angle is used as the corresponding button rotation angle for each patterned button. Finally, the motor signal is calculated based on the rotation angle to control the alignment mechanism to rotate the button by the specified rotation angle.

[0019] II. An Automatic Alignment System for Clothing Button Patterns Based on Visual Detection

[0020] The automatic alignment system includes an automatic feeding device, an image acquisition module, a quality recognition module, an image post-processing module, and an alignment mechanism. The button-attaching machine on the garment production line is equipped with these modules. The automatic feeding device presses down the garment fabric and sends the patterned buttons to the button-attaching machine. The image acquisition module captures the original button images, pre-processes them, and then sends them to the quality recognition module. The quality recognition module checks the button quality based on the original button images. If the pattern quality is good, it sends the pre-processed button image to the image post-processing module. The image post-processing module calculates the rotation angle, outputs a motor signal, and sends it to the alignment mechanism. The alignment mechanism rotates each button according to the motor signal, aligning the patterns on each button.

[0021] The quality recognition module includes a deep neural network model for detecting the quality of button patterns. The deep neural network model for detecting the quality of button patterns is obtained through the following steps:

[0022] The original training set, consisting of multiple preprocessed button images, is input into a least-squares generative adversarial neural network for sample augmentation to obtain augmented button images.

[0023] The final training set consists of the original training set and augmented button images. The original training set is then input into the transfer learning network GoogLeNet for pre-training to obtain the pre-trained transfer learning network.

[0024] The network feature parameters of the pre-trained transfer learning network are then transferred to the long short-term memory deep neural network and used as the initial network parameters. The long short-term memory deep neural network is then trained using the final training set to obtain a trained long short-term memory deep neural network, which is then used as a deep neural network model for detecting the quality of button patterns.

[0025] In the image post-processing module, the Harris corner extraction algorithm is used to extract the pixel coordinates of each feature corner point in the preprocessed button image corresponding to each pattern button with good pattern quality. The angle difference between the pixel coordinates of each feature corner point corresponding to each pattern button and the pixel coordinates of each feature corner point in the template is calculated using the following formula:

[0026]

[0027] Where, α ij x represents the angle difference of the i-th feature corner point corresponding to each patterned button. i ,y i This represents the two pixel coordinates of the i-th feature corner point corresponding to each patterned button, x j ,y j This represents the two pixel coordinates of the j-th feature corner point in the template, x o,y o This represents the two pixel coordinates of the origin in the template;

[0028] An angle difference set is formed by all the angle differences corresponding to each patterned button. The angle intervals of the angle difference sets corresponding to each patterned button are divided according to a preset resolution. The average angle of the angle interval with the highest frequency of angle occurrence is calculated and used as the original angle of each angle difference set. The original angles of each set of angle differences The angle obtained by adding the compensation angle β to the button rotation angle is used as the corresponding button rotation angle for each patterned button. Finally, the motor signal is calculated based on the rotation angle to control the alignment mechanism to rotate the button by the specified rotation angle.

[0029] III. A storage medium storing a computer program

[0030] When the computer program is executed by the processor, it implements any of the methods described.

[0031] IV. A storage medium

[0032] The computer program referred to therein is an instruction that implements the method.

[0033] The beneficial effects of this invention are:

[0034] This invention uses LSGAN to expand the button pattern image, increasing the sample size and generalization ability of the neural network; it uses LSTM and GoogLeNet to detect and recognize the button pattern image quality, improving the efficiency and accuracy of detection and recognition; and it employs image post-processing methods to calculate the rotation angle of a high-quality button pattern image relative to the template image, outputting a rotation signal to improve the accuracy of the rotation angle. Attached Figure Description

[0035] Figure 1 This is a flowchart of the main steps of a method for automatic alignment of clothing button patterns based on visual detection in an embodiment of the present invention;

[0036] Figure 2 This is a schematic diagram illustrating the two types of results that the quality detection model needs to identify and classify in this embodiment of the invention;

[0037] Figure 3 This is a basic structural diagram of the LSGAN network in an embodiment of the present invention;

[0038] Figure 4 This is a basic structural diagram of the GoogLeNet network in an embodiment of the present invention;

[0039] Figure 5 This is a schematic diagram of image post-processing in an embodiment of the present invention. Detailed Implementation

[0040] The embodiments of the present invention are described in detail below, and the embodiments are illustrated in the accompanying drawings.

[0041] like Figure 1 As shown, the method of the present invention includes the following steps:

[0042] 1) Patterned buttons are buttons with a pattern printed on the surface of the button. Collect original button images of multiple patterned buttons, preprocess the original button images respectively, and obtain the corresponding preprocessed button images.

[0043] In step 1), the original button image is converted to grayscale and the region of interest is extracted to obtain a preprocessed button image. The region of interest extraction depends on the shape of the button, and is generally circular or square.

[0044] 2) Input multiple pre-processed button images into a deep neural network model for quality detection and recognition of button patterns. Output the quality detection results for each button pattern. If the pattern quality is good, proceed to the next step; otherwise, terminate the process by removing the button pattern and not sewing it on. Figure 2 As shown, button pattern quality issues are categorized into two types: good and bad.

[0045] In step 2), the deep neural network model for detecting the quality of button patterns is obtained through the following steps:

[0046] The original training set, consisting of multiple preprocessed button images, is input into a least-squares generative adversarial neural network (LSGAN) for sample augmentation to obtain augmented button images. The LSGAN network structure is as follows: Figure 3 As shown, Figure 3 (a) is a schematic diagram of the LSGAN generator structure. Figure 3 (b) is a schematic diagram of the LSGAN discriminator structure. The loss function L(D) of the discriminator D(x) in the LSGAN network is as follows:

[0047]

[0048] In the formula, px represents the true data distribution, pz represents the noise distribution, G(z) represents the generator with multilayer sensing capability, D(x) represents the discriminator with classification capability, a represents the generated sample, b represents the true sample, and E x～px (·) represents the expected value of the difference between the generator-generated sample and the real sample, E z～pz (·) represents the expected value corresponding to the discriminator's ability to identify the differences between the generator-generated samples and the real samples, and min represents the minimum value of the loss function.

[0049] The loss function L(G) of the generator G in the LSGAN network is as follows:

[0050]

[0051] In the formula, c indicates that the judge D(x) treats the sample generated by the generator G(x) as the real sample.

[0052] Image data augmentation based on LSGAN faces challenges in actual production processes due to limitations in on-site production conditions, making it difficult to obtain sufficient sample data for subsequent training. The interplay between the LSGAN generator and discriminator can generate a large number of sample images with varying rotation angles and quality. In this case, the original number of poor-quality button pattern images was 359; after augmentation using LSGAN, this number can reach 600. The final training set, composed of the original training set and the augmented button images, enhances the generalization ability of the transfer learning network. The original training set is input into the GoogLeNet transfer learning network for pre-training, resulting in the pre-trained network. The feature extraction function in the GoogLeNet transfer learning network is:

[0053]

[0054] In the formula, Represents the feature extraction function. Indicates the weight value. This represents the bias, and max(·) represents taking the maximum value.

[0055] The activation function uses the ReLU function, and its function expression is:

[0056] ReLU(x) = max(0,x)

[0057] Features extracted from button pattern images using the GoogLeNet network can accelerate the training process of the LSTM (Long Short-Term Memory) deep neural network. Transfer learning is then used to transfer the trained GoogLeNet network parameters to a new model, aiding in its training and solving, thereby reducing the training time of the LSTM network and decreasing its dependence on data samples. The structure of the GoogLeNet network is as follows... Figure 4 As shown, the pre-trained GoogLeNet enables the backbone network LSTM to acquire strong feature extraction capabilities from the outset, allowing the network model to recognize specific features in image data such as button pattern images.

[0058] Therefore, the network feature parameters of the pre-trained transfer learning network are transferred to the Long Short-Term Memory (LSTM) deep neural network and used as the initial network parameters. The LSTM deep neural network is then trained using the final training set to obtain a trained LSTM deep neural network, which serves as the deep neural network for detecting the quality of button patterns. The trained LSTM deep neural network can extract local features from button pattern images, and its prediction performance on button pattern image quality is superior to that of a single network.

[0059] 3) Perform image post-processing on the pre-processed button images corresponding to multiple patterned buttons to obtain the corresponding button rotation angles. Rotate the corresponding patterned buttons according to the button rotation angles to align the patterns on multiple patterned buttons.

[0060] In step 3), the rotation angle of each pattern button with good pattern quality is calculated using the feature corner point extraction method and template matching algorithm.

[0061] like Figure 5 As shown, Figure 5 (a) is a schematic diagram of template matching for calculating the rotation angle. Figure 5 (b) is a schematic diagram of extracting corner points from the sample image. In 3), the Harris corner point extraction algorithm is used to extract the pixel coordinates of each feature corner point in the preprocessed button image corresponding to each pattern button with good pattern quality. The angle difference between the pixel coordinates of each feature corner point corresponding to each pattern button and the pixel coordinates of each feature corner point in the template is calculated by the following formula:

[0062]

[0063] Where, α ij x represents the angle difference of the i-th feature corner point corresponding to each patterned button. i ,y i This represents the two pixel coordinates of the i-th feature corner point corresponding to each patterned button, x j ,y j This represents the two pixel coordinates of the j-th feature corner point in the template, x o ,y o This represents the two pixel coordinates of the origin in the template;

[0064] The set of corresponding angle differences Ω{α} is composed of all the angle differences corresponding to each patterned button. ij Let i = 1, 2, ..., n, j = 1, 2, ..., m, where n is the number of feature corner points calculated using the feature corner point extraction algorithm, and m is the number of feature corner points in the template. In specific implementation, m = n = 7. The angle intervals corresponding to the angle difference sets of each pattern button are divided according to the preset resolution. The average angle of the angle interval with the highest angle occurrence frequency is calculated and used as the original angle of each angle difference set. The original angles of each set of angle differences The angle obtained by adding the compensation angle β to the original angle is taken as the button rotation angle corresponding to each patterned button. Finally, the motor signal is calculated based on the rotation angle to control the alignment mechanism to rotate the button by the specified rotation angle. In practice, based on the rotation angle and the actual motor model information, a pulse signal is calculated and output to control the motor, thereby controlling the alignment mechanism to rotate the button and align it with the corresponding position.

[0065] In practice, the expression for the set of angle differences is as follows:

[0066] Ω{α ij}={90.1°,79.2°,65.5°,9.1°,85.5°,103.2°,23.3°,79.1°,89.5°,9.1°,85.6°,103.5°,24.2°,62°,135.8°,148°,90.7°,114°,12°,15°,66.3°,132°,12 °, 13°, 159.8°, 90.1°, 69°, 89° 28°, 89°, 23°, 158°, 143°, 100°, 56.5° 20.5°, 138°, 129°, 57.1°, 65°, 89.8°, 42°, 109°, 105°, 58.5°, 75°, 105°, 165°, 90.9°

[0067] In actual processing, there is a certain angle compensation. In this example, β = 1° is set, meaning that 1° is added to each angle value in the set. Dividing the statistical intervals with a resolution of 2°, and based on the frequency distribution statistics, we can see that the most angles are distributed between [89°, 91°]. Therefore, the rotation angle is the average of the angles within this interval.

[0068] The vision-based automatic button pattern alignment system includes an automatic feeding device, an image acquisition module, a quality recognition module, an image post-processing module, and an alignment mechanism. The button attaching machine on the garment production line is equipped with these modules. The automatic feeding device presses down the garment fabric and delivers the patterned buttons to the button attaching machine according to a pre-set program. The image acquisition module acquires the original button images, pre-processes them, and then sends them to the quality recognition module. The quality recognition module checks the button quality based on the original images. If the pattern quality is good, the pre-processed button image is sent to the image post-processing module. If the pattern quality is good, a button rejection signal is sent to the alignment mechanism. The image post-processing module calculates the rotation angle, outputs a motor signal, and sends it to the alignment mechanism. The alignment mechanism rotates each button according to the motor signal, ensuring the buttons are rotated to the desired correct position and the patterns on each button are aligned.

[0069] The quality identification module includes a pre-trained deep neural network model, which is obtained through the following steps:

[0070] The original training set, consisting of multiple preprocessed button images, is input into the least squares generative adversarial neural network for sample augmentation to obtain augmented button images. The original training set and the augmented button images form the final training set. The original training set is then input into the transfer learning network GoogLeNet for pre-training to obtain the pre-trained transfer learning network.

[0071] The network feature parameters of the pre-trained transfer learning network are then transferred to the long short-term memory deep neural network and used as the initial network parameters. The long short-term memory deep neural network is then trained using the final training set to obtain a trained long short-term memory deep neural network, which is then used as a deep neural network to detect the quality of button patterns.

[0072] In the image post-processing module, the Harris corner extraction algorithm is used to extract the pixel coordinates of each feature corner point in the preprocessed button image corresponding to each pattern button with good pattern quality. The angle difference between the pixel coordinates of each feature corner point corresponding to each pattern button and the pixel coordinates of each feature corner point in the template is calculated using the following formula:

[0073]

[0074] Where, α ij x represents the angle difference of the i-th feature corner point corresponding to each patterned button. i ,y i This represents the two pixel coordinates of the i-th feature corner point corresponding to each patterned button, x j ,y jThis represents the two pixel coordinates of the j-th feature corner point in the template, x o ,y o This represents the two pixel coordinates of the origin in the template;

[0075] An angle difference set is formed by all the angle differences corresponding to each patterned button. The angle intervals of the angle difference sets corresponding to each patterned button are divided according to a preset resolution. The average angle of the angle interval with the highest frequency of angle occurrence is calculated and used as the original angle α of each angle difference set. ij * The original angle α of each set of angle differences ij * The angle obtained by adding the compensation angle β to the angle is taken as the button rotation angle corresponding to each patterned button, i.e., α. ij * +β, and finally, based on the rotation angle, calculate the motor signal to control the alignment mechanism to rotate the button by the rotation angle α. ij * +β. In practice, based on the rotation angle and the actual motor model information, a pulse signal for controlling the motor is calculated and output, thereby controlling the alignment mechanism to rotate the button and align it with the corresponding position.

Claims

1. A method for automatic alignment of garment button patterns based on visual detection, characterized in that, Includes the following steps: 1) Collect original button images of multiple patterned buttons, preprocess the original button images respectively, and obtain corresponding preprocessed button images respectively; 2) Input multiple pre-processed button images into a deep neural network model for quality detection and recognition of button patterns, and output the quality detection results for multiple patterned buttons. If the pattern quality is good in the quality detection results, proceed to the next step. 3) Perform image post-processing on the pre-processed button images corresponding to multiple patterned buttons to obtain the corresponding button rotation angles. Rotate the corresponding patterned buttons according to the button rotation angles to align the patterns on multiple patterned buttons. In step 3), the rotation angle of each pattern button with good pattern quality is calculated using the feature corner point extraction method and template matching algorithm. Specifically, the Harris corner extraction algorithm is used to extract the pixel coordinates of each feature corner point in the preprocessed button image corresponding to each pattern button with good pattern quality. The angle difference between the pixel coordinates of each feature corner point corresponding to each pattern button and the pixel coordinates of each feature corner point in the template is calculated using the following formula: in, This represents the angle difference of the i-th feature corner point corresponding to each patterned button. This represents the two pixel coordinates of the i-th feature corner point corresponding to each patterned button. This represents the two pixel coordinates of the j-th feature corner point in the template. This represents the two pixel coordinates of the origin in the template; An angle difference set is formed by all the angle differences corresponding to each patterned button. The angle intervals of the angle difference sets corresponding to each patterned button are divided according to a preset resolution. The average angle of the angle interval with the highest frequency of angle occurrence is calculated and used as the original angle of each angle difference set. The original angles of each set of angle differences From the perspective of compensation The summed angles serve as the button rotation angles for each patterned button. Finally, the motor signal is calculated based on the rotation angles to control the alignment mechanism to rotate the buttons by the specified angles. + .

2. The method for automatic alignment of clothing button patterns based on visual detection according to claim 1, characterized in that, In step 1), after grayscale processing and region of interest extraction of the original button image, a preprocessed button image is obtained.

3. The method for automatic alignment of clothing button patterns based on visual detection according to claim 1, characterized in that, In step 2), the deep neural network model for detecting the quality of button patterns is obtained through the following steps: The original training set, consisting of multiple preprocessed button images, is input into a least-squares generative adversarial neural network for sample augmentation to obtain augmented button images. The final training set consists of the original training set and augmented button images. The original training set is then input into the transfer learning network GoogLeNet for pre-training to obtain the pre-trained transfer learning network. The network feature parameters of the pre-trained transfer learning network are then transferred to the long short-term memory deep neural network and used as the initial network parameters. The long short-term memory deep neural network is then trained using the final training set to obtain a trained long short-term memory deep neural network. The trained long short-term memory deep neural network is then used as a deep neural network model to detect the quality of button patterns.

4. A vision-based automatic alignment system for clothing button patterns, characterized in that, The system includes an automatic feeding device, an image acquisition module, a quality recognition module, an image post-processing module, and an alignment mechanism. The button attaching machine on the garment production line is equipped with these modules. The automatic feeding device presses down the garment fabric and sends the patterned buttons to the button attaching machine. The image acquisition module acquires the original button image, performs pre-processing, and then sends it to the quality recognition module. The quality recognition module detects the button quality based on the original button image. If the pattern quality is good, the pre-processed button image is sent to the image post-processing module. The image post-processing module calculates the rotation angle, outputs a motor signal, and sends it to the alignment mechanism. The alignment mechanism rotates each button according to the motor signal to align the patterns on each button. In the image post-processing module, the Harris corner extraction algorithm is used to extract the pixel coordinates of each feature corner point in the preprocessed button image corresponding to each pattern button with good pattern quality. The angle difference between the pixel coordinates of each feature corner point corresponding to each pattern button and the pixel coordinates of each feature corner point in the template is calculated using the following formula: in, This represents the angle difference of the i-th feature corner point corresponding to each patterned button. This represents the two pixel coordinates of the i-th feature corner point corresponding to each patterned button. This represents the two pixel coordinates of the j-th feature corner point in the template. This represents the two pixel coordinates of the origin in the template; An angle difference set is formed by all the angle differences corresponding to each patterned button. The angle intervals of the angle difference sets corresponding to each patterned button are divided according to a preset resolution. The average angle of the angle interval with the highest frequency of angle occurrence is calculated and used as the original angle of each angle difference set. The original angles of each set of angle differences From the perspective of compensation The summed angles serve as the button rotation angles for each patterned button. Finally, the motor signal is calculated based on the rotation angles to control the alignment mechanism to rotate the buttons by the specified angles. + .

5. The automatic alignment system for clothing button patterns based on visual detection according to claim 4, characterized in that, The quality recognition module includes a deep neural network model for detecting the quality of button patterns. The deep neural network model for detecting the quality of button patterns is obtained through the following steps: The original training set, consisting of multiple preprocessed button images, is input into a least-squares generative adversarial neural network for sample augmentation to obtain augmented button images. The final training set consists of the original training set and augmented button images. The original training set is then input into the transfer learning network GoogLeNet for pre-training to obtain the pre-trained transfer learning network. The network feature parameters of the pre-trained transfer learning network are then transferred to the long short-term memory deep neural network and used as the initial network parameters. The long short-term memory deep neural network is then trained using the final training set to obtain a trained long short-term memory deep neural network, which is then used as a deep neural network model for detecting the quality of button patterns.

6. A storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the method described in any one of claims 1 to 3.

Images