Micro-vibration motor armature defect detection method based on semantic segmentation

Abstract

The application discloses a micro-vibration motor armature defect detection method based on semantic segmentation, which comprises the following steps: firstly, performing semantic segmentation on a micro-vibration motor image to be detected based on a constructed semantic segmentation network to obtain a feature map, wherein the semantic segmentation network comprises a converter ST, a feature fusion network FPN and a full convolution network FCN; and then performing visual display on the feature map obtained through semantic segmentation, and further identifying the feature map to determine whether there is an armature defect.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of microelectronics technology, and relates to micro-vibration motor defect detection, in particular to a micro-vibration motor armature defect detection method based on semantic segmentation. BACKGROUND

[0002] With the development of modernization, micro-machines have been widely used in industry. The quality of the core armature component directly affects the performance of the micro-motor. When a motor with a defective armature is used in other equipment, it will cause huge economic losses. In order to detect such damage, over the past few decades, various machine vision-based detection methods have been proposed. Among these advanced methods, deep learning techniques have become increasingly popular in recent years. With the progress of image classification methods based on deep neural networks, a large number of studies have emerged using image classification algorithms to detect micro-motor armature surface defects.

[0003] Feng et al. disclosed a double-branch surface defect detection system consisting of ResNet and FPN (Feng T, Liu J, Fang X, Wang J, Zhou L. A Double-Branch Surface Detection System for Armatures in Vibration Motors with Miniature Volume Based on ResNet-101 and FPN. Sensors. 2020; 20(8): 2360). Feng et al. disclosed an armature welding surface detection method based on CNN and image comparison, which converts the image classification problem into an image comparison problem to reduce the classification difficulty caused by the different distribution of negative samples (T. Feng, S. Huang, J. Liu, J. Wang and X. Fang, Welding Surface Inspection of Armatures via CNN and Image Comparison, in IEEE Sensors Journal, Vol. 21, No. 19, pp. 21696-21704, 1 Oct. 1, 2021, doi: 10.1109 / JSEN.2021.3079334). Li et al. disclosed a micro-motor armature surface defect detection method based on channel attention, which tried to use channel attention mechanism to enhance the feature extraction ability of the classification algorithm to improve the detection accuracy.

[0004] The above-mentioned micro-motor armature defect recognition methods are mainly based on image classification algorithms, which can only perform binary classification on the images of the armature surface, i.e. whether there is a fault, and cannot realize defect position positioning and defect category evaluation. SUMMARY

[0005] The purpose of the present application is to provide a micro-vibration motor armature defect detection method based on semantic segmentation, which realizes the positioning and defect type recognition of the micro-vibration motor armature defect.

[0006] To achieve the above purpose, the present application adopts the following technology.

[0007] The micro-vibration motor armature defect detection method based on semantic segmentation provided by the present application comprises the following steps:

[0008] S1 performs semantic segmentation on the micro-vibration motor image to be tested based on the constructed semantic segmentation network to obtain a feature map; the semantic segmentation network comprises a transformer ST, a feature fusion network FPN and a full convolutional network FCN; the transformer ST is used for feature extraction of different scales on the micro-vibration motor image to be tested; the feature fusion network FPN is used for fusing the extracted features of different scales; the full convolutional network FCN is used for extracting semantic information from the fused feature map, completing semantic segmentation, and outputting a feature image corresponding to each defect;

[0009] S2 visually displays the feature image obtained by semantic segmentation, and then identifies the feature map according to the set standard to determine whether there is an armature defect; the set standard comprises: (1) when the output feature image of the channel corresponding to the tin bead defect contains a tin bead, it indicates that there is a tin bead defect; (2) when the output feature image of the channel corresponding to the tin scraping defect contains a tin segment whose length is greater than a set threshold, it indicates that there is a tin scraping defect; (3) when the output feature image of the channel corresponding to the virtual welding defect contains a copper wire in a specified area, it indicates that there is a virtual welding defect.

[0010] In the above step S1, the transformer ST (Swin Transformer) is based on a multi-head attention mechanism, which can select a specific part in a visual area and then focus on it. Therefore, the present application attempts to use the transformer ST as the backbone of the semantic segmentation network. The hierarchical structure of the transformer ST makes its required computational complexity lower; but the hierarchical design inevitably causes a large amount of target information to be lost. Therefore, the feature fusion network FPN can be used to fuse feature maps of different scales to obtain as much global information as possible. Finally, the full convolutional network FCN is used to extract semantic information from the feature map; the output channel number of the full convolutional network FCN is 3, i.e. the total number of defect types.

[0011] The above transformer ST network structure adopts the conventional network results disclosed in the art, see Z. Liu et al., "Swin Transformer: Hierarchical Vision Transformer using Shifted Windows," 2021 IEEE / CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada, 2021, pp. 9992-10002, doi: 10.1109 / ICCV48922.2021.00986. The hierarchical structure of the transformer ST is divided into four stages with the number of channels increasing in turn. In order to describe the change of the feature map in the hierarchical structure, the size of the feature map is represented in the form of (height, width, channels). H, W and C represent the height, width and channel of the input image respectively. From the first stage to the fourth stage, the size of the feature map output by each stage is (H / 4, W / 4, 32C), (H / 8, W / 8, 64C), (H / 16, W / 16, 128C) and (H / 32, W / 32, 256C) respectively. The output feature map of each stage is sent to the FPN for fusion at different scales.

[0012] The above feature fusion network FPN branch contains a bottom-up path, a top-down path and a lateral connection. The fourth stage features extracted by the transformer ST are first upsampled to restore the feature map to the same spatial size as the third stage; then the two layers are connected to merge their feature maps, obtaining the fusion features of the fourth stage and the third stage feature map. The same principle is applied to the second stage again, and the fourth stage and the third stage fusion feature map is upsampled and superimposed with the feature extracted in the second stage to obtain the fusion feature map of the fourth stage, the third stage and the second stage; then the fusion features of the fourth stage, the third stage and the second stage are upsampled and superimposed with the feature extracted in the first stage to obtain the final fusion feature map. Through this operation, the shallow layer containing the bottom layer features and rich detail information (such as the edge of the object) can also contain rich strong semantic information.

[0013] The above full convolution network FCN includes a first convolutional layer with a step size of 1, a convolution kernel size of 3x3 and a channel number of 32C, a transposed convolutional layer with a step size of 2, a convolution kernel size of 2x2 and a channel number of 32C, and a second convolutional layer with a step size of 1 and a convolution kernel size of 1x1. The second convolutional layer compresses the output channel number of the feature map to 3, i.e. the total number of defect types.

[0014] In step S2, the feature map with a channel number of 3 obtained by the semantic segmentation network is visualized to obtain the position information of the three types of defects. Each channel corresponds to the output of a defect extraction feature image, and when the corresponding defect does not exist, a blank image is output. Among them, the tin beads are relatively obvious, and whether the tin beads are contained in the output feature map can be directly judged according to the mask. The tin plucking is determined according to the length of the tin segment attached to the copper wire, so the length of the tin segment in the feature map can be calculated, and if the tin segment is greater than the set threshold, it is considered that the tin plucking occurs, and the tin segment is visualized as a tin plucking defect. In addition, the detection of the virtual welding defect is essentially the detection of the copper wire on the terminal, in order to prevent confusion with other copper wires in the armature, it is stipulated that if the copper wire is detected in the specified area (i.e. the terminal area) in the image, it is considered that the virtual welding defect occurs.

[0015] Compared with the prior art, the micro-vibration motor armature defect detection method based on semantic segmentation provided by the present application has the following beneficial effects:

[0016] The present application first uses the converter ST to perform multi-scale feature extraction on the image to be detected, then fuses the extracted features through the feature fusion network FPN, then extracts semantic information from the fused feature map through the full convolution network FCN, and outputs through the channel corresponding to the total number of defect types, and finally generates a visual image containing the defect position based on the extracted semantic information, and identifies the defect type based on the position information. Through the present application, the position of the micro-vibration motor armature surface defect can be accurately positioned while the defect is identified. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 The micro-vibration motor armature defect image and the label image; (a) is a defect-free original image, (b) is a defect-free mask image, (c) is a tin plucking defect original image, (d) is a tin plucking defect mask image, (e) is a tin bead defect original image, (f) is a tin bead defect mask image, (g) is a virtual welding defect original image, and (h) is a virtual welding defect mask image. The above mask image is obtained by processing the corresponding original image through the semantic segmentation network given above.

[0018] Figure 2 The structure diagram of the constructed semantic segmentation network. DETAILED DESCRIPTION

[0019] The technical solutions of the embodiments of the present application will be described clearly and completely in combination with the drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor belong to the present application.

[0020] Embodiment

[0021] The images used in this embodiment are captured by the imaging system from a fixed angle with a resolution of 540*1024. The data set contains a total of 500 images; among them, 400 images for the training set and 100 images for the test set. Through analysis of the collected motor armature image surface data, three main defects are obtained as shown in Table 1. Each motor armature can contain multiple defects, and the characteristics of the defects are different. The corresponding features are extracted by the semantic segmentation network provided by the present application, and pixel-level labeling is performed as shown in Figure 1 The data set used in this embodiment contains a total of three labels: tin segment, tin bead and virtual welding. Among them, the tin bead has obvious characteristics, which can be directly judged according to whether the tin bead mask is contained in the output feature map. The tin segment is determined according to the length of the tin segment attached to the copper wire, so the length of the tin segment in the feature map can be calculated, and if the tin segment is greater than the set threshold, it is considered that tin segment occurs, and the tin segment is immediately visualized as a tin segment defect. In addition, detecting virtual welding defects is essentially detecting copper wires on the terminal. In order to prevent confusion with other copper wires in the armature, it is stipulated here that if copper wire is detected in the specified area (i.e. the terminal area) in the lower left corner of the image, it is considered that a virtual welding defect occurs.

[0022] Table 1 Armature defect information

[0023]

[0024] Based on the above analysis, the micro-vibration motor armature defect detection method based on semantic segmentation provided by this embodiment includes the following steps:

[0025] S1. Perform semantic segmentation on the micro-vibration motor image to be tested based on the constructed semantic segmentation network to obtain a feature map.

[0026] As shown in Figure 2 , the semantic segmentation network includes a converter ST, a feature fusion network FPN and a full convolutional network FCN.

[0027] The converter ST is used to extract features of different scales for the micro-vibration motor image on the to-be-processed side. The network structure of the converter ST adopts a conventional network structure disclosed in the prior art, see Z. Liu et al., "SwinTransformer: Hierarchical Vision Transformer using Shifted Windows," 2021 IEEE / CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada, 2021, pp. 9992-10002, doi: 10.1109 / ICCV48922.2021.00986. The hierarchical structure of the converter ST is divided into four stages with the number of channels increasing in turn. In order to describe the change of the feature map in the hierarchical structure, the size of the feature map is represented in the form of (height, width, channels). H, W and C represent the height, width and channel of the input image respectively. From the first stage to the fourth stage, the size of the feature map output by each stage is (H / 4, W / 4, 32C), (H / 8, W / 8, 64C), (H / 16, W / 16, 128C) and (H / 32, W / 32, 256C) respectively. The output feature map of each stage is sent to the FPN for fusion at different scales.

[0028] The feature fusion network FPN is used to fuse the features of different scales extracted. The feature fusion network FPN branch includes a bottom-up path, a top-down path and a lateral connection. The fourth stage features extracted by the converter ST are first up-sampled to restore the feature map to the same spatial size as the third stage; then the two layers are connected to merge their feature maps, obtaining the fusion features of the fourth stage and the third stage feature maps. The same principle is applied to the second stage again, the fourth stage and the third stage fusion feature maps are up-sampled, and then superimposed with the features extracted in the second stage to obtain the fusion feature maps of the fourth stage, the third stage and the second stage; then the fusion features of the fourth stage, the third stage and the second stage are up-sampled, and then superimposed with the features extracted in the first stage to obtain the final fusion feature map. Through this operation, the shallow layer containing the bottom layer features and rich detail information (such as the edge of the object) can also contain rich strong semantic information. The feature map output from the final output of the FPN is selected for subsequent calculation in this embodiment, and the size of the feature map is (H / 4, W / 4, 32C).

[0029] The full convolutional network FCN is used to extract semantic information from the fused feature map to complete semantic segmentation. The full convolutional network FCN sequentially comprises four first convolutional layers with a step of 1, a convolution kernel of 3*3 and a channel number of 32C, one transposed convolutional layer with a step of 2, a convolution kernel size of 2*2 and a channel number of 32C, and one second convolutional layer with a step of 1 and a convolution kernel size of 1*1. The second convolutional layer compresses the output channel number of the feature map to 3, that is, the total number of defect types, and each channel corresponds to an output feature map of one defect. When the corresponding defect does not exist, the output image is a blank image.

[0030] S2 visually displays the feature image obtained by semantic segmentation, and then identifies the feature map according to the set standard to determine whether the armature defect exists.

[0031] According to the foregoing analysis, the set standard of the embodiment includes: (1) when the output feature image of the channel corresponding to the tin bead defect contains a tin bead, it indicates that the tin bead defect exists; (2) when the length of the tin segment in the output feature image of the channel corresponding to the tin pulling defect is greater than a set threshold, it indicates that the tin pulling defect exists; (3) when the copper wire exists in the specified area of the output feature image of the channel corresponding to the virtual welding defect, it indicates that the virtual welding defect exists.

[0032] The tin bead and virtual welding defects can be directly discriminated according to the output feature image corresponding to the defect, and the tin pulling can be discriminated by calculating the length of the tin segment through post-processing.

[0033] In the application process, first, the semantic segmentation network is trained by using the training set, and cross entropy is used as the loss function, and then the network parameters are optimized by the Adam optimization algorithm until the loss value meets the set requirement. Then, the trained semantic segmentation network is used to test the test set according to the steps S1-S2 given above. Through the micro-vibration motor armature defect detection method based on semantic segmentation provided by the application, the defect recognition accuracy can reach 95%, and the accurate positioning of the defect can be realized.

[0034] Those skilled in the art will appreciate that the embodiments described herein are intended to help the reader understand the principles of the application and should be understood as not limiting the scope of protection of the application to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations according to the technical inspiration disclosed in the application without departing from the essence of the application, and these modifications and combinations are still within the scope of protection of the application.

Claims

1. A method for detecting armature defects in a micro-vibration motor based on semantic segmentation, characterized in that, Includes the following steps: S1 uses a constructed semantic segmentation network to perform semantic segmentation on the image of the micro-vibration motor under test to obtain a feature map. The semantic segmentation network includes a converter ST, a feature fusion network FPN, and a fully convolutional network FCN. The converter ST is used to extract features at different scales from the image of the micro-vibration motor under test. The feature fusion network FPN is used to fuse the extracted features at different scales. The fully convolutional network FCN is used to extract semantic information from the fused feature map, complete semantic segmentation, and output feature images corresponding to each defect. S2 visualizes the feature image obtained from semantic segmentation, and then identifies the feature image according to the set criteria to determine whether there is an armature defect. The set criteria include: (1) when the output feature image of the channel corresponding to the solder ball defect contains solder balls, it indicates that there is a solder ball defect; (2) when the length of the solder segment in the output feature image of the channel corresponding to the solder stripping defect is greater than the set threshold, it indicates that there is a solder stripping defect; (3) when there is a copper wire in the specified area of ​​the output feature image of the channel corresponding to the cold solder joint defect, it indicates that there is a cold solder joint defect.

2. The method for detecting armature defects in a micro-vibration motor based on semantic segmentation according to claim 1, characterized in that, The hierarchical structure of the converter ST is divided into four stages with the number of channels increasing sequentially.

3. The method for detecting armature defects in a micro-vibration motor based on semantic segmentation according to claim 2, characterized in that, From the first stage to the fourth stage, the size of the feature map output by each stage is (H / 4, W / 4, 32C), (H / 8, W / 8, 64C), (H / 16, W / 16, 128C), and (H / 32, W / 32, 256C), respectively, where H, W, and C represent the height, width, and channels of the input image, respectively.

4. The method for detecting armature defects in a micro-vibration motor based on semantic segmentation according to claim 2 or 3, characterized in that, The Feature Fusion Network (FPN) first upsamples the fourth-stage features extracted by the converter ST, restoring its feature map to the same spatial size as the third stage; then, it connects these two layers and merges their feature maps to obtain the fused features of the fourth-stage and third-stage feature maps. Similarly, the fused feature maps of the fourth and third stages are upsampled and then superimposed with the features extracted in the second stage to obtain the fused feature maps of the fourth, third, and second stages; then the fused features of the fourth, third, and second stages are upsampled and then superimposed with the features extracted in the first stage to obtain the final fused feature map.

5. The method for detecting armature defects in a micro-vibration motor based on semantic segmentation according to claim 4, characterized in that, The fully convolutional network (FCN) consists of four first convolutional layers with a stride of 1 and a kernel size of 3×3, one transposed convolutional layer with a stride of 2 and a kernel size of 2×2, and one second convolutional layer with a stride of 1 and a kernel size of 1×1, arranged sequentially.

6. The method for detecting armature defects in a micro-vibration motor based on semantic segmentation according to claim 5, characterized in that; The second convolutional layer has 3 output channels, which represents the total number of defect types.

Images