Ball screw driving surface pitting defect detection method based on CA attention mechanism and RepGhost network
By introducing the CA attention mechanism and RepGhost network into the DeepLab v3+ model, the RG-DeepLab v3+ model is constructed, and the limitations of detailed information capture and processing in ball screw-driven surface pitting defect detection are solved, achieving more efficient and accurate defect detection.
Patent Information
- Application Number
- CN202411912425.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-24
- Publication Date
- 2025-04-15
AI Technical Summary
The prior art has limitations in the detection of detailed information capture and processing in ball screw-driven surface pitting defects, resulting in a degradation of detection performance.
The ball screw-driven surface pitting defect detection method based on the CA attention mechanism and the RepGhost network is adopted. By adding the Coordinate Attention (CA) attention mechanism after the ASPP module of the DeepLab v3+ model, and using the lightweight RepGhost network as the backbone feature extraction network, an improved DeepLab v3+ model, namely RG-DeepLab v3+.
The model's expression ability and spatial sensitivity to image features are improved, especially when dealing with tiny pitting defects, and the segmentation performance of the ball screw-driven surface pitting defects is significantly improved.
Smart Images
Figure CN119832265A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of defect detection technology, and specifically to a method for detecting pitting defects on the surface of a ball screw drive based on a CA attention mechanism and a RepGhost network. Background Art
[0002] As the core execution unit in machine tools, the main function of ball screw drive is to accurately convert rotational motion into linear motion and simultaneously achieve high-precision positioning and transmission. However, once pitting occurs on the surface of ball screw drive, it will have a significant negative impact on its performance and reliability. In extreme cases, it may even cause complete failure, which is the main cause of unplanned failures of machine tools. The detection of pitting defects on the surface of traditional ball screw drives mainly relies on manual inspection. This method is not only inefficient and costly, but also the obtained machine tool component status data often has high uncertainty and time lag. Therefore, it is particularly urgent to develop a method that can automatically detect and evaluate faults in a timely and accurate manner to make up for the shortcomings of traditional manual detection.
[0003] With the rapid development of deep learning technology, semantic segmentation technology based on convolutional neural networks has gradually been applied to the field of industrial defect detection. Among them, DeepLabV3+, as the latest version of the DeepLab series of models, has demonstrated excellent performance in semantic segmentation tasks. DeepLabV3+ adopts an encoder-decoder architecture. The encoder part consists of a backbone network and a dilated spatial pooling pyramid module, and the decoder part includes upsampling and convolution modules. This design enables DeepLabV3+ to achieve more accurate segmentation in complex image scenes, especially in processing details such as object edges.
[0004] Although DeepLabV3+ performs well in many aspects, it still has certain limitations in capturing and processing detailed information. In the encoder stage, DeepLabV3+ directly fuses the feature maps that are downsampled 16 times, which will lead to the loss of some image detail information. At the same time, relying solely on the atrous spatial pyramid pooling module is not enough to fully capture multi-scale information, especially when surface defects account for a small proportion of the image, which will lead to a decrease in detection performance. Summary of the invention
[0005] The present invention provides a ball screw drive surface pitting defect detection method based on CA attention mechanism and RepGhost network, which solves the problem that DeepLab v3+ still has certain limitations in capturing and processing detail information.
[0006] To achieve the above objectives, the present invention is implemented by the following technical scheme: a method for detecting pitting defects on the surface of a ball screw drive based on a CA attention mechanism and a RepGhost network, comprising the following steps:
[0007] S1: Obtain an image of the ball screw drive surface, annotate it at the semantic level, and obtain a data set after data processing;
[0008] S2: Use the lightweight RepGhost network to replace the original ResNet network of the DeepLab v3+ model as the backbone feature extraction network. RepGhostNet adopts the RepGhost Bottleneck structure based on the RepGhost module, which reduces the number of model parameters and calculations while maintaining high feature extraction and fusion capabilities;
[0009] S3: Add the Coordinate Attention (CA) mechanism after the ASPP module of the DeepLab v3+ model. By decomposing the input feature map, aggregating features in the vertical and horizontal directions, generating spatial attention weights, and enhancing the model's ability to express image features and spatial sensitivity;
[0010] S4: Combine the RepGhostNet in step S2 with the CA attention mechanism in step S3 to build an improved DeepLab v3+ model, namely RG-DeepLab v3+. This model uses RepGhostNet in the encoder's backbone feature extraction network to extract multi-scale features, applies the CA attention mechanism after the ASPP module to enhance feature representation, and finally restores the spatial information of the image through the decoder, thus achieving high-precision semantic segmentation while reducing the number of model parameters and computational complexity.
[0011] S5: The RG-DeepLab v3+ model is trained and tested on a ball screw drive surface pitting defect image dataset, using cross entropy as the loss function, setting appropriate hyperparameters, and using stochastic gradient descent (SGD) for optimization training.
[0012] The annotation described in step S1 is to use the Ploygon form of the labelme tool to annotate the pitting defects in the surface image of the ball screw drive.
[0013] The data processing described in step S1 is to use a batch processing tool to convert the image semantic labels in json format into corresponding mask image files in png format, and then rotate, block, translate, adjust the brightness, and randomly add noise to the ball screw drive surface pitting defect image and its mask image to obtain a data set.
[0014] The RepGhost Bottleneck structure described in step S2 first halves the number of channels of the input feature map through a 1×1 convolutional layer, applies the ReLU activation function to enhance nonlinearity, and keeps the number of channels unchanged after convolution through the first RepGhost module. Then, the SE channel attention layer is used to reweight the channel features to improve the model's sensitivity to important features. To restore the number of channels, the feature map is adjusted again through a 1×1 convolutional layer. Finally, the processed feature map is scaled and added to the initial input feature map to fuse the feature information. During inference, the RepGhost bottleneck contains only two branches, which saves memory resources and improves inference speed.
[0015] The ASPP module described in step S3 includes a 1×1 convolution module, a dilated convolution module, and a pooling pyramid module. First, the input feature map is reduced in dimension or channel adjusted by a 1×1 convolution, and then a plurality of dilated convolution layers with different expansion rates are used to extract multi-scale features, and the global average pooling layer is combined to obtain global context information, and finally the output of the dilated convolution layer is feature fused with the output of the global pooling layer to obtain a fused feature map.
[0016] The ASPP module in step S3 has limitations in capturing spatial information, and its ability to express fine-grained details and contextual dependencies in complex scenes needs to be improved. Specifically, in image scenes with occlusion, illumination changes, and large scale differences, the segmentation accuracy of the ASPP module needs to be further improved.
[0017] The CA attention mechanism is added after the ASPP module in step S3 to enhance the model's ability to identify key features and its sensitivity to spatial relationships. The mechanism encodes positional spatial information into channel attention, uses two specialized modules to capture attention features in the vertical and horizontal directions respectively, and uses these two modules to replace the traditional global average pooling process. Through this design, the CA mechanism can more effectively extract and utilize spatial information, thereby improving the performance of the model.
[0018] The CA attention mechanism can be used to treat any intermediate feature tensor X = [x 1 ,x 2 ,...,x c ]∈R H×W×C , converted to a transformed tensor Y of the same size as X, Y = [y 1 ,y 2 ,...,y c ]∈R H×W×C , enhance the learning ability of the network model. Use two spatial range pooling kernels (H, 1) and (1, W) to encode each channel in the horizontal and vertical directions respectively, where the output of the cth channel at height h is Z ch (h), the output of the cth channel at width w is Z c w (w), the specific formula is as follows:
[0019]
[0020]
[0021] The CA attention mechanism is implemented by sharing a 1×1 convolution transformation function F 1 The feature layer is normalized to obtain the intermediate feature map f that encodes spatial information in the horizontal and vertical directions. The specific formula is as follows:
[0022] f=δ(F 1 ([Z h ,Z W ])) (3)
[0023]
[0024] Where δ is a nonlinear activation function, [,] is the connection operation in the spatial dimension, Represents the set of real numbers.
[0025] The f will be split into two separate tensors along the spatial dimension and Then f h With f w Perform convolution and activation operations, and finally obtain the output result y of the CA attention mechanism through fusion c (i,j), the specific formula is as follows:
[0026]
[0027] where x c (i, j) is the value of the input feature map at position (i, j).
[0028] The present invention provides a method for detecting pitting defects on the surface of a ball screw drive based on a CA attention mechanism and a RepGhost network. It has the following beneficial effects:
[0029] 1. The present invention uses a lightweight RepGhost network as the backbone feature extraction network. RepGhostNet adopts the RepGhost Bottleneck structure based on the RepGhost module, which reduces the number of model parameters and the amount of calculation while maintaining a high feature extraction and fusion capability, making it more suitable for practical application deployment;
[0030] 2. The present invention adds a CA attention mechanism after the ASPP module of the DeepLab v3+ model. By decomposing the input feature map, aggregating features in the vertical and horizontal directions, and generating spatial attention weights, the model's ability to express image features and spatial sensitivity are enhanced, especially when dealing with tiny pitting defects;
[0031] 3. The RG-DeepLab v3+ model constructed by combining RepGhostNet with the CA attention mechanism exhibits superior surface pitting defect segmentation performance.
[0032] In summary, the method proposed in the present invention provides a new idea for accurate and efficient semantic segmentation of ball screw drive surface pitting defect images, and has broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] Figure 1 This is a flow chart of the method for detecting pitting defects on the surface of a ball screw drive proposed by the present invention;
[0034] Figure 2 An image display diagram collected for the present invention;
[0035] Figure 3 A mask image display diagram of the present invention;
[0036] Figure 4 It is a curve diagram of the loss function change during the training process of the present invention;
[0037] Figure 5 This is the RepGhost Bottleneck structure diagram of the present invention, Figure 5 (a) is the structure during training, Figure 5 (b) is the structure during reasoning;
[0038] Figure 6 This is the principle diagram of the CA attention mechanism of the present invention;
[0039] Figure 7 This is the RG-DeepLab v3+ network model architecture diagram of the present invention;
[0040] Figure 8 It is a curve diagram of the model evaluation index of the present invention;
[0041] Fig. 9 This is a diagram of the prediction results of the present invention. DETAILED DESCRIPTION
[0042] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0043] Example:
[0044] Reference Figure 1-3 The embodiment of the present invention provides a method for detecting pitting defects on the surface of a ball screw drive based on a CA attention mechanism and a RepGhost network, comprising the following steps:
[0045] Firstly, a camera is installed on the ball screw drive nut to obtain the image of the ball screw drive surface, and the Labelme software is used to perform semantic-level annotation on it. Then, a batch processing tool is used to convert the image semantic labels in json format into mask image files in corresponding png format. Then, the pitting defect image on the ball screw drive surface and its mask image are rotated, blocked, translated, brightness adjusted, and noise is randomly added to obtain the data set used in the present invention.
[0046] Subsequently, the dataset was used to train the RG-DeepLab v3+ model proposed in the present invention. During the training process, the input size of the model proposed in the present invention is 150×150, the batch size is 16, the multilinear process is 8, the momentum factor is 0.95, the initial learning rate is 0.01, and the weight decay is 0.0005. The number of samples per batch is 16, the multilinear process is 8, the momentum factor is 0.937, the initial learning rate is 0.01, and the weight decay is 0.0005. Stochastic gradient descent (SGD) is selected as the optimizer, the number of iterations is 40,000, and the cross entropy is used as the loss function. The model training parameters are shown in Table 1.
[0047] Table 1 Model training parameters
[0048]
[0049] Reference Figure 4 ,By analyzing the change curve of the loss function of the training process of the RG-DeepLab v3+ model, it is found that the model tends to converge after 25,000 iterations.
[0050] Reference Figure 5RepGhostNet is an efficient and lightweight network architecture that uses RepGhostBottleneck as the basic building block of the network. The RepGhost Bottleneck structure first halves the number of channels of the input feature map through a 1×1 convolutional layer, applies the ReLU activation function to enhance nonlinearity, and keeps the number of channels unchanged after convolution through the first RepGhost module. Then, the SE channel attention layer is used to reweight the channel features to improve the model's sensitivity to important features. To restore the number of channels, the feature map is adjusted again through a 1×1 convolutional layer. Finally, the processed feature map is scaled and added to the initial input feature map to fuse the feature information. During inference, the RepGhost bottleneck contains only two branches, which saves memory resources and improves inference speed.
[0051] Reference Figure 6 The CA attention mechanism is a mechanism for enhancing the feature expression ability of the network. Its implementation involves two core steps: coordinate information embedding and coordinate attention generation. First, in the coordinate information embedding stage, a global pooling decomposition is performed on the input feature map, and feature aggregation is performed in the horizontal and vertical directions respectively to generate two direction-aware feature maps. Then, in the coordinate attention generation stage, the two direction-aware feature maps are concatenated and feature transformed through a shared 1×1 convolution module, followed by batch normalization and activation function processing to obtain an attention weight map. Finally, the attention weight map is applied to the height and width directions of the input feature map respectively, and the final output feature map is obtained through multiplication weighted operations. This mechanism effectively integrates position information and significantly improves the feature expression ability.
[0052] Reference Figure 7 The present invention combines the aforementioned RepGhostNet with the CA attention mechanism to construct an improved DeepLab v3+ model, namely RG-DeepLab v3+. This model uses RepGhostNet to extract multi-scale features in the backbone feature extraction network of the encoder, applies the CA attention mechanism to enhance feature representation after the ASPP module, and finally restores the spatial information of the image through the decoder, thereby achieving high-precision semantic segmentation while reducing the number of model parameters and computational complexity;
[0053] Reference Figure 8-9 In order to verify the superiority of the RG-DeepLab v3+ model, the present invention conducted a prediction experiment on the RG-DeepLab v3+ model on the ball screw drive surface dataset. Through visual effect comparison and quantitative indicator analysis, the excellent performance of RG-DeepLab v3+ in the task of ball screw drive surface pitting defect detection was confirmed.
[0054] In summary, the proposed method for detecting pitting defects on the surface of ball screw drives based on CA attention mechanism and RepGhost network uses RepGhostNet to extract multi-scale features in the backbone feature extraction network of DeepLab v3+ model encoder, and CA attention mechanism is applied after ASPP module to enhance feature representation. The prediction results show that RG-DeepLabv3+ has excellent performance advantages and provides a new solution for efficient and accurate detection of pitting defects on the surface of ball screw drives.
[0055] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.
Claims
1. A ball screw drive surface pitting defect detection method based on CA attention mechanism and RepGhost network, characterized in that: The following steps are involved: S1: Obtain an image of the ball screw drive surface, annotate it at the semantic level, and obtain a data set after data processing; S2: Use the lightweight RepGhost network to replace the original ResNet network of the DeepLab v3+ model as the backbone feature extraction network. RepGhostNet adopts the RepGhost Bottleneck structure based on the RepGhost module, which reduces the number of model parameters and calculations while maintaining high feature extraction and fusion capabilities; S3: Add the Coordinate Attention (CA) mechanism after the ASPP module of the DeepLab v3+ model. By decomposing the input feature map, aggregating features in the vertical and horizontal directions, generating spatial attention weights, and enhancing the model's ability to express image features and spatial sensitivity; S4: Combine the RepGhostNet in step S2 with the CA attention mechanism in step S3 to build an improved DeepLab v3+ model, namely RG-DeepLab v3+. This model uses RepGhostNet in the encoder's backbone feature extraction network to extract multi-scale features, applies the CA attention mechanism after the ASPP module to enhance feature representation, and finally restores the spatial information of the image through the decoder, thus achieving high-precision semantic segmentation while reducing the number of model parameters and computational complexity. S5: The RG-DeepLab v3+ model is trained and tested on a ball screw drive surface pitting defect image dataset, using cross entropy as the loss function, setting appropriate hyperparameters, and using stochastic gradient descent (SGD) for optimization training.
2. According to claim 1, a method for detecting pitting defects on the surface of a ball screw drive based on a CA attention mechanism and a RepGhost network is characterized in that: The annotation described in step S1 is to use the Ploygon form of the labelme tool to annotate the pitting defects in the surface image of the ball screw drive.
3. According to claim 1, a method for detecting pitting defects on the surface of a ball screw drive based on a CA attention mechanism and a RepGhost network is characterized in that: The data processing described in step S1 is to use a batch processing tool to convert the image semantic labels in json format into corresponding mask image files in png format, and then rotate, block, translate, adjust the brightness, and randomly add noise to the ball screw drive surface pitting defect image and its mask image to obtain a data set.
4. According to claim 1, a method for detecting pitting defects on the surface of a ball screw drive based on a CA attention mechanism and a RepGhost network is characterized in that: The RepGhost Bottleneck structure described in step S2 first halves the number of channels of the input feature map through a 1×1 convolution layer, applies the ReLU activation function to enhance nonlinearity, and keeps the number of channels unchanged after convolution through the first RepGhost module; then, the SE channel attention layer is used to reweight the channel features to improve the model's sensitivity to important features; to restore the number of channels, the feature map is adjusted again through a 1×1 convolution layer; finally, the processed feature map is scaled and added to the initial input feature map to fuse the feature information; during inference, the RepGhost bottleneck contains only two branches, which saves memory resources and improves the inference speed.
5. According to claim 1, a method for detecting pitting defects on the surface of a ball screw drive based on a CA attention mechanism and a RepGhost network is characterized in that: The ASPP module described in step S3 includes a 1×1 convolution module, a dilated convolution module and a pooling pyramid module; first, the input feature map is reduced in dimension or the channel is adjusted by a 1×1 convolution, and then a plurality of dilated convolution layers with different expansion rates are used to extract multi-scale features, and the global average pooling layer is combined to obtain the global context information, and finally, the output of the dilated convolution layer is feature fused with the output of the global pooling layer to obtain the fused feature map.
6. According to claim 1, a method for detecting pitting defects on the surface of a ball screw drive based on a CA attention mechanism and a RepGhost network is characterized in that: The ASPP module in step S3 has limitations in capturing spatial information, and its ability to express fine-grained details and contextual dependencies in complex scenes needs to be improved. Specifically, in image scenes with occlusion, illumination changes, and large scale differences, the segmentation accuracy of the ASPP module needs to be further improved.
7. According to claim 1, a method for detecting pitting defects on the surface of a ball screw drive based on a CA attention mechanism and a RepGhost network is characterized in that: The CA attention mechanism is added after the ASPP module in step S3 to enhance the model's ability to identify key features and its sensitivity to spatial relationships. The mechanism encodes positional spatial information into channel attention, uses two specialized modules to capture attention features in the vertical and horizontal directions respectively, and uses these two modules to replace the traditional global average pooling process. Through this design, the CA mechanism can more effectively extract and utilize spatial information, thereby improving the performance of the model.
8. According to claim 1, a method for detecting pitting defects on the surface of a ball screw drive based on a CA attention mechanism and a RepGhost network is characterized in that: The CA attention mechanism can focus any intermediate feature tensor X=[x1,x2,...,x c ]∈R H×W×C , converted to a transformed tensor Y of the same size as X, Y = [y1, y2, ..., y c ]∈R H ×W×C , enhance the learning ability of the network model; use two spatial range pooling kernels (H, 1) and (1, W) to encode each channel in the horizontal and vertical directions respectively, where the output of the cth channel at height h is The output of the cth channel at width w is The specific formula is as follows:
9. According to claim 1, a method for detecting pitting defects on the surface of a ball screw drive based on a CA attention mechanism and a RepGhost network is characterized in that: The CA attention mechanism normalizes the feature layer through a shared 1×1 convolution transformation function F1 to obtain an intermediate feature map f that encodes spatial information in the horizontal and vertical directions. The specific formula is as follows: f=δ(F1([Z h ,Z W ])) (3) Where δ is a nonlinear activation function, [,] is the connection operation in the spatial dimension, Represents the set of real numbers.
10. According to claim 1, a method for detecting pitting defects on the surface of a ball screw drive based on a CA attention mechanism and a RepGhost network is characterized in that: The f will be split into two separate tensors along the spatial dimension and Then f h With f w Perform convolution and activation operations, and finally obtain the output result y of the CA attention mechanism through fusion c (i,j), the specific formula is as follows: where x c (i, j) is the value of the input feature map at position (i, j).