Method for detecting straight fluted tap through polarization imaging based on deep learning
Through deep learning-based polarization imaging technology, combined with Mask R-CNN model and custom loss function, the problem that traditional detection methods are difficult to accurately detect the micro structural details of the tap is solved, and high-precision, fast and non-destructive detection of the tap is achieved, reducing the Manpower consumption.
Patent Information
- Application Number
- CN202411392194.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-08
- Publication Date
- 2024-12-31
AI Technical Summary
Traditional imaging technology is difficult to accurately detect the tiny structural details and surface defects of the tap, and manual inspection is time-consuming and labor-intensive, making it impossible to achieve lossless, fast and accurate detection.
Using a polarization imaging detection method based on deep learning, a polarization camera is used to acquire multi-polarization angle images, denoising and contrast enhancement are performed through RDN residual dense neural network, and feature segmentation and parameter measurement are combined with Mask R-CNN model to introduce custom losses. Function and attention mechanisms use geometric prior knowledge to optimize the segmentation process to achieve accurate identification of tap defects and wear.
High-precision detection of taps is achieved, manpower and material consumption is reduced, detection is improved, detection is faster and accurate, the limitations of traditional measurement tools are overcome, and complex features can be handled.
Smart Images
Figure CN119228776A_ABST
Abstract
Description
(I) Technical field
[0001] The present invention is a method for detecting straight groove taps by polarization imaging based on deep learning. It utilizes polarization imaging technology combined with a deep learning neural network to detect wear and defects of taps caused by improper cutting parameters and other operations when processing straight groove taps, and combines image processing technology to detect tap parameters. It belongs to the field of optical image information extraction technology. (II) Background technology
[0002] Polarization imaging can reveal the optical properties of an object's surface, which are often difficult to capture using traditional imaging techniques. By analyzing the polarization state of light, the technology can significantly improve the contrast of the image, allowing the tiny structural details and surface defects of the tap to be clearly displayed. In the field of industrial inspection, especially in the inspection of taps with extremely high precision requirements, polarization imaging technology can provide richer and more intuitive surface information than other imaging methods. In addition, as a non-contact inspection method, polarization imaging avoids physical contact with the inspected object, thereby reducing potential damage to precision tools such as taps, while also improving the safety and reliability of the inspection process. Therefore, we considered using polarization imaging technology to inspect taps and obtain their surface information at the same time.
[0003] Although polarization imaging technology has obvious advantages in image acquisition, how to accurately extract features from these images and conduct effective analysis remains a technical challenge. Traditionally, the use of Wangongxian to detect the features and measurements of taps often requires workers to perform a large number of repeated manual corrections, which is not only time-consuming and labor-intensive, but may not work well when dealing with complex or unknown features. Deep learning, as an advanced data analysis technology, can automatically learn complex patterns and features in image data through training. In polarization image processing, deep learning models, especially convolutional neural networks (CNNs), have been shown to have excellent performance in image classification, feature extraction, and pattern recognition. Through deep learning, a powerful image processing framework is constructed to automatically identify and quantify the wear state and various features of taps, thereby achieving non-destructive, fast, and accurate detection. This paper proposes a straight groove tap detection method based on polarization imaging of MaskR-CNN neural network for wear, defects, and tap parameters of straight groove taps. The purpose is to use deep learning models to perform feature segmentation on polarization images, thereby achieving high-precision detection of taps. This method not only overcomes the limitations of traditional measurement tools, but also gives full play to the advantages of polarization imaging in processing complex features. (III) Summary of the invention
[0004] The purpose of the present invention is to provide a method for detecting straight groove taps by polarization imaging based on deep learning. It solves the problem of clearly displaying tiny structural details and surface defects in taps in traditional imaging technology, and solves the problem that manual contact and manual visual inspection of taps cannot achieve non-destructive, fast and accurate inspection, greatly reduces the consumption of manpower and material resources, and realizes accurate identification and parameter measurement of tap defects and wear.
[0005] The object of the present invention is achieved in that:
[0006] A method for detecting straight groove taps by polarization imaging based on deep learning, comprising the following steps:
[0007] Step S1: Use a polarization camera to collect multi-polarization angle images of straight groove taps at different shooting azimuths, including 0°, 45°, and 90° polarization images, to form a training set, a validation set, and a test set. Use the RDN residual dense neural network to denoise, enhance contrast, and perform matching calibration calculations on the polarization images to obtain polarization degree and polarization angle images.
[0008] Step S2: Annotate the polarization degree and polarization angle images of the training set and validation set. The annotations include the tap end, core, blade back, cutting tooth features, as well as wear and defect areas. Use the model pre-trained on ImageNet as a basis to train the Mask R-CNN model. By evaluating the model's accuracy, recall rate, and F1 score on the validation set, select the ideal weight file and obtain the neural network model.
[0009] Step S3: Use the Mask R-CNN segmentation model to detect defects and wear on the polarized images of the straight groove taps in the test set, and segment key features such as the end, core, blade back, and cutting teeth.
[0010] Step S4: Use image processing technology to extract the mask of the segmented image, and generate a classification probability map for each pixel. Use the segmented mask to perform calibration measurement and calculate the size, including the end diameter, core diameter, blade back width, number of cutting teeth, radial back of the cutting cone, back angle and front angle.
[0011] Step S5: Use standard size parameter products to verify the measurement results. Finally, compare the test size parameters with the standard size parameters, select unqualified products and classify them into different models.
[0012] Preferably, step S2: train the Mask R-CNN model using the labeled training set. A transfer learning strategy is adopted during the training process, using a model pre-trained on the large dataset ImageNet as a starting point. Adding new structures based on the pre-trained model can accelerate the learning process and improve the model's recognition accuracy for specific defect features of taps. The new structures added to the pre-trained model are:
[0013] These include: introducing a custom loss function to give higher weights to specific defects of the tap (such as wear and cracks) so that the model can pay more attention to these key defects. And adding an attention mechanism to make the model pay more attention to key areas of the tap surface, such as the back of the blade surface and the cutting tooth surface, to improve the accuracy of defect detection.
[0014] Preferably, the process of introducing a custom loss function and adding a DANet attention mechanism is:
[0015] A custom loss function based on Focal Loss is introduced, which uses a dynamically adjusted factor that is automatically adjusted according to the frequency of the target category to increase the model's sensitivity to rare defects. The model pays more attention to these key defects by reducing the weights of easy-to-classify samples and increasing the weights of difficult-to-classify samples. An attention module is added after each convolutional layer, which learns the importance weights of each channel through global average pooling and fully connected layers, and then applies these weights to the input feature map.
[0016] Preferably, the Mask R-CNN model based on ResNet is used for tap feature segmentation, and a new structure is added to this part:
[0017] Includes: Optimizing the segmentation process through geometric prior knowledge. In the mask prediction branch, the geometric parameters of the tap (such as straight groove width, end diameter, core diameter, blade back width, etc.) are combined to guide the segmentation network through custom geometric constraints. These geometric prior knowledge helps the model to more accurately follow the actual shape and size restrictions of the tap during segmentation. Reasonable size restrictions are set to guide the model to make more accurate region proposals and avoid generating small or irregular regions that do not conform to the actual size of the tap. Finally, the segmentation results are optimized by removing small areas and merging adjacent areas.
[0018] Preferably, the size is calculated using the segmented mask, and the parameters including the end diameter, core diameter, blade width, number of cutting teeth, radial relief of the cutting cone, back angle and front angle are measured. The specific measurement method is as follows:
[0019] These include: By identifying the masks of the tip and core, the tip diameter and core diameter can be determined using the minimum circumscribed circle method. Identifying the mask of the blade back and measuring the boundaries of the masked area to determine the blade back width. By analyzing the mask of the cutting teeth, the number of teeth can be calculated. By identifying specific points in the mask, the back angle and front angle can be determined by measuring the shortest distance between two edges and the angle between them. (IV) Description of the drawings
[0020] Figure 1 System flow chart of the method for inspecting straight groove taps using deep learning-based polarization imaging.
[0021] Figure 2 This is the training flow chart of the MaskR-CNN deep learning model in step S2. (V) Specific implementation methods
[0022] The present invention will be further described below in conjunction with specific embodiments.
[0023] Figure 1 The specific flow chart of the method for detecting straight groove taps by polarization imaging based on deep learning is given. The data acquisition model structure is composed of FD-1665 polarization imager (1, lens 2, 0°, 45°, 90° polarizers 3, 3 CCD sensors 4, three-way prism beam splitter).
[0024] The training set and test set images collected by the polarization imager in the system are denoised by the RDN residual dense neural network model to obtain the processed images, and then the three-angle polarization images are spatially aligned through feature point matching. The corresponding polarization degree and polarization angle images are then obtained through matching calibration calculation. The trained Mask R-CNN model is used to segment the polarization images of the straight groove taps in the test set, and the images are detected for defects and wear.
[0025] Image processing technology is used to extract the mask of the segmented image and generate a classification probability map for each pixel. The segmented mask is used for calibration measurement to calculate parameters including end diameter, core diameter, blade back width, number of cutting teeth, radial back of cutting cone, back angle and front angle.
[0026] These parameters are then compared to known calibration data to verify the accuracy of the measurements and ensure consistency with actual dimensions.
[0027] Compare the test size parameters with the standard size parameters, select unqualified products based on the comparison results, and classify them into different models.
[0028] Figure 2This is the training flow chart of the Mask R-CNN model described in step S2. First, the labeling tool LabelMe is used to manually label the tap end, core, blade back, cutting tooth features, and wear and defect areas to generate labeling data (JSON format).
[0029] These labeled data are converted to the COCO format required by Mask R-CNN.
[0030] When initializing the model, set the network structure, including the backbone network ResNet, the region proposal network (RPN) and the mask branch, and use pre-trained weights to accelerate training. During the forward propagation process, the input image generates a feature map through the backbone network, and the RPN applies a sliding window strategy on the feature map to generate a series of region candidate boxes through a 3×3 convolution kernel. These candidate boxes are potential target areas.
[0031] RPN generates the final region segmentation by predicting the target score of each candidate box and regressing the bounding box of the target. To enhance the feature expression ability and improve the model performance, Mask R-CNN uses residual connections to combine low-level and high-level features, reduce information loss and accelerate model training.
[0032] At the same time, the attention mechanism is introduced to enable the model to more effectively focus on key areas in the image, such as the thread groove of the tap and the tool edge. The attention mechanism enhances the model's focus on important features by assigning weights to each feature, thereby improving the accuracy of detection and segmentation.
[0033] The model then feeds these region proposals into deeper convolutional and fully connected layers for refinement. The mask branch generates pixel-level masks for each region, applying custom geometric constraints and post-processing techniques to further improve the accuracy of the segmentation results, including removing noisy areas (small areas) and merging adjacent segmented areas to ensure that the segmentation results are more consistent with the actual tap shape.
[0034] During the back-propagation process, the gradient is calculated and the model parameters are updated using the optimization algorithm. The Focal Loss custom loss function is added to the classification loss calculation part of the model to improve the model's sensitivity to rare defects. It is combined with the classification loss, bounding box regression loss, and mask loss during the training process. Finally, the minimum total loss is calculated and back-propagation is performed.
[0035] During the training process, the validation set is used to evaluate the model performance, check whether it is overfitting, and adjust the learning rate and batch size based on the validation results, and select the ideal weight file based on the training results. In the testing phase, the performance of the model on the test set is evaluated to ensure its generalization ability and actual application performance. Finally, the trained model can be deployed to actual application scenarios. The above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention is described in detail with reference to specific embodiments, a person skilled in the art should understand that the technical solutions of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solutions of the present invention, which should be included in the scope of the claims of the present invention. The technologies, shapes, and structural parts not described in detail in the present invention are all well-known technologies.
Claims
1. A straight groove tap detection system based on polarization imaging detection based on deep learning, characterized in that: The system comprises the following steps: Step S1: Use a polarization camera to collect multi-polarization angle images of straight groove taps at different shooting azimuths, including 0°, 45°, and 90° polarization images, to form a training set, a validation set, and a test set. Use the RDN residual dense neural network to denoise, enhance contrast, and perform matching calibration calculations on the polarization images to obtain polarization degree and polarization angle images. Step S2: Annotate the polarization degree and polarization angle images of the training set and validation set. The annotations include the tap end, core, blade back, cutting tooth features, as well as wear and defect areas. Use the model pre-trained on ImageNet as a basis to train the Mask R-CNN model. By evaluating the model's accuracy, recall rate, and F1 score on the validation set, select the ideal weight file and obtain the neural network model. Step S3: Use the Mask R-CNN segmentation model to detect defects and wear on the polarized images of the straight groove taps in the test set, and segment key features such as the end, core, blade back, and cutting teeth. Step S4: Use image processing technology to extract the mask of the segmented image, and generate a classification probability map for each pixel. Use the segmented mask to perform calibration measurement and calculate the size, including the end diameter, core diameter, blade back width, number of cutting teeth, radial back of the cutting cone, back angle and front angle. Step S5: Use standard size parameter products to verify the measurement results. Finally, compare the test size parameters with the standard size parameters, select unqualified products and classify them into different models.
2. The detection system according to claim 1, characterized in that: The transfer learning strategy is adopted to train the model based on the pre-trained model on the large dataset ImageNet. New structures are added to the training model, including: adding a custom loss function based on FocalLoss, using a dynamically adjusted factor that is automatically adjusted according to the frequency of the target category to increase the model's sensitivity to rare defects. By reducing the weight of easy-to-classify samples and increasing the weight of difficult-to-classify samples, the model pays more attention to these key defects.
3. The detection system according to claim 1, characterized in that: The model is trained based on the model pre-trained on the large dataset ImageNet using a transfer learning strategy. New structures are added to the pre-trained model, including: adding an attention mechanism to make the model pay more attention to key areas of the tap surface, such as the blade back surface and cutting tooth surface, to improve the accuracy of defect detection. A DANet attention module is added after each convolutional layer, which learns the importance weights of each channel through global average pooling and fully connected layers, and then applies these weights to the input feature map.
4. The segmentation method according to claim 1, characterized in that: The Mask R-CNN segmentation model is used to detect defects and wear in the polarized images of the straight groove taps in the test set, and to segment key features such as the end, core, blade back, and cutting teeth. A new structure is introduced, including: optimizing the segmentation process through geometric prior knowledge. In the mask prediction branch, the geometric parameters of the tap (such as straight groove width, end diameter, core diameter, blade back width, etc.) are combined to guide the segmentation network through custom geometric constraints. These geometric prior knowledge helps the model to more accurately follow the actual shape and size restrictions of the tap during segmentation. Reasonable size restrictions are set to guide the model for more accurate segmentation and avoid generating small or irregular areas that do not conform to the actual size of the tap. Finally, the segmentation results are optimized by removing small areas and merging adjacent areas.
5. The detection method according to claim 1, characterized in that: The mask is extracted from the segmented image, and a classification probability map is generated for each pixel. The segmented mask is used for calibration measurement and size calculation, including end diameter, core diameter, blade width, number of cutting teeth, radial back of cutting cone, back angle and front angle. Including: Converting pixel size to actual physical size requires calibration of the system. Place a calibration plate in the image. Extract the mask in the image and generate a classification probability map for each pixel. By comparing the pixel size of the calibration object in the image with the actual size, calculate the conversion ratio between the pixel and the actual physical size. Then, apply this ratio to convert the segmented mask pixel size to the actual physical size. Finally, use known calibration data to verify the measurement results to ensure that the actual sizes of all measured parameters (such as end diameter, core diameter, etc.) are consistent with the sizes calculated in the image.
6. The detection method according to claim 1, characterized in that: The segmented mask is used for calibration measurement and size calculation, including the end diameter, core diameter, blade width, number of cutting teeth, radial relief of the cutting cone, back angle and front angle. The specific measurement method includes: by identifying the mask of the end and the core, the end diameter and the core diameter can be determined by the minimum circumscribed circle. The blade width is determined by identifying the mask of the blade and measuring the boundary of the mask area. The number of teeth can be calculated by analyzing the mask of the cutting teeth. By identifying specific points in the mask, the back angle and front angle are determined by measuring the shortest distance between two edges and the angle between them.
7. The detection method according to claim 1, characterized in that: The use of polarization cameras combined with deep learning technology to inspect straight groove taps has several characteristics compared to other traditional measurement methods. These include: polarization imaging can reveal the optical properties of the surface of an object, which are often difficult to capture with traditional imaging technology. By analyzing the polarization state of light, this technology can significantly improve the contrast of the image, allowing the surface defects and wear of the tap to be clearly displayed. Through deep learning, a powerful image processing framework is built to automatically identify and quantify the wear state and various characteristics of the tap, thereby achieving non-destructive, fast, and accurate detection.