A method for detecting and grading defects on the inner wall surface of a part

Abstract

The present invention discloses a method for detecting and grading defects on the inner wall surface of a part, including: collecting a video or image of the inner wall surface of a part, and marking defect features offline; constructing a defect detection model for the inner wall surface of a part, optimizing the model feature extraction and fusion network, improving the model's ability to learn small defect features while maintaining the detection speed, improving the model's bounding box loss function, accelerating model convergence, and improving the bounding box regression accuracy; setting hyperparameters in the defect detection model, adjusting the hyperparameters and training to obtain a corresponding model, selecting the best defect detection model from the training model to detect the defects in the collected video or image, and outputting defect confidence, bounding box and mask; calculating defect area and equivalent diameter rating parameters based on the defect mask; and rating the inner wall surface defects of the part by the equivalent diameter value in the rating parameter. The present invention has great practical engineering application value in reducing the workload of inner wall surface defect detection of parts and improving the efficiency and accuracy of defect detection.

Description

Technical Field

[0001] The present invention relates to the technical field of nondestructive testing, and in particular to a method for detecting and grading defects on the inner wall surface of a part based on machine vision. Background Art

[0002] Anti-lock Brake Systems (ABS) is one of the most important safety devices for automobiles. The quality of the inner wall surface of its parts directly affects the assembly accuracy and performance. At present, the quality inspection of the inner wall surface of parts relies on manual visual inspection, but factors such as long-term eye fatigue of inspectors and subjective inconsistency of judgment standards lead to low defect recognition rate and poor real-time inspection. Therefore, the use of this method to detect defects inside parts is of great significance to ensure product quality and process improvement.

[0003] In the prior art of the method for detecting and evaluating the inner wall surface defects of parts, there are several comparative patents and documents as follows:

[0004] Patent CN 112102278 A discloses a method for detecting defects on the machined surface of metal workpieces based on computer vision. This method uses traditional image processing methods such as binarization, Hough circle detection, and morphological processing to detect defects such as sand holes, scratches, and bump scars on the metal surface. However, this method has high requirements for imaging quality and low anti-interference ability, and only classifies three structural defects based on the defect area, and does not have a defect rating function.

[0005] Patent 202310159962.0 proposes a multi-order feature interactive target detection model and its construction method, device and application. This method uses a spatial aggregation module in the YOLOv8 head network to better capture intermediate-order features, but it is difficult to learn the deep feature information of tiny targets such as trachoma based only on intermediate-order features, and it is difficult to ensure recognition accuracy.

[0006] Patent CN 115953386 A discloses a lightweight gear surface defect detection method based on MSTA-YOLOv5. The method features a backbone network that uses the ShuffleNetv2 architecture and a neck that uses FPN+PAN, which has the advantage of being lightweight, but with the increase in lightweight, its accuracy is greatly lost and its practical application is limited.

[0007] Patent CN115700741 A discloses a method for detecting fabric defects based on an improved YOLOv5. This method adds the SE module to Backbone and replaces the Leaky ReLU function with the ActivateOrNot activation function to improve the model accuracy, but the amount of calculation is large and some target bounding boxes are prone to overlap.

[0008] Jin Ying and others from China University of Metrology[1] The article "Surface Defect Detection of the Inner Groove of the Plunger-Type Brake Master Cylinder Based on LabVIEW and Mask R-CNN" was published in the 5th issue of "Modern Manufacturing Engineering" in 2020. The article uses Mask R-CNN to detect sand holes, scratches, and vibration marks on the surface of the inner groove of the automobile plunger-type brake master cylinder. However, the defect recognition rate of this method is only 93.6%, and no specific parameters such as defect location, area, and equivalent diameter are output, which limits its practical application. [1] Jin Ying, Wang Xueying, Duan Linmao. Surface Defect Detection of the Inner Groove of the Plunger-Type Brake Master Cylinder Based on LabVIEW and Mask R-CNN [J]. Modern Manufacturing Engineering, 2020, 476(5): 125. Summary of the invention

[0009] In order to solve the above technical problems, the present invention proposes a method for detecting and grading defects on the inner wall surface of parts. The method constructs a self-attention feature extraction network and a feature fusion network through a dynamic sparse attention mechanism BiFormer, adopts the minimum point distance intersection-over-union ratio MPDIoU to optimize the model bounding box loss function, completes the construction of the BM-YOLOv8 part inner wall surface defect detection model and accurate defect detection, and completes the defect grade assessment according to the assessment method.

[0010] The purpose of the present invention is achieved through the following technical solutions:

[0011] A method for detecting and grading defects on the inner wall surface of a part, comprising:

[0012] Step A collects video or images of the inner wall surface of the part and annotates defect features offline;

[0013] Step B builds a defect detection model for the inner wall surface of the part, optimizes the model feature extraction network and feature fusion network, improves the model's ability to learn small defect features while maintaining the detection speed, improves the model's bounding box loss function, accelerates model convergence, and improves bounding box regression accuracy;

[0014] Step C sets hyperparameters in the defect detection model, adjusts the hyperparameters and trains to obtain the corresponding model, selects the best defect detection model from the trained model to detect the defects in the collected video or image, and outputs the defect confidence, bounding box and mask;

[0015] Step D calculates the defect area S and equivalent diameter D based on the defect mask e Rating parameters;

[0016] Step E: Equivalent diameter D in the rating parameter e The value is used to rate the surface defects on the inner wall of the part.

[0017] Compared with the prior art, one or more embodiments of the present invention may have the following advantages:

[0018] The present invention can detect and identify the surface defects of the inner wall of parts based on video or image detection, and grade the degree of the defects, which can be effectively applied to the detection and identification of the surface defects of the inner wall of parts. The method has great practical engineering application value in reducing the workload of the surface defects detection of the inner wall of parts and improving the efficiency and accuracy of defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a flow chart of the method for detecting and grading defects on the inner wall surface of parts;

[0020] Figure 2a This is the YOLOv8 model structure diagram

[0021] Figure 2b This is the BM-YOLOv8 model structure diagram;

[0022] Figure 3a It is a schematic diagram of bounding box loss calculation parameters;

[0023] Figure 3b This is a schematic diagram of the MPDIoU principle;

[0024] Figure 4 This is the result diagram of the defect detection on the inner wall of the part. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be further described in detail below in conjunction with embodiments and drawings.

[0026] like Figure 1 The figure shows the method for detecting and grading defects on the inner wall surface of a part, including:

[0027] Step 10: Collect video or image of the inner wall surface of the part and mark defect features offline;

[0028] Step 20 builds a part inner wall surface defect detection model, optimizes the model feature extraction network and feature fusion network, improves the model's ability to learn small defect features while maintaining the detection speed, improves the model's bounding box loss function, accelerates model convergence, and improves bounding box regression accuracy;

[0029] Step 30: setting hyperparameters in the defect detection model, adjusting the hyperparameters and training to obtain the corresponding model, selecting the best defect detection model from the trained model to detect defects in the collected video or image, and outputting defect confidence, bounding box and mask;

[0030] Step 40 calculates the defect area S and equivalent diameter D based on the defect mask e Rating parameters;

[0031] Step 50: Equivalent diameter D in the rating parametere The value is used to rate the surface defects on the inner wall of the part.

[0032] The above step 10 specifically includes:

[0033] The video or image of the inner wall surface of the part is collected by a CCD camera, and the defect features of sand holes and long scratches are annotated offline using Labelme open source software.

[0034] The above step 20 specifically includes:

[0035] Construct a part inner wall surface defect detection model based on BM-YOLOv8. Figure 2b As shown, the BM-YOLOv8 model uses YOLOv8 as the basic model (such as Figure 2a As shown in the figure, the dynamic sparse self-attention mechanism BiFomer is used to optimize the feature extraction network and feature fusion network of the YOLOv8 model, which improves the network's ability to learn small defect features while maintaining the detection speed. The bounding box loss function of the YOLOv8 model is optimized based on the minimum point distance intersection-over-union ratio MPDIoU, which improves the regression performance of the bounding box when the width and height values ​​of the real and predicted boxes are different, accelerates the model convergence, and completes the accurate detection of defects on the inner and outer surfaces of parts.

[0036] In the feature extraction network, the BiFormer module is added to the last layer of Backbone, so that the network can adaptively adjust the attention weight according to the input image features, flexibly control the intensity and range of attention, and enhance the network's ability to express tiny features. In the feature fusion network, the BiFormer module is added before the Upsample structure in the PAN-FPN upsampling stage and after the C2f module in the downsampling stage, so that the model can maintain high-quality perception of deep discriminative features and high-level semantic information in the feature fusion stage.

[0037] The BiFomer module includes three parts: image input region division and projection, region-to-region routing of directed graphs, and Token-to-token attention mechanism; the details are as follows:

[0038] ① Image input area division and projection

[0039] Let H be the height of the input image, W be the width of the input image, and α be the number of channels of the input image. Then the two-dimensional feature map X of the input image is:

[0040] X∈R H×W×α (1)

[0041] After dividing into S×S blocks of non-overlapping areas, each area contains HW / S 2 feature vectors, then after linear projection of the input image, the query tensor Q, key tensor K, and value tensor V can be expressed as:

[0042]

[0043] Where W q ,W k ,W v They are query, key, and value projection weights respectively;

[0044] ②Region-to-region routing with intention graph

[0045] Inter-regional correlation matrix A r The solution is:

[0046] A r =Q r (K r ) T (3)

[0047] In the formula, Q r is the region-level query tensor, K r is the region-level key tensor;

[0048] According to formula (3), we can solve the inter-region index routing matrix M: r for:

[0049] M r =topkIndex(A r ) (4)

[0050] Where topkIndex(·) is the index function of the topk region;

[0051] ③Token-to-token attention mechanism

[0052] The key tensor K of the clustered region g for:

[0053] K g =gather(K,M r ) (5)

[0054] Similarly, the value tensor V of the aggregated area g for:

[0055] V g =gather(V,M r ) (6)

[0056] According to equations (5) and (6), the dynamic sparse attention mechanism can be solved as follows:

[0057]

[0058] Where, d kis the key dimension, and LE(V) is the multi-scale token local enhancement component obtained by performing a deep convolution on V (the convolution kernel is 5×5).

[0059] Considering the different shapes and angles of surface defects on the inner wall of parts, and the bounding box regression loss function of the YOLOv8 model It is difficult to optimize when the target prediction box and the real box have the same aspect ratio but different height and width values. Therefore, based on the minimum point distance intersection-over-union (MPDIoU), the distance between the upper left and lower right points of the prediction box and the real box is minimized to optimize the bounding box loss function. The specific steps are as follows:

[0060] By geometric analysis Figure 3a It can be seen that the existing bounding box regression loss function All calculation parameters, such as: minimum enclosing rectangle area |A|, coordinates of the center point of the real frame Width gt , height h gt And the coordinates of the center point of the prediction box Width prd , height h prd It can be determined by the coordinates of the four vertices in the upper left and lower right of the predicted box and the real box, that is, MPDIoU contains The superior performance of Figure 3b As shown, is the prediction box, is the real frame, The real frame The upper left and lower right vertices of The prediction boxes are The upper left and lower right vertices, then the minimum circumscribed rectangle area |A|, the coordinates of the center point of the real box Width gt , height h gt And the coordinates of the center point of the prediction box Width prd , height h prd The solution is:

[0061]

[0062]

[0063]

[0064] Furthermore, the prediction box Real frame The minimum point distance intersection and union ratio MPDIoU between the upper left and lower right vertices is expressed as:

[0065]

[0066] In the formula,

[0067] further for:

[0068]

[0069] Optimized model bounding box loss function for:

[0070]

[0071] In the formula, is the DFL loss function.

[0072] The above step 30 specifically includes:

[0073] The hyperparameters that are set or adjusted multiple times in the defect detection model include: initial weight, weight decay coefficient, learning rate, batch size, activation function, and number of units;

[0074] The performance evaluation indicators of the defect detection model are: mAP, F1 score, recognition rate and FPS;

[0075] According to the performance evaluation index of the defect detection model, the best model is selected from multiple training models to detect the collected video or image, and the confidence, bounding box and mask of sand holes and long scratches are visually output.

[0076] The above step 40 specifically includes:

[0077] Calculate defect area S and equivalent diameter D based on defect mask e Rating parameters; specifically include:

[0078] Let u be the horizontal pixel coordinate of the image, v be the vertical pixel coordinate of the image, and the defect area S of sand holes and long scratches is:

[0079]

[0080] Equivalent diameter D e for:

[0081]

[0082] Combined with the spatial posture conversion relationship between the imaging plane and the object plane, the physical unit value of the rating parameter is calculated.

[0083] The above step 50 specifically includes:

[0084] If the equivalent diameter D in the rating parameter eIf the formula (16) is satisfied, it is evaluated as a sand hole defect, a level 1 defect;

[0085] 0.4mm≤D e <0.6mm (16)

[0086] If the equivalent diameter D in the rating parameter e If the formula (17) is satisfied, it is evaluated as a sand hole defect, a level 2 defect;

[0087] 0.6mm≤D e <1.0mm (17)

[0088] If the equivalent diameter D in the rating parameter e If the formula (18) is satisfied, it is evaluated as a sand hole defect, a level 3 defect;

[0089] 1.0mm≤D e <1.5mm (18)

[0090] If the equivalent diameter D in the rating parameter e If formula (19) is satisfied, it is evaluated as a long scratch defect, a level 4 defect;

[0091] D e ≥1.5mm (19).

[0092] [Example 1]

[0093] Figure 4 This is the result of the defect detection on the inner wall surface of the part by the present invention. A CCD camera is used to collect the video or image of the inner wall surface of the part, and the sand hole and long scratch defect features are annotated offline through the Labelme open source software; a BM-YOLOv8 part inner wall surface defect detection model is constructed to detect sand holes and long scratch defects in the collected video or image, and output defect confidence, bounding box, and mask; the defect rating parameter is calculated, and the defect area S = 0.846mm 2 , equivalent diameter D e =0.955mm; since 0.6mm≤D e <1.0mm can be rated as Grade 2 sand hole defect.

[0094] Although the embodiments disclosed in the present invention are as above, the above contents are only embodiments adopted for facilitating the understanding of the present invention and are not intended to limit the present invention. Any technician in the technical field to which the present invention belongs can make any modifications and changes in the form and details of the implementation without departing from the spirit and scope disclosed in the present invention, but the patent protection scope of the present invention shall still be subject to the scope defined in the attached claims.

Claims

1. A method for detecting and grading defects on the inner wall surface of a part, characterized in that: The method comprises: Step A collects video or images of the inner wall surface of the part and annotates defect features offline; Step B builds a defect detection model for the inner wall surface of the part, optimizes the model feature extraction network and feature fusion network, improves the model's ability to learn small defect features while maintaining the detection speed, improves the model's bounding box loss function, accelerates model convergence, and improves bounding box regression accuracy; Step C sets hyperparameters in the defect detection model, adjusts the hyperparameters and trains to obtain the corresponding model, selects the best defect detection model from the trained model to detect the defects in the collected video or image, and outputs the defect confidence, bounding box and mask; Step D calculates the defect area S and equivalent diameter D based on the defect mask e Rating parameters; Step E: Equivalent diameter D in the rating parameter e The value is used to rate the surface defects on the inner wall of the part; The step B specifically comprises: The part inner wall surface defect detection model is a BM-YOLOv8 model. The BM-YOLOv8 model uses YOLOv8 as the basic model, optimizes the feature extraction network and feature fusion network of the YOLOv8 model through the dynamic sparse self-attention mechanism BiFormer, and optimizes the YOLOv8 model bounding box loss function based on the minimum point distance intersection and union ratio MPDIoU to complete the accurate detection of defects on the inner and outer surfaces of parts; The BM-YOLOv8 model feature extraction network includes adding the BiFormer module to the last layer of Backbone, so that the network can adaptively adjust the attention weight according to the input image features; the BM-YOLOv8 model feature fusion network includes: adding the BiFormer module before the Upsample structure in the PAN-FPN upsampling stage and after the C2f module in the downsampling stage; The BiFormer module consists of three parts: image input region division and projection, region-to-region routing of directed graphs, and token-to-token attention. Specifically, it includes: ① Image input area division and projection Let H be the height of the input image, W be the width of the input image, and α be the number of channels of the input image. Then the two-dimensional feature map X of the input image is: X∈R H×W×α (1) After dividing into S×S blocks of non-overlapping areas, each area contains HW / S 2 feature vectors, then after linear projection of the input image, the query tensor Q, key tensor K, and value tensor V can be expressed as: Where W q ,W k ,W v They are query, key, and value projection weights respectively; ②Region-to-region routing with intention graph Inter-regional correlation matrix A r The solution is: A r =Q r (K r ) T (3) In the formula, Q r is the region-level query tensor, K r is the region-level key tensor; According to formula (3), we can solve the inter-region index routing matrix M: r for: M r =topkIndex(A r ) (4) Where topkIndex(·) is the index function of the topk region; ③Token-to-token attention mechanism The key tensor K of the clustered region g for: K g =gather(K,M r ) (5) Similarly, the value tensor V of the aggregated area g for: V g =gather(V,M r ) (6) According to equations (5) and (6), the dynamic sparse attention mechanism can be solved as follows: Where, d k is the key dimension, LE(V) is the multi-scale token local enhancement component obtained by deep convolution of V; The bounding box loss function is optimized by minimizing the distance between the upper left and lower right points of the predicted box and the true box using the minimum point distance intersection and union ratio MPDIoU. The specific steps are as follows: set up is the prediction box, is the real frame, The real frame The upper left and lower right vertices of The prediction boxes are The upper left and lower right vertices of the prediction box Real frame The MPDIoU of the minimum distance between the upper left and lower right vertices is expressed as: In the formula, further for: Optimized model bounding box loss function for: In the formula, is the DFL loss function; The hyperparameters that are set or adjusted multiple times in the defect detection model include: initial weight, weight decay coefficient, learning rate, batch size, activation function, and number of units; The performance evaluation indicators of the defect detection model are: mAP, F1 score, recognition rate and FPS; According to the performance evaluation index of the defect detection model, the best model is selected from multiple training models to detect the collected video or image, and the confidence level, bounding box and mask of sand holes and long scratches are visually output; The step D specifically comprises: Calculate defect area S and equivalent diameter D based on the mask where the defect is identified e Rating parameters; specifically include: Let u be the horizontal pixel coordinate of the image, v be the vertical pixel coordinate of the image, and the defect area S is: Defect equivalent diameter D e for: Combined with the spatial posture conversion relationship between the imaging plane and the object plane, the physical unit value of the defect rating parameter is calculated; The step E specifically comprises: If the equivalent diameter D in the rating parameter e If the formula (9) is satisfied, it is evaluated as a sand hole defect, a level 1 defect; 0.4mm≤D e <0.6mm (13) If the equivalent diameter D in the rating parameter e If the formula (10) is satisfied, it is evaluated as a sand hole defect, a level 2 defect; 0.6mm≤D e <1.0mm (14) If the equivalent diameter D in the rating parameter e If the formula (11) is satisfied, it is evaluated as a sand hole defect, a level 3 defect; 1.0mm≤D e <1.5mm (15) If the equivalent diameter D in the rating parameter e If formula (12) is satisfied, it is evaluated as a long scratch defect, a level 4 defect; D e ≥1.5mm (16) 2. The method for detecting and grading inner wall surface defects of a part according to claim 1, characterized in that: In the step A, a CCD camera is used to collect a video or image of the interior of the part; and Labelme open source software is used to mark the defect features of sand holes and long scratches offline.

3. The method for detecting and grading inner wall surface defects of a part according to claim 1, characterized in that: The BM-YOLOv8 model loss function includes: optimizing the bounding box loss function by minimizing the distance between the upper left and lower right points of the predicted box and the true box using the minimum point distance intersection-over-union ratio MPDIoU.

Images