Improvement method of target detection model, detection method and intelligent welding equipment

The Lite-Deformable-DETR model addresses the challenge of balancing speed and accuracy in weld defect detection by using depth-wise separable convolution and a simplified encoder, enhancing detection efficiency and precision.

CN119091274APending Publication Date: 2024-12-06WUHU XINGJIAN INTELLIGENT ROBOT CO LTD +1
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202411127804.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-16
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

Existing weld defect detection methods face challenges in balancing detection speed and accuracy, particularly in industrial settings where computational load affects detection efficiency and accuracy.

Method used

The method employs a Lite-Deformable-DETR target detection model enhanced with depth-wise separable convolution, inverse residual structure, single-layer encoder, and attention modules to reduce computational burden while maintaining accuracy.

Benefits of technology

The improved model achieves efficient and accurate weld defect detection by reducing computational requirements and model complexity, enabling high-speed and precise defect identification.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119091274A_ABST
    Figure CN119091274A_ABST
Patent Text Reader

Abstract

The invention provides an improvement method of a target detection model, a detection method and intelligent welding equipment, and relates to the technical field of automatic detection of weld joints. The improved method comprises the following steps: replacing standard convolution in an original model with depth separable convolution; replacing a residual structure in the original model with an inverse residual structure, and setting a linear activation function and a bottleneck structure in the inverse residual structure; the encoder is set to be a single-layer encoder. In this way, through mutual combination of the depth separable convolution, the inverse residual structure and the single-layer encoder, the intelligent welding equipment can give consideration to the detection speed and the detection precision of the weld joint at the same time.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of automatic weld detection, and in particular to an improved method for a target detection model, a detection method and intelligent welding equipment. Background Art

[0002] As the cornerstone of social development and production, welding is widely used in various fields. Welding quality directly affects the performance, safety and market competitiveness of products. However, in the actual production process, due to differences in various working conditions, defects may appear in the welds, which may have an adverse effect on the quality and performance of industrial products. In order to ensure product quality and stable production, weld defect detection has become a key technology, and traditionally, manual visual inspection methods are mainly used. However, this method has the problems of low detection efficiency, high false detection rate and missed detection rate, and high labor intensity, which makes it difficult to meet the detection needs of welds.

[0003] On the other hand, with the widespread use of intelligent robots in the field of welding, intelligent welding robots are widely used in all steps of the welding process. At the same time, with the combination of visual recognition technology and intelligent robots, it has become a reality to combine intelligent robots and visual recognition to detect welds.

[0004] In addition, defect detection algorithms based on machine vision usually include multiple different steps, such as image preprocessing, image segmentation, feature extraction, and classification or quantification based on feature vectors. Each step is very critical, and the design of each step affects the final detection effect. Different machine vision inspection tasks require the above traditional machine vision inspection algorithm steps to be individually designed according to the application scenario and task objectives.

[0005] Compared with the traditional image processing process, the deep learning network relies more on the self-learning of the machine, and obtains the parts that need to be learned or paid attention to through the artificially designed feature processing structure. In weld defect detection, determining the type of defect is only the first step. After that, it is necessary to determine the location and coordinates of the defect, which requires further processing by the algorithm. In this regard, most of the existing technologies have improved the general detection framework to suit the detection of industrial data.

[0006] At the same time, the computational load of the front-end hardware has always been a problem. To ensure the detection accuracy of the weld, a certain amount of computing must be guaranteed. To reduce the computational load, the amount of computing must be reduced, which in turn affects the detection accuracy of the weld. Therefore, how to balance the detection speed and detection accuracy of the weld has become a major technical problem. Summary of the invention

[0007] The technical problem to be solved by the present invention is how to enable intelligent welding equipment to take into account both the detection speed and the detection accuracy of the weld.

[0008] To solve the above problems, the present invention provides a method for improving a target detection model, wherein the target detection model is used to detect surface defects of welds, and the target detection model is a Lite-Deformable-DETR target detection model. The improved method comprises:

[0009] Use depth-wise separable convolution to replace the standard convolution in the original model;

[0010] An inverse residual structure is used to replace the residual structure in the original model, and a linear activation function and a bottleneck structure are set in the inverse residual structure;

[0011] Set the encoder to a single-layer encoder.

[0012] Optionally, using depthwise separable convolution to replace the standard convolution in the original model includes:

[0013] Convolving the depthwise convolution with the pointwise convolution into the depthwise separable convolution;

[0014] Setting the depthwise convolution to process spatial information in length and width directions;

[0015] The cross-correlation operation of multi-channel features is utilized, and the spatial information across channels is processed using the point-by-point convolution.

[0016] Optionally, multiple stacked lightweight attention layers and feedforward neural networks are set in the single-layer encoder.

[0017] Optionally, an attention module and a multi-size variable attention module are set in the decoder of the target detection model,

[0018] Make the attention module interact between targets,

[0019] The multi-scale variable attention module is enabled to extract features from the output of the encoder.

[0020] Optionally, setting the encoder to a single-layer encoder includes:

[0021] Based on the convolution kernel size, number of input channels, and number of convolution kernel groups of the original model, the parameter reduction rate of the improved model relative to the original model is obtained.

[0022] Optionally, based on the convolution kernel size, the number of input channels, and the number of convolution kernel groups of the original model, the parameter reduction rate of the improved model relative to the original model is obtained, including:

[0023] Using the first formula (1): [(D k ·Dk ·M)·(N·D F ·D F )] (1) Calculate the computational effort of the standard convolution;

[0024] Among them, D k is the convolution kernel size, M is the number of input channels, N is the number of convolution kernel groups, and D F is the output image feature size, (D k ·D k ·M) is the multiplication amount of a convolution, (N·D F ·D F ) is the number of elements of the output feature map;

[0025] Using the third formula (3): [(D k ·D k )·(M·D F ·D F )+M·(N·D F ·D F )] (3) Calculate the computational effort required to obtain the depthwise separable convolution;

[0026] Where: (D k ·D k )·(M·D F ·D F ) is the computational complexity of the depthwise convolution; M·(N·D F ·D F ) is the amount of point-by-point convolution calculation;

[0027] Based on the first formula (1) and the third formula (3), the parameter reduction rate of the improved model relative to the original model is calculated.

[0028] Alternatively, the first formula (1) is used: [(D k ·D k ·M)·(N·D F ·D F )] (1) After calculating the amount of computation required for the standard convolution, the following are included:

[0029] Using the second formula (2): [D k ·D k ·M·N] (2) Calculate the parameters of the standard convolution;

[0030] Using the third formula (3): [(D k ·D k )·(M·D F ·D F )+M·(N·D F ·D F )] (3) The computational effort of obtaining the depthwise separable convolution includes:

[0031] Using the fourth formula (4): [D k ·D k ·M+1·1·M·N] (4) Calculate the parameters of the depthwise separable convolution;

[0032] Introducing the channel number factor α and the output feature scaling factor ρ into the third formula (3), the fifth formula (5) of the computational amount of the depthwise separable convolution is obtained:

[0033] (D k ·D k )·(αM·ρD F ·ρD F )+αM·(αN·ρD F ·ρD F ) (5);

[0034] Comparing the first formula (1) with the third formula (3), a sixth formula (6) is obtained:

[0035]

[0036] The sixth formula (6) is simplified to the seventh formula (7):

[0037]

[0038] in, is the parameter reduction rate of the improved model relative to the original model.

[0039] In addition, the present invention also provides a method for detecting weld surface defects based on a target detection model, wherein the target detection model is obtained after being improved by the improvement method, and the detection method comprises:

[0040] The collected weld surface image information is input into the neck inverse residual structure with depthwise separable convolution to obtain a multi-scale feature map with spatial information;

[0041] Inputting the multi-scale feature map with spatial information into a single-layer encoder to obtain a feature vector with position and context information;

[0042] Input the feature vector with position and context information into a decoder to generate a predicted result;

[0043] Based on the predicted results, a mating ratio between the predicted results and the real information is calculated;

[0044] Classification and regression prediction are performed based on the mating ratio, and the detection result is obtained by calculation.

[0045] In addition, the present invention also provides an intelligent welding device, which is used to execute the detection method.

[0046] The technical effects of the present invention include at least:

[0047] In the present invention, depth-separable convolution is used to replace standard convolution, and the combination of depth-separable convolution and point-by-point convolution can effectively reduce the computational burden. By adjusting the number of channels and output feature scaling, the size of the model and the computational requirements are greatly reduced. At the same time, in order to ensure the accuracy of detection, the inverse residual structure is used to replace the conventional residual structure. The inverse residual structure adds a linear activation function and a bottleneck structure, which helps to improve the accuracy and detail capture ability of the model. On this basis, in order to further improve efficiency, the encoder of the original model is simplified, only one layer is retained, and the depth-separable convolution is combined with the inverse residual structure, aiming to reduce the model complexity and the number of parameters, thereby improving the model efficiency and speed. In this way, through the combination of depth-separable convolution, inverse residual structure and single-layer encoder, it is achieved that the intelligent welding equipment can take into account both the detection speed and detection accuracy of the weld. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A schematic flow chart of a method for improving the target detection model according to a specific embodiment of the present invention;

[0049] Figure 2 is a schematic flow chart of the working process of the target detection model after being improved by the improvement method according to the specific implementation mode of the present invention;

[0050] Figure 3 is a schematic structural diagram of the target detection model after being improved by the improvement method according to the specific implementation mode of the present invention;

[0051] Figure 4 A schematic diagram of a depth-separable convolution process in the target detection model according to a specific embodiment of the present invention;

[0052] Figure 5 A schematic diagram of a standard convolution process in the original model according to a specific embodiment of the present invention;

[0053] Figure 6 is a schematic diagram of the inverse residual structure of a specific implementation mode of the present invention;

[0054] Figure 7 A schematic diagram of an MPI data set according to a specific embodiment of the present invention;

[0055] Figure 8 A schematic diagram of a NEU-DET steel defect dataset according to a specific embodiment of the present invention;

[0056] Fig. 9 A schematic diagram of a GC-10 industrial defect data set according to a specific embodiment of the present invention;

[0057] Fig.10 It is a schematic diagram of the multi-model prediction results of a specific embodiment of the present invention. DETAILED DESCRIPTION

[0058] In order to make the above-mentioned purpose, features and advantages of the present invention more obvious and easy to understand, the specific implementation mode of the present invention is described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the embodiments of the present invention. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. The embodiments of the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed below.

[0059] Unless otherwise defined, all technical and scientific terms used in the present invention have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0060] See also Figures 1 to 10 This embodiment provides a method for improving a target detection model, wherein the target detection model is used to detect surface defects of a weld, and the target detection model is a Lite-Deformable-DETR target detection model. The improved method includes:

[0061] Use depth-wise separable convolution to replace the standard convolution in the original model;

[0062] An inverse residual structure is used to replace the residual structure in the original model, and a linear activation function and a bottleneck structure are set in the inverse residual structure;

[0063] Set the encoder to a single-layer encoder.

[0064] In the present invention, depth-separable convolution is used to replace standard convolution, and the combination of depth-separable convolution and point-by-point convolution can effectively reduce the computational burden. By adjusting the number of channels and output feature scaling, the size of the model and the computational requirements are greatly reduced. At the same time, in order to ensure the accuracy of detection, the inverse residual structure is used to replace the conventional residual structure. The inverse residual structure adds a linear activation function and a bottleneck structure, which helps to improve the accuracy and detail capture ability of the model. On this basis, in order to further improve efficiency, the encoder of the original model is simplified, only one layer is retained, and the depth-separable convolution is combined with the inverse residual structure, aiming to reduce the model complexity and the number of parameters, thereby improving the model efficiency and speed. In this way, through the combination of depth-separable convolution, inverse residual structure and single-layer encoder, it is achieved that the intelligent welding equipment can take into account both the detection speed and detection accuracy of the weld.

[0065] In addition, the detection process of the intelligent welding equipment using the target detection model in this embodiment can be: through the image acquisition device of the intelligent welding equipment, such as a CCD camera, the surface image of the weld to be detected is collected to obtain the detection sample image; the detection sample image is imported into the Lite-Deformable-DETR model stored in the storage medium of the industrial computer, and the target detection model in this embodiment performs feature extraction according to the trained network structure weight. Finally, the location coordinates and type information of the defect are obtained through the prediction layer.

[0066] Further, see Figures 1 to 10 , using depthwise separable convolution to replace the standard convolution in the original model includes:

[0067] Convolving the depthwise convolution with the pointwise convolution into the depthwise separable convolution;

[0068] Setting the depthwise convolution to process spatial information in length and width directions;

[0069] The cross-correlation operation of multi-channel features is utilized, and the spatial information across channels is processed using the point-by-point convolution.

[0070] Further, see Figures 1 to 10 , multiple stacked lightweight attention layers and feedforward neural networks are set in the single-layer encoder.

[0071] Further, see Figures 1 to 10 , an attention module and a multi-size variable attention module are set in the decoder of the target detection model,

[0072] Make the attention module interact between targets,

[0073] The multi-scale variable attention module is enabled to extract features from the output of the encoder.

[0074] Combining the above characteristics, it can be seen that, on the one hand, the present invention uses depthwise separable convolution to replace the standard convolution in the original model. This special convolution method used by depthwise convolution can process spatial information in the length and width directions; point-by-point convolution can process cross-channel spatial information through the cross-correlation operation of multi-channel features. The composite convolution of the two, namely depthwise separable convolution, can effectively utilize multi-scale spatial information while reducing the amount of calculation.

[0075] In addition, the present invention builds an inverse residual feature extraction network to replace the original feature extraction network. The original model uses the standard Resnet50 as the feature extraction network. The network has a strong feature extraction capability. However, for weld crack features, its learning and expression capabilities are too strong, which will lead to overfitting. The prediction results of the original model of the prior art are shown in Table 1.

[0076] In Table 1, the training results of the sub-models at different stages of the original feature extraction network show that for target detection of magnetic particle inspection, its feature learning ability will easily lead to overfitting, and the training accuracy has not improved. It can be considered that in the deep convolution kernel, most of the convolution kernel parameters are 0, while the number of parameters and the amount of calculation have been improved by one stage.

[0077] In addition, the inverse residual structure designed by the present invention will upgrade the dimension of the image processing features to generate more accurate and high-quality output. Depthwise separable convolution will reduce the quality of input image features due to its lightweight processing, while the inverse residual structure is usually used in situations where the model needs to actively intervene in the input data to improve the output quality. The two can complement each other well.

[0078] In addition, a single-layer encoder is used in the present invention to replace the 6-layer encoder in the original model. In the multi-layer encoder structure in the original model, each layer relies on the output of the previous layer for feature extraction and processing. As shown in Table 2, based on the training results of the encoder layer of the original model of the prior art, this stacking method will bring about the problem of high computational complexity and does not bring about an improvement in accuracy. Reducing the encoder layer to one layer can simplify the model structure, reduce computational overhead, and avoid the loss of information in the multi-layer transmission process, thereby achieving a significant improvement in efficiency while also ensuring the accuracy of the data.

[0079] Further, see Figures 1 to 10 , setting the encoder to a single-layer encoder includes:

[0080] Based on the convolution kernel size, number of input channels, and number of convolution kernel groups of the original model, the parameter reduction rate of the improved model relative to the original model is obtained.

[0081] Further, see Figures 1 to 10Based on the convolution kernel size, number of input channels and number of convolution kernel groups of the original model, the parameter reduction rate of the improved model relative to the original model is obtained:

[0082] Using the first formula (1): [(D k ·D k ·M)·(N·D F ·D F )] (1) Calculate the computational effort of the standard convolution;

[0083] Among them, D k is the convolution kernel size, M is the number of input channels, N is the number of convolution kernel groups, and D F is the output image feature size, (D k ·D k ·M) is the multiplication amount of a convolution, (N·D F ·D F ) is the number of elements of the output feature map;

[0084] Using the third formula (3): [(D k ·D k )·(M·D F ·D F )+M·(N·D F ·D F )] (3) Calculate the computational effort required to obtain the depthwise separable convolution;

[0085] Where: (D k ·D k )·(M·D F ·D F ) is the computational complexity of the depthwise convolution; M·(N·D F ·D F ) is the amount of point-by-point convolution calculation;

[0086] Based on the first formula (1) and the third formula (3), the parameter reduction rate of the improved model relative to the original model is calculated.

[0087] Further, see Figures 1 to 10 , using the first formula (1): [(D k ·D k ·M)(N·D F ·D F )] (1) After calculating the amount of computation required for the standard convolution, the following are included:

[0088] Using the second formula (2): [D k ·D k ·M·N] (2) Calculate the parameters of the standard convolution;

[0089] Using the third formula (3): [(Dk ·D k )·(M·D F ·D F )+M·(N·D F ·D F )] (3) The computational effort of obtaining the depthwise separable convolution includes:

[0090] Using the fourth formula (4): [D k ·D k ·M+1·1·M·N] (4) Calculate the parameters of the depthwise separable convolution;

[0091] Introducing the channel number factor α and the output feature scaling factor ρ into the third formula (3), the fifth formula (5) of the computational amount of the depthwise separable convolution is obtained:

[0092] (D k ·D k )·(αM·ρD F ·ρD F )+αM·(αN·ρD F ·ρD F )] (5);

[0093] Comparing the first formula (1) with the third formula (3), a sixth formula (6) is obtained:

[0094]

[0095] The sixth formula (6) is simplified to the seventh formula (7):

[0096]

[0097] in, is the parameter reduction rate of the improved model relative to the original model.

[0098] The parameter reduction rate can be used as a basis for judging the improvement of model efficiency, thereby determining the improvement speed. This is helpful for the preliminary judgment of the detection time.

[0099] At the same time, by integrating the processing features of depth-separable convolution and the data volume expressions of the two, it can be seen that in a single calculation process of standard convolution, the number of parameters is N times that of depth-based convolution. Through depth-based convolution, the number of convolution parameters can be reduced to one-Nth of the original, thereby greatly reducing the amount of calculation and the number of parameters.

[0100] In addition, this embodiment also provides a method for detecting weld surface defects based on a target detection model, wherein the target detection model is obtained after being improved by the improvement method, and the detection method comprises:

[0101] The collected weld surface image information is input into the neck inverse residual structure with depthwise separable convolution to obtain a multi-scale feature map with spatial information;

[0102] Inputting the multi-scale feature map with spatial information into a single-layer encoder to obtain a feature vector with position and context information;

[0103] Input the feature vector with position and context information into a decoder to generate a predicted result;

[0104] Based on the predicted results, a mating ratio between the predicted results and the real information is calculated;

[0105] Classification and regression prediction are performed based on the mating ratio, and the detection result is obtained by calculation.

[0106] In other words, the basic principle process of this detection method is:

[0107] First, the defect image to be detected is input into the improved feature extraction network of the model, and its image information is input into the neck residual block with depthwise separable convolution to obtain a multi-scale feature map with spatial information.

[0108] Secondly, the multi-scale feature map with spatial information is further input into the improved encoder layer in the Lite-Deformable-DETR model, which consists of multiple stacked lightweight attention layers and feedforward neural networks, and finally obtains a set of feature vectors with position and context information, which are input into the decoder layer.

[0109] Thirdly, the decoder consists of a self-attention module and a multi-scale deformable attention module. The self-attention module is used to interact between objects, while the multi-scale deformable attention module is used to extract features from the output of the encoder.

[0110] Finally, the predicted results are generated, and the mating ratio between them and the real information is calculated. Classification and regression prediction are performed based on the mating ratio to obtain the final target detection result.

[0111] The algorithm can adapt to the weld defect detection needs of different materials and surface features by adjusting and optimizing multiple parameters, while reducing the load requirements on the detection hardware and improving the efficiency and accuracy of the detection.

[0112] In practical applications, the algorithm model involved in the present invention can be used not only for weld detection, but also for deployment scenarios where non-destructive testing of various large-scale high-altitude structures is required, such as port machinery, large bridges, wind power generation equipment, etc.

[0113] In addition, this embodiment also provides an intelligent welding device, which is used to execute the detection method.

[0114] Since the technical effect achieved by the intelligent welding equipment is the same as that of the detection method, the intelligent welding equipment will not be explained any more.

[0115] The improved target detection model in this embodiment is experimentally verified:

[0116] The lightweight Lite-Deformable-DETR target detection model involved in the present invention is compared with a variety of existing target detection technologies under experimental conditions, and tested through multiple sets of actual defect data. An experimental platform is built for the detection of convolutional networks and original end-to-end networks. The experiment is based on the Ubuntu system. The comparison models are DETR, Deformable-DETR, Faster_rcnn and Yolov3. The specific training environment and parameters are shown in Table 3.

[0117] During the training of various data sets, the compared models all used their publicly available best weight files for training and prediction.

[0118] In order to ensure the credibility of the training results of multiple different training models on the same dataset, the data of the dataset is checked to remove or correct any missing values ​​and outliers. The ratio of training set and validation set in the dataset is unified to ensure that the performance evaluation of the model on different datasets is representative.

[0119] The hyperparameter settings of multiple models were kept consistent, the optimizer was uniformly set to Adam optimizer, and the training rounds were 100. The same random seed was set during the training process to ensure the consistency of randomness during data loading and training, thereby reducing the impact of accidental factors on the training results.

[0120] During the training process, each model is trained independently, and the loss value and evaluation index of each training process are recorded. The model performance is evaluated on the validation set and the test set respectively. The validation set is used for parameter adjustment, and the test set is used for final performance evaluation to ensure the reliability of the results. At the same time, all models are evaluated using the same evaluation index to ensure consistent evaluation standards.

[0121] Through the above steps, the results of multiple different models trained on the same dataset are guaranteed to be credible. This systematic and standardized training and evaluation method can minimize bias and error and provide reliable model performance comparison and analysis.

[0122] The present invention designs a lightweight Lite-Deformable-DETR target detection model that can be deployed on the front-end low-load computing hardware. As shown in Table 4, through experimental verification, for the original Deformable-DETR model, its lightweight Lite model has significant advantages in both model parameters and computational complexity. The number of parameters is only 17.7M, the amount of computation is 70.6Flops, the scale is reduced by 56%, the amount of computation is reduced by 43%, and the accuracy loss is reduced by 4.7%mAP. DETR and Faster_rcnn are 88.8 and 84.8 respectively on AP50, with good performance but relatively high parameter and computational complexity. On the contrary, Yolov3 has lower performance and computational complexity, with an AP50 of 77.3 and a parameter of 61.5M.

[0123] See also Figure 7 , 8 As shown in Table 5, the training effect of NEU-DET is similar to that of MPI. The Lite model performs well in the AP50 index, reaching a high level of 74.7. At the same time, the number of parameters and the amount of computation are significantly lower than other models, which are 17.7M and 70.6Flops respectively. For the original Deformable-DETR model, its lightweight Lite model reduces the model size by 56%, the amount of computation by 43%, and the accuracy loss is only reduced by 1.4% mAP, showing high cost performance and computational efficiency.

[0124] Although Deformable-DETR has the best performance on AP50, reaching 76.1, its parameter count and computational complexity are 40.7M and 124Flops, respectively, which are relatively high. The with ResNet sub-model has found a balance between performance, parameter count, and computational complexity, but the performance improvement is limited. In contrast, DETR, Faster_rcnn, and Yolov3 perform relatively average in terms of AP50 and computational complexity. Among them, Faster_rcnn has the highest computational complexity, reaching 134Flops, while Yolov3 has the highest parameter count but the worst performance, with an AP50 of only 66.3.

[0125] See also Fig. 9 , 10As shown in Table 6, the scale and image feature quantity of the GC-10 dataset are higher than those of the other two datasets. Deformable-DETR continues to lead in AP50, reaching 68.2, which is still the highest despite a slight decrease. Lite maintains the lowest parameter quantity of 17.7M, while its AP50 performance is 55 and the computational cost is 70.6Flops, showing a relatively high efficiency. With ResNet, a balance is reached in performance, parameter quantity and computational cost, with AP50 of 66.4, parameter quantity of 36.9M and computational cost of 55.2Flops. However, the performance of DETR is significantly lower than that of other models, with Faster_rcnn having the highest computational cost and Yolov3 having the largest parameter quantity. The results of the Lite model in this dataset show that it has reached the bottleneck of lightweight model training. For large-capacity high-definition datasets, lightweight models still have limitations.

[0126] Aiming at the low load of front-end computing hardware in actual industrial detection, the present invention optimizes the end-to-end target detection network based on the Transformer architecture and obtains the improved model Lite-Deformable-DETR. While maintaining a small loss of accuracy, the number and complexity of model parameters are greatly reduced. It can achieve high-efficiency and high-precision weld surface defect detection under the condition of low-load computing hardware at the front end.

[0127] Although the disclosure is disclosed as above, the protection scope of the disclosure is not limited thereto. Those skilled in the art may make various changes and modifications without departing from the spirit and scope of the disclosure, and these changes and modifications will fall within the protection scope of the present invention.

[0128] Table 1 Original feature extraction network deconstruction training

[0129]

[0130]

[0131] Table 2 Encoder layer decomposition training

[0132] Model iter mAP(%) AR(%) Params(M) FPS(f / s) layer1 4300 68.1 69.8 40.8 19.79 layer2 4300 72.5 50.0 42.1 19.02 layer3 4300 71.0 93.1 43.4 17.40 layer4 4300 74.0 94.9 44.6 14.45 layer5 4300 82.8 94.2 45.9 16.73 layer6 4300 77.0 92.2 47.2 16.11

[0133] Table 3 Multi-model training parameters

[0134]

[0135] Table 4. Training results of multiple models based on MPI dataset

[0136]

[0137]

[0138] Table 5. Training results of multiple models based on NEU-DET dataset

[0139] Model Epoch AP AP50 AP75 Param(M) Flops DETR 100 38.2 70.9 36.1 41.5 59.8 Deformable-DETR 100 42.5 76.1 41.8 40.7 124 Lite-Df-DETR 100 41.3 74.7 43.7 17.7 70.6 Lit-Df-DETR-ResNet 100 40.7 74.5 39.5 36.9 55.2 Faster_rcnn 100 36.5 69.3 34 41.3 134 Yolov3 100 27.1 66.3 14.2 61.5 69.9

[0140] Table 6 Model training results based on GC-10 dataset

[0141]

[0142]

Claims

1. A method for improving a target detection model, characterized in that: The target detection model is used to detect surface defects of welds. The target detection model is a Lite-Deformable-DETR target detection model. The improved method includes: Use depth-wise separable convolution to replace the standard convolution in the original model; An inverse residual structure is used to replace the residual structure in the original model, and a linear activation function and a bottleneck structure are set in the inverse residual structure; Set the encoder to a single-layer encoder.

2. The improved method according to claim 1, characterized in that: Using depth-wise separable convolution to replace the standard convolution in the original model includes: Convolving the depthwise convolution with the pointwise convolution into the depthwise separable convolution; Setting the depthwise convolution to process spatial information in length and width directions; The cross-correlation operation of multi-channel features is utilized, and the spatial information across channels is processed using the point-by-point convolution.

3. The improved method according to claim 1, characterized in that: A plurality of stacked lightweight attention layers and a feedforward neural network are arranged in the single-layer encoder.

4. The improved method according to claim 1, characterized in that: An attention module and a multi-size variable attention module are set in the decoder of the target detection model, Make the attention module interact between targets, The multi-scale variable attention module is enabled to extract features from the output of the encoder.

5. The improved method according to any one of claims 1 to 4, characterized in that: After setting the encoder to a single layer encoder includes: Based on the convolution kernel size, number of input channels, and number of convolution kernel groups of the original model, the parameter reduction rate of the improved model relative to the original model is obtained.

6. The improved method according to claim 5, characterized in that: Based on the convolution kernel size, number of input channels, and number of convolution kernel groups of the original model, the parameter reduction rate of the improved model relative to the original model is obtained: Using the first formula (1): [(D k ·D k ·M)·(N·D F ·D F )] (1) Calculate the computational effort of the standard convolution; Among them, D k is the convolution kernel size, M is the number of input channels, N is the number of convolution kernel groups, and D F is the output image feature size, (D k ·D k ·M) is the multiplication amount of a convolution, (N·D F ·D F ) is the number of elements of the output feature map; Using the third formula (3): [(D k ·D k )·(M·D F ·D F )+M·(N·D F ·D F )] (3) Calculate the computational effort required to obtain the depthwise separable convolution; Where: (D k ·D k )·(M·D F ·D F ) is the computational complexity of the depthwise convolution; M·(N·D F ·D F ) is the amount of point-by-point convolution calculation; Based on the first formula (1) and the third formula (3), the parameter reduction rate of the improved model relative to the original model is calculated.

7. The improved method according to claim 6, characterized in that: Using the first formula (1): [(D k ·D k ·M)·(N·D F ·D F )] (1) After calculating the amount of computation required for the standard convolution, the following are included: Using the second formula (2): [D k ·D k ·M·N] (2) Calculate the parameters of the standard convolution; Using the third formula (3): [(D k ·D k )·(M·D F ·D F )+M·(N·D F ·D F )] (3) The computational effort of obtaining the depthwise separable convolution includes: Using the fourth formula (4): [D k ·D k ·M+1·1·M·N] (4) Calculate the parameters of the depthwise separable convolution; Introducing the channel number factor α and the output feature scaling factor ρ into the third formula (3), the fifth formula (5) of the computational amount of the depthwise separable convolution is obtained: (D k ·D k )·(αM·ρD F ·ρD F )+αM·(αN·ρD F ·ρD F ) (5): Comparing the first formula (1) with the third formula (3), a sixth formula (6) is obtained: The sixth formula (6) is simplified to the seventh formula (7): in, is the parameter reduction rate of the improved model relative to the original model.

8. A method for detecting weld surface defects based on a target detection model, characterized in that: The target detection model is obtained by being improved by the improvement method described in any one of claims 1 to 7, and the detection method comprises: The collected weld surface image information is input into the neck inverse residual structure with depthwise separable convolution to obtain a multi-scale feature map with spatial information; Inputting the multi-scale feature map with spatial information into a single-layer encoder to obtain a feature vector with position and context information; Input the feature vector with position and context information into a decoder to generate a predicted result; Based on the predicted results, a mating ratio between the predicted results and the real information is calculated; Classification and regression prediction are performed based on the mating ratio, and the detection result is obtained by calculation.

9. An intelligent welding device, characterized in that: The intelligent welding equipment is used to execute the detection method described in claim 8.

Citation Information

Patent Citations

  • Lightweight large-scale building high-altitude steel structure weld defect identification model and weld quality detection device and method

    CN116342531A