Defect detection model training method, detection method, equipment, medium and product

Abstract

The present invention discloses a defect detection model training method, detection method, equipment, medium and product, wherein the training method comprises constructing a sample data set; wherein the sample data set comprises a plurality of samples, each sample comprises a weld defect image and its defect category; constructing a weld defect detection model; wherein the weld defect detection model adopts an improved YOLOv8 model, introduces a high-resolution feature map P2 in the neck network, and adopts a dynamic detection head in the head network; while the weld defect detection model is trained using the sample data set, the weld defect detection model is pruned using pruning technology to obtain a target defect detection model. The present invention can accurately detect and locate tiny defects in welds, and solves the problem of low detection accuracy of small-sized and diversified weld defects by traditional methods.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision and machine learning, and in particular relates to a steel bridge deck weld defect detection model training method, detection method, equipment, storage medium and product. Background Art

[0002] The structural integrity of steel bridge decks is critical to bridge safety, especially welds, which are the most vulnerable areas to defects. Weld defects, such as holes, slag inclusions, and cracks, not only threaten the safety of bridges, but may also lead to serious structural failure if not discovered and treated in time. Traditional weld defect detection methods, including manual visual inspection and basic automated image analysis, are widely used, but have problems such as low efficiency, high cost, and high false detection rate.

[0003] With the continuous advancement of industrial automation, although automated weld defect detection technology based on computer vision has been proposed and applied, it is still insufficient when facing small-sized and diversified weld defects in complex environments, especially in the accuracy and efficiency of detecting small-sized weld defects, which needs to be improved.

[0004] Existing automated weld defect detection technologies cannot effectively solve the problem of accurate detection of small-sized and diverse weld defects in complex environments. On the one hand, traditional anchor-based detection methods, such as YOLOv3, YOLOv4, YOLOv5, and Faster R-CNN, often preset the size and shape of defects and cannot flexibly cope with the diversity of weld defects in size and shape, resulting in unsatisfactory results in processing small-sized weld defects, reducing the sensitivity and accuracy of detection. On the other hand, although deep learning-based convolutional neural network technology has shown good detection performance in multiple application scenarios, it often does not work well in high-noise environments or in the detection of extremely small defects; at the same time, these methods usually rely on complex parameter adjustments and large training data sets, and often cannot meet the requirements of real-time detection in terms of processing speed. It is still a challenge to achieve a balance between high efficiency and high accuracy in highly complex backgrounds. In addition, for irregular or deformed defect shapes, the flexibility and accuracy of existing technologies are far from ideal, especially in key applications such as steel bridge deck weld detection, where their performance limitations are significant and fail to fully meet the stringent requirements of industrial production. Summary of the invention

[0005] The purpose of the present invention is to provide a defect detection model training method, detection method, equipment, medium and product to solve the problem that traditional methods have low detection accuracy and poor real-time detection for small-sized and diversified weld defects.

[0006] The present invention solves the above technical problems through the following technical solutions: a weld defect detection model training method, comprising:

[0007] Constructing a sample data set; wherein the sample data set includes a plurality of samples, each sample including a weld defect image and its defect category;

[0008] Construct a weld defect detection model; wherein the weld defect detection model adopts an improved YOLOv8 model, and the improved YOLOv8 model includes a backbone network, a neck network and a head network, the backbone network includes a first CBS module, a feature layer P2, a feature layer P3, a feature layer P4 and a feature layer P5 connected in sequence, and the neck network includes a first upsampling layer, a first splicing layer, a first C2f module, a second upsampling layer, a second splicing layer, a second C2f module, a third upsampling layer, a third splicing layer, a third C2f module, a second CBS module, a fourth splicing layer, a fourth C2f module, a third CBS module, a fifth splicing layer, a fifth C2f module, a fourth CBS module, a sixth splicing layer and The sixth C2f module; the head network includes a first motion detection head, a second motion detection head, a third motion detection head and a fourth motion detection head; the feature layer P5 is connected to the first upsampling layer and the sixth splicing layer, the feature layer P4 is connected to the first splicing layer, the feature layer P3 is connected to the second splicing layer, the feature layer P2 is connected to the third splicing layer, the first C2f module is connected to the fifth splicing layer, the second C2f module is connected to the fourth splicing layer, the third C2f module is connected to the first motion detection head, the fourth C2f module is connected to the second motion detection head, the fifth C2f module is connected to the third motion detection head, and the sixth C2f module is connected to the fourth motion detection head;

[0009] While the weld defect detection model is trained using the sample data set, the weld defect detection model is pruned using pruning technology to obtain a target defect detection model.

[0010] Furthermore, the feature layer P2, feature layer P3 and feature layer P4 each include a CBS module and a C2f module connected in sequence; the feature layer P5 includes a CBS module, a C2f module and an SPPF layer connected in sequence.

[0011] Furthermore, the first motion detection head, the second motion detection head, the third motion detection head and the fourth motion detection head all include scale attention, channel attention and spatial attention connected in sequence; the scale attention includes an average pooling layer, a 1×1 convolution layer, a ReLu activation layer and a Sigmoid activation layer connected in sequence, the channel attention includes an offset layer, and an index layer, a 3×3 convolution layer and a Sigmoid activation layer connected in sequence, and the spatial attention includes an average pooling layer, a fully connected layer, a ReLu activation layer and a normalization layer connected in sequence.

[0012] Furthermore, while the weld defect detection model is trained using the sample data set, the weld defect detection model is pruned using pruning technology, specifically including:

[0013] Using the sample data set to train and verify the weld defect detection model to obtain a current model;

[0014] Prune the current model using pruning technology to obtain the current pruned model;

[0015] Fine-tune the current pruned model using the sample data set to obtain the current fine-tuned model;

[0016] Determine whether the parameter quantity and detection accuracy of the current fine-tuned model meet the set requirements; if so, use the current fine-tuned model as the target defect detection model output; if not, repeat the pruning and fine-tuning steps until the parameter quantity and detection accuracy of the fine-tuned model meet the set requirements.

[0017] Furthermore, an amplitude-based pruning method is used to prune the current model or the current fine-tuned model, specifically including:

[0018] Calculate the importance score of the weight parameters of each layer in the current model or the current fine-tuned model;

[0019] Sort the weight parameters of all layers in the current model or the current fine-tuned model according to the importance score;

[0020] According to the single pruning ratio, the weight parameters ranked later are pruned to obtain the current pruned model.

[0021] Furthermore, in training and fine-tuning, the EIoU loss function is used; wherein the specific expression of the EIoU loss function is:

[0022]

[0023] Among them, L EIoUrepresents the enhanced loss value, IOU represents the intersection-over-union ratio between the predicted box and the real box, b represents the coordinates of the center point of the predicted box, and b gt represents the center point coordinates of the real box, ω represents the width of the predicted box, ω gt represents the width of the real box, h represents the height of the predicted box, and h gt represents the height of the real box, ω c Indicates the width of the minimum bounding rectangle of the prediction box, h c It represents the height of the minimum bounding rectangle of the prediction box, and ρ() represents the Euclidean distance calculation function.

[0024] Based on the same concept, the present invention also provides a weld defect detection method, comprising:

[0025] Acquire the weld defect image to be detected;

[0026] Calling a target defect detection model; wherein the target defect detection model is trained using the weld defect detection model training method described above;

[0027] Using the target defect detection model to detect the weld defect image to be detected, and obtaining a preliminary prediction result; wherein the preliminary prediction result includes a predicted target frame and a predicted category;

[0028] The preliminary prediction results are filtered and de-overlapped to obtain final prediction results.

[0029] Based on the same concept, the present invention also provides an electronic device, including a memory, a processor, and a computer program / instructions stored in the memory, wherein the processor executes the computer program / instructions to implement the weld defect detection model training method or weld defect detection method as described above.

[0030] Based on the same concept, the present invention also provides a computer-readable storage medium having a computer program / instruction stored thereon, which, when executed by a processor, implements the weld defect detection model training method or weld defect detection method as described above.

[0031] Based on the same concept, the present invention also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the weld defect detection model training method or weld defect detection method as described above.

[0032] Beneficial Effects

[0033] Compared with the prior art, the advantages of the present invention are:

[0034] The present invention replaces the traditional anchor frame detection head of the traditional YOLOv8 model with a dynamic detection head, and dynamically adjusts and fuses features through the adaptive attention mechanism introduced in the dynamic detection head, without relying on a fixed anchor frame, thereby improving the model's detection capability for small and irregular targets; the shallow features output by the feature layer P2 are introduced into the neck network, and the shallow features are spliced ​​with the deep feature map, thereby increasing the model's attention to small targets and improving the model's detection capability for small targets; the present invention can accurately detect and locate tiny defects in welds, solving the problem of low detection accuracy of small-sized and diversified weld defects in traditional methods.

[0035] The present invention not only uses pruning technology to prune the model while training it, and adopts the EIoU loss function, which not only makes the model more efficient in real-time processing scenarios, but also greatly improves the stability and reliability of the model in changeable and complex environments. It is particularly suitable for real-time and accurate detection of steel bridge deck weld defects. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solution of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only one embodiment of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0037] Figure 1 is a flow chart of a weld defect detection model training method in an embodiment of the present invention;

[0038] Figure 2 It is a diagram of the improved YOLOv8 model architecture in an embodiment of the present invention;

[0039] Figure 3 is a structural diagram of a motion detection head in an embodiment of the present invention;

[0040] Figure 4 is a relationship diagram of the predicted box, the real box and the minimum bounding rectangle in an embodiment of the present invention. DETAILED DESCRIPTION

[0041] The following is a clear and complete description of the technical solutions in the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] The technical solution of the present application is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0043] Example 1

[0044] like Figure 1 As shown, a weld defect detection model training method provided by an embodiment of the present invention comprises the following steps:

[0045] Step S1: Construct a sample dataset.

[0046] The sample data set includes multiple samples, each of which includes a weld defect image and its defect category. In this embodiment, the weld defect images are obtained by laboratory self-production and on-site shooting. The weld defect types include undercut, weld bead, hole and slag inclusion. The size of the weld defect image is 1280×1280. The number of samples in the sample data set is 2133. The sample data set is divided into a training set, a validation set and a test set. The number of samples in the training set is 1494, the number of samples in the validation set is 426, and the number of samples in the test set is 213.

[0047] Step S2: construct a weld defect detection model.

[0048] The weld defect detection model uses the improved YOLOv8 model. Figure 2 As shown, the improved YOLOv8 model includes a backbone network (Backbone), a neck network (Neck) and a head network (Head), the backbone network includes a first CBS module, a feature layer P2, a feature layer P3, a feature layer P4 and a feature layer P5 connected in sequence, and the neck network includes a first upsampling layer (Upsample), a first concatenation layer (Concat), a first C2f module, a second upsampling layer, a second concatenation layer, a second C2f module, a third upsampling layer, a third concatenation layer, a third C2f module, a second CBS module, a fourth concatenation layer, a fourth C2f module, a third CBS module, a fifth concatenation layer, and a fifth C2f module connected in sequence. The fifth C2f module, the fourth CBS module, the sixth splicing layer and the sixth C2f module; the head network includes the first motion detection head, the second motion detection head, the third motion detection head and the fourth motion detection head; the feature layer P5 is connected to the first upsampling layer and the sixth splicing layer, the feature layer P4 is connected to the first splicing layer, the feature layer P3 is connected to the second splicing layer, the feature layer P2 is connected to the third splicing layer, the first C2f module is connected to the fifth splicing layer, the second C2f module is connected to the fourth splicing layer, the third C2f module is connected to the first motion detection head, the fourth C2f module is connected to the second motion detection head, the fifth C2f module is connected to the third motion detection head, and the sixth C2f module is connected to the fourth motion detection head.

[0049] In this embodiment, feature layer P2, feature layer P3 and feature layer P4 all include CBS modules and C2f modules connected in sequence; feature layer P5 includes CBS modules, C2f modules and SPPF layers connected in sequence. Feature layer P2, feature layer P3, feature layer P4 and feature layer P5 correspond to features of different scales, respectively, allowing the model to detect targets of different scales simultaneously.

[0050] The C in the CBS module represents the convolution block Conv, B represents BatchNorm2d, and S represents the SiLu activation function. The CBS module encapsulates the combined operation of convolution, batch normalization, and activation function. The CBS module fuses feature maps of different scales to improve the accuracy of target detection.

[0051] The C2f module (Cross Stage Partial Layer with 2Convs) is an optimized feature extraction unit. The C2f module includes a convolution block Conv, multiple Bottle necks, a splicing layer, and a CBS module. The input feature map is received by the convolution block Conv in the C2f module and generates an intermediate feature map. Figure 1 Part of it is directly passed to the splicing layer, and the other part is passed to multiple bottle necks for further processing. The feature map input to the bottle neck undergoes a series of convolution, normalization and activation operations. The feature map generated at the end is spliced ​​with the feature map directly passed to the splicing layer. The spliced ​​feature map enters the CBS module for processing to generate an output feature map. The C2f module extracts the basic features of the input feature map through the convolution block Conv, and further extracts and enhances the features through multiple bottle necks. Multiple bottle necks can capture more complex patterns and details; the directly transmitted feature map and the processed feature map are fused through the splicing operation, so that the model can comprehensively utilize multi-scale and multi-level information. The C2f module effectively reduces the amount of calculation and enhances the gradient flow through the design of partial connection and separation paths, thereby improving the efficiency of feature extraction. The introduction of the C2f module enhances the model's ability to obtain stronger feature expression capabilities while maintaining high computational efficiency.

[0052] The SPPF layer (Spatial Pyramid Pooling Fast) is a fast version of the spatial pyramid pooling SPP. The function of SPP is to extract and encode features of images at different scales. It can rescale input images of any size to a fixed size and generate a feature vector of fixed length. SPPF changes the parallel structure of SPP into a serial structure, which increases the receptive field, reduces the amount of calculation, and plays the same role as SPP.

[0053] The backbone network Backbone is responsible for extracting features from the input image (such as weld defect images). The backbone network adopts the CSPDarknet structure, which can effectively reduce the amount of calculation and improve the efficiency of feature extraction; at the same time, the C2f module is used to improve the gradient flow and model performance. The backbone network processes the input image as follows: the input image whose format and size meet the model requirements is input into the backbone network, and the input image is subjected to preliminary feature extraction by the first CBS module. The feature map obtained by the preliminary feature extraction is sequentially subjected to multi-scale feature extraction by feature layers P2, P3, P4 and P5, and multi-scale information from high-resolution details to low-resolution semantics is obtained, so that the model can handle targets of different scales; the feature map is further enriched by the SPPF layer, and the model's perception of targets of different scales is enhanced, and the receptive field is expanded without significantly increasing the computational complexity; the processed multi-scale feature map is transmitted to the neck network Neck for further feature fusion and target detection to improve the model's detection capability of multi-scale targets.

[0054] In the neck network Neck, feature maps of different scales are respectively upsampled and transferred to the feature layer with higher resolution, and then spliced ​​with the feature map with higher resolution. The spliced ​​feature map not only contains the semantic information of the deep features, but also retains the detail information of the shallow features, thus realizing the effective combination of deep abstract features and shallow detail features. Especially when processing small-sized targets, it can provide richer spatial details, thereby improving the model's ability to detect defects of different scales in complex backgrounds.

[0055] The spliced ​​feature maps are respectively input to the motion detection heads of the head network Head to perform target detection. Figure 3 As shown, the first motion detection head, the second motion detection head, the third motion detection head and the fourth motion detection head all include scale attention (π L ), channel attention (π C ) and spatial attention (π S ). Each dynamic detection head applies attention mechanisms in three dimensions: scale, channel, and space. Each attention mechanism processes feature maps at different levels, aiming to dynamically adjust the weight and expression of feature maps, thereby enhancing the model's ability to detect targets of various scales, positions, and task types. The head network Head not only improves the accuracy of target detection, but also maintains computational efficiency.

[0056] The scale attention includes the average pooling layer, 1×1 convolution layer, ReLu activation layer and Sigmoid activation layer connected in sequence. The scale attention weight is generated through the scale attention. The scale attention mechanism dynamically adjusts the scale response of the feature map according to the target size, so that the model can more accurately capture the information of targets of different scales, improve the detection accuracy of targets of different scales, and significantly improve the detection performance of the model in complex scenes.

[0057] Channel attention includes an offset layer, an index layer, a 3×3 convolution layer, and a Sigmoid activation layer connected in sequence. Channel attention weights are generated through channel attention. The channel attention mechanism allows the model to flexibly adjust the activation state of each channel in different tasks and scenarios to optimize the effect of feature expression.

[0058] Spatial attention includes an average pooling layer, a fully connected layer, a ReLu activation layer, and a normalization layer connected in sequence. The spatial attention generates the attention weight of the spatial position. The spatial attention mechanism enhances the model's sensitivity to the spatial distribution of the target and improves the detection accuracy, especially in complex backgrounds.

[0059] The feature maps processed by scale attention, channel attention and spatial attention are used for target classification, bounding box regression and center point regression to finally generate the detection results.

[0060] The improved YOLOv8 model of the present invention introduces the high-resolution feature P2 in the neck network Neck part, so that the model can provide richer detail information when detecting small-sized targets and subtle defects, thereby significantly improving the detection accuracy; the motion detection head is introduced in the head network Head part, and the multi-attention mechanism of the motion detection head is used to further process the spliced ​​feature map. The motion detection head integrates the dynamic attention mechanism of three dimensions of scale, channel and space, so that the model can flexibly adjust the weight of the feature map according to the specific characteristics of the target and task requirements. This multi-dimensional dynamic adjustment mechanism greatly improves the detection accuracy and robustness of the model when dealing with multi-scale and multi-type targets. Through the introduction of channel attention and spatial attention, the motion detection head can flexibly adjust the channel and spatial expression of the features when different tasks and scenes change, thereby improving the model's classification accuracy and positioning ability for the target.

[0061] Step S3: While training the weld defect detection model using the sample data set, the weld defect detection model is pruned using pruning technology to obtain a target defect detection model.

[0062] In a specific embodiment of the present invention, while training the weld defect detection model using a sample data set, the weld defect detection model is pruned using a pruning technique, specifically including:

[0063] Step S3.1: Using the sample data set to train and verify the weld defect detection model to obtain the current model;

[0064] Step S3.2: Pruning the current model using pruning technology to obtain the current pruned model;

[0065] Step S3.3: fine-tune the current pruned model using the sample data set to obtain the current fine-tuned model;

[0066] Step S3.4: Determine whether the parameter quantity and detection accuracy of the current fine-tuned model meet the set requirements; if so, use the current fine-tuned model as the target defect detection model output;

[0067] If not, proceed to step S3.2 based on the current fine-tuned model.

[0068] Pruning is a method of optimizing a neural network model by reducing unimportant parameters in the model, aiming to reduce the complexity of the model while maintaining or improving the performance of the model as much as possible. In a specific embodiment of the present invention, an amplitude-based pruning method (i.e., LAMP strategy) is used to prune the current model or the current fine-tuned model. The amplitude-based pruning method (i.e., LAMP strategy) has the following advantages:

[0069] (1) No need for additional hyperparameter adjustment: Compared with some methods that require manual setting of pruning parameters, the LAMP strategy is simpler and does not require additional parameter adjustment; (2) Maintain or even improve model performance: Pruning not only reduces the number of parameters, but also improves model performance in some cases; (3) Global pruning: The LAMP strategy focuses on the overall structure of the model, not just the optimization of a single layer; this global pruning method can maintain the integrity of the data flow in the model and reduce the loss of important information during the pruning process.

[0070] In a specific embodiment of the present invention, an amplitude-based pruning method is used to prune the current model or the current fine-tuned model, specifically including:

[0071] Step S3.21: Calculate the importance score of the weight parameters of each layer in the current model or the current fine-tuned model. The specific calculation formula is:

[0072]

[0073] Among them, score(u; W) represents the importance score of the weight parameter of the u-th neuron, W[u] represents the weight parameter of the u-th neuron, υ represents the set of all neurons in the group where the u-th neuron belongs, ∑ υ≥u (W[v]) 2Represents the sum of squares of weight parameters of all neurons in the group where the u-th neuron is located. For the weight parameters of each layer, the importance score is calculated according to the sum of squares of the weight parameters. The smaller the importance score, the smaller the contribution, and it can be considered for priority pruning.

[0074] Step S3.22: Sort the weight parameters of all layers in the current model or the current fine-tuned model according to the importance scores.

[0075] Step S3.23: According to the single pruning ratio, prune the weight parameters that are ranked later to obtain the current pruned model.

[0076] Starting from the weight parameter with the smallest importance score, prune it step by step until the single pruning ratio is reached. In step S3.4, the setting requirement of the model parameter quantity is that the entire model reaches the preset sparsity, and sparsity refers to the proportion of the remaining weight parameters in the model. For example, the model parameter quantity is reduced from 4.17M to 2.056M, and the pruning reaches a sparsity of 53.44%, which improves the operating efficiency of the model in practical applications, and the average precision (mAP) of the model is increased from 89% to 90%.

[0077] The LAMP strategy cuts off the non-critical connections of the model, making the model lighter while retaining important information. This not only reduces the amount of computation in the model, but also makes the model more focused on learning key features, improving generalization and stability. By reducing overfitting and reducing noise interference in the model, LAMP makes the model more efficient in learning key features.

[0078] Since the pruned model may have performance degradation, it is necessary to use the sample data set to fine-tune the pruned model so that the simplified model maintains good detection performance. The pruned and fine-tuned model is lightweight and maintains good detection performance.

[0079] In training and fine-tuning, the present invention adopts the EIoU loss function; wherein, the specific expression of the EIoU loss function is:

[0080]

[0081] Among them, L EIOU represents the enhanced loss value, IOU represents the intersection-over-union ratio between the predicted box and the real box, b represents the coordinates of the center point of the predicted box, and b gt represents the center point coordinates of the real box, ω represents the width of the predicted box, ω gt represents the width of the real box, h represents the height of the predicted box, and h gt represents the height of the real box, ω c Indicates the width of the minimum bounding rectangle of the prediction box, h cIt represents the height of the minimum bounding rectangle of the prediction box, and ρ() represents the Euclidean distance calculation function. Figure 4 The relationship between the predicted box, the true box, and the minimum bounding rectangle is shown.

[0082] The traditional YOLOv8 model adopts the CIOU loss function in design. It introduces the loss of the detection box size on the basis of DIOU, including the loss calculation of length and width, in order to enhance the stability of the target box regression. This improvement avoids the divergence problem that may occur in IOU and GIOU during training. However, when the aspect ratio of the predicted box is the same as that of the true box, the aspect ratio penalty term of CIOU will always be zero, which is obviously an unreasonable design. When further analyzing the gradients of the width and height of the aspect ratio term in CIOU, it is found that the gradients of the two are a pair of opposite numbers, which means that the width and height cannot be increased or decreased at the same time. This limits the flexibility of the model to some extent and seems unreasonable.

[0083] In order to overcome the limitations of CIOU, the present invention uses the EIOU loss function instead. EIoU is an improvement on the CIOU loss term. It considers the aspect ratio of the predicted box and the true box separately, so as to deal with the influence of the aspect ratio more finely. Specifically, EIOU decomposes the aspect ratio penalty term in CIOU and independently calculates the length and width of the predicted box and the true box, as shown in formula (2). This method can more effectively solve the problem that the aspect ratio penalty term in CIOU is always zero, optimize the impact of aspect ratio changes on the loss function, and improve the performance and stability of the model.

[0084] The EIOU loss function consists of IOU loss, distance loss, and height-width loss, which work together on the model to optimize performance. Among them, the height-width loss is particularly critical, which directly minimizes the height and width differences between the bounding box of the predicted target (i.e., the predicted box) and the true bounding box (i.e., the true box). This mechanism not only speeds up the convergence of the model, but also significantly improves the accuracy of positioning; by carefully adjusting the height and width of the target bounding box, the EIOU loss function effectively enhances the performance of the model in the target detection task, ensuring more accurate and stable detection results.

[0085] Table 1 Detection results of the traditional YOLOv8 model and the improved YOLOv8 model of the present invention

[0086]

[0087] In order to verify the effectiveness of the method of the present invention, the traditional YOLOv8 model and the improved YOLOv8 model of the present invention are trained and verified by using the sample data set, and then the two trained models are used to detect the test set of the sample data set; the traditional YOLOv8 model and the improved YOLOv8 model of the present invention are trained and verified by using the PCB-DET data set, and then the test set of the PCB-DET data set is detected by using the two trained models, and the detection results are shown in Table 1. Each sample in the PCB-DET data set is a PCB defect, and the defect types include rat bite, short circuit, spur, miscellaneous copper, missing hole, and open circuit. The PCB-DET data set is divided into a training set, a verification set, and a test set. The training set includes 555 images, the verification set includes 69 images, and the test set includes 69 images.

[0088] As shown in Table 1, compared with the traditional YOLOv8 model, the improved YOLOv8 model of the present invention has significantly improved the detection accuracy and recall rate of all types of defects, which shows that the improved YOLOv8 model of the present invention effectively makes up for the shortcomings of the anchor-free mechanism in terms of recall rate. In the sample data set, hole defects and slag inclusion defects are mainly small and medium-sized targets. The average accuracy of the traditional YOLOv8 model for these defects is only 64.5% and 81.7%; while the improved YOLOv8 model of the present invention significantly enhances the recognition accuracy of small-scale targets, especially the introduction of high-resolution feature map P2 in the neck network, which greatly enhances the capture of small target details. This improvement improves the detection accuracy of hole defects by 23.8%. For slag inclusion defects, due to the inclusion of targets of different sizes, the improvement of the improved YOLOv8 model of the present invention is relatively small, with an increase of 4%. The relatively large undercut and weld bump defects have larger sizes, so the improvements of the improved YOLOv8 model of the present invention are 2.1% and 1.3% respectively, indicating that the applicability and effect of the improved YOLOv8 model of the present invention on a wide range of defect types are improved.

[0089] Similar to the analysis of the sample dataset, the defects of the PCB-DET dataset usually exist in a complex background and are characterized by multiple small-sized defects. In terms of the mean average precision, the improved YOLOv8 model of the present invention has significantly improved the detection performance of all defects. Compared with the traditional YOLOv8 model, the mean average precision of the improved YOLOv8 model of the present invention on open circuits and copper defects is improved by 11.7% and 11.9%, respectively, indicating that the improved YOLOv8 model of the present invention performs well in handling the detection of small-sized defects.

[0090] Table 2 Detection comparison results of the improved YOLOv8 model of the present invention and the most advanced target detection model

[0091]

[0092] In order to further verify the effectiveness of the method of the present invention, the improved YOLOv8 model of the present invention is compared with the most advanced target detection models, including the single-stage detection model YOLOv5 model, the YOLOv8 model and the two-stage detection model Faster R-CNN model. All experiments were conducted on the same hardware devices, such as Intel Xeon Platinum 8352V CPU, NVIDIA GeForce RTX 4090 GPU. In order to compare the detection performance more fairly, all models were randomly trained from scratch without using pre-trained weights. The test results are shown in Table 2.

[0093] As can be seen from Table 2, in terms of accuracy, the improved YOLOv8 model of the present invention reaches 86.7%, which is higher than the 83.6% of the traditional YOLOv8 model in marking defects. This is because the improved YOLOv8 model of the present invention has better performance in distinguishing normal and defective welds, thereby reducing the misjudgment rate. In terms of recall rate, the improved YOLOv8 model of the present invention has also been significantly improved, with a specific recall rate of 85.7%, while the recall rate of the traditional YOLOv8 model is 75.2%, an increase of 10.5%, indicating that the improved YOLOv8 model of the present invention can more comprehensively detect real defects and greatly reduce the risk of missed detection, which is crucial to ensuring the safety of the structure.

[0094] In terms of mean average precision (mAP@0.5), the improved YOLOv8 model of the present invention also shows its superiority, reaching 90.1%, which is nearly 8 percentage points higher than 82.1% of the traditional YOLOv8 model. This improvement reflects the stability and reliability of the improved YOLOv8 model of the present invention under different confidence thresholds, especially when dealing with complex scenarios and diverse defect types.

[0095] The operating efficiency of the improved YOLOv8 model of the present invention is also improved. The model parameter amount of the improved YOLOv8 model of the present invention is only 2.056M, which is lighter than the 3.011M of the traditional YOLOv8 model, so that the operating speed of the improved YOLOv8 model of the present invention is greatly improved to 361.1 frames per second, compared with 272.3 frames per second of the traditional YOLOv8 model, which is more suitable for real-time monitoring applications. This improvement not only reduces the demand for computing resources, but also improves the feasibility and flexibility of the model in practical applications.

[0096] In general, the improved YOLOv8 model of the present invention not only performs well in detection accuracy, but also performs very well in speed and model size, and is very suitable for real-time steel bridge deck weld defect detection. Such performance enables the model to meet the needs of real-time monitoring while ensuring high accuracy.

[0097] Example 2

[0098] A weld defect detection method provided by an embodiment of the present invention comprises the following steps:

[0099] Step S1: Acquire the weld defect image to be detected;

[0100] Step S2: calling a target defect detection model; wherein the target defect detection model is trained using the weld defect detection model training method described in Example 1;

[0101] Step S3: using the target defect detection model to detect the weld defect image to be detected, and obtaining a preliminary prediction result; wherein the preliminary prediction result includes a predicted target frame and a predicted category;

[0102] Step S4: Filter and remove overlaps on the preliminary prediction results to obtain the final prediction results.

[0103] Before the weld defect image to be detected is input into the target defect detection model, it needs to be preprocessed to meet the input requirements of the target defect detection model. Preprocessing includes image resizing, normalization and other operations. Letterbox resize is used to resize the weld defect image to be detected. Letterbox resize is mainly used to resize the image to the specified input size without changing the original aspect ratio of the image. Specifically, Letterbox resize first calculates the image scaling ratio according to the target size so that the image fills the long or short side of the target size as much as possible; then, the unfilled part is filled (usually filled with gray or black) to ensure that the size of the final image is exactly the same as the target size, which avoids deformation of the image due to stretching or compression, thereby retaining the geometric features in the original image. Letterbox resize adjusts the image to the input size required by the model while maintaining the integrity of the image content. Normalization scales the pixel values ​​of the image to between 0 and 1, or normalizes the mean and standard deviation according to the statistical data during model training.

[0104] In order to obtain a prediction result with high confidence, in a specific embodiment of the present invention, a confidence threshold is used to filter the preliminary prediction results, and the predicted target boxes with confidence less than the confidence threshold are filtered out to ensure that only the predicted target boxes with high confidence are transmitted, thereby reducing the computational burden of subsequent processing.

[0105] In order to remove overlapping prediction target boxes and retain the best prediction results, the non-maximum suppression method (NMS) is used to perform de-overlapping on the prediction results after filtering. Specifically, for each category of prediction target boxes, they are sorted from high to low according to the confidence level, and the highest confidence level is first selected as the benchmark. The overlapping area (i.e., intersection over union, IoU) between the prediction target box corresponding to the benchmark and other prediction target boxes is calculated, and the prediction target boxes with overlapping areas greater than the set threshold are removed. Finally, a set of non-overlapping or small-overlapping and optimal prediction target boxes is obtained, which improves the detection accuracy of the model.

[0106] The improved YOLOv8 model of the present invention introduces the high-resolution feature map P2 into the neck network and adopts an advanced dynamic detection head, which provides rich spatial information and detail capture capabilities, and realizes the accurate recognition and positioning of small-scale weld defects in complex industrial environments; the dynamic detection head not only enhances the model's ability to capture weld details, but also significantly improves the accuracy and efficiency of detection through its improved feature fusion and attention mechanism.

[0107] The EIOU loss function is used to optimize the positioning of the prediction box, and the aspect ratio and center point errors are adjusted specifically to reduce the occurrence of false positives and false negatives, significantly improve the accuracy of positioning, reduce errors in model training, and improve the accuracy of the prediction box. The model pruning technology reduces the number of model parameters and computational load, enhances the practicality of the model and the efficiency of deployment in actual industrial applications. Experiments show that the number of model parameters is reduced by about 53%, which improves the operating efficiency in actual applications. In addition, the introduction of the dynamic detection head further reduces the computing requirements while maintaining high detection accuracy, effectively balancing performance and resource consumption.

[0108] Example 3

[0109] An embodiment of the present invention also provides an electronic device, which includes: a memory, a processor, and a computer program / instructions stored in the memory, and the processor executes the computer program / instructions to implement the weld defect detection model training method or weld defect detection method in the embodiment of the present application.

[0110] Although not shown, the electronic device includes a processor, which can perform various appropriate operations and processes according to the program and / or data stored in the read-only memory (ROM) or the program and / or data loaded from the storage portion into the random access memory (RAM). The processor can be a multi-core processor, or it can include multiple processors. In some embodiments, the processor can include a general main processor and one or more special coprocessors, such as a central processing unit, a graphics processing unit (GPU), a neural network processor (NPU), a digital signal processor (DSP), etc. In RAM, various programs and data required for device operation are also stored. The processor, ROM and RAM are connected to each other via a bus. The input / output (I / O) interface is also connected to the bus.

[0111] The processor and memory are used together to execute the program / instructions stored in the memory. When the program / instructions are executed by the computer, the methods, steps or functions described in the above embodiments can be implemented.

[0112] Although not shown, an embodiment of the present invention further provides a computer-readable storage medium having a computer program / instruction stored thereon, which, when executed by a processor, implements the weld defect detection model training method or weld defect detection method in the embodiment of the present application.

[0113] Storage media in embodiments of the present invention include permanent and non-permanent, removable and non-removable items that can be implemented by any method or technology to store information. Examples of storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0114] Readable storage media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0115] Although not shown, an embodiment of the present invention further provides a computer program product, including: a computer program / instruction, which, when executed by a processor, implements the weld defect detection model training method or weld defect detection method in the embodiment of the present application.

[0116] What is disclosed above is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or modifications within the technical scope disclosed in the present invention, which should be covered within the protection scope of the present invention.

Claims

1. A weld defect detection model training method, characterized in that: The training method comprises: Constructing a sample data set; wherein the sample data set includes a plurality of samples, each sample including a weld defect image and its defect category; Construct a weld defect detection model; wherein the weld defect detection model adopts an improved YOLOv8 model, and the improved YOLOv8 model includes a backbone network, a neck network and a head network, the backbone network includes a first CBS module, a feature layer P2, a feature layer P3, a feature layer P4 and a feature layer P5 connected in sequence, and the neck network includes a first upsampling layer, a first splicing layer, a first C2f module, a second upsampling layer, a second splicing layer, a second C2f module, a third upsampling layer, a third splicing layer, a third C2f module, a second CBS module, a fourth splicing layer, a fourth C2f module, a third CBS module, a fifth splicing layer, a fifth C2f module, a fourth CBS module, a sixth splicing layer and The sixth C2f module; the head network includes a first motion detection head, a second motion detection head, a third motion detection head and a fourth motion detection head; the feature layer P5 is connected to the first upsampling layer and the sixth splicing layer, the feature layer P4 is connected to the first splicing layer, the feature layer P3 is connected to the second splicing layer, the feature layer P2 is connected to the third splicing layer, the first C2f module is connected to the fifth splicing layer, the second C2f module is connected to the fourth splicing layer, the third C2f module is connected to the first motion detection head, the fourth C2f module is connected to the second motion detection head, the fifth C2f module is connected to the third motion detection head, and the sixth C2f module is connected to the fourth motion detection head; While the weld defect detection model is trained using the sample data set, the weld defect detection model is pruned using pruning technology to obtain a target defect detection model.

2. The weld defect detection model training method according to claim 1, characterized in that: The feature layer P2, feature layer P3 and feature layer P4 all include a CBS module and a C2f module connected in sequence; the feature layer P5 includes a CBS module, a C2f module and an SPPF layer connected in sequence.

3. The weld defect detection model training method according to claim 1, characterized in that: The first motion detection head, the second motion detection head, the third motion detection head and the fourth motion detection head all include scale attention, channel attention and spatial attention connected in sequence; the scale attention includes an average pooling layer, a 1×1 convolution layer, a ReLu activation layer and a Sigmoid activation layer connected in sequence, the channel attention includes an offset layer, and an index layer, a 3×3 convolution layer and a Sigmoid activation layer connected in sequence, and the spatial attention includes an average pooling layer, a fully connected layer, a ReLu activation layer and a normalization layer connected in sequence.

4. The weld defect detection model training method according to any one of claims 1 to 3, characterized in that: While training the weld defect detection model using the sample data set, pruning the weld defect detection model using a pruning technique specifically includes: Using the sample data set to train and verify the weld defect detection model to obtain a current model; Prune the current model using pruning technology to obtain the current pruned model; Fine-tune the current pruned model using the sample data set to obtain the current fine-tuned model; Determine whether the parameter quantity and detection accuracy of the current fine-tuned model meet the set requirements; if so, use the current fine-tuned model as the target defect detection model output; if not, repeat the pruning and fine-tuning steps until the parameter quantity and detection accuracy of the fine-tuned model meet the set requirements.

5. The weld defect detection model training method according to claim 4, characterized in that: The current model or the current fine-tuned model is pruned using an amplitude-based pruning method, including: Calculate the importance score of the weight parameters of each layer in the current model or the current fine-tuned model; Sort the weight parameters of all layers in the current model or the current fine-tuned model according to the importance score; According to the single pruning ratio, the weight parameters ranked later are pruned to obtain the current pruned model.

6. The weld defect detection model training method according to claim 4, characterized in that: In training and fine-tuning, the EIoU loss function is used; wherein the specific expression of the EIoU loss function is: Among them, L EIOU represents the enhanced loss value, IOU represents the intersection-over-union ratio between the predicted box and the real box, b represents the coordinates of the center point of the predicted box, and b gt represents the center point coordinates of the real box, ω represents the width of the predicted box, ω gt represents the width of the real box, h represents the height of the predicted box, and h gt represents the height of the real box, ω c Indicates the width of the minimum bounding rectangle of the prediction box, h c represents the height of the minimum bounding rectangle of the prediction box, and ρ( ) represents the Euclidean distance calculation function.

7. A weld defect detection method, characterized in that: The detection method comprises: Acquire the weld defect image to be detected; Calling a target defect detection model; wherein the target defect detection model is trained using the weld defect detection model training method according to any one of claims 1 to 6; Using the target defect detection model to detect the weld defect image to be detected, and obtaining a preliminary prediction result; wherein the preliminary prediction result includes a predicted target frame and a predicted category; The preliminary prediction results are filtered and de-overlapped to obtain final prediction results.

8. An electronic device comprising a memory, a processor, and a computer program / instruction stored in the memory, characterized in that: The processor executes the computer program / instructions to implement the weld defect detection model training method according to any one of claims 1 to 6 or the weld defect detection method according to claim 7.

9. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instruction is executed by a processor, the weld defect detection model training method according to any one of claims 1 to 6 or the weld defect detection method according to claim 7 is implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the weld defect detection model training method according to any one of claims 1 to 6 or the weld defect detection method according to claim 7 is implemented.

Images