Weld defect detection method and computer program product

Abstract

The application discloses a weld defect detection method and a computer program product, and relates to the technical field of image processing. The method comprises the following steps: inputting a to-be-detected weld image into a target detection model, wherein the target detection model comprises a feature extraction layer and a feature fusion layer, a multi-scale edge feature optimization submodel is arranged in the feature extraction layer, and a high-level screening feature pyramid network is arranged in the feature fusion layer; performing feature enhancement on the to-be-detected weld image through the multi-scale edge feature optimization submodel, and obtaining an enhanced feature map; performing feature fusion on the enhanced feature map through the high-level screening feature pyramid network, and obtaining a target feature; performing weld defect detection on the to-be-detected weld image based on the target feature, and obtaining a weld defect category. Through hierarchical optimization, local calculation is used to replace global calculation, and redundant features are inhibited through dynamic screening, so that the model calculation resources are significantly reduced under the premise of ensuring the defect detection accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a welding seam defect detection method and a computer program product. BACKGROUND

[0002] The wide application of steel materials in the fields of vehicle manufacturing and bridge construction, especially in the welding process of automobile parts, has continuously improved the requirements for the quality of steel materials. However, steel materials are easily affected by uncertain factors such as environment during the production process, especially in the welding seam area, often appearing defects such as cracks, pores, and incomplete penetration. These defects not only affect the performance of vehicle parts, but also may cause safety hazards in actual use. Therefore, accurate detection of welding seam defects is particularly important.

[0003] Although the detection method based on machine learning improves the automation level to a certain extent, it is limited by the actual terminal edge device computing resources when processing defects of different scales, and it is difficult to meet the demand for high efficiency and high precision of modern production.

[0004] Therefore, how to reduce the model computing resources while ensuring the accuracy of defect detection is a problem that needs to be solved at present. SUMMARY

[0005] The main purpose of the present application is to provide a welding seam defect detection method and a computer program product, which aims to solve the technical problem of insufficient model computing resources while ensuring the accuracy of defect detection.

[0006] To achieve the above-mentioned purpose, the present application provides a welding seam defect detection method, which comprises:

[0007] inputting a to-be-detected welding seam image into a target detection model, wherein the target detection model comprises a feature extraction layer and a feature fusion layer, a multi-scale edge feature optimization sub-model is arranged in the feature extraction layer, and a high-level screening feature pyramid network is arranged in the feature fusion layer;

[0008] performing feature enhancement on the to-be-detected welding seam image through the multi-scale edge feature optimization sub-model to obtain an enhanced feature map;

[0009] performing feature fusion on the enhanced feature map through the high-level screening feature pyramid network to obtain a target feature;

[0010] performing welding seam defect detection on the to-be-detected welding seam image based on the target feature to obtain a welding seam defect category.

[0011] In an embodiment, the step of performing feature enhancement on the to-be-detected welding seam image through the multi-scale edge feature optimization sub-model to obtain an enhanced feature map comprises:

[0012] perform adaptive multi-scale pooling processing on the to-be-tested weld image through the multi-scale edge feature optimization sub-model to obtain multi-scale edge features;

[0013] perform edge enhancement on contour information in the multi-scale edge features to generate an enhanced edge feature map;

[0014] perform dual-domain feature screening on the enhanced edge feature map to obtain screened features;

[0015] align the screened features through bilinear interpolation and splice the screened features with the multi-scale edge features to generate an enhanced feature map.

[0016] In an embodiment, the screened features include first screened features and second screened features, and the step of performing dual-domain feature screening on the enhanced edge feature map to obtain screened features includes:

[0017] in the spatial domain, locate degenerative areas and suppress irrelevant background noise through an attention mechanism to generate the first screened features;

[0018] in the frequency domain, extract high-frequency edge signals and enhance the detail expression capability of complex areas to generate the second screened features.

[0019] In an embodiment, the step of performing feature fusion on the enhanced feature map through the high-level screened feature pyramid network to obtain target features includes:

[0020] perform channel attention weight distribution on the features of each level of the enhanced feature map through the high-level screened feature pyramid network to generate standardized features;

[0021] perform feature alignment on the standardized features to obtain target high-level features;

[0022] perform feature fusion on the screened low-level features and the target high-level features to obtain target features.

[0023] In an embodiment, the step of performing feature alignment on the standardized features to obtain target high-level features includes:

[0024] perform deconvolution upsampling on high-level features in the standardized features to obtain initial high-level features;

[0025] perform size adjustment on the initial high-level features through bilinear interpolation to obtain intermediate high-level features;

[0026] convert the intermediate high-level features into attention weights to obtain target high-level features.

[0027] In an embodiment, the step of performing feature fusion on the screened low-level features and the target high-level features to obtain target features comprises:

[0028] performing weighted screening on the low-level features to obtain screened low-level features;

[0029] performing element-by-element addition on the target high-level features and the screened low-level features to generate fused multi-scale features;

[0030] performing bidirectional fusion from top to bottom and from bottom to top on the fused multi-scale features to obtain target features.

[0031] In an embodiment, before the step of inputting the to-be-tested weld image into the target detection model, the method comprises:

[0032] inputting the weld image data into an initial detection model for model training, and iteratively optimizing a hyperparameter combination based on an evolutionary optimization algorithm to obtain an optimal hyperparameter combination;

[0033] initializing model parameters of the initial detection model according to the optimal hyperparameter combination to obtain an intermediate detection model;

[0034] training the intermediate detection model using a bounding box regression loss function to obtain a trained intermediate detection model, the bounding box regression loss function being used to improve the positioning accuracy of weld defect detection;

[0035] performing lightweight pruning and fine-tuning training on the trained intermediate detection model to obtain the target detection model.

[0036] In an embodiment, the step of inputting the weld image data into an initial detection model for model training, iteratively optimizing a hyperparameter combination based on an evolutionary optimization algorithm to obtain an optimal hyperparameter combination comprises:

[0037] taking the average precision in the intersection over union threshold interval of weld defect detection as the optimization target to construct a fitness function;

[0038] inputting the weld image data into the initial detection model for model training, and evaluating the performance of the hyperparameter combination according to the fitness function to obtain the top N groups of hyperparameters with the highest fitness scores;

[0039] performing cross-recombination and random mutation operations on the top N groups of hyperparameters to generate a first generation of hyperparameters;

[0040] returning to the step of evaluating the performance of the hyperparameter combination according to the fitness function until the number of iterations reaches a preset number of iterations, and outputting the optimal hyperparameter combination.

[0041] In an embodiment, the step of performing lightweight pruning and fine-tuning training on the trained intermediate detection model to obtain a target detection model comprises:

[0042] performing channel importance evaluation on the trained intermediate detection model based on an L1 norm, removing redundant channels with weight values lower than a preset threshold to obtain a pruned intermediate detection model;

[0043] performing fine-tuning training on the pruned intermediate detection model using the weld seam image data and the bounding box regression loss function to obtain a target detection model.

[0044] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which comprises a computer program, wherein the computer program is executed by a processor to implement the steps of the weld defect detection method as described above.

[0045] The one or more technical solutions proposed in the present application have at least the following technical effects:

[0046] inputting the to-be-tested weld seam image into a target detection model, wherein the target detection model comprises a feature extraction layer and a feature fusion layer, a multi-scale edge feature optimization sub-model is arranged in the feature extraction layer, and a high-level screening feature pyramid network is arranged in the feature fusion layer; performing feature enhancement on the to-be-tested weld seam image through the multi-scale edge feature optimization sub-model to obtain an enhanced feature map; performing feature fusion on the enhanced feature map through the high-level screening feature pyramid network to obtain a target feature; performing weld defect detection on the to-be-tested weld seam image based on the target feature to obtain a weld defect category. The multi-scale edge feature optimization sub-model is introduced, the local receptive field constraint and the edge feature weight distribution mechanism are used to focus on the saliency features of the weld edge region. The high-level screening feature pyramid network is used to suppress non-key features from participating in fusion, avoid redundant calculation of the whole image, and reduce the computing resources. Through hierarchical optimization, local calculation is used to replace global calculation, and redundant features are suppressed through dynamic screening, thereby significantly reducing the model computing resources while ensuring the defect detection accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0047] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and serve to explain the principles of the present application together with the specification.

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, for those skilled in the art, other drawings can also be obtained based on these drawings without any creative effort.

[0049] Figure 1A flowchart of a first embodiment of the weld defect detection method of the present application;

[0050] Figure 2 A flowchart of a second embodiment of the weld defect detection method of the present application;

[0051] Figure 3 A structural diagram of a multi-scale edge feature optimization submodel in an embodiment of the present application;

[0052] Figure 4 A diagram of a dual-domain feature screening mechanism in an embodiment of the present application;

[0053] Figure 5 A flowchart of a third embodiment of the weld defect detection method of the present application;

[0054] Figure 6 A structural diagram of a high-level screening feature pyramid network in an embodiment of the present application;

[0055] Figure 7 A structural diagram of an SFF in an embodiment of the present application;

[0056] Figure 8 A diagram of Shape-IoU in an embodiment of the present application;

[0057] Figure 9 An optimization result diagram of a YOLO11 model in an embodiment of the present application.

[0058] The purposes, functional features and advantages of the present application will be further described with reference to the accompanying drawings in conjunction with the embodiments. DETAILED DESCRIPTION

[0059] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and do not limit the present application.

[0060] In order to better understand the technical solutions of the present application, the following will be described in detail in conjunction with the drawings and specific embodiments of the specification.

[0061] Since the traditional weld defect detection technology mainly relies on X-ray, ultrasonic wave, magnetic powder and other means. And the detection results are processed by manual. However, these methods are easily affected by subjective factors, resulting in frequent missed detection and false detection. Although deep learning has become the mainstream technology for weld defect detection, it still faces the problem of limited computing power of edge terminal devices. The large model parameter quantity and computing demand make it difficult for existing methods to be deployed in practical applications, and the existing technology fails to find an ideal balance between detection accuracy and computing resources.

[0062] The application provides a solution, inputting a to-be-tested weld image into a target detection model, wherein the target detection model comprises a feature extraction layer and a feature fusion layer, a multi-scale edge feature optimization sub-model is arranged in the feature extraction layer, and a high-level screening feature pyramid network is arranged in the feature fusion layer; the to-be-tested weld image is subjected to feature enhancement through the multi-scale edge feature optimization sub-model to obtain an enhanced feature map; the enhanced feature map is subjected to feature fusion through the high-level screening feature pyramid network to obtain a target feature; and the to-be-tested weld image is subjected to weld defect detection based on the target feature to obtain a weld defect category. The multi-scale edge feature optimization sub-model is introduced, the saliency features of the weld edge region are focused through a local receptive field constraint and an edge feature weight distribution mechanism, the high-level screening feature pyramid network is adopted to suppress non-key features from participating in fusion, the full-image redundant calculation is avoided, and the calculation resource is reduced. Through hierarchical optimization, the global calculation is replaced by local calculation, the redundant features are suppressed through dynamic screening, the model calculation resource is significantly reduced on the premise of ensuring the defect detection accuracy.

[0063] Based on this, the embodiment of the application provides a weld defect detection method, referring to Figure 1 , Figure 1 The flowchart of the first embodiment of the weld defect detection method of the application is shown in the figure.

[0064] In this embodiment, the weld defect detection method comprises steps S10-S40:

[0065] Step S10, inputting a to-be-tested weld image into a target detection model, wherein the target detection model comprises a feature extraction layer and a feature fusion layer, a multi-scale edge feature optimization sub-model is arranged in the feature extraction layer, and a high-level screening feature pyramid network is arranged in the feature fusion layer.

[0066] It should be noted that the target detection model can be understood as a computer vision model based on deep learning, which is used for locating and identifying weld defects in the to-be-tested weld image. For example, the target detection model can be a YOLO11 network, which can be composed of four main parts: an input layer (Input), a feature extraction layer (Backbone), a feature fusion layer (Neck) and an output layer (Head), which are respectively responsible for processing input features, extracting image features, performing multi-scale feature fusion and target detection. The multi-scale edge feature optimization sub-model can be understood as a sub-module embedded in the feature extraction layer, and the high-level screening feature pyramid network can be understood as a network embedded in the feature fusion layer.

[0067] Step S20, performing feature enhancement on the to-be-tested weld image through the multi-scale edge feature optimization sub-model to obtain an enhanced feature map.

[0068] It should be noted that the multi-scale edge feature refinement sub-model (Multi-Scale Edge Feature Refinement, MS-EFR) can improve the perception ability of the target detection model to the edge region. The enhanced feature map can be understood as a feature map processed by the multi-scale edge feature refinement sub-model, and the gradient response of the weld edge region is significantly enhanced, and the feature response of the non-edge region (such as the smooth weld surface) is suppressed.

[0069] In step S30, the enhanced feature map is fused by the high-level screening feature pyramid network to obtain a target feature.

[0070] It should be noted that the high-level screening feature pyramid network (High-level Screening-feature Pyramid Networks, HSFPN) can be understood as an improved feature pyramid network, which can effectively extract and fuse multi-scale weld defect information at different feature levels, thereby significantly solving the detection problem caused by the size difference of defects. The target feature is a feature filtered and fused by the high-level screening feature pyramid network. For example, the target feature can include semantic information (such as defect shape, position, size) and context-related features of multi-scale weld defects.

[0071] In step S40, based on the target feature, a weld defect detection is performed on the weld image to be tested to obtain a weld defect category.

[0072] For example, based on the target feature, the classification network and the regression network can be used to simultaneously output the defect category and the position coordinates. The weld defect category can include pores, cracks, incomplete fusion, slag inclusion, etc.

[0073] In this embodiment, the weld image to be tested is input into the target detection model, wherein the target detection model includes a feature extraction layer and a feature fusion layer, the multi-scale edge feature refinement sub-model is arranged in the feature extraction layer, and the high-level screening feature pyramid network is arranged in the feature fusion layer; the feature of the weld image to be tested is enhanced by the multi-scale edge feature refinement sub-model to obtain an enhanced feature map; the enhanced feature map is fused by the high-level screening feature pyramid network to obtain a target feature; and based on the target feature, a weld defect detection is performed on the weld image to be tested to obtain a weld defect category. The multi-scale edge feature refinement sub-model is introduced, the local receptive field constraint and the edge feature weight distribution mechanism are used to focus on the salient features of the weld edge region. The high-level screening feature pyramid network is used to suppress non-key features from participating in fusion, avoid redundant calculation of the whole image, and reduce the calculation resources. Through hierarchical optimization, local calculation is used to replace global calculation, and redundant features are suppressed through dynamic screening. Under the premise of ensuring the defect detection accuracy, the model calculation resources are significantly reduced.

[0074] Reference Figure 2 , Figure 2 The flowchart of the second embodiment of the weld defect detection method is shown in FIG. 2B. Based on the first embodiment shown in FIG. 2A, the second embodiment of the weld defect detection method is proposed. Figure 1

[0075] In the second embodiment, the step S20 comprises:

[0076] Step S201, the multi-scale edge feature optimization sub-model is used to perform adaptive multi-scale pooling processing on the weld image to be detected, and multi-scale edge features are obtained.

[0077] For example, adaptive multi-scale pooling processing can be understood as a pooling operation that dynamically selects a pool kernel size (such as 2x2, 3x3, 5x5). The optimal scale can be automatically matched according to the local gradient distribution of the input feature map. The multi-scale edge feature can be understood as a feature map set after adaptive multi-scale pooling, which can include multi-level local and global feature information. For example, the multi-scale edge feature optimization sub-model can be extracted by the edge feature extraction module (Edge Feature Extraction Module) as shown in FIG. 3B. In specific implementation, different scales of the input feature map are pooled using adaptive average pooling (AdaptiveAvgPool) to capture multi-level local and global feature information, so as to ensure that the model can comprehensively understand the multi-scale feature structure of the image. Figure 3

[0078] Step S202, the contour information in the multi-scale edge feature is edge enhanced to generate an enhanced edge feature map.

[0079] It should be noted that edge enhancement can be understood as a feature enhancement method based on gradient direction consistency constraint. For example, the edge gradient extracted by the edge detection operator (such as Sobel operator) can be nonlinearly amplified (such as sigmoid weighting) to realize edge sharpening, and the correlation of adjacent pixel gradients can be used to filter isolated noise points (based on Markov random field model) to realize noise suppression. The enhanced edge feature map can be quantified by signal-to-noise ratio. For example, the edge enhancement module (Edge Enhancer) is used to extract and enhance edge information, which can make the network more sensitive to the edge contour in the image. Figure 3

[0080] Step S203, the enhanced edge feature map is subjected to double-domain feature screening to obtain screened features.

[0081] ​​​It should be noted that the screening features include first screening features and second screening features, and step S203 includes: in the spatial domain, locating the degradation area and suppressing irrelevant background noise through the attention mechanism to generate the first screening features. In the frequency domain, the high-frequency edge signal is extracted and the detail expression ability of the complex region is enhanced to generate the second screening features.

[0082] It can be understood that the first screening features correspond to the spatial domain, and the second screening features correspond to the frequency domain. For example, a dual-domain feature screening mechanism (DSM) can be referred to as Figure 4 DSM is introduced in the MS-EFR module, which adaptively selects the information highly related to the task in the multi-scale features by focusing on the more critical regions in the image (such as complex edge and high-frequency signal region), thereby significantly improving the accuracy of feature selection and the overall performance of the model. Specifically, DSM includes two key modules: spatial selection module (SSM) and frequency selection module (FSM). Among them, SSM effectively locates the severely degraded regions in the image through the selection mechanism in the spatial domain; and FSM focuses on the frequency domain and emphasizes high-frequency edge information to enhance the model's recovery ability for complex regions. Finally, DSM can reduce the interference of irrelevant features by combining the selection mechanisms of the spatial domain and the frequency domain, highlight the key regions and important features, and further improve the model's perception and recovery ability for the target task.

[0083] In step S204, the screening features are aligned through bilinear interpolation and spliced with the multi-scale edge features to generate enhanced feature maps.

[0084] For example, the features of different scales are adjusted to a uniform size through bilinear interpolation to solve the problem of mismatched feature scales. The edge-enhanced features are spliced with the multi-scale features, and the features are further fused and optimized through convolution layers to finally generate enhanced feature maps containing multi-scale information and edge details, thereby improving the feature representation ability and detection performance of the model.

[0085] In this embodiment, the multi-scale edge feature optimization sub-model effectively enhances the expression of edge information through multi-scale feature extraction, edge information enhancement, and feature fusion mechanism, can efficiently select the key features related to the task, and significantly improves the edge perception and target detection ability of the target detection model.

[0086] Referring to Figure 5 , Figure 5 FIG. 3 is a flowchart of a third embodiment of the weld defect detection method, which is based on the above Figure 2The second embodiment shown, proposes a third embodiment of the weld defect detection method of the present application.

[0087] In the third embodiment, the step S30 comprises:

[0088] Step S301, by the high-level screening feature pyramid network, the channel attention weight distribution of each level feature of the enhanced feature map is generated. The standardized feature is generated.

[0089] Exemplarily, the structure of the high-level screening feature pyramid network (HSFPN) is as follows: Figure 6 As shown, the feature information highly related to the task target can be screened through the feature selection module (Feature Selection Module). Specifically, the feature selection module assigns a weight value to each channel by using the channel attention mechanism (Channel Attention, CA), automatically learns and identifies the feature channel that contributes more to the task, thereby strengthening the attention to the key information, suppressing the interference of irrelevant or redundant information, and improving the overall performance of the model. It should be noted that in order to eliminate the dimension mismatch problem of different scale feature maps when they are fused, the feature selection module adjusts and optimizes the dimensions of the feature maps through a 1x1 convolution layer to obtain the standardized feature.

[0090] Step S302, aligning the standardized features to obtain target high-level features.

[0091] It should be noted that step S302 comprises: deconvolution up-sampling the high-level features in the standardized features to obtain initial high-level features; adjusting the size of the initial high-level features by bilinear interpolation to obtain intermediate high-level features; converting the intermediate high-level features into attention weights to obtain target high-level features.

[0092] Exemplarily, the standardized features are up-sampled by transposed convolution to obtain initial high-level features, which expand their spatial dimensions and have finer spatial resolution. Then, the bilinear interpolation is used to further adjust the initial high-level features after up-sampling, so that their spatial dimensions remain consistent with the low-level features, and the intermediate high-level features are obtained. The intermediate high-level features are converted into attention weights to obtain target high-level features, which are used to screen and filter the low-level features, highlighting the important information related to the task target.

[0093] Step S303, the low-level features after screening and the target high-level features are fused to obtain target features.

[0094] It should be noted that step S303 comprises: performing weighted screening on the low-level features to obtain screened low-level features; performing element-by-element addition on the target high-level features and the screened low-level features to generate fused multi-scale features; and performing bidirectional fusion from top to bottom and from bottom to top on the fused multi-scale features to obtain target features.

[0095] It should be noted that selective feature fusion (SFF) can be used as the core of the feature fusion module, and key semantic information in low-scale features is screened and filtered by using high-level features as weights, so as to realize effective fusion of multi-level features and enhance the perception ability of the model to the target. Exemplarily, Figure 7 An exemplary diagram of SFF is shown in FIG. 3. Specifically, SFF expands the spatial size of the target high-level features by using transposed convolution (T-Conv) to match the size of the low-scale features. The target high-level features are upsampled or downsampled by using bilinear interpolation to ensure that the size is consistent with the low-scale features. Channel attention mechanism (CA) is used to generate weights to screen the low-scale features and retain task-related information. The filtered low-scale features are added to the target high-level features element by element to realize efficient fusion of multi-scale features.

[0096] It should be noted that f att represents a feature map obtained by using bilinear interpolation for upsampling or downsampling, and is specifically represented by formula (1). f out represents the target features output by the SFF module, and is represented by formula (2).

[0097] f att = BL(T-conv(f high )) (1)

[0098] f out = f low *CA(f att )+f att (2)

[0099] In formulas (1) and (2), f high represents high-level weld defect features extracted by the network, f low represents weld detail features from the shallow layer of the network, and f att represents a feature map obtained by using bilinear interpolation for upsampling or downsampling. BL refers to the bilinear interpolation method for adjusting the size of the feature map; T-Conv represents transposed convolution, which reconstructs and expands the feature map by learning parameters; and CA represents the channel attention mechanism, which is used to dynamically adjust the importance of the feature channels.

[0100] In this embodiment, through multi-level feature fusion and pyramid structure, HSFPN can effectively extract and fuse multi-scale weld defect information at different feature levels, fully combine high-level semantic information and low-scale positioning information, and improve the performance and robustness of the model in multi-scale detection tasks.

[0101] In one embodiment, based on the above embodiments, before the step of inputting the to-be-detected weld image into the target detection model, the step includes: inputting the weld image data into an initial detection model for model training, and iteratively optimizing the hyperparameter combination based on an evolutionary optimization algorithm to obtain an optimal hyperparameter combination; initializing the model parameters of the initial detection model according to the optimal hyperparameter combination to obtain an intermediate detection model; training the intermediate detection model using a bounding box regression loss function to obtain a trained intermediate detection model, the bounding box regression loss function being used to improve the positioning accuracy of weld defect detection; and performing lightweight pruning and fine-tuning training on the trained intermediate detection model to obtain the target detection model.

[0102] It should be noted that the initial detection model can be understood as an original neural network architecture without training, and the initial detection model can include randomly initialized weight parameters. The evolutionary optimization algorithm (such as genetic algorithm) can be a population optimization algorithm simulating the biological evolution process through selection, crossover, mutation, fitness evaluation, etc. The fitness of each individual (hyperparameter combination) is evaluated by training the YOLO model on the training data set and calculating its performance indicators (such as mAP, loss function value, etc.). The intermediate detection model can be understood as a model initialized with the optimal hyperparameters and preliminarily trained. The bounding box regression loss function (Shape-IoU) can be understood as a loss function for optimizing the position prediction of the detection box. Shape-IoU not only considers the distance, shape and angle between the labeled box and the predicted box, but also considers the proportion and shape difference of the bounding box, thereby effectively improving the regression accuracy. Lightweight pruning is used to remove redundant parameters or channels in the neural network that contribute less to the output, and fine-tuning can continue training on the pruned model with a lower learning rate to restore the accuracy lost due to pruning.

[0103] For example, a schematic diagram of Shape-IoU is shown in Figure 8 , wherein the calculation formula involved is represented as formula (3) to formula (10).

[0104]

[0105] L Shape-IoU =1-IoU+d shape +0.5Ω shape (10)

[0106] In formulas (3) to (10), ww and hh represent weight values related to the width and height of the real frame, respectively, B and B gt represent the predicted frame and the real labeled frame (Ground Truth Box, GT frame), respectively, w and w gt represent the width of the anchor frame and the GT frame, respectively, h and h gt represent the height of the anchor frame and the GT frame, respectively, w w represents the relative degree of difference in width between the predicted frame and the real frame, w h represents the relative degree of difference in height between the predicted frame and the real frame. w w and ww h represent the minimum value related to the width and height. scale represents an exponential parameter for scaling the width and height of the real frame. |B∩B gt | represents the intersection area of the predicted frame and the real labeled frame, |B∪B gt | represents the union area of the predicted frame and the real labeled frame. d shape is used to measure the difference between the predicted frame and the real labeled frame in shape position. x c and represent the center horizontal coordinates of the predicted frame and the real labeled frame, where c represents a normalization parameter for normalizing the coordinate difference. y c and represent the center vertical coordinates of the predicted frame and the real labeled frame. Ω shape is used to comprehensively measure the difference between the predicted frame and the real labeled frame in shape. t represents a loop variable, and e is a natural constant, approximately equal to 2.71828. L Shape-IoU represents the bounding box regression loss function, where IoU represents the intersection over union.

[0107] In the present embodiment, by removing connections, neurons or parameters that contribute less to the performance of the model, the storage requirements of the model can be effectively reduced, and by continuously optimizing and training the initial detection model, the computational complexity is reduced, the detection speed and inference efficiency are further improved while maintaining the accuracy of the model, and the improved model can efficiently run in terminal devices, meeting the real-time requirements in actual applications.

[0108] In an embodiment, the step of inputting the weld seam image data into the initial detection model for model training, and iteratively optimizing the hyperparameter combination based on the evolutionary optimization algorithm to obtain the optimal hyperparameter combination, comprises: taking the average precision in the intersection over union threshold interval of the weld defect detection as the optimization objective to construct a fitness function; inputting the weld seam image data into the initial detection model for model training, and evaluating the performance of the hyperparameter combination according to the fitness function to obtain the top N groups of hyperparameters with the highest fitness scores; performing cross-recombination and random mutation operations on the top N groups of hyperparameters to generate the first generation of hyperparameters; returning to the step of evaluating the performance of the hyperparameter combination according to the fitness function until the number of iterations reaches the preset number of iterations, and outputting the optimal hyperparameter combination.

[0109] For example, the weld seam image data set is input into the initial detection model, and the hyperparameter combination is iteratively optimized based on the evolutionary optimization algorithm. The fitness function can be defined, for example, taking the average precision (mAP) in the intersection over union (IoU) threshold interval in the weld defect detection task as the core index, wherein the mAP weight corresponding to the first IoU threshold (such as 0.5) is 90%, and the mAP weight corresponding to the second IoU threshold interval (such as 0.5-0.95) is 10%. In each generation of evolution, the model is trained using the weld seam image data, the optimal hyperparameter combination is retained according to the fitness score, and a new generation of hyperparameters is generated through cross-recombination and random mutation. After 50 (preset number of iterations) generations of optimization, the optimal hyperparameter combination is output. Figure 9 A plurality of scatter plots are shown in the figure, each corresponding to a hyperparameter. The best value of each hyperparameter is marked by a plus sign. This process generates a new generation of hyperparameter combinations with better performance by combining the characteristics of the best individuals in the early generations. Figure 9In the above table, lr0 represents the initial learning rate, which is used to control the parameter update step size. lrf represents the final learning rate, which is the value to which the learning rate decays in the later training stage. momentum represents the momentum, which is used to accelerate convergence and avoid local optimum. weight_decay represents the weight decay parameter, which is used to limit the size of weight values through L2 regularization during model training to prevent overfitting. warmup_epochs represents the number of training epochs corresponding to the learning rate warm-up phase, during which the learning rate is gradually increased from an initial minimum value to a preset value linearly to avoid unstable gradients in the early training stage. warmup_momentum represents the momentum value used in the learning rate warm-up phase, which is usually set to be lower than the momentum parameter in the normal training phase to stabilize the optimization process. box represents the weight coefficient of the bounding box regression loss function, which is used to adjust the contribution of the error between the predicted box position and the true box to the total loss. cls represents the weight coefficient of the class classification loss function, which is used to control the proportion of classification prediction error in the total loss. df1 represents the adjustment factor of the focal loss function, which dynamically adjusts the loss weight of difficult samples to improve the detection ability of small defects. hsv_h represents the random adjustment intensity of the image hue (Hue) channel during data augmentation, taking a normalized proportion value in the range [0, 0.5]. hsv_s represents the random adjustment intensity of the image saturation (Saturation) channel during data augmentation, taking a normalized proportion value in the range [0, 0.7]. hsv_v represents the random adjustment intensity of the image lightness (Value) channel during data augmentation, taking a normalized proportion value in the range [0, 0.4]. degrees represents the angle range parameter of image random rotation, defining the maximum absolute value of rotation angle (unit: degrees). translate represents the amplitude parameter of image random translation, defining the maximum proportion of horizontal and vertical translation distance to image size. scale represents the amplitude parameter of image random scaling, defining the proportion range of image size adjustment, for example, [0.5, 1.5] represents scaling to 50%-150% of the original size. shear represents the intensity parameter of image random shear transformation, defining the maximum absolute value of shear angle (unit: degrees). perspective represents the intensity parameter of image perspective transformation, controlling the amplitude of three-dimensional space projection transformation, taking a normalized value in the range [0, 0.001]. flipud represents the probability threshold of image up-down flipping, which performs vertical flipping when the random number is less than the threshold. fliplr represents the probability threshold of image left-right flipping, which performs horizontal flipping when the random number is less than the threshold. bgr represents the color channel order conversion flag bit, which is set to True to convert the channel order of the input image from BGR to RGB format. mosaic represents the enable probability of Mosaic data augmentation, representing the probability threshold of combining four images into one composite image in a single training iteration.mixup represents the enabling probability of Mixup data augmentation, representing the probability threshold of linearly mixing two images and labels in a single training iteration. copy_paste represents the enabling probability of Copy-Paste data augmentation, representing the probability threshold of randomly copying a target region from other training images and pasting it to the current image.

[0110] In this embodiment, by using the evolutionary optimization algorithm to automatically search for the optimal hyperparameter combination, the tediousness and subjectivity of manual parameter tuning are avoided, and the efficiency and effectiveness of model optimization are improved. The fitness function can objectively and quantitatively evaluate the performance of different hyperparameter combinations, ensuring that the evolution of each generation of hyperparameters is towards improving detection accuracy. Through multiple iterations of optimization, new hyperparameter combinations are continuously generated and their performance is evaluated until the preset number of iterations is reached, ensuring the sufficiency and stability of the optimization process.

[0111] In one embodiment, the step of performing lightweight pruning and fine-tuning training on the trained intermediate detection model to obtain the target detection model comprises: performing channel importance evaluation on the trained intermediate detection model based on the L1 norm, removing redundant channels with weight values below a preset threshold to obtain a pruned intermediate detection model; and performing fine-tuning training on the pruned intermediate detection model using the weld seam image data and the bounding box regression loss function to obtain the target detection model.

[0112] For example, the complexity of the network structure can be reduced by deleting entire convolution kernels, channels or layers, which can typically significantly reduce the amount of computation. The L1 pruning method is used to sort the weights in the network by calculating the absolute value of each weight, and selectively pruning the connections with the smallest weight values. L1 pruning can efficiently reduce the parameter size of the model while maintaining high detection accuracy. The pruned model is further trained at a lower learning rate to recover the accuracy lost due to pruning.

[0113] In this embodiment, the channel pruning method based on the L1 norm is used to compress the model, which can effectively reduce the storage requirements of the model by removing connections, neurons or parameters that contribute less to the performance of the model. While maintaining the accuracy of the model, the detection speed and inference efficiency are further improved, enabling the improved model to run efficiently in terminal devices and meet the real-time requirements in actual applications.

[0114] To clarify the descriptions of the above embodiments and implementation methods, this application provides some experimental data. Commonly used evaluation metrics in the field of object detection were employed in the experiments, including Precision (P), Recall (R), Average Precision (AP), and Mean Average Precision (mAP). Precision measures the accuracy of the model's predictions, i.e., the proportion of actually positive samples among those predicted as positive; Recall measures the model's recall capability, i.e., the proportion of actually positive samples correctly predicted. AP represents the average precision of a single class at different thresholds, reflecting the model's overall detection capability for that class; while mAP is the average of AP across all classes, comprehensively considering both Precision and Recall, and more comprehensively reflecting the model's detection performance in multi-class scenarios. mAP@50 represents the average precision at an IoU threshold of 0.5, while mAP@50-95, under a more stringent evaluation standard, calculates the average precision by varying the IoU threshold from 0.5 to 0.95 in 0.05 steps, providing a more detailed evaluation of model performance. The calculation formulas for each metric are as follows:

[0115]

[0116] In formulas (11)-(14), P is precision, representing the proportion of true positives among the samples judged as positive by the classifier. TP (True-Positive) is the number of true positives, i.e., the number of samples that are actually positive and correctly predicted as positive by the classifier. FP (False-Positive) is the number of false positives, i.e., the number of samples that are actually negative but incorrectly predicted as positive by the classifier. R is recall, representing the proportion of true positives correctly predicted as positive by the classifier. FN (False-Negative) is the number of false negatives, i.e., the number of samples that are actually positive but incorrectly predicted as negative by the classifier. AP is average precision, which is a calculation of the area under the precision-recall curve, obtained by integrating the precision at different recall levels. mAP is mean average precision, commonly used in tasks such as multi-class object detection, and is the average of the average precision for multiple classes, where n is the total number of classes, p i It is the average precision of the i-th category.

[0117] To comprehensively consider the lightweight effect of the model, the evaluation indexes of the number of model parameters (Parameters), the model size (Model Size) and the computing performance (GFLOPs) are introduced. Among them, Parameters represents the total number of trainable parameters in the model, which is directly related to the storage requirement and the computing overhead; GFLOPs (Giga Floating Point Operations) measures the number of floating point operations required for one forward propagation, reflecting the computing complexity of the model. Lower Parameters and GFLOPs indicate that the model is smaller in size and lower in computing cost.

[0118] It should be noted that the above examples are only used for understanding the present application and do not constitute a limitation on the weld defect detection method of the present application. More forms of simple transformation based on this technical concept are within the protection scope of the present application.

[0119] The present application also provides a computer program product comprising a computer program which, when executed by a processor, implements the steps of the weld defect detection method as described above.

[0120] The computer program product provided by the present application can solve the technical problem of insufficient model computing resources under the premise of ensuring defect detection accuracy. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as those of the weld defect detection method provided by the above-mentioned embodiments, and are not repeated here.

[0121] The above only describes some embodiments of the present application, and does not limit the patent scope of the present application. Any equivalent structural transformation made by using the content of the present application specification and drawings, or direct / indirect application in other related technical fields is included in the patent protection scope of the present application.

Claims

1. A method of detecting a weld defect, characterized by, The method comprises: inputting the to-be-tested weld image into a target detection model, wherein the target detection model comprises a feature extraction layer and a feature fusion layer, a multi-scale edge feature optimization sub-model is arranged in the feature extraction layer, and a high-level screening feature pyramid network is arranged in the feature fusion layer; performing feature enhancement on the to-be-tested weld image through the multi-scale edge feature optimization sub-model to obtain an enhanced feature map; performing feature fusion on the enhanced feature map through the high-level screening feature pyramid network to obtain a target feature; performing weld defect detection on the to-be-tested weld image based on the target feature to obtain a weld defect category; the step of performing feature enhancement on the to-be-tested weld image through the multi-scale edge feature optimization sub-model to obtain an enhanced feature map comprises: performing adaptive multi-scale pooling processing on the to-be-tested weld image through the multi-scale edge feature optimization sub-model to obtain multi-scale edge features; performing edge enhancement on the contour information in the multi-scale edge features to generate an enhanced edge feature map; performing double-domain feature screening on the enhanced edge feature map to obtain screened features; aligning the screened features through bilinear interpolation and splicing the screened features with the multi-scale edge features to generate an enhanced feature map; the screened features comprise first screened features and second screened features, and the step of performing double-domain feature screening on the enhanced edge feature map to obtain screened features comprises: in a spatial domain, locating a degradation area and suppressing irrelevant background noise through an attention mechanism to generate the first screened features; in a frequency domain, extracting a high-frequency edge signal and enhancing the detail expression capability of a complex area to generate the second screened features.

2. The method of claim 1, wherein, the step of performing feature fusion on the enhanced feature map through the high-level screening feature pyramid network to obtain a target feature comprises: performing channel attention weight distribution on the features at each level of the enhanced feature map through the high-level screening feature pyramid network to generate standardized features; performing feature alignment on the standardized features to obtain target high-level features; performing feature fusion on the screened low-level features and the target high-level features to obtain a target feature.

3. The method of claim 2, wherein, the step of performing feature alignment on the standardized features to obtain target high-level features comprises: performing deconvolution upsampling on the high-level features in the standardized features to obtain initial high-level features; performing size adjustment on the initial high-level features through bilinear interpolation to obtain intermediate high-level features; converting the intermediate high-level features into attention weights to obtain target high-level features.

4. The method of claim 3, wherein, the step of performing feature fusion on the screened low-level features and the target high-level features to obtain a target feature comprises: performing weighted screening on the low-level features to obtain screened low-level features; performing element-by-element addition on the target high-level features and the screened low-level features to generate fused multi-scale features; performing bidirectional fusion from top to bottom and from bottom to top on the fused multi-scale features to obtain a target feature.

5. The method of claim 1, wherein, before the step of inputting the to-be-tested weld image into a target detection model, The weld seam image data is input into an initial detection model for model training, and a combination of hyperparameters is iteratively optimized based on an evolutionary optimization algorithm to obtain an optimal combination of hyperparameters; Model parameters of the initial detection model are initialized based on the optimal combination of hyperparameters to obtain an intermediate detection model; The intermediate detection model is trained using a bounding box regression loss function to obtain a trained intermediate detection model, and the bounding box regression loss function is used to improve the positioning accuracy of weld defect detection; The trained intermediate detection model is subjected to lightweight pruning and fine-tuning to obtain a target detection model.

6. The method of claim 5, wherein, The step of inputting the weld seam image data into the initial detection model for model training, iteratively optimizing the combination of hyperparameters based on the evolutionary optimization algorithm, and obtaining the optimal combination of hyperparameters includes: An average precision in an intersection over union threshold interval of weld defect detection is taken as an optimization objective to construct a fitness function; The weld seam image data is input into the initial detection model for model training, and the performance of the combination of hyperparameters is evaluated according to the fitness function to obtain the top N combinations of hyperparameters with the highest fitness scores; The top N combinations of hyperparameters are subjected to cross-recombination and random mutation operations to generate a first generation of hyperparameters; The step of evaluating the performance of the combination of hyperparameters according to the fitness function is returned until the number of iterations reaches a preset number of iterations, and the optimal combination of hyperparameters is output.

7. The method of claim 5, wherein, The step of performing lightweight pruning and fine-tuning on the trained intermediate detection model to obtain the target detection model includes: The trained intermediate detection model is subjected to channel importance evaluation based on L1 norm, redundant channels with weight values lower than a preset threshold are removed, and a pruned intermediate detection model is obtained; The weld seam image data and the bounding box regression loss function are reused to fine-tune the pruned intermediate detection model to obtain the target detection model.

8. A computer program product, characterised in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the weld defect detection method according to any one of claims 1 to 7.

Images