Welding defect image preprocessing method based on multi-scale feature fusion

Abstract

The invention discloses a welding defect image preprocessing method based on multi-scale feature fusion. The method comprises the steps that a molten pool camera is used for collecting various welding state data and preprocessing the welding state data; performing feature extraction on the preprocessed image through a multi-scale convolution kernel to obtain a multi-scale feature map; extracting features in the feature maps of multiple scales through a convolutional neural network model, and carrying out feature fusion on the features; and carrying out pooling operation on the feature map after feature fusion. According to the method, efficient feature extraction and fusion can be carried out on the welding image, and the welding defect identification precision and efficiency are improved.

Description

Technical Field

[0001] This application relates to the field of image processing technologies, and particularly to a method for preprocessing welding defect images based on multi-scale feature fusion. Background Art

[0002] In recent years, multi-feature fusion technologies have been continuously developed in the field of image classification, and many studies have been dedicated to improving model performance through multi-level feature combinations. Existing research has shown that the combination of the main color feature and deep learning models can significantly improve classification accuracy. For example, the dual-encoder architecture developed in Jiang Xingguo, Li Ming. Image Restoration Algorithm Based on Multi-Feature Fusion [J]. Electronic Measurement Technology, 2024, 48(18): 80-88. fuses texture and structural features through an attention mechanism, improving the PSNR index by 1.18% on a public dataset. Although this scheme verifies the effectiveness of cross-modal feature fusion, its linear weighting strategy uses fixed weight coefficients and is vulnerable to dynamic noise interference in high-noise scenarios such as welding defect images, resulting in a decline in feature discriminability.

[0003] Regarding the optimization of multi-scale feature extraction, Chen Jiawei, Ji Tianyao, Mei Guang, etc. A Non-Intrusive Load Monitoring Method Based on Multi-Scale Feature Fusion and Multi-Task Learning Framework [J]. Power System Technology, 2024, 48(5): 2074-2083. proposed a composite network structure combining IBN-Net and UNet, which captures the temporal feature differences of device loads through a multi-time scale feature pyramid, improving the decomposition accuracy of non-intrusive load monitoring by 15.5%. However, this method relies on the dense stacking of 7×7 large-size convolutional kernels, generating a computational load of up to 3.2T FLOPs in real-time welding defect detection, making it difficult to meet the stringent requirements for inference speed in industrial fields.

[0004] In terms of adaptive feature fusion, the multi-view semi-supervised graph model (MVSG) proposed by Liang Z, et al. A low-cost machine learning framework for drug interaction prediction via multi-feature fusion [J]. Physical Chemistry Chemical Physics, 2024. achieved a 6% improvement in the accuracy of drug interaction prediction by dynamically adjusting similarity measurement parameters. However, the construction of the graph structure of this model requires preset node association rules, and when facing randomly distributed defects such as spatter and pores in welding images, it is prone to feature redundancy problems caused by prior knowledge biases.

[0005] From the perspective of the existing technologies, there are two bottlenecks in the current multi-feature fusion method for welding defect classification tasks: First, traditional denoising methods such as Gaussian filtering will over-smooth the image, resulting in the loss of key detail features such as micro-cracks and lack of fusion, and the fixed-weight fusion strategy is difficult to adapt to the dynamic noise environment; Second, the dynamic parameter adjustment mechanism is limited by the prior constraints of the graph structure and cannot effectively process the unstructured feature space distribution in the welding defect images. Summary of the Invention

[0006] In view of this, the present application provides a preprocessing method for welding defect images based on multi-scale feature fusion. Through the technical solution of parallel feature extraction and feature fusion using multi-scale convolutional kernels, this technical solution can be used in an automated welding defect detection system to efficiently extract and fuse features from welding images, so as to improve the accuracy and efficiency of subsequent classification and recognition.

[0007] The present application discloses a preprocessing method for welding defect images based on multi-scale feature fusion, which includes:

[0008] Step 1: Use a molten pool camera to collect various welding state data and preprocess it; various welding state data includes burn-through state, over-welding state, lack of welding state, and full-welding state;

[0009] Step 2: Extract features from the preprocessed image through multi-scale convolutional kernels to obtain feature maps of multiple scales;

[0010] Step 3: Extract features from the feature maps of multiple scales through a convolutional neural network model and perform feature fusion on them;

[0011] Step 4: Perform a pooling operation on the feature map after feature fusion.

[0012] Further, the Step 1 includes:

[0013] Use a molten pool camera to collect various welding state data, and divide the collected various welding state data into training images and test images; crop all the training images and test images so that the sizes of the cropped images are the same;

[0014] Adopt the Gaussian filtering algorithm to filter the cropped images to remove the high-frequency noise in the images, and perform normalization on the denoised images to ensure that the pixel values of the images are within a unified numerical range.

[0015] Further, the Step 2 includes:

[0016] The convolutional layer converts the input image into a feature map by setting different numbers and sizes of convolutional kernels and passes it to the next layer. The input image is the preprocessed image.

[0017] Furthermore, the conversion of the input image into a feature map is achieved through the following formula:

[0018]

[0019] wherein, represents the j-th feature map of the current l-th layer; represents the i-th feature map of the (l - 1)-th layer; M is the number of input feature maps; represents the number of convolution kernels, is the bias term; f(Δ) is a non-linear activation function.

[0020] Furthermore, step 3 includes:

[0021] The convolutional neural network model includes a data feature fusion layer, a depth feature fusion layer, a feature abstract representation layer, and a feature high-level representation layer: the data feature fusion layer, the depth feature fusion layer, the feature abstract representation layer, and the feature high-level representation layer are the first convolutional layer, the second convolutional layer, the third convolutional layer, and the fourth convolutional layer respectively;

[0022] The data feature fusion layer, the depth feature fusion layer, the feature abstract representation layer, and the feature high-level representation layer extract image features from different scales to simultaneously capture the local detail features and global structure features of the image, and improve the recognition ability of the convolutional neural network model for defects of different sizes.

[0023] The extracted multi-scale features are fused by concatenation or weighted summation to ensure the integration of low-level features and high-level features; low-level features include edges and textures; high-level features include shapes and defect categories.

[0024] Furthermore, the data feature fusion layer extracts basic low-level features from the feature map obtained in step 2, uses multiple convolution kernels of different sizes to simultaneously capture the local detail features and global structure features in the feature map; after convolution, the size of the feature map is reduced through a pooling layer, and multiple convolution kernels act on different regions of the feature map simultaneously, and the features of different regions are integrated through a fusion operation;

[0025] The depth feature fusion layer integrates the low-level features extracted by the data feature fusion layer and the deep features learned in the current layer, and through multi-scale convolution kernels, enables the convolutional neural network model to extract features of different scales simultaneously in one convolution operation, and the output after convolution is dimension-reduced through a pooling layer;

[0026] The feature abstraction representation layer removes redundant noise in the feature map and abstractly represents the features of the feature map. It uses a convolution kernel smaller than that used in the deep feature fusion layer to remove redundant details and noise through finer convolution operations. The pooling layer reduces the size of the feature map while retaining important spatial information;

[0027] In the feature high-level representation layer, the convolutional neural network model removes redundant features by using a convolution kernel larger than that used in the feature abstraction representation layer and extracts more critical representative features. Through deep convolutional operations, more important high-level features are extracted from the abstract features of the feature abstraction representation layer, and the high-level features are combined with the low-level features extracted by the data feature fusion layer, the deep feature fusion layer, and the feature abstraction representation layer.

[0028] Further, step 4 includes:

[0029] Pooling operation: Use the max pooling layer to downsample the fused feature map and select the maximum value within the region of each pooling window.

[0030] Further, after step 4, it further includes:

[0031] Feature map flattening: Flatten the pooled feature map and convert it into a one-dimensional vector to facilitate input into the fully connected layer for final classification. Use the Softmax layer to predict the probability of defects in each category and output the classification result of welding defects.

[0032] Due to the adoption of the above technical solutions, the present application has the following advantages:

[0033] 1. Improve the accuracy of welding defect recognition: The present application can capture defect features at different scales by parallelly extracting image features with multi-scale convolution kernels. It can effectively identify defects regardless of their size and shape. Compared with the existing single-scale convolutional network, the present application can perform more accurate recognition in complex welding images, especially for tiny defects, thus significantly improving the accuracy of welding defect classification.

[0034] 2. Enhance the robustness to complex welding images: Through the feature fusion mechanism, the present application can effectively integrate low-level detail information and high-level abstract information, enabling the network to still stably identify welding defects in the face of image noise, illumination changes, or other interference factors. Compared with the limitations of traditional methods, the present application has stronger robustness in various complex welding scenarios and can adapt to a wider range of application environments.

[0035] 3. Improve computational efficiency and real-time performance: In this application, multi-scale parallel convolutional kernels are used to extract features. At the same time, pooling layers are combined to reduce the dimensionality of the features, significantly reducing the computational amount. Especially in industrial production environments, it can quickly process a large number of welding images, achieve real-time welding defect detection, and avoid the disadvantages of slow calculation and inability to be applied in real time of traditional deep learning models.

[0036] 4. Simplify operation and maintenance: Since this application uses a deep learning model, the training process of the model does not require manual intervention and complex feature design. It completely automatically learns features by the system. This greatly simplifies the operation complexity of traditional manual feature extraction and classification systems, improves the convenience of operation and the automation level of the system, and at the same time reduces the need for manual intervention.

[0037] 5. Reduce costs and resource consumption: Through effective feature fusion and computational optimization, this application not only improves the detection accuracy, but also increases the computational efficiency and reduces the demand for hardware resources. Especially in the deployment of industrial equipment, it can save computational resources and power consumption, improve the utilization rate and operation efficiency of the equipment. Compared with traditional methods that require a large amount of calculation and manual feature design, this application reduces the hardware and labor costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of this application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings described below are only some of the embodiments recorded in the embodiments of this application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings.

[0039] Figure 1 It is a schematic flowchart of a welding defect image preprocessing method based on multi-scale feature fusion according to an embodiment of this application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0040] The present application will be further described in conjunction with the drawings and embodiments. The described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art shall fall within the scope of protection of the embodiments of the present application.

[0041] The following technical problems mainly exist in the prior art: The accuracy of existing image processing methods is insufficient: Traditional welding defect recognition methods rely on manual feature extraction and image processing techniques, and cannot effectively capture the complex and variable defect features in welding images, resulting in low classification accuracy. Limited feature extraction ability: In the welding defect recognition of existing convolutional neural networks (CNNs), a convolutional kernel of a single size is usually used for feature extraction, resulting in insufficient recognition ability for defects of different scales, especially poor results when dealing with tiny or large defects. Lack of an effective feature fusion mechanism: Existing deep learning models lack multi-level and cross-level feature fusion, and cannot fully integrate low-level and high-level feature information, affecting the generalization ability and robustness of the model. Low computational efficiency: Traditional deep learning models have a large amount of computation, especially when dealing with high-dimensional image features, which easily leads to a slow processing speed and cannot meet the requirements of real-time detection in industrial fields. Complex defect types and image scenes: Existing methods perform poorly in multi-category defects and complex welding scenes, and are easily affected by image noise, illumination changes, and the diversity of defect types, resulting in a decline in classification effects. Low degree of automation: Traditional methods require a large amount of manual intervention and experience, and the automation level is low, and high-efficiency and seamless defect recognition and classification cannot be achieved in actual production.

[0042] In view of the problems existing in the above prior art, the present application provides a welding defect image preprocessing method based on multi-scale feature fusion, which improves the accuracy and efficiency of welding defect recognition through an innovative multi-scale convolutional kernel parallel feature extraction and feature fusion mechanism.

[0043] See Figure 1 , an embodiment of a welding defect image preprocessing method based on multi-scale feature fusion provided by the present application includes:

[0044] Step 1: Use a molten pool camera to collect various welding state data and preprocess it; various welding state data include burn-through state, over-welding state, lack-of-welding state, and full-welding state;

[0045] Step 2: Extract features from the preprocessed image through a multi-scale convolutional kernel to obtain feature maps of multiple scales;

[0046] Step 3: Extract features from the feature maps of multiple scales through a convolutional neural network model and perform feature fusion on them;

[0047] Step 4: Perform a pooling operation on the feature map after feature fusion.

[0048] Optionally, the said Step 1 includes:

[0049] Collect various welding state data using a molten pool camera, and divide the collected various welding state data into training images and test images; crop all the training images and test images so that the sizes of the cropped images are the same;

[0050] Use the Gaussian filtering algorithm to filter the cropped images to remove high-frequency noise in the images, and perform normalization on the denoised images to ensure that the pixel values of the images are within a unified numerical range.

[0051] Specifically, a total of 2584 images with a size of 1280×720 are collected, 1603 of which are used as training images and 981 as test images. Since the size of the input images is large, in order to reduce the computational amount and make the features of each image region more concentrated and clear, all the collected images are cropped, and the size of the cropped images is 512×512. The cropping operation does not change the content of the images, but only crops out a sub-image with a size of 512×512 by selecting the central region of the images. The cropping process formula is:

[0052] x cropped = Crop(x, 512, 512)

[0053] where x is the original image, and x cropped is the cropped image with a size of 512×512.

[0054] In the images during the welding process, due to the imaging characteristics of the molten pool camera, light changes, and environmental interference, there may be noise in the images, which affects the training effect of the subsequent model. To improve the image quality and reduce the impact of noise on the model performance, denoising technology is used to process the cropped images.

[0055] The denoising operation uses the Gaussian filtering algorithm, which can effectively remove high-frequency noise in the images. For the input image x cropped , its denoising operation is achieved by applying the Gaussian filter G σ :

[0056] x denoised = G σ *x cropped

[0057] where G σ is a Gaussian filter kernel with a standard deviation of σ, * represents the convolution operation, and x denoised is the denoised image. The denoising operation mainly filters out high-frequency noise, retains the low-frequency information of the images, and reduces the interference during model training.

[0058] After image denoising, normalization will be continued to ensure that the pixel values of the image are within a unified numerical range, facilitating the efficient training of the neural network. The normalization operation scales the pixel values of the image to the range [0, 1] to reduce the training instability caused by excessive differences in pixel values.

[0059] The formula for normalization is:

[0060]

[0061] where x denoised is the denoised image, and x normalized is the normalized image, and the pixel value range after normalization is [0, 1].

[0062] After the above processing, the final image set will contain 1603 training images and 981 test images. The size of each image is 512×512 and within the unified normalization range.

[0063] Optionally, step 2 includes:

[0064] The convolutional layer converts the input image into a feature map by setting different numbers and sizes of convolutional kernels and passes it to the next layer. The input image is the preprocessed image.

[0065] Optionally, the conversion of the input image into a feature map is achieved through the following formula:

[0066]

[0067] where represents the j-th feature map of the current l-th layer; represents the i-th feature map of the (l - 1)-th layer; M is the number of input feature maps; represents the number of convolutional kernels, is the bias term; f(Δ) is a non-linear activation function.

[0068] Optionally, step 3 includes:

[0069] The convolutional neural network model includes a data feature fusion layer, a depth feature fusion layer, a feature abstract representation layer, and a feature high-level representation layer: the data feature fusion layer, the depth feature fusion layer, the feature abstract representation layer, and the feature high-level representation layer are the first convolutional layer, the second convolutional layer, the third convolutional layer, and the fourth convolutional layer respectively;

[0070] The data feature fusion layer, the depth feature fusion layer, the feature abstract representation layer, and the feature high-level representation layer extract image features from different scales to simultaneously capture the local detail features and global structure features of the image, improving the recognition ability of the convolutional neural network model for different-sized defects.

[0071] The extracted multi-scale features are fused by concatenation or weighted summation to ensure the integration of low-level features and high-level features; the low-level features include edges and textures; the high-level features include shapes and defect categories.

[0072] The convolutional neural network model of this application introduces an innovative feature fusion mechanism in the traditional convolutional neural network framework. This mechanism aims to improve the accuracy and robustness of welding defect recognition by deeply fusing features at different levels and scales. The feature fusion mechanism runs through all levels of the model, from the data feature fusion layer to the high-level feature representation layer. The convolutional operation at each layer is not just local feature extraction, but through multi-dimensional information fusion, it improves the model's learning ability for complex image features.

[0073] In the design, special designs are made for features at different levels to ensure that each layer can fully learn and fuse the deep features from the previous layer, and at the same time enhance the model's expressive ability with multi-scale convolutional kernels. Specifically, through feature fusion between convolutional layers, the model not only retains the low-level features of the image, but also ensures the extraction and information integration of high-level features, making the network more robust and accurate in classification tasks.

[0074] Optionally, the data feature fusion layer extracts basic low-level features from the feature maps obtained in step 2, using multiple convolutional kernels of different sizes (such as 3×3 and 5×5) to simultaneously capture local detail features and global structure features in the feature maps; after convolution, the size of the feature maps is reduced through a pooling layer, reducing the computational complexity and improving the computational efficiency of the model. In this process, the application of the feature fusion mechanism is reflected in the combination of the output features of the two convolutional kernels. Multiple convolutional kernels act on different regions of the image simultaneously, and the features of different regions are integrated through a fusion operation, enabling the network to learn richer image features at the first layer;

[0075] The deep feature fusion layer integrates the low-level features extracted by the data feature fusion layer and the deep features learned in the current layer. The size of the convolutional kernel can be of multiple sizes (5×5 and 7×7) to more comprehensively capture the complex information in the welding image. Through the design of multi-scale convolutional kernels, the model can extract features at different scales simultaneously in one convolutional operation, thus achieving more refined feature learning. At the same time, the output after convolution is further dimension-reduced through a pooling layer, simplifying the data and reducing the number of parameters to help improve the computational efficiency.

[0076] The feature abstraction representation layer focuses on removing redundant noise in the image and abstractly representing the image features. This layer uses smaller convolutional kernels (3×3) with the aim of removing unnecessary details and noise through finer convolutional operations and further strengthening the abstract learning of important features. In particular, with smaller convolutional kernels, the network can enhance its sensitivity to minute changes in welding defects. The feature fusion mechanism at this layer is reflected in the integration of detailed features and abstract features extracted by convolutional kernels of different sizes. The pooling layer further reduces the size of the feature map while retaining important spatial information;

[0077] In the feature high-level representation layer, the convolutional neural network model removes redundant features by using larger convolutional kernels (5×5) and focuses on extracting more critical representative features. Through deep convolutional operations, this layer can further extract more important high-level features from the abstract features of the previous layer and combine these features with the low-level features extracted from the previous several layers (data feature fusion layer, depth feature fusion layer, feature abstraction representation layer). The feature fusion mechanism at this layer further improves the model's ability to identify welding defects, especially helping the model capture key information such as the shape, size, and location of the defects.

[0078] Optionally, step 4 includes:

[0079] Pooling operation: Use the max pooling layer to downsample the fused feature map. The pooling window is usually 2×2 with a stride of 2, that is, select the maximum value in each 2×2 region.

[0080] Max pooling retains the most important features by selecting the maximum value within the region while reducing the data dimension. The feature map after pooling becomes more compact, the data dimension is reduced, effectively reducing the computational amount. The pooling operation also improves the model's robustness to slight translations or noise changes in the image, making the model more adaptable.

[0081] Optionally, after step 4, it further includes:

[0082] Feature map flattening: Flatten the pooled feature map (Flatten) to convert it into a one-dimensional vector, providing input for the subsequent fully connected layer. It reduces the size of the feature map, reduces the computational amount, and improves the computational efficiency of the model. It enhances the model's robustness to noise, image changes, and defect location changes.

[0083] This application uses convolutional kernels of different sizes for parallel computing in the same convolutional layer to extract image features from different scales. This method can simultaneously capture the local detailed features and global structural features of the image, thereby improving the model's ability to identify defects of different sizes.

[0084] The extracted multi-scale features are fused by concatenation or weighted summation to ensure the effective integration of low-level features (such as edges, textures, etc.) and high-level features (such as shapes, defect categories, etc.). The fused features are more abundant, which helps to improve the accuracy of subsequent classification. After feature extraction, a pooling layer is used to downsample the feature map. The pooling operation helps to reduce the size of the feature map, lower the computational complexity, and enhance the robustness to image noise.

[0085] The data after feature extraction and pooling will be flattened and input into a fully connected layer for final classification. The Softmax layer is used to predict the probability of each type of defect, and the classification results of welding defects are output.

[0086] The Adam optimizer can also be used to train the convolutional neural network model. By gradually reducing the learning rate through a learning rate decay strategy, the training stability of the convolutional neural network model is improved and the convergence of the convolutional neural network model is accelerated.

[0087] This application significantly improves the feature representation ability and classification accuracy of welding defect images by parallelly extracting image features with multi-scale convolutional kernels and combining a feature fusion mechanism with an efficient pooling layer. This solution is applicable to the automatic recognition of welding defects and can provide an efficient and accurate defect detection solution in industrial fields.

[0088] Multi-scale convolutional kernel parallel feature extraction technology: The first key innovation of this application is to use multi-scale convolutional kernels to parallelly extract image features. By simultaneously using convolutional kernels of different sizes in the same convolutional layer, image features can be extracted from different scales, which can capture local details and global structure information simultaneously. This multi-scale feature extraction method enhances the model's ability to recognize welding defects, especially in the case of large variations in defect sizes. Protection point: The parallel use method of multi-scale convolutional kernels, especially the design of combining convolutional kernels of different sizes for feature extraction in a single convolutional layer.

[0089] Feature fusion mechanism: The feature fusion mechanism proposed in this application aims to effectively integrate the features extracted by multiple convolutional kernels. The features extracted by different convolutional kernels represent different information levels of the image. The feature fusion mechanism combines multi-level features through concatenation (concat) or weighted summation, thus forming a richer and more meaningful feature representation. This fusion method ensures the full integration of low-level detailed features and high-level abstract features, improving the accuracy in the subsequent classification stage. Protection point: Fusing the features extracted by multi-scale convolutional kernels through concatenation or weighted summation to ensure the effective integration of different-level features.

[0090] This application processes the welding image by denoising and feature extraction to help the subsequent deep learning model learn and classify welding defects more effectively. It not only supports efficient image feature extraction but also can dynamically adjust the parameters of the convolutional kernel and the feature fusion strategy according to the characteristics of different welding images. Protection point: The design of the welding defect image preprocessing device, especially the use of multi-scale convolutional kernels and the implementation of feature fusion operations in hardware devices.

[0091] Efficient computing and feature processing capabilities: To improve computing efficiency, the device designed in this application can reduce the computing time by processing multiple convolutional kernels in parallel. At the same time, the feature fusion operation ensures that there is no excessive redundancy when the network processes high-dimensional features, thus improving the overall computing efficiency and recognition accuracy. Protection point: The design of optimizing the computing process and improving the processing efficiency by parallel computing of multi-scale convolutional kernels.

[0092] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application and not to limit them. Although this application has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: modifications or equivalent replacements can still be made to the specific implementation manners of this application, and any modifications or equivalent replacements that do not depart from the spirit and scope of this application should be covered within the protection scope of the claims of this application.

Claims

1. A welding defect image preprocessing method based on multi-scale feature fusion, characterized in that: include: Step 1: Use the weld pool camera to collect various welding state data and pre-process them; Various welding status data include burn-through status, over-welding status, under-welding status and full-welding status; Step 2: Extract features from the preprocessed image using a multi-scale convolution kernel to obtain feature maps at multiple scales; Step 3: Extract features from feature maps of multiple scales through a convolutional neural network model and perform feature fusion; Step 4: Perform pooling operation on the feature map after feature fusion.

2. The method according to claim 1, characterized in that The step 1 comprises: Use a molten pool camera to collect various welding state data, and divide the collected various welding state data into training images and test images; crop all training images and test images so that the sizes of the cropped images are the same; The cropped image is filtered using Gaussian filtering algorithm to remove high-frequency noise in the image, and the denoised image is normalized to ensure that the pixel values ​​of the image are within a uniform numerical range.

3. The method according to claim 1, characterized in that The step 2 comprises: The convolution layer converts the input image into a feature map by setting convolution kernels of different numbers and sizes, and passes it to the next layer. The input image is the preprocessed image.

4. The method according to claim 3, characterized in that The input image is converted into a feature map using the following formula: in, Represents the jth feature map of the current l layer; represents the i-th feature map of layer l-1; M is the number of input feature maps; Represents the number of convolution kernels, is a bias term; f(Δ) is a nonlinear activation function.

5. The method according to claim 1, characterized in that The step 3 comprises: The convolutional neural network model includes a data feature fusion layer, a deep feature fusion layer, a feature abstract representation layer and a feature high-level representation layer: the data feature fusion layer, the deep feature fusion layer, the feature abstract representation layer and the feature high-level representation layer are respectively the first convolution layer, the second convolution layer, the third convolution layer and the fourth convolution layer; Image features are extracted from different scales through the data feature fusion layer, deep feature fusion layer, feature abstract representation layer and feature advanced representation layer to simultaneously capture the local detail features and global structural features of the image, thereby improving the convolutional neural network model's ability to recognize defects of different sizes; The extracted multi-scale features are fused by concatenation or weighted summation to ensure that low-level features are integrated with high-level features; low-level features include edges and textures; high-level features include shapes and defect categories.

6. The method according to claim 5, characterized in that The data feature fusion layer extracts basic low-level features from the feature map obtained in step 2, and uses multiple convolution kernels of different sizes to simultaneously capture local detail features and global structural features in the feature map; after convolution, the size of the feature map is reduced by the pooling layer, and multiple convolution kernels act on different regions of the feature map at the same time, and the features of different regions are integrated through fusion operations; The deep feature fusion layer integrates the low-level features extracted by the data feature fusion layer and the deep-level features learned by the current layer, and uses multi-scale convolution kernels to enable the convolutional neural network model to simultaneously extract features of different scales in one convolution operation. The output after convolution is processed by dimensionality reduction through the pooling layer; The feature abstraction representation layer removes redundant noise in the feature map and abstractly expresses the features of the feature map. It uses a smaller convolution kernel than the convolution kernel used in the deep feature fusion layer to remove redundant details and noise through a finer convolution operation. The pooling layer reduces the size of the feature map while retaining important spatial information. In the feature high-level representation layer, the convolutional neural network model removes redundant features and extracts more critical representative features by using a convolution kernel larger than the convolution kernel used in the feature abstract representation layer. Through deep convolution operations, more important high-level features are extracted from the abstract features of the feature abstract representation layer, and the high-level features are combined with the low-level features extracted by the data feature fusion layer, the deep feature fusion layer, and the feature abstract representation layer.

7. The method according to claim 1, characterized in that The step 4 comprises: Pooling operation: Use the maximum pooling layer to downsample the fused feature map and select the maximum value in the area within each pooling window.

8. The method according to claim 1, characterized in that After step 4, the method further includes: Feature map flattening: Flatten the pooled feature map and convert it into a one-dimensional vector to facilitate input into the fully connected layer for final classification. Use the Softmax layer to predict the probability of defects in each category and output the classification results of welding defects.

Images