Welding defect identification method and platform for few samples

Abstract

The invention discloses a few-sample-oriented welding defect identification method and platform, and relates to the technical field of welding defect identification, and the method comprises the following steps: collecting and marking an original welding image, collecting a molten pool gray scale image with an optical filter in real time through a visual system in a welding process, and adjusting and generating various defect images in combination with process parameters; the actual welding defect is used as a label; the welding original image is preprocessed, wherein welding pool area cutting, data enhancement operation and standardized data division are included; performing staged training on a welding defect identification model, extracting spatial features through a backbone network, performing cross fusion on horizontal and vertical features through a multi-dimensional feature fusion module to enhance molten pool positioning, and generating high-order statistical features through a Brownian distance covariance matrix module; the method comprises the following steps: identifying defects in an actual welding process, constructing a prototype welding network space based on a meta-learning strategy, generating class prototype vectors of various defects, realizing classification of query images through similarity measurement, and dynamically adjusting process parameters.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of welding defect recognition, and particularly relates to a welding defect recognition method and platform for few-shot learning. BACKGROUND

[0002] Welding, as a basic process in modern manufacturing, plays an irreplaceable key role in the field of equipment manufacturing. However, due to factors such as low assembly accuracy and heat input fluctuation, various defects often occur at the welded joints, which directly affect the overall integrity and service reliability of the welded structure. Welding defect recognition based on molten pool images is the most commonly used real-time defect recognition method at present, mainly including traditional machine learning methods and deep learning methods. Machine learning methods rely on data preprocessing and feature engineering, which are low in efficiency and difficult to cope with noise and light changes. Deep learning methods significantly improve recognition accuracy and robustness through automatic feature extraction and end-to-end learning. However, there are still challenges, such as the complexity of the welding process, the difficulty of obtaining high-quality images, the high cost and time-consuming of manual annotation, which limits the number of samples, and the imbalance of class sample quantity, which leads to model bias towards classes with more samples.

[0003] To solve the above problems, researchers have introduced few-shot learning methods, especially the metric-based method has received widespread attention. This method learns image representation through similarity measurement. The image features extracted by the network can be regarded as random vectors in a high-dimensional space, and their similarity is usually measured by statistical features such as mean and covariance combined with Euclidean distance or KL divergence. However, these methods mainly consider the marginal distribution, ignoring the joint distribution and the nonlinear relationship between variables. Although some methods try to use joint distribution, the computational cost is high and not suitable for real-time applications. Therefore, it is urgent to propose a welding defect recognition method and platform for few-shot learning. SUMMARY

[0004] To solve the above technical problems, the present application proposes a welding defect recognition method and platform for few-shot learning, which introduces Brown distance covariance metric into the welding defect recognition network, effectively models and extracts the nonlinear dependence between the input welding image and the welding defect class prototype vector, and realizes high-accuracy classification in complex scenarios.

[0005] In one aspect to achieve the above object, the present application provides a welding defect recognition method for few-shot learning, comprising:

[0006] Welding original image acquisition and labeling, real-time acquisition of molten pool gray scale images with filter through a vision system during the welding process, generation of multi-class defect images combined with process parameter adjustment, and actual welding defects as labels;

[0007] The welding original image preprocessing includes molten pool region clipping, data enhancement operation and standard data division.

[0008] The phased training of the welding defect recognition model extracts spatial features through the backbone network, cross-fuses horizontal and vertical direction features through the multi-dimensional feature fusion module to enhance molten pool positioning, and generates high-order statistical features through the Brown distance covariance matrix module.

[0009] The defect recognition in the actual welding process is based on a meta-learning strategy to construct a prototype welding network space, generate class prototype vectors of various defects, realize the classification of the query image through similarity measurement, and dynamically adjust the process parameters.

[0010] Optionally, in the welding original image acquisition and labeling, the visual system includes a camera integrated with a neutral density filter and a band-pass filter, and a lateral illumination diode laser for reducing plasma interference; the actual welding defect label is determined by observing the weld surface, back surface and cross section after welding.

[0011] Optionally, in the welding original image preprocessing, the data enhancement operation includes flipping, adding Gaussian noise, and randomly adjusting brightness and contrast of the molten pool image; after the data set is divided, the standard training data set is constituted by standardizing the number of samples of each category.

[0012] Optionally, the backbone network adopts a four-group residual block stacking structure, each residual block contains a 3x3 convolution kernel, a group normalization layer and a residual connection, which is used to output a first welding feature vector.

[0013] Optionally, the operation of the multi-dimensional feature fusion module includes: performing horizontal and vertical direction pooling on the first welding feature vector respectively, generating a first spatial weight feature vector through matrix multiplication fusion, generating a second spatial weight feature vector through convolution, group normalization and Sigmoid activation, and multiplying the first welding feature vector with the second welding feature vector channel by channel to output a second welding feature vector.

[0014] Optionally, the operation of the Brown distance covariance matrix module includes: calculating the Euclidean distance matrix of the input feature vector, subtracting the row and column mean values through centering processing, and generating a welding feature vector representing the nonlinear dependence relationship.

[0015] Optionally, in the phased training, the meta-training phase constructs a support set and a query set according to an N-way K-shot task; the class prototype vector is generated by averaging the feature vectors of the same class support set samples; the similarity measurement is realized by calculating the matrix trace inner product of the query feature vector and the class prototype vector.

[0016] In another aspect to achieve the above object, the present application also provides a welding defect recognition platform for few-shot learning, comprising:

[0017] Visual system, industrial computer, edge controller, motion mechanism, laser welding head, fiber laser, protective gas device, IO module;

[0018] The visual system includes a camera and a diode laser for illumination, the camera is installed on the coaxial optical port of the laser welding head to capture real-time images of the molten pool, the diode laser is installed on the side of the laser welding head to irradiate the area to be welded, the industrial computer pre-processes the collected data and transmits the processing result to the edge controller, the edge controller judges the current welding defect category through the deployed model and feeds back the judgment result to the industrial computer, at the same time, the industrial computer calculates and issues the position data of the motion mechanism according to the teaching trajectory, receives the collection data from the camera and issues the control signal to the IO module to adjust the power size of the fiber laser, the flow of the protective gas device and the real-time triggering of the camera collection signal.

[0019] The technical effect of the present application is that a welding defect recognition method and platform are disclosed, a meta-learning training strategy is adopted, a prototype welding network space is constructed, each welding defect class prototype vector is generated, and the similarity between the input welding image and the welding defect class prototype vector is used to recognize the category of the welding defect image. Compared with the traditional method, this method performs even better under sparse samples, significantly reduces the dependence on large-scale labeled data, and improves the adaptability and robustness of the system in complex welding environments. The Brown distance covariance metric is introduced into the welding defect recognition network, effectively modeling and extracting the nonlinear dependence between the input welding image and the welding defect class prototype vector, achieving high accuracy classification in complex scenarios while ensuring the simplicity and efficiency of the calculation process, suitable for real-time scenarios. A multi-dimensional feature fusion module is designed, which effectively identifies and locates the molten pool area by cross-fusing horizontal and vertical direction features, significantly improving the feature extraction capability under limited samples. BRIEF DESCRIPTION OF DRAWINGS

[0020] The accompanying drawings, which form a part of this application, are included to provide a further understanding of the application and are incorporated in and constitute a part of this application. The embodiments of this application and their description together with the drawings serve to explain the application. In the drawings:

[0021] Figure 1 The molten pool images corresponding to various welding defects of the embodiments of the present application;

[0022] Figure 2 The flowchart of a welding defect recognition method for few samples of an embodiment of the present application;

[0023] Figure 3 The model structure diagram of an embodiment of the present application;

[0024] Figure 4 A schematic diagram of a backbone network of an embodiment of the present application;

[0025] Figure 5 A schematic diagram of a multi-dimensional feature fusion (MFF) module of an embodiment of the present application;

[0026] Figure 6 A schematic diagram of a model stage-by-stage training method of an embodiment of the present application;

[0027] Figure 7 A test result diagram of an embodiment of the present application;

[0028] Figure 8 A schematic diagram of a welding defect recognition platform of an embodiment of the present application;

[0029] Figure 9 A physical diagram of a welding defect recognition platform of an embodiment of the present application. DETAILED DESCRIPTION

[0030] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments.

[0031] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a group of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown herein.

[0032] As shown in Figures 1-7 , the present embodiment provides a welding defect recognition method for few-shot learning, which comprises:

[0033] Step S1: Referring to Figure 1 , welding original image acquisition and labeling, specifically, real-time acquisition of welding original images in the welding process through a vision system, six types of defect corresponding welding original images are obtained by changing laser power, offset and other parameters, and actual welding defects are used as model labels;

[0034] Specifically, the welding original image is a gray-scale image captured by a vision system, the vision system includes a camera and a diode laser for illumination, the camera further includes a neutral density filter and an 808 nanometer bandpass filter, which are integrated into the camera lens, the camera captures images at a speed of 50 frames per second, the size of each original image is 384x340 pixels, the diode laser power is set to 3W, and the working wavelength is 808 nanometers, which can illuminate the welding area and reduce the plasma interference generated during the welding process, the actual welding defects are accurately obtained by observing the welding surface, back surface and cross section after the welding is completed, and are used as labels for model training;Figure 1 It can be seen that the actual welding defects are set here to include incomplete penetration, good, excessive penetration, burn-through, weld offset and weld misalignment, and each set of process parameters is tested multiple times to obtain sufficient data.

[0035] Step S2: Welding original image preprocessing, including:

[0036] Step S21: Region of interest cropping, specifically, the welding original image is cropped to extract the relevant part of the molten pool and reduce the image size, obtaining a molten pool image;

[0037] Step S22: Data augmentation, specifically, the molten pool image is flipped, Gaussian noise is added, the brightness and contrast of the image are randomly adjusted;

[0038] Step S23: Label assignment, specifically, an integer sequence starting from 0 is used as the label of the molten pool image, representing the defect category of the image, 0 represents incomplete penetration, 1 represents good, 2 represents excessive penetration, 3 represents burn-through, 4 represents weld offset, and 5 represents weld misalignment.

[0039] Step S24: Dataset division, the dataset contains a known sub-dataset and an unknown sub-dataset, the known sub-dataset includes 35,000 stainless steel 304 molten pool images, and the 35,000 labeled molten pool images are randomly divided into a preliminary training set, a validation set and a test set according to a ratio of 0.8:0.1:0.1. The unknown sub-dataset contains 21,133 stainless steel 304 molten pool images, and the unknown sub-dataset is collected under different welding process parameters from the known sub-dataset, and is specifically used to test the performance of the method. Subsequently, the number of molten pool images of each defect category in the preliminary training set is standardized to be the same, forming a standard training dataset D L .

[0040] Step S3: Refer to Figures 3-6 , the architecture design of the welding defect recognition model and the phased training strategy include the following steps:

[0041] Step S31: Refer to Figure 3 , the architecture design of the welding defect recognition model includes a backbone network, a multi-dimensional feature fusion module (MFF), and a Brown distance covariance matrix module (BDCM);

[0042] Step S311: Refer to Figure 4 , the backbone network adopts a four-group residual block stacking structure, each residual block contains S 3x3 convolution kernels, a group normalization layer and a residual connection, which is used to extract the spatial features of the molten pool image and obtain a first welding feature vector;

[0043] Step S312: Referring to 5, the MFF module extracts the spatial weight feature through bidirectional pooling, and the specific algorithm is as follows:

[0044] The first welding feature vector is subjected to horizontal direction and vertical direction pooling operations, the horizontal direction generates a horizontal feature vector, and the vertical direction generates a vertical feature vector;

[0045] The horizontal feature vector and the vertical feature vector are fused through matrix multiplication to generate a first spatial weight feature vector, and the calculation formula is:

[0046]

[0047] wherein, is the first spatial weight feature vector, z x is the horizontal feature vector, z y is the vertical feature vector;

[0048] The first spatial weight feature vector is subjected to 2D convolution, group normalization and sigmoid activation function to generate a second spatial weight feature vector, and the calculation formula is:

[0049]

[0050] wherein, f is the second spatial weight feature vector, σ is the sigmoid activation function, GN is the group normalization operation, and Conv is the 2D convolution operation;

[0051] The second spatial weight feature vector is multiplied with the first welding feature vector channel by channel to obtain a second welding feature vector M out output by the MFF module, and the calculation formula is:

[0052]

[0053] wherein, M out is the second welding feature vector, is the first welding feature vector;

[0054] Step S313: The BDCM module generates high-order statistical features based on the Brown distance covariance matrix, and the specific algorithm is as follows:

[0055] For two pool images, after being extracted by the backbone network, vectors A ∈ R p and B ∈ R q are obtained, the dimensions of which are p and q respectively, and it is assumed that wherein is the Euclidean distance matrix calculated from the discrete observations of vector A, and m is the number of observations. Similarly, the Euclidean distance matrix is calculated, wherein n is the number of observation values. In the following discussion, it is assumed that m is equal to n. The form of the Euclidean distance matrix is as follows:

[0056]

[0057] The matrix X = (x kl ) is called a welding feature vector. Specifically, x kl is defined as follows:

[0058]

[0059] where the last three terms are the matrix lth column, the kth row, and the mean of all elements, respectively. The matrix Y can also be calculated in a similar manner, so that the input molten pool image is converted into a welding feature vector by the welding defect recognition model;

[0060] Step S32: Referring to Figure 6 , the welding defect recognition model divides the training strategy into stages, including the following steps:

[0061] Step S321: Pre-training stage, used to initialize the network parameters of the welding defect recognition model, and the standard training data set D L is input into the model in batches, and the defect category prediction is output by the fully connected classifier;

[0062] Step S322: Meta-training stage, based on the N-way K-shot task construction strategy, N defect categories are randomly selected for each training task, each category contains K support samples and L query samples, respectively, to form the support set and the query set where y j is the class label of the specific molten pool image z j .

[0063] Step S3221: Generation of welding feature vectors of query samples, in each training task, N×L molten pool images in the query set D que are input into the welding defect recognition model to generate welding feature vectors;

[0064] Step S3222: Generation of class prototype vectors, in each training task, K molten pool images of the same class in the support set D sup are input into the welding defect recognition model to generate class prototype vectors, including:

[0065] Specifically, support set sample screening, K molten pool images of a certain class are selected from the support set D sup to form a subset;

[0066] ​Specifically, the welding feature vector extraction encodes each specific molten pool image in the subset through the welding defect recognition model to obtain a welding feature vector;

[0067] Specifically, the mean value of the welding feature vectors corresponding to the molten pool images of the same class is calculated as the class prototype vector, and the calculation formula is as follows:

[0068]

[0069] Wherein, v represents a certain class in the support set D sup , M v is the class prototype vector, K is the number of support samples, O v is the subset, X θ (z j ) is the welding feature vector, and θ is the network parameter;

[0070] Step S3223: Similarity measurement, extracting the welding feature vector of the molten pool image in the query set D que , and calculating the similarity score of each class prototype vector, the calculation formula is:

[0071] s j，v =τ·tr(X θ (z j ) T M v );

[0072] Wherein, s j，v is the similarity score between the welding feature vector of the molten pool image and the class v prototype vector, tr(·) represents the trace of the matrix, T represents the transpose of the matrix, τ is a learnable scaling parameter, and here the similarity scores between six class prototype vectors (un-melted, good, over-melted, burn-through, weld offset and weld misalignment) need to be calculated.

[0073] Since the Brown distance covariance matrix is symmetric, the similarity score can be regarded as the inner product of the vectorization of the upper triangular part of the two matrices, which simplifies the calculation.

[0074] Step S3224: Classification decision, including:

[0075] Specifically, the probability distribution generation converts the similarity score into a class probability distribution through the softmax function, and the calculation formula is:

[0076]

[0077] Specifically, the welding defect classification decision selects the defect class corresponding to the maximum probability according to the class probability distribution of the query sample as the recognition result, and the calculation formula is:

[0078]

[0079] Probability of the under-penetration category If the probabilities of the remaining categories, such as good, over-penetration, burn-through, weld offset and weld misalignment, are all less than 0.8, the current recognition result is determined as the under-penetration defect category.

[0080] Specifically, the loss is optimized to measure the error between the predicted probability and the true label by using a cross-entropy loss function, and the network parameters are updated by a gradient descent algorithm, and the calculation formula is:

[0081]

[0082] Step S4: The defect recognition of the actual welding process is specifically:

[0083] The trained model is deployed in the edge controller. In the actual welding process, the camera continuously collects the welding original image, and the collected image is first transmitted to the industrial computer for image preprocessing, and the preprocessed data is then transmitted to the edge controller. Combined with the deployed model, the system identifies the welding defect type in real time, and transmits the recognition result to the industrial computer for real-time recording and display. Once a defect is detected, the system automatically triggers an early warning feedback in time, and dynamically adjusts the corresponding process parameters according to the defect type to optimize the welding quality.

[0084] Reference Figure 7 The present application has achieved excellent welding defect recognition effect in various welding scenes. Using only 5% of the labeled data, high-accuracy results are achieved, and excellent performance and generalization ability are exhibited in complex environments, which has broad application prospects in real-time quality monitoring of industrial welding defects.

[0085] In the present application, a welding defect recognition method oriented to few-shot samples is proposed. A meta-learning training strategy is adopted to construct a prototype welding network space, generate prototype vectors of each welding defect class, and identify the class of the welding defect image according to the similarity between the input welding image and the welding defect class prototype vector. Compared with traditional methods, this method performs even better under sparse samples, significantly reduces the dependence on large-scale labeled data, and improves the adaptability and robustness of the system in complex welding environments.

[0086] In the present application, the Brown distance covariance measure is introduced into the welding defect recognition network, effectively modeling and extracting the nonlinear dependence relationship between the input welding image and the welding defect class prototype vector, achieving high-accuracy classification in complex scenes, while ensuring the simplicity and efficiency of the calculation process, and being suitable for real-time scenarios.

[0087] In this invention, a multi-dimensional feature fusion module is designed. By cross-fusing horizontal and vertical features, the melt pool region can be effectively identified and located, significantly improving the feature extraction capability under limited sample conditions.

[0088] like Figures 8-9 As shown, this embodiment also provides a welding defect identification platform for small sample sizes, including:

[0089] Vision systems, industrial computers, edge controllers, motion mechanisms, laser welding heads, fiber lasers, protective gas devices, and I / O modules;

[0090] The vision system includes a camera and a diode laser for illumination. The camera is mounted on the coaxial optical port of the laser welding head to capture real-time images of the molten pool. The diode laser is mounted on the side of the laser welding head to illuminate the area to be welded, reducing plasma interference during the welding process. The industrial control computer preprocesses the acquired data and transmits the processing results to the edge controller. The edge controller determines the current welding defect category based on the deployed model and feeds back the determination result to the industrial control computer. Simultaneously, the industrial control computer calculates and sends the position data of the motion mechanism based on the taught trajectory, receives the acquired data from the camera, and sends control signals to the I / O module to adjust the power of the fiber laser, the flow rate of the protective gas device, and the real-time trigger signal of the camera's acquisition signal.

[0091] This embodiment proposes a welding defect recognition method and platform for limited sample sizes. It employs a meta-learning training strategy to construct a prototype welding network space, generating prototype vectors for each welding defect class. The category of the welding defect image is then identified based on the similarity between the input welding image and the prototype vectors of the welding defect classes. Compared to traditional methods, this method performs even better with sparse samples, significantly reducing the dependence on large-scale labeled data and improving the system's adaptability and robustness in complex welding environments. By introducing the Brownian distance covariance metric into the welding defect recognition network, the nonlinear dependency between the input welding image and the prototype vectors of the welding defect classes is effectively modeled and extracted, achieving high-accuracy classification in complex scenarios while maintaining a simple and efficient computational process, making it suitable for real-time scenarios. A multi-dimensional feature fusion module is designed, which effectively identifies and locates the molten pool region by cross-fusing horizontal and vertical features, significantly improving feature extraction capabilities under limited sample conditions.

[0092] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A welding defect recognition method for few-shot learning, characterized by, Comprise: Welding original image acquisition and labeling, real-time acquisition of gray images of molten pool with filter during welding process through visual system, combined with process parameter adjustment to generate multi-class defect images, and actual welding defects as labels; Welding original image preprocessing, including molten pool region clipping, data enhancement operation and standardized data division; Stage-by-stage training of welding defect recognition model, through backbone network to extract spatial features, cross fusion of horizontal and vertical direction features through multi-dimensional feature fusion module to enhance molten pool positioning, and high-order statistical features generated through Brown distance covariance matrix module; Defect recognition in actual welding process, based on meta-learning strategy to construct prototype welding network space, generate class prototype vectors of various defects, realize classification of query images through similarity measurement, and dynamically adjust process parameters.

2. The few-shot oriented welding defect recognition method of claim 1, wherein In the welding original image acquisition and labeling, the visual system comprises a camera integrated with neutral density filter and band-pass filter, and a lateral illumination diode laser for reducing plasma interference; the actual welding defect label is determined through post-welding observation of weld surface, back surface and cross section.

3. The few-shot oriented welding defect recognition method of claim 1, wherein In the welding original image preprocessing, the data enhancement operation includes flipping, adding Gaussian noise, randomly adjusting brightness and contrast of the molten pool image; after data set division, standard training data set is constituted by standardizing the number of samples in each category.

4. The few-shot oriented welding defect recognition method of claim 1, wherein The backbone network adopts four groups of residual block stacking structure, each residual block contains 3x3 convolution kernel, group normalization layer and residual connection, for outputting first welding feature vector.

5. The few-shot oriented welding defect recognition method of claim 1, wherein The operation of the multi-dimensional feature fusion module includes: performing horizontal and vertical direction pooling on the first welding feature vector respectively, generating first spatial weight feature vector through matrix multiplication fusion, generating second spatial weight feature vector through convolution, group normalization and Sigmoid activation, and multiplying the first welding feature vector with the second spatial weight feature vector channel by channel to output second welding feature vector.

6. The few-shot oriented welding defect recognition method of claim 1, wherein The operation of the Brown distance covariance matrix module includes: calculating the Euclidean distance matrix of the input feature vector, subtracting the row and column mean and global mean through centering processing, and generating the welding feature vector representing the nonlinear dependence relationship.

7. The few-shot oriented welding defect recognition method of claim 1, wherein In the stage-by-stage training, the meta-training stage constructs support set and query set according to N-way K-shot task; the class prototype vector is generated by averaging the feature vectors of the same class support set samples; the similarity measurement is realized by calculating the matrix trace inner product of the query feature vector and the class prototype vector.

8. A platform for few-shot oriented welding defect recognition method according to any one of claims 1-7, characterized in that, Comprise: Visual system, industrial computer, edge controller, motion mechanism, laser welding head, fiber laser, protective gas device, IO module; The visual system includes a camera mounted on the coaxial optical port of the laser welding head for capturing real-time images of the molten pool, and a diode laser for illumination mounted on the side of the laser welding head, which irradiates the area to be welded. The industrial computer pre-processes the collected data and transmits the processing results to the edge controller. The edge controller determines the current welding defect category through the deployed model and feeds back the determination result to the industrial computer. At the same time, the industrial computer calculates and issues the position data of the motion mechanism according to the teaching trajectory, receives the collected data from the camera, and issues control signals to the IO module to adjust the power of the fiber laser, the flow of the shielding gas device, and the real-time triggering of the camera acquisition signal.

Images