A method and system for monitoring laser welding quality

Abstract

The present invention discloses a method and system for monitoring the quality of laser welding. The laser beam is measured by a photodiode and a high-speed camera, and the beam is focused on the surface of the workpiece by a scanner. At each sampling time, the voltage of the photodiode amplifier is recorded, and the image of the high-speed camera is recorded to complete data acquisition. Based on the data acquisition, the data is preprocessed, the high-speed camera image is cropped, and each cropped high-speed camera image is assigned to a high-speed camera sample. Through a cascade system, a multi-sensor combined feature photodiode block, a feature high-speed camera block, a classification photodiode block, and a classification high-speed camera block are used to monitor the quality of laser welding through these four blocks; the prediction result of the laser welding quality monitoring is made through the machine learning method of feature engineering and decision tree or by a deep neural network. The inference time is effectively shortened, and the data inference is made more accurate.

Description

Technical Field

[0001] The present invention relates to the technical field of quality monitoring, and particularly relates to a laser welding quality monitoring method and system. Background Art

[0002] Laser welding has the characteristics of fast welding speed, narrow heat affected zone, high penetration depth and high automation degree, and plays an important role in many industries such as automobile and shipbuilding.

[0003] However, in the process of implementing the technical solution of the present invention in the embodiments of the present application, the inventors found that the above technologies have at least the following technical problems:

[0004] However, in the actual welding process, welding defects such as cracks and pores will inevitably occur, and one welding defect may cause the failure of the entire component. In order to quickly detect defective components, quality monitoring of industrial processes is carried out, and signal data is collected through sensors such as photodiodes, spectrometers, ultraviolet sensors, X-ray sensors, and high-speed cameras. Then, different processing algorithms are used for analysis, including machine learning methods such as decision trees and support vector machines, as well as single-sensor systems (SSS) or multi-sensor systems (MSS) combined with convolutional neural networks to detect the welding process. However, in the prior art, laser welding has problems that in a complex multi-sensor system, due to the long inference time, the quality monitoring time may far exceed the production time, resulting in the impact on production output and inaccurate data. Summary of the Invention

[0005] By providing a laser welding quality monitoring method and system in the embodiments of the present application, the problems of long inference time and inaccurate data in laser welding in the prior art are solved, and the effective shortening of the inference time and more accurate data inference are realized.

[0006] The embodiments of the present application provide a laser welding quality monitoring method, including:

[0007] S1. Measuring the laser beam through a photodiode and a high-speed camera, focusing the beam on the workpiece surface by a scanner, and at each sampling time, recording the voltage of the photodiode amplifier and the image of the high-speed camera to complete data acquisition;

[0008] S2. Based on the data acquisition, preprocessing the data, cropping the high-speed camera images, and assigning each cropped high-speed camera image to a high-speed camera sample, and performing data augmentation operations of rotation and flipping on the high-speed camera images through block-by-block marking;

[0009] S3. Through a cascade system, combining multi-sensor features of a photodiode block, a feature high-speed camera block, a classification photodiode block, and a classification high-speed camera block, and monitoring the laser welding quality through these four blocks;

[0010] S4. Make a prediction result for the laser welding quality monitoring through machine learning methods of feature engineering and decision trees or by deep neural networks.

[0011] Furthermore, use a fiber laser with an infrared wavelength of 1070 nm to measure light with a wavelength of 300–950 nm in the welding area, focus the beam onto the workpiece surface through a scanner, record the voltage of the photodiode amplifier at each sampling time, and record the images of the high-speed camera;

[0012] The time series sampling rate of the photodiode is 250 kHz, one sample is recorded every 4 μs, and the sampling rate of the high-speed camera is 20 kHz, and one image is generated every 50 μs.

[0013] Furthermore, the definition of the cascade system CS is as follows:

[0014] ;

[0015] Let be the output of the classifier for pre-judgment based on the photodiode signal.

[0016] When p < 0.5, the classifier selects an anomaly, and when p ≥ 0.5, it selects a reference. The closer p is to 0 or 1, the higher the decision confidence of the classifier;

[0017] Let be a fixed threshold. If p < r or , then the decision of the classifier is definite;

[0018] If the first condition is satisfied, the first condition is , then it is considered that the classifier's judgment of the anomaly is acceptable. If the second condition is satisfied, the second condition is then it is considered that the classifier's judgment of the reference is acceptable. If the result of the classifier is uncertain, a final decision is made based on the high-speed camera data in the next step.

[0019] Furthermore, calculate seven statistical features and the median on the photodiode signal block for each high-speed camera image;

[0020] Calculate a binary mask image with the same size as the high-speed camera image for each high-speed camera image. The pixel values in the mask are considered 1 at those positions where the pixel values in the high-speed camera image are greater than or equal to h, and 0 at positions where the high-speed camera pixel values are smaller;

[0021] According to h, the binary mask can contain information about the size, shape or spatter of the weld, and 11 geometric features are extracted, including area, number of regions, area of the largest region, ratio of the largest region to the area, convex hull, ratio of area to convex hull, perimeter, ratio of perimeter to area, area of the fitted ellipse, length of the ellipse and width of the ellipse;

[0022] Through different experience thresholds ,Each image can have several geometric features;

[0023] After feature extraction, DT is used for classification and the quality of the cracks is measured using the CART algorithm;

[0024] The algorithm uses the Gini index to select features. It randomly selects two samples from the data set, calculates the probability that their categories are inconsistent, and selects the feature with the smallest Gini index.

[0025] Based on the selected features, the dataset is divided into different subsets, and a decision tree is recursively generated for each subset.

[0026] Furthermore, the feature photodiode block and the feature high-speed camera block are composed of convolutional layers, and the classification photodiode block and the classification high-speed camera block are composed of fully connected layers;

[0027] Feature blocks are composed of composition, It is a 2D convolutional layer with a filter size of 3×3, followed by batch normalization, max pooling, ReLU as the activation function, and k is the number of filters;

[0028] The classification block consists of a flattening layer and a fully connected layer DI, where D represents the weight matrix of the fully connected layer and l is a neuron;

[0029] When 1≠1, the ReLU activation function is used, and when 1=1, the Sigmoid activation function is used, and the dropout rate of the dropout layer is 0.5;

[0030] During the training of the neural network, binary cross entropy is used as the loss function and the weights are initialized with random values.

[0031] A laser welding quality monitoring system, comprising:

[0032] The data acquisition module is used to measure the laser beam through a photodiode and a high-speed camera, focus the beam on the workpiece surface using a scanner, and record the voltage of the photodiode amplifier and the image of the high-speed camera at each sampling time to complete the data acquisition;

[0033] The data preprocessing module is used to preprocess data based on data acquisition, crop the high-speed camera images, and assign each cropped high-speed camera image to a high-speed camera sample. Through block-by-block labeling, data augmentation operations of rotating and flipping the high-speed camera images are performed;

[0034] The quality monitoring module is used to perform laser welding quality monitoring through a cascaded system, a multi-sensor combined feature photodiode block, a feature high-speed camera block, a classification photodiode block, and a classification high-speed camera block;

[0035] The prediction result module is used to make a prediction result for laser welding quality monitoring through machine learning methods of feature engineering and decision trees or by a deep neural network.

[0036] Furthermore, a fiber laser with an infrared wavelength of 1070 nm is used to measure light with a wavelength of 300–950 nm in the welding area. The beam is focused on the workpiece surface through a scanner. At each sampling time, the voltage of the photodiode amplifier is recorded, and the images of the high-speed camera are recorded;

[0037] The time series sampling rate of the photodiode is 250 kHz, and one sample is recorded every 4 μs. The sampling rate of the high-speed camera is 20 kHz, and one image is generated every 50 μs.

[0038] Furthermore, the definition of the cascaded system CS is as follows:

[0039] ;

[0040] Let be the output of the classifier for pre-judgment based on the photodiode signal.

[0041] When p < 0.5, the classifier selects an anomaly. When p ≥ 0.5, it selects a reference. The closer p is to 0 or 1, the higher the decision confidence of the classifier;

[0042] Let be a fixed threshold. If p < r or , the decision of the classifier is certain;

[0043] If the first condition is satisfied, the first condition is , it is considered that the classifier's judgment of an anomaly is acceptable. If the second condition is satisfied, the second condition is then it is considered that the classifier's judgment of a reference is acceptable. If the result of the classifier is uncertain, a final decision is made based on the high-speed camera data in the next step.

[0044] Further, calculate seven statistical features and the median on the photodiode signal block for each high-speed camera image;

[0045] Calculate a binary mask image with the same size as the high-speed camera image for each high-speed camera image. The mask is considered to have a value of 1 at those positions where the pixel value in the high-speed camera image is greater than or equal to h, and a value of 0 at positions where the high-speed camera pixel value is smaller;

[0046] According to h, the binary mask can contain information about the weld size, shape, or spatter, and extract 11 geometric features including area, number of regions, area of the largest region, ratio of the largest region to the area, convex hull, ratio of the area to the convex hull, perimeter, ratio of the perimeter to the area, area of the fitted ellipse, length of the ellipse, and width of the ellipse;

[0047] Through different empirical thresholds Each image can have several geometric features;

[0048] After feature extraction, use DT for classification and measure the quality of cracks using the CART algorithm;

[0049] The Gini index is used in the algorithm to select features. Randomly select two samples from the dataset, calculate the probability of class inconsistency, and select the feature with the smallest Gini index;

[0050] According to the selected features, divide the dataset into different subsets and recursively generate decision trees for each subset.

[0051] Further, the feature photodiode block and the feature high-speed camera block are composed of convolutional layers, and the classification photodiode block and the classification high-speed camera block are composed of fully connected layers;

[0052] The feature block consists of consisting of is a 2D convolutional layer with a filter size of 3×3, followed by batch normalization, max pooling, and ReLU as the activation function, where k is the number of filters;

[0053] The classification block consists of a flattening layer and a fully connected layer DI, where D represents the weight matrix of the fully connected layer, is the neuron;

[0054] When ReLU activation function is used, and when Sigmoid activation function is used, and the dropout rate of the dropout layer is 0.5;

[0055] During the training process of the neural network, use binary cross-entropy as the loss function and initialize with random weights.

[0056] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0057] Due to the proposed cascade system, quality monitoring can be carried out quickly and accurately through a two-stage structure; the first-stage structure uses classical machine learning of feature engineering and DT to check and analyze simple data such as time series, and makes a definite classification of some welds. In the uncertain field, the second-stage structure uses a convolutional network to make a final decision based on image data. Practice has proved that CS can outperform SSS in terms of accuracy and inference time. Compared with MSS, the data of different sensors in CS do not have to be transmitted to general-purpose hardware, thus effectively reducing the inference time and effectively improving productivity. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 is a flowchart of a laser welding quality monitoring method;

[0059] Figure 2 is a schematic diagram of a device for laser welding quality monitoring;

[0060] Figure 3 is a schematic diagram of two single-sensor systems;

[0061] Figure 4 is a schematic diagram of a multi-sensor system;

[0062] Figure 5 is a schematic diagram of a cascade system. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0063] In order to better understand the above technical solutions, the above technical solutions will be described in detail below in conjunction with the accompanying drawings of the specification and specific embodiments.

[0064] See Figure 1 , S1. Measure the laser beam through a photodiode and a high-speed camera, use a scanner to focus the beam on the workpiece surface, and at each sampling time, record the voltage of the photodiode amplifier and record the image of the high-speed camera to complete data acquisition;

[0065] Specifically, the actual equipment of the entire system is as Figure 2 shown. A 2D galvanometer scanner using two mirrors is used to guide the laser beam generated in the laser to two thin metal plates. The data acquisition device uses two measurement systems, a photodiode (PD) and a high-speed camera (HSC). A fiber laser with an infrared wavelength of 1070 nm is used to measure the light with a wavelength of 300–950 nm in the welding area, and then the scanner is used to focus the beam on the workpiece surface. At each sampling time, the voltage of the photodiode amplifier is recorded and the image of the high-speed camera is recorded.

[0066] The time series sampling rate of the photodiode is 250 kHz, and a sample is recorded every 4 μs. The sampling rate of the high-speed camera is 20 kHz, and an image is generated every 50 μs. The higher sampling rate of the photodiode allows for the detection of shorter anomalies, and the original data volume is smaller compared to the high-speed camera images, enabling faster processing. The high-speed camera images provide geometric shape information not present in the photodiode signals and have spatial resolution.

[0067] S2. Based on data acquisition, preprocess the data. Crop the high-speed camera images and assign each cropped high-speed camera image to a high-speed camera sample. Through block-by-block labeling, perform data augmentation operations of rotation and flipping on the high-speed camera images;

[0068] Specifically, the high-speed camera sample represents a block, and a label is assigned to each block. Determine whether the label reference is abnormal. Through block-by-block labeling, defects can be located along the welding path, and data augmentation operations of rotation and flipping are performed on the high-speed camera images.

[0069] Preprocess the data for subsequent system use. First, crop the high-speed camera images to a size of 100×100 pixels and scale them to a value range of [0,1]. Then, assign each high-speed camera image to 13 photodiode samples, which represent a block, and assign a label to each block, namely reference or abnormal. Abnormal refers to the location where anomalies such as gaps or spatter are introduced, and reference refers to the location where neither an anomaly is caused nor a visible anomaly exists in the recorded photodiode signals or high-speed camera images. Through block-by-block labeling, defects can be located along the welding path. Finally, for the robustness of the model, data augmentation operations of rotation and flipping are performed on the high-speed camera images.

[0070] S3. Through a cascaded system, combine the features of multiple sensors, including photodiode blocks, high-speed camera blocks, classification photodiode blocks, and classification high-speed camera blocks, and perform laser welding quality monitoring through these four blocks;

[0071] Specifically, three quality monitoring methods, namely the single sensor system (SSS), the multi-sensor system (MSS), and the cascaded system (CS), are described subsequently. Generally, the measured values are mapped to quality-related quantities . The label , where 0 represents abnormal and 1 represents reference. Each time series in the n photodiode time series is defined as , where the first index represents the block number, and the second index represents the sample number within the block. The image of the high-speed camera is defined as

[0072] ;

[0073] The dataset S consists of n triples, .

[0074] The single-sensor system SSS monitors the process based on data from one sensor. For the input photodiode and high-speed camera data, SSS is defined as follows:

[0075] ;

[0076] Figure 3 Two SSSs are shown. The left one takes the photodiode signal as input, and the right one takes the high-speed camera image as input, and respectively represent the optimized prediction models. Each prediction model consists of two blocks, one is a feature block that extracts important features, and the other is a classification block that determines the welding quality.

[0077] The multi-sensor system MSS uses data from multiple sensors. The MSS composed of photodiode and high-speed camera data as input is defined as follows:

[0078] ;

[0079] Figure 4 A MSS and its prediction model are shown. The MSS consists of a classification block and two feature blocks, that is, one feature block for each sensor data, where the features are fused and processed for prediction. The classification block has a similar high-speed camera algorithm structure to the SSS. The difference is that the dimension of the photodiode features will be processed to the dimension of the high-speed camera. Compared with the SSS, the advantage of the MSS is more comprehensive quality assessment, however, the inference time will be longer.

[0080] The cascaded system CS combines the advantages of the two systems. It provides the possibility of using multiple sensors as the MSS <multi-sensor system>. Compared with the MSS, only part of the valid data is selected for analysis to obtain the welding quality instead of all the data, which speeds up the training speed,

[0081] See Figure 5 A two-stage CS is shown. Let be the output of the classifier for pre-judgment based on the photodiode signal. When p < 0.5, the classifier selects the anomaly, and when p ≥ 0.5, it selects the reference. The closer p is to 0 or 1, the higher the decision confidence of the classifier. Let be a fixed threshold. If p < r or , the decision of the classifier is definite. If the first condition is satisfied, the first condition is , it is considered that the classifier's judgment of anomalies is acceptable. If the second condition is met, the second condition is , it is considered that the classifier's judgment of the reference is acceptable. If the result of the classifier is uncertain, a final decision is made based on the high-speed camera data in the next step. Formally, the cascade system CS is defined as follows:

[0082] ;

[0083] The above three quality monitoring methods are all created by combining four blocks: the feature photodiode block, the feature high-speed camera block, the classification photodiode block, and the classification high-speed camera block. Each block can be composed of machine learning methods of feature engineering and decision tree (DT) or deep neural network (NN). For unified experiments, when one block uses machine learning or deep learning methods, the other blocks also use the same methods.

[0084] S4. Make a prediction result for laser welding quality monitoring through machine learning methods of feature engineering and decision tree or by deep neural network.

[0085] Specific feature engineering and decision tree. The features of the photodiode signal can be extracted manually or automatically by tsfresh (time series feature extraction based on scalable hypothesis testing). Seven statistical features, namely the mean, standard deviation, maximum value, minimum value, distance between the maximum and minimum values, kurtosis, and skewness, are calculated for each of the 13 samples in the photodiode data block to obtain several time series features. The features of the high-speed camera images are divided into statistical and geometric features. First, seven statistical features calculated on the photodiode signal block and the median are calculated for each high-speed camera image. Second, a binary mask image with the same size as the high-speed camera image is calculated for each high-speed camera image. The pixel values of the mask at those positions are considered as 1 value, and at the positions where the high-speed camera pixel values are smaller, they are considered as 0 value. According to h, the binary mask can contain information about the weld size, shape, or spatter, and 11 geometric features, namely area, number of regions, area of the largest region, ratio of the largest region to the area, convex hull, ratio of the area to the convex hull, perimeter, ratio of the perimeter to the area, area of the fitted ellipse, length of the ellipse, and width of the ellipse, are extracted. Through different empirical thresholds , each image can have several geometric features.

[0086] After feature extraction, DT is used for classification, and the CART algorithm is used to measure the quality of cracks. Among them, the CART algorithm is a classification and regression algorithm. The Gini index is used in the algorithm to select features. Two samples are randomly selected from the dataset, the probability of class inconsistency is calculated, and the feature with the smallest Gini index is selected. According to the selected features, the dataset is divided into different subsets, and decision trees are recursively generated for each subset. When generating subtrees, the Gini index is still used to select features, and the majority voting method is used at the leaf nodes to determine the classification results. To prevent overfitting, post-pruning is used to prune the decision trees.

[0087] Deep neural network. The feature photodiode and the feature high-speed camera are composed of convolutional layers, and the classification photodiode and the high-speed camera are composed of fully connected layers. The feature block consists of consisting of is a 2D convolutional layer with a filter size of 3×3, followed by batch normalization, max pooling, and ReLU as the activation function, where k is the number of filters. The classification block consists of a flattening layer and a fully connected layer DI, where D represents the weight matrix of the fully connected layer, is the neuron. The fully connected layer calculates the output by multiplying the input vector by the weight matrix and adding the bias vector. Here, "D" represents that weight matrix, and its dimension determines the connection relationship and transformation method between the input neuron and the output neuron (represented by "l" here). Through training, the value of the weight matrix "D" will be continuously adjusted to enable the model to better complete tasks such as classification. When ReLU activation function is used, and when Sigmoid activation function is used, and the dropout rate of the dropout layer is 0.5. During the training process of the neural network, binary cross-entropy is used as the loss function and is initialized with random weights.

[0088] The technical solutions in the above embodiments of the present application at least have the following technical effects or advantages:

[0089] Due to the proposed cascade system, quality monitoring can be quickly and accurately carried out through a two-stage structure; the first-stage structure uses classical machine learning of feature engineering and DT to check and analyze simple data such as time series, and makes certain classifications for some welds. For uncertain areas, the second-stage structure uses a convolutional network to make a final decision based on image data. Practice has proved that CS can outperform SSS in terms of accuracy and inference time. Compared with MSS, the data of different sensors in CS do not have to be transmitted to general-purpose hardware, thus effectively reducing the inference time and effectively improving productivity.

[0090] A laser welding quality monitoring system, comprising:

[0091] The data acquisition module is used to measure the laser beam through a photodiode and a high-speed camera, focus the beam on the workpiece surface using a scanner, record the voltage of the photodiode amplifier and the image of the high-speed camera at each sampling time to complete data acquisition;

[0092] The data preprocessing module is used to preprocess the data based on the data acquisition, crop the high-speed camera images, and assign each cropped high-speed camera image to a high-speed camera sample. Through block-by-block labeling, data augmentation operations of rotating and flipping the high-speed camera images are performed;

[0093] The high-speed camera sample represents a block, and a label is assigned to each block. Determine whether the label reference is abnormal. Through block-by-block labeling, defects can be located along the welding path, and data augmentation operations of rotating and flipping the high-speed camera images are performed;

[0094] The quality monitoring module is used to monitor the laser welding quality through a cascaded system, a multi-sensor combined feature photodiode block, a feature high-speed camera block, a classification photodiode block, and a classification high-speed camera block;

[0095] The prediction result module is used to make a prediction result on the laser welding quality monitoring through machine learning methods of feature engineering and decision trees or by a deep neural network.

[0096] Furthermore, a fiber laser with an infrared wavelength of 1070 nm is used to measure the light with a wavelength of 300–950 nm in the welding area, focus the beam on the workpiece surface using a scanner, record the voltage of the photodiode amplifier and the image of the high-speed camera at each sampling time;

[0097] The time series sampling rate of the photodiode is 250 kHz, and one sample is recorded every 4 μs. The sampling rate of the high-speed camera is 20 kHz, and one image is generated every 50 μs.

[0098] Furthermore, the definition of the cascaded system CS is as follows:

[0099] ;

[0100] Let be the output of the classifier for pre-judgment based on the photodiode signal,

[0101] When p < 0.5, the classifier selects abnormal, and when p ≥ 0.5, it selects reference. The closer p is to 0 or 1, the higher the decision confidence of the classifier;

[0102] Let be a fixed threshold. If p < r or , the decision of the classifier is certain;

[0103] If the first condition is satisfied, it is considered that the classifier's judgment of abnormality is acceptable. If the second condition is satisfied, it is considered that the classifier's judgment of reference is acceptable. If the result of the classifier is uncertain, a final decision is made based on the high-speed camera data in the next step.

[0104] Furthermore, seven statistical features and the median on the photodiode signal block are calculated for each high-speed camera image;

[0105] A binary mask image with the same size as the high-speed camera image is calculated for each high-speed camera image. The mask is considered to have a value of 1 at those positions where the pixel value in the high-speed camera image is greater than or equal to h, and a value of 0 at positions where the high-speed camera pixel value is smaller;

[0106] According to h, the binary mask can contain information about the weld size, shape, or spatter, and eleven geometric features are extracted: area, number of regions, area of the largest region, ratio of the largest region to the area, convex hull, ratio of the area to the convex hull, perimeter, ratio of the perimeter to the area, area of the fitted ellipse, length of the ellipse, and width of the ellipse;

[0107] Through different empirical thresholds , each image can have several geometric features;

[0108] After feature extraction, DT is used for classification, and the CART algorithm is used to measure the quality of cracks;

[0109] The Gini index is used in the algorithm to select features. Two samples are randomly selected from the dataset, the probability of their class inconsistency is calculated, and the feature with the smallest Gini index is selected;

[0110] According to the selected features, the dataset is divided into different subsets, and a decision tree is recursively generated for each subset.

[0111] Furthermore, the feature photodiode block and the feature high-speed camera block are composed of convolutional layers, and the classification photodiode block and the classification high-speed camera block are composed of fully connected layers;

[0112] The feature block is composed of and is a 2D convolutional layer with a filter size of 3×3, followed by batch normalization, max pooling, and ReLU as the activation function, where k is the number of filters; is a 2D convolutional layer with a filter size of 3×3, then batch normalization, max pooling, and ReLU as the activation function, and k is the number of filters;

[0113] The classification block consists of a flattening layer and a fully connected layer DI, where D represents the weight matrix of the fully connected layer and l is the neuron;

[0114] When Use the ReLU activation function when Use the Sigmoid activation function when , and the dropout rate of the dropout layer is 0.5;

[0115] During the training process of the neural network, use binary cross-entropy as the loss function and initialize with random weights.

[0116] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0117] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, and the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the specified functions in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0118] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including instruction means, and the instruction means implements the specified functions in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0119] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the specified functions in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0120] Although the preferred embodiments of the present invention have been described, additional changes and modifications can be made to these embodiments by those skilled in the art once they learn the basic creative concept. Therefore, the appended claims are intended to be construed to include the preferred embodiments as well as all changes and modifications falling within the scope of the present invention.

[0121] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these modifications and variations.

Claims

1. A method for monitoring the quality of laser welding, characterized in that, Including, S1, Measuring the laser beam through a photodiode and a high-speed camera, focusing the beam onto the workpiece surface using a scanner, recording the voltage of the photodiode amplifier and the image of the high-speed camera at each sampling time to complete data acquisition; S2, Based on the data acquisition, preprocessing the data, cropping the high-speed camera images, and assigning each cropped high-speed camera image to a high-speed camera sample. Through block-by-block labeling, data augmentation operations of rotation and flipping are performed on the high-speed camera images; S3. Through a cascaded system, a multi-sensor combined feature photodiode block, a feature high-speed camera block, a classification photodiode block, and a classification high-speed camera block are used to monitor the quality of laser welding through these four blocks; the cascaded system f cs includes, When, obtain the optimized prediction model ; When other situations occur, obtain an optimized prediction model ; CS represents a cascade system, and respectively represent optimized prediction models; Let be the output of a classifier that performs pre-determination based on a photodiode signal When p < 0.5, the classifier selects anomalies, and when p ≥ 0.5, it selects references. The closer p is to 0 or 1, the higher the decision confidence of the classifier; Let be a fixed threshold. If p < r or p > 1 - r, the decision of the classifier is certain; If the first condition is satisfied, where the first condition is , then the classifier's determination of an anomaly is considered acceptable. If the second condition is satisfied, where the second condition is , then the classifier's determination of a reference is considered acceptable. If the classifier's result is inconclusive, then a final decision is made based on high-speed camera data in the next step; S4, Making a prediction result for laser welding quality monitoring through machine learning methods of feature engineering and decision trees or by a deep neural network.

2. The laser welding quality monitoring method according to claim 1, wherein In step S1, it includes: Using a fiber laser with an infrared wavelength of 1070 nm to measure light with a wavelength of 300–950 nm in the welding area, focusing the beam onto the workpiece surface using a scanner, recording the voltage of the photodiode amplifier and the image of the high-speed camera at each sampling time; The time series sampling rate of the photodiode is 250 kHz, and one sample is recorded every 4 μs. The sampling rate of the high-speed camera is 20 kHz, and one image is generated every 50 μs.

3. The laser welding quality monitoring method according to claim 1, characterized in that, In step S4, it includes: Calculating seven statistical features and the median on the photodiode signal block for each high-speed camera image; Calculating a binary mask image with the same size as the high-speed camera image for each high-speed camera image. The pixel values in the mask are considered 1 at those positions where the pixel values in the high-speed camera image are greater than or equal to h, and 0 at positions where the high-speed camera pixel values are smaller; According to h, the binary mask contains information about the weld size, shape, or spatter, and 11 geometric features are extracted, including area, number of regions, area of the largest region, ratio of the largest region to the area, convex hull, ratio of the area to the convex hull, perimeter, ratio of the perimeter to the area, area of the fitted ellipse, length of the ellipse, and width of the ellipse; Via different empirical thresholds , each image has several geometric features; After feature extraction, classification is performed using DT, and the CART algorithm is used to measure the quality of cracks; The Gini index is used in the algorithm to select features. Two samples are randomly selected from the dataset, the probability of their class inconsistency is calculated, and the feature with the smallest Gini index is selected; According to the selected features, the dataset is divided into different subsets, and decision trees are recursively generated for each subset.

4. A laser welding quality monitoring method according to claim 1, characterized in that, In the deep neural network method in step S4, it includes: The feature photodiode block and the feature high-speed camera block are composed of convolutional layers, and the classification photodiode block and the classification high-speed camera block are composed of fully connected layers; The feature block consists of and is a 2D convolutional layer with a filter size of 3×3, followed by batch normalization, max pooling, and ReLU as the activation function, where k is the number of filters; The classification block consists of a flattening layer and a fully connected layer DI, where D represents the weight matrix of the fully connected layer and l is the neuron; The ReLU activation function is used when l ≠ 1, and the Sigmoid activation function is used when l = 1. The dropout rate of the dropout layer is 0.5; During the training process of the neural network, binary cross-entropy is used as the loss function and is initialized with random weights.

5. A laser welding quality monitoring system, characterized in that, Including, The data acquisition module is used to measure the laser beam through a photodiode and a high-speed camera, focus the beam onto the workpiece surface using a scanner, record the voltage of the photodiode amplifier and the image of the high-speed camera at each sampling time to complete data acquisition; The data preprocessing module is used to preprocess the data based on the data acquisition, crop the high-speed camera images, and assign each cropped high-speed camera image to a high-speed camera sample. Through block-by-block marking, data augmentation operations of rotation and flipping are performed on the high-speed camera images; The quality monitoring module is used to monitor the quality of laser welding through a cascaded system, a multi-sensor combined feature photodiode block, a feature high-speed camera block, a classification photodiode block, and a classification high-speed camera block, and perform quality monitoring of laser welding through these four blocks; the cascaded system f cs includes When, obtain the optimized prediction model ; When other situations occur, obtain an optimized prediction model ; CS represents a cascade system, and respectively represent optimized prediction models; Let be the output of a classifier that makes a preliminary decision based on a photodiode signal When p < 0.5, the classifier selects anomalies, and when p ≥ 0.5, it selects references. The closer p is to 0 or 1, the higher the decision confidence of the classifier; Let be a fixed threshold, and if p < r or p > 1 - r, the decision of the classifier is certain; If the first condition is met, where the first condition is , then the classifier's judgment of anomaly is considered acceptable. If the second condition is met, where the second condition is , then the classifier's judgment of reference is considered acceptable. If the classifier's result is uncertain, then a final decision is made based on the high-speed camera data in the next step; The prediction result module is used to make prediction results for laser welding quality monitoring through machine learning methods of feature engineering and decision trees or by a deep neural network.

6. The laser welding quality monitoring system according to claim 5, characterized in that, In the data acquisition module, it includes: Measure the light with a wavelength of 300–950 nm in the welding area using a fiber laser with an infrared wavelength of 1070 nm, focus the beam onto the workpiece surface using a scanner, record the voltage of the photodiode amplifier and the image of the high-speed camera at each sampling time; The time series sampling rate of the photodiode is 250 kHz, and one sample is recorded every 4 μs. The sampling rate of the high-speed camera is 20 kHz, and one image is generated every 50 μs.

7. The laser welding quality monitoring system according to claim 5, wherein, In the prediction result module, it includes: Calculate seven statistical features and the median on the photodiode signal block for each high-speed camera image; Calculate a binary mask image with the same size as the high-speed camera image for each high-speed camera image. The pixel value of the mask is considered 1 at those positions where the pixel value in the high-speed camera image is greater than or equal to h, and 0 at positions where the high-speed camera pixel value is smaller; According to h, the binary mask contains information about the weld size, shape, or spatter, and extracts 11 geometric features including area, number of regions, area of the largest region, ratio of the largest region to the area, convex hull, ratio of the area to the convex hull, perimeter, ratio of the perimeter to the area, area of the fitted ellipse, length of the ellipse, and width of the ellipse; By different experience thresholds , each image has a number of geometric features; After feature extraction, use DT for classification and measure the quality of cracks using the CART algorithm; The Gini index is used in the algorithm to select features. Randomly select two samples from the dataset, calculate the probability of class inconsistency, and select the feature with the smallest Gini index; According to the selected features, divide the dataset into different subsets and recursively generate decision trees for each subset.

8. A laser welding quality monitoring system according to claim 5, characterized in that, In the prediction result module, it also includes: The feature photodiode block and the feature high-speed camera block are composed of convolutional layers, and the classification photodiode block and the classification high-speed camera block are composed of fully connected layers; The feature block consists of and is a 2D convolutional layer with a filter size of 3×3, followed by batch normalization, max pooling, and ReLU as the activation function, where k is the number of filters; The classification block consists of a flattening layer and a fully connected layer DI, where D represents the weight matrix of the fully connected layer and l is the neuron; Use the ReLU activation function when l ≠ 1, use the Sigmoid activation function when l = 1, and the dropout rate of the dropout layer is 0.5; During the training process of the neural network, use binary cross-entropy as the loss function and initialize with random weights.

Images