Laser welding quality monitoring method and system

Abstract

The invention discloses a laser welding quality monitoring method and system. A laser beam is measured through a photodiode and a high-speed camera, the laser beam is focused to the surface of a workpiece through a scanner, at each sampling time, the voltage of a photodiode amplifier and an image of the high-speed camera are recorded, data acquisition is completed, and the laser welding quality is monitored based on the data acquisition. The method comprises the following steps of: preprocessing data, cutting high-speed camera images, distributing each cut high-speed camera image to a high-speed camera sample, and combining a characteristic photodiode block, a characteristic high-speed camera block, a classification photodiode block and a classification high-speed camera block through a cascade system and a plurality of sensors; laser welding quality monitoring is carried out through the four blocks; and a prediction result is made for laser welding quality monitoring through a feature engineering and decision tree machine learning method or a deep neural network. Reasoning time is effectively shortened, and data reasoning is more accurate.

Description

Technical Field

[0001] The present invention relates to the technical field of quality monitoring, and in particular to a laser welding quality monitoring method and system. Background Art

[0002] Laser welding has the characteristics of fast welding speed, narrow heat-affected zone, deep penetration and high degree of automation, and plays an important role in many industries such as automobiles and shipbuilding.

[0003] However, the inventors of this application found that the above technology has at least the following technical problems in the process of implementing the technical solution of the invention in the embodiment of this application: However, in the actual welding process, welding defects such as cracks and pores are inevitable, and one welding defect may cause the entire component to fail. In order to quickly detect defective components, the industrial process will be quality monitored, and signal data will be collected through sensors such as photodiodes, spectrometers, UV sensors, X-ray sensors, and high-speed cameras. Then different processing algorithms are used for analysis, including machine learning methods such as decision trees and support vector machines, and single sensor systems (SSS) or multi-sensor systems (MSS) combined with convolutional neural networks to detect the welding process. However, in the existing technology, laser welding exists, but the complex multi-sensor system has a long inference time, which may cause the quality monitoring time to far exceed the production time, resulting in production output being affected and the data being inaccurate. Summary of the invention

[0004] The embodiments of the present application provide a laser welding quality monitoring method and system, which solves the problems of long laser welding reasoning time and inaccurate data in the prior art, and effectively shortens the reasoning time and makes data reasoning more accurate.

[0005] The present application provides a laser welding quality monitoring method, comprising: S1. Measure the laser beam through a photodiode and a high-speed camera, use a scanner to focus the beam onto the workpiece surface, record the voltage of the photodiode amplifier and the image of the high-speed camera at each sampling time to complete data acquisition; S2, based on data acquisition, preprocesses the data, crops the high-speed camera image, and assigns each cropped high-speed camera image to a high-speed camera sample. By marking block by block, the high-speed camera image is rotated and flipped for data enhancement. S3, through the cascade system, the multi-sensor combination feature photodiode block, feature high-speed camera block, classification photodiode block and classification high-speed camera block, the laser welding quality is monitored through these four blocks; S4. Make predictions for laser welding quality monitoring through feature engineering and decision tree machine learning methods or deep neural networks.

[0006] Furthermore, a fiber laser with an infrared wavelength of 1070 nm was used to measure light with a wavelength of 300–950 nm in the welding area, and the beam was focused onto the workpiece surface by a scanner. At each sampling time, the voltage of the photodiode amplifier was recorded, and the image of the high-speed camera was recorded; The time series sampling rate of the photodiode was 250 kHz, recording one sample every 4 μs, and the sampling rate of the high-speed camera was 20 kHz, producing one image every 50 μs.

[0007] Furthermore, the cascade system CS is defined as follows: ; set up is the output of the classifier based on the pre-decision of the photodiode signal, When p < 0.5, the classifier selects the anomaly, and when p ≥ 0.5, it selects the reference. The closer p is to 0 or 1, the higher the classifier's decision confidence; set up is a fixed threshold, if p<r or , then the decision of the classifier is certain; If the first condition is met, the first condition is , then the classifier's judgment on the anomaly is considered acceptable if the second condition is met, which is The classifier's judgment on the reference is considered acceptable. If the classifier's result is uncertain, a final decision is made based on the high-speed camera data in the next step.

[0008] Furthermore, seven statistical features and median values ​​are calculated on the photodiode signal blocks for each high-speed camera image; For each high-speed camera image, a binary mask image with the same size as the high-speed camera image is calculated, and the mask is considered to have a value of 1 at those positions where the pixel value in the high-speed camera image is greater than or equal to h, and a value of 0 at those positions where the pixel value of the high-speed camera is smaller; According to h, the binary mask can contain information about the size, shape or spatter of the weld, and 11 geometric features are extracted, including area, number of regions, area of ​​the largest region, ratio of the largest region to the area, convex hull, ratio of area to convex hull, perimeter, ratio of perimeter to area, area of ​​the fitted ellipse, length of the ellipse and width of the ellipse; Through different experience thresholds , each image can have several geometric features; After feature extraction, DT is used for classification and the quality of cracks is measured using the CART algorithm; The algorithm uses the Gini index to select features. Two samples are randomly selected from the data set, the probability of their categories being inconsistent is calculated, and the feature with the smallest Gini index is selected. Based on the selected features, the dataset is divided into different subsets, and a decision tree is recursively generated for each subset.

[0009] Furthermore, the feature photodiode block and the feature high-speed camera block are composed of convolutional layers, and the classification photodiode block and the classification high-speed camera block are composed of fully connected layers; The feature block consists of composition, It is a 2D convolutional layer with a filter size of 3×3, followed by batch normalization, max pooling, ReLU as the activation function, and k is the number of filters; The classification block consists of a flattening layer and a fully connected layer DI, where D represents the weight matrix of the fully connected layer and l is a neuron; When 1≠1, the ReLU activation function is used, and when 1=1, the Sigmoid activation function is used, and the dropout rate of the dropout layer is 0.5; During the training of the neural network, binary cross entropy is used as the loss function and the weights are initialized with random values.

[0010] A laser welding quality monitoring system, comprising: A data acquisition module is used to measure the laser beam through a photodiode and a high-speed camera, focus the beam on the workpiece surface using a scanner, record the voltage of the photodiode amplifier and the image of the high-speed camera at each sampling time to complete data acquisition; A data preprocessing module is used for preprocessing the data based on data acquisition, cropping the high-speed camera image, and assigning each cropped high-speed camera image to a high-speed camera sample, and performing data enhancement operations of rotating and flipping the high-speed camera image by marking block by block; A quality monitoring module for monitoring the quality of laser welding through a cascade system, a multi-sensor combination feature photodiode block, a feature high-speed camera block, a classification photodiode block and a classification high-speed camera block; The prediction result module is used to make prediction results for laser welding quality monitoring through feature engineering and decision tree machine learning methods or deep neural networks.

[0011] Furthermore, a fiber laser with an infrared wavelength of 1070 nm was used to measure light with a wavelength of 300–950 nm in the welding area, and the beam was focused onto the workpiece surface by a scanner. At each sampling time, the voltage of the photodiode amplifier was recorded, and the image of the high-speed camera was recorded; The time series sampling rate of the photodiode was 250 kHz, recording one sample every 4 μs, and the sampling rate of the high-speed camera was 20 kHz, producing one image every 50 μs.

[0012] Furthermore, the cascade system CS is defined as follows: ; set up is the output of the classifier based on the pre-decision of the photodiode signal, When p < 0.5, the classifier selects the anomaly, and when p ≥ 0.5, it selects the reference. The closer p is to 0 or 1, the higher the classifier's decision confidence; set up is a fixed threshold, if p<r or , then the decision of the classifier is certain; If the first condition is met, the first condition is , then the classifier's judgment on the anomaly is considered acceptable if the second condition is met, which is The classifier's judgment on the reference is considered acceptable. If the classifier's result is uncertain, a final decision is made based on the high-speed camera data in the next step.

[0013] Furthermore, seven statistical features and median values ​​are calculated on the photodiode signal blocks for each high-speed camera image; For each high-speed camera image, a binary mask image with the same size as the high-speed camera image is calculated, and the mask is considered to have a value of 1 at those positions where the pixel value in the high-speed camera image is greater than or equal to h, and a value of 0 at those positions where the pixel value of the high-speed camera is smaller; According to h, the binary mask can contain information about the size, shape or spatter of the weld, and 11 geometric features are extracted, including area, number of regions, area of ​​the largest region, ratio of the largest region to the area, convex hull, ratio of area to convex hull, perimeter, ratio of perimeter to area, area of ​​the fitted ellipse, length of the ellipse and width of the ellipse; Through different experience thresholds , each image can have several geometric features; After feature extraction, DT is used for classification and the quality of cracks is measured using the CART algorithm; The algorithm uses the Gini index to select features. Two samples are randomly selected from the data set, the probability of their categories being inconsistent is calculated, and the feature with the smallest Gini index is selected. Based on the selected features, the dataset is divided into different subsets, and a decision tree is recursively generated for each subset.

[0014] Furthermore, the feature photodiode block and the feature high-speed camera block are composed of convolutional layers, and the classification photodiode block and the classification high-speed camera block are composed of fully connected layers; The feature block consists of composition, It is a 2D convolutional layer with a filter size of 3×3, followed by batch normalization, max pooling, ReLU as the activation function, and k is the number of filters; The classification block consists of a flattening layer and a fully connected layer DI, where D represents the weight matrix of the fully connected layer. It’s a neuron; when When ReLU activation function is used, The Sigmoid activation function is used, and the dropout rate of the dropout layer is 0.5; During the training of the neural network, binary cross entropy is used as the loss function and the weights are initialized with random values.

[0015] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: Due to the proposed cascade system, quality monitoring can be performed quickly and accurately through a two-level structure; the first level structure uses feature engineering and DT's classic machine learning to check and analyze simple data such as time series, and classifies some welds for certainty. In uncertain areas, a convolutional network is used in the second level structure to make a final decision based on image data. Practice has proved that CS can outperform SSS in terms of accuracy and reasoning time. Compared with MSS, the data of different sensors in CS do not have to be transmitted to general hardware, which effectively reduces the reasoning time and effectively improves productivity. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a flow chart of the laser welding quality monitoring method; Figure 2 Schematic diagram of a device for monitoring laser welding quality; Figure 3 Schematic diagram of two single sensor systems; Figure 4 It is a schematic diagram of a multi-sensor system; Figure 5 Schematic diagram of the cascade system. DETAILED DESCRIPTION

[0017] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0018] See also Figure 1 , S1. Measure the laser beam through a photodiode and a high-speed camera, use a scanner to focus the beam onto the workpiece surface, record the voltage of the photodiode amplifier and the image of the high-speed camera at each sampling time to complete data acquisition; Specifically, the actual equipment of the entire system is as follows Figure 2 As shown, a 2D galvanometer scanner using two mirrors directs the laser beam generated in the laser onto two thin metal plates. The data acquisition equipment uses two measurement systems: a photodiode (PD) and a high-speed camera (HSC). A fiber laser with an infrared wavelength of 1070nm is used to measure light with a wavelength of 300–950nm in the welding area, and then a scanner is used to focus the beam onto the workpiece surface. At each sampling time, the voltage of the photodiode amplifier is recorded, and the image of the high-speed camera is recorded.

[0019] The photodiode time series is sampled at 250 kHz, recording a sample every 4 μs, while the high-speed camera is sampled at 20 kHz, producing an image every 50 μs. The higher sampling rate of the photodiode allows for the detection of shorter anomalies, and the amount of raw data is smaller than that of the high-speed camera image, allowing for faster processing. The high-speed camera image provides geometric information not available in the photodiode signal, with spatial resolution.

[0020] S2. Based on data acquisition, the data is preprocessed, the high-speed camera image is cropped, and each cropped high-speed camera image is assigned to a high-speed camera sample. By marking block by block, the high-speed camera image is rotated and flipped for data enhancement; Specifically, the high-speed camera sample represents a block, and a label is assigned to each block to determine whether the label reference is abnormal. By marking each block, defects can be located along the welding path. The high-speed camera image is rotated and flipped for data enhancement.

[0021] The data is preprocessed for use in subsequent systems. First, the high-speed camera image is cropped to a size of 100×100 pixels and scaled to a value range of [0,1]. Then, each high-speed camera image is assigned to 13 photodiode samples, representing a block, and each block is assigned a label, namely reference or anomaly. Anomaly refers to the location where anomalies such as gaps or spatters are introduced, and reference refers to the location where no anomalies are caused or visible in the recorded photodiode signal or high-speed camera image. By labeling block by block, defects can be located along the welding path. Finally, for the robustness of the model, the high-speed camera image is subjected to data augmentation operations such as rotation and flipping.

[0022] S3. Through the cascade system, the multi-sensor combination feature photodiode block, feature high-speed camera block, classification photodiode block and classification high-speed camera block are used to monitor the laser welding quality; Specifically, the following describes three quality monitoring methods: single sensor system (SSS), multi-sensor system (MSS) and cascade system (CS). Generally speaking, the measured value Mapping to mass-related quantities On. Tags , where 0 represents anomaly and 1 represents reference. Each of the n photodiode time series is defined as , where the first index represents the block number, and the second index represents the sample number within the block. The image of the high-speed camera is defined as ; The dataset S consists of n triplets. .

[0023] A single sensor system (SSS) performs process monitoring based on data from one sensor. For photodiode and high-speed camera data as input, the SSS definition is as follows: ; Figure 3 Two SSS are shown, with the photodiode signal as input on the left and the high-speed camera image as input on the right. and They represent the optimized prediction models respectively. Each prediction model consists of two blocks, one is the feature block to extract important features, and the other is the classification block to determine the welding quality.

[0024] A multi-sensor system MSS uses data from multiple sensors. The MSS consisting of photodiode and high-speed camera data as input is defined as follows: ; Figure 4 An MSS and its prediction model are shown. The MSS consists of a classification block and two feature blocks, one for each sensor data, in which the features are fused and processed for prediction. The classification block has a high-speed camera algorithm structure similar to SSS, except that the dimensions of the photodiode features are processed into the dimensions of the high-speed camera. Compared with SSS, the advantage of MSS is a more comprehensive quality assessment, but the inference time will be longer.

[0025] The cascade system CS combines the advantages of both systems. It provides the possibility to use multiple sensors as MSS (Multi-Sensor System). Compared with MSS, only part of the valid data is analyzed to obtain the welding quality, instead of all the data, which speeds up the training. See also Figure 5 A two-stage CS is shown. is the output of the classifier based on the pre-decision of the photodiode signal. When p < 0.5, the classifier selects the abnormality, and when p ≥ 0.5, it selects the reference. The closer p is to 0 or 1, the higher the confidence of the classifier's decision. is a fixed threshold, if p<r or , then the decision of the classifier is certain. If the first condition is met, the first condition is , then the classifier's judgment on the anomaly is considered acceptable if the second condition is met, which is , the classifier’s judgment on the reference is considered acceptable. If the result of the classifier is uncertain, a final decision is made based on the high-speed camera data in the next step. Formally, the cascade system CS is defined as follows: ; The three quality monitoring methods described above are created by combining four blocks: feature photodiode block, feature high-speed camera block, classification photodiode block, and classification high-speed camera block. Each block can be machine learning methods through feature engineering and decision trees (DT) or composed of deep neural networks (NN). In order to unify the experiments, when one block uses machine learning or deep learning methods, the other blocks also use them.

[0026] S4. Make predictions for laser welding quality monitoring through feature engineering and decision tree machine learning methods or deep neural networks.

[0027] Specific feature engineering and decision trees. The features of the photodiode signal can be extracted manually or automatically by tsfresh (time series feature extraction based on scalable hypothesis testing). Seven statistical features, including mean, standard deviation, maximum, minimum, distance between maximum and minimum values, kurtosis and skewness, are calculated on each of the 13 samples of the photodiode data block to obtain several time series features. The features of high-speed camera images are divided into statistical and geometric features. First, the seven statistical features and the median calculated on the photodiode signal block are calculated for each high-speed camera image. Secondly, for each high-speed camera image, a binary mask image with the same size as the high-speed camera image is calculated. The pixel value of the mask in the high-speed camera image The positions where the pixel value of the high-speed camera is smaller are considered as 1 value, and the positions where the pixel value of the high-speed camera is smaller are considered as 0 value. According to h, the binary mask can contain information about the size, shape or spatter of the weld, and 11 geometric features are extracted: area, number of regions, area of ​​the largest region, ratio of the largest region to area, convex hull, ratio of area to convex hull, perimeter, ratio of perimeter to area, area of ​​the fitted ellipse, length of the ellipse and width of the ellipse. By different empirical thresholds , each image can have several geometric features.

[0028] After feature extraction, DT is used for classification, and the quality of cracks is measured using the CART algorithm. The CART algorithm is a classification regression algorithm. The Gini index is used to select features. Two samples are randomly selected from the data set, the probability of inconsistent categories is calculated, and the feature with the smallest Gini index is selected. According to the selected features, the data set is divided into different subsets, and a decision tree is recursively generated for each subset. When generating subtrees, the Gini index is continued to be used to select features, and the majority voting method is used at the leaf nodes to determine the classification results. In order to prevent overfitting, post-pruning is used to prune the decision tree.

[0029] Deep neural network. The feature photodiode and feature high-speed camera are composed of convolutional layers, and the classification photodiode and high-speed camera are composed of fully connected layers. The feature block is composed of composition, It is a 2D convolutional layer with a filter size of 3×3, followed by batch normalization, maximum pooling, ReLU as the activation function, and k is the number of filters. The classification block consists of a flattening layer and a fully connected layer DI, where D represents the weight matrix of the fully connected layer. The fully connected layer calculates the output by multiplying the input vector by the weight matrix and adding the bias vector. The "D" here represents the weight matrix, and its dimension determines the connection relationship and transformation method between the input neuron and the output neuron (here "l" represents the neuron). Through training, the value of the weight matrix "D" will be continuously adjusted so that the model can better complete tasks such as classification. When ReLU activation function is used, The Sigmoid activation function is used, and the dropout rate of the dropout layer is 0.5. In the training process of the neural network, binary cross entropy is used as the loss function and the weights are initialized with random values.

[0030] The technical solutions in the above embodiments of the present application have at least the following technical effects or advantages: Due to the proposed cascade system, quality monitoring can be performed quickly and accurately through a two-level structure; the first level structure uses feature engineering and DT's classic machine learning to check and analyze simple data such as time series, and classifies some welds for certainty. In uncertain areas, a convolutional network is used in the second level structure to make a final decision based on image data. Practice has proved that CS can outperform SSS in terms of accuracy and reasoning time. Compared with MSS, the data of different sensors in CS do not have to be transmitted to general hardware, which effectively reduces the reasoning time and effectively improves productivity.

[0031] A laser welding quality monitoring system, comprising: A data acquisition module is used to measure the laser beam through a photodiode and a high-speed camera, focus the beam on the workpiece surface using a scanner, record the voltage of the photodiode amplifier and the image of the high-speed camera at each sampling time to complete data acquisition; A data preprocessing module is used for preprocessing the data based on data acquisition, cropping the high-speed camera image, and assigning each cropped high-speed camera image to a high-speed camera sample, and performing data enhancement operations of rotating and flipping the high-speed camera image by marking block by block; The high-speed camera sample represents a block, and a label is assigned to each block to determine whether the label reference is abnormal. By marking each block, defects can be located along the welding path. The high-speed camera image is rotated and flipped for data enhancement; A quality monitoring module for monitoring the quality of laser welding through a cascade system, a multi-sensor combination feature photodiode block, a feature high-speed camera block, a classification photodiode block and a classification high-speed camera block; The prediction result module is used to make prediction results for laser welding quality monitoring through feature engineering and decision tree machine learning methods or deep neural networks.

[0032] Furthermore, a fiber laser with an infrared wavelength of 1070 nm was used to measure light with a wavelength of 300–950 nm in the welding area, and the beam was focused onto the workpiece surface by a scanner. At each sampling time, the voltage of the photodiode amplifier was recorded, and the image of the high-speed camera was recorded; The time series sampling rate of the photodiode was 250 kHz, recording one sample every 4 μs, and the sampling rate of the high-speed camera was 20 kHz, producing one image every 50 μs.

[0033] Furthermore, the cascade system CS is defined as follows: ; set up is the output of the classifier based on the pre-decision of the photodiode signal, When p < 0.5, the classifier selects the anomaly, and when p ≥ 0.5, it selects the reference. The closer p is to 0 or 1, the higher the classifier's decision confidence; set up is a fixed threshold, if p<r or , then the decision of the classifier is certain; If the first condition is met, the classifier's judgment on the anomaly is considered acceptable. If the second condition is met, the classifier's judgment on the reference is considered acceptable. If the result of the classifier is inconclusive, a final decision is made based on the high-speed camera data in the next step.

[0034] Furthermore, seven statistical features and median values ​​are calculated on the photodiode signal blocks for each high-speed camera image; For each high-speed camera image, a binary mask image with the same size as the high-speed camera image is calculated, and the mask is considered to have a value of 1 at those positions where the pixel value in the high-speed camera image is greater than or equal to h, and a value of 0 at those positions where the pixel value of the high-speed camera is smaller; According to h, the binary mask can contain information about the size, shape or spatter of the weld, and 11 geometric features are extracted, including area, number of regions, area of ​​the largest region, ratio of the largest region to the area, convex hull, ratio of area to convex hull, perimeter, ratio of perimeter to area, area of ​​the fitted ellipse, length of the ellipse and width of the ellipse; Through different experience thresholds , each image can have several geometric features; After feature extraction, DT is used for classification and the quality of cracks is measured using the CART algorithm; The algorithm uses the Gini index to select features. Two samples are randomly selected from the data set, the probability of their categories being inconsistent is calculated, and the feature with the smallest Gini index is selected. Based on the selected features, the dataset is divided into different subsets, and a decision tree is recursively generated for each subset.

[0035] Furthermore, the feature photodiode block and the feature high-speed camera block are composed of convolutional layers, and the classification photodiode block and the classification high-speed camera block are composed of fully connected layers; The feature block consists of composition, It is a 2D convolutional layer with a filter size of 3×3, followed by batch normalization, max pooling, ReLU as the activation function, and k is the number of filters; The classification block consists of a flattening layer, a fully connected layer DI, where D represents the weight matrix of the fully connected layer, and l is a neuron; when When ReLU activation function is used, The Sigmoid activation function is used, and the dropout rate of the dropout layer is 0.5; During the training of the neural network, binary cross entropy is used as the loss function and the weights are initialized with random values.

[0036] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0037] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0038] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0039] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0040] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0041] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A laser welding quality monitoring method, characterized in that: include: S1. Measure the laser beam through a photodiode and a high-speed camera, use a scanner to focus the beam onto the workpiece surface, record the voltage of the photodiode amplifier and the image of the high-speed camera at each sampling time to complete data acquisition; S2, based on data acquisition, preprocesses the data, crops the high-speed camera image, and assigns each cropped high-speed camera image to a high-speed camera sample. By marking block by block, the high-speed camera image is rotated and flipped for data enhancement. S3, through the cascade system, the multi-sensor combination feature photodiode block, feature high-speed camera block, classification photodiode block and classification high-speed camera block, the laser welding quality is monitored through these four blocks; S4. Make predictions for laser welding quality monitoring through feature engineering and decision tree machine learning methods or deep neural networks.

2. A laser welding quality monitoring method as claimed in claim 1, characterized in that: Step S1 includes: A fiber laser with an infrared wavelength of 1070 nm was used to measure light with a wavelength of 300–950 nm in the welding area. The beam was focused onto the workpiece surface by a scanner. At each sampling time, the voltage of the photodiode amplifier was recorded and the image of the high-speed camera was recorded. The time series sampling rate of the photodiode was 250 kHz, recording one sample every 4 μs, and the sampling rate of the high-speed camera was 20 kHz, producing one image every 50 μs.

3. A laser welding quality monitoring method as claimed in claim 1, characterized in that: Step S3 includes: The definition of the cascade system is as follows: ； CS stands for cascade system; set up is the output of the classifier based on the pre-decision of the photodiode signal, When p < 0.5, the classifier selects the anomaly, and when p ≥ 0.5, it selects the reference. The closer p is to 0 or 1, the higher the classifier's decision confidence; set up is a fixed threshold, if p<r or , then the decision of the classifier is certain; If the first condition is met, the first condition is , then the classifier's judgment on the anomaly is considered acceptable if the second condition is met, which is , the classifier’s judgment on the reference is considered acceptable. If the classifier’s result is uncertain, a final decision is made based on the high-speed camera data in the next step.

4. A laser welding quality monitoring method as claimed in claim 1, characterized in that: Step S4 includes: Seven statistical features and median values ​​were calculated on the photodiode signal blocks for each high-speed camera image; For each high-speed camera image, a binary mask image with the same size as the high-speed camera image is calculated, and the mask is considered to have a value of 1 at those positions where the pixel value in the high-speed camera image is greater than or equal to h, and a value of 0 at those positions where the pixel value of the high-speed camera is smaller; According to h, the binary mask can contain information about the size, shape or spatter of the weld, and 11 geometric features are extracted, including area, number of regions, area of ​​the largest region, ratio of the largest region to the area, convex hull, ratio of area to convex hull, perimeter, ratio of perimeter to area, area of ​​the fitted ellipse, length of the ellipse and width of the ellipse; Through different experience thresholds , each image can have several geometric features; After feature extraction, DT is used for classification and the quality of cracks is measured using the CART algorithm; The algorithm uses the Gini index to select features. Two samples are randomly selected from the data set, the probability of their categories being inconsistent is calculated, and the feature with the smallest Gini index is selected. Based on the selected features, the dataset is divided into different subsets, and a decision tree is recursively generated for each subset.

5. A laser welding quality monitoring method as claimed in claim 1, characterized in that: The deep neural network method in step S4 includes: The feature photodiode block and the feature high-speed camera block are composed of convolutional layers, and the classification photodiode block and the classification high-speed camera block are composed of fully connected layers; The feature block consists of composition, It is a 2D convolutional layer with a filter size of 3×3, followed by batch normalization, max pooling, ReLU as the activation function, and k is the number of filters; The classification block consists of a flattening layer and a fully connected layer DI, where D represents the weight matrix of the fully connected layer and l is a neuron; When l≠1, the ReLU activation function is used, and when l=1, the Sigmoid activation function is used, and the dropout rate of the dropout layer is 0.5; During the training of the neural network, binary cross entropy is used as the loss function and the weights are initialized with random values.

6. A laser welding quality monitoring system, characterized in that: include: A data acquisition module is used to measure the laser beam through a photodiode and a high-speed camera, focus the beam on the workpiece surface using a scanner, record the voltage of the photodiode amplifier and the image of the high-speed camera at each sampling time to complete data acquisition; A data preprocessing module is used for preprocessing the data based on data acquisition, cropping the high-speed camera image, and assigning each cropped high-speed camera image to a high-speed camera sample, and performing data enhancement operations of rotating and flipping the high-speed camera image by marking block by block; A quality monitoring module for monitoring the quality of laser welding through a cascade system, a multi-sensor combination feature photodiode block, a feature high-speed camera block, a classification photodiode block and a classification high-speed camera block; The prediction result module is used to make prediction results for laser welding quality monitoring through feature engineering and decision tree machine learning methods or deep neural networks.

7. A laser welding quality monitoring system as claimed in claim 6, characterized in that: The data acquisition module includes: A fiber laser with an infrared wavelength of 1070 nm was used to measure light with a wavelength of 300–950 nm in the welding area. The beam was focused onto the workpiece surface by a scanner. At each sampling time, the voltage of the photodiode amplifier was recorded and the image of the high-speed camera was recorded. The time series sampling rate of the photodiode was 250 kHz, recording one sample every 4 μs, and the sampling rate of the high-speed camera was 20 kHz, producing one image every 50 μs.

8. A laser welding quality monitoring system as claimed in claim 6, characterized in that: The quality control module includes: The definition of the cascade system CS is as follows: ； CS stands for cascade system; set up is the output of the classifier based on the pre-decision of the photodiode signal, When p < 0.5, the classifier selects the anomaly, and when p ≥ 0.5, it selects the reference. The closer p is to 0 or 1, the higher the classifier's decision confidence; set up is a fixed threshold, if p<r or , then the decision of the classifier is certain; If the first condition is met, the first condition is , then the classifier's judgment on the anomaly is considered acceptable if the second condition is met, which is , the classifier’s judgment on the reference is considered acceptable. If the classifier’s result is uncertain, a final decision is made based on the high-speed camera data in the next step.

9. A laser welding quality monitoring system as claimed in claim 6, characterized in that: The prediction result module includes: Seven statistical features and median values ​​were calculated on the photodiode signal blocks for each high-speed camera image; For each high-speed camera image, a binary mask image with the same size as the high-speed camera image is calculated, and the mask is considered to have a value of 1 at those positions where the pixel value in the high-speed camera image is greater than or equal to h, and a value of 0 at those positions where the pixel value of the high-speed camera is smaller; According to h, the binary mask can contain information about the size, shape or spatter of the weld, and 11 geometric features are extracted, including area, number of regions, area of ​​the largest region, ratio of the largest region to the area, convex hull, ratio of area to convex hull, perimeter, ratio of perimeter to area, area of ​​the fitted ellipse, length of the ellipse and width of the ellipse; Through different experience thresholds , each image can have several geometric features; After feature extraction, DT is used for classification and the quality of cracks is measured using the CART algorithm; The algorithm uses the Gini index to select features. Two samples are randomly selected from the data set, the probability of their categories being inconsistent is calculated, and the feature with the smallest Gini index is selected. Based on the selected features, the dataset is divided into different subsets, and a decision tree is recursively generated for each subset.

10. A laser welding quality monitoring system as claimed in claim 6, characterized in that: The prediction result module also includes: The feature photodiode block and the feature high-speed camera block are composed of convolutional layers, and the classification photodiode block and the classification high-speed camera block are composed of fully connected layers; The feature block consists of composition, It is a 2D convolutional layer with a filter size of 3×3, followed by batch normalization, max pooling, ReLU as the activation function, and k is the number of filters; The classification block consists of a flattening layer and a fully connected layer DI, where D represents the weight matrix of the fully connected layer and l is a neuron; When l≠1, the ReLU activation function is used, and when l=1, the Sigmoid activation function is used, and the dropout rate of the dropout layer is 0.5; During the training of the neural network, binary cross entropy is used as the loss function and the weights are initialized with random values.

Images