State detection and online learning method for laser welding manufacturing

Abstract

The invention relates to the technical field of laser welding quality detection, and particularly discloses a laser welding manufacturing-oriented state detection and online learning method, which comprises the following steps of: constructing a multi-material welding defect image data set; constructing an initial welding defect detection model; training the initial welding defect detection model according to the multi-material welding defect image data set to obtain a trained welding defect detection model; and evaluating the trained welding defect detection model, and after the evaluation is passed, inputting the current to-be-detected welding image into the trained welding defect detection model for detection so as to output a defect category detection result of the current to-be-detected welding image. According to the method, a more accurate and real-time quality monitoring means is provided for the laser welding process, and application and development of the intelligent manufacturing technology in the welding field are promoted; by achieving intelligent control over the welding process, the industrial production efficiency is improved, the production cost is reduced, and meanwhile the product quality is guaranteed.

Description

Technical Field

[0001] This invention relates to the field of laser welding quality inspection technology, and more specifically, to a condition detection and online learning method for laser welding manufacturing. Background Technology

[0002] Laser welding, as a core technology in modern industrial precision machining, faces key challenges in practical applications, including welding process stability, precise quality control, and improved automation levels. These challenges mainly stem from the interference of multiple random factors on weld formation quality, such as differences in base material microstructure, laser power fluctuations, changes in welding environment temperature and humidity, and uneven surface cleanliness of the base material, thus affecting product performance. Traditional laser welding processes rely on operator experience and judgment, resulting in inherent defects such as strong subjective dependence, delayed real-time feedback, and difficulty in quantitative evaluation. This leads to significant challenges in controlling quality consistency, making it difficult to meet the stringent standards of high-end fields such as aerospace and energy equipment, and also restricting further improvements in production efficiency.

[0003] In the field of weld defect detection, the current mainstream technologies are still mainly manual visual inspection and traditional machine vision. Manual inspection is limited by the professional level and fatigue of the inspectors, resulting in low efficiency, unstable accuracy, and high false positive and false negative rates. Traditional machine vision methods rely on manually designed features (such as edges and textures), and have insufficient ability to extract features of complex defects (such as microcracks and low-contrast porosity), making it difficult to break through the bottleneck in recognition accuracy.

[0004] To address the aforementioned issues, deep learning-based intelligent detection technology has become an important research direction. This technology, through models such as Convolutional Neural Networks (CNN) and Faster R-CNN, leverages data-driven automatic feature learning capabilities to achieve efficient detection and classification of weld defects. This breaks through the reliance on manual feature engineering in traditional methods, laying the technological foundation for automated and intelligent inspection of weld quality in industrial equipment. It effectively improves the robustness and efficiency of defect identification, driving laser welding technology towards higher precision and reliability. However, despite the existence of traditional and deep learning-based automatic detection methods, the detection accuracy in complex industrial scenarios needs improvement, and real-time data interaction between the detection system and welding equipment is not yet fully established, making it difficult to achieve full-process automation. Essentially, this is a challenge of synergistically optimizing "process stability" and "quality control accuracy." The lack of welding data is a key factor restricting the application of deep learning and the realization of full-process automation in laser welding, affecting model training and optimization results. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the existing technology and provide a state detection and online learning method for laser welding manufacturing, so as to solve the problems of complex imaging, random defect distribution, scarce labeled samples and catastrophic forgetting in the existing technology.

[0006] As a first aspect of the present invention, a method for condition detection and online learning in laser welding manufacturing is provided, comprising the following steps:

[0007] Step S1: Construct a multi-material welding defect image dataset;

[0008] Step S2: Construct an initial welding defect detection model;

[0009] Step S3: Train the initial welding defect detection model based on the multi-material welding defect image dataset to obtain the trained welding defect detection model;

[0010] Step S4: Evaluate the trained welding defect detection model. If the evaluation is successful, input the current welding image to be detected into the trained welding defect detection model for detection, and output the defect category detection result of the current welding image to be detected.

[0011] Furthermore, step S1 also includes:

[0012] During the laser welding manufacturing process, industrial cameras are used to collect images of six types of welding defects in stainless steel and four types of welding defects in aluminum alloy. The number of images of each type of welding defect is increased to more than 1,500. Then, five levels of noise, blur, glare, and brightness interference are simulated on the increased welding defect images to obtain the multi-material welding defect image dataset.

[0013] Among them, the six types of welding defects for stainless steel include good weld, burn-through, contamination, incomplete penetration, insufficient shielding gas, and hump; the four types of welding defects for aluminum alloys include penetration, burn-through, incomplete penetration, and hump.

[0014] Furthermore, the simulation of noise, blur, glare, and brightness interference of level 5 intensity on the amplified welding defect image also includes:

[0015] The amplified welding defect image is simulated with Gaussian noise interference of intensity level 5 by adjusting the standard deviation σ; the amplified welding defect image is simulated with Gaussian blur interference of intensity level 5 by adjusting the standard deviation σ; the amplified welding defect image is simulated with glare interference of intensity level 5 by adjusting the intensity; and the amplified welding defect image is simulated with luminance interference of intensity level 5 by adjusting the luminance factor.

[0016] Furthermore, step S2 also includes:

[0017] The initial welding defect detection model includes a CNN feature extractor and a fully connected layer. The CNN feature extractor includes an initial convolutional block, a cascaded convolutional block, and a global average pooling layer. The welding defect image is input into the initial convolutional block for feature extraction, and the initial convolutional block outputs a 56×56×64 feature map. Then, the 56×56×64 feature map is input into the cascaded convolutional block for processing, and the cascaded convolutional block outputs a 7×7×512 feature map. Next, the 7×7×512 feature map is input into the global average pooling layer for compression, and the global average pooling layer outputs a 1×1×512 compact feature vector. Finally, the 1×1×512 compact feature vector is input into the fully connected layer for category prediction, and the fully connected layer outputs the defect category prediction result for the welding defect image.

[0018] The fully connected layer initially has 10 output dimensions, corresponding to the predicted probabilities of 10 types of welding defects. For each new type of welding defect, the output dimension increases by one. The discriminative power between the new and old classes is enhanced by calculating the cosine similarity between the input features of the new class and the centers of the old classes. The cosine similarity is calculated as cosine... s The formula for calculating similarity(x,y) is:

[0019]

[0020] Here, x and y are two vectors, · represents the dot product, and ||x|| and ||y|| are the norms of vectors x and y, respectively.

[0021] Furthermore, step S3 also includes:

[0022] The six types of stainless steel welding defect images in the multi-material welding defect image dataset are used as the base class dataset. The base class dataset is input into the initial welding defect detection model for training to obtain the current welding defect detection model. The current welding defect detection model is used as the teacher model for subsequent knowledge distillation.

[0023] Next, based on the teacher model, four types of welding defect images of aluminum alloy in the multi-material welding defect image dataset are introduced as incremental categories. The parameters of the CNN feature extractor of the teacher model are fixed, and the output dimension of the fully connected layer is dynamically expanded. The current welding defect detection model is trained through a two-stage joint optimization strategy to obtain the student model after each incremental training. Then, the performance of the student model after each incremental training is verified in multiple dimensions.

[0024] The two-stage joint optimization strategy includes a first-stage optimization strategy that uses the teacher model to generate soft labels to pass on historical classification experience, and a second-stage optimization strategy that integrates hard label cross-entropy loss and soft label distillation loss. Here, the soft label refers to the probability distribution of defect categories output by the teacher model for the base class dataset, and the hard label refers to the one-hot encoding of the new welding defect image.

[0025] Furthermore, in the first-stage optimization strategy, a teacher-student model architecture is adopted. The teacher model is trained based on the base class dataset and its parameters are fixed, serving only as knowledge. The student model initially inherits the parameters of the CNN feature extractor of the teacher model. The fully connected layer is dynamically expanded according to the number of new categories, but its parameters are randomly initialized. The soft labels generated by the teacher model are used to optimize only the parameters of the fully connected layer and fit the soft labels using classification loss.

[0026] In the second-stage optimization strategy, old-class welding defect images and new-class welding defect images are mixed in a 1:1 ratio as the training set. The loss function is the sum of the classification loss and the distillation loss. The distillation loss is weighted by the importance of the feature maps to constrain the difference between the feature maps of the old and new models. When updating parameters, the CNN feature extractor only updates the non-critical feature parameters related to the new-class welding defect images. The old class weights of the fully connected layers are frozen, and the new class weights are optimized with a learning rate of 0.01.

[0027] Furthermore, in the second-stage optimization strategy, the current welding defect detection model is trained by constraining the feature space through the feature map importance weighting and distillation loss;

[0028] (1) Feature map importance weighting: The gradient norm of the feature maps of each layer of the CNN feature extractor is calculated using Taylor expansion and used as the importance. The formula is:

[0029]

[0030] in, The importance of the c-th feature map in the l-th layer for the t-th task, Z l,c It is the c-th feature map in the l-th layer. This is the old task classification loss, where F represents the Frobenius norm;

[0031] By iterating through old task examples, backpropagation is used to obtain the gradient norm of the feature maps of each layer of the CNN feature extractor, which is then accumulated into the importance matrix. In the middle, and normalized, the importance matrix The formula is as follows:

[0032]

[0033] Among them, C lIt is the number of channels in the l-th layer of the CNN feature extractor;

[0034] (2) Distillation Loss Constraint on Feature Space: By constraining the differences in feature maps between the old and new models, key features of the old task are preserved, and distillation loss is applied. The calculation formula is:

[0035]

[0036] Among them, Z' l,c Z is the c-th feature map in the l-th layer of the old model. l,c This is the c-th feature map of the new model in layer l, where B is the batch size, L is the total number of convolutional layers in the CNN feature extractor, and b is the training sample for the current training round. Forward propagation simultaneously acquires feature maps from both the old and new models, and the distillation loss is calculated. and classification loss L cls Weighted summation to obtain the total loss

[0037] Classification loss L cls The calculation formula is:

[0038]

[0039] Where N is the number of samples in the current training round, y i p is the true label of the i-th training sample. i It is the predicted probability of the i-th training sample;

[0040] Total loss The calculation formula is:

[0041]

[0042] Where, λ disc Distillation loss The fixed weight parameter, λ t These are weight coefficients that are dynamically adjusted according to the task stage t. Initially, at t=1, the parameters of the CNN feature extractor and the fully connected layer are randomly initialized. In subsequent tasks, at t>1, the parameters of the CNN feature extractor are partially updated through training with new types of welding defect images, and the parameters of the fully connected layer are dynamically expanded. The dynamic expansion mechanism of the fully connected layer's parameters is centered on the weight matrix W of the fully connected layer. During initial base class training, the output dimension of the fully connected layer is 6, corresponding to the weight matrix W. When training the new k-th class, the weight matrix is ​​expanded to...

[0043] Furthermore, the evaluation of the trained welding defect detection model also includes:

[0044] Take the average incremental accuracy and forgetting rate as the evaluation metrics for the trained welding defect detection model;

[0045] (1) The average incremental accuracy refers to the average of the classification accuracies of the trained welding defect detection model on the seen classes in all subsequent tasks. For each subsequent task t, calculate the accuracy ACC of the trained welding defect detection model on all seen classes in this subsequent task t to calculate the average incremental accuracy ACC of the trained welding defect detection model inc The calculation formula is:

[0046]

[0047] where T is the total number of subsequent tasks, and ACC t is the accuracy of the trained welding defect detection model on all seen classes in the t-th subsequent task;

[0048] (2) The forgetting rate refers to the difference between the accuracy of the current model on the old task and the accuracy of the old model on this old task. Among them, when the trained welding defect detection model is obtained at the t-th subsequent task, select the old task s < t, and calculate the accuracy ACC of the trained welding defect detection model on the old task s current,s and the accuracy ACC of the welding defect detection model at the completion of the old task s on the old task s old,s to calculate the forgetting rate FR of the trained welding defect detection model. The calculation formula is:

[0049] FR = ACC old,s -ACC current,s

[0050] where a negative forgetting rate FR indicates forgetting, and a positive value indicates backward transfer.

[0051] A state detection and online learning method for laser welding manufacturing provided by the present invention has the following beneficial effects: It not only provides a more accurate and real-time quality monitoring means for the laser welding process, but also promotes the application and development of intelligent manufacturing technology in the welding field; By realizing the intelligent control of the welding process, it helps to improve industrial production efficiency, reduce production costs, and ensure product quality at the same time, providing reliable technical support for high-standard industries. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The drawings are used to provide a further understanding of the present invention and constitute a part of the specification, and are used together with the following specific embodiments to explain the present invention, but do not constitute a limitation to the present invention.

[0053] Figure 1 The flowchart illustrates a condition monitoring and online learning method for laser welding manufacturing provided by this invention.

[0054] Figure 2 The flowchart illustrates a specific implementation method for a condition monitoring and online learning approach in laser welding manufacturing provided by this invention.

[0055] Figure 3 This is a schematic diagram of the welding defect detection model provided by the present invention.

[0056] Figure 4 This diagram illustrates the training process of the welding defect detection model provided by the present invention. Detailed Implementation

[0057] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a condition detection and online learning method for laser welding manufacturing proposed according to the present invention. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the protection scope of the present invention.

[0058] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of the invention described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0059] This embodiment provides a condition detection and online learning method for laser welding manufacturing, such as... Figure 1-2 As shown, the condition monitoring and online learning method for laser welding manufacturing includes the following steps:

[0060] Step S1: Construct a multi-material welding defect image dataset;

[0061] Preferably, step S1 further includes:

[0062] During the laser welding manufacturing process, industrial cameras are used to collect images of six types of welding defects in stainless steel and four types of welding defects in aluminum alloy. The number of images of each type of welding defect is increased to more than 1,500 through data amplification methods such as horizontal flipping. Then, five levels of noise, blur, glare, and brightness interference are simulated on the amplified welding defect images to obtain the multi-material welding defect image dataset.

[0063] Among them, the six types of welding defects for stainless steel include good weld, burn-through, contamination, incomplete penetration, insufficient shielding gas, and hump; the four types of welding defects for aluminum alloys include penetration, burn-through, incomplete penetration, and hump.

[0064] It should be noted that the laser welding manufacturing process video was captured by an industrial camera, and the video was analyzed frame by frame to obtain image data of 6 types of welding defects in stainless steel and 4 types of welding defects in aluminum alloy, covering types such as good welds and burn-through. Through data amplification methods such as horizontal flipping and vertical flipping, the sample size of each type exceeded 1,500 images.

[0065] Specifically, the simulation of noise, blur, glare, and brightness interference of level 5 intensity on the amplified welding defect image also includes:

[0066] To simulate a complex industrial environment, Gaussian noise interference of intensity level 5 is simulated on the amplified welding defect image by adjusting the standard deviation σ (σ = 10 / 32.5 / 55 / 77.5 / 100); Gaussian blur interference of intensity level 5 is simulated on the amplified welding defect image by adjusting the standard deviation σ (1 to 5); glare interference of intensity level 5 is simulated on the amplified welding defect image by adjusting the intensity (100 to 200); and brightness interference of intensity level 5 is simulated on the amplified welding defect image by adjusting the brightness factor (0.2 to 2.0).

[0067] Step S2: Construct an initial welding defect detection model;

[0068] Preferably, such as Figure 3 As shown, step S2 further includes:

[0069] The initial welding defect detection model includes a CNN feature extractor and a fully connected layer. The CNN feature extractor extracts general features, and the fully connected layer is dynamically adjusted to adapt to new tasks. The CNN feature extractor includes an initial convolutional block, cascaded convolutional blocks, and a global average pooling layer. The welding defect image is input into the initial convolutional block for feature extraction, which outputs a 56×56×64 feature map to quickly extract basic common features of the welding defect image, such as weld edges and molten pool contours, laying the foundation for subsequent in-depth extraction of complex features. Then, the 56×56×64 feature map is input into the cascaded convolutional block for processing. The cascaded convolutional block sequentially processes the data to obtain 28×28×128, 14×14×256, and 7×7×512 feature maps, with the cascaded convolutional block outputting a 7×7×512 feature map. The 12-dimensional feature map, based on the extraction of basic common features, mines high-level features such as the detailed texture of defects and cross-regional correlations. Then, the 7×7×512 feature map is input into the global average pooling layer for compression. The global average pooling layer outputs a compact feature vector of 1×1×512. Finally, the compact feature vector of 1×1×512 is input into the fully connected layer for category prediction. The fully connected layer outputs the defect category prediction result of the welding defect image dynamically expanding with "old class + new class" (vertical dots on the right, old class above and new class below). At the same time, in order to prevent the forgetting of old class knowledge, a knowledge distillation constraint is introduced. First, the feature importance is calculated to quantify the importance of old class features, and then the feature extractor parameters are constrained by backpropagation of distillation loss. This fully covers the input, feature extraction, classification, knowledge distillation, and changes in data flow and feature map size.

[0070] The fully connected layer initially has 10 output dimensions, corresponding to the predicted probabilities of 10 categories of welding defects (6 for stainless steel and 4 for aluminum alloy). For each new category of welding defect, the output dimension increases by one. The cosine similarity between the new and old classes is calculated to enhance the discriminative power between them (avoiding overfitting of the new class). s The formula for calculating similarity(x,y) is:

[0071]

[0072] Here, x and y are two vectors, · represents the dot product, and ||x|| and ||y|| are the norms of vectors x and y, respectively.

[0073] In this embodiment of the invention, taking a stainless steel-aluminum alloy hybrid dataset as an example, feature stability is enhanced through normalization. Then, multiple convolutional blocks are cascaded to form an initial convolutional block and cascaded convolutional blocks. Each convolutional block includes a convolutional layer, a normalization layer (to accelerate training and reduce internal covariate bias), an activation function (ReLU, introducing nonlinearity to enhance feature representation), and a pooling layer, outputting a feature map (such as...). Figure 3 The 7×7×512 feature map in the image is converted into a feature vector (e.g., 7×7×512 feature map) through a global average pooling layer (GAP). Figure 3 (1×1×512 eigenvectors in the data).

[0074] In this embodiment of the invention, the output dimension of the fully connected layer dynamically expands with the task increment. Dimension refers to the number of category probabilities output by the classification head, corresponding one-to-one with the total number of defect categories to be distinguished. Dynamically expanding the dimension allows the model to adapt to the continuously increasing number of defect categories and output a corresponding number of prediction results. The output of the fully connected layer is a set of numerical values, each corresponding to the predicted probability of a category. "The output dimension is 10-dimensional" means that this set of numerical values ​​has 10 elements, each corresponding to the predicted probability of one of the 10 defect categories.

[0075] Step S3: Train the initial welding defect detection model based on the multi-material welding defect image dataset to obtain the trained welding defect detection model;

[0076] Preferably, such as Figure 4 As shown, step S3 further includes:

[0077] The six types of stainless steel welding defect images in the multi-material welding defect image dataset are used as the base class dataset. The base class dataset is input into the initial welding defect detection model for training to obtain the current welding defect detection model. The current welding defect detection model is used as the teacher model for subsequent knowledge distillation.

[0078] Next, based on the teacher model, four types of welding defect images of aluminum alloy in the multi-material welding defect image dataset are introduced as incremental categories. The parameters of the CNN feature extractor of the teacher model are fixed, and the output dimension of the fully connected layer is dynamically expanded. The current welding defect detection model is trained through a two-stage joint optimization strategy to obtain the student model after each incremental training. Then, the performance of the student model after each incremental training is verified in multiple dimensions.

[0079] The two-stage joint optimization strategy includes a first-stage optimization strategy that uses the teacher model to generate soft labels to pass on historical classification experience, and a second-stage optimization strategy that integrates hard label cross-entropy loss and soft label distillation loss. Here, the soft label refers to the probability distribution of defect categories output by the teacher model for the base class dataset, and the hard label refers to the one-hot encoding of the new welding defect image.

[0080] Specifically, such as Figure 4 As shown, Task 1 involves inputting six types of welding defect images for stainless steel (good weld, burn-through, contamination, incomplete penetration, insufficient shielding gas, and hump). An initial welding defect detection model is used for base class training, outputting a stable teacher model. This teacher model identifies the six types of stainless steel defects through a fully connected layer. Tasks 2-5 sequentially introduce four types of welding defect images for aluminum alloy (penetration, burn-through, incomplete penetration, and hump). Only one new type of welding defect image is added each time for model iteration, resulting in a student model. The student model inherits the "knowledge" of the teacher model and accurately classifies the newly added aluminum alloy defects through a fully connected layer. The total number of tasks is 5.

[0081] In the detection inference process during training, the cross-material detection workflow involves inputting a stainless steel-aluminum alloy hybrid dataset, preprocessing it, generating feature vectors through a CNN feature extractor, and then having a dynamic classification head output a probability distribution based on the current number of defect categories. This probability distribution is combined with class center cosine similarity to achieve cross-material defect classification. Furthermore, for multi-environment testing, simulated industrial interference of five levels, including noise and blur, is used to generate multi-environment test data.

[0082] Specifically, in the first stage optimization strategy, a teacher-student model architecture is adopted. The teacher model is trained on the base class dataset and its parameters are fixed, serving only as knowledge. The student model initially inherits the parameters of the CNN feature extractor of the teacher model. The fully connected layer is dynamically expanded according to the number of new categories, but the parameters are randomly initialized. Using the soft labels generated by the teacher model, the student model only optimizes the parameters of the fully connected layer and uses classification loss to fit the soft labels in order to quickly learn the category distribution pattern of the old task. This stage can further improve the convergence speed of the student model for the old categories compared to random initialization, thereby improving the learning efficiency of the student model.

[0083] In the second-stage optimization strategy, old-class welding defect images and new-class welding defect images are mixed in a 1:1 ratio as the training set. The loss function is the sum of the classification loss and the distillation loss. The distillation loss is weighted by the importance of the feature maps to constrain the difference between the feature maps of the old and new models. When updating parameters, the CNN feature extractor only updates the non-critical feature parameters related to the new-class welding defect images. The old class weights of the fully connected layers are frozen, and the new class weights are optimized with a learning rate of 0.01.

[0084] In the incremental learning process, during task initialization, a training set containing the new category and an old task example set containing 20 representative samples from each category are divided, and the old model parameters are loaded to retain old knowledge. By randomly sampling 2000 samples from the old task example set, the gradient norm of the feature maps at each layer is calculated to locate key features (such as the melt pool contour) and generate an importance matrix. When updating parameters, the backbone network (convolutional layer) only updates non-critical parameters related to the new task, while the classification head freezes the old class weights and randomly initializes and trains the new class weights. The optimizer uses the SGD optimizer, configured with an initial learning rate of 0.1, momentum of 0.9, decay of 0.1 every 20 epochs, batch size of 128, and 90 epochs per task.

[0085] More specifically, in the second-stage optimization strategy, in order to solve the catastrophic forgetting in incremental learning (features of the old task are covered by the new task), the current welding defect detection model (student model) is trained by constraining the feature space through the feature map importance weighting and distillation loss;

[0086] (1) Feature map importance weighting: Traditional knowledge distillation applies importance to all features. Figure 1 Treating all features equally leads to key old features being covered by new task features. The impact of the Taylor expansion approximates the model update on the old task loss, and the gradient norm of the feature maps of each layer of the CNN feature extractor is calculated as the importance, using the following formula:

[0087]

[0088] in, The importance of the c-th feature map in the l-th layer for the t-th task, Z l,c It is the c-th feature map in the l-th layer. It is the old task classification loss, where F represents the Frobenius norm (F-norm), which is used to measure the "size" of the gradient matrix;

[0089] By iterating through old task examples, backpropagation is used to obtain the gradient norm of the feature maps of each layer of the CNN feature extractor, which is then accumulated into the importance matrix. In the middle, and normalized, the importance matrix The formula is as follows:

[0090]

[0091] Among them, C l It is the number of channels in the l-th layer of the CNN feature extractor;

[0092] (2) Distillation loss constrains the feature space: By constraining the differences in feature maps between the old and new models (teacher model and student model) (higher importance feature maps are penalized more severely), key features of the old task are preserved, and distillation loss is applied. The calculation formula is:

[0093]

[0094] Among them, Z' l,cZ is the c-th feature map in layer l of the old model (teacher model). l,c This is the c-th feature map of the new model (student model) in layer l, where B is the batch size, L is the total number of convolutional layers in the CNN feature extractor, and b is the training sample for the current training round. Forward propagation simultaneously acquires feature maps from both the old and new models, and the distillation loss is calculated. and classification loss L cls Weighted summation to obtain the total loss

[0095] Classification loss L cls The calculation formula is:

[0096]

[0097] Where N is the number of samples in the current training round, y i p is the true label of the i-th training sample. i It is the predicted probability of the i-th training sample;

[0098] Total loss The calculation formula is:

[0099]

[0100] Where, λ disc Distillation loss The fixed weight parameter, λ t These are weight coefficients that are dynamically adjusted according to the task stage t. Used together, these two factors allow the model to achieve both λ and t-weighting during incremental learning. disc Forcefully retain key old features, while also being able to use λ t Adaptively adjust constraint strength to avoid excessive constraints affecting the learning efficiency of new classes;

[0101] The parameter update strategy for the feature extractor involves random initialization of both the CNN feature extractor and the fully connected layer parameters during the initial task (t=1). During subsequent tasks (t>1), the CNN feature extractor parameters are partially updated (adjusting only non-critical features sensitive to the new task) through training with images of new welding defects. The fully connected layer parameters are dynamically expanded (weights for new categories are randomly initialized, while weights for older categories are frozen or fine-tuned). The dynamic expansion mechanism of the fully connected layer's parameters is centered on the weight matrix W. During initial base class training, the output dimension of the fully connected layer is 6 (e.g., 6 categories for stainless steel), corresponding to the weight matrix W. When training the new k-th class, the weight matrix is ​​expanded to...

[0102] In the embodiment of the present invention, the CNN feature extractor extracts hierarchical features through multiple convolutional blocks, generates a compact feature vector by combining global average pooling, and retains the key features of the old task through feature map importance weighted distillation. This design balances the stability and flexibility of features in class-incremental learning, effectively alleviates catastrophic forgetting, and is particularly suitable for industrial scenarios such as welding defect detection that require incremental learning of new defect categories. The output dimension of the fully connected layer dynamically expands with the defect categories, and the calculation of class center cosine similarity is introduced to enhance the discrimination between new and old categories.

[0103] Step S4: Evaluate the trained welding defect detection model. After passing the evaluation, input the current welding image to be detected into the trained welding defect detection model for detection, so as to output the detection result of the defect category of the current welding image to be detected.

[0104] Preferably, in the evaluation of the trained welding defect detection model, it further includes:

[0105] Use the average incremental accuracy and forgetting rate as the evaluation indicators of the trained welding defect detection model; the average incremental accuracy is used to measure the comprehensive generalization ability of the student model for new and old classes, and the forgetting rate is used to measure the degree of catastrophic forgetting;

[0106] (1) The average incremental accuracy refers to the average value of the classification accuracies of the trained welding defect detection model on the seen classes in all subsequent tasks. For each subsequent task t, calculate the accuracy ACC of the trained welding defect detection model on all seen classes (classes of the first t tasks) in this subsequent task t , so as to calculate the average incremental accuracy ACC of the trained welding defect detection model inc , and the calculation formula is:

[0107]

[0108] where T is the total number of subsequent tasks, and ACC t is the accuracy of the trained welding defect detection model on all seen classes in the t-th subsequent task;

[0109] (2) The forgetting rate refers to the difference between the accuracy of the current model on the old task and the accuracy of the old model on this old task. Among them, when the trained welding defect detection model is obtained in the t-th subsequent task, select the old task s < t, and calculate the accuracy ACC of the trained welding defect detection model on the old task s current,s and the accuracy ACC of the welding defect detection model at the completion of the old task s on the old task s old,s , so as to calculate the forgetting rate FR of the trained welding defect detection model, and the calculation formula is:

[0110] FR = Acc old,s -ACC current,s

[0111] Here, a negative forgetting rate b indicates forgetting, while a positive value indicates backward transfer.

[0112] In this embodiment of the invention, taking the stainless steel-aluminum alloy dataset (10 classes) as an example, a defect detection and online learning method that expands the categories by incremental learning is adopted. The overall process is as follows: (1) First, the 6 types of defects in stainless steel are used as the base class dataset. Welding images are collected by an industrial camera and expanded to a single class sample size of ≥1500 images. The CNN feature extractor with 3 levels of residual blocks and the fully connected layer with an initial dimension of 6 are input for training to obtain the teacher model as the basis for subsequent knowledge distillation; (2) Then, the 4 types of defects in aluminum alloy are introduced as incremental categories on the basis of the teacher model (1 new category is added each time). The backbone network parameters of the teacher model are fixed and the output dimension of the fully connected layer is dynamically expanded (1 new dimension is added each time). In the first stage, the soft label with temperature T=2 is generated by the teacher model to transmit the base class knowledge. In the second stage, the cross-entropy loss of hard labels and the feature map importance of weight λ=0.8 are fused. A two-stage joint optimization strategy using weighted distillation loss is employed to train the student model, enabling collaborative optimization of new and old knowledge and mitigating catastrophic forgetting. Soft labels refer to the probability distribution of the teacher model's output on old task data, while hard labels are the one-hot encoding of new task data. For example, the label for "weld penetration" defects in aluminum alloys (4 types) is (0,0,1,0), with 1 only at the corresponding category position and 0 for the rest. Then, the average accuracy and forgetting rate of the student model after each incremental training are validated in multiple dimensions. The tasks are divided into Task 1, training on the 6 base classes of stainless steel (teacher model), and Tasks 2-5, which sequentially add 4 types of defects in aluminum alloys (1 type each time). A total of 5 tasks gradually expand the output dimension of the fully connected layer from 6 dimensions to 10 dimensions. This process achieves incremental optimization and industrial scenario adaptation of the welding defect detection model through dynamic expansion of the fully connected layer and a feature importance-weighted distillation mechanism.

[0113] The present invention provides a state detection and online learning method for laser welding manufacturing, including an online learning framework based on adaptive feature consolidation, a generalization ability verification method based on multi-material defect dataset construction and environmental simulation, a model building method based on knowledge distillation and dynamic classification head, and a model training and evaluation method based on two-stage joint optimization.

[0114] Furthermore, in the aforementioned state detection and online learning method for laser welding manufacturing, the adaptive feature consolidation online learning framework balances new and old knowledge through feature importance weighted distillation to mitigate catastrophic forgetting. Specifically, it uses Taylor expansion to approximate the impact of model updates on the loss of old tasks, calculates the gradient norm of each layer's feature map as an importance index, locates key low-frequency features, constructs a weighted distillation loss function, applies stronger penalties to feature maps with high importance, and forces the new and old models to maintain consistency in key features.

[0115] Furthermore, the construction of the multi-material defect dataset and environmental simulation involved collecting 10 types of weld images, including 6 types of stainless steel and 4 types of aluminum alloy. Through data amplification, the sample size of each type was increased to more than 1,500 images. Simulation data was generated according to 5 intensity levels for 4 types of interference factors, including noise, blur, glare, and brightness. A multi-environment test set containing 20 categories was constructed to quantify the robustness of the model under different working conditions.

[0116] Furthermore, the knowledge distillation and dynamic classification head model construction adopts multi-level convolutional blocks of convolutional neural networks to extract hierarchical features, generates compact feature vectors through global average pooling, designs an scalable fully connected classification layer, the output dimension dynamically increases with the defect category, and introduces class center cosine similarity calculation to enhance the distinction between new and old categories. At the same time, the teacher model generates soft labels to convey the category distribution knowledge of the old task, and integrates hard labels and soft labels to train the student model.

[0117] Furthermore, in the two-stage joint optimization model training and evaluation, the first stage is to fix the teacher model to generate soft labels, and the second stage is to integrate hard labels and soft labels to train the student model. The average incremental accuracy and forgetting rate are used as evaluation indicators. The average incremental accuracy is the average classification accuracy of the model on the seen classes in all incremental stages, and the forgetting rate is the difference between the accuracy of the current model on the old task and the accuracy of the old model on the old task.

[0118] In summary, the state detection and online learning method for laser welding manufacturing provided by this invention is an online learning detection algorithm using a knowledge distillation framework based on adaptive feature consolidation. This algorithm addresses the real-time, robustness, and incremental learning problems in laser welding defect detection, and is applicable to multi-material and multi-environment welding quality monitoring in industrial automated production. Specifically, a multi-material weld dataset containing six types of stainless steel and four types of aluminum alloy is constructed. Data amplification, including horizontal flipping, and simulation of noise, blur, glare, and brightness interference at five levels are performed to generate multi-environment test data. An online learning model based on knowledge distillation is designed. Hierarchical features are extracted through the basic convolutional layers of a CNN feature extractor. A dynamic classification head dynamically expands the output dimension according to the defect category, and class center cosine similarity calculation is introduced to avoid overfitting to new classes. Weighted distillation based on feature map importance is used, and Taylor expansion quantifies the gradient norm of old task features, imposing stronger constraints on key features to mitigate catastrophic forgetting. The training process employs a two-stage joint optimization, fusing hard / soft label loss to update the model. Experiments show that this method exhibits good generalization ability and robustness under cross-material detection and complex environmental interference, providing effective support for real-time laser welding quality detection.

[0119] This invention delves into the application of online learning technology in the field of laser welding. Online learning offers advantages such as strong adaptability and high flexibility, enabling real-time updates and optimization of models based on new data. By acquiring laser welding images online in real time and constructing an intelligent laser welding image database, this project will study online monitoring technology for laser welding. This will enable online monitoring and identification of laser welding data under different environments (such as low light, various noise levels, etc.) and data on different welding materials (such as iron, zinc, etc.). This method overcomes the technical difficulties of insufficient real-time feedback and overly subjective judgment in traditional welding processes, significantly improving welding quality, accelerating production pace, reducing defect rates, and lowering energy consumption. This paper not only provides a more accurate and real-time quality monitoring method for the laser welding process but also promotes the application and development of intelligent manufacturing technology in the welding field. By realizing intelligent control of the welding process, it helps improve industrial production efficiency, reduce production costs, and ensure product quality, providing reliable technical support for high-standard industries.

[0120] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A condition monitoring and online learning method for laser welding manufacturing, characterized in that, The condition monitoring and online learning method for laser welding manufacturing includes the following steps: Step S1: Construct a multi-material welding defect image dataset; Step S2: Construct an initial welding defect detection model; Step S3: Train the initial welding defect detection model based on the multi-material welding defect image dataset to obtain the trained welding defect detection model; Step S4: Evaluate the trained welding defect detection model. If the evaluation is successful, input the current welding image to be detected into the trained welding defect detection model for detection, and output the defect category detection result of the current welding image to be detected.

2. The condition detection and online learning method for laser welding manufacturing according to claim 1, characterized in that, Step S1 further includes: During the laser welding manufacturing process, industrial cameras are used to collect images of six types of welding defects in stainless steel and four types of welding defects in aluminum alloy. The number of images of each type of welding defect is increased to more than 1,500. Then, five levels of noise, blur, glare, and brightness interference are simulated on the increased welding defect images to obtain the multi-material welding defect image dataset. Among them, the six types of welding defects for stainless steel include good weld, burn-through, contamination, incomplete penetration, insufficient shielding gas, and hump; the four types of welding defects for aluminum alloys include penetration, burn-through, incomplete penetration, and hump.

3. The condition detection and online learning method for laser welding manufacturing according to claim 2, characterized in that, The simulation of noise, blur, glare, and brightness interference of level 5 intensity on the amplified welding defect image also includes: The amplified welding defect image is simulated with Gaussian noise interference of intensity level 5 by adjusting the standard deviation σ; the amplified welding defect image is simulated with Gaussian blur interference of intensity level 5 by adjusting the standard deviation σ; the amplified welding defect image is simulated with glare interference of intensity level 5 by adjusting the intensity; and the amplified welding defect image is simulated with luminance interference of intensity level 5 by adjusting the luminance factor.

4. The condition detection and online learning method for laser welding manufacturing according to claim 2, characterized in that, Step S2 further includes: The initial welding defect detection model includes a CNN feature extractor and a fully connected layer. The CNN feature extractor includes an initial convolutional block, a cascaded convolutional block, and a global average pooling layer. The welding defect image is input into the initial convolutional block for feature extraction, and the initial convolutional block outputs a 56×56×64 feature map. Then, the 56×56×64 feature map is input into the cascaded convolutional block for processing, and the cascaded convolutional block outputs a 7×7×512 feature map. Next, the 7×7×512 feature map is input into the global average pooling layer for compression, and the global average pooling layer outputs a 1×1×512 compact feature vector. Finally, the 1×1×512 compact feature vector is input into the fully connected layer for category prediction, and the fully connected layer outputs the defect category prediction result for the welding defect image. The fully connected layer initially has 10 output dimensions, corresponding to the predicted probabilities of 10 types of welding defects. For each new type of welding defect, the output dimension increases by one. The discriminative power between the new and old classes is enhanced by calculating the cosine similarity between the input features of the new class and the centers of the old classes. The cosine similarity is calculated as cosine... s The formula for calculating similarity(x,y) is: Here, x and y are two vectors, · represents the dot product, and ||x|| and ||y|| are the norms of vectors x and y, respectively.

5. The condition detection and online learning method for laser welding manufacturing according to claim 4, characterized in that, Step S3 further includes: The six types of stainless steel welding defect images in the multi-material welding defect image dataset are used as the base class dataset. The base class dataset is input into the initial welding defect detection model for training to obtain the current welding defect detection model. The current welding defect detection model is used as the teacher model for subsequent knowledge distillation. Next, based on the teacher model, four types of welding defect images of aluminum alloy in the multi-material welding defect image dataset are introduced as incremental categories. The parameters of the CNN feature extractor of the teacher model are fixed, and the output dimension of the fully connected layer is dynamically expanded. The current welding defect detection model is trained through a two-stage joint optimization strategy to obtain the student model after each incremental training. Then, the performance of the student model after each incremental training is verified in multiple dimensions. The two-stage joint optimization strategy includes a first-stage optimization strategy that uses the teacher model to generate soft labels to pass on historical classification experience, and a second-stage optimization strategy that integrates hard label cross-entropy loss and soft label distillation loss. Here, the soft label refers to the probability distribution of defect categories output by the teacher model for the base class dataset, and the hard label refers to the one-hot encoding of the new welding defect image.

6. The condition detection and online learning method for laser welding manufacturing according to claim 5, characterized in that, In the first stage optimization strategy, a teacher-student model architecture is adopted. The teacher model is trained on the base class dataset and the parameters are fixed. It is only used as knowledge. The student model initially inherits the parameters of the CNN feature extractor of the teacher model. The fully connected layer is dynamically expanded according to the number of new categories, but the parameters are randomly initialized. The soft labels generated by the teacher model are used. The student model only optimizes the parameters of the fully connected layer and uses classification loss to fit the soft labels. In the second-stage optimization strategy, old-class welding defect images and new-class welding defect images are mixed in a 1:1 ratio as the training set. The loss function is the sum of the classification loss and the distillation loss. The distillation loss is weighted by the importance of the feature maps to constrain the difference between the feature maps of the old and new models. When updating parameters, the CNN feature extractor only updates the non-critical feature parameters related to the new-class welding defect images. The old class weights of the fully connected layers are frozen, and the new class weights are optimized with a learning rate of 0.

01.

7. The condition detection and online learning method for laser welding manufacturing according to claim 6, characterized in that, In the second-stage optimization strategy, the current welding defect detection model is trained by constraining the feature space through the feature map importance weighting and distillation loss; (1) Feature map importance weighting: The gradient norm of the feature maps of each layer of the CNN feature extractor is calculated using Taylor expansion and used as the importance. The formula is: in, The importance of the c-th feature map in the l-th layer for the t-th task, Z l,c It is the c-th feature map in the l-th layer. This is the old task classification loss, where F represents the Frobenius norm; By iterating through old task examples, backpropagation is used to obtain the gradient norm of the feature maps of each layer of the CNN feature extractor, which is then accumulated into the importance matrix. In the middle, and normalized, the importance matrix The formula is as follows: Among them, C l It is the number of channels in the l-th layer of the CNN feature extractor; (2) Distillation Loss Constraint on Feature Space: By constraining the differences in feature maps between the old and new models, key features of the old task are preserved, and distillation loss is applied. The calculation formula is: Among them, Z' l,c Z is the c-th feature map in the l-th layer of the old model. l,c This is the c-th feature map of the new model in layer l, where B is the batch size, L is the total number of convolutional layers in the CNN feature extractor, and b is the training sample for the current training round. Forward propagation simultaneously acquires feature maps from both the old and new models, and the distillation loss is calculated. and classification loss L cls Weighted summation to obtain the total loss Classification loss L cls The calculation formula is: Where N is the number of samples in the current training round, y i p is the true label of the i-th training sample. i It is the predicted probability of the i-th training sample; Total loss The calculation formula is: Where, λ disc Distillation loss The fixed weight parameter, λ t These are weight coefficients that are dynamically adjusted according to the task stage t. Initially, at t=1, the parameters of the CNN feature extractor and the fully connected layer are randomly initialized. In subsequent tasks, at t>1, the parameters of the CNN feature extractor are partially updated through training with new types of welding defect images, and the parameters of the fully connected layer are dynamically expanded. The dynamic expansion mechanism of the fully connected layer's parameters is centered on the weight matrix W of the fully connected layer. During initial base class training, the output dimension of the fully connected layer is 6, corresponding to the weight matrix W. When training the new k-th class, the weight matrix is ​​expanded to...

8. The condition detection and online learning method for laser welding manufacturing according to claim 7, characterized in that, The evaluation of the trained welding defect detection model also includes: The average incremental accuracy and forgetting rate are used as evaluation metrics for the trained welding defect detection model. (1) The average incremental accuracy refers to the average classification accuracy of the trained welding defect detection model on the seen classes across all subsequent tasks. For each subsequent task t, the accuracy ACC of the trained welding defect detection model on all seen classes in that subsequent task is calculated. t The average incremental accuracy (ACC) of the trained welding defect detection model is calculated. inc The calculation formula is: Where T is the total number of subsequent tasks, ACC t Let be the accuracy of the trained welding defect detection model on all seen classes in the t-th subsequent task; (2) The forgetting rate refers to the difference between the accuracy of the current model on the old task and the accuracy of the old model on the same old task. Among them, when the trained welding defect detection model is obtained at the t-th subsequent task, an old task s < t is selected, and the accuracy ACC of the trained welding defect detection model on the old task s is calculated current,s and the accuracy ACC of the welding defect detection model when the old task s is completed on the old task s old,s , so as to calculate the forgetting rate FR of the trained welding defect detection model. The calculation formula is: FR=ACC old,s -ACC current,s Among them, the forgetting rate FR is negative to indicate forgetting and positive to indicate backward migration.

Images