A state detection and online learning method for laser welding manufacturing

Abstract

The application relates to the technical field of laser welding quality detection, and particularly discloses a state detection and online learning method for laser welding manufacturing, which comprises the following steps: constructing a multi-material welding defect image dataset; constructing an initial welding defect detection model; training the initial welding defect detection model according to the multi-material welding defect image dataset to obtain a trained welding defect detection model; evaluating the trained welding defect detection model, and when the evaluation is passed, inputting a current welding image to be detected into the trained welding defect detection model for detection to output a defect category detection result of the current welding image to be detected. The application not only provides a more accurate and real-time quality monitoring method for the laser welding process, but also promotes the application and development of intelligent manufacturing technology in the welding field; through intelligent control of the welding process, the industrial production efficiency can be improved, the production cost can be reduced, and the product quality can be ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of laser welding quality detection, and more particularly to a state detection and online learning method for laser welding manufacturing. BACKGROUND

[0002] As a core technology of modern industrial precision machining, laser welding faces key challenges in welding process stability, quality precision control, and automation level improvement in practical applications. These challenges are mainly caused by multiple random factors such as differences in base material structure, laser power fluctuations, changes in welding environment temperature and humidity, and uneven base material surface cleanliness, which interfere with the weld formation quality and further affect the product service performance. Traditional laser welding processes rely on the experience of operators, which has strong subjective dependence, real-time feedback lag, and difficulty in quantitative evaluation, resulting in difficulty in quality consistency control and difficulty in meeting the stringent standards of high-end fields such as aerospace and energy equipment, and restricting the further improvement of production efficiency.

[0003] In the field of weld defect detection, the current mainstream technology is still mainly based on manual inspection and traditional machine vision. Manual detection is limited by the professional level and fatigue state of the detection personnel, and has problems such as low efficiency, unstable precision, and high false detection and missed detection rates; traditional machine vision methods rely on manual design of features (such as edges, textures, etc.), and have insufficient feature extraction capability for complex defects (such as micro-cracks, low-contrast pores), making it difficult to break through the bottleneck of recognition accuracy.

[0004] To address the above problems, intelligent detection technology based on deep learning has become an important research direction. This technology uses models such as convolutional neural networks (CNN) and Faster R-CNN to achieve efficient detection and classification of weld defects through data-driven automatic feature learning, breaking the dependence on manual feature engineering of traditional methods, laying a technical foundation for automated and intelligent detection of industrial equipment weld quality, effectively improving the robustness and detection efficiency of defect recognition, and promoting the development of laser welding technology towards high precision and high reliability. However, although there are traditional and deep learning automatic detection methods, the detection accuracy in industrial complex scenarios needs to be improved, and the real-time data interaction between the detection system and the welding equipment has not been completed, making it difficult to achieve full-process automation. Essentially, it is a problem of collaborative optimization of "process stability" and "quality control precision". The lack of welding data is a key factor that restricts the application of deep learning and the realization of full-process automation of laser welding, affecting the model training and optimization effect. SUMMARY

[0005] The present application aims to overcome the deficiencies in the prior art, and provides a state detection and online learning method for laser welding manufacturing, to solve the problems of complex imaging, random defect distribution, scarce labeled samples and catastrophic forgetting in the prior art.

[0006] As a first aspect of the present application, a state detection and online learning method for laser welding manufacturing is provided, comprising the following steps:

[0007] Step S1: constructing a multi-material welding defect image dataset;

[0008] Step S2: constructing an initial welding defect detection model;

[0009] Step S3: training the initial welding defect detection model according to the multi-material welding defect image dataset to obtain a trained welding defect detection model;

[0010] Step S4: evaluating the trained welding defect detection model, and when the evaluation passes, inputting a current welding image to be detected into the trained welding defect detection model for detection to output a defect category detection result of the current welding image to be detected.

[0011] Further, in the step S1, further comprising:

[0012] In the laser welding manufacturing process, 6 types of welding defect images of stainless steel and 4 types of welding defect images of aluminum alloy are collected by an industrial camera, and the number of each type of welding defect image is expanded to more than 1500, and then the expanded welding defect images are simulated with 5 levels of intensity of noise, blur, glare, and brightness interference to obtain the multi-material welding defect image dataset;

[0013] Among them, the 6 types of welding defects of stainless steel include good weld, burn-through, contamination, incomplete penetration, insufficient protective gas, and hump, and the 4 types of welding defects of aluminum alloy include complete penetration, burn-through, incomplete penetration, and hump.

[0014] Further, in the step S2, further comprising:

[0015] The expanded welding defect images are simulated with 5 levels of intensity of Gaussian noise interference by adjusting the standard deviation σ, 5 levels of intensity of Gaussian blur interference by adjusting the standard deviation σ, 5 levels of intensity of glare interference by adjusting the intensity, and 5 levels of intensity of brightness interference by adjusting the brightness factor.

[0016] Further, in the step S2, further comprising:

[0017] The initial welding defect detection model comprises a CNN feature extractor and a full connection layer, the CNN feature extractor comprises an initial convolution block, a cascaded convolution block and a global average pooling layer, a welding defect image is input into the initial convolution block for feature extraction, the initial convolution block outputs a feature map of 56x56x64, then the feature map of 56x56x64 is input into the cascaded convolution block for processing, the cascaded convolution block outputs a feature map of 7x7x512, then the feature map of 7x7x512 is input into the global average pooling layer for compression, the global average pooling layer outputs a compact feature vector of 1x1x512, finally the compact feature vector of 1x1x512 is input into the full connection layer for class prediction, and the full connection layer outputs a defect class prediction result of the welding defect image;

[0018] The output dimension of the full connection layer is initially 10 dimensions, corresponding to the prediction probabilities of 10 welding defect classes respectively, and the output dimension increases by 1 dimension for each newly added welding defect class, and the cosine similarity between the input features of the new class and the center of the old class is calculated to enhance the discrimination between the new and old classes; wherein the cosine similarity cos s The calculation formula of the cosine similarity cos

[0019]

[0020] Wherein, x and y are two vectors, · represents dot product, and ||x|| and ||y|| are the norms of vectors x and y respectively.

[0021] Further, the step S3 further comprises:

[0022] The 6 welding defect images of stainless steel in the multi-material welding defect image data set are taken as a base class data set, the base class data set is input into the initial welding defect detection model for training to obtain a current welding defect detection model, and the current welding defect detection model is taken as a teacher model for subsequent knowledge distillation;

[0023] Then, 4 welding defect images of aluminum alloy in the multi-material welding defect image data set are sequentially introduced as incremental classes on the basis of the teacher model, the parameters of the CNN feature extractor of the teacher model are fixed, the output dimension of the full connection layer is dynamically expanded, the current welding defect detection model is trained through a two-stage joint optimization strategy to obtain a student model after each incremental training, and then the student model after each incremental training is subjected to multi-dimensional performance verification;

[0024] The two-stage joint optimization strategy includes a first-stage optimization strategy of generating soft labels by a teacher model to pass historical classification experience and a second-stage optimization strategy of fusing a hard label cross-entropy loss and a soft label distillation loss; the soft label refers to a defect category probability distribution law output by the teacher model for the base class data set, and the hard label refers to a one-hot encoding of the new welding defect image.

[0025] Further, in the first-stage optimization strategy, a teacher-student model architecture is adopted, the teacher model is trained based on the base class data set and has fixed parameters, and only serves as knowledge, the student model initially inherits the parameters of the CNN feature extractor of the teacher model, the full connection layer is dynamically expanded according to the number of new categories but the parameters are randomly initialized, the student model only optimizes the parameters of the full connection layer by using the soft label generated by the teacher model, and the soft label is fitted by using a classification loss;

[0026] In the second-stage optimization strategy, old class welding defect images and new class welding defect images are mixed in a 1:1 ratio as a training set, and a loss function is a sum of a classification loss and a distillation loss, wherein the distillation loss is weighted by feature map importance to constrain the feature map difference between the new and old models, and when the parameters are updated, the CNN feature extractor only updates the non-key feature parameters related to the new class welding defect image, and the old class weight of the full connection layer is frozen and the new class weight is optimized at a learning rate of 0.01.

[0027] Further, in the second-stage optimization strategy, the current welding defect detection model is trained by the feature map importance weighting and the distillation loss constraint on the feature space;

[0028] (1) Feature map importance weighting: the gradient norm of each layer of the CNN feature extractor is calculated as importance by Taylor expansion, and the formula is:

[0029]

[0030] wherein, is the importance of the cth feature map in the lth layer at the tth task, Z l,c is the cth feature map in the lth layer, is an old task classification loss, and F represents a Frobenius norm;

[0031] By traversing the old task examples, the gradient norm of each layer of the CNN feature extractor is obtained by back propagation, is accumulated in the importance matrix , and is normalized, and the formula of the importance matrix is as follows:

[0032]

[0033] wherein, C lis the number of channels in the l-th layer of the CNN feature extractor;

[0034] (2) Distillation loss constraint feature space: by constraining the feature map difference between the old and new models, the key features of the old task are preserved, and the distillation loss The calculation formula is:

[0035]

[0036] Wherein, Z' l,c is the c-th feature map of the old model in the l-th layer, Z l,c is the c-th feature map of the new model in the l-th layer, B is the batch size, L is the total number of convolutional layers of the CNN feature extractor, and b is the training sample of the current round of training; The feature maps of the new and old models are obtained by forward propagation, and the distillation loss is calculated, and the classification loss L cls is weighted and summed to obtain the total loss

[0037] The calculation formula of the classification loss L cls is:

[0038]

[0039] Wherein, N is the number of samples in the current round of training, y i is the true label of the i-th training sample, p i is the prediction probability of the i-th training sample;

[0040] The calculation formula of the total loss is:

[0041]

[0042] Wherein, λ disc is the fixed weight parameter of the distillation loss , λ t is a weight coefficient dynamically adjusted with the task phase t; wherein, at the initial task, i.e. t = 1, the parameters of the CNN feature extractor and the parameters of the fully connected layer are randomly initialized, and at the subsequent task, i.e. t > 1, the parameters of the CNN feature extractor are updated and the parameters of the fully connected layer are dynamically expanded through the training of new class welding defect images; wherein, the dynamic expansion mechanism of the parameters of the fully connected layer takes the weight matrix W of the fully connected layer as the core, and the output dimension of the fully connected layer is 6 during the initial base class training, corresponding to the weight matrix The weight matrix is expanded to

[0043] Further, the evaluation of the trained welding defect detection model further comprises:

[0044] The average incremental accuracy and the forgetting rate are taken as the evaluation indexes of the trained welding defect detection model;

[0045] (1) The average incremental accuracy refers to the average value of the classification accuracy of the trained welding defect detection model on seen classes in all subsequent tasks. t The average incremental accuracy ACC of the trained welding defect detection model is calculated by calculating the accuracy ACC of the trained welding defect detection model on all seen classes of each subsequent task t. inc The calculation formula is as follows:

[0046]

[0047] Wherein, T is the total number of subsequent tasks, ACC is the accuracy of the trained welding defect detection model on all seen classes of the tth subsequent task. t

[0048] (2) The forgetting rate refers to the difference between the accuracy of the current model on the old task and the accuracy of the old model on the old task. current,s The trained welding defect detection model is obtained at the tth subsequent task, an old task s < t is selected, and the accuracy ACC of the trained welding defect detection model on the old task s is calculated. old,s The accuracy ACC of the welding defect detection model on the old task s when the old task s is completed.

[0049] The forgetting rate FR of the trained welding defect detection model is calculated by the calculation formula as follows: old,s current,s

[0050] Wherein, the forgetting rate FR is negative value indicating forgetting, and positive value indicating reverse transfer.

[0051] The state detection and online learning method for laser welding manufacturing provided by the application has the following beneficial effects: not only provides a more accurate and real-time quality monitoring means for the laser welding process, but also promotes the application and development of intelligent manufacturing technology in the welding field; by realizing intelligent control of the welding process, the industrial production efficiency is improved, the production cost is reduced, the product quality is guaranteed, and reliable technical support is provided for high-standard industries. BRIEF DESCRIPTION OF DRAWINGS

[0052] The accompanying drawings are used to provide a further understanding of the application, and constitute a part of the specification, and are used together with the following specific embodiments to explain the application, but do not constitute a limitation on the application. ​​

[0053] Figure 1 The flowchart illustrates a condition monitoring and online learning method for laser welding manufacturing provided by this invention.

[0054] Figure 2 The flowchart illustrates a specific implementation method for a condition monitoring and online learning approach in laser welding manufacturing provided by this invention.

[0055] Figure 3 This is a schematic diagram of the welding defect detection model provided by the present invention.

[0056] Figure 4 This diagram illustrates the training process of the welding defect detection model provided by the present invention. Detailed Implementation

[0057] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a condition detection and online learning method for laser welding manufacturing proposed according to the present invention. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the protection scope of the present invention.

[0058] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of the invention described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0059] This embodiment provides a condition detection and online learning method for laser welding manufacturing, such as... Figures 1-2 As shown, the condition monitoring and online learning method for laser welding manufacturing includes the following steps:

[0060] Step S1: Construct a multi-material welding defect image dataset;

[0061] Preferably, step S1 further includes:

[0062] During the laser welding manufacturing process, industrial cameras are used to collect images of six types of welding defects in stainless steel and four types of welding defects in aluminum alloy. The number of images of each type of welding defect is increased to more than 1,500 through data amplification methods such as horizontal flipping. Then, five levels of noise, blur, glare, and brightness interference are simulated on the amplified welding defect images to obtain the multi-material welding defect image dataset.

[0063] Among them, the six types of welding defects for stainless steel include good weld, burn-through, contamination, incomplete penetration, insufficient shielding gas, and hump; the four types of welding defects for aluminum alloys include penetration, burn-through, incomplete penetration, and hump.

[0064] It should be noted that the laser welding manufacturing process video was captured by an industrial camera, and the video was analyzed frame by frame to obtain image data of 6 types of welding defects in stainless steel and 4 types of welding defects in aluminum alloy, covering types such as good welds and burn-through. Through data amplification methods such as horizontal flipping and vertical flipping, the sample size of each type exceeded 1,500 images.

[0065] Specifically, the simulation of noise, blur, glare, and brightness interference of level 5 intensity on the amplified welding defect image also includes:

[0066] To simulate a complex industrial environment, Gaussian noise interference of intensity level 5 is simulated on the amplified welding defect image by adjusting the standard deviation σ (σ = 10 / 32.5 / 55 / 77.5 / 100); Gaussian blur interference of intensity level 5 is simulated on the amplified welding defect image by adjusting the standard deviation σ (1 to 5); glare interference of intensity level 5 is simulated on the amplified welding defect image by adjusting the intensity (100 to 200); and brightness interference of intensity level 5 is simulated on the amplified welding defect image by adjusting the brightness factor (0.2 to 2.0).

[0067] Step S2: Construct an initial welding defect detection model;

[0068] Preferably, such as Figure 3 As shown, step S2 further includes:

[0069] The initial welding defect detection model includes a CNN feature extractor and a fully connected layer. The CNN feature extractor extracts general features, and the fully connected layer is dynamically adjusted to adapt to new tasks. The CNN feature extractor includes an initial convolutional block, cascaded convolutional blocks, and a global average pooling layer. The welding defect image is input into the initial convolutional block for feature extraction, which outputs a 56×56×64 feature map to quickly extract basic common features of the welding defect image, such as weld edges and molten pool contours, laying the foundation for subsequent in-depth extraction of complex features. Then, the 56×56×64 feature map is input into the cascaded convolutional block for processing. The cascaded convolutional block sequentially processes the data to obtain 28×28×128, 14×14×256, and 7×7×512 feature maps, with the cascaded convolutional block outputting a 7×7×512 feature map. The 12-dimensional feature map, based on the extraction of basic common features, mines high-level features such as the detailed texture of defects and cross-regional correlations. Then, the 7×7×512 feature map is input into the global average pooling layer for compression. The global average pooling layer outputs a compact feature vector of 1×1×512. Finally, the compact feature vector of 1×1×512 is input into the fully connected layer for category prediction. The fully connected layer outputs the defect category prediction result of the welding defect image dynamically expanding with "old class + new class" (vertical dots on the right, old class above and new class below). At the same time, in order to prevent the forgetting of old class knowledge, a knowledge distillation constraint is introduced. First, the feature importance is calculated to quantify the importance of old class features, and then the feature extractor parameters are constrained by backpropagation of distillation loss. This fully covers the input, feature extraction, classification, knowledge distillation, and changes in data flow and feature map size.

[0070] The fully connected layer initially has 10 output dimensions, corresponding to the predicted probabilities of 10 categories of welding defects (6 for stainless steel and 4 for aluminum alloy). For each new category of welding defect, the output dimension increases by one. The cosine similarity between the new and old classes is calculated to enhance the discriminative power between them (avoiding overfitting of the new class). s The formula for calculating similarity(x,y) is:

[0071]

[0072] Here, x and y are two vectors, · represents the dot product, and ||x|| and ||y|| are the norms of vectors x and y, respectively.

[0073] In this embodiment of the invention, taking a stainless steel-aluminum alloy hybrid dataset as an example, feature stability is enhanced through normalization. Then, multiple convolutional blocks are cascaded to form an initial convolutional block and cascaded convolutional blocks. Each convolutional block includes a convolutional layer, a normalization layer (to accelerate training and reduce internal covariate bias), an activation function (ReLU, introducing nonlinearity to enhance feature representation), and a pooling layer, outputting a feature map (such as...). Figure 3 The 7×7×512 feature map in the image is converted into a feature vector (e.g., 7×7×512 feature map) through a global average pooling layer (GAP). Figure 3 (1×1×512 eigenvectors in the data).

[0074] In this embodiment of the invention, the output dimension of the fully connected layer dynamically expands with the task increment. Dimension refers to the number of category probabilities output by the classification head, corresponding one-to-one with the total number of defect categories to be distinguished. Dynamically expanding the dimension allows the model to adapt to the continuously increasing number of defect categories and output a corresponding number of prediction results. The output of the fully connected layer is a set of numerical values, each corresponding to the predicted probability of a category. "The output dimension is 10-dimensional" means that this set of numerical values ​​has 10 elements, each corresponding to the predicted probability of one of the 10 defect categories.

[0075] Step S3: Train the initial welding defect detection model based on the multi-material welding defect image dataset to obtain the trained welding defect detection model;

[0076] Preferably, such as Figure 4 As shown, step S3 further includes:

[0077] The six types of stainless steel welding defect images in the multi-material welding defect image dataset are used as the base class dataset. The base class dataset is input into the initial welding defect detection model for training to obtain the current welding defect detection model. The current welding defect detection model is used as the teacher model for subsequent knowledge distillation.

[0078] Next, based on the teacher model, four types of welding defect images of aluminum alloy in the multi-material welding defect image dataset are introduced as incremental categories. The parameters of the CNN feature extractor of the teacher model are fixed, and the output dimension of the fully connected layer is dynamically expanded. The current welding defect detection model is trained through a two-stage joint optimization strategy to obtain the student model after each incremental training. Then, the performance of the student model after each incremental training is verified in multiple dimensions.

[0079] The two-stage joint optimization strategy includes a first-stage optimization strategy that uses the teacher model to generate soft labels to pass on historical classification experience, and a second-stage optimization strategy that integrates hard label cross-entropy loss and soft label distillation loss. Here, the soft label refers to the probability distribution of defect categories output by the teacher model for the base class dataset, and the hard label refers to the one-hot encoding of the new welding defect image.

[0080] Specifically, such as Figure 4 As shown, Task 1 involves inputting six types of welding defect images for stainless steel (good weld, burn-through, contamination, incomplete penetration, insufficient shielding gas, and hump). An initial welding defect detection model is used for base class training, outputting a stable teacher model. This teacher model identifies the six types of stainless steel defects through a fully connected layer. Tasks 2-5 sequentially introduce four types of welding defect images for aluminum alloy (penetration, burn-through, incomplete penetration, and hump). Only one new type of welding defect image is added each time for model iteration, resulting in a student model. The student model inherits the "knowledge" of the teacher model and accurately classifies the newly added aluminum alloy defects through a fully connected layer. The total number of tasks is 5.

[0081] In the detection inference process during training, the cross-material detection workflow involves inputting a stainless steel-aluminum alloy hybrid dataset, preprocessing it, generating feature vectors through a CNN feature extractor, and then having a dynamic classification head output a probability distribution based on the current number of defect categories. This probability distribution is combined with class center cosine similarity to achieve cross-material defect classification. Furthermore, for multi-environment testing, simulated industrial interference of five levels, including noise and blur, is used to generate multi-environment test data.

[0082] Specifically, in the first stage optimization strategy, a teacher-student model architecture is adopted. The teacher model is trained on the base class dataset and its parameters are fixed, serving only as knowledge. The student model initially inherits the parameters of the CNN feature extractor of the teacher model. The fully connected layer is dynamically expanded according to the number of new categories, but the parameters are randomly initialized. Using the soft labels generated by the teacher model, the student model only optimizes the parameters of the fully connected layer and uses classification loss to fit the soft labels in order to quickly learn the category distribution pattern of the old task. This stage can further improve the convergence speed of the student model for the old categories compared to random initialization, thereby improving the learning efficiency of the student model.

[0083] In the second-stage optimization strategy, old-class welding defect images and new-class welding defect images are mixed in a 1:1 ratio as the training set. The loss function is the sum of the classification loss and the distillation loss. The distillation loss is weighted by the importance of the feature maps to constrain the difference between the feature maps of the old and new models. When updating parameters, the CNN feature extractor only updates the non-critical feature parameters related to the new-class welding defect images. The old class weights of the fully connected layers are frozen, and the new class weights are optimized with a learning rate of 0.01.

[0084] In the incremental learning process, during task initialization, a training set containing the new category and an old task example set containing 20 representative samples from each category are divided, and the old model parameters are loaded to retain old knowledge. By randomly sampling 2000 samples from the old task example set, the gradient norm of the feature maps at each layer is calculated to locate key features (such as the melt pool contour) and generate an importance matrix. When updating parameters, the backbone network (convolutional layer) only updates non-critical parameters related to the new task, while the classification head freezes the old class weights and randomly initializes and trains the new class weights. The optimizer uses the SGD optimizer, configured with an initial learning rate of 0.1, momentum of 0.9, decay of 0.1 every 20 epochs, batch size of 128, and 90 epochs per task.

[0085] More specifically, in the second-stage optimization strategy, in order to solve the catastrophic forgetting in incremental learning (features of the old task are covered by the new task), the current welding defect detection model (student model) is trained by constraining the feature space through the feature map importance weighting and distillation loss;

[0086] (1) Feature map importance weighting: Traditional knowledge distillation applies importance to all features. Figure 1 Treating all features equally leads to key old features being covered by new task features. The impact of the Taylor expansion approximates the model update on the old task loss, and the gradient norm of the feature maps of each layer of the CNN feature extractor is calculated as the importance, using the following formula:

[0087]

[0088] in, The importance of the c-th feature map in the l-th layer for the t-th task, Z l,c It is the c-th feature map in the l-th layer. It is the old task classification loss, where F represents the Frobenius norm (F-norm), which is used to measure the "size" of the gradient matrix;

[0089] By iterating through old task examples, backpropagation is used to obtain the gradient norm of the feature maps of each layer of the CNN feature extractor, which is then accumulated into the importance matrix. In the middle, and normalized, the importance matrix The formula is as follows:

[0090]

[0091] Among them, C l It is the number of channels in the l-th layer of the CNN feature extractor;

[0092] (2) Distillation loss constrains the feature space: By constraining the differences in feature maps between the old and new models (teacher model and student model) (higher importance feature maps are penalized more severely), key features of the old task are preserved, and distillation loss is applied. The calculation formula is:

[0093]

[0094] Among them, Z' l,cZ is the c-th feature map in layer l of the old model (teacher model). l,c This is the c-th feature map of the new model (student model) in layer l, where B is the batch size, L is the total number of convolutional layers in the CNN feature extractor, and b is the training sample for the current training round. Forward propagation simultaneously acquires feature maps from both the old and new models, and the distillation loss is calculated. and classification loss L cls Weighted summation to obtain the total loss

[0095] Classification loss L cls The calculation formula is:

[0096]

[0097] Where N is the number of samples in the current training round, y i p is the true label of the i-th training sample. i It is the predicted probability of the i-th training sample;

[0098] Total loss The calculation formula is:

[0099]

[0100] Where, λ disc Distillation loss The fixed weight parameter, λ t These are weight coefficients that are dynamically adjusted according to the task stage t. Used together, these two factors allow the model to achieve both λ and t-weighting during incremental learning. disc Forcefully retain key old features, while also being able to use λ t Adaptively adjust constraint strength to avoid excessive constraints affecting the learning efficiency of new classes;

[0101] The parameter update strategy for the feature extractor involves random initialization of both the CNN feature extractor and the fully connected layer parameters during the initial task (t=1). During subsequent tasks (t>1), the CNN feature extractor parameters are partially updated (adjusting only non-critical features sensitive to the new task) through training with images of new welding defects. The fully connected layer parameters are dynamically expanded (weights for new categories are randomly initialized, while weights for older categories are frozen or fine-tuned). The dynamic expansion mechanism of the fully connected layer's parameters is centered on the weight matrix W. During initial base class training, the output dimension of the fully connected layer is 6 (e.g., 6 categories for stainless steel), corresponding to the weight matrix W. When training the new k-th class, the weight matrix is ​​expanded to...

[0102] In the embodiments of the present invention, the CNN feature extractor extracts hierarchical features through multiple convolutional blocks, generates a compact feature vector by combining global average pooling, and retains the key features of the old task through importance-weighted distillation of feature maps. This design balances the stability and flexibility of features in class incremental learning, effectively alleviates catastrophic forgetting, and is particularly suitable for industrial scenarios such as welding defect detection that require incremental learning of new defect categories. The output dimension of the fully connected layer dynamically expands with the defect categories, and the cosine similarity calculation of class centers is introduced to enhance the discrimination between new and old categories.

[0103] Step S4: Evaluate the trained welding defect detection model. After passing the evaluation, input the current welding image to be detected into the trained welding defect detection model for detection, so as to output the defect category detection result of the current welding image to be detected.

[0104] Preferably, in the evaluation of the trained welding defect detection model, it further includes:

[0105] Use the average incremental accuracy and forgetting rate as the evaluation metrics for the trained welding defect detection model; the average incremental accuracy is used to measure the comprehensive generalization ability of the student model for new and old classes, and the forgetting rate is used to measure the degree of catastrophic forgetting;

[0106] (1) The average incremental accuracy refers to the average of the classification accuracies of the trained welding defect detection model on the seen classes in all subsequent tasks. For each subsequent task t, calculate the accuracy ACC of the trained welding defect detection model on all seen classes (classes of the first t tasks) in this subsequent task t , so as to calculate the average incremental accuracy ACC of the trained welding defect detection model inc , and the calculation formula is:

[0107]

[0108] Among them, T is the total number of subsequent tasks, and ACC t is the accuracy of the trained welding defect detection model on all seen classes in the t-th subsequent task;

[0109] (2) The forgetting rate refers to the difference between the accuracy of the current model on the old task and the accuracy of the old model on this old task. Among them, when obtaining the trained welding defect detection model at the t-th subsequent task, select the old task s < t, and calculate the accuracy ACC of the trained welding defect detection model on the old task s current,s and the accuracy ACC of the welding defect detection model at the completion of the old task s on the old task s old,s , so as to calculate the forgetting rate FR of the trained welding defect detection model, and the calculation formula is:

[0110] FR = Acc old,s -ACC current,s

[0111] Here, the forgetting rate b is negative to indicate forgetting, and positive to indicate backward transfer.

[0112] In this embodiment of the invention, taking the stainless steel-aluminum alloy dataset (10 classes) as an example, a defect detection and online learning method that expands the categories by incremental learning is adopted. The overall process is as follows: (1) First, the 6 types of defects in stainless steel are used as the base class dataset. Welding images are collected by an industrial camera and expanded to a single class sample size of ≥1500 images. The CNN feature extractor with 3 levels of residual blocks and the fully connected layer with an initial dimension of 6 are input for training to obtain the teacher model as the basis for subsequent knowledge distillation; (2) Then, the 4 types of defects in aluminum alloy are introduced as incremental categories on the basis of the teacher model (1 new category is added each time). The backbone network parameters of the teacher model are fixed and the output dimension of the fully connected layer is dynamically expanded (1 new dimension is added each time). In the first stage, the soft label with temperature T=2 is generated by the teacher model to transmit the base class knowledge. In the second stage, the cross-entropy loss of hard labels and the feature map importance of weight λ=0.8 are fused. A two-stage joint optimization strategy using weighted distillation loss is employed to train the student model, enabling collaborative optimization of new and old knowledge and mitigating catastrophic forgetting. Soft labels refer to the probability distribution of the teacher model's output on old task data, while hard labels are the one-hot encoding of new task data. For example, the label for "weld penetration" defects in aluminum alloys (4 types) is (0,0,1,0), with 1 only at the corresponding category position and 0 for the rest. Then, the average accuracy and forgetting rate of the student model after each incremental training are validated in multiple dimensions. The tasks are divided into Task 1, training on the 6 base classes of stainless steel (teacher model), and Tasks 2-5, which sequentially add 4 types of defects in aluminum alloys (1 type each time). A total of 5 tasks gradually expand the output dimension of the fully connected layer from 6 dimensions to 10 dimensions. This process achieves incremental optimization and industrial scenario adaptation of the welding defect detection model through dynamic expansion of the fully connected layer and a feature importance-weighted distillation mechanism.

[0113] The present invention provides a state detection and online learning method for laser welding manufacturing, including an online learning framework based on adaptive feature consolidation, a generalization ability verification method based on multi-material defect dataset construction and environmental simulation, a model building method based on knowledge distillation and dynamic classification head, and a model training and evaluation method based on two-stage joint optimization.

[0114] Furthermore, in the aforementioned state detection and online learning method for laser welding manufacturing, the adaptive feature consolidation online learning framework balances new and old knowledge through feature importance weighted distillation to mitigate catastrophic forgetting. Specifically, it uses Taylor expansion to approximate the impact of model updates on the loss of old tasks, calculates the gradient norm of each layer's feature map as an importance index, locates key low-frequency features, constructs a weighted distillation loss function, applies stronger penalties to feature maps with high importance, and forces the new and old models to maintain consistency in key features.

[0115] Furthermore, the construction of the multi-material defect dataset and environmental simulation involved collecting 10 types of weld images, including 6 types of stainless steel and 4 types of aluminum alloy. Through data amplification, the sample size of each type was increased to more than 1,500 images. Simulation data was generated according to 5 intensity levels for 4 types of interference factors, including noise, blur, glare, and brightness. A multi-environment test set containing 20 categories was constructed to quantify the robustness of the model under different working conditions.

[0116] Furthermore, the knowledge distillation and dynamic classification head model construction adopts multi-level convolutional blocks of convolutional neural networks to extract hierarchical features, generates compact feature vectors through global average pooling, designs an scalable fully connected classification layer, the output dimension dynamically increases with the defect category, and introduces class center cosine similarity calculation to enhance the distinction between new and old categories. At the same time, the teacher model generates soft labels to convey the category distribution knowledge of the old task, and integrates hard labels and soft labels to train the student model.

[0117] Furthermore, in the two-stage joint optimization model training and evaluation, the first stage is to fix the teacher model to generate soft labels, and the second stage is to integrate hard labels and soft labels to train the student model. The average incremental accuracy and forgetting rate are used as evaluation indicators. The average incremental accuracy is the average classification accuracy of the model on the seen classes in all incremental stages, and the forgetting rate is the difference between the accuracy of the current model on the old task and the accuracy of the old model on the old task.

[0118] In summary, the state detection and online learning method for laser welding manufacturing provided by this invention is an online learning detection algorithm using a knowledge distillation framework based on adaptive feature consolidation. This algorithm addresses the real-time, robustness, and incremental learning problems in laser welding defect detection, and is applicable to multi-material and multi-environment welding quality monitoring in industrial automated production. Specifically, a multi-material weld dataset containing six types of stainless steel and four types of aluminum alloy is constructed. Data amplification, including horizontal flipping, and simulation of noise, blur, glare, and brightness interference at five levels are performed to generate multi-environment test data. An online learning model based on knowledge distillation is designed. Hierarchical features are extracted through the basic convolutional layers of a CNN feature extractor. A dynamic classification head dynamically expands the output dimension according to the defect category, and class center cosine similarity calculation is introduced to avoid overfitting to new classes. Weighted distillation based on feature map importance is used, and Taylor expansion quantifies the gradient norm of old task features, imposing stronger constraints on key features to mitigate catastrophic forgetting. The training process employs a two-stage joint optimization, fusing hard / soft label loss to update the model. Experiments show that this method exhibits good generalization ability and robustness under cross-material detection and complex environmental interference, providing effective support for real-time laser welding quality detection.

[0119] This invention delves into the application of online learning technology in the field of laser welding. Online learning offers advantages such as strong adaptability and high flexibility, enabling real-time updates and optimization of models based on new data. By acquiring laser welding images online in real time and constructing an intelligent laser welding image database, this project will study online monitoring technology for laser welding. This will enable online monitoring and identification of laser welding data under different environments (such as low light, various noise levels, etc.) and data on different welding materials (such as iron, zinc, etc.). This method overcomes the technical difficulties of insufficient real-time feedback and overly subjective judgment in traditional welding processes, significantly improving welding quality, accelerating production pace, reducing defect rates, and lowering energy consumption. This paper not only provides a more accurate and real-time quality monitoring method for the laser welding process but also promotes the application and development of intelligent manufacturing technology in the welding field. By realizing intelligent control of the welding process, it helps improve industrial production efficiency, reduce production costs, and ensure product quality, providing reliable technical support for high-standard industries.

[0120] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A condition monitoring and online learning method for laser welding manufacturing, characterized in that, The condition monitoring and online learning method for laser welding manufacturing includes the following steps: Step S1: Construct a multi-material welding defect image dataset; Step S2: Construct an initial welding defect detection model; Step S3: Train the initial welding defect detection model based on the multi-material welding defect image dataset to obtain the trained welding defect detection model; Step S4: Evaluate the trained welding defect detection model. If the evaluation is successful, input the current welding image to be detected into the trained welding defect detection model for detection, and output the defect category detection result of the current welding image to be detected. Step S3 further includes: The six types of stainless steel welding defect images in the multi-material welding defect image dataset are used as the base class dataset. The base class dataset is input into the initial welding defect detection model for training to obtain the current welding defect detection model. The current welding defect detection model is used as the teacher model for subsequent knowledge distillation. Next, based on the teacher model, four types of welding defect images of aluminum alloy in the multi-material welding defect image dataset are introduced as incremental categories. The parameters of the CNN feature extractor of the teacher model are fixed, and the output dimension of the fully connected layer is dynamically expanded. The current welding defect detection model is trained through a two-stage joint optimization strategy to obtain the student model after each incremental training. Then, the performance of the student model after each incremental training is verified in multiple dimensions. The two-stage joint optimization strategy includes a first-stage optimization strategy that uses a teacher model to generate soft labels to pass on historical classification experience, and a second-stage optimization strategy that integrates hard label cross-entropy loss and soft label distillation loss. Here, soft labels refer to the probability distribution of defect categories output by the teacher model for the base class dataset, and hard labels refer to the one-hot encoding of new welding defect images. In the first stage optimization strategy, a teacher-student model architecture is adopted. The teacher model is trained on the base class dataset and the parameters are fixed. It is only used as knowledge. The student model initially inherits the parameters of the CNN feature extractor of the teacher model. The fully connected layer is dynamically expanded according to the number of new categories, but the parameters are randomly initialized. The soft labels generated by the teacher model are used. The student model only optimizes the parameters of the fully connected layer and uses classification loss to fit the soft labels. In the second-stage optimization strategy, old-class welding defect images and new-class welding defect images are mixed in a 1:1 ratio as the training set. The loss function is the sum of the classification loss and the distillation loss. The distillation loss is weighted by the importance of the feature maps to constrain the difference between the feature maps of the old and new models. When updating the parameters, the CNN feature extractor only updates the non-critical feature parameters related to the new-class welding defect images. The old class weights of the fully connected layers are frozen, and the new class weights are optimized with a learning rate of 0.

01. In the second-stage optimization strategy, the current welding defect detection model is trained by constraining the feature space through the feature map importance weighting and distillation loss; (1) Feature map importance weighting: The gradient norm of the feature maps of each layer of the CNN feature extractor is calculated by Taylor expansion and used as the importance. The formula is: ； in, It is the first The first task The first in the layer The importance of each feature map It is the first The first in the layer Each feature map It is a loss due to the old task classification. Denotes the Frobenius norm; By iterating through old task examples, backpropagation is used to obtain the gradient norm of the feature maps of each layer of the CNN feature extractor, which is then accumulated into the importance matrix. In the middle, and normalized, the importance matrix The formula is as follows: ； in, It is the first CNN feature extractor The number of channels in the layer; (2) Distillation loss constrains the feature space: by constraining the differences in feature maps between the old and new models, key features of the old task are preserved, and the distillation loss is applied. The calculation formula is: ； in, The old model is in the first The first in the layer Each feature map The new model is in the first The first in the layer Feature map It refers to the batch size. It is the total number of convolutional layers in the CNN feature extractor. These are the training samples for the current training round; forward propagation simultaneously acquires feature maps of both the old and new models, and the distillation loss is calculated. and classification loss Weighted summation to obtain the total loss ; Classification loss The calculation formula is: ； in, It is the number of samples in the current training round. It is the first The true labels of each training sample It is the first The predicted probability of each training sample; Total loss The calculation formula is: ； in, Distillation loss Fixed weight parameters, It is according to the task phase Dynamically adjusted weighting coefficients; where, during the initial task, i.e. =1, the parameters of the CNN feature extractor and the fully connected layer are randomly initialized, and will be used in subsequent tasks, i.e. When the value is greater than 1, the parameters of the CNN feature extractor are updated through training with images of new types of welding defects, and the parameters of the fully connected layer are dynamically expanded. The dynamic parameter expansion mechanism of the fully connected layer uses the weight matrix of the fully connected layer... W As the core, the output dimension of the fully connected layer during the initial base class training is 6, corresponding to the weight matrix. , added During training, the weight matrix is ​​expanded to .

2. The condition detection and online learning method for laser welding manufacturing according to claim 1, characterized in that, Step S1 further includes: During the laser welding manufacturing process, industrial cameras are used to collect images of six types of welding defects in stainless steel and four types of welding defects in aluminum alloy. The number of images of each type of welding defect is increased to more than 1,500. Then, five levels of noise, blur, glare, and brightness interference are simulated on the increased welding defect images to obtain the multi-material welding defect image dataset. Among them, the six types of welding defects for stainless steel include good weld, burn-through, contamination, incomplete penetration, insufficient shielding gas, and hump; the four types of welding defects for aluminum alloys include penetration, burn-through, incomplete penetration, and hump.

3. The condition detection and online learning method for laser welding manufacturing according to claim 2, characterized in that, The simulation of noise, blur, glare, and brightness interference of level 5 intensity on the amplified welding defect image also includes: The amplified welding defect image is simulated with Gaussian noise interference of intensity level 5 by adjusting the standard deviation σ; the amplified welding defect image is simulated with Gaussian blur interference of intensity level 5 by adjusting the standard deviation σ; the amplified welding defect image is simulated with glare interference of intensity level 5 by adjusting the intensity; and the amplified welding defect image is simulated with luminance interference of intensity level 5 by adjusting the luminance factor.

4. The condition detection and online learning method for laser welding manufacturing according to claim 2, characterized in that, Step S2 further includes: The initial welding defect detection model includes a CNN feature extractor and a fully connected layer. The CNN feature extractor includes an initial convolutional block, a cascaded convolutional block, and a global average pooling layer. The welding defect image is input into the initial convolutional block for feature extraction, and the initial convolutional block outputs a 56×56×64 feature map. Then, the 56×56×64 feature map is input into the cascaded convolutional block for processing, and the cascaded convolutional block outputs a 7×7×512 feature map. Next, the 7×7×512 feature map is input into the global average pooling layer for compression, and the global average pooling layer outputs a 1×1×512 compact feature vector. Finally, the 1×1×512 compact feature vector is input into the fully connected layer for category prediction, and the fully connected layer outputs the defect category prediction result for the welding defect image. The fully connected layer initially has 10 output dimensions, corresponding to the predicted probabilities of 10 types of welding defects. For each new type of welding defect, the output dimension increases by one. The discriminative power between the new and old classes is enhanced by calculating the cosine similarity between the input features of the new class and the centers of the old classes. The calculation formula is: ； in, and There are two vectors. Represents the dot product. and They are vectors The norm of .

5. The condition detection and online learning method for laser welding manufacturing according to claim 1, characterized in that, The evaluation of the trained welding defect detection model also includes: The average incremental accuracy and forgetting rate are used as evaluation metrics for the trained welding defect detection model. (1) Average incremental accuracy refers to the average classification accuracy of the trained welding defect detection model on the seen classes across all subsequent tasks. For each subsequent task The accuracy of the trained welding defect detection model on all known classes in the subsequent task is calculated. The average incremental accuracy of the trained welding defect detection model is calculated. The calculation formula is: ； in, This represents the total number of subsequent tasks. The trained welding defect detection model is used in the first... Accuracy of all seen classes in each subsequent task; (2) The forgetting rate refers to the difference between the accuracy of the current model on the old task and the accuracy of the old model on the same old task. Among them, in the th... The trained welding defect detection model is obtained during the subsequent task, and the old task is selected. s < The trained welding defect detection model was calculated for the old task. s accuracy and completing old tasks s The welding defect detection model in the old task s accuracy To calculate the forgetting rate of the trained welding defect detection model. The calculation formula is: ； Among them, the forgetting rate Negative values ​​indicate forgetting, while positive values ​​indicate reverse migration.

Images