Laser vision welding seam automatic tracking method and system based on sustainable learning

Abstract

The invention discloses a laser vision welding seam automatic tracking method and system based on sustainable learning. The method comprises the following steps: acquiring an initial welding seam image to obtain coordinate values of initial welding seam feature points under a pixel coordinate system; constructing an initial semantic segmentation network model, and training the pruned semantic segmentation network model in sequence by adopting the preprocessed training subsets to obtain a lightweight semantic segmentation network model; when a novel welding seam is welded, a lightweight semantic segmentation network model is updated and trained based on a playback continuous learning strategy, a welding seam image collected by an embedded industrial control computer is denoised to obtain pure and noiseless welding seam stripe image data, and the pure and noiseless welding seam stripe image data is sent to a robot control cabinet in real time to be processed. And a welding gun on the welding robot is controlled to move along the welding seam track of the workpiece to complete automatic welding seam tracking. According to the method, the problem of disastrous forgetting of an existing model can be effectively solved, and the welding robustness can be greatly improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of weld seam tracking, and particularly relates to a laser vision automatic weld seam tracking method and system based on sustainable learning. Background Technique

[0002] As an important part of industrial manufacturing, the welding process is widely used in industries such as automobile processing, shipbuilding, and general machinery. With the development of industrial automation, automatic welding of robots has been widely used in industrial production. However, in the "manual teaching - learning" automatic welding mode, although automatic welding has achieved a certain degree of automation, its anti - interference ability is weak, with large errors, and it is difficult to meet the requirements of high efficiency, high precision, and high quality in current welding production applications. Therefore, with the continuous development of deep learning, the automatic weld seam tracking method based on deep learning using a structured light vision sensor has been widely used due to its strong robustness, high precision, and other advantages.

[0003] In the prior art, the automatic weld seam tracking method based on deep learning often completes the deep learning task according to the paradigm of supervised learning. When facing the diverse environment of weld seam structures, such methods often need to re - learn new types of laser stripes and adjust the algorithm model parameters to adapt to the welding of new types of weld seams. Over time, the existing weld seam tracking methods will encounter the "catastrophic forgetting problem", that is, after the weld seam tracking method adapts to the new weld seam structure distribution, the robustness and tracking accuracy on the old weld seam types will decline. Therefore, in the current environment of increasingly diverse weld seam structures, there is an urgent need for a weld seam tracking method that can overcome the "catastrophic forgetting" problem. Summary of the Invention

[0004] The purpose of the present invention is to overcome the above - mentioned deficiencies in the prior art, and provide a laser vision automatic weld seam tracking method based on sustainable learning. This method updates and trains the lightweight semantic segmentation network model in the embedded industrial control machine through a replay - based continuous learning strategy, thereby effectively overcoming the catastrophic forgetting problem in the prior art. When the lightweight semantic segmentation network model learns new weld seam types, it will not affect the tracking accuracy of the previous old weld seam types, and can greatly improve the welding robustness.

[0005] Meanwhile, another purpose of the present invention is to provide a laser vision automatic weld seam tracking system based on sustainable learning.

[0006] The purpose of the present invention is achieved through the following technical solutions:

[0007] A laser vision automatic weld seam tracking method based on sustainable learning includes the following steps:

[0008] S1. Before the welding work starts, the industrial camera in the laser vision sensor sends the collected initial weld seam image to the embedded industrial control machine. The embedded industrial control machine performs an initialization operation on the initial weld seam image to obtain the coordinate values of the initial weld seam feature points in the pixel coordinate system of the initial weld seam image, and converts them into three-dimensional coordinate values based on the basic coordinate system of the welding robot;

[0009] S2. Construct an initial semantic segmentation network model and perform initialization training. Replace the ordinary convolution layer of the standard module in the initialized and trained semantic segmentation network model with a depthwise separable convolution layer to obtain a semantic segmentation network model; perform channel pruning operation on the semantic segmentation network model to obtain a pruned semantic segmentation network model; use the preprocessed training subset to sequentially train the pruned semantic segmentation network model to obtain a trained semantic segmentation network model, and use the trained semantic segmentation network model as a lightweight semantic segmentation network model; save the lightweight semantic segmentation network model and the weld seam training subset used during its training to the embedded industrial control machine;

[0010] S3. When welding starts, identify whether the weld seam on the workpiece belongs to a new weld seam type; if the weld seam on the workpiece is of an unknown weld seam type, collect weld seam image data through the industrial camera in the laser vision sensor, and make the weld seam image data into a training sample set after preprocessing. Use the replay continuous learning strategy to update and train the lightweight semantic segmentation network model in the embedded industrial control machine to obtain a trained and updated lightweight semantic segmentation network model, and replace the original lightweight semantic segmentation network model in the embedded industrial control machine with the trained and updated lightweight semantic segmentation network model; if it is identified that the weld seam on the workpiece is of a known weld seam type, jump to step S4;

[0011] S4. Use the lightweight semantic segmentation network in the embedded industrial control machine to denoise the weld seam image collected by the embedded industrial control machine to obtain pure and noise-free weld seam stripe image data; use an efficient convolution operator to perform a tracking algorithm process on the pure and noise-free weld seam stripe image data to calculate the pixel coordinate values of the weld seam feature points corresponding to the collected weld seam image; convert the pixel coordinate values of the weld seam feature points into three-dimensional coordinate values in the base coordinate system of the welding robot, and send them to the robot control cabinet for processing in real time, control the robot control cabinet of the welding robot to process, and control the welding torch on the welding robot to move along the weld seam trajectory of the workpiece to complete automatic weld seam tracking.

[0012] Preferably, the specific steps of step S1 are as follows:

[0013] S11. Before the welding work starts, adjust the position and posture of the robotic arm of the welding robot so that the end of the welding torch is above the starting position of the weld of the workpiece to be welded and the welding torch is perpendicular to the surface of the workpiece, and make the laser vision sensor fixed on the welding torch in the optimal working position; the optimal working position refers to the position where the laser vision sensor can capture clear welding images during the welding process and will not interfere with the workpiece to be welded;

[0014] S12. The industrial camera in the laser vision sensor acquires the initial weld image of the workpiece surface and sends the initial weld image to the embedded industrial control machine. The embedded industrial control machine performs threshold processing by calling the library functions of the Halcon software and performs initialization operations using the morphological correction method to obtain the coordinate values of the initial weld feature points in the pixel coordinate system;

[0015] S13. The embedded industrial control machine converts the coordinate values of the initial weld feature points in the pixel coordinate system into three-dimensional coordinate values based on the basic coordinate system of the welding robot.

[0016] Preferably, the specific steps of step S2 are as follows:

[0017] S21. Construct an initial semantic segmentation network model and perform initialization training to obtain the initialized and trained semantic segmentation network model; the initial semantic segmentation network model uses the ENet neural network model; the ENet network model includes an initialization module and a standard module;

[0018] S22. Replace the ordinary convolutional layer of the standard module in the initialized and trained semantic segmentation network model with a depthwise separable convolutional layer to obtain the semantic segmentation network model; the depthwise separable convolutional layer includes a depth convolutional layer and a pointwise convolutional layer;

[0019] S23. Repeatedly perform channel pruning operations on the semantic segmentation network model by setting the normalization layer scaling factor weight threshold. When the segmentation accuracy of the model drops to a preset value, stop the pruning operation and perform fine-tuning training on the pruned semantic segmentation network model to obtain the pruned semantic segmentation network model; the loss function during the fine-tuning training of the pruned semantic segmentation network model is expressed as follows:

[0020]

[0021] where x and y are the input and output of the normalization layer in sequence, γ is the scaling factor of the normalization layer, θ is the parameter of the model, l() is the standard cross-entropy loss function, and λ is the weight constant of the regularization term;

[0022] S24. Obtain weld images of different types, and construct multiple corresponding weld training subsets according to different weld types. Only image data of the same weld type is included in the weld training subsets. Process the training subsets using the threshold segmentation method, extract the laser stripe masks corresponding to the weld images in the training subsets, and use the laser stripe masks as labels. Sequentially train the pruned semantic segmentation network model with the processed training subsets to obtain a trained semantic segmentation network model, and use the trained semantic segmentation network model as a lightweight semantic segmentation network model;

[0023] S25. Save the lightweight semantic segmentation network model and the weld training subsets used during its training to the embedded industrial control machine.

[0024] Preferably, the specific steps of step S3 are as follows:

[0025] S31. At the start of welding, identify whether the weld on the workpiece belongs to a new weld type. If the weld on the workpiece is of an unknown weld type, control the industrial camera in the laser vision sensor to move along the weld and continuously collect weld images. If it is recognized that the weld image is of a known weld type, jump to step S4;

[0026] S32. Construct a new weld image dataset based on the weld images collected in step S31. Use the threshold segmentation method to extract the laser stripe masks corresponding to the weld images in the new weld image dataset, and use the laser stripe masks as labels for marking. Divide a training subset from the processed new weld image dataset and perform clipping processing on the training subset;

[0027] S33. Update and train the lightweight semantic segmentation network model in the embedded industrial control machine using the replay continuous learning strategy; and save the updated and trained lightweight semantic segmentation network model and replace it at the position of the original lightweight semantic segmentation network model in the embedded industrial control machine;

[0028] S34. Use the contribution score value to screen the training samples of the lightweight semantic segmentation network model obtained in step S33. When the contribution score value is greater than the preset threshold, the sample is saved as a training subset to the embedded industrial control machine;

[0029] S35. Use the bilinear interpolation method to reduce the sample size of the training subset in the embedded industrial control machine.

[0030] Preferably, the specific process of step S33 is as follows:

[0031] After extracting the lightweight semantic segmentation network model from the storage space of the embedded industrial control machine, copy the lightweight semantic segmentation network model into a lightweight semantic segmentation network model A and a lightweight semantic segmentation network model B with the same model parameters in the cloud;

[0032] S332. Randomly extract samples from the training subset containing labels in step S32 and input them into the lightweight semantic segmentation network model A for training, and calculate the learning loss value of the lightweight semantic segmentation network model A;

[0033] S333. Randomly extract old training samples from the storage space of the embedded industrial control machine and input them into the lightweight semantic segmentation network model B for inference to obtain pseudo-labels, bring the pseudo-labels into the recall loss function, and calculate the recall loss value of the lightweight semantic segmentation network model A;

[0034] S334. Sum the learning loss value and the recall loss value to obtain the total loss function;

[0035] S335. Repeat steps S332 - S34 to update the model parameters by the stochastic gradient descent method until the training cycle reaches the preset value;

[0036] S336. Save the updated and trained lightweight semantic segmentation network model and replace it with the original lightweight semantic segmentation network model in the embedded industrial control machine.

[0037] Preferably, the total loss function in step S334 is expressed as follows:

[0038]

[0039] Among them, L k (θ) is the learning loss value of the model, L 1:k-1 (θ) is the recall loss value of the model, is the pseudo-label, λ is the hyperparameter, and l(,) is the standard cross-entropy loss function.

[0040] Preferably, the specific expression of the contribution score value in step S34 is as follows:

[0041]

[0042] Among them, is the contribution score of the sample, can measure the contribution of the sample to the parameter change. To prevent the parameter change from being too small and inconvenient for calculation, a constant term ξ is added and set to 1; ω i can measure the contribution of the sample to the reduction of the current batch loss function: The contribution of each parameter θ to the change in the loss of the k-th batch during training.

[0043] The method of using bilinear interpolation described in step S35 is adopted to reduce the sample size of the training subset in the embedded industrial control machine.

[0044] Since as the types of weld seams processed by the lightweight semantic segmentation network model increase, the storage cost of the training subset stored in the embedded industrial control machine becomes higher and higher. Therefore, in order to reduce the storage cost, at the cost of adding noise to the samples, the bilinear interpolation method is adopted to sequentially reduce the size of the training subset stored in the embedded industrial control machine, which is specifically expressed as follows:

[0045] H j+1 = H j ×α, W j+1 = W j ×α, j = 1, 2, …, k - 1

[0046] Where, H j and W j respectively represent the length and width of the j-th batch of training subsets, and α represents the reduction ratio.

[0047] A laser vision weld automatic tracking system based on sustainable learning, including a welding robot, a welding torch, a welding robot control cabinet, supporting welding equipment, a laser vision sensor, an embedded industrial control machine, and a welding workbench. The workpiece is fixed on the welding workbench. The laser vision sensor is installed on the welding torch. The welding torch is installed at the end of the welding robot. The supporting welding equipment provides energy, welding materials, and a protector for the welding torch. The laser vision sensor, the welding robot control cabinet, the laser vision sensor, and the embedded industrial control machine are connected through communication. The embedded industrial control machine includes a processor and a memory. Non-temporary computer instructions are stored on the memory. When the non-temporary computer instructions are run by the processor, the above-mentioned laser vision weld automatic tracking method based on sustainable learning is executed.

[0048] Preferably, the laser vision sensor includes a sensor housing, an industrial camera, a light-transmitting partition, and a laser generator. The industrial camera and the laser generator are fixedly arranged in the sensor housing. The light-transmitting partition is fixedly installed on the sensor housing, and the light-transmitting partition is arranged in front of the industrial camera and the laser generator.

[0049] The present invention has the following advantages over the prior art:

[0050] (1) The automatic laser vision weld seam tracking method based on sustainable learning of the present invention updates and trains the lightweight semantic segmentation network model in the embedded industrial control machine by adopting a replay-based continuous learning strategy, thereby effectively overcoming the catastrophic forgetting problem in the prior art. When the lightweight semantic segmentation network model learns new weld seam types, it will not affect the tracking accuracy of the previous old weld seam types, and can greatly improve the welding robustness.

[0051] (2) The automatic laser vision weld seam tracking method based on sustainable learning of the present invention can select the sample subset that contributes the most to the decrease in the loss value after each update training of the lightweight semantic segmentation network model, and use the bilinear interpolation method to shrink the sample subset, thereby being able to relieve the storage cost pressure and endowing the model with the ability to learn more weld seam types. Description of the Drawings

[0052] Figure 1 is a schematic flow chart of the automatic laser vision weld seam tracking method based on sustainable learning provided in Embodiment 1 of the present invention;

[0053] Figure 2 is a schematic structural diagram of the initialization module of the ENet neural network model in Embodiment 1 of the present invention;

[0054] Figure 3 is a schematic structural diagram of the initialization module of the ENet neural network model in Embodiment 1 of the present invention;

[0055] Figure 4 is a schematic flow chart of the method for pruning the semantic segmentation network model in Embodiment 1 of the present invention;

[0056] Figure 5 is a schematic structural diagram of the automatic laser vision weld seam tracking system based on sustainable learning provided in Embodiment 2 of the present invention;

[0057] Figure 6 is a schematic structural diagram of the laser vision sensor in Embodiment 2 of the present invention;

[0058] In the figure: 1 - supporting welding equipment; 2 - welding robot; 3 - support pad; 4 - workpiece; 5 - welding torch; 6 - external connection part of the laser sensor; 7 - laser vision sensor; 71 - sensor housing; 72 - industrial camera; 73 - light-transmitting partition; 74 - laser generator; 8 - welding workbench; 9 - embedded industrial control machine; 10 - control cabinet. Detailed Embodiments

[0059] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are only a part rather than all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the scope of protection of the present invention.

[0060] The technical solutions of the present invention will be further described below with reference to the accompanying drawings and through specific implementation manners.

[0061] Embodiment 1

[0062] As Figures 1 - 4 shown, a laser vision weld automatic tracking method based on sustainable learning includes the following steps:

[0063] S1. Before the welding operation starts, the industrial camera 72 in the laser vision sensor 7 sends the collected initial weld image to the embedded industrial control machine 9. The embedded industrial control machine 9 performs an initialization operation on the initial weld image to obtain the coordinate values of the initial weld feature points in the pixel coordinate system of the initial weld image, and converts them into three-dimensional coordinate values based on the basic coordinate system of the welding robot 2.

[0064] The specific steps of step S1 are as follows:

[0065] S11. Before the welding operation starts, adjust the position and posture of the mechanical arm of the welding robot 2 so that the end of the welding torch 5 is above the starting position of the weld of the workpiece to be welded and the welding torch 5 is perpendicular to the surface of the workpiece 4, and make the laser vision sensor 7 fixed on the welding torch 5 in the best working position. The best working position refers to the position where the laser vision sensor 7 can capture a clear welding image during the welding process and will not interfere with the workpiece 4 to be welded.

[0066] S12. The industrial camera 72 in the laser vision sensor 7 acquires the initial weld image on the surface of the workpiece 4 and sends the initial weld image to the embedded industrial control machine 9. The embedded industrial control machine 9 performs threshold processing by calling the library functions of the Halcon software and performs an initialization operation using the morphological correction method to obtain the coordinate values of the initial weld feature points in the pixel coordinate system.

[0067] S13. The embedded industrial control machine 9 converts the coordinate values of the initial weld feature points in the pixel coordinate system into three-dimensional coordinate values based on the basic coordinate system of the welding robot 2.

[0068] S2. Construct an initial semantic segmentation network model and perform initialization training. Replace the ordinary convolutional layers in the standard modules of the initialized and trained semantic segmentation network model with depthwise separable convolutional layers to obtain a semantic segmentation network model. Perform channel pruning on the semantic segmentation network model to obtain a pruned semantic segmentation network model. Use the preprocessed training subsets to sequentially train the pruned semantic segmentation network model to obtain a trained semantic segmentation network model, and use the trained semantic segmentation network model as a lightweight semantic segmentation network model. Save the lightweight semantic segmentation network model and the weld training subsets used during its training to the embedded industrial control machine 9.

[0069] The specific steps of step S2 are as follows:

[0070] S21. Construct an initial semantic segmentation network model and perform initialization training to obtain an initialized and trained semantic segmentation network model. The initial semantic segmentation network model uses an ENet neural network model. The ENet network model includes an initialization module and a standard module.

[0071] S22. Replace the ordinary convolutional layers in the standard modules of the initialized and trained semantic segmentation network model with depthwise separable convolutional layers to obtain a semantic segmentation network model. The depthwise separable convolutional layer includes a depthwise convolutional layer and a pointwise convolutional layer.

[0072] S23. Repeatedly perform channel pruning on the semantic segmentation network model by setting the scaling factor weight threshold of the normalization layer. When the segmentation accuracy of the model drops to a preset value, stop the pruning operation and perform fine-tuning training on the pruned semantic segmentation network model to obtain a pruned semantic segmentation network model. The loss function during the fine-tuning training of the pruned semantic segmentation network model is expressed as follows:

[0073]

[0074] where x and y are the input and output of the normalization layer in sequence, γ is the scaling factor of the normalization layer, θ is the parameter of the model, l() is the standard cross-entropy loss function, and λ is the weight constant of the regularization term.

[0075] S24. Obtain weld images of different types and construct multiple corresponding weld training subsets according to different weld types. Only image data of the same weld type is included in the weld training subsets. Use the method of threshold segmentation to process the training subsets, extract the laser stripe masks corresponding to the weld images in the training subsets, and use the laser stripe masks as labels. Use the processed training subsets to sequentially train the pruned semantic segmentation network model to obtain a trained semantic segmentation network model, and use the trained semantic segmentation network model as a lightweight semantic segmentation network model.

[0076] S25. Save the lightweight semantic segmentation network model and the weld training subset used during its training to the embedded industrial control machine 9.

[0077] S3. At the start of welding, identify whether the weld on the workpiece 4 belongs to a new weld type. If the weld on the workpiece 4 is of an unknown weld type, collect weld image data through the industrial camera 72 in the laser vision sensor 7, preprocess the weld image data and make it into a training sample set, and update and train the lightweight semantic segmentation network model in the embedded industrial control machine 9 using the replay continuous learning strategy to obtain an updated and trained lightweight semantic segmentation network model, and replace the original lightweight semantic segmentation network model in the embedded industrial control machine 9 with the updated and trained lightweight semantic segmentation network model. If it is identified that the weld on the workpiece 4 is of a known weld type, jump to step S4.

[0078] The specific steps of step S3 are as follows:

[0079] S31. At the start of welding, identify whether the weld on the workpiece 4 belongs to a new weld type. If the weld on the workpiece is of an unknown weld type, control the industrial camera 72 in the laser vision sensor 7 to move along the weld and continuously collect weld images. If it is identified that the weld image is of a known weld type, jump to step S4.

[0080] S32. Construct a new weld image dataset based on the weld images collected in step S31, use the threshold segmentation method to extract the laser stripe mask corresponding to the weld images in the new weld image dataset, and mark the laser stripe mask as a label. Divide a training subset from the processed new weld image dataset and perform cropping processing on the training subset.

[0081] S33. Update and train the lightweight semantic segmentation network model in the embedded industrial control machine 9 using the replay continuous learning strategy; and save the updated and trained lightweight semantic segmentation network model and replace it in the position of the original lightweight semantic segmentation network model in the embedded industrial control machine 9.

[0082] The specific process of step S33 is as follows:

[0083] S331. After extracting the lightweight semantic segmentation network model from the storage space of the embedded industrial control machine 9, copy the lightweight semantic segmentation network model into a lightweight semantic segmentation network model A and a lightweight semantic segmentation network model B with the same model parameters in the cloud.

[0084] S332. Randomly extract samples from the training subset containing tags obtained in step S32 and input them into the lightweight semantic segmentation network model A for training, and calculate the learning loss value of the lightweight semantic segmentation network model A;

[0085] S333. Randomly extract old training samples from the storage space of the embedded industrial control machine 9 and input them into the lightweight semantic segmentation network model B for inference to obtain pseudo-labels, bring the pseudo-labels into the recall loss function, and calculate the recall loss value of the lightweight semantic segmentation network model A;

[0086] S334. Sum the learning loss value and the recall loss value to obtain the total loss function;

[0087] The total loss function described in step S334 is expressed as follows:

[0088]

[0089] Among them, L k (θ) is the learning loss value of the model, and L 1:k-1 (θ) is the recall loss value of the model. is the pseudo-label, λ is the hyperparameter, and l(,) is the standard cross-entropy loss function.

[0090] S335. Repeat steps S332 - S34 to update the model parameters by the stochastic gradient descent method until the training cycle reaches the preset value;

[0091] S336. Save the updated and trained lightweight semantic segmentation network model and replace it at the original position of the lightweight semantic segmentation network model in the embedded industrial control machine 9.

[0092] S34. Screen the training samples of the lightweight semantic segmentation network model obtained in step S33 using the contribution score value. When the contribution score value is greater than the preset threshold, the sample is saved as a training subset in the embedded industrial control machine 9. The process of screening the training samples is expressed as follows:

[0093]

[0094] Among them, represents the i-th sample in the dataset of the k-th type of weld seam; is the score evaluating the contribution of this sample to the decrease in the loss value during training:

[0095] Specifically, the specific expression of the contribution score value is as follows:

[0096]

[0097] Among them, is the contribution score of the sample, which can measure the sample 's contribution to the parameter change. To prevent the parameter change from being too small and inconvenient for calculation, a constant term ξ is added and set to 1; ω i can measure the sample 's contribution to the reduction of the loss function in the current batch: is the contribution of each parameter θ to the loss change in the k-th batch training.

[0098] S35. Use the bilinear interpolation method to reduce the sample size of the training subset in the embedded industrial control machine 9.

[0099] The step S35 uses the bilinear interpolation method to reduce the sample size of the training subset in the embedded industrial control machine 9.

[0100] Since as the types of welds processed by the lightweight semantic segmentation network model increase, the storage cost of the training subset saved in the embedded industrial control machine 9 becomes higher and higher. Therefore, in order to reduce the storage cost, at the cost of adding noise to the sample, the bilinear interpolation method is used to sequentially reduce the size of the training subset saved in the embedded industrial control machine 9, which is specifically expressed as follows:

[0101] H j+1 = H j × α, W j+1 = W j × α, j = 1, 2, …, k - 1

[0102] where, H j and W j respectively represent the length and width of the k-th batch training subset, both set to 300; α represents the reduction ratio, set to 0.5.

[0103] S4. Use the lightweight semantic segmentation network in the embedded industrial control machine 9 to denoise the weld image collected by the embedded industrial control machine 9 to obtain pure and noise-free weld stripe image data; use an efficient convolution operator to perform a tracking algorithm on the pure and noise-free weld stripe image data to calculate the pixel coordinate values of the weld feature points corresponding to the collected weld image; convert the pixel coordinate values of the weld feature points into three-dimensional coordinate values in the base coordinate system of the welding robot 2 and send them to the welding robot control cabinet 10 for processing in real time, control the welding robot control cabinet 10 for processing, and control the welding torch 5 on the welding robot 2 to move along the weld track of the workpiece 4 to complete automatic weld tracking.

[0104] Embodiment 2

[0105] As Figure 5As shown in the figure, a laser vision weld automatic tracking system based on sustainable learning includes a welding robot 2, a welding torch 5, a welding robot control cabinet 10, a supporting welding equipment 1, a laser vision sensor 7, a supporting cushion block 3, an embedded industrial control computer 9, and a welding workbench 8. The supporting cushion block 3 is placed on the welding workbench 8, and the workpiece 4 is fixed on the supporting cushion block 3. Moreover, the inclination angle of the workpiece 4 can be adjusted by the supporting cushion block 3, so that the workpiece 4 has different postures, and different welding conditions are also available when the welding robot 2 is welding. The laser vision sensor 7 is installed on the welding torch 5 through an external connector 6. The welding torch 5 is installed at the end of the welding robot 2. The supporting welding equipment 1 provides energy, welding materials, and a protector for the welding torch 5. The laser vision sensor 7, the welding robot control cabinet 10, and the embedded industrial control computer 9 are communicatively connected. The embedded industrial control computer 9 includes a processor and a memory. A non-temporary computer instruction is stored on the memory. When the non-temporary computer instruction is run by the processor, it executes the laser vision weld automatic tracking method based on sustainable learning as described in Embodiment 1.

[0106] As Figure 5 shown in the figure, the welding torch 5 is installed at the end of the welding robot 2 through a welding torch clamping mechanism. The welding torch clamping mechanism consists of a fixture fixing seat and bolts and nuts, and is used to place the welding torch 5 in the fixture fixing seat and tightly connect it with bolts and nuts.

[0107] The supporting welding equipment 1 includes a welding machine and a shielding gas cylinder. The welding machine uses MOTOWELD-RD350 of Yaskawa brand and is used for wire feeding and wire withdrawing of the robot. The shielding gas cylinder is filled with carbon dioxide (20%) and nitrogen (80%) to play a role in protection during the welding process.

[0108] Specifically, the welding robot control cabinet 10, the welding robot 2, and the supporting welding equipment 1 are connected by cable wires. The spatial position of the laser vision sensor 7 and the welding torch 5 changes with the movement of the welding robot 2.

[0109] As Figure 6 shown in the figure, the laser vision sensor 7 includes a sensor housing 71 with black oxide treatment, an industrial camera 72, a light-transmitting partition 73, and a laser generator 74. The industrial camera 72 and the laser generator 74 are fixed inside the sensor housing 71. The light-transmitting partition 73 is fixed on the sensor housing 71 and is located in front of the industrial camera 72 and the laser generator 74. The laser generator 74 is tightly connected to the sensor housing 71 by bolts and nuts and forms an angle of 30° with the industrial camera 72.

[0110] Specifically, a G-type fixture is provided on the welding workbench 8, and the workpiece 4 is placed on the support pad 3 and clamped and positioned by two or more G-type fixtures.

[0111] In the description of the present invention, it should be noted that unless otherwise clearly specified and agreed, the terms "set", "installed", and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0112] The above embodiments are the preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications made without departing from the spirit and principle of the present invention shall be equivalent replacement methods and are all included in the protection scope of the present invention.

Claims

1. A laser vision weld seam automatic tracking method based on sustainable learning, characterized in that: The following steps are involved: S1. Before welding begins, the industrial camera in the laser vision sensor sends the collected initial weld image to the embedded industrial control computer, which performs initialization operation on the initial weld image, obtains the coordinate value of the initial weld feature point in the initial weld image in the pixel coordinate system, and converts it into a three-dimensional coordinate value based on the basic coordinate system of the welding robot; S2. Construct an initial semantic segmentation network model and perform initialization training, replace the ordinary convolution layer of the standard module in the semantic segmentation network model after initialization training with a depth-separable convolution layer to obtain a semantic segmentation network model; perform channel pruning operation on the semantic segmentation network model to obtain a pruned semantic segmentation network model; use the preprocessed training subset to train the pruned semantic segmentation network model in sequence to obtain a trained semantic segmentation network model, and use the trained semantic segmentation network model as a lightweight semantic segmentation network model; save the lightweight semantic segmentation network model and the weld training subset used in its training to the embedded industrial control computer; S3, when welding starts, identify whether the weld on the workpiece belongs to a new weld type; if the weld on the workpiece is of an unknown weld type, collect weld image data through the industrial camera in the laser vision sensor, and pre-process the weld image data to make a training sample set, and use the playback-based continuous learning strategy to update the lightweight semantic segmentation network model in the embedded industrial control machine to obtain a lightweight semantic segmentation network model after training and updating, and replace the original lightweight semantic segmentation network model in the embedded industrial control machine with the lightweight semantic segmentation network model after training and updating; if the weld on the workpiece is identified as a known weld type, jump to step S4; S4, using the lightweight semantic segmentation network in the embedded industrial control computer to perform denoising on the weld image collected by the embedded industrial control computer to obtain pure and noise-free weld stripe image data; using an efficient convolution operator to perform tracking algorithm processing on the pure and noise-free weld stripe image data to calculate the pixel coordinate values ​​of the weld feature points corresponding to the collected weld image; The pixel coordinate values ​​of the weld feature points are converted into three-dimensional coordinate values ​​in the welding robot base coordinate system, and sent to the robot control cabinet for processing in real time. The welding robot control cabinet is controlled to process, and the welding gun on the welding robot is controlled to move along the weld trajectory of the workpiece to complete automatic weld tracking.

2. The laser vision weld seam automatic tracking method based on sustainable learning according to claim 1 is characterized in that: The specific steps of step S1 are as follows: S11. Before welding begins, adjust the position and posture of the welding robot arm so that the end of the welding gun is located above the starting position of the weld seam of the workpiece to be welded and the welding gun is perpendicular to the workpiece surface, and the laser vision sensor fixed on the welding gun is in the best working position; the best working position refers to the position where the laser vision sensor can capture a clear welding image and will not interfere with the welded workpiece during welding; S12, the industrial camera in the laser vision sensor obtains an initial weld image on the workpiece surface, and sends the initial weld image to the embedded industrial control computer, the embedded industrial control computer performs threshold processing by calling the library function of the Halcon software, and performs initialization operation using the morphological correction method to obtain the coordinate value of the initial weld feature point in the pixel coordinate system; S13, the embedded industrial control computer converts the coordinate value of the initial weld feature point in the pixel coordinate system into a three-dimensional coordinate value based on the basic coordinate system of the welding robot.

3. The laser vision weld seam automatic tracking method based on sustainable learning according to claim 1 is characterized in that: The specific steps of step S2 are as follows: S21, constructing an initial semantic segmentation network model and performing initialization training to obtain a semantic segmentation network model after initialization training; the initial semantic segmentation network model adopts an ENet neural network model; the ENet network model includes an initialization module and a standard module; S22, replacing the common convolution layer of the standard module in the initialized trained semantic segmentation network model with a depthwise separable convolution layer to obtain a semantic segmentation network model; the depthwise separable convolution layer includes a depthwise convolution layer and a pointwise convolution layer; S23, repeatedly performing channel pruning operation on the semantic segmentation network model by setting a normalized layer scaling factor weight threshold, when the segmentation accuracy of the model drops to a preset value, stopping the pruning operation, and fine-tuning the pruned semantic segmentation network model to obtain a pruned semantic segmentation network model; the loss function of the pruned semantic segmentation network model during fine-tuning training is expressed as follows: Among them, x and y are the input and output of the normalization layer respectively, γ is the scaling factor of the normalization layer, θ is the parameter of the model, l() is the standard cross entropy loss function, and λ is the weight constant of the regularization term; S24, obtaining different types of weld images, and constructing a plurality of corresponding weld training subsets according to different weld types, wherein the weld training subsets only contain image data of the same weld type; processing the training subsets by using a threshold segmentation method, extracting laser stripe masks corresponding to the weld images in the training subsets, and using the laser stripe masks as labels, sequentially training the pruned semantic segmentation network models with the processed training subsets to obtain a trained semantic segmentation network model, and using the trained semantic segmentation network model as a lightweight semantic segmentation network model; S25. Save the lightweight semantic segmentation network model and the weld training subset used for training to the embedded industrial control computer.

4. The laser vision weld seam automatic tracking method based on sustainable learning according to claim 1 is characterized in that: The specific steps of step S3 are as follows: S31, when welding starts, identifying whether the weld on the workpiece is of a new weld type. If the weld on the workpiece is of an unknown weld type, controlling the industrial camera in the laser vision sensor to move along the weld and continuously collecting weld images; if the weld image is identified as a known weld type, jumping to step S4; S32, constructing a new weld image dataset based on the weld image collected in step S31, extracting the laser stripe mask corresponding to the weld image in the new weld image dataset by using a threshold segmentation method, and marking the laser stripe mask as a label; Dividing a training subset from the processed new weld image data set, and performing a trimming process on the training subset; S33, using a playback-based continuous learning strategy to update and train the lightweight semantic segmentation network model in the embedded industrial control machine; and saving and replacing the updated lightweight semantic segmentation network model to the original lightweight semantic segmentation network model position in the embedded industrial control machine; S34, using the contribution score value to screen the training samples of the lightweight semantic segmentation network model obtained in step S33, when the contribution score value is greater than a preset threshold, the sample is saved as a training subset to the embedded industrial control computer; S35. Use a bilinear interpolation method to reduce the sample size of the training subset in the embedded industrial control computer.

5. The laser vision weld seam automatic tracking method based on sustainable learning according to claim 4 is characterized in that: The specific process of step S33 is as follows: S331, after extracting the lightweight semantic segmentation network model from the storage space of the embedded industrial control machine, copying the lightweight semantic segmentation network model into lightweight semantic segmentation network model A and lightweight semantic segmentation network model B with the same model parameters in the cloud; S332, randomly extracting samples from the training subset containing labels described in step S32 and inputting them into the lightweight semantic segmentation network model A for training, and calculating the learning loss value of the lightweight semantic segmentation network model A; S333, randomly extracting old training samples from the storage space of the embedded industrial control computer and inputting them into the lightweight semantic segmentation network model B for inference to obtain pseudo labels, bringing the pseudo labels into the recall loss function, and calculating the recall loss value of the lightweight semantic segmentation network model A; S334, summing the learning loss value and the recall loss value to obtain a total loss function; S335, repeating steps S332-S34 to update the model parameters by stochastic gradient descent method until the training cycle reaches the preset value; S336, saving the updated trained lightweight semantic segmentation network model and replacing it to the original lightweight semantic segmentation network model position in the embedded industrial control machine.

6. The laser vision weld seam automatic tracking method based on sustainable learning according to claim 5 is characterized in that: The total loss function in step S334 is expressed as follows: Among them, L k (θ) is the learning loss value of the model, L 1:k-1 (θ) is the recall loss value of the model, is a pseudo label, λ is a hyperparameter, and l(,) is the standard cross entropy loss function.

7. The laser vision weld seam automatic tracking method based on sustainable learning according to claim 4 is characterized in that: The specific expression of the contribution score value in step S34 is as follows: in, is the contribution score of the sample, Samples can be measured Contribution to parameter change. In order to prevent the parameter change from being too small and inconvenient to calculate, a constant term ξ is added and set to 1; ω i Samples can be measured Contribution to the reduction of the loss function of the current batch: is the contribution of each parameter θ to the change in loss of the kth batch during training.

8. The laser vision weld seam automatic tracking method based on sustainable learning according to claim 4 is characterized in that: The specific process of reducing the sample size of the training subset in the embedded industrial control machine by using the bilinear interpolation method in step S35 is as follows: As the number of weld types processed by the lightweight semantic segmentation network model increases, the storage cost of the training subsets saved by the embedded industrial control machine becomes increasingly high. Therefore, in order to reduce the storage cost, a bilinear interpolation method is used at the cost of adding noise to the sample to successively reduce the size of the training subsets saved by the embedded industrial control machine, as specifically expressed as follows: H j+1 =H j ×α,W j+1 =W j ×α,j=1,2,…,k-1 Among them, H j and W j They are respectively represented as the length and width of the j-th batch of training subsets, and α is represented as the reduction factor.

9. A system for implementing the laser vision weld seam automatic tracking method based on sustainable learning as claimed in claim 1, characterized in that: It includes a welding robot, a welding gun, a welding robot control cabinet, supporting welding equipment, a laser vision sensor, an embedded industrial control machine and a welding workbench, the workpiece is fixed on the welding workbench, the laser vision sensor is installed on the welding gun, the welding gun is installed at the end of the welding robot, the supporting welding equipment provides energy, welding materials and a protector for the welding gun, the laser vision sensor, the welding robot control cabinet, the laser vision sensor and the embedded industrial control machine are connected through communication, the embedded industrial control machine includes a processor and a memory, the memory stores non-temporary computer instructions, when the non-temporary computer instructions are executed by the processor, the laser vision weld automatic tracking method based on sustainable learning as described in any one of claims 1-7 is executed.

10. The laser vision weld seam automatic tracking system based on sustainable learning according to claim 9 is characterized in that: The laser vision sensor includes a sensor housing, an industrial camera, a light-transmitting partition and a laser generator. The industrial camera and the laser generator are fixedly arranged in the sensor housing, the light-transmitting partition is fixedly installed on the sensor housing, and the light-transmitting partition is arranged in front of the industrial camera and the laser generator.

Images