Laser welding robot defect detection method and system based on visual guidance

Abstract

The invention relates to the technical field of image recognition, and provides a laser welding robot defect detection method and system based on visual guidance. The method comprises the steps that a laser welding robot is monitored in real time through a visual sensor, and a welding synchronous image is obtained; carrying out adaptive defect detection on the image to obtain a first welding defect map, and carrying out multi-scale evolution prediction to construct a second map; molten pool defect compensation is optimized according to the second map, and a first strategy is obtained; weld defect compensation is optimized, and a second strategy is obtained; fusing defect compensation is optimized, a third strategy is obtained, welding optimization control is carried out in combination with the first strategy and the second strategy, and therefore the technical problems that due to the fact that a traditional laser welding defect detection method depends on single-scale feature analysis and lacks multi-angle cooperative detection capacity, the welding quality is unstable, and defects cannot be corrected in time are solved. The technical effects of improving the welding quality, reducing defects and optimizing the welding process precision are achieved.

Description

Technical Field

[0001] This application relates to the field of image recognition technology, specifically to a method and system for defect detection in laser welding robots based on vision guidance. Background Technology

[0002] Laser welding technology, as a highly efficient and precise welding method, has been widely used in aerospace, automotive manufacturing, electronic equipment, and many other fields. It uses a high-energy laser beam to melt the workpiece, achieving the welding purpose. Compared with traditional welding methods, laser welding has significant advantages such as high precision, a low heat-affected zone, and high welding efficiency. However, welding defects often occur during laser welding, such as unstable molten pools, uneven welds, and poor fusion. These defects not only affect welding quality but may also lead to a decrease in the structural strength of the product, affecting its service life. To improve the quality and efficiency of laser welding, defect detection and real-time compensation have become particularly important. Traditional defect detection methods rely on manual experience or offline sampling, which suffer from insufficient real-time performance and weak multi-area collaborative monitoring capabilities. On the one hand, single-area monitoring (such as focusing only on the molten pool or weld) cannot simultaneously capture the dynamic defect characteristics of the molten pool, weld, and fusion area, leading to missed or misjudged defects. On the other hand, detection algorithms with fixed thresholds cannot adapt to the diversity of defect morphologies during welding (such as the scale difference between micropores and cracks), causing compensation strategies to lag or over-correct, making it difficult to achieve precise control of welding quality. Summary of the Invention

[0003] This application provides a vision-guided laser welding robot defect detection method and system, aiming to solve the technical problems of traditional laser welding defect detection methods, which rely on single-scale feature analysis and lack multi-angle collaborative detection capabilities, resulting in unstable welding quality and inability to correct defects in a timely manner. The system achieves the technical effects of real-time identification and multi-scale dynamic prediction of welding defects through vision guidance, intelligent optimization of welding control scheme, improved welding quality, reduced defects, and optimized welding process accuracy.

[0004] The first aspect disclosed in this application provides a vision-guided defect detection method for laser welding robots. The method includes: real-time monitoring of the laser welding robot using a vision sensor to obtain a welding synchronization image, the welding synchronization image including a molten pool region image, a weld seam region image, and a fusion region image; performing region-adaptive defect detection on the welding synchronization image to obtain a first welding defect map, and performing multi-scale evolution prediction on the first welding defect map to construct a second welding defect map; optimizing the welding control scheme of the laser welding robot based on the second welding defect map to obtain a first welding optimization strategy; optimizing the welding control scheme based on the second welding defect map to obtain a second welding optimization strategy; optimizing the welding control scheme based on the second welding defect map to obtain a third welding optimization strategy; and combining the first and second welding optimization strategies for welding optimization control.

[0005] Another aspect of this application discloses a vision-guided laser welding robot defect detection system, comprising: a real-time monitoring module for real-time monitoring of the laser welding robot using a vision sensor to obtain synchronous welding images, including images of the molten pool region, weld seam region, and fusion region; a defect evolution module for performing region-adaptive defect detection on the synchronous welding images to obtain a first welding defect map, and performing multi-scale evolution prediction on the first welding defect map to construct a second welding defect map; a molten pool defect compensation module for optimizing the welding control scheme of the laser welding robot based on the second welding defect map to obtain a first welding optimization strategy; a weld seam defect compensation module for optimizing the welding control scheme based on the second welding defect map to obtain a second welding optimization strategy; and a welding optimization and control module for optimizing the welding control scheme based on the second welding defect map to obtain a third welding optimization strategy, and combining the first and second welding optimization strategies for welding optimization and control.

[0006] One or more technical solutions provided in this application have at least the following technical effects or advantages: The aforementioned vision-guided defect detection method for laser welding robots first uses a vision sensor to monitor the laser welding robot in real time, acquiring synchronous images of the molten pool, weld seam, and fusion zone. Then, by performing region-adaptive defect detection on these images, a preliminary defect map is generated, and further multi-scale evolution prediction is performed to obtain a more accurate defect map. Next, based on this defect map, the welding control scheme is optimized, and compensation optimizations are performed for defects in the molten pool, weld seam, and fusion zone, resulting in three optimization strategies. Finally, these optimization strategies are combined to form a comprehensive optimization and control scheme for the welding process, ensuring improved welding quality.

[0007] The above description is merely an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, specific embodiments of this application are given below. Attached Figure Description

[0008] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0009] Figure 1 This is a flowchart illustrating a vision-guided laser welding robot defect detection method in one embodiment.

[0010] Figure 2 This is an architecture diagram of a vision-guided laser welding robot defect detection system in one embodiment.

[0011] Figure labeling: Real-time monitoring module 11, Defect evolution module 12, Molten pool defect compensation module 13, Weld defect compensation module 14, Welding optimization and control module 15. Detailed Implementation

[0012] This application provides a vision-guided laser welding robot defect detection method and system, which solves the technical problems of traditional laser welding defect detection methods that rely on single-scale feature analysis and lack multi-angle collaborative detection capabilities, resulting in unstable welding quality and inability to correct defects in a timely manner. The system achieves the technical effects of real-time identification and multi-scale dynamic prediction of welding defects through vision guidance, intelligent optimization of welding control scheme, improved welding quality, reduced defects, and optimized welding process accuracy.

[0013] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0014] It should be noted that the terms “comprising” and “having”, and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or modules that are not explicitly listed or that are inherent to such process, method, product, or device.

[0015] Example 1, as Figure 1 As shown, this application provides a vision-guided defect detection method for laser welding robots, the method comprising: The laser welding robot is monitored in real time using a vision sensor to obtain synchronous welding images, which include images of the molten pool area, the weld seam area, and the fusion area.

[0016] In this embodiment, a vision sensor is used to monitor the laser welding robot in real time, acquiring image data of key areas during the welding process. The acquired image data is then segmented according to the feature vector of each area, resulting in synchronous welding images including images of the molten pool area, weld seam area, and fusion area. The molten pool area image corresponds to the area where metal melts during laser welding, determining the quality and stability of the weld. By capturing images of the molten pool area, the size, shape, temperature changes, and dynamic behavior of the molten pool can be monitored in real time, providing basic data for subsequent defect detection and control. The weld seam area image corresponds to the weld seam, the joint connecting two metal workpieces. The quality of the weld seam directly affects the structural strength of the welded part. The weld seam area image captures detailed information about the metal connection line formed during the welding process, including the width, depth, uniformity, and presence of defects such as cracks and porosity. The fusion area image corresponds to the fusion region, the area where the edge of the molten pool fuses with the surface of the metal to be welded. Good fusion ensures the strength and sealing of the welded joint. The fusion area image is used to monitor the fusion of the molten pool and the substrate, checking for over-melting, insufficient fusion, or welding defects. By acquiring synchronized images of these three types of regions, real-time changes during the welding process can be accurately captured, providing high-quality image data for subsequent defect detection, prediction, and compensation.

[0017] Furthermore, this application provides real-time monitoring of a laser welding robot using a vision sensor to obtain synchronous welding images, including: A welding monitoring image is obtained based on the visual sensor; the welding monitoring image is enhanced to obtain a welding enhanced image; regional feature detection is performed on the welding enhanced image to obtain a multi-dimensional regional feature vector; the welding enhanced image is adaptively segmented based on the multi-dimensional regional feature vector to generate the welding synchronization image.

[0018] Preferably, firstly, the laser welding process is monitored in real time using a vision sensor (such as a high-resolution camera or industrial camera) to acquire welding monitoring images of various areas during the welding process. These images include multiple areas such as the molten pool area, weld seam area, and fusion area, recording the dynamic changes of these areas during the welding process and providing important raw data for subsequent defect detection and quality control. Subsequently, the acquired welding monitoring images are enhanced. The purpose of image enhancement is to improve image quality, making key features in the image (such as the contours, color contrast, and details of the welding area) clearer. Common enhancement methods include contrast enhancement, edge enhancement, and noise removal. Through enhancement processing, the welding monitoring images can be converted into enhanced welding images, helping to highlight the features of the welding area and reduce interference from environmental noise, thereby improving the accuracy of subsequent processing. For example, histogram equalization can be used to enhance image contrast, improving brightness and making details in the welding area clearer. Then, edge detection algorithms (such as the Sobel operator) are applied to highlight the contours of the welding area, helping to better identify the edges of the weld and molten pool. Median filtering is then used to remove noise from the image, preserving important features while reducing environmental interference, ensuring the image quality of the welding area is suitable for subsequent processing and analysis. Next, the enhanced welding image is input into a pre-trained convolutional neural network (CNN) for region feature detection. The CNN extracts features from different regions of the image, such as texture, shape, color, and spatial location, through a series of convolutional layers, pooling layers, and activation functions (such as ReLU). Fully connected layers or pooling layers then output multi-dimensional feature vectors. These feature vectors contain deep learning features of the corresponding regions, fully reflecting the key feature information of the welding area. The output layer summarizes these feature vectors to form a complete multi-dimensional region feature vector, which is used in subsequent image segmentation. The CNN used is pre-trained using welding images from different regions and labeled regional features. Specifically, historical welding image data is first acquired, and each image is labeled with the welding region and its corresponding features using manual or automated tools. Then, a CNN model is initialized, including input layers, convolutional layers, pooling layers, fully connected layers, and output layers. By inputting the prepared historical data into the CNN, steps such as forward propagation, loss calculation, backpropagation, and parameter optimization are performed, allowing it to gradually learn how to extract features from different regions of the welding image data. After training, the model is validated using unseen data. If the accuracy on this unseen data meets the preset accuracy, the current CNN is stored; otherwise, the model is optimized by adjusting the learning rate, batch size, adding regularization, and adjusting the network structure.Then, based on the generated multi-region feature vector, the welding enhancement image is adaptively segmented. In this process, the image is divided into multiple regions with different features (such as molten pool, weld, fusion zone, etc.) according to the positional features of each region in the multi-region feature vector, thus forming a series of synchronous welding images. These synchronous welding images contain images of various important regions (such as molten pool, weld, fusion zone) during the welding process, providing complete and accurate data support for subsequent welding defect detection, defect identification, compensation and optimization strategies.

[0019] Furthermore, this application provides that the multi-dimensional region feature vector includes a molten pool region feature vector, a weld region feature vector, and a fusion region feature vector.

[0020] Preferably, the multi-dimensional feature vectors extracted by CNN include multiple multi-dimensional feature vectors, namely, the molten pool region feature vector, the weld seam region feature vector, and the fusion region feature vector. The molten pool region feature vector includes information such as the region's geometry, temperature distribution, texture features, and spatial location, reflecting the stability of the molten pool and its dynamic changes during the welding process. The weld seam region feature vector includes information such as the weld seam's shape, width, uniformity, texture, and spatial location, reflecting the strength and reliability of the welded structure. The fusion region feature vector includes information such as the fusion region's shape, size, fusion depth, and spatial location, reflecting whether there is sufficient fusion between the molten pool and the substrate, and whether there are instances of incomplete fusion or over-melting. These feature vectors comprehensively reflect the key information of different welding regions in the image, providing a basis for image segmentation.

[0021] Regional adaptive defect detection is performed on the welding synchronous image to obtain a first welding defect map, and multi-scale evolution prediction is performed on the first welding defect map to construct a second welding defect map.

[0022] In one embodiment, after obtaining the welding synchronization image, region-adaptive defect detection is performed on the welding synchronization image. This involves performing multi-dimensional defect detection on the images of various regions in the welding image to accurately identify welding defects in different regions, such as insufficient molten pool, weld cracks, or poor fusion. By summarizing the detection results of different regions, a first welding defect atlas is generated. This first welding defect atlas contains the defect type, defect features, and location detected in each region. Subsequently, the first welding defect atlas is input into a Long Short-Term Memory (LSTM) network with different time scales. By predicting the welding process at different time scales, the short-term, medium-term, and long-term trends of defects during the welding process are captured. The LSTM is constructed in a similar manner to the one described above, using forward propagation, loss calculation, backpropagation, and parameter optimization. Through multi-scale evolution prediction, based on the information in the first welding defect atlas, corresponding scale defect prediction results can be generated at each time scale. By weighting the prediction results according to the weighting coefficients of each time scale, a scale-fused defect prediction result is obtained. This scale-fused defect prediction result is then used as the basic data to construct the second welding defect atlas. This second welding defect atlas reflects the state of welding defects after multi-scale evolution, including the evolution trend of defects and the possible range of changes. It provides more accurate and comprehensive data support for subsequent welding control scheme optimization and defect compensation, enabling intelligent adjustment and optimization based on the evolution law of defects, thereby ensuring the stability and improvement of welding quality.

[0023] Furthermore, this application provides a method for performing region-adaptive defect detection on the synchronous welding image to obtain a first weld defect map, including: Multidimensional defect detection is performed on the molten pool region image to obtain molten pool defect detection results; multidimensional defect detection is performed on the weld region image to obtain weld defect detection results; multidimensional defect detection is performed on the fusion region image to obtain fusion defect detection results; and a first welding defect map is constructed based on the molten pool defect detection results, the weld defect detection results, and the fusion defect detection results.

[0024] Preferably, for the segmented molten pool region images in the welding synchronization image, a molten pool defect detection channel is used to perform multi-dimensional defect detection on these molten pool region images. This molten pool defect detection channel is constructed based on multiple molten pool defect detection branches, each responsible for the identification and detection of a defect category. After the molten pool defect detection channel completes the identification and detection of the input molten pool region image, a molten pool defect detection result is generated, which includes information such as the current defect type, defect features, and defect location. Subsequently, for the segmented weld region images and fusion region images, the weld defect detection channel and fusion defect detection channel are also used to perform corresponding multi-dimensional defect detection, thereby obtaining the weld defect detection result for the weld region image and the fusion defect detection result for the fusion region image. Subsequently, based on the obtained molten pool defect detection results, weld defect detection results, and fusion defect detection results, the defect information of the three is integrated to construct the first welding defect map. The first welding defect map is a comprehensive defect description map that contains defect information of all key areas in the welding process. It can provide the spatial distribution of defects and reflect the state of welding quality, providing data support for subsequent defect analysis, welding compensation, and optimization strategies.

[0025] Furthermore, this application provides multi-dimensional defect detection based on the molten pool region image to obtain molten pool defect detection results, including: Based on N molten pool defect categories, defect detection records are backtracked to obtain N molten pool defect detection record libraries, where N is a positive integer greater than 1. Multiple defect detection learners are integrated and fused for training based on each molten pool defect detection record library to obtain N molten pool defect detection branches. Loss minimization distillation is performed on the N molten pool defect detection branches to obtain molten pool defect detection channels. The molten pool region image is input into the molten pool defect detection channels to obtain the molten pool defect detection results.

[0026] Optionally, firstly, based on the predetermined N categories of molten pool defects (such as molten pool too large, too small, molten pool fluctuation, etc.), through multiple welding experiments and on-site data collection, the defect situation under different molten pool conditions is recorded and traced back, and these data are organized and stored in N molten pool defect detection record databases, where N is a positive integer greater than 1, representing the dataset of different molten pool defect types. Subsequently, for each melt pool defect detection record library, multiple defect detection learners (i.e., multiple machine learning models) are integrated and trained. These machine learning models can be different types of algorithms, such as fully connected neural networks, deep neural networks, multilayer perceptrons, support vector machines, decision trees, random forests, etc. They are trained on each melt pool defect category respectively. Each model learns the features extracted from the melt pool area image (such as image texture, shape, temperature distribution, etc.) and learns how to identify various defects in the melt pool. Then, through ensemble fusion methods (such as weighted average, voting mechanism, etc.), the prediction results of multiple learners are fused to improve the robustness and accuracy of the model. In this way, after training with N melt pool defect detection record libraries of N melt pool defect categories, N melt pool defect detection branches can be obtained, and each branch is responsible for the identification and prediction of different melt pool defect types. During training, to improve detection performance and reduce model complexity, loss minimization distillation is performed on the obtained N molten pool defect detection branches. The core idea of ​​distillation is to extract optimal knowledge from multiple trained learners and integrate this knowledge into a smaller, more efficient detection channel. In this process, the integrated N molten pool defect detection branches are selected as teachers to output high-precision defect detection results. A lightweight CNN is then designed as a student to learn the detection logic of the teacher model. Next, the feature maps of the intermediate layers of the teacher model are aligned with the corresponding layers of the student model. The difference in the output probability distribution between the student and teacher models is minimized using KL divergence loss, and cross-entropy loss is combined to optimize the detection accuracy and localization ability of the student model. After distillation, the student model becomes the molten pool defect detection channel, which not only maintains high-precision defect recognition but also improves inference efficiency. Finally, the loss-minimization distilled molten pool defect detection channel is used to process images of the molten pool area in the actual welding process. When the image of the molten pool area is input into the molten pool defect detection channel, the molten pool defect detection channel will use a trained model to detect molten pool defects in the image and output molten pool defect detection results including the detected defect type, defect features, defect location, etc. These detection results can provide reliable data support for real-time monitoring and defect compensation in the welding process, ensuring efficient identification and processing of molten pool defects in the welding process.

[0027] Based on the second welding defect map, the welding control scheme of the laser welding robot is optimized by performing weld pool defect compensation, and a first welding optimization strategy is obtained.

[0028] In one embodiment, after obtaining the second weld defect map, defect information of the molten pool region is extracted from the second weld defect map, and the weld control scheme is made to compensate for molten pool defects based on this defect information. Multiple possible adjustment schemes are formulated, and then these possible schemes are optimized multiple times based on molten pool compensation risk factors (such as molten pool overcompensation risk, molten pool compensation weld interference risk, etc.) to form the final welding optimization first strategy. This welding optimization first strategy can accurately compensate for molten pool defects, thereby optimizing the welding process, improving welding quality and reducing the generation of defects.

[0029] Furthermore, this application provides a first welding optimization strategy for the laser welding robot's welding control scheme based on the second welding defect map, to compensate for weld pool defects and obtain welding optimization. This strategy includes: Based on the second welding defect map, molten pool defects are captured to obtain a molten pool defect map; based on the molten pool defect map, molten pool defect compensation decisions are made for the welding control scheme to obtain a first welding adjustment group; based on the molten pool compensation risk factor, the first welding adjustment group is optimized by molten pool compensation risk constraint to obtain a second welding adjustment group; based on the molten pool compensation risk factor, the second welding adjustment group is optimized by comprehensive risk analysis to generate a third welding adjustment group; based on the third welding adjustment group, cost minimization optimization is performed to generate the first welding optimization strategy.

[0030] Preferably, firstly, defects in the molten pool region during the welding process are captured using a second welding defect map. Specifically, key defect parameters (such as defect location, defect type, and defect size) related to the molten pool region are extracted from the second welding defect map to construct a molten pool defect map. Then, based on the obtained molten pool defect map, all existing defect types are identified. Based on the defect types and adjustment experience, the parameter adjustment direction is determined. Then, welding parameters in the welding control scheme (such as laser power, welding speed, and welding current) are randomly adjusted multiple times in that direction to correct the molten pool state, thereby generating multiple possible welding adjustment schemes. These welding adjustment schemes form the first welding adjustment group. Next, a welding synchronization model is used to simulate the welding adjustment schemes in the first welding adjustment group, and a molten pool compensation risk analysis model is used to evaluate the simulation results, obtaining the risk coefficient for each welding adjustment scheme. By comparing these risk coefficients with corresponding thresholds, welding adjustment schemes that meet the threshold requirements are extracted and added to the second welding adjustment group. Then, based on the molten pool compensation risk factor, a comprehensive risk analysis is performed on the second welding adjustment group using a weighted approach. The comprehensive risk coefficient of each welding adjustment scheme in the second group is calculated. By comparing these comprehensive risk coefficients with corresponding thresholds, welding adjustment schemes that meet the threshold requirements are selected to form the third welding adjustment group. Finally, the welding adjustment schemes in the third group are iterated. For each iterated scheme, resource consumption and time consumption are extracted from the corresponding simulation data. The comprehensive cost of the welding adjustment scheme is obtained by weighting these two factors. By comparing the comprehensive costs of each welding adjustment scheme, the scheme with the lowest comprehensive cost is eliminated and selected as the first welding optimization strategy. It is important to note that the data involved in the weighting calculation is normalized before weighting to ensure they are on the same scale. Furthermore, if either the second or third welding adjustment group is empty, it indicates that no suitable scheme has been found. In this case, multiple welding adjustment schemes are reconstructed, and the above steps are repeated. The final welding optimization strategy can compensate and adjust in time before defects appear in the molten pool, ensuring welding quality and thus effectively improving welding quality and reducing the defect rate.

[0031] Furthermore, this application provides a method for optimizing the weld pool compensation risk constraint of the first weld adjustment group based on the weld pool compensation risk factor to obtain a second weld adjustment group, including: Based on the molten pool compensation risk factors, a molten pool compensation risk analysis model is trained. These factors include molten pool overcompensation risk, molten pool compensation weld interference risk, and molten pool compensation fusion interference risk. Based on the first welding adjustment group, a first welding adjustment scheme is extracted. Based on the welding synchronization image, a welding synchronization model is constructed, and molten pool simulation compensation is performed using the first welding adjustment scheme to obtain first compensation simulation data. The first compensation simulation data is input into the molten pool compensation risk analysis model to obtain a first molten pool compensation risk sequence. Based on the molten pool compensation risk factors, molten pool compensation risk constraints are constructed. If the first molten pool compensation risk sequence satisfies the molten pool compensation risk constraints, the first welding adjustment scheme is added to the second welding adjustment group.

[0032] Optionally, pre-set molten pool compensation risk factors are first obtained. These factors include molten pool overcompensation risk, molten pool compensation weld interference risk, and molten pool compensation fusion interference risk. Overcompensation risk refers to the risk of an excessively large or unstable molten pool due to overcompensation during defect compensation. Weld interference risk refers to the risk that changes in the molten pool state during compensation may interfere with the weld area, thus affecting weld quality. Fusion interference risk refers to the risk that molten pool compensation may cause abnormal changes in the fusion area, leading to poor or uneven fusion. For each molten pool compensation risk factor, corresponding risk factors are labeled in historical welding data, including the overcompensation risk coefficient, weld interference risk coefficient, and fusion interference risk coefficient. The labeled historical data is then input into a multilayer perceptron. Iterative training is performed using the same training steps as described above, such as forward propagation, loss calculation, backpropagation, and parameter optimization, to obtain a risk analysis model corresponding to each molten pool compensation risk factor. The three risk analysis models are then connected in parallel to form the final molten pool compensation risk analysis model. Subsequently, based on the previously obtained first group of welding adjustments, one adjustment scheme was randomly extracted as the first welding adjustment scheme. Then, based on the welding synchronization images and basic welding data (such as material properties, process parameters, etc.), a welding synchronization model was constructed using 3D simulation software and the finite element method to simulate the dynamics of the molten pool. This model can accurately reflect various key parameters in the welding process. Afterwards, the first welding adjustment scheme was loaded into the welding synchronization model. The welding synchronization model would adjust the process parameters according to the first welding adjustment scheme, simulating the process of molten pool compensation, thereby obtaining the first compensation simulation data. This first compensation simulation data records the impact of molten pool compensation on the molten pool, weld, and fusion zone under a specific welding adjustment scheme, such as molten pool temperature gradient, weld reinforcement height, and fusion line offset. Then, the first compensation simulation data is loaded into the molten pool compensation risk analysis model. The molten pool compensation risk analysis model predicts the potential risks in the molten pool compensation process based on the first compensation simulation data according to the three internal parallel models. In this way, the three parallel models will generate three risk coefficients, namely the molten pool overcompensation risk coefficient, the molten pool compensation weld interference risk coefficient, and the molten pool compensation fusion interference risk coefficient. Then, the output layer integrates these three risk coefficients into a set, which is output as the first molten pool compensation risk sequence.Then, based on current business needs, molten pool compensation risk constraints are constructed for each type of molten pool compensation risk factor. These constraints include over-compensation risk constraints, weld interference risk constraints, and fusion interference risk constraints. The over-compensation risk constraint limits the maximum size or variation range of the molten pool to prevent instability caused by over-compensation. The weld interference risk constraint ensures that the molten pool compensation process does not interfere with the quality of the weld area, avoiding issues such as uneven welds or cracks. The fusion interference risk constraint controls the impact of the molten pool compensation process, preventing over-compensation from leading to over-melting or under-fusion of the fusion area. If all risk coefficients in the first molten pool compensation risk sequence are less than or equal to the corresponding constraint thresholds in the molten pool compensation risk constraints, the first welding adjustment scheme is added to the second welding adjustment group. This second welding adjustment group contains a set of optimized welding control schemes that effectively improve molten pool defects while minimizing risks, ensuring the stability and quality of the welding process.

[0033] Furthermore, this application provides a method for comprehensive risk analysis and optimization of the second welding adjustment group based on the molten pool compensation risk factor to generate a third welding adjustment group, including: Based on the weight allocation of the molten pool compensation risk factors, a comprehensive risk analysis model for molten pool compensation is obtained; based on the comprehensive risk analysis model for molten pool compensation, a comprehensive risk analysis is performed on each welding adjustment scheme in the second welding adjustment group to obtain a comprehensive risk set for molten pool compensation; based on the comprehensive risk set for molten pool compensation, the second welding adjustment group is optimized and screened according to the comprehensive risk threshold for molten pool compensation to generate the third welding adjustment group.

[0034] Optionally, for each weld pool compensation risk factor, a weight value is assigned based on business needs and expert decisions. These weights reflect the relative importance of different risk factors in the overall welding adjustment process. Based on these weights, a weighted summation formula is constructed and encapsulated as a comprehensive risk analysis model for weld pool compensation. Subsequently, the over-compensation risk coefficient, weld interference risk coefficient, and fusion interference risk coefficient of each welding adjustment scheme in the second welding adjustment group are input into the comprehensive risk analysis model for weighted calculation to obtain the comprehensive risk coefficient for weld pool compensation of each welding adjustment scheme. By storing these comprehensive risk coefficients, a comprehensive risk set for weld pool compensation is constructed. Then, each comprehensive risk coefficient in the comprehensive risk set is compared with a preset comprehensive risk threshold for weld pool compensation. If the comprehensive risk coefficient of a welding adjustment scheme is less than the threshold, the risk of the welding adjustment scheme is considered to be within an acceptable range, meeting the optimization objective, and should be retained in the next stage. If the comprehensive risk coefficient of molten pool compensation for a certain welding adjustment scheme is greater than or equal to the threshold, the scheme is considered to have excessive risk and should be eliminated. Through screening, welding adjustment schemes that meet the comprehensive risk constraint condition for molten pool compensation are grouped into a third group of welding adjustments. The schemes in this third group are all low-risk, high-efficiency adjustment schemes that can effectively compensate for molten pool defects while avoiding other potential risks and interferences, providing the optimal control strategy for subsequent welding processes and ensuring welding quality and safety.

[0035] Based on the second welding defect map, the welding control scheme is optimized by performing weld defect compensation, and a second welding optimization strategy is obtained.

[0036] In one embodiment, similar to weld pool defect compensation optimization, the welding control scheme is also optimized for weld defect compensation based on the second welding defect map. Specifically, defect information of the weld area is first extracted from the second welding defect map. This information reflects the types of defects that may occur in the weld area during the welding process, such as weld cracks, uneven weld depth, or porosity. Based on this defect information, multiple possible adjustment schemes are formulated, and these adjustment schemes are optimized according to the weld compensation risk factor to generate a second welding optimization strategy. This strategy can accurately compensate for weld defects, thereby optimizing the welding process, improving welding quality, and reducing the generation of weld defects.

[0037] Furthermore, this application provides a second welding optimization strategy for optimizing the welding control scheme by performing weld defect compensation based on the second welding defect map, including: Weld defects are captured based on the second welding defect map to obtain a weld defect map; weld defect compensation decisions are made for the welding control scheme based on the weld defect map to obtain a fourth welding adjustment group; weld compensation risk constraints are optimized on the fourth welding adjustment group based on weld compensation risk factors to obtain a fifth welding adjustment group, whereby weld compensation risk factors include weld overcompensation risk, weld compensation molten pool interference risk, and weld compensation fusion interference risk; comprehensive risk analysis is performed on the fifth welding adjustment group based on the weld compensation risk factors to generate a sixth welding adjustment group; cost minimization optimization is performed on the sixth welding adjustment group to generate the second welding optimization strategy.

[0038] Preferably, firstly, defects in the weld area during the welding process are captured using the second welding defect atlas. Specifically, key defect parameters (such as defect location and type) related to the weld area are extracted from the second welding defect atlas to construct a weld defect atlas. Then, based on the obtained weld defect atlas, the parameter adjustment direction is determined using the same method described above. Next, welding parameters in the welding control scheme (such as laser power, welding speed, and welding current) are randomly adjusted multiple times in that direction to compensate for weld defects, thereby generating multiple possible welding adjustment schemes. These welding adjustment schemes will form the fourth welding adjustment group. Afterward, the welding adjustment schemes in the fourth welding adjustment group are simulated using a welding synchronization model, and the simulation results are evaluated using a weld compensation risk analysis model to obtain the risk coefficient of each welding adjustment scheme. By comparing these risk coefficients with corresponding thresholds, welding adjustment schemes that meet the threshold requirements are extracted and added to the fifth welding adjustment group. Then, based on the weld compensation risk factor, a comprehensive risk analysis is performed on the fifth group of welding adjustments using a weighted approach. The comprehensive risk coefficient of each welding adjustment scheme in the fifth group is calculated. By comparing these comprehensive risk coefficients with corresponding thresholds, welding adjustment schemes that meet the threshold requirements are selected to form the sixth group of welding adjustments. The weld compensation risk factor includes weld overcompensation risk, weld compensation molten pool interference risk, and weld compensation fusion interference risk. Finally, the welding adjustment schemes in the sixth group are traversed. For each traversed scheme, the same cost minimization optimization process is performed to obtain the welding adjustment scheme with the lowest comprehensive cost. This welding adjustment scheme is used as the final welding optimization second strategy. This second welding optimization strategy can reduce costs and resource consumption during the welding process while ensuring welding quality, thereby improving overall production efficiency.

[0039] Based on the second welding defect map, the welding control scheme is optimized by fusion defect compensation to obtain a third welding optimization strategy. The welding optimization and control are then performed by combining the first welding optimization strategy and the second welding optimization strategy.

[0040] In one embodiment, similar to the optimization of molten pool defect compensation and weld defect compensation, a similar approach is used for fusion defect compensation optimization. In this process, defect information in the fusion region is first extracted, and based on this information, multiple possible adjustment schemes are formulated. Subsequently, these adjustment schemes are optimized according to the fusion compensation risk factor, thereby generating a third welding optimization strategy. Then, this third welding optimization strategy is combined with the previously obtained first welding optimization strategy (adjustment scheme for molten pool defects) and second welding optimization strategy (adjustment scheme for weld defects), applying the parameters involved in the three strategies to the welding process to comprehensively optimize all critical areas (molten pool, weld, and fusion region) in the laser welding process. This ensures that defects in each area are effectively compensated, thereby improving welding quality and reducing defect occurrence.

[0041] In summary, the embodiments of this application have at least the following technical effects: This application embodiment first performs real-time monitoring of the laser welding robot using a vision sensor to obtain synchronous welding images, including images of the molten pool region, the weld seam region, and the fusion region. Then, it performs region-adaptive defect detection on the synchronous welding images to obtain a first welding defect map, and performs multi-scale evolution prediction on the first welding defect map to construct a second welding defect map. Next, based on the second welding defect map, it optimizes the welding control scheme of the laser welding robot by compensating for molten pool defects to obtain a first welding optimization strategy. Then, based on the second welding defect map, it optimizes the welding control scheme by compensating for weld seam defects to obtain a second welding optimization strategy. Finally, based on the second welding defect map, it optimizes the welding control scheme by compensating for fusion defects to obtain a third welding optimization strategy. The welding optimization is then combined with the first and second welding optimization strategies for overall welding optimization and control. These technological effects collectively solve the technical problems of traditional laser welding defect detection methods, which rely on single-scale feature analysis and lack multi-angle collaborative detection capabilities, resulting in unstable welding quality and inability to correct defects in a timely manner. They achieve the technical effects of real-time identification and multi-scale dynamic prediction of welding defects through visual guidance, intelligent optimization of welding control schemes, improved welding quality, reduced defects, and optimized welding process precision.

[0042] Example 2, based on the same inventive concept as the vision-guided laser welding robot defect detection method in the foregoing examples, such as... Figure 2As shown, this application provides a vision-guided laser welding robot defect detection system, the system comprising: a real-time monitoring module 11: performing real-time monitoring of the laser welding robot using a vision sensor to obtain welding synchronous images, the welding synchronous images including molten pool area images, weld seam area images, and fusion area images; a defect evolution module 12: performing regional adaptive defect detection on the welding synchronous images to obtain a first welding defect map, and performing multi-scale evolution prediction on the first welding defect map to construct a second welding defect map; a molten pool defect compensation module 13: optimizing the welding control scheme of the laser welding robot based on the second welding defect map to obtain a first welding optimization strategy; a weld seam defect compensation module 14: optimizing the welding control scheme based on the second welding defect map to obtain a second welding optimization strategy; and a welding optimization management module 15: optimizing the welding control scheme based on the second welding defect map to obtain a third welding optimization strategy, and combining the first and second welding optimization strategies for welding optimization management.

[0043] Furthermore, the real-time monitoring module 11 is also used to perform the following methods: A welding monitoring image is obtained based on the visual sensor; the welding monitoring image is enhanced to obtain a welding enhanced image; regional feature detection is performed on the welding enhanced image to obtain a multi-dimensional regional feature vector; the welding enhanced image is adaptively segmented based on the multi-dimensional regional feature vector to generate the welding synchronization image.

[0044] Furthermore, the real-time monitoring module 11 is also used to perform the following methods: The multi-dimensional region feature vectors include the molten pool region feature vector, the weld region feature vector, and the fusion region feature vector.

[0045] Furthermore, the defect evolution module 12 is also used to perform the following method: Multidimensional defect detection is performed on the molten pool region image to obtain molten pool defect detection results; multidimensional defect detection is performed on the weld region image to obtain weld defect detection results; multidimensional defect detection is performed on the fusion region image to obtain fusion defect detection results; and a first welding defect map is constructed based on the molten pool defect detection results, the weld defect detection results, and the fusion defect detection results.

[0046] Furthermore, the defect evolution module 12 is also used to perform the following method: Based on N molten pool defect categories, defect detection records are backtracked to obtain N molten pool defect detection record libraries, where N is a positive integer greater than 1. Multiple defect detection learners are integrated and fused for training based on each molten pool defect detection record library to obtain N molten pool defect detection branches. Loss minimization distillation is performed on the N molten pool defect detection branches to obtain molten pool defect detection channels. The molten pool region image is input into the molten pool defect detection channels to obtain the molten pool defect detection results.

[0047] Furthermore, the molten pool defect compensation module 13 is also used to perform the following method: Based on the second welding defect map, molten pool defects are captured to obtain a molten pool defect map; based on the molten pool defect map, molten pool defect compensation decisions are made for the welding control scheme to obtain a first welding adjustment group; based on the molten pool compensation risk factor, the first welding adjustment group is optimized by molten pool compensation risk constraint to obtain a second welding adjustment group; based on the molten pool compensation risk factor, the second welding adjustment group is optimized by comprehensive risk analysis to generate a third welding adjustment group; based on the third welding adjustment group, cost minimization optimization is performed to generate the first welding optimization strategy.

[0048] Furthermore, the molten pool defect compensation module 13 is also used to perform the following method: Based on the molten pool compensation risk factors, a molten pool compensation risk analysis model is trained. These factors include molten pool overcompensation risk, molten pool compensation weld interference risk, and molten pool compensation fusion interference risk. Based on the first welding adjustment group, a first welding adjustment scheme is extracted. Based on the welding synchronization image, a welding synchronization model is constructed, and molten pool simulation compensation is performed using the first welding adjustment scheme to obtain first compensation simulation data. The first compensation simulation data is input into the molten pool compensation risk analysis model to obtain a first molten pool compensation risk sequence. Based on the molten pool compensation risk factors, molten pool compensation risk constraints are constructed. If the first molten pool compensation risk sequence satisfies the molten pool compensation risk constraints, the first welding adjustment scheme is added to the second welding adjustment group.

[0049] Furthermore, the molten pool defect compensation module 13 is also used to perform the following method: Based on the weight allocation of the molten pool compensation risk factors, a comprehensive risk analysis model for molten pool compensation is obtained; based on the comprehensive risk analysis model for molten pool compensation, a comprehensive risk analysis is performed on each welding adjustment scheme in the second welding adjustment group to obtain a comprehensive risk set for molten pool compensation; based on the comprehensive risk set for molten pool compensation, the second welding adjustment group is optimized and screened according to the comprehensive risk threshold for molten pool compensation to generate the third welding adjustment group.

[0050] Furthermore, the weld defect compensation module 14 is also used to perform the following method: Weld defects are captured based on the second welding defect map to obtain a weld defect map; weld defect compensation decisions are made for the welding control scheme based on the weld defect map to obtain a fourth welding adjustment group; weld compensation risk constraints are optimized on the fourth welding adjustment group based on weld compensation risk factors to obtain a fifth welding adjustment group, whereby weld compensation risk factors include weld overcompensation risk, weld compensation molten pool interference risk, and weld compensation fusion interference risk; comprehensive risk analysis is performed on the fifth welding adjustment group based on the weld compensation risk factors to generate a sixth welding adjustment group; cost minimization optimization is performed on the sixth welding adjustment group to generate the second welding optimization strategy.

[0051] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments of this specification. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing are possible or may be advantageous.

[0052] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0053] This specification and accompanying drawings are merely illustrative examples of this application and are intended to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from its scope. Therefore, if such modifications and modifications fall within the scope of this application and its equivalents, this application intends to include such modifications and modifications.

Claims

1. A vision-guided defect detection method for laser welding robots, characterized in that, The method includes: The laser welding robot is monitored in real time using a vision sensor to obtain synchronous welding images, which include images of the molten pool area, the weld seam area, and the fusion area. Regional adaptive defect detection is performed on the welding synchronous image to obtain a first welding defect map, and multi-scale evolution prediction is performed on the first welding defect map to construct a second welding defect map. Based on the second welding defect map, the welding control scheme of the laser welding robot is optimized by performing weld pool defect compensation, and a first welding optimization strategy is obtained. Based on the second welding defect map, the welding control scheme is optimized by weld defect compensation to obtain a second welding optimization strategy. Based on the second welding defect map, the welding control scheme is optimized by fusion defect compensation to obtain a third welding optimization strategy. The welding optimization and control are then performed by combining the first welding optimization strategy and the second welding optimization strategy.

2. The vision-guided laser welding robot defect detection method as described in claim 1, characterized in that, Regional adaptive defect detection is performed on the synchronous welding image to obtain a first weld defect map, including: Multidimensional defect detection is performed based on the image of the molten pool region to obtain the molten pool defect detection results; Multidimensional defect detection is performed on the weld area image to obtain weld defect detection results; Multidimensional defect detection is performed based on the image of the fusion region to obtain the fusion defect detection result; Based on the detection results of the molten pool defects, the detection results of the weld defects, and the detection results of the fusion defects, a first atlas of welding defects is constructed.

3. The vision-guided laser welding robot defect detection method as described in claim 2, characterized in that, Multidimensional defect detection is performed based on the image of the molten pool region to obtain molten pool defect detection results, including: Based on N molten pool defect categories, defect detection records are backtracked to obtain N molten pool defect detection record libraries, where N is a positive integer greater than 1; Based on each molten pool defect detection record library, multiple defect detection learners are integrated and fused for training to obtain N molten pool defect detection branches. Loss-minimum distillation is performed on the N molten pool defect detection branches to obtain the molten pool defect detection channel; The image of the molten pool region is input into the molten pool defect detection channel to obtain the molten pool defect detection result.

4. The defect detection method for a vision-guided laser welding robot as described in claim 1, characterized in that, Based on the second welding defect map, the welding control scheme of the laser welding robot is optimized by performing weld pool defect compensation, resulting in a first welding optimization strategy, including: Based on the second welding defect map, molten pool defects are captured to obtain a molten pool defect map. Based on the molten pool defect map, a molten pool defect compensation decision is made for the welding control scheme to obtain the first group of welding adjustments. Based on the molten pool compensation risk factor, the first group of welding adjustments is optimized by performing molten pool compensation risk constraint search to obtain the second group of welding adjustments. Based on the aforementioned molten pool compensation risk factor, a comprehensive risk analysis and optimization is performed on the second group of welding adjustments to generate a third group of welding adjustments. The first welding optimization strategy is generated by minimizing the cost based on the third welding adjustment group.

5. The vision-guided laser welding robot defect detection method as described in claim 4, characterized in that, Based on the molten pool compensation risk factor, the first group of welding adjustments is optimized using molten pool compensation risk constraints to obtain the second group of welding adjustments, including: Based on the molten pool compensation risk factors, a molten pool compensation risk analysis model is trained. The molten pool compensation risk factors include molten pool overcompensation risk, molten pool compensation weld interference risk, and molten pool compensation fusion interference risk. Based on the first group of welding adjustments, extract the first welding adjustment scheme; Based on the welding synchronization image, a welding synchronization model is constructed, and the molten pool is simulated and compensated in combination with the first welding adjustment scheme to obtain the first compensation simulation data. The first compensation simulation data is input into the molten pool compensation risk analysis model to obtain the first molten pool compensation risk sequence. Based on the aforementioned molten pool compensation risk factors, construct molten pool compensation risk constraints; If the first molten pool compensation risk sequence satisfies the molten pool compensation risk constraint, the first welding adjustment scheme is added to the second welding adjustment group.

6. The vision-guided laser welding robot defect detection method as described in claim 4, characterized in that, Based on the aforementioned molten pool compensation risk factor, a comprehensive risk analysis and optimization is performed on the second group of welding adjustments to generate a third group of welding adjustments, including: Based on the weight allocation of the aforementioned molten pool compensation risk factors, a comprehensive risk analysis model for molten pool compensation is obtained. Based on the comprehensive risk analysis model for molten pool compensation, a comprehensive risk analysis is performed on each welding adjustment scheme in the second group of welding adjustments to obtain a comprehensive risk set for molten pool compensation. Based on the comprehensive risk set of the molten pool compensation, the second group of welding adjustment is optimized and screened according to the comprehensive risk threshold of the molten pool compensation, and the third group of welding adjustment is generated.

7. The vision-guided laser welding robot defect detection method as described in claim 1, characterized in that, Based on the second welding defect map, the welding control scheme is optimized for weld defect compensation to obtain a second welding optimization strategy, including: Weld defects are captured based on the second welding defect map to obtain a weld defect map. Based on the weld defect map, a weld defect compensation decision is made for the welding control scheme to obtain the fourth group of welding adjustment. Based on the weld compensation risk factors, the fourth group of welding adjustments is optimized by performing weld compensation risk constraint search to obtain the fifth group of welding adjustments. The weld compensation risk factors include weld overcompensation risk, weld compensation molten pool interference risk, and weld compensation fusion interference risk. Based on the weld seam compensation risk factors, a comprehensive risk analysis and optimization of the fifth group of welding adjustments is performed to generate the sixth group of welding adjustments. The second welding optimization strategy is generated by minimizing the cost based on the sixth welding adjustment group.

8. The vision-guided laser welding robot defect detection method as described in claim 1, characterized in that, The laser welding robot is monitored in real time using a vision sensor to obtain synchronous welding images, including: Welding monitoring images are obtained based on the vision sensor; The welding monitoring image is enhanced to obtain an enhanced welding image; Based on the weld enhancement image, perform region feature detection to obtain a multivariate region feature vector; The welding enhancement image is adaptively segmented based on the multi-region feature vector to generate the welding synchronization image.

9. The vision-guided laser welding robot defect detection method as described in claim 8, characterized in that, The multi-dimensional region feature vectors include the molten pool region feature vector, the weld region feature vector, and the fusion region feature vector.

10. A vision-guided laser welding robot defect detection system, characterized in that, The system is used to perform the vision-guided laser welding robot defect detection method according to any one of claims 1-9, the system comprising: Real-time monitoring module: The laser welding robot is monitored in real time by a vision sensor to obtain synchronous welding images, which include images of the molten pool area, the weld seam area, and the fusion area. Defect evolution module: Performs region-adaptive defect detection on the welding synchronous image to obtain a first welding defect map, and performs multi-scale evolution prediction on the first welding defect map to construct a second welding defect map; Molten pool defect compensation module: Based on the second welding defect map, the welding control scheme of the laser welding robot is optimized for molten pool defect compensation to obtain the first welding optimization strategy; Weld defect compensation module: Based on the second welding defect map, the welding control scheme is optimized for weld defect compensation to obtain a second welding optimization strategy; Welding optimization and control module: Based on the second welding defect map, the welding control scheme is optimized by fusion defect compensation to obtain a third welding optimization strategy, and welding optimization and control is performed by combining the first welding optimization strategy and the second welding optimization strategy.

Images