Laser welding control method and system based on machine vision

Abstract

The invention discloses a laser welding control method and system based on machine vision, and relates to the technical field of laser welding, the system collects operation data and welding image data of laser equipment in real time by integrating a sensor group and an industrial camera, and preprocesses the operation data and the welding image data to ensure the accuracy and the reliability of the data; the deep learning module identifies related features in the image data by using a convolutional neural network CNN and a recurrent neural network RNN, and performs feature extraction; the data analysis module processes the extracted features to obtain a light spot shape distortion index SSD, a pulse characteristic index MCT and a fusion depth index WPD; the comprehensive analysis module calculates the indexes to obtain a comprehensive welding quality index CWQI; and the welding evaluation control module evaluates, regulates and controls the welding quality by utilizing a historical standard welding quality fusion depth interval and a preset quality threshold value, so that the welding quality is controlled by collecting welding piece data and collecting welding laser in real time.

Description

Technical Field

[0001] The present invention relates to the technical field of laser welding, and in particular to a laser welding control method and system based on machine vision. Background Art

[0002] Traditional laser welding quality control methods have many shortcomings; for example, parameter adjustments during the welding process mostly rely on manual experience and lack the support of real-time data, resulting in unstable welding quality; for data collection of welded parts and real-time monitoring of welding lasers, traditional methods usually use a single sensor or manual detection, which is not easy to comprehensively and accurately obtain the various key parameters of the welding process; this situation makes it difficult to effectively control welding quality and prone to welding defects such as cold welds and burn-throughs; in addition, traditional methods have weak real-time feedback and control capabilities during the welding process, and it is not easy to adjust welding parameters in time according to actual welding conditions, resulting in low welding efficiency and difficulty in ensuring quality.

[0003] The above status quo and shortcomings are mainly due to the fact that traditional welding control methods lack real-time data support and comprehensive monitoring means for the welding process; traditional methods usually rely on manual experience to adjust welding parameters, and are not easy to achieve real-time feedback and intelligent regulation; this not only increases the complexity and uncertainty of welding operations, but may also lead to fluctuations and instability in welding quality; when problems occur during the welding process, it is often difficult to discover and take effective measures in time, resulting in welding defects; in this case, it will not only affect the welding quality, cause rework and waste of resources, but may also have a negative impact on production efficiency and product consistency; therefore, there is an urgent need for a laser welding control system that can perform real-time monitoring and intelligent regulation to solve the shortcomings of traditional methods and improve welding quality and efficiency. Summary of the Invention

[0004] In view of the shortcomings of the existing technology, the present invention provides a laser welding control method and system based on machine vision, which solves the problems mentioned in the background technology.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solutions: a laser welding control method and system based on machine vision, including a welding data acquisition module, a deep learning module, a data analysis module, a comprehensive analysis module and a welding evaluation control module;

[0006] The welding data acquisition module is used to install an integrated sensor group inside the laser equipment to collect the operating data of the laser equipment in real time, and set an industrial camera to capture the welding image data of the laser equipment in real time during the welding process, and pre-process the collected operating data and welding image data;

[0007] The deep learning module is used to construct a deep learning model based on the convolutional neural network (CNN) and the recurrent neural network (RNN) as the basic framework of the deep learning model, identify relevant features in the collected image data, and perform feature extraction;

[0008] The data analysis module is used to process the feature vectors output by the deep learning module, perform multi-level data analysis and correlation analysis, perform dimensionless processing and comprehensive calculation on the extracted data set, and respectively obtain the spot shape distortion index SSD, pulse characteristic index MCT and penetration depth index WPD;

[0009] The comprehensive analysis module is used to perform correlation calculation on the acquired spot shape distortion index SSD, pulse characteristic index MCT and penetration index WPD to obtain a comprehensive welding quality index CWQI;

[0010] The welding evaluation control module is used to perform a preliminary evaluation based on the historical standard welding quality penetration range of laser welding, the preset first penetration threshold A and the second penetration threshold B, and the obtained penetration index WPD, perform an initial adjustment on the equipment according to the evaluation results, and start a second evaluation mechanism. The second evaluation mechanism is used to perform a secondary comparative evaluation of the preset first welding quality threshold M and the second welding quality threshold N with the obtained comprehensive welding quality index CWQI, and generate control information according to the evaluation results.

[0011] Preferably, the welding data acquisition module includes an image acquisition unit, a sensor acquisition unit and a data preprocessing unit;

[0012] The image acquisition unit is used to set an industrial camera with a frame rate of more than 5000 above the welding area. At the same time, the lens of the industrial camera is set to a 105mm fixed-focus lens. During the welding process of the laser welding equipment, the welding area is photographed 36 times per second, and the spot image and weld image of the welding area are collected in real time;

[0013] The sensor acquisition unit is used to install an integrated sensor group inside the laser equipment to collect the operating data of the laser equipment in real time and classify and summarize it to generate an operating data set. The integrated sensor group includes a voltage sensor, a current sensor, a timing sensor, a photoelectric sensor, an encoding sensor and a photodiode sensor. The operating data set includes voltage V, current I, welding time sj, pulse frequency f, welding speed hj, pulse energy E and pulse width w;

[0014] The voltage V, current I and welding time sj are obtained through the voltage sensor, current sensor and timing sensor;

[0015] Obtain the pulse frequency f through the photoelectric sensor;

[0016] The welding speed hj is obtained through the encoding sensor;

[0017] The pulse energy E and pulse width w are obtained by the photodiode sensor;

[0018] The data preprocessing unit includes a parameter preprocessing unit and a picture preprocessing unit;

[0019] The parameter preprocessing unit is used to perform data cleaning, filtering, denoising and dimensionless processing on the running data set;

[0020] The image preprocessing unit is used to enhance the weld and spot boundaries in the video, grayscale the video, and Gaussian blur the collected image data by using morphological operations and image processing software OpenCV.

[0021] Preferably, the deep learning module includes a feature recognition unit and a feature extraction unit;

[0022] The feature recognition unit includes a data annotation unit and a deep learning unit;

[0023] The data annotation unit is used to import a large amount of collected historical weld image data and historical spot image data into the CVAT annotation tool, and annotate the images with welds and light spots through the annotation function of the CVAT annotation tool;

[0024] By marking the weld shape features of the weld frame image, the weld shape features include weld width features and weld penetration features;

[0025] By marking the features of the light spot frame image, the light spot features include light spot shape features and light spot radius features;

[0026] The deep learning unit includes an image recognition unit and a model training unit;

[0027] The image recognition unit integrates a convolutional neural network (CNN) and a recurrent neural network (RNN) as a model framework to construct a deep learning model. The deep learning model extracts local features from the image through the convolution layer and pooling layer of the convolutional neural network (CNN), classifies the weld shape and the spatial distribution of the light spot using a fully connected layer, identifies the weld shape and the spatial distribution characteristics of the light spot, and uses the Canny edge detection algorithm to mark the boundary information of the weld and the light spot. At the same time, the recurrent neural network (RNN) captures the temporal information and dependency of the data through a recursive structure and a long short-term memory unit, and identifies the weld morphology changes and the light spot pulse characteristics.

[0028] The model training unit is used to divide the annotated historical weld image data and spot image data into a training set and a validation set, import the training set into the constructed deep learning model, iteratively train the deep learning model, and then verify the accuracy of the deep learning model in recognizing weld image features and spot image features through the validation set, and iteratively learn and optimize the deep learning model;

[0029] The feature extraction unit is used to import the weld image set and spot image set collected in real time into the deep learning model to perform feature extraction, divide the image into several regions according to pixel grayscale and gradient through the watershed algorithm, separate the weld area and the spot area from the image, measure the size of the segmented weld area, measure the size of the segmented spot area to obtain the size of the spot area, and obtain an image data set by calculating the size of the weld area and the size of the obtained spot area. The image data set includes the weld width hk, the penetration depth D, the spot shape S and the spot radius R.

[0030] Preferably, the data analysis module includes a penetration analysis unit, a pulse analysis unit and a spot analysis unit;

[0031] The penetration analysis unit includes a heat input calculation unit and a penetration calculation unit;

[0032] The heat input calculation unit is used to extract the current I, voltage V and welding time sj based on the acquired operation data set and image data set, and perform summary calculation to obtain the heat input index Q;

[0033] The heat input index Q is calculated by the following formula:

[0034]

[0035] Where k is the correction constant, represents the index;

[0036] The penetration calculation unit is used to extract the welding speed hj, weld width hk and penetration D based on the acquired operation data set and image data set, and the extracted data and the heat input index Q obtained by the heat input calculation unit are dimensionlessly processed and then summarized and calculated to obtain the penetration index WPD;

[0037] The penetration index WPD is obtained by the following formula:

[0038]

[0039] Where, Represents the adjustment factor used to balance the effects of speed, heat input and weld size on penetration.

[0040] Preferably, the pulse analysis unit is used to extract the pulse frequency f, pulse width w and pulse energy E based on the acquired operating data set, and perform summary calculation to obtain the pulse characteristic index MCT;

[0041] The pulse characteristic index MCT is obtained by the following formula:

[0042]

[0043] Where, Indicates the pulse energy change rate, the rate of change of pulse energy per unit time, Indicates the change in pulse energy, represents the corresponding time variation, represents the weight value, Indicates the maximum value of the pulse frequency, Indicates the maximum value of the pulse width, Indicates the maximum value of the pulse energy.

[0044] Preferably, the light spot analysis unit is used to extract the light spot shape S and the light spot radius R according to the extracted image data set, and perform summary calculation to obtain the light spot shape distortion index SSD;

[0045] The spot shape distortion index SSD is obtained by the following formula:

[0046]

[0047] Where Si represents the spot shape parameter of the i-th sampling point, represents the average spot shape parameter, n is the number of sampling points, Indicates the standard deviation of the spot shape parameters, the change of the spot shape between multiple measurements or different sampling points, represents the spot radius of the i-th sampling point, represents the average value of the spot radius, Indicates the standard deviation of the spot radius, the change of the spot radius between multiple measurements or different sampling points, Indicates the weight value.

[0048] Preferably, the comprehensive analysis module is used to perform dimensionless processing on the acquired spot shape distortion index SSD, pulse characteristic index MCT and penetration index WPD, and then perform summary calculation to obtain a comprehensive welding quality index CWQI;

[0049] The comprehensive welding quality index CWQI is calculated by the following formula:

[0050]

[0051] Where, represents the inverse sine function, log represents the logarithmic function, and A represents the normalization factor.

[0052] Preferably, the welding evaluation control module includes a first evaluation unit and a second evaluation unit;

[0053] The first evaluation unit performs a preliminary evaluation by presetting a first penetration threshold A and a second penetration threshold B based on the historical standard welding quality penetration range of laser welding, and then combines the obtained penetration index WPD to analyze the penetration quality information of the current welding process. The specific evaluation scheme is as follows;

[0054] When the first penetration threshold A is less than the penetration index WPD and less than the second penetration threshold B, it indicates that the welding process meets the quality standards;

[0055] When the penetration index WPD is less than or equal to the first penetration threshold A, the welding is unqualified. In this case, the heat input is increased by 20%, the welding speed is reduced by 20%, and the adjusted parameters are iteratively calculated and evaluated by the penetration analysis unit and the first evaluation unit until the welding is qualified.

[0056] When the penetration index WPD ≥ the second penetration threshold B, it indicates that the welding is unqualified. In this case, the heat input is reduced by 20%, the welding speed is increased by 20%, and the second evaluation result is obtained. The adjusted parameters are iteratively calculated and evaluated by the penetration analysis unit and the first evaluation unit until the welding is qualified.

[0057] At this point, a second evaluation is performed through the second evaluation unit to comprehensively analyze the causes of welding quality.

[0058] Preferably, the second evaluation unit performs a secondary comparative evaluation by presetting a first welding quality threshold M and a second welding quality N based on the average value of historical welding quality, and then compares and evaluates the obtained comprehensive welding quality index CWQI to analyze the welding quality of the current welding process. The specific evaluation scheme is as follows;

[0059] When the preset first welding quality threshold M is less than the comprehensive welding quality index CWQI and less than the preset second welding quality threshold N, it indicates that the welding quality is qualified. At this time, the current welding parameters and process are maintained.

[0060] When the comprehensive welding quality index CWQI is less than or equal to the preset first welding quality threshold M, it means that the welding quality is unqualified. At this time, the pulse energy is increased by 30%, the pulse width is reduced by 20%, the pulse frequency is increased by 30%, and the spot radius is reduced by 20%.

[0061] When the comprehensive welding quality index CWQI ≥ the preset second welding quality threshold N, it means that the welding quality is unqualified. At this time, the pulse energy is reduced by 20%, the pulse width is increased by 30%, the pulse frequency is reduced by 20%, and the spot radius is increased by 30%.

[0062] A laser welding control method based on machine vision comprises the following steps:

[0063] S1. Install an integrated sensor group inside the laser equipment to collect the operating data of the laser equipment in real time, set an industrial camera to capture the welding image data of the laser equipment in real time during the welding process, and pre-process the collected operating data and welding image data;

[0064] S2. Based on the convolutional neural network (CNN) and recurrent neural network (RNN) as the basic framework of the deep learning model, a deep learning model is constructed to identify relevant features in the collected image data and perform feature extraction;

[0065] S3. Process the output feature vectors, perform multi-level data parsing and correlation analysis, perform dimensionless processing and comprehensive calculation on the extracted data set, and obtain the spot shape distortion index SSD, pulse characteristic index MCT and penetration index WPD respectively;

[0066] S4, performing correlation calculation on the acquired spot shape distortion index SSD, pulse characteristic index MCT and penetration index WPD to obtain a comprehensive welding quality index CWQI;

[0067] S5. Based on the historical standard welding quality penetration range of laser welding, a first penetration threshold A and a second penetration threshold B are preset and preliminary evaluated with the obtained penetration index WPD. The equipment is initially adjusted according to the evaluation results, and a second evaluation mechanism is started. The second evaluation mechanism is used to preset the first welding quality threshold M and the second welding quality threshold N and perform a secondary comparative evaluation with the obtained comprehensive welding quality index CWQI, and generate control information according to the evaluation results.

[0068] The present invention provides a laser welding control method and system based on machine vision, which has the following beneficial effects:

[0069] (1) The system collects welding operation data and image data in real time by installing an integrated sensor group inside the laser equipment, and uses convolutional neural network (CNN) and recurrent neural network (RNN) to extract and identify features of deep learning models, further realizing efficient monitoring and evaluation of the welding process. This system can not only collect and preprocess data in real time, but also accurately extract the features of welds and light spots through deep learning models, significantly improving the monitoring accuracy of welding quality.

[0070] (2) The system uses multi-level data analysis and dimensionless processing to comprehensively calculate the extracted feature vectors and obtain the spot shape distortion index SSD, pulse characteristic index MCT and penetration index WPD; these indices can reflect the key parameters in the welding process, and the comprehensive welding quality index CWQI is calculated through the comprehensive analysis module, thereby achieving a comprehensive evaluation of the welding quality; by setting the preset penetration threshold and welding quality threshold, the system can perform preliminary and secondary evaluations of the welding quality, ensure the high quality stability of the welding process, and achieve dynamic optimization of the welding process by automatically adjusting the parameters.

[0071] (3) By introducing machine vision and deep learning technologies, the system has greatly improved the data collection, processing and analysis capabilities of the welding process. Through multi-sensor data fusion and image data analysis, it provides a more comprehensive and accurate welding quality assessment method. The introduction of the comprehensive welding quality index (CWQI) makes welding quality assessment more scientific and accurate, provides reliable technical support for the intelligent control of laser welding, and significantly improves welding quality and production efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 This is a flow chart of a laser welding control system based on machine vision according to the present invention;

[0073] Figure 2 This is a schematic diagram of the steps of a laser welding control method based on machine vision of the present invention; DETAILED DESCRIPTION

[0074] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0075] Example 1

[0076] See also Figure 1 The present invention provides a laser welding control method and system based on machine vision. To achieve the above objectives, the present invention is implemented through the following technical solutions: including a welding data acquisition module, a deep learning module, a data analysis module, a comprehensive analysis module and a welding evaluation control module;

[0077] The welding data acquisition module is used to install an integrated sensor group inside the laser equipment to collect the operating data of the laser equipment in real time, and set an industrial camera to capture the welding image data of the laser equipment in real time during the welding process, and pre-process the collected operating data and welding image data;

[0078] The deep learning module is used to construct a deep learning model based on the convolutional neural network (CNN) and the recurrent neural network (RNN) as the basic framework of the deep learning model, identify relevant features in the collected image data, and perform feature extraction;

[0079] The data analysis module is used to process the feature vectors output by the deep learning module, perform multi-level data analysis and correlation analysis, perform dimensionless processing and comprehensive calculation on the extracted data set, and respectively obtain the spot shape distortion index SSD, pulse characteristic index MCT and penetration depth index WPD;

[0080] The comprehensive analysis module is used to perform correlation calculation on the acquired spot shape distortion index SSD, pulse characteristic index MCT and penetration index WPD to obtain a comprehensive welding quality index CWQI;

[0081] The welding evaluation control module is used to perform a preliminary evaluation based on the historical standard welding quality penetration range of laser welding, the preset first penetration threshold A and the second penetration threshold B, and the obtained penetration index WPD, perform an initial adjustment on the equipment according to the evaluation results, and start a second evaluation mechanism. The second evaluation mechanism is used to perform a secondary comparative evaluation of the preset first welding quality threshold M and the second welding quality threshold N with the obtained comprehensive welding quality index CWQI, and generate control information according to the evaluation results.

[0082] In this embodiment, by integrating the welding data acquisition module, deep learning module, data analysis module, comprehensive analysis module and welding evaluation control module, the laser welding system can achieve comprehensive and efficient quality control; the welding data acquisition module uses an integrated sensor group and an industrial camera to capture the operating data and welding image data of the laser equipment in real time to ensure the integrity and timeliness of the information; the deep learning module is based on the convolutional neural network CNN and recurrent neural network RNN deep learning model framework to accurately identify and extract relevant features in the welding image, thereby improving the intelligent level of data processing; the data analysis module performs multi-level parsing and correlation analysis on the extracted feature vectors, and uses statistical and machine learning models to perform infinite Through outline processing and comprehensive calculation, the spot shape distortion index SSD, pulse characteristic index MCT and penetration index WPD are obtained, providing detailed quality assessment indicators; the comprehensive analysis module correlates the above indices to generate the comprehensive welding quality index CWQI, which provides a comprehensive quantitative basis for quality assessment; finally, the welding assessment control module uses historical standard welding quality data and preset thresholds to perform preliminary and secondary assessments of the penetration index WPD and comprehensive welding quality index CWQI, and generates control information based on the assessment results to ensure precise adjustment and optimization of welding equipment; overall, this system greatly improves the monitoring and control level of laser welding quality, and enhances production efficiency and product reliability.

[0083] Example 2

[0084] This embodiment is explained in Example 1, please refer to Figure 1 ,Specifically: the welding data acquisition module includes an image acquisition unit, a sensor acquisition unit and a data preprocessing unit;

[0085] The image acquisition unit is used to set an industrial camera with a frame rate of more than 5000 above the welding area. At the same time, the lens of the industrial camera is set to a 105mm fixed-focus lens. During the welding process of the laser welding equipment, the welding area is photographed 36 times per second, and the spot image and weld image of the welding area are collected in real time;

[0086] The sensor acquisition unit is used to install an integrated sensor group inside the laser equipment to collect the operating data of the laser equipment in real time and classify and summarize it to generate an operating data set. The integrated sensor group includes a voltage sensor, a current sensor, a timing sensor, a photoelectric sensor, an encoding sensor and a photodiode sensor. The operating data set includes voltage V, current I, welding time sj, pulse frequency f, welding speed hj, pulse energy E and pulse width w;

[0087] The parameter preprocessing unit is used to perform data cleaning, filtering, denoising and dimensionless processing on the running data set;

[0088] The image preprocessing unit is used to enhance the weld and spot boundaries in the video, grayscale the video, and Gaussian blur the collected image data by using morphological operations and image processing software OpenCV.

[0089] In this embodiment, through the comprehensive application of the image acquisition unit, sensor acquisition unit and data preprocessing unit of the welding data acquisition module, the laser welding system can achieve efficient real-time data acquisition and preprocessing; the image acquisition unit captures 36 high-quality images of the welding area per second through a high-frame rate industrial camera and a fixed-focus lens, ensuring the clarity and details of the light spot and weld images; the sensor acquisition unit uses a variety of sensors to monitor the key operating parameters of the laser equipment in real time, including voltage, current, welding time, pulse frequency, welding speed, pulse energy and pulse width, to form a comprehensive operating data set; the data preprocessing unit processes the operating data through dimensionless processing, and uses morphological operations and OpenCV software to enhance, grayscale and Gaussian blur the image data, thereby improving the accuracy and applicability of the data; the data acquisition module integrates data acquisition and preprocessing processes, which not only improves the real-time monitoring and analysis capabilities of the welding process, but also provides high-quality basic data for subsequent feature extraction and data analysis, thereby significantly improving the accuracy and efficiency of laser welding quality control.

[0090] Example 3

[0091] This embodiment is explained in Example 1, please refer to Figure 1 ,Specifically: the deep learning module includes a feature recognition unit and a feature extraction unit;

[0092] The feature recognition unit includes a data annotation unit and a deep learning unit;

[0093] The data annotation unit is used to import a large amount of collected historical weld image data and historical spot image data into the CVAT annotation tool, and annotate the images with welds and light spots through the annotation function of the CVAT annotation tool;

[0094] By marking the weld shape features of the weld frame image, the weld shape features include weld width features and weld penetration features;

[0095] By marking the features of the light spot frame image, the light spot features include light spot shape features and light spot radius features;

[0096] The deep learning unit includes an image recognition unit and a model training unit;

[0097] The image recognition unit integrates a convolutional neural network (CNN) and a recurrent neural network (RNN) as a model framework to construct a deep learning model. The deep learning model extracts local features from the image through the convolution layer and pooling layer of the convolutional neural network (CNN), classifies the weld shape and the spatial distribution of the light spot using a fully connected layer, identifies the weld shape and the spatial distribution characteristics of the light spot, and uses the Canny edge detection algorithm to mark the boundary information of the weld and the light spot. At the same time, the recurrent neural network (RNN) captures the temporal information and dependency of the data through a recursive structure and a long short-term memory unit, and identifies the weld morphology changes and the light spot pulse characteristics.

[0098] The model training unit is used to divide the annotated historical weld image data and spot image data into a training set and a validation set, import the training set into the constructed deep learning model, iteratively train the deep learning model, and then verify the accuracy of the deep learning model in recognizing weld image features and spot image features through the validation set, and iteratively learn and optimize the deep learning model;

[0099] The feature extraction unit is used to import the weld image set and spot image set collected in real time into the deep learning model to perform feature extraction, divide the image into several regions according to pixel grayscale and gradient through the watershed algorithm, separate the weld area and the spot area from the image, measure the size of the segmented weld area, measure the size of the segmented spot area to obtain the size of the spot area, and obtain an image data set by calculating the size of the weld area and the size of the obtained spot area. The image data set includes the weld width hk, the penetration depth D, the spot shape S and the spot radius R.

[0100] In this embodiment, by integrating a feature recognition unit with a feature extraction unit, the deep learning module significantly enhances the intelligence and accuracy of the laser welding system. The feature recognition unit utilizes a data annotation unit and a deep learning unit, using the CVAT annotation tool and a deep learning model to perform detailed annotation and identification of weld and spot images, ensuring high data quality and accuracy. The feature extraction unit employs a watershed algorithm to accurately separate the weld and spot regions from real-time image acquisition, measure their dimensions, and generate an image dataset containing weld width hk, penetration depth D, spot shape S, and spot radius R. This approach not only improves the efficiency and accuracy of data processing but also enhances the reliability and stability of welding quality monitoring through iterative training and optimization of the deep learning model, ultimately improving production efficiency and product quality.

[0101] Example 4

[0102] This embodiment is explained in Example 1, please refer to Figure 1 , specifically: the data analysis module includes a penetration analysis unit, a pulse analysis unit and a spot analysis unit;

[0103] The penetration analysis unit includes a heat input calculation unit and a penetration calculation unit;

[0104] The heat input calculation unit is used to extract the current I, voltage V and welding time sj based on the acquired operation data set and image data set, and perform summary calculation to obtain the heat input index Q;

[0105] The heat input index Q is calculated by the following formula:

[0106]

[0107] Where k is the correction constant, represents the index;

[0108] The penetration calculation unit is used to extract the welding speed hj, weld width hk and penetration D based on the acquired operation data set and image data set, and the extracted data and the heat input index Q obtained by the heat input calculation unit are dimensionlessly processed and then summarized and calculated to obtain the penetration index WPD;

[0109] The penetration index WPD is obtained by the following formula:

[0110]

[0111] Where, Represents the adjustment factor used to balance the effects of speed, heat input and weld size on penetration.

[0112] The pulse analysis unit is used to extract the pulse frequency f, pulse width w and pulse energy E based on the acquired operating data set, and perform summary calculation to obtain the pulse characteristic index MCT;

[0113] The pulse characteristic index MCT is obtained by the following formula:

[0114]

[0115] Where, Indicates the pulse energy change rate, the rate of change of pulse energy per unit time, Indicates the change in pulse energy, represents the corresponding time variation, represents the weight value, Indicates the maximum value of the pulse frequency, Indicates the maximum value of the pulse width, Indicates the maximum value of the pulse energy.

[0116] The light spot analysis unit is used to extract the light spot shape S and the light spot radius R according to the extracted image data set, and perform summary calculation to obtain the light spot shape distortion index SSD;

[0117] The spot shape distortion index SSD is obtained by the following formula:

[0118]

[0119] Where Si represents the spot shape parameter of the i-th sampling point, represents the average spot shape parameter, n is the number of sampling points, Indicates the standard deviation of the spot shape parameters, the change of the spot shape between multiple measurements or different sampling points, represents the spot radius of the i-th sampling point, represents the average value of the spot radius, Indicates the standard deviation of the spot radius, the change of the spot radius between multiple measurements or different sampling points, Indicates the weight value.

[0120] The comprehensive analysis module is used to perform dimensionless processing on the acquired spot shape distortion index SSD, pulse characteristic index MCT and penetration index WPD, and then perform summary calculation to obtain a comprehensive welding quality index CWQI;

[0121] The comprehensive welding quality index CWQI is calculated by the following formula:

[0122]

[0123] Where, represents the inverse sine function, log represents the logarithmic function, and A represents the normalization factor.

[0124] In this embodiment, the laser welding system can accurately evaluate and control welding quality through the penetration analysis unit, pulse analysis unit, and spot analysis unit of the data analysis module. The penetration analysis unit calculates the heat input index Q by extracting parameters such as current, voltage, and welding time, and then calculates the penetration index WPD by combining the welding speed, weld width, and penetration depth, thereby achieving precise control of the welding depth. The pulse analysis unit calculates the pulse characteristic index MCT based on the pulse frequency, pulse width, and pulse energy, effectively evaluating the impact of pulse characteristics on welding quality. The spot analysis unit extracts the spot shape and spot radius based on image data and calculates the spot shape distortion index SSD to ensure the stability of the spot shape. The comprehensive analysis module non-dimensionalizes the spot shape distortion index SSD, the pulse characteristic index MCT, and the penetration index WPD, and calculates the comprehensive welding quality index CWQI, which provides a reliable basis for comprehensive evaluation of welding quality. Through the integration of these modules, the system significantly improves the accuracy and stability of the welding process, improves production efficiency and product reliability, reduces quality defects and rework costs, and ultimately brings significant economic benefits and competitive advantages.

[0125] Example 5

[0126] This embodiment is explained in Example 4. Please refer to Figure 1 ,Specifically: the welding evaluation control module includes a first evaluation unit and a second evaluation unit;

[0127] The first evaluation unit performs a preliminary evaluation by presetting a first penetration threshold A and a second penetration threshold B based on the historical standard welding quality penetration range of laser welding, and then combines the obtained penetration index WPD to analyze the penetration quality information of the current welding process. The specific evaluation scheme is as follows;

[0128] When the first penetration threshold A is less than the penetration index WPD and less than the second penetration threshold B, it indicates that the welding process meets the quality standards;

[0129] When the penetration index WPD is less than or equal to the first penetration threshold A, the welding is unqualified. In this case, the heat input is increased by 20%, the welding speed is reduced by 20%, and the adjusted parameters are iteratively calculated and evaluated by the penetration analysis unit and the first evaluation unit until the welding is qualified.

[0130] When the penetration index WPD ≥ the second penetration threshold B, it indicates that the welding is unqualified. In this case, the heat input is reduced by 20%, the welding speed is increased by 20%, and the second evaluation result is obtained. The adjusted parameters are iteratively calculated and evaluated by the penetration analysis unit and the first evaluation unit until the welding is qualified.

[0131] At this point, a second evaluation is performed through the second evaluation unit to comprehensively analyze the causes of welding quality.

[0132] The second evaluation unit performs a secondary comparative evaluation based on the average value of the historical welding quality, a preset first welding quality threshold M and a preset second welding quality N, and then compares and evaluates the obtained comprehensive welding quality index CWQI to analyze the welding quality of the current welding process. The specific evaluation scheme is as follows;

[0133] When the preset first welding quality threshold M is less than the comprehensive welding quality index CWQI and less than the preset second welding quality threshold N, it indicates that the welding quality is qualified. At this time, the current welding parameters and process are maintained.

[0134] When the comprehensive welding quality index CWQI is less than or equal to the preset first welding quality threshold M, it means that the welding quality is unqualified. At this time, the pulse energy is increased by 30%, the pulse width is reduced by 20%, the pulse frequency is increased by 30%, and the spot radius is reduced by 20%.

[0135] When the comprehensive welding quality index CWQI ≥ the preset second welding quality threshold N, it means that the welding quality is unqualified. At this time, the pulse energy is reduced by 20%, the pulse width is increased by 30%, the pulse frequency is reduced by 20%, and the spot radius is increased by 30%.

[0136] In this embodiment, by introducing the first evaluation unit and the second evaluation unit of the welding evaluation control module, the laser welding system realizes multi-level and refined quality monitoring and regulation; the first evaluation unit adjusts the heat input and welding speed in time according to the preset penetration threshold, through preliminary evaluation of the penetration index WPD, to ensure that the penetration quality is within the standard range; if the initial evaluation fails, iterative adjustment is performed until it passes; the second evaluation unit further performs a secondary evaluation of the comprehensive welding quality index CWQI according to the preset welding quality threshold to ensure that the welding quality meets the standard; this double-layer evaluation mechanism not only improves the accuracy and reliability of the welding process, but also significantly improves the stability and consistency of the welding quality, bringing higher efficiency and better product quality to the production process.

[0137] Example 6

[0138] See also Figure 2 , a laser welding control method based on machine vision, comprising the following steps:

[0139] S1. Install an integrated sensor group inside the laser equipment to collect the operating data of the laser equipment in real time, set an industrial camera to capture the welding image data of the laser equipment in real time during the welding process, and pre-process the collected operating data and welding image data;

[0140] S2. Based on the convolutional neural network (CNN) and recurrent neural network (RNN) as the basic framework of the deep learning model, a deep learning model is constructed to identify relevant features in the collected image data and perform feature extraction;

[0141] S3. Process the output feature vectors, perform multi-level data parsing and correlation analysis, perform dimensionless processing and comprehensive calculation on the extracted data set, and obtain the spot shape distortion index SSD, pulse characteristic index MCT and penetration index WPD respectively;

[0142] S4, performing correlation calculation on the acquired spot shape distortion index SSD, pulse characteristic index MCT and penetration index WPD to obtain a comprehensive welding quality index CWQI;

[0143] S5. Based on the historical standard welding quality penetration range of laser welding, a first penetration threshold A and a second penetration threshold B are preset and preliminary evaluated with the obtained penetration index WPD. The equipment is initially adjusted according to the evaluation results, and a second evaluation mechanism is started. The second evaluation mechanism is used to preset the first welding quality threshold M and the second welding quality threshold N and perform a secondary comparative evaluation with the obtained comprehensive welding quality index CWQI, and generate control information according to the evaluation results.

[0144] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A laser welding control system based on machine vision, characterized by: It includes welding data acquisition module, deep learning module, data analysis module, comprehensive analysis module and welding evaluation control module; The welding data acquisition module is used to install an integrated sensor group inside the laser equipment to collect the operating data of the laser equipment in real time, and set an industrial camera to capture the welding image data of the laser equipment in real time during the welding process, and pre-process the collected operating data and welding image data; The deep learning module is used to construct a deep learning model based on the convolutional neural network (CNN) and the recurrent neural network (RNN) as the basic framework of the deep learning model, identify relevant features in the collected image data, and perform feature extraction; The data analysis module is used to process the feature vectors output by the deep learning module, perform multi-level data analysis and correlation analysis, perform dimensionless processing and comprehensive calculation on the extracted data set, and respectively obtain the spot shape distortion index SSD, pulse characteristic index MCT and penetration depth index WPD; The comprehensive analysis module is used to perform correlation calculation on the acquired spot shape distortion index SSD, pulse characteristic index MCT and penetration index WPD to obtain a comprehensive welding quality index CWQI; The welding evaluation control module is used to perform a preliminary evaluation based on the historical standard welding quality penetration range of laser welding, the preset first penetration threshold A and the second penetration threshold B, and the obtained penetration index WPD, perform an initial adjustment on the equipment according to the evaluation results, and start a second evaluation mechanism. The second evaluation mechanism is used to perform a secondary comparative evaluation of the preset first welding quality threshold M and the second welding quality threshold N with the obtained comprehensive welding quality index CWQI, and generate control information according to the evaluation results.

2. The machine vision-based laser welding control system according to claim 1, characterized in that: The welding data acquisition module includes an image acquisition unit, a sensor acquisition unit and a data preprocessing unit; The image acquisition unit is used to set an industrial camera with a frame rate of more than 5000 above the welding area. At the same time, the lens of the industrial camera is set to a 105mm fixed-focus lens. During the welding process of the laser welding equipment, the welding area is photographed 36 times per second, and the spot image and weld image of the welding area are collected in real time; The sensor acquisition unit is used to install an integrated sensor group inside the laser equipment to collect the operating data of the laser equipment in real time and classify and summarize it to generate an operating data set. The integrated sensor group includes a voltage sensor, a current sensor, a timing sensor, a photoelectric sensor, an encoding sensor and a photodiode sensor. The operating data set includes voltage V, current I, welding time sj, pulse frequency f, welding speed hj, pulse energy E and pulse width w; The data preprocessing unit includes a parameter preprocessing unit and a picture preprocessing unit; The parameter preprocessing unit is used to perform data cleaning, filtering, denoising and dimensionless processing on the running data set; The image preprocessing unit is used to enhance the weld and spot boundaries in the video, grayscale the video, and Gaussian blur the collected image data by using morphological operations and image processing software OpenCV.

3. The machine vision-based laser welding control system according to claim 1, characterized in that: The deep learning module includes a feature recognition unit and a feature extraction unit; The feature recognition unit includes a data annotation unit and a deep learning unit; The data annotation unit is used to import a large amount of collected historical weld image data and historical spot image data into the CVAT annotation tool, and annotate the images with welds and light spots through the annotation function of the CVAT annotation tool; By marking the weld shape features of the weld frame image, the weld shape features include weld width features and weld penetration features; By marking the features of the light spot frame image, the light spot features include light spot shape features and light spot radius features; The deep learning unit includes an image recognition unit and a model training unit; The image recognition unit integrates a convolutional neural network (CNN) and a recurrent neural network (RNN) as a model framework to construct a deep learning model. The deep learning model extracts local features from the image through the convolution layer and pooling layer of the convolutional neural network (CNN), classifies the weld shape and the spatial distribution of the light spot using a fully connected layer, identifies the weld shape and the spatial distribution characteristics of the light spot, and uses the Canny edge detection algorithm to mark the boundary information of the weld and the light spot. At the same time, the recurrent neural network (RNN) captures the temporal information and dependency of the data through a recursive structure and a long short-term memory unit, and identifies the weld morphology changes and the light spot pulse characteristics. The model training unit is used to divide the annotated historical weld image data and spot image data into a training set and a validation set, import the training set into the constructed deep learning model, iteratively train the deep learning model, and then verify the accuracy of the deep learning model in recognizing weld image features and spot image features through the validation set, and iteratively learn and optimize the deep learning model; The feature extraction unit is used to import the weld image set and spot image set collected in real time into the deep learning model to perform feature extraction, divide the image into several regions according to pixel grayscale and gradient through the watershed algorithm, separate the weld area and the spot area from the image, measure the size of the segmented weld area, measure the size of the segmented spot area to obtain the size of the spot area, and obtain an image data set by calculating the size of the weld area and the size of the obtained spot area. The image data set includes the weld width hk, the penetration depth D, the spot shape S and the spot radius R.

4. The machine vision-based laser welding control system according to claim 1, characterized in that: The data analysis module includes a penetration analysis unit, a pulse analysis unit and a spot analysis unit; The penetration analysis unit includes a heat input calculation unit and a penetration calculation unit; The heat input calculation unit is used to extract the current I, voltage V and welding time sj based on the acquired operation data set and image data set, and perform summary calculation to obtain the heat input index Q; The heat input index Q is calculated by the following formula: Where k is the correction constant, represents the index; The penetration calculation unit is used to extract the welding speed hj, weld width hk and penetration D based on the acquired operation data set and image data set, and the extracted data and the heat input index Q obtained by the heat input calculation unit are dimensionlessly processed and then summarized and calculated to obtain the penetration index WPD; The penetration index WPD is obtained by the following formula: Where, represents the adjustment factor.

5. The machine vision-based laser welding control system according to claim 4, characterized in that: The pulse analysis unit is used to extract the pulse frequency f, pulse width w and pulse energy E based on the acquired operating data set, and perform summary calculation to obtain the pulse characteristic index MCT; The pulse characteristic index MCT is obtained by the following formula: Where, represents the pulse energy change rate, Indicates the change in pulse energy, represents the corresponding time variation, Indicates the weight value.

6. The machine vision-based laser welding control system according to claim 4, characterized in that: The light spot analysis unit is used to extract the light spot shape S and the light spot radius R according to the extracted image data set, and perform summary calculation to obtain the light spot shape distortion index SSD; The spot shape distortion index SSD is obtained by the following formula: Where Si represents the spot shape parameter of the i-th sampling point, represents the average spot shape parameter, n is the number of sampling points, represents the standard deviation of the spot shape parameters, represents the spot radius of the i-th sampling point, represents the average value of the spot radius, represents the standard deviation of the spot radius, Indicates the weight value.

7. The machine vision-based laser welding control system according to claim 1, characterized in that: The comprehensive analysis module is used to perform dimensionless processing on the acquired spot shape distortion index SSD, pulse characteristic index MCT and penetration index WPD, and then perform summary calculation to obtain a comprehensive welding quality index CWQI; The comprehensive welding quality index CWQI is calculated by the following formula: Where, represents the inverse sine function, log represents the logarithmic function, and A represents the normalization factor.

8. The machine vision-based laser welding control system according to claim 4, characterized in that: The welding evaluation control module includes a first evaluation unit and a second evaluation unit; The first evaluation unit performs a preliminary evaluation by presetting a first penetration threshold A and a second penetration threshold B based on the historical standard welding quality penetration range of laser welding, and then combines the obtained penetration index WPD to analyze the penetration quality information of the current welding process. The specific evaluation scheme is as follows; When the first penetration threshold A is less than the penetration index WPD and less than the second penetration threshold B, it indicates that the welding process meets the quality standards; When the penetration index WPD is less than or equal to the first penetration threshold A, the welding is unqualified. In this case, the heat input is increased by 20%, the welding speed is reduced by 20%, and the adjusted parameters are iteratively calculated and evaluated by the penetration analysis unit and the first evaluation unit until the welding is qualified. When the penetration index WPD ≥ the second penetration threshold B, it indicates that the welding is unqualified. In this case, the heat input is reduced by 20%, the welding speed is increased by 20%, and the second evaluation result is obtained. The adjusted parameters are iteratively calculated and evaluated by the penetration analysis unit and the first evaluation unit until the welding is qualified. At this point, a second evaluation is performed through the second evaluation unit to comprehensively analyze the causes of welding quality.

9. The machine vision-based laser welding control system according to claim 8, characterized in that: The second evaluation unit performs a secondary comparative evaluation based on the average value of the historical welding quality, a preset first welding quality threshold M and a preset second welding quality N, and then compares and evaluates the obtained comprehensive welding quality index CWQI to analyze the welding quality of the current welding process. The specific evaluation scheme is as follows; When the preset first welding quality threshold M is less than the comprehensive welding quality index CWQI and less than the preset second welding quality threshold N, it indicates that the welding quality is qualified, and the current welding parameters and process are maintained; When the comprehensive welding quality index CWQI is less than or equal to the preset first welding quality threshold M, it means that the welding quality is unqualified. At this time, the pulse energy is increased by 30%, the pulse width is reduced by 20%, the pulse frequency is increased by 30%, and the spot radius is reduced by 20%. When the comprehensive welding quality index CWQI ≥ the preset second welding quality threshold N, it means that the welding quality is unqualified. At this time, the pulse energy is reduced by 20%, the pulse width is increased by 30%, the pulse frequency is reduced by 20%, and the spot radius is increased by 30%.

10. A laser welding control method based on machine vision, applied to the laser welding control system based on machine vision according to any one of claims 1 to 9, characterized in that: The following steps are involved: S1. Install an integrated sensor group inside the laser equipment to collect the operating data of the laser equipment in real time, set an industrial camera to capture the welding image data of the laser equipment in real time during the welding process, and pre-process the collected operating data and welding image data; S2. Based on the convolutional neural network (CNN) and recurrent neural network (RNN) as the basic framework of the deep learning model, a deep learning model is constructed to identify relevant features in the collected image data and perform feature extraction; S3. Process the output feature vectors, perform multi-level data parsing and correlation analysis, perform dimensionless processing and comprehensive calculation on the extracted data set, and obtain the spot shape distortion index SSD, pulse characteristic index MCT and penetration index WPD respectively; S4, performing correlation calculation on the acquired spot shape distortion index SSD, pulse characteristic index MCT and penetration index WPD to obtain a comprehensive welding quality index CWQI; S5. Based on the historical standard welding quality penetration range of laser welding, a first penetration threshold A and a second penetration threshold B are preset and preliminary evaluated with the obtained penetration index WPD. The equipment is initially adjusted according to the evaluation results, and a second evaluation mechanism is started. The second evaluation mechanism is used to preset the first welding quality threshold M and the second welding quality threshold N and perform a secondary comparative evaluation with the obtained comprehensive welding quality index CWQI, and generate control information according to the evaluation results.

Images