Robot welding control method and system

Abstract

The invention discloses a robot welding control method and system. The robot welding control method comprises the following steps that vision, temperature and force sense data of a welding target are collected through a multi-mode sensor; inputting the acquired multi-modal data into a multi-modal deep learning model; a reinforcement learning algorithm is used for optimizing the initial welding path, and the welding path and welding parameters are dynamically adjusted according to real-time sensor data in the welding process; the welding quality is monitored in real time, and welding parameters are dynamically adjusted according to the feedback control module; and multi-modal sensor data, a welding path, welding parameters and welding quality evaluation indexes in the welding process are recorded. According to the method, the complex state in the welding process can be comprehensively and accurately reflected through multi-modal sensor data fusion, and compared with a method only depending on single-modal data in the prior art, the boundary, temperature distribution and mechanical changes of the welding area can be more accurately recognized, and the welding accuracy is improved. Therefore, the accuracy of welding path planning and parameter adjustment is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of robot welding control, and in particular to a robot welding control method and system. Background Art

[0002] In modern industrial manufacturing, welding technology is one of the key processes for connecting metal materials and is widely used in the fields of automobile manufacturing, aerospace, shipbuilding, energy equipment, etc. With the rapid development of industrial automation and intelligent manufacturing, traditional manual welding has been gradually replaced by robot welding, which has the advantages of high efficiency, high precision, good consistency, etc., and can significantly improve production efficiency and product quality.

[0003] In the prior art, welding path planning and parameter adjustment mainly rely on single-modal sensor data. Since single-modal data cannot fully reflect the complex state of the welding process, the accuracy of welding path planning and parameter adjustment is insufficient, which in turn affects the stability and consistency of welding quality. For example, during the welding process, relying solely on visual data cannot accurately identify the temperature distribution of the welding area, which may lead to improper adjustment of welding parameters and thus welding defects. Summary of the invention

[0004] The object of the present invention is to provide a robot welding control method and system to solve the problems raised in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] A robot welding control method comprises the following steps:

[0007] Step S1, collecting visual, temperature and force data of the welding target through a multimodal sensor, wherein the multimodal sensor includes a high-resolution industrial camera, an infrared thermal imager and a force sensor;

[0008] Step S2, inputting the collected multimodal data into a multimodal deep learning model, wherein the multimodal deep learning model includes a convolutional neural network and a long short-term memory network, which is used to identify the welding area and generate an initial welding path;

[0009] Step S3, using a reinforcement learning algorithm to optimize the initial welding path, and dynamically adjusting the welding path and welding parameters according to real-time sensor data during the welding process;

[0010] Step S4, real-time monitoring of welding quality and dynamic adjustment of welding parameters according to the feedback control module;

[0011] Step S5, recording the multimodal sensor data, welding path, welding parameters and welding quality evaluation indicators during the welding process for subsequent analysis and optimization.

[0012] In the present invention, in step S1, multimodal sensor data collection includes:

[0013] Step S101, using a high-resolution industrial camera to collect an RGB image of a welding target to obtain the surface texture and geometric shape of the target;

[0014] Step S102, using an infrared thermal imager to collect a temperature distribution image of the welding target and identify temperature changes in the welding area;

[0015] Step S103: Use a force sensor to monitor the contact force and torque during the welding process in real time.

[0016] In the present invention, in step S2, the training process of the multimodal deep learning model includes:

[0017] Step S201, taking the RGB image, the temperature distribution image and the force data as input, and the bounding box of the welding area and the initial welding path as output;

[0018] Step S202, using a convolutional neural network to extract image features and a long short-term memory network to process time series data;

[0019] Step S203, optimizing the model parameters through the loss function to ensure the accuracy of the welding area and the initial path generation.

[0020] In the present invention, in step S3, the optimization process of the reinforcement learning algorithm includes:

[0021] Step S301, taking the initial welding path and real-time sensor data as input, and taking welding path adjustment and welding parameter optimization as output;

[0022] Step S302, using a deep Q network as a reinforcement learning algorithm to maximize welding quality and minimize welding error through a reward function;

[0023] Step S303, during the welding process, dynamically adjust the welding path and welding parameters according to the real-time data of the infrared thermal imager and the force sensor.

[0024] In the present invention, in step S4, the real-time monitoring and feedback of welding quality includes:

[0025] Step S401, using a high frame rate industrial camera to shoot the welding process in real time, combining the data of the infrared thermal imager and the force sensor to generate a real-time evaluation index of the welding quality;

[0026] Step S402, input the evaluation index into the feedback control module, and the module dynamically adjusts the welding parameters, including welding current and welding speed, according to the preset quality standard.

[0027] In the present invention, in step S5, the welding process data recording and analysis includes:

[0028] Step S501, storing the multimodal sensor data, welding path, welding parameters and welding quality evaluation indexes in the welding process into a database;

[0029] Step S502, using a data analysis tool to analyze the stored data, identify common problems and optimization points in the welding process, and generate optimization suggestions.

[0030] In the present invention, the multimodal deep learning model and reinforcement learning algorithm are run on an embedded GPU platform to ensure real-time performance and computational efficiency.

[0031] In the present invention, the feedback control module adopts a fuzzy control algorithm to dynamically adjust welding parameters according to real-time evaluation indicators of welding quality.

[0032] In the present invention, the force sensor is installed at the end of the robot to monitor the contact force and torque during the welding process in real time and feed the data back to the reinforcement learning algorithm.

[0033] A robot welding control system, comprising:

[0034] at least one processor;

[0035] at least one memory for storing at least one program;

[0036] When the at least one program is executed by the at least one processor, the at least one processor implements any one of the methods described above.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] 1. The present invention can comprehensively and accurately reflect the complex state of the welding process through multi-modal sensor data fusion. Compared with the method in the prior art that only relies on single modal data, the present invention can more accurately identify the boundary, temperature distribution and mechanical changes of the welding area, thereby significantly improving the accuracy of welding path planning and parameter adjustment. For example, during the welding process, the temperature data provided by the infrared thermal imager is combined with the mechanical data provided by the force sensor, which can effectively avoid welding defects caused by uneven temperature distribution or mechanical changes.

[0039] 2. The present invention adopts reinforcement learning and PID control algorithms, which can dynamically adjust the welding path and parameters according to real-time sensor data to ensure that the welding quality meets the preset standards. Compared with the static welding control method in the prior art, the present invention can monitor the welding quality in real time and dynamically optimize the welding parameters to minimize the welding error and improve the consistency of the welding quality. For example, during the welding process, the PID control algorithm can dynamically adjust the welding parameters according to the real-time error to ensure that the welding quality is stable and meets the process requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 The figure is a flow chart of a robot welding control method of the present invention. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] See also Figure 1 , the present invention provides a technical solution:

[0043] A robot welding control method comprises the following steps:

[0044] Step S1, collecting visual, temperature and force data of the welding target through a multimodal sensor, wherein the multimodal sensor includes a high-resolution industrial camera, an infrared thermal imager and a force sensor;

[0045] Specifically, multimodal sensor data collection includes:

[0046] Step S101, using a high-resolution industrial camera to collect an RGB image of the welding target to obtain the surface texture and geometric shape of the target. A Basler ace series industrial camera with a resolution of 5 million pixels and a frame rate of 30fps is used. The camera is installed on a robot arm at a distance of about 50 cm from the welding target to collect the RGB image of the welding target. The collected image is denoised by an image processing algorithm to ensure image quality.

[0047] Step S102, using an infrared thermal imager to collect a temperature distribution image of the welding target and identify the temperature change of the welding area. A FLIRA65 infrared thermal imager with a resolution of 640×480 and a temperature measurement range of -20°C to 150°C is used. The image is installed on a robot arm about 30 cm away from the welding target to collect a temperature distribution image of the welding area. The thermal imaging data is calibrated by a temperature calibration algorithm to ensure the accuracy of the temperature measurement.

[0048] Step S103, use a force sensor to monitor the contact force and torque during the welding process in real time. Use an ATIOmega160 force sensor installed at the end of the robot to monitor the contact force and torque during the welding process in real time. Use a low-pass filter to filter the force data to remove high-frequency noise.

[0049] In step S1, during the welding process, the industrial camera provides visual information, the infrared thermal imager provides temperature information, and the force sensor provides mechanical information. The above data needs to be fused through Kalman filtering to generate a more accurate state estimate. Provide reliable input for subsequent path planning and parameter adjustment. The formula is as follows:

[0050]

[0051] in, represents the estimated value of the state at the kth moment, represents the predicted state value at the kth moment, K k represents the Kalman gain, z k represents the observed value at the kth moment, H k represents the observation matrix, P k|k-1 represents the prediction error covariance matrix, R k represents the observation noise covariance matrix.

[0052] Step S2, inputting the collected multimodal data into a multimodal deep learning model, wherein the multimodal deep learning model includes a convolutional neural network and a long short-term memory network, which is used to identify the welding area and generate an initial welding path;

[0053] Specifically, the training process of the multimodal deep learning model includes:

[0054] Step S201, taking the RGB image, the temperature distribution image and the force data as input, and the bounding box of the welding area and the initial welding path as output;

[0055] Step S202, using a convolutional neural network to extract image features and a long short-term memory network to process time series data;

[0056] Step S203, optimizing the model parameters through the loss function to ensure the accuracy of the welding area and the initial path generation;

[0057] In this embodiment, the PyTorch framework is used, the loss function is a combination of cross entropy loss and mean square error, the optimizer is Adam, the learning rate is 0.001, the training data set contains 10,000 sets of multimodal data, and the A* algorithm is used to generate the initial welding path according to the welding area bounding box output by the deep learning model. The path planning goal is to minimize the welding length and maximize the welding quality.

[0058] In step S2, the multimodal deep learning model needs to complete the welding area classification and path regression tasks at the same time. The joint loss function L of the cross entropy loss and the mean square error can ensure that the model performs well in both classification and regression tasks, thereby improving the accuracy of welding area recognition. The formula is as follows:

[0059]

[0060] Among them, L represents the total loss, N represents the number of samples, and y i represents the true value of the i-th sample, Represents the predicted value of the i-th sample.

[0061] Step S3, using a reinforcement learning algorithm to optimize the initial welding path, and dynamically adjusting the welding path and welding parameters according to real-time sensor data during the welding process;

[0062] Specifically, the optimization process of the reinforcement learning algorithm includes:

[0063] Step S301, taking the initial welding path and real-time sensor data as input, and taking welding path adjustment and welding parameter optimization as output;

[0064] Step S302, using a deep Q network as a reinforcement learning algorithm to maximize welding quality and minimize welding error through a reward function;

[0065] Step S303, during the welding process, dynamically adjust the welding path and welding parameters according to the real-time data of the infrared thermal imager and the force sensor.

[0066] In this embodiment, a deep Q network (DQN) is used as a reinforcement learning algorithm. The state space is the welding path and real-time sensor data, and the action space is the welding path adjustment and welding parameter optimization. The reward function is designed according to the welding error and welding quality evaluation index. The goal is to minimize the welding error and maximize the welding quality. The TensorFlow framework is used. The training data set contains 5000 sets of welding paths and sensor data. The number of training rounds is 1000. During the welding process, the welding path and welding parameters are dynamically adjusted according to the real-time data of the infrared thermal imager and the force sensor to ensure the welding quality.

[0067] In step S3, during the welding process, welding error and welding quality are optimization targets. By designing the reward function R, the reinforcement learning algorithm can dynamically adjust the welding path and parameters to minimize the welding error and maximize the welding quality. The formula is as follows:

[0068] R=-α·error+β·quality

[0069] Among them, R represents the reward value, α represents the error weight coefficient, β represents the quality weight coefficient, error represents the welding error, and quality represents the welding quality evaluation index.

[0070] Step S4, real-time monitoring of welding quality and dynamic adjustment of welding parameters according to the feedback control module;

[0071] Specifically, real-time monitoring and feedback of welding quality include:

[0072] Step S401, using a high frame rate industrial camera to shoot the welding process in real time, combining the data of the infrared thermal imager and the force sensor to generate a real-time evaluation index of the welding quality;

[0073] Step S402, input the evaluation index into the feedback control module, and the module dynamically adjusts the welding parameters, including welding current and welding speed, according to the preset quality standard.

[0074] In step S4, the PID control formula u(t) is used to dynamically adjust the welding parameters, including but not limited to the welding current, welding speed, etc., to ensure that the welding quality meets the preset standards. The formula is as follows:

[0075]

[0076] Where u(t) represents the control output, K p Represents proportional gain, K i Indicates the integral gain, K d represents the differential gain, and e(t) represents the error value.

[0077] In this embodiment, a high frame rate industrial camera is used to capture the welding process in real time, and the data from an infrared thermal imager and a force sensor are combined to generate real-time evaluation indicators of welding quality, including weld width, weld depth and temperature distribution. A fuzzy control algorithm is used to dynamically adjust welding parameters based on the real-time evaluation indicators of welding quality to ensure that the welding quality meets preset standards.

[0078] Step S5, recording multimodal sensor data, welding path, welding parameters and welding quality evaluation indicators during the welding process for subsequent analysis and optimization;

[0079] Specifically, welding process data recording and analysis include:

[0080] Step S501, storing the multimodal sensor data, welding path, welding parameters and welding quality evaluation indexes in the welding process into a database;

[0081] Step S502, using a data analysis tool to analyze the stored data, identify common problems and optimization points in the welding process, and generate optimization suggestions.

[0082] In this embodiment, the multimodal sensor data, welding path, welding parameters and welding quality evaluation indicators in the welding process are stored in a MySQL database in the data storage format of JSON. The Pandas and Matplotlib libraries in Python are used to analyze the stored data, identify common problems and optimization points in the welding process, and generate optimization suggestions.

[0083] In the present invention, the multimodal deep learning model and reinforcement learning algorithm are run on an embedded GPU platform to ensure real-time performance and computational efficiency.

[0084] In the present invention, the feedback control module adopts a fuzzy control algorithm to dynamically adjust welding parameters according to real-time evaluation indicators of welding quality.

[0085] In the present invention, the force sensor is installed at the end of the robot to monitor the contact force and torque during the welding process in real time and feed the data back to the reinforcement learning algorithm.

[0086] A robot welding control system, comprising:

[0087] at least one processor;

[0088] at least one memory for storing at least one program;

[0089] When the at least one program is executed by the at least one processor, the at least one processor implements any one of the methods described above.

[0090] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0091] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A robot welding control method, characterized in that: The following steps are involved: Step S1, collecting visual, temperature and force data of the welding target through a multimodal sensor, wherein the multimodal sensor includes a high-resolution industrial camera, an infrared thermal imager and a force sensor; Step S2, inputting the collected multimodal data into a multimodal deep learning model, wherein the multimodal deep learning model includes a convolutional neural network and a long short-term memory network, which is used to identify the welding area and generate an initial welding path; Step S3, using a reinforcement learning algorithm to optimize the initial welding path, and dynamically adjusting the welding path and welding parameters according to real-time sensor data during the welding process; Step S4, real-time monitoring of welding quality and dynamic adjustment of welding parameters according to the feedback control module; Step S5, recording the multimodal sensor data, welding path, welding parameters and welding quality evaluation indicators during the welding process for subsequent analysis and optimization.

2. A robot welding control method according to claim 1, characterized in that: In step S1, multimodal sensor data collection includes: Step S101, using a high-resolution industrial camera to collect an RGB image of a welding target to obtain the surface texture and geometric shape of the target; Step S102, using an infrared thermal imager to collect a temperature distribution image of the welding target and identify temperature changes in the welding area; Step S103: Use a force sensor to monitor the contact force and torque during the welding process in real time.

3. A robot welding control method according to claim 1, characterized in that: In step S2, the training process of the multimodal deep learning model includes: Step S201, taking the RGB image, the temperature distribution image and the force data as input, and the bounding box of the welding area and the initial welding path as output; Step S202, using a convolutional neural network to extract image features and a long short-term memory network to process time series data; Step S203, optimizing the model parameters through the loss function to ensure the accuracy of the welding area and the initial path generation.

4. A robot welding control method according to claim 1, characterized in that: In step S3, the optimization process of the reinforcement learning algorithm includes: Step S301, taking the initial welding path and real-time sensor data as input, and taking welding path adjustment and welding parameter optimization as output; Step S302, using a deep Q network as a reinforcement learning algorithm to maximize welding quality and minimize welding error through a reward function; Step S303, during the welding process, dynamically adjust the welding path and welding parameters according to the real-time data of the infrared thermal imager and the force sensor.

5. A robot welding control method according to claim 1, characterized in that: In step S4, the real-time monitoring and feedback of welding quality includes: Step S401, using a high frame rate industrial camera to shoot the welding process in real time, combining the data of the infrared thermal imager and the force sensor to generate a real-time evaluation index of the welding quality; Step S402, input the evaluation index into the feedback control module, and the module dynamically adjusts the welding parameters, including welding current and welding speed, according to the preset quality standard.

6. A robot welding control method according to claim 1, characterized in that: In step S5, the welding process data recording and analysis includes: Step S501, storing the multimodal sensor data, welding path, welding parameters and welding quality evaluation indexes in the welding process into a database; Step S502, using a data analysis tool to analyze the stored data, identify common problems and optimization points in the welding process, and generate optimization suggestions.

7. A robot welding control method according to claim 1, characterized in that: The multimodal deep learning model and reinforcement learning algorithm are run on an embedded GPU platform to ensure real-time performance and computational efficiency.

8. A robot welding control method according to claim 1, characterized in that: The feedback control module adopts a fuzzy control algorithm to dynamically adjust welding parameters according to real-time evaluation indicators of welding quality.

9. A robot welding control method according to claim 1, characterized in that: The force sensor is installed at the end of the robot to monitor the contact force and torque during the welding process in real time and feed the data back to the reinforcement learning algorithm.

10. A robot welding control system, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method according to any one of claims 1 to 9.

Images