Welding robot constraint performance visual servo control method and system
Through constrained performance visual servo control and laser visual weld tracking technology, the problems of slow convergence speed and field of view of welding robot visual servo control are solved, and high-precision welding operations are achieved, reducing the scrap rate and defect rate.
Patent Information
- Application Number
- CN202411216596.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-02
- Publication Date
- 2024-11-15
AI Technical Summary
The existing visual servo control of welding robots has problems such as slow convergence speed, unstable control process, and servo failure due to limited camera field of view, making it difficult to ensure high accuracy and consistency of welding operations.
The constrained performance visual servo control method is adopted, and the image feature deviation of the computer robot joint motion relationship and combined with laser visual weld tracking technology, a faster convergence speed and a smoother motion trajectory are achieved, avoiding servo failure caused by field of vision constraints.
It effectively reduces the scrap rate and defect rate in welding operations, improves the accuracy and quality of robot welding, and realizes a high-precision automatic welding mode.
Smart Images

Figure CN118951528A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of welding robot vision sensing, and in particular relates to a welding robot constraint performance visual servo control method and system. Background Art
[0002] With the continuous development of my country's manufacturing industry, the demand for welding processing industry continues to grow. Welding is a professional technical job. Enterprises need to invest a lot of time and financial resources to train a mature welding technician. It is difficult to ensure the consistency of product quality of manual welding, which is not conducive to mass production. Long-term welding work will also reduce the production efficiency of welders. With the increasing application of industrial robots, using welding robots to replace manual welding can solve this problem to a certain extent.
[0003] Visual servo is a control method that integrates many disciplines such as image processing, robotics, and control theory. It mainly controls the robot end to reach the desired target position through visual feature information, and completes the robot visual servo control by using the error of image features as the input of robot servo control. With the continuous development of visual sensing technology, the application of visual sensing technology in weld trajectory tracking is becoming more and more extensive. Compared with the application of other types of sensors in weld trajectory tracking, visual sensors have the advantages of no contact with the welding workpiece, high measurement accuracy, and anti-electromagnetic interference, and are suitable for various types of weld trajectory tracking.
[0004] After years of research and exploration, robot visual servo control theory and visual sensing technology have become relatively mature, but their application in the field of robot welding is relatively rare. On the one hand, the arc light and other interferences in the welding process are not conducive to the extraction of visual information. On the other hand, it is necessary to establish a relatively complete visual servo control model for robot weld tracking. However, the visual servo control in the existing technology has the problems of slow convergence speed, unstable control process, and visual servo failure due to the limited field of view of the camera. Summary of the invention
[0005] The purpose of the present invention is to provide a welding robot constraint performance visual servo control method and system, which constrains and controls the visual servo through performance functions, so that the improved visual servo converges faster and has a smoother motion trajectory, and integrates laser visual weld tracking technology to effectively solve the problem of servo failure caused by camera field of view constraints, and effectively reduces the scrap rate and defect rate of the robot welding system during the welding operation as a whole, thereby realizing high-precision robot welding operations.
[0006] The technical solution adopted by the present invention is as follows: A welding robot constraint performance visual servo control method, comprising: S1: Collect images containing weld features, and extract image features after image preprocessing; S2: Compare the extracted image features with the expected image features to obtain image feature deviations; S3: Obtain the image feature deviation value as the input of the constraint performance visual servo control, and obtain the relationship between the image feature and the robot joint motion from the calculated value of the image Jacobian matrix; S4: The robust state feedback controller calculates the control amount according to the image feature deviation and drives the robot end to move toward the target workpiece; S5: The laser vision sensor collects the image of the initial frame in real time and transmits it to the computer for image processing, extracts the image weld feature points, and performs spatial pose conversion on the pixel coordinates of the weld image feature points to obtain spatial pose information; S6: Based on the spatial posture information, the inverse kinematics solution is performed through the Movet toolbox to control the robot end to move to the weld feature point and complete the robot weld tracking trajectory.
[0007] The step of collecting an image containing weld features in S1 and extracting image features after preprocessing the image also includes: The robot welding hardware platform that can be compared with the camera is used to simulate the virtual robot welding system through the ABB robotsdudio simulation platform 3D modeling, and the image visual servo control system with constraint performance is integrated for simulation experiments; After the robot welding hardware platform is built, the welded pressure vessel pipe workpiece is placed on the welding area of the welding hardware platform and clamped. The camera and laser vision sensor are installed at the end of the robotic arm, and the camera calibration experiment is debugged to obtain the camera's intrinsic parameter matrix, extrinsic parameter matrix and distortion parameters. Then the laser vision sensor and the ABB robot's hand-eye calibration are completed to calculate the pose conversion matrix from the robot's polar coordinates to the camera end. The weld image is collected by the camera, and constraints are imposed on the image pixels collected by the camera in the visual servo scheme. The original weld image is then subjected to image preprocessing operations such as image grayscale, ROI weld extraction, image binarization, and image morphological processing to obtain image features.
[0008] The step of comparing the extracted image features with the expected image features to obtain image feature deviations in S2 further includes: This step requires pre-defining transient and steady-state responses, defining the characteristic errors of the upper and lower boundaries of the pixel plane as an overall error matrix, and selecting appropriate parameters to ensure that the image features remain within the camera field of view and the pre-defined transient and steady-state errors.
[0009] In S3, the obtained deviation value is used as the input of the constraint performance visual servo control. The step of obtaining the relationship between the image feature and the robot joint motion from the calculated value includes calculating the overall error matrix based on the expected image feature error and the actual image feature error, and using this as the input of the constraint performance visual servo control. When the hand is on the eye, the movement speed of the camera is the same as the movement speed of the end actuator of the robot arm. Therefore, by substituting the camera movement speed and the image coordinate movement speed, the interaction matrix of all feature points can be obtained after the image Jacobian matrix conversion, realizing the mapping relationship between the robot motion and the weld image features.
[0010] In S4, the robust state feedback controller calculates the control quantity according to the image feature deviation, and the step of driving the robot end to move toward the target workpiece includes comparing the extracted image features with the expected image features, and inputting the calculated image feature deviation into the controller as feedback. Based on the robust control theory, the state feedback controller considers the joint posture, speed and angle of the system, and the robot executes the control command synchronously, and feeds back the joint position and motion state to the controller in real time, forming a closed-loop control system.
[0011] In S4, the laser vision sensor collects images in real time and transmits them to the computer for processing. The steps of extracting the weld feature points in the image and converting the pixel coordinates of the feature points into a spatial posture state include: applying constrained image visual servoing to guide the robot field of view to a suitable welding workpiece area, and adjusting the end of the robot to irradiate the stripes of the laser sensor to the starting point of the weld, and then obtaining the pixel information of the feature points of the initial weld image through image processing, and converting the pixel information into the spatial posture information of the robot tool coordinate system.
[0012] In a preferred embodiment, it also includes a laser sensing system, an image comparison module, an input module, a control module, a processing module and a tracking module. The laser sensing system includes a camera and a laser transmitter arranged at the end of the robot arm. The line laser emitted by the laser transmitter irradiates the surface of the workpiece to form laser stripes. After the laser stripes are reflected by the surface of the workpiece, a weld image is presented on the industrial camera. The image is preprocessed to extract image features. The image comparison module is used to compare the image features with the expected image features to obtain image feature deviations. The input module uses the obtained image feature deviation value as the input of the constraint performance visual servo control, and the image Jacobian matrix is used to obtain the image feature deviation value. The relationship between the image feature and the robot joint motion is calculated, and the control module signal is transmitted to the robust state feedback controller, so that the robust state feedback controller calculates the control amount according to the image feature deviation and drives the robot end to move toward the target workpiece. The processing module collects the image of the initial frame in real time by the laser vision sensor and transmits it to the computer for image processing, and extracts the image weld feature points, and performs spatial posture conversion on the pixel coordinates of the weld image feature points to obtain spatial posture information. The tracking module controls the robot end to move to the weld feature point through the Movet toolbox according to the spatial posture information through inverse kinematics solution, and completes the robot weld tracking trajectory.
[0013] In a preferred embodiment, it also includes visual servo control based on constraint performance, joint controller, multi-sensor fusion calibration module, weld trajectory recognition module and weld tracking module, which are used to control and guide the welding gun at the end of the robot arm to reach the predetermined position where the weld starts, wherein after the camera collects the weld image, the actual image features are calculated after image processing, and the expected features and constraint pixels are predefined by the performance function. The information includes the value of the feature error, and the joint controller is used to receive the motion instruction, and the above feature error is used as the input of the robust state feedback controller, and the appropriate control amount is selected to substitute, so that the image feature deviation tends to 0, and the visual servo guides the welding gun at the end of the robot arm to reach the starting point of the weld; Among them, the constraint performance-based visual servo control is controlled and guided by the constructed platform, and the platform of the architecture is: a robot weld tracking architecture of image visual servoing, which has a multi-sensor fusion calibration module. Through the mutual interlocking of multiple sensors, the relationship between the position and posture of each sensor is determined. The weld trajectory recognition module is placed at the front end of the welding gun, scans and obtains the welding point according to the trajectory recognition, and then transmits it to the motion control model. The real-time weld tracking module includes a real-time weld tracking module, a spatial posture receiving module and a robot joint control module. During the welding process, the real-time weld tracking module continuously collects weld position information through the weld trajectory recognition module, and combines the hand-eye calibration matrix and the real-time posture of the robot to convert the weld coordinates into the robot basic coordinate system for real-time tracking.
[0014] In a preferred embodiment, the image preprocessing step also includes line laser centerline extraction and calibration plate corner point extraction. The image is acquired and preprocessed, and the edge of the laser beam in the image is detected using a line laser centerline equation or a fitted line laser plane equation. The internal and external parameters of the camera are calibrated and sampled and photographed to complete the calibration plate corner point extraction.
[0015] In a preferred embodiment, it also includes: Collect and process the camera image to confirm whether it is at the initial position of the weld; The weld image processing is inconsistent, the tracking target is extracted, and the tracking data is imported. The camera vision is adjusted and tracked according to the weld feature points; The weld image processing is consistent, and the camera vision tracks the weld feature points; Based on the tracking characteristics, the tracker predicts parameter updates until the weld end point is reached; If the end point is not reached, the camera re-collects images; Reach the end point and end the weld tracking task.
[0016] In a preferred embodiment, image visual servoing is also included: Image visual servoing, which guides the robot welding gun to the starting point of the weld, and the laser vision sensor at the front end of the welding gun collects the image of the initial frame and transmits it to the computer for image processing; The laser vision sensor collects images in real time and transmits them to the computer. The computer extracts and analyzes the pixel coordinates of the weld feature points in the image and converts the pixel coordinates into the spatial posture of the robot end. The process is solved by inverse kinematics through the Movel toolbox, and finally the welding gun at the end of the robot reaches the specified position. After the welding gun at the end of the robot reaches the specified position, the camera determines whether it has reached the last frame of the weld image. If not, it returns to the step of the laser vision sensor collecting images in real time and transmitting them to the computer. If it has, the process ends.
[0017] The technical effects achieved by the present invention are: The present invention proposes a method for image visual servo control based on constraint performance, which constrains and controls the visual servo through performance functions, so that the improved visual servo converges faster and has a smoother motion trajectory, which is advanced in theory and technology. The present invention fully integrates image visual servo technology and laser visual weld tracking technology into the robot welding system, realizing an automatic welding mode. Compared with the teaching programming mode and the offline programming mode, it has the advantage of being able to perform dynamic adjustments, thereby improving the welding quality of the robot welding method and reducing the scrap rate. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Figure 1 It is a schematic diagram of the overall process of the present invention; Figure 2 is a system flow chart of the present invention; Figure 3 The constraint performance visual servoing flow chart of the present invention; Figure 4 It is a schematic diagram of the weld tracking software architecture of the present invention; Figure 5 It is a flow chart of the calibration algorithm of the present invention; Figure 6 The weld image tracking algorithm flow of the present invention; Figure 7 This is a flow chart of the robot control module of the present invention. DETAILED DESCRIPTION
[0019] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.
[0020] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.
[0021] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure or characteristic that may be included in at least one implementation of the present invention. The phrase "in a preferred embodiment" that appears in different places in this specification does not refer to the same embodiment, nor is it a separate or selective embodiment that is mutually exclusive with other embodiments.
[0022] Secondly, the present invention is described in detail with reference to the schematic diagram. When describing the embodiments of the present invention in detail, for the sake of convenience, the cross-sectional diagrams showing the device structure will not be partially enlarged according to the general scale, and the schematic diagrams are only examples, which should not limit the scope of protection of the present invention. In addition, in actual production, the three-dimensional dimensions of length, width and depth should be included.
[0023] Please refer to the attached Figure 1-Figure 6 As shown, the present invention provides a welding robot constraint performance visual servo control method, the specific steps are as follows: S1: Collect images containing weld features, and extract image features after image preprocessing; S2: Compare the extracted image features with the expected image features to obtain image feature deviations; S3: Obtain the image feature deviation value as the input of the constraint performance visual servo control, and obtain the relationship between the image feature and the robot joint motion from the calculated value of the image Jacobian matrix; S4: The robust state feedback controller calculates the control amount according to the image feature deviation and drives the robot end to move toward the target workpiece; S5: The laser vision sensor collects the image of the initial frame in real time and transmits it to the computer for image processing, extracts the image weld feature points, and performs spatial pose conversion on the pixel coordinates of the weld image feature points to obtain spatial pose information; S6: Based on the spatial posture information, the inverse kinematics solution is performed through the Movet toolbox to control the robot end to move to the weld feature point and complete the robot weld tracking trajectory.
[0024] Among them, the Movet toolbox solves the robot's inverse kinematics based on the spatial posture information of the weld feature points, controls the robot to move to the specified position, and the camera tracks the weld features in real time. It calculates the deviation between the robot's current posture and the weld feature points based on the position of the feature points, and transmits the deviation data to the Movet toolbox for robot joint control, thereby completing the robot's weld tracking trajectory. When the camera recognizes the features at the end of the weld, the welding task of the welder is stopped.
[0025] S1 also includes a robot welding hardware platform that can be compared with the camera. A virtual robot welding system is simulated through three-dimensional modeling of the ABB robotsdudio simulation platform, and an image visual servo control system with constraint performance is integrated to conduct simulation experiments; and after the robot welding hardware platform is built, the welded pressure vessel pipeline workpiece is placed on the welding area of the welding hardware platform for clamping and fixing. The camera can be one of the RealSense cameras or high-definition industrial cameras, or other industrial cameras that are more convenient for capturing images. The camera and the laser vision sensor are installed at the end of the robot arm, and the camera calibration experiment is debugged to obtain the camera's intrinsic parameter matrix, extrinsic parameter matrix and distortion parameters. Then the laser vision sensor and the ABB robot's hand-eye calibration are completed, and the posture conversion matrix from the robot's polar coordinates to the end of the camera is calculated. When the camera collects the weld image, constraints are imposed on the image pixels collected by the camera in the visual servo scheme, and then the original weld image is subjected to image grayscale, ROI weld extraction, image binarization, and image morphology processing image preprocessing operations to obtain image features.
[0026] Among them, the image features obtained by the above-mentioned image processing operation are not only limited to the image processing listed in this application. Various image processing algorithms or image edge detection in the prior art can be combined or updated according to actual production conditions.
[0027] The step of comparing the extracted image features with the expected image features to obtain the image feature deviation in S2 includes requiring the step to predefine transient and steady-state responses, defining the feature errors of the upper and lower boundaries of the pixel plane as an overall error matrix, respectively, and selecting appropriate parameters to ensure that the image features remain within the camera field of view and the predefined transient and steady-state errors.
[0028] In S3, the obtained deviation value is used as the input of the constraint performance visual servo control. The step of obtaining the relationship between the image feature and the robot joint motion from the calculated value includes calculating the overall error matrix based on the expected image feature error and the actual image feature error, and using this as the input of the constraint performance visual servo control. When the hand is on the eye, the movement speed of the camera is the same as the movement speed of the end actuator of the robot arm. Therefore, by substituting the camera movement speed and the image coordinate movement speed, the interaction matrix of all feature points can be obtained after the image Jacobian matrix conversion, realizing the mapping relationship between the robot motion and the weld image features.
[0029] In S4, the robust state feedback controller calculates the control quantity according to the image feature deviation. The step of driving the robot end to move toward the target workpiece includes comparing the extracted image features with the expected image features and inputting the calculated image feature deviation into the controller as feedback. Based on the robust control theory, the state feedback controller considers the joint posture, speed and angle of the system. The robot executes the control command synchronously and feeds back the joint position and motion state to the controller in real time to form a closed-loop control system.
[0030] In S4, the laser vision sensor collects images in real time and transmits them to the computer for processing. The steps of extracting the weld feature points in the image and converting the pixel coordinates of the feature points into a spatial posture state include: applying constrained image visual servoing to guide the robot field of view to a suitable welding workpiece area, and adjusting the end of the robot to irradiate the stripes of the laser sensor to the starting point of the weld, and then obtaining the pixel information of the feature points of the initial weld image through image processing, and converting the pixel information into the spatial posture information of the robot tool coordinate system.
[0031] See also Figure 2-Figure 3 , the welding robot constraint performance visual servo control system also includes: Laser sensing system, image comparison module, input module, control module, processing module and tracking module. The laser sensing system includes a camera and a laser transmitter at the end of the robotic arm. The line laser emitted by the laser transmitter irradiates the surface of the workpiece to form laser stripes. After the laser stripes are reflected by the surface of the workpiece, a weld image is presented on the industrial camera. After image preprocessing, image features are extracted. The image comparison module is used to compare the image features with the expected image features to obtain image feature deviations. The input module uses the obtained image feature deviation value as the input of the constraint performance visual servo control, and obtains the image features from the calculated value of the image Jacobian matrix. In relation to the robot joint movement, the control module signal is transmitted to the robust state feedback controller, so that the robust state feedback controller calculates the control amount according to the image feature deviation and drives the robot end to move toward the target workpiece. The processing module collects the image of the initial frame in real time by the laser vision sensor and transmits it to the computer for image processing, and extracts the image weld feature points, and transforms the pixel coordinates of the weld image feature points into spatial posture to obtain spatial posture information. The tracking module controls the robot end to move to the weld feature point through the Movel toolbox according to the spatial posture information through inverse kinematics solution, and completes the robot weld tracking trajectory.
[0032] The laser sensing system mentioned in the above structure is composed of a CMOS industrial camera, a laser emitter, a lens, a mirror, a baffle and a housing. The baffle at the joint has protection functions such as splash protection and heat resistance. The sensor is installed on the welding gun, the internal industrial camera is parallel to the front surface of the housing, and the laser emitter is placed at a fixed angle. When the welding task is performed, the laser vision sensor maintains a certain height between the workpiece to be welded, and the line laser emitted by the internal laser emitter irradiates the surface of the workpiece, forming a laser stripe. After being reflected by the surface of the workpiece, the corresponding image is presented on the industrial camera through the lens.
[0033] In the above structure, Figure 2 The IBVS in the figure stands for constraint-based visual servo control, which can use image information to control the movement of the robot, enabling it to complete tasks such as target tracking, docking, and grasping, so that the welding gun can synchronously track the workpiece and make corresponding instructions.
[0034] See also Figure 4 , visual servo control based on constraint performance is used to control and guide the welding gun at the end of the robot arm to the predetermined position where the weld starts, wherein after the camera captures the weld image, the actual image features are calculated after image processing, and the desired features and constraint pixels are predefined by the performance function, and this piece of information contains the numerical value of the feature error; The joint controller is used to receive motion instructions, take the above characteristic errors as the input of the robust state feedback controller, select appropriate control quantities to substitute, so that the image characteristic deviation tends to 0, and the visual servo guides the welding gun at the end of the robot arm to reach the starting point of the weld; Among them, the visual servo control based on constraint performance is controlled and guided by the architecture platform, and the architecture platform is: the robot weld tracking architecture of image visual servoing, which has The multi-sensor fusion calibration module determines the relationship between the position and posture of each sensor through the mutual interlocking of multiple sensors; Weld seam trajectory recognition module, which is placed at the front end of the welding gun, scans and obtains welding points based on trajectory recognition, and then transmits them to the motion control model; The weld tracking module includes a real-time weld tracking module, a spatial posture receiving module and a robot joint control module. During the welding process, the real-time weld tracking module continuously collects weld position information through the weld trajectory recognition module, and combines the hand-eye calibration matrix and the robot's real-time posture to convert the weld coordinates into the robot's basic coordinate system for real-time tracking.
[0035] Among them, the core control modules of the bottom layer of the robot weld tracking architecture of image visual servoing mainly include robot communication, trajectory planning, image processing and weld trajectory recognition. The software part mainly includes some OpenCV algorithm libraries, calibration code posture recognition algorithms, robot control tracking programs, etc.
[0036] Based on the above structure, the robot weld tracking architecture of image visual servo constructed by the camera at the end of the robotic arm can continuously collect and identify the weld position information of the weld trajectory, and calculate the actual image features after image processing. It overcomes the problem that the image features in the robot welding servo vision process exceed the field of view and the collected image is accompanied by interference such as arc light and spatter, which leads to the failure of visual servo. It also predefines the expected features and constrained pixels through the performance function, and applies constrained visual servo to the overall robot welding system of weld tracking, thereby improving the welding accuracy, visual servo convergence speed and control efficiency of the robot welding system.
[0037] See also Figure 5, and also includes the steps of image preprocessing including line laser centerline extraction and calibration plate corner point extraction. By acquiring the image and preprocessing the image, the edge of the laser beam in the image is detected using the line laser centerline equation or the fitted line laser plane equation, the internal and external parameters of the camera are calibrated and sampled for shooting, and the calibration plate corner point extraction is completed; the calibration plate corner point extracts the pre-designed corner point coordinates, adjusts the internal and external parameters of the camera for calibration and sampled for shooting, and after completion, the calibration corner points in the sample are detected, and the calibration plate corner point coordinates are generated. In the camera calibration process, the calibration plate plane equation calculates and corrects the camera internal parameters and improves the image accuracy. The robot motion parameters include the values obtained by solving the translation matrix and rotation matrix constraints, fitting multiple sets of constraint equations, and calculating the hand-eye relationship matrix. The values are used as instructions to guide and constrain the motion posture and travel of the robot joints.
[0038] Based on the above, the accuracy of weld tracking depends on the calibration accuracy of each module, and the calibration of the visual sensor is the key link. It is necessary to complete the calibration of each sensor of the weld tracking system, including the camera, laser camera and robot hand-eye calibration.
[0039] See also Figure 6 , also includes the weld image tracking process of the robot weld tracking architecture of image visual servoing: Collect and process the camera image to confirm whether it is at the initial position of the weld; The weld image processing is inconsistent, the tracking target is extracted, and the tracking data is imported. The camera vision is adjusted and tracked according to the weld feature points; The weld image processing is consistent, and the camera vision tracks the weld feature points; Based on the tracking characteristics, the tracker predicts parameter updates until the weld end point is reached; If the end point is not reached, the camera will re-capture the image; Reach the end point and end the weld tracking task.
[0040] Among them, the image processing module mainly calls the image information collected by the camera in real time for processing, calculates the characteristic points of the weld through the weld laser image collected by the visual sensor, and performs image processing on the initial weld laser image to obtain the characteristic points of the weld.
[0041] See also Figure 7 , also includes image visual servoing, which guides the robot welding gun to the starting point of the weld, and the laser vision sensor at the front end of the welding gun collects the image of the initial frame and transmits it to the computer for image processing; The laser vision sensor collects images in real time and transmits them to the computer. The computer extracts and analyzes the pixel coordinates of the weld feature points in the image and converts the pixel coordinates into the spatial posture of the robot end. The process is solved by inverse kinematics through the Movel toolbox, and finally the welding gun at the end of the robot reaches the specified position. After the welding gun at the end of the robot reaches the specified position, the camera determines whether it has reached the last frame of the weld image. If not, it returns to the step of the laser vision sensor collecting images in real time and transmitting them to the computer. If it has, the process ends.
[0042] Among them, the Movet module and Soket module of the ROS platform are used to realize the motion control of the ABB robot, the end welding gun posture calculation, and the visual data communication of the sensor, completing the weld tracking and welding operations of the laser vision sensor.
[0043] As indicated in the above steps, the visual sensor calibration is used to visually convey and collect visual data, and the robot motion parameters are closely integrated with each calibration. When the vision is conveyed to the robot motion parameters, the matrix equation in the robot motion parameters directly calculates the posture value, and uses this value as an instruction to guide and constrain the motion posture travel of the robot joint, so that the improved visual servo converges faster and the motion trajectory is smoother. In addition, the above-mentioned laser visual weld tracking technology is integrated to effectively solve the problem of servo failure caused by camera field of view constraints, and the overall scrap rate and defect rate of the robot welding system during the welding operation are effectively reduced. High-precision robot welding operations are achieved, and the lean level and sustainable development level of the manufacturing industry are improved. On the one hand, the servo control system based on constraint performance is integrated, and on the other hand, the weld tracking system is integrated to overcome the problem that the traditional image visual servo system cannot complete the image visual servo control in complex environments well, especially the image feature points are easily lost during the servo process, causing the problem of visual servo failure, and realize the automatic welding mode, improve the welding quality of the robot welding method, and reduce the scrap rate of the robot welding operation.
[0044] The working principle of the present invention is as follows: by fixing a camera and a laser vision sensor at the end of a six-axis robot arm to collect weld image information, the image pixels of the fixed camera are actively added with upper and lower boundary constraints through a controller, weld feature extraction is performed after image processing, data is transmitted to a computer to convert the image information into two-dimensional information, and the difference between the actual image feature and the expected image feature is calculated to obtain the image feature deviation, complete the calculation of the feedback control amount, and use this as the input of the IBVS control system, use the robust state feedback controller with constraint performance to realize FOV constraints and predefined transient and steady-state responses, and then obtain the interaction matrix between the image features and the camera motion through the image Jacobian matrix, and finally input the motion command to the robot joint controller, the image visual servo guides the robot end welding gun to reach the desired posture, and then the laser vision sensor collects the laser stripes in real time and extracts the weld feature points therein and transmits them to the computer to control the robot welding, completing the robot welding operation of image visual servoing and weld tracking.
[0045] The above is only a preferred embodiment of the present invention. It should be noted that, for those skilled in the art, several improvements and modifications can be made without departing from the principles of the present invention, and these improvements and modifications should also be considered as the protection scope of the present invention. The structures, devices and operating methods not specifically described and explained in the present invention shall be implemented according to the conventional means in the art unless otherwise specified and limited.
Claims
1. A welding robot constraint performance visual servo control method, characterized in that: include: Collect images containing weld features, and extract image features after image preprocessing; The extracted image features are compared with the expected image features to obtain image feature deviations; The image feature deviation value is obtained as the input of the constraint performance visual servo control, and the relationship between the image feature and the robot joint motion is obtained from the calculated value of the image Jacobian matrix; The robust state feedback controller calculates the control amount according to the image feature deviation and drives the robot end to move toward the target workpiece; The laser vision sensor collects the initial frame image in real time and transmits it to the computer for image processing, extracts the image weld feature points, and performs spatial pose conversion on the pixel coordinates of the weld image feature points to obtain spatial pose information; According to the spatial posture information, the inverse kinematics solution is performed through the Movet toolbox to control the robot end to move to the weld feature point and complete the robot weld tracking trajectory.
2. A welding robot constraint performance visual servo control method according to claim 1, characterized in that: The step of collecting an image containing weld features and extracting image features after preprocessing the image includes: The robot welding hardware platform that can be compared with the camera is used to simulate the virtual robot welding system through the ABB robotsdudio simulation platform 3D modeling, and the image visual servo control system with constraint performance is integrated for simulation experiments; After the robot welding hardware platform is built, the welded pressure vessel pipe workpiece is placed on the welding area of the welding hardware platform and clamped. The camera and laser vision sensor are installed at the end of the robotic arm, and the camera calibration experiment is debugged to obtain the camera's intrinsic parameter matrix, extrinsic parameter matrix and distortion parameters. Then the laser vision sensor and the ABB robot's hand-eye calibration are completed to calculate the pose conversion matrix from the robot's polar coordinates to the camera end. The weld image is collected by the camera, and constraints are imposed on the image pixels collected by the camera in the visual servo scheme. The original weld image is then subjected to image preprocessing operations such as image grayscale, ROI weld extraction, image binarization, and image morphological processing to obtain image features.
3. A welding robot constraint performance visual servo control method according to claim 1, characterized in that: The step of comparing the extracted image features with the expected image features to obtain the image feature deviation comprises: This step requires pre-defining transient and steady-state responses, defining the characteristic errors of the upper and lower boundaries of the pixel plane as an overall error matrix, and selecting appropriate parameters to ensure that the image features remain within the camera field of view and the pre-defined transient and steady-state errors.
4. A welding robot constraint performance visual servo control method according to claim 1, characterized in that: The obtained deviation value is used as the input of the constraint performance visual servo control, and the steps of obtaining the relationship between the image feature and the robot joint motion from the calculated value include: The overall error matrix is calculated based on the expected image feature error and the actual image feature error, and used as the input of the constraint performance visual servo control. When the hand is on the eye, the movement speed of the camera is the same as the movement speed of the end actuator of the robot arm. Therefore, by substituting the camera movement speed and the image coordinate movement speed, the interaction matrix of all feature points can be obtained after the image Jacobian matrix conversion, realizing the mapping relationship between robot motion and weld image features.
5. The visual servo control method for constraint performance of a welding robot according to claim 1, characterized in that: The robust state feedback controller calculates the control amount according to the image feature deviation and drives the robot end to move toward the target workpiece: The extracted image features are compared with the expected image features, and the calculated image feature deviation is input into the controller as feedback. Based on the robust control theory, the state feedback controller takes into account the joint posture, speed and angle of the system. The robot executes the control commands synchronously and feeds back the joint position and motion state to the controller in real time to form a closed-loop control system.
6. A welding robot constraint performance visual servo control method according to claim 1, characterized in that: The laser vision sensor collects images in real time and transmits them to the computer for processing, extracts the weld feature points in the image, and transforms the pixel coordinates of the feature points into spatial pose: The constrained image visual servoing guides the robot's field of view to the appropriate welding workpiece area, and by adjusting the robot end, the laser sensor stripes are irradiated to the starting point of the weld. The feature point pixel information of the initial weld image is obtained through image processing, and the pixel information is converted into the spatial pose information of the robot tool coordinate system.
7. A welding robot constraint performance visual servo control system, characterized in that: Used to operate and process to implement the method described in any one of claims 1 to 6, including: A laser sensing system, wherein the laser sensing system comprises a camera and a laser emitter provided at the end of the robot arm. The line laser emitted by the laser emitter is irradiated onto the surface of the workpiece to form laser stripes. After the laser stripes are reflected by the surface of the workpiece, a weld image is presented on the industrial camera. The image features are extracted after preprocessing the image. An image comparison module, the image comparison module is used to compare the image feature with the expected image feature to obtain an image feature deviation; An input module, wherein the input module obtains image feature deviation values as input of constraint performance visual servo control, and obtains the relationship between image features and robot joint motions from the calculated values of the image Jacobian matrix; A control module, wherein the control module transmits a signal to a robust state feedback controller, so that the robust state feedback controller calculates a control amount according to an image feature deviation and drives the robot end to move toward a target workpiece; A processing module, wherein the processing module collects the image of the initial frame in real time by the laser vision sensor and transmits it to the computer for image processing, extracts the image weld feature points, performs spatial pose conversion on the pixel coordinates of the weld image feature points, and obtains spatial pose information; The tracking module controls the robot end to move to the weld feature point through the inverse kinematics solution of the Movet toolbox according to the spatial posture information, and completes the robot weld tracking trajectory.
8. A welding robot constraint performance visual servo control system according to claim 7, characterized in that: Also includes: Visual servo control based on constraint performance is used to control and guide the welding gun at the end of the robot arm to the predetermined position where the weld starts. After the camera captures the weld image, the actual image features are calculated after image processing. The expected features and constraint pixels are predefined by the performance function. This information contains the value of the feature error. The joint controller is used to receive motion instructions, take the above characteristic errors as the input of the robust state feedback controller, select appropriate control quantities to substitute, so that the image characteristic deviation tends to 0, and the visual servo guides the welding gun at the end of the robot arm to reach the starting point of the weld; The constraint-based visual servo control is controlled and guided by the platform of the architecture, which is an image visual servo robot weld tracking architecture with The multi-sensor fusion calibration module determines the relationship between the position and posture of each sensor through the mutual interlocking of multiple sensors; Weld seam trajectory recognition module, which is placed at the front end of the welding gun, scans and obtains welding points based on trajectory recognition, and then transmits them to the motion control model; The weld tracking module comprises a real-time weld tracking module, a spatial posture receiving module and a robot joint control module. During the welding process, the real-time weld tracking module continuously collects weld position information through the weld trajectory recognition module, and combines the hand-eye calibration matrix and the real-time posture of the robot to convert the weld coordinates into the robot basic coordinate system for real-time tracking.
9. A welding robot constraint performance visual servo control system according to claim 8, characterized in that: The robot weld tracking module also includes: Collect and process the camera image to confirm whether it is at the initial position of the weld; The weld image processing is inconsistent, the tracking target is extracted, and the tracking data is imported. The camera vision is adjusted and tracked according to the weld feature points; The weld image processing is consistent, and the camera vision tracks the weld feature points; Based on the tracking characteristics, the tracker predicts parameter updates until the weld end point is reached; If the end point is not reached, the camera re-collects images; Reach the end point and end the weld tracking task.
10. A welding robot constraint performance visual servo control system according to claim 9, characterized in that: Also includes: Image visual servoing, which guides the robot welding gun to the starting point of the weld, and the laser vision sensor at the front end of the welding gun collects the image of the initial frame and transmits it to the computer for image processing; The laser vision sensor collects images in real time and transmits them to the computer. The computer extracts and analyzes the pixel coordinates of the weld feature points in the image and converts the pixel coordinates into the spatial posture of the robot end. The process is solved by inverse kinematics through the Movel toolbox, and finally the welding gun at the end of the robot reaches the specified position. After the welding gun at the end of the robot reaches the specified position, the camera determines whether it has reached the last frame of the weld image. If not, it returns to the step of the laser vision sensor collecting images in real time and transmitting them to the computer. If it has, the process ends.
Citation Information
Cited By
Uncalibrated visual servo method, system and equipment for six-degree-of-freedom mechanical arm and medium
CN121004612A
A method, system, device and medium for uncalibrated visual servoing for a six degree of freedom robotic arm
CN121004612B
Constraint visual servo method, system and device based on imitation learning and medium
CN121083667A
A constraint visual servoing method, system, device and medium based on imitation learning
CN121083667B
Robot projection welding visual positioning method and device for vehicle body welding
CN121373703A
