Grabbing control system for robot visual guidance

Abstract

The invention discloses a grabbing control system for robot visual guidance, and belongs to the technical field of grabbing control systems.The grabbing control system comprises a visual perception module used for collecting workpiece images, extracting workpiece feature information and calculating accurate coordinates of workpieces in space; the robot main body module is used for receiving a system instruction to drive a carrier to move to a target position; the grabbing execution module is used for being in contact with the workpiece, grabbing the workpiece stably and transferring the workpiece to a designated position to be placed without damage; the system integration module is used for guaranteeing the safety, operability and stability of the feeding and discharging process; according to the system, workpiece information can be accurately collected, the posture of the grabbing execution end is analyzed based on the workpiece position, the grabbing posture is corrected in real time according to the grabbing dynamic state, and the workpiece grabbing precision and stability are improved.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of a grabbing control system, in particular to a grabbing control system for robot vision guidance. BACKGROUND

[0002] In a workpiece feeding and discharging grabbing scene, the robot vision guidance grabbing control system is a cooperative system of vision perception, motion control and operation execution, which accurately positions the workpiece through vision, guides the robot to complete grabbing, carrying and placing actions, and solves the problems of poor flexibility, poor adaptability to workpiece posture and weak ability to change position of traditional mechanical positioning.

[0003] A patent with publication number CN216505198U discloses an industrial robot feeding and discharging system, which comprises a first processing and transporting device, a first feeding platform, a second feeding platform, a second processing and transporting device, a second discharging platform, a first discharging platform and a robot. The first processing and transporting device, the first feeding platform, the second feeding platform, the second processing and transporting device, the second discharging platform and the first discharging platform are arranged in a circle around the robot and are sequentially arranged. The robot comprises a robot arm and a robot hand installed at the end of the robot arm. The robot hand is suitable for grabbing or carrying workpieces. The industrial robot feeding and discharging system has high automation, and is convenient for production management and real-time control.

[0004] A patent with publication number CN217072375U discloses a mechanical arm feeding and discharging control system, which comprises a mechanical arm, a grabbing module, a falling prevention module, an alarm module and a control module. The grabbing module is used for grabbing workpieces and is arranged on the mechanical arm and is in communication connection with the control module. The falling prevention module is used for acquiring falling information of the workpieces and is arranged on the mechanical arm and is in communication connection with the control module. The alarm module is used for sending alarm information and is in communication connection with the control module. The control module is used for controlling the grabbing module to grab workpieces and controlling the alarm module to send alarm information. The system can monitor whether the materials are in a stable transportation state during the process of grabbing materials.

[0005] As the patent with publication number CN112403932A discloses an automatic workpiece transfer system and a control method thereof, including a protective fence, a grabbing robot, a turnover unit, an up and down feeding channel unit, a material frame buffer unit and a workpiece material channel are arranged in the protective fence respectively, and the turnover unit, the up and down feeding channel unit, the workpiece material channel and the material frame buffer unit are distributed in a ring around the grabbing robot in turn, a control unit is arranged on the outside of the protective fence, the system has strong compatibility, can not only visually identify and classify and select a plurality of parts, but also can make the workpiece keep a unified posture, facilitate subsequent processing of the workpiece, improve the utilization efficiency of the production line, reduce the labor intensity of workers, and realize the automation function of the robot up and down feeding link of the assembly line; however, metal parts and smooth workpieces are prone to highlight and shadow in strong light or low light environment, resulting in detection distortion, which is not conducive to image acquisition and analysis of workpiece position and posture, and meanwhile, the posture of the end effector does not match the workpiece during grabbing, which is prone to cause the workpiece to fall or collide, and is not conducive to accurate grabbing of the workpiece.

[0006] In view of the above problems, it is urgent to make innovative design on the basis of the original grabbing control system. SUMMARY

[0007] The purpose of the present application is to provide a robot vision guided grabbing control system to solve the problem of metal parts and smooth workpieces prone to highlight and shadow in strong light or low light environment, resulting in detection distortion, which is not conducive to image acquisition and analysis of workpiece position and posture, and meanwhile, the posture of the end effector does not match the workpiece during grabbing, which is prone to cause the workpiece to fall or collide, and is not conducive to accurate grabbing of the workpiece.

[0008] To achieve the above purpose, the present application provides the following technical scheme: a robot vision guided grabbing control system, the grabbing control system comprises: A visual perception module is used for collecting workpiece images, extracting workpiece feature information, calculating the accurate coordinates of the workpiece in space, quickly estimating and correcting the grabbing posture according to the collected workpiece information, and providing a precise material taking basis.

[0009] A robot main body module is used for receiving system instructions to drive the carrier to move to the target position, which includes a motion control unit, a moving unit and a position feedback unit, wherein the motion control unit is responsible for receiving the workpiece coordinate data of the visual perception module, planning the motion trajectory in combination with the robot kinematics model, generating the robot control instruction, and coordinating the synchronization of visual positioning and robot action in real time, the position feedback unit collects the position information of the robot motion in real time and feeds back to the motion control unit to realize the instruction and actual position closed loop correction.

[0010] The gripping execution module is used to contact and stably grip the workpiece, and transfer the workpiece to the designated position for placement without damage. The main structure of the gripping execution module is a fixture, which is designed and controlled based on the workpiece material, shape and surface features.

[0011] The system integration module ensures the safety, operability, and stability of the loading and unloading process. It includes a human-machine interaction unit, a safety protection unit, an external equipment collaboration unit, and a data storage and traceability unit. The human-machine interaction unit enables parameter configuration, status monitoring, manual operation, and fault alarms. The safety protection unit prevents the robot's movement from causing injury to operators and equipment, and assists in construction in accordance with industrial safety standards. The external equipment collaboration unit is responsible for coordinating the timing logic of various devices and is used in loading and unloading scenarios, where the robot's main module works in conjunction with other equipment on the production line. The data storage and traceability unit records production data, supporting quality traceability and production optimization.

[0012] Preferably, the visual perception module calculates the precise coordinate parameters of the workpiece in the spatial coordinate system based on three-dimensional visual positioning technology. Specifically, it includes: a light source, which senses the dynamic changes of ambient light in real time, overcomes ambient light interference, highlights the edges, holes, and texture features of the workpiece, and ensures image stability; an image acquisition unit, consisting of an industrial camera, lens, and image acquisition card, whose core task is to convert the optical information of the target object or scene illuminated by the light source into two-dimensional images or point cloud data that can be processed subsequently; a visual processing unit, which runs visual algorithms to complete image preprocessing, feature extraction, coordinate calculation, and data output; and a pose estimation unit, which extracts the workpiece image features based on the image information processed by the visual processing unit and matches them with features in a preset template or database to determine the type and approximate position of the workpiece. The pose estimation unit integrates multiple algorithms, including SIFT, SURF, PnP, gradient descent, and Levenberg-Marquardt algorithms, and calculates the dynamic situation of the workpiece and the grasping in real time to quickly analyze the difference between the current grasping posture and the preset ideal posture.

[0013] Preferably, the light source possesses adaptive illumination and reflection suppression technology, specifically: based on sensor technology, when the ambient light increases, the resistance of the photoresistor decreases accordingly, and vice versa. Through the change in resistance value, the sensor converts the ambient light intensity information into an electrical signal. Based on signal analysis and processing, it determines whether the current ambient light meets the requirements for image acquisition. In the lampshade or lens part of the light source, a material and structural design with diffuse reflection characteristics are adopted, so that the light emitted by the light source is scattered, making the light more uniform and softer to illuminate the surface of the workpiece, reducing concentrated reflection of light. Most of the reflections on the surface of metal parts and smooth workpieces are polarized light. Adding a polarizing filter in front of the light source or camera lens can selectively filter out polarized reflected light. By performing algorithm processing on the acquired image, the influence of reflection is further weakened. Based on deep learning image enhancement algorithms, the reflective areas in the image are identified and analyzed, and the brightness, contrast, and color saturation parameters of the area are adjusted in a targeted manner to make the image information of the reflective area clearer and restore the true features of the workpiece surface.

[0014] Preferably, the Levenberg-Marquardt algorithm is used to optimize pose parameters, achieving high-precision pose estimation by minimizing the difference between visual measurements and theoretical values; the feature points of the workpiece in three-dimensional space are... In the coordinate system of the robot's end effector, the projected coordinates of the feature points are: The camera intrinsic parameter matrix is ​​K; the projection relationship of feature points from the world coordinate system to the image coordinate system is as follows:

[0015] in, Let be the scale factor. The Levenberg-Marquardt algorithm aims to minimize the sum of squares of reprojection errors, calculated as follows:

[0016] in, It is the theoretical projected coordinates calculated from the pose parameter x, i.e., the error function. The total error is .

[0017] Preferably, the position feedback unit collects the robot's actual position and attitude in real time through encoders, visual odometry, and high-precision LiDAR sensors to form closed-loop control. Specifically, this includes joint feedback, where the encoder of each joint directly measures the angle. , and the angle of the instruction The error is The error signal is sent to the motion control unit, where the control command is corrected via PID control; global position feedback uses visual markers to obtain the robot's position in the world coordinate system. , and target location The deviation is used to correct the path, and its formula is: .

[0018] Preferably, the collaborative logic of the motion control unit, the movement unit, and the position feedback unit is as follows: the visual perception module outputs the workpiece coordinates. The motion control unit will The robot's position is converted into a target pose, its trajectory is planned, and control commands are generated. The mobilization unit executes the commands, driving the robot to move towards the target. The position feedback unit collects the actual position in real time. The error is calculated and fed back to the motion control unit; the motion control unit performs closed-loop correction according to the error adjustment command until the target position is reached.

[0019] Preferably, the gripping execution module includes: a pneumatic gripper, which drives a cylinder and a gripper with compressed air to achieve opening and closing actions; an electric gripper, which drives a gear and a lead screw with a servo motor and a stepper motor to achieve precise opening and closing; a vacuum suction cup gripper, which generates negative pressure with a vacuum pump and a vacuum generator to adsorb the surface of the workpiece to achieve gripping; and a customized gripper, which is a special gripper designed according to the special shape of the workpiece with irregular grooves and multiple protrusions, including a chuck and a claw.

[0020] Preferably, the gripping execution module has adaptive clamping force control capability, which monitors the clamping force in real time through a force sensor. , and preset threshold By comparing the results, a closed-loop control system can be formed.

[0021] in, The driving signal ensures that the clamping force is within a safe range.

[0022] Preferably, the gripping execution module includes gripping posture calibration features. Combined with the workpiece pose output by the vision perception module, the gripper needs to adjust its own posture before contacting the workpiece to ensure it is aligned with the normal direction of the workpiece surface and to avoid lateral slippage. Based on the degrees of freedom of the robot's end effector, the gripper's posture parameters are adjusted, specifically including: rotational adjustment, and adjustment around three coordinate axes. rotation angle This aligns the fixture coordinate system with the workpiece surface coordinate system; fine-tuning involves making small translations along the normal direction and in the plane perpendicular to the normal before contacting the workpiece to ensure that the fixture is directly opposite and centered to the workpiece surface.

[0023] Preferably, the workpiece transfer and placement process of the gripping execution module includes: a transfer process in which the movement speed is planned according to the workpiece quality and stability requirements, and path tracking is achieved through position feedback from the robot's main module; and a placement process in which a soft landing strategy is adopted, the speed is reduced upon contact with the target surface, and a release action is triggered through force feedback.

[0024] Compared with the prior art, the beneficial effects of the present invention are: On industrial production lines, light sources work closely with robotic arms that assist in loading and unloading workpieces. They sense dynamic changes in ambient light in real time and make corresponding adjustments quickly to ensure that workpieces can be clearly illuminated under any lighting conditions. This also reduces the possibility of strong reflections on the workpiece surface, making the reflections on the workpiece surface more uniform, reducing the occurrence of bright areas, improving the accuracy of workpiece information collection, and providing a foundation for precise workpiece gripping.

[0025] As the robot approaches the workpiece, the pose estimation unit quickly analyzes the difference between the current grasping posture and the preset ideal posture based on the image information fed back in real time by the vision sensor. Once a difference is detected, the pose estimation unit reacts quickly and dynamically adjusts the grasping posture. Throughout the grasping process, the pose estimation unit continuously receives real-time feedback and makes continuous dynamic adjustments to ensure that the robot can grasp the workpiece in the best posture, thereby improving the success rate and stability of grasping. Attached Figure Description

[0026] Fig. 1 This is a diagram showing the system module composition of the present invention.

[0027] Fig. 2 This is a flowchart of the capture control process of the present invention. Detailed Implementation

[0028] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] This application provides a robot vision-guided grasping control system, which includes: The visual perception module is used to acquire workpiece images, extract workpiece feature information, calculate the precise coordinates of the workpiece in space, and quickly estimate and correct the grasping posture based on the acquired workpiece information, providing a basis for accurate material handling.

[0030] The main robot module is used to receive system commands to drive the carrier to move to the target position. It includes a motion control unit, a movement unit, and a position feedback unit. The motion control unit receives workpiece coordinate data from the vision perception module, combines it with the robot's kinematic model, plans the motion trajectory, generates robot control commands, and coordinates the synchronization between visual positioning and robot actions in real time. The position feedback unit collects the robot's position information in real time and feeds it back to the motion control unit to achieve closed-loop correction between commands and actual positions.

[0031] The gripping execution module is used to contact and stably grip the workpiece, and transfer the workpiece to the designated position for placement without damage. The main structure of the gripping execution module is a fixture, which is designed and controlled based on the workpiece material, shape and surface features.

[0032] The system integration module ensures the safety, operability, and stability of the loading and unloading process. It includes a human-machine interaction unit, a safety protection unit, an external equipment collaboration unit, and a data storage and traceability unit. The human-machine interaction unit enables parameter configuration, status monitoring, manual operation, and fault alarms. The safety protection unit prevents the robot's movement from causing injury to operators and equipment, and assists in construction in accordance with industrial safety standards. The external equipment collaboration unit is responsible for coordinating the timing logic of various devices and is used in loading and unloading scenarios, where the robot's main module works in conjunction with other equipment on the production line. The data storage and traceability unit records production data, supporting quality traceability and production optimization.

[0033] Example 1: To better understand the above technical solution, the following will provide a detailed description of the technical solution in conjunction with the accompanying drawings and specific implementation methods. (Refer to...) Figs. 1-2 As shown in this embodiment of the present application, a robot vision-guided grasping control system includes the following grasping control method: S1. After the system issues the start command, the image acquisition device of the vision perception module acquires images of the workpiece placed in the area to be grasped according to the preset acquisition frequency and resolution, and obtains clear and complete workpiece images.

[0034] In this embodiment, the visual perception module calculates the precise coordinate parameters of the workpiece in the spatial coordinate system based on three-dimensional visual positioning technology. Specifically, it includes: a light source, which senses the dynamic changes of ambient light in real time, overcomes ambient light interference, highlights the edges, holes, and texture features of the workpiece, and ensures image stability; an image acquisition unit, which consists of an industrial camera, lens, and image acquisition card, whose core task is to convert the optical information of the target object or scene illuminated by the light source into two-dimensional images or point cloud data that can be processed later; a visual processing unit, which runs visual algorithms to complete image preprocessing, feature extraction, coordinate calculation, and data output; and a pose estimation unit, which extracts the workpiece image features based on the image information processed by the visual processing unit and matches them with features in a preset template or database to determine the type and approximate position of the workpiece. The pose estimation unit integrates multiple algorithms, including SIFT, SURF, PnP, gradient descent, and Levenberg-Marquardt algorithms, and calculates the dynamic situation of the workpiece and the grasping in real time to quickly analyze the difference between the current grasping posture and the preset ideal posture.

[0035] In this embodiment, the light source possesses adaptive lighting and reflection suppression technology. Specifically, based on sensor technology, when the ambient light increases, the resistance of the photoresistor decreases accordingly, and vice versa. Through the change in resistance, the sensor converts the ambient light intensity information into an electrical signal. Based on signal analysis and processing, it determines whether the current ambient light meets the requirements for image acquisition. In the lampshade or lens of the light source, a material and structural design with diffuse reflection characteristics are adopted, scattering the light emitted by the light source, making the light more uniform and softer to illuminate the workpiece surface, and reducing concentrated reflection of light. Most of the reflections on the surfaces of metal parts and smooth workpieces are polarized light. Adding a polarizing filter in front of the light source or camera lens can selectively filter out polarized reflected light. By performing algorithmic processing on the acquired image, the influence of reflection is further weakened. Based on deep learning image enhancement algorithms, reflective areas in the image are identified and analyzed, and the brightness, contrast, and color saturation parameters of the area are adjusted accordingly to make the image information of the reflective area clearer and restore the true features of the workpiece surface.

[0036] In practical implementation, a deep learning-based image enhancement algorithm is used to identify and analyze reflective areas in the image. Then, parameters such as brightness, contrast, and color saturation of these areas are adjusted to make the image information of the reflective areas clearer and restore the true features of the workpiece surface. In the inspection of metal parts, the algorithm can automatically identify and correct overly bright areas caused by reflection, making scratches, dents, and other defects on the part surface clearly visible.

[0037] S2, the visual perception module performs feature extraction processing on the acquired workpiece image. Through edge detection, contour recognition and other algorithms, it extracts the key feature points of the workpiece, calculates the precise spatial coordinates of the workpiece in the entire system coordinate system, and transmits the coordinate data to the motion control unit of the robot main module in real time.

[0038] In practice, the visual perception module first receives the original image of the workpiece captured by a camera or other device. It usually performs preprocessing such as noise reduction, grayscale conversion, and contrast enhancement to eliminate interference for subsequent feature extraction. The algorithm identifies the grayscale difference boundaries between the workpiece and the background, as well as between different parts of the workpiece, to determine the basic shape and contour of the workpiece. Based on edge detection, the discrete edge points are connected into a complete workpiece contour through contour tracking, fitting, and other algorithms. Key feature points are further selected. Through camera calibration, the pixel coordinates of the extracted feature points are converted into three-dimensional physical coordinates in the system coordinate system. The calculated precise coordinates are transmitted to the motion control unit of the robot's main module in a low-latency manner, directly serving as the target position instructions for the robot's grasping, assembly, and positioning actions.

[0039] S3. The visual perception module analyzes the current placement posture of the workpiece based on its spatial coordinates and shape characteristics, compares it with the preset optimal grasping posture, calculates the posture deviation value, generates posture correction instructions based on the deviation value, and transmits them to the motion control unit to provide a basis for the robot to adjust the grasping position and posture in the future.

[0040] In this embodiment, the Levenberg-Marquardt algorithm is used to optimize pose parameters, achieving high-precision pose estimation by minimizing the difference between visual measurements and theoretical values; the feature points of the workpiece in three-dimensional space are... In the coordinate system of the robot's end effector, the projected coordinates of the feature points are: The camera intrinsic parameter matrix is ​​K; the projection relationship of feature points from the world coordinate system to the image coordinate system is as follows:

[0041] in, Let be the scale factor. The Levenberg-Marquardt algorithm aims to minimize the sum of squares of reprojection errors, calculated as follows:

[0042] in, It is the theoretical projected coordinates calculated from the pose parameter x, i.e., the error function. The total error is .

[0043] It should be noted that by adaptively adjusting the damping coefficient, it is suitable for complex scenarios when the robot arm is grasping. By iteratively optimizing the reprojection error, the attitude estimation error can be controlled at the millimeter or even sub-millimeter level, meeting the requirements of precision grasping. Compared with the ICP algorithm, the Levenberg-Marquardt algorithm converges faster, usually within 5-20 iterations, which can meet the frame rate requirements of real-time control of the robot arm. The Levenberg-Marquardt algorithm achieves high-precision optimization of the robot arm's grasping attitude by constructing an error function, solving a damped linear equation system, and adaptively adjusting the iteration step size, ensuring that the robot arm's end effector and the workpiece's attitude are accurately matched.

[0044] S4. After receiving the workpiece spatial coordinates and posture correction instructions from the vision perception module, the motion control unit of the robot main module plans the robot's motion trajectory from the current initial position to the workpiece grasping position by combining its own pre-stored robot kinematic model, ensuring that the robot's motion process is stable and efficient, while avoiding collisions with surrounding equipment or workpieces.

[0045] It should be noted that the performance of the motion control unit directly affects the stability, accuracy, and efficiency of the robot's operation. Through precise calculations of forward and inverse kinematics, the error between the actual position of the end effector and the visually perceived target coordinates is controlled within 0.1mm-1mm, meeting the requirements of precision assembly. By optimizing trajectory planning, the motion path is shortened while avoiding collisions. For example, a traditional fixed trajectory requires 10 seconds to complete a grasp, which can be shortened to 6 seconds after optimization, significantly improving the production line cycle time. Smooth trajectory planning reduces the impact of joint start-stop, extending the service life of the robotic arm, while the collision detection mechanism avoids equipment maintenance costs caused by misoperation.

[0046] In practice, the motion control unit embeds collision detection logic simultaneously when planning the trajectory. This is crucial for preventing equipment damage and personnel injury in industrial scenarios. It is mainly achieved in two ways: static environment pre-storage, where the system pre-stores static models of the work scene, such as the position of the conveyor belt, the size and coordinates of surrounding equipment, and the placement area of ​​other fixed workpieces. When planning the trajectory, it will automatically avoid these areas. For example, if the trajectory passes through the coordinate range of the conveyor belt support, the system will automatically adjust the path to avoid the support by more than 5cm. Dynamic real-time monitoring, some high-end systems will combine sensors to monitor the motion process in real time. If the robot suddenly touches an unexpected object while moving, the force sensor will detect abnormal resistance, and the motion control unit will immediately trigger an emergency stop or retraction command to avoid hard collisions that could cause deformation of the robotic arm or damage to the workpiece.

[0047] S5. The motion control unit generates corresponding robot control commands based on the planned motion trajectory and sends the commands to the motion unit of the robot's main module. This drives the robot's joints to move according to the commands, moving the robot's end effector towards the workpiece's gripping position. During the movement, the position feedback unit collects the actual position information of the robot's end effector in real time and feeds it back to the motion control unit.

[0048] In this embodiment, the position feedback unit collects the robot's actual position and posture in real time through encoders, visual odometry, and high-precision LiDAR sensors to form closed-loop control. Specifically, this includes joint feedback, where the encoder of each joint directly measures the angle. , and the angle of the instruction The error is The error signal is sent to the motion control unit, where the control command is corrected via PID control; global position feedback uses visual markers to obtain the robot's position in the world coordinate system. , and target location The deviation is used to correct the path, and its formula is: .

[0049] It should be noted that the position feedback unit collects the actual position of the end effector in real time to provide a basis for correcting deviations. The core of the position feedback is the position sensor, and common types include encoders, laser rangefinders, and vision sensors. The actual position data collected by the sensor, such as the actual angle of the joint and the actual coordinates of the end effector, will be transmitted back to the motion control unit in real time through a high-speed communication bus. The transmission delay is usually controlled at the millisecond level to ensure that the feedback data is synchronized with the actual movement.

[0050] In practical implementation, even if the robot experiences joint wear and motor step loss due to long-term use, closed-loop control can correct the position deviation in real time and keep it within the accuracy range, preventing a decrease in operational accuracy due to mechanical aging. If the robot encounters slight external interference during movement, such as airflow blowing the end effector or slight vibration of the conveyor belt, the position feedback can immediately detect the deviation, and the motion control unit can quickly adjust the instructions to prevent the end effector from going off-track. In mass production, such as when multiple workpieces are gripped per hour, closed-loop control can ensure that the gripping position and movement speed of each workpiece are completely consistent, avoiding product quality instability caused by individual differences, such as uneven gripping force leading to part deformation.

[0051] S6. The motion control unit compares the actual position information collected by the position feedback unit with the planned theoretical position information in real time, calculates the position deviation, adjusts the motion parameters of the moving unit, realizes closed-loop correction between the command and the actual position, and ensures that the robot end effector can accurately reach the workpiece gripping position to meet the gripping accuracy requirements.

[0052] In this embodiment, the collaborative logic of the motion control unit, the movement unit, and the position feedback unit is as follows: the visual perception module outputs the workpiece coordinates. The motion control unit will The robot's position is converted into a target pose, its trajectory is planned, and control commands are generated. The mobilization unit executes the commands, driving the robot to move towards the target. The position feedback unit collects the actual position in real time. The error is calculated and fed back to the motion control unit; the motion control unit performs closed-loop correction according to the error adjustment command until the target position is reached.

[0053] It should be noted that the closed-loop correction of the motion control unit is not a single action, but a continuous iterative dynamic process. Each step revolves around reducing position deviation, which can be divided into three key steps, forming a millisecond-level closed loop. First, the motion control unit simultaneously receives two types of core data: theoretical position information and actual position information, and performs real-time comparison at the millisecond level. After comparison, the unit calculates the position deviation value through a preset algorithm. During the calculation, attitude deviations, such as the deviation of the end effector rotation angle, are also taken into account to ensure that not only the position is correct, but the attitude is also correct. Second, the motion parameters are dynamically adjusted and corrected. This is the execution link of the closed-loop correction. Based on the calculated deviation value, the motion control unit adjusts the motion parameters sent to the moving unit in real time to specifically offset the deviation.

[0054] In industrial settings, closed-loop calibration can dynamically adjust to ensure the final grasping accuracy meets standards, offset inherent mechanical errors, resist external dynamic interference, and ensure consistency in batch operations when there are minor errors in the preceding stages or external interference occurs during operation.

[0055] S7. When the robot end effector reaches the preset gripping position, the motion control unit sends a gripping command to the gripping execution module. Combined with the previously configured fixture parameters, the control unit controls the fixture movement. After the fixture clamps the workpiece, the clamping status is confirmed by detection elements such as pressure sensors, and a gripping success signal is fed back to the motion control unit and the system integration module. The human-machine interaction unit of the system integration module displays the gripping success status in real time, while the data storage and traceability unit records data such as gripping time and clamping force.

[0056] In this embodiment, the gripping execution module includes: a pneumatic gripper, which drives a cylinder and a gripper with compressed air to achieve opening and closing actions; an electric gripper, which drives a gear and a lead screw with a servo motor and a stepper motor to achieve precise opening and closing; a vacuum suction cup gripper, which generates negative pressure through a vacuum pump and a vacuum generator to adsorb the surface of the workpiece to achieve gripping; and a customized gripper, which is a special gripper designed according to the special shape of the workpiece with irregular grooves and multiple protrusions, including a chuck and a claw.

[0057] In this embodiment, the gripping execution module has adaptive clamping force control capability, which monitors the clamping force in real time through a force sensor. , and preset threshold By comparing the results, a closed-loop control system can be formed.

[0058] in, The driving signal ensures that the clamping force is within a safe range.

[0059] In this embodiment, the gripping execution module includes gripping posture calibration features. Combined with the workpiece pose output by the vision perception module, the gripper needs to adjust its own posture before contacting the workpiece to ensure that it is consistent with the normal direction of the workpiece surface and avoid lateral slippage. According to the degrees of freedom of the robot's end effector, the posture parameters of the gripper are adjusted, specifically including: rotational adjustment, around three coordinate axes. rotation angle This aligns the fixture coordinate system with the workpiece surface coordinate system; fine-tuning involves making small translations along the normal direction and in the plane perpendicular to the normal before contacting the workpiece to ensure that the fixture is directly opposite and centered to the workpiece surface.

[0060] In practice, after the fixture clamps the workpiece, it does not immediately proceed to the next step. Instead, it needs to physically verify whether it is truly gripped securely. This relies on detection components such as pressure sensors to form a small closed loop of action → detection → feedback. The pressure sensor is usually integrated inside the fixture's jaws or in the power source circuit to collect the actual clamping force in real time. If the actual clamping force is within the target clamping force range, it is determined that the clamping is qualified and the workpiece has been stably gripped. If the actual clamping force is lower than the lower limit, it is determined that the clamping is not tight enough. If it is higher than the upper limit, it is determined that the clamping is over-clamped. Both situations will result in a gripping failure. After the detection is completed, the gripping execution module will send signals to the two core modules to achieve state synchronization. If the gripping is successful, the motion control unit will start the next action, such as moving the workpiece to the assembly station. If the gripping fails, it will trigger retry logic, such as releasing the fixture, readjusting the position, and then gripping again, or alarm logic, and send feedback to the system integration module to transmit the status signals of successful or failed gripping, providing a basis for subsequent human-machine interaction display and data recording.

[0061] It should be noted that the data storage and traceability unit automatically records all key data related to this capture. This data is the core basis for production quality traceability and equipment maintenance. Common recorded content includes time data, status data, and equipment data. This data will be stored in a local database or cloud system. If workpiece damage occurs later, or the capture success rate decreases, historical data can be retrieved by keywords such as capture time and fixture number to quickly locate the root cause of the problem.

[0062] S8. After receiving the successful gripping signal from the gripping execution module, the motion control unit restarts the trajectory planning algorithm based on the previously entered workpiece target placement position coordinates, combined with the robot's current position and the layout of surrounding equipment. It plans the robot's transfer trajectory from the gripping position to the target placement position, drives the robot to move the gripped workpiece to the target placement position, and the robot carries the workpiece to the target placement position.

[0063] In this embodiment, the gripping execution module transfers and places the workpiece, including: the transfer process, which plans the movement speed according to the workpiece quality and stability requirements, and realizes path tracking through the position feedback of the robot's main module; and the placement process, which adopts a soft landing strategy, reduces the speed when contacting the target surface, and triggers a release action through force feedback.

[0064] It should be noted that the motion control unit calls the pre-stored work scene environment model, which includes the fixed coordinates and dynamic range of surrounding equipment, conveyor belts, and other workpieces. When planning the trajectory, it automatically avoids the above-mentioned areas. Common constraint scenarios include: static obstacle avoidance, such as when there is a fixed equipment support in the transfer path, the trajectory will automatically bypass the area to ensure a safe distance between the robotic arm and the support; dynamic obstacle avoidance, if there is a mobile device in the scene, the unit will receive its position information in real time through sensors and dynamically adjust the trajectory. By optimizing the trajectory planning, the transfer time can be shortened while avoiding collisions, thereby improving the overall cycle time of the production line. Deviation correction during the transfer process can ensure the accuracy of the final placement position of the workpiece and provide a stable reference position for subsequent processes.

[0065] Example 2: To better understand the above technical solution, the following will provide a detailed explanation of the specific implementation method using actual examples. To provide efficient and accurate solutions for various practical application scenarios, a robot vision-guided grasping control system is proposed. The grasping control system includes: The visual perception module is used to acquire workpiece images, extract workpiece feature information, calculate the precise coordinates of the workpiece in space, and quickly estimate and correct the grasping posture based on the acquired workpiece information, providing a basis for accurate material handling.

[0066] The visual perception module calculates the precise coordinate parameters of the workpiece in the spatial coordinate system based on 3D visual positioning technology. Specifically, it includes: a light source that senses the dynamic changes of ambient light in real time, overcomes ambient light interference, highlights workpiece edges, holes, and texture features, and ensures image stability; an image acquisition unit composed of an industrial camera, lens, and image acquisition card, whose core task is to convert the optical information of the target object or scene illuminated by the light source into a 2D image or point cloud data for subsequent processing; a visual processing unit that runs visual algorithms to complete image preprocessing, feature extraction, coordinate calculation, and data output; and a pose estimation unit that, based on the image information processed by the visual processing unit, extracts workpiece image features and matches them with features from a preset template or database to determine the type and approximate location of the workpiece. The pose estimation unit integrates multiple algorithms, including SIFT, SURF, PnP, gradient descent, and Levenberg-Marquardt algorithms, to quickly analyze the difference between the current grasping posture and the preset ideal posture by calculating the dynamic situation of the workpiece and the grasping process in real time.

[0067] The visual perception module is equipped with a high-resolution industrial camera, a customized optical lens, and a supplementary lighting system. It adopts differentiated acquisition strategies for different scenarios: in static scenarios, it uses trigger-based acquisition, which triggers the camera to capture images when the sensor detects that the workpiece has reached the designated area, ensuring that the images are free of motion blur; in dynamic scenarios, it uses high-speed continuous shooting and frame synchronization technology, combined with a conveyor belt speed compensation algorithm, to select clear and complete workpiece images from continuous frames.

[0068] The main robot module is used to receive system commands to drive the carrier to move to the target position. It includes a motion control unit, a movement unit, and a position feedback unit. The motion control unit receives workpiece coordinate data from the vision perception module, combines it with the robot's kinematic model, plans the motion trajectory, generates robot control commands, and coordinates the synchronization between visual positioning and robot actions in real time. The position feedback unit collects the robot's position information in real time and feeds it back to the motion control unit to achieve closed-loop correction between commands and actual positions.

[0069] Based on the starting and target positions output by the vision module, the main robot module uses joint space trajectory planning or Cartesian space trajectory planning algorithms to generate smooth motion trajectories. For example, in precision grasping scenarios, polynomial interpolation trajectory planning is used to ensure that the robot's speed and acceleration change continuously, avoiding workpiece displacement caused by impact. In high-speed grasping scenarios, trapezoidal velocity planning is used to achieve rapid start-up → uniform speed operation → smooth deceleration, balancing efficiency and stability.

[0070] By combining the robot's kinematic model, the target coordinates in Cartesian space are converted into angle commands for each joint and sent to the servo driver of the mobile unit via pulse signals or bus communication. Due to the slight delay in visual acquisition and data transmission, the motion control unit needs to use timestamp synchronization technology to associate the coordinate data of the vision module with the robot's movement time to avoid deviations where the robot still moves according to the old coordinates even after the visual coordinates have been updated. For example, in dynamic grasping scenarios, the motion control unit will calculate the displacement of the workpiece during the command transmission delay based on the timestamp of the image acquired by the vision module and correct the target coordinates in real time to achieve dynamic tracking and grasping.

[0071] The gripping execution module is used to contact and stably grip the workpiece, and transfer the workpiece to the designated position for placement without damage. The main structure of the gripping execution module is a fixture, which is designed and controlled based on the workpiece material, shape and surface features.

[0072] Based on the workpiece's material, shape, weight, and surface characteristics, a suitable fixture structure and control logic are designed to ensure stable gripping and damage-free placement. Miniature pressure sensors are installed at the grippers or suction cups to monitor the gripping and suction forces in real time. When the force value is abnormal, a signal is immediately sent to the system to stop the gripping action, preventing workpiece damage or drop. In some high-precision scenarios, the fixture is equipped with a micro-displacement platform, which can make minor adjustments to the workpiece's posture after gripping to ensure the accuracy of subsequent assembly. For workpieces that are prone to slipping, such as cylindrical parts, the fixture can be designed with anti-slip textures or multiple suction cup arrays to improve gripping stability.

[0073] The system integration module ensures the safety, operability, and stability of the loading and unloading process. It includes a human-machine interaction unit, a safety protection unit, an external equipment collaboration unit, and a data storage and traceability unit. The human-machine interaction unit enables parameter configuration, status monitoring, manual operation, and fault alarms. The safety protection unit prevents the robot's movement from causing injury to operators and equipment, and assists in construction in accordance with industrial safety standards. The external equipment collaboration unit is responsible for coordinating the timing logic of various devices and is used in loading and unloading scenarios, where the robot's main module works in conjunction with other equipment on the production line. The data storage and traceability unit records production data, supporting quality traceability and production optimization.

[0074] The human-machine interface provides a visual operation interface, allowing operators to set core parameters and display the real-time operating status of each module of the system. It adopts a graphical and color-coded warning method and supports a manual debugging mode. Operators can manually control the movement of each joint of the robot and the opening and closing of the gripper through the handle or interface buttons for system installation and debugging or workpiece handling in emergencies. When the system malfunctions, such as visual recognition failure, excessive robot position deviation, or safety door opening, the unit will remind the operator through audible and visual alarms and push the fault information to the administrator's mobile APP.

[0075] The safety protection unit implements multi-level protection through hardware protection and software logic. A safety light curtain is set in the robot's working area. When a person enters the danger zone, the light curtain triggers a signal, and the robot immediately slows down to 25%. If the person continues to enter, an emergency stop is triggered. A safe zone is set through the robot controller. If a program error causes the robot to exceed the safe zone, the system automatically cuts off the power supply. An emergency stop button and a manual release button are set on the operation panel. When the robot jams, the operator can press the emergency stop button and then use the manual release button to open the clamp and remove the workpiece, avoiding equipment damage caused by forced disassembly.

[0076] The external device collaboration unit communicates with other equipment on the production line through the industrial bus to solve the problem of low efficiency caused by the independent operation of each device. Taking the logistics sorting scenario as an example, the system needs to coordinate the actions of the conveyor belt, vision camera, robot, and sorting chute. The conveyor belt transports the package to the camera recognition area → the camera recognizes the package type and sends coordinates → the robot grabs the package according to the coordinates → the sorting chute opens the corresponding channel according to the package type. The timing deviation of the entire process is ≤100ms. When a device fails, the unit immediately sends a signal to the robot to pause the grabbing action and at the same time notifies the upstream equipment to stop feeding to avoid the accumulation of workpieces.

[0077] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A robot vision-guided grasping control system, characterized in that, The grasping control system includes: The visual perception module is used to acquire workpiece images, extract workpiece feature information, calculate the precise coordinates of the workpiece in space, and quickly estimate and correct the grasping posture based on the acquired workpiece information, providing a basis for accurate material handling. The main robot module is used to receive system commands to drive the carrier to move to the target position. It includes a motion control unit, a movement unit, and a position feedback unit. The motion control unit receives workpiece coordinate data from the vision perception module, combines it with the robot's kinematic model, plans the motion trajectory, generates robot control commands, and coordinates the synchronization between visual positioning and robot actions in real time. The position feedback unit collects the position information of the robot's movement in real time and feeds it back to the motion control unit to achieve closed-loop correction between commands and actual positions. The gripping execution module is used to contact and stably grip the workpiece and transfer it to a designated position for placement without damage. The main structure of the gripping execution module is a fixture, which is designed and controlled based on the workpiece's material, shape, and surface features. The system integration module ensures the safety, operability, and stability of the loading and unloading process. It includes a human-machine interaction unit, a safety protection unit, an external equipment collaboration unit, and a data storage and traceability unit. The human-machine interaction unit enables parameter configuration, status monitoring, manual operation, and fault alarms. The safety protection unit prevents the robot's movement from causing injury to operators and equipment, and assists in construction in accordance with industrial safety standards. The external equipment collaboration unit is responsible for coordinating the timing logic of various devices and is used in loading and unloading scenarios, where the robot's main module works in conjunction with other equipment on the production line. The data storage and traceability unit records production data, supporting quality traceability and production optimization.

2. The robot vision-guided grasping control system according to claim 1, characterized in that: The visual perception module calculates the precise coordinate parameters of the workpiece in the spatial coordinate system based on three-dimensional visual positioning technology, specifically including: The light source senses the dynamic changes of ambient light in real time, overcomes ambient light interference, highlights the edges, holes, and texture features of the workpiece, and ensures image stability. The image acquisition unit consists of an industrial camera, a lens, and an image acquisition card. Its core task is to convert the optical information of the target object or scene illuminated by the light source into two-dimensional images or point cloud data that can be processed later. The visual processing unit runs visual algorithms to complete image preprocessing, feature extraction, coordinate calculation, and data output. The pose estimation unit extracts workpiece image features based on the image information processed by the vision processing unit and matches them with features in a preset template or database to determine the type and approximate position of the workpiece. The pose estimation unit integrates multiple algorithms, including SIFT, SURF, PnP, gradient descent, and Levenberg-Marquardt algorithms. By calculating the dynamic situation of the workpiece and the grasping in real time, it quickly analyzes the difference between the current grasping posture and the preset ideal posture.

3. A robot vision-guided grasping control system according to claim 2, characterized in that: The light source features adaptive illumination and reflection suppression technology, specifically: Based on sensor technology, when the ambient light increases, the resistance of the photoresistor will decrease accordingly, and vice versa. By changing the resistance, the sensor converts the ambient light intensity information into an electrical signal. Based on the analysis and processing of the signal, it can determine whether the current ambient light meets the requirements for image acquisition. In the lampshade or lens part of the light source, materials and structural designs with diffuse reflection characteristics are used to scatter the light emitted by the light source, so that the light shines more evenly and softly on the surface of the workpiece, reducing concentrated reflection of light. Most of the reflections on the surfaces of metal parts and smooth workpieces are polarized light. By adding a polarizing filter in front of the light source or camera lens, polarized reflected light can be selectively filtered out. By processing the acquired images using algorithms, the impact of reflections is further reduced. Based on deep learning, the image enhancement algorithm identifies and analyzes the reflective areas in the image, and adjusts the brightness, contrast, and color saturation parameters of the area accordingly, making the image information of the reflective areas clearer and restoring the true features of the workpiece surface.

4. A robot vision-guided grasping control system according to claim 2, characterized in that: The Levenberg-Marquardt algorithm is used to optimize pose parameters and achieve high-precision pose estimation by minimizing the difference between visual measurements and theoretical values. The feature points of the workpiece in three-dimensional space are ; In the coordinate system of the robot's end effector, the projected coordinates of the feature point are: ; The camera intrinsic parameter matrix is ​​K; The projection relationship of feature points from the world coordinate system to the image coordinate system is as follows: ； in, Let be the scale factor. The Levenberg-Marquardt algorithm aims to minimize the sum of squares of reprojection errors, calculated as follows: ； in, It is the theoretical projected coordinates calculated from the pose parameter x, i.e., the error function. The total error is .

5. A robot vision-guided grasping control system according to claim 1, characterized in that: The position feedback unit collects the robot's actual position and attitude in real time through encoders, visual odometry, and high-precision LiDAR sensors to form closed-loop control, specifically including: Joint feedback: the encoder for each joint directly measures the angle. , and the angle of the instruction The error is The error signal is sent to the motion control unit, and the control command is corrected by PID adjustment. Global position feedback utilizes visual markers to obtain the robot's position in the world coordinate system. , and target location The deviation is used to correct the path, and its formula is: .

6. A robot vision-guided grasping control system according to claim 5, characterized in that: The specific collaborative logic of the motion control unit, the movement unit, and the position feedback unit is as follows: The vision perception module outputs workpiece coordinates. ; The motion control unit will Convert the robot's target pose, plan the trajectory, and generate control commands; The mobile unit executes commands, driving the robot to move towards the target; The position feedback unit collects the actual position in real time. The error is calculated and fed back to the motion control unit; The motion control unit performs closed-loop correction based on error adjustment commands until the target position is reached.

7. A robot vision-guided grasping control system according to claim 1, characterized in that: The gripper in the grasping and execution module includes: Pneumatic grippers use compressed air to drive cylinders and grippers to achieve opening and closing actions; Electric clamps achieve precise opening and closing by driving gears and lead screws with servo motors and stepper motors; Vacuum suction cup clamps use a vacuum pump or vacuum generator to create negative pressure, which adsorbs the surface of the workpiece to achieve gripping. Customized fixtures are designed specifically for workpieces with irregular grooves or multiple protrusions, including chucks and jaws.

8. A robot vision-guided grasping control system according to claim 7, characterized in that: The gripping execution module has adaptive clamping force control capability, which monitors the clamping force in real time through a force sensor. , and preset threshold By comparing the results, a closed-loop control system can be formed. ； in, The driving signal ensures that the clamping force is within a safe range.

9. A robot vision-guided grasping control system according to claim 8, characterized in that: The gripping execution module includes gripping posture calibration features. Combined with the workpiece pose output by the vision perception module, the gripper needs to adjust its own posture before contacting the workpiece to ensure it is aligned with the normal direction of the workpiece surface and avoid lateral slippage. Based on the degrees of freedom of the robot's end effector, the gripper's posture parameters are adjusted, specifically including: Rotation adjustment, around three coordinate axes rotation angle This aligns the fixture coordinate system with the workpiece surface coordinate system. Before contacting the workpiece, make a small translation along the normal direction and in the plane perpendicular to the normal to ensure that the fixture is aligned with and centered on the workpiece surface.

10. A robot vision-guided grasping control system according to claim 9, characterized in that: The grasping and execution module transfers and places the workpiece, including: During the transfer process, the movement speed is planned according to the workpiece quality and stability requirements, and path tracking is achieved through the position feedback of the robot's main module; During placement, a soft landing strategy is employed, reducing speed upon contact with the target surface and triggering a release action through force feedback.

Images