Pneumatic robot control method and system based on visual feedback

Abstract

The invention discloses a pneumatic robot control method and system based on visual feedback, and belongs to the field of pneumatic servo control. Visual information is introduced into a control closed loop, real-time accurate feedback of the tail end space position of the pneumatic robot is achieved, the complementary advantage is formed by combining the high-frequency characteristic of the displacement sensor, and therefore the trajectory tracking precision is improved while modeling errors are restrained. In addition, the system adaptability and the anti-interference capability are effectively improved by adopting an auxiliary control method of an uncalibrated visual identification point. The method is suitable for the field of pneumatic servo control, and realizes efficient and accurate control on a pneumatic driving robot.

Description

Technical Field

[0001] This invention relates to a pneumatic robot control method and system based on visual feedback, belonging to the field of pneumatic servo control. Background Technology

[0002] Traditional robot control systems often employ a single feedback architecture based on displacement sensors. While these sensors can provide high-precision local position feedback, their spatial position calculation is highly dependent on the mechanism's kinematic model, and the modeling accuracy is easily affected by factors such as mechanism deformation, assembly errors, and environmental vibrations. In pneumatic drive systems, the nonlinear dynamic response problem caused by the compressibility of gas further amplifies the positional uncertainty of the end effector, leading to reduced accuracy.

[0003] Visual technology can directly acquire the absolute pose information between a robot's end effector and its environment through non-contact measurement methods, and reconstruct the three-dimensional spatial mapping relationship. Integrating a visual recognition system with a robot control system can significantly improve the control accuracy of the robot's end effector. However, existing visual servoing technologies still have significant drawbacks: First, the delays introduced by image acquisition, processing, and coordinate calculation processes lead to phase lag in the control loop, making it difficult to meet the requirements of high-speed and high-dynamic response; second, visual systems are sensitive to changes in lighting, target occlusion, and image noise, resulting in a significant decrease in positioning stability in low-contrast or complex background scenes; third, monocular cameras, due to the lack of depth information, require multi-camera fusion or complex calibration processes, greatly increasing system complexity and deployment costs.

[0004] To address the aforementioned issues, employing a control method assisted by uncalibrated visual markers can effectively improve system adaptability and anti-interference capabilities. By introducing visual information into the control closed loop, real-time and accurate feedback of the robot's end effector spatial position can be achieved. This, combined with the high-frequency characteristics of displacement sensors, creates complementary advantages, thereby suppressing modeling errors while improving trajectory tracking accuracy. This dual-feedback fusion mechanism provides a new approach to overcoming the bottleneck of high-precision control for pneumatic robots. Summary of the Invention

[0005] The purpose of this invention is to address the problem of large end-effector errors in traditional pneumatic robot control, which relies solely on displacement sensors for displacement feedback and obtains end-effector trajectories through kinematic calculations. This invention provides a visual feedback-based pneumatic robot control method and system. By introducing visual information into the control closed loop, this invention achieves real-time and accurate feedback of the robot's end-effector spatial position. Combined with the high-frequency characteristics of displacement sensors, it forms complementary advantages, thereby improving trajectory tracking accuracy while suppressing modeling errors. Furthermore, the use of an auxiliary control method without calibration visual markers effectively enhances the system's adaptability and anti-interference capabilities. This invention is applicable to the field of pneumatic servo control, enabling efficient and precise control of pneumatically driven robots.

[0006] The objective of this invention is achieved through the following technical solution:

[0007] A pneumatic robot control method and system based on visual feedback includes the following steps:

[0008] Step 1: Perform image acquisition from a single camera, binarize and preprocess the acquired images using Gaussian filtering, and then obtain the camera image coordinates of the visual markers carried by the robot's end effector using a geometric feature matching method. ;

[0009] Step 2: Convert the camera image coordinates of the visual markers obtained in Step 1. With three-dimensional spatial position Substitute into equation (1) to calculate the camera's visual projection matrix. To obtain the three-dimensional spatial position With camera image coordinates The mapping relationship;

[0010] Step 3: For multiple groups and Centroid normalization was performed to obtain multiple sets of normalized camera image coordinates for visual marker points. and the corresponding three-dimensional spatial position Substitute into the objective function based on the Huber loss function Further optimization of the visual projection matrix ;

[0011] Step 4: Use at least two cameras to observe from different perspectives to obtain the three-dimensional spatial position of the visual marker point. This allows us to further obtain the three-dimensional spatial position of the robot's end effector;

[0012] Step 5: The three-dimensional spatial position of the robot end effector obtained in Step 4 is used as feedback information, becoming visual feedback; the visual feedback and the displacement sensor information together constitute dual feedback control; the entire robot control is a dual-loop control, the inner loop is a motion control closed loop using displacement sensor information; the outer loop is a closed loop using visual feedback, providing high-precision robot target position information for the inner loop control.

[0013] The three-dimensional spatial position described in step two With camera image coordinates The mapping relationship;

[0014] (1)

[0015] in, Scale factor;

[0016] The scale factor in equation (1) Elimination yields

[0017] (2)

[0018] The third step is to implement the following method:

[0019] According to equation (2), multiple sets of three-dimensional spatial positions can be selected. and camera image coordinates This constitutes an over-constrained system;

[0020] To improve the three-dimensional spatial position of visual markers and camera image coordinates Stability, for multiple groups and Perform centroid normalization;

[0021] , ,

[0022] ,

[0023] in, This represents the total number of visual marker locations across the entire workspace.

[0024] 3D spatial position of visual markers and camera image coordinates The normalized coordinates are

[0025] , ,

[0026] ,

[0027] Camera image coordinates after normalization of multiple sets of visual markers and the corresponding three-dimensional spatial position The visual projection matrix is ​​optimized using equation (3). ;

[0028] (3)

[0029] in, The objective function is based on the Huber loss function. The Huber loss function is expressed as follows:

[0030]

[0031] in, For the threshold, Denotes the Euclidean norm. Indicates the first The visual marker point at the th Normalized coordinates at each position, This represents the total number of visual markers detected at a given location.

[0032] The implementation method for step four is as follows:

[0033] The value of a single camera is obtained through equation (3). Multiple cameras are used to observe from different perspectives, and the projection equations of each camera provide two independent constraints, namely... and Orientation error: The two cameras provide four constraints, which are sufficient to solve for the three-dimensional spatial position of the visual markers. ;

[0034] The calculated visual projection matrix After performing inverse normalization, we get:

[0035]

[0036] Establish the objective function (4) for minimizing the 3D point reprojection under multi-camera observation, and obtain the 3D position of the visual marker points carried by the robot end effector through equation (4):

[0037] (4)

[0038] in, The total number of cameras observing simultaneously ( ), For the first The image coordinates of the visual markers carried by the robot's end effector in the camera. For the first The inverse normalized visual projection matrix of the camera;

[0039] The three-dimensional spatial position of the robot's end effector is then obtained.

[0040] Step five is implemented as follows: the entire robot control is a dual-loop control. The vision closed loop provides high-precision target position information, thereby enabling the robot motion control closed loop to achieve high-precision robot control, forming a vision-motion dual-loop robot control. The motion control closed loop adopts composite active disturbance rejection control to classify and suppress disturbances received by the pneumatic robot system: a disturbance observer is designed to estimate time-varying load force disturbances that have a significant impact on the system in real time, such as equivalent driving force disturbances caused by load changes, and to perform dynamic compensation; at the same time, other unmodeled dynamics and system uncertainties are regarded as total disturbances, and an extended state observer is used to observe and dynamically compensate for them in real time, realizing disturbance classification observation and cooperative suppression.

[0041] The number of groups mentioned in step three is 6 or more.

[0042] The number of cameras mentioned in step four is at least two.

[0043] Step 3 optimizes the visual projection matrix The process employs an objective function based on the Huber loss function. .

[0044] Beneficial effects:

[0045] 1. This invention establishes a mapping relationship between camera space and actual space through a visual projection matrix, thereby achieving accurate estimation of the end-point three-dimensional spatial position under uncalibrated conditions.

[0046] 2. By introducing visual feedback information, this invention compensates for the error in the spatial position of the robot end effector calculated by the displacement sensor, thereby improving the spatial trajectory accuracy of the robot end effector. Attached Figure Description

[0047] Figure 1 This is a schematic diagram of a vision-based dual-feedback closed-loop control system.

[0048] Figure 2 It is a vision-based dual-feedback closed-loop control block diagram;

[0049] Figure label:

[0050] 1-Air source, 2-Proportional valve, 3-Proportional valve, 4-Proportional valve, 5-Cylinder, 6-Cylinder, 7-Cylinder, 8-Displacement sensor, 9-Displacement sensor, 10-Displacement sensor, 11-Robot body, 12-Industrial camera, 13-Industrial camera, 14-Controller, 15-PC. Detailed Implementation

[0051] The present invention will now be described in detail with reference to the accompanying drawings and embodiments. The technical problems solved by the present invention and its beneficial effects are also described. It should be noted that the described examples are only intended to facilitate understanding of the present invention and do not constitute any limitation thereof.

[0052] like Figure 1As shown, the vision-based dual-feedback closed-loop control system for a pneumatic robot comprises a robot body 11, an air source 1, proportional valves 2, 3, and 4, cylinders 5, 6, and 7, displacement sensors 8, 9, and 10, an industrial camera 12 and 13, a controller 14, and a PC 15. The displacement sensors are mounted on the cylinders, the industrial cameras are connected to the PC, and both the displacement sensors and the PC are connected to the controller. The controller's output signal controls the proportional valves, which are connected to the cylinders.

[0053] The gas source provides a gas pressure of 0.6 MPa; the displacement sensor has a range of 200 mm and outputs a voltage signal of 0-10 V; the proportional valve has a range of 0-10 V; and the controller has AD and DA functions.

[0054] The operation process of a robot control system employing vision-based dual-feedback closed-loop control is described as follows: During movement, the pneumatic robot collects the extension and retraction displacement of its actuator cylinders via displacement sensors and feeds this cylinder displacement signal back to the controller. Simultaneously, a dual-camera vision system acquires images of visual markers carried by the robot's end effector. After image processing processes such as image preprocessing and target recognition, the target's image coordinates are obtained. The three-dimensional spatial coordinates of the end effector are then optimized and calculated using a visual projection matrix. This visual feedback information is fed back to the controller. Finally, a control algorithm that fuses the information from both sensors controls the movement of the pneumatic robot's cylinders, forming a dual-feedback control process.

[0055] The specific steps for implementing the vision-based dual-feedback closed-loop control method are as follows:

[0056] A pneumatic robot control method based on visual feedback includes the following steps:

[0057] Step 1: Perform image acquisition from a single camera, binarize and preprocess the acquired images using Gaussian filtering, and then obtain the camera image coordinates of the visual markers carried by the robot's end effector using a geometric feature matching method. ;

[0058] Step 2: Convert the camera image coordinates of the visual markers obtained in Step 1. With three-dimensional spatial position Substitute into equation (1) to calculate the camera's visual projection matrix. To obtain the three-dimensional spatial position With camera image coordinates Mapping relationship:

[0059] (1)

[0060] in, Scale factor;

[0061] The scale factor in equation (1) Elimination yields

[0062] (2)

[0063] Step 3: As can be seen from equation (2), the three-dimensional spatial positions of each group are... and camera image coordinates Two independent equations can be provided. Visual projection matrix. It contains 12 unknown parameters and requires at least 6 sets of three-dimensional spatial positions. and camera image coordinates Only then can the visual projection matrix be determined. However, a small amount of data is insufficient to accurately establish the mapping relationship of the visual markers carried by the robot's end effector across the entire workspace. For multiple sets... and Centroid normalization was performed to obtain multiple sets of normalized camera image coordinates for visual marker points. and the corresponding three-dimensional spatial position Substitute into the objective function based on the Huber loss function Further optimization of the visual projection matrix ;

[0064] According to equation (2), multiple sets of three-dimensional spatial positions can be selected. and camera image coordinates This constitutes an over-constrained system;

[0065] To improve the three-dimensional spatial position of visual markers and camera image coordinates Stability, for multiple groups and Perform centroid normalization;

[0066] , ,

[0067] ,

[0068] in, This represents the total number of visual marker locations across the entire workspace.

[0069] 3D spatial position of visual markers and camera image coordinates The normalized coordinates are

[0070] , ,

[0071] ,

[0072] Camera image coordinates after normalization of multiple sets of visual markers and the corresponding three-dimensional spatial position The visual projection matrix is ​​optimized using equation (3). ;

[0073] (3)

[0074] in, The objective function is based on the Huber loss function. The Huber loss function is expressed as follows:

[0075]

[0076]

[0077] in, For the threshold, Denotes the Euclidean norm. Indicates the first The visual marker point at the th Normalized coordinates at each position, This represents the total number of visual markers detected at a given location.

[0078] Step 4: Use at least two cameras to observe from different perspectives to obtain the three-dimensional spatial position of the visual marker point. This allows us to further obtain the three-dimensional spatial position of the robot's end effector;

[0079] The value of a single camera is obtained through equation (3). Multiple cameras are used to observe from different perspectives, and the projection equations of each camera provide two independent constraints, namely... and Orientation error: The two cameras provide four constraints, which are sufficient to solve for the three-dimensional spatial position of the visual markers. ;

[0080] The calculated visual projection matrix After performing inverse normalization, we get:

[0081]

[0082] Establish the objective function (4) for minimizing the 3D point reprojection under multi-camera observation, and obtain the 3D position of the visual marker points carried by the robot end effector through equation (4):

[0083] (4)

[0084] in, The total number of cameras observing simultaneously ( ), For the first The image coordinates of the visual markers carried by the robot's end effector in the camera. For the first The inverse normalized visual projection matrix of the camera;

[0085] Coordinates of the camera image at the robot's end effector and The three-dimensional spatial position of the robot's end effector can be obtained using the visual projection matrix obtained in Equation 5. .

[0086] Step 5: The three-dimensional spatial position of the robot end effector obtained in Step 4 is used as feedback information, becoming visual feedback; the visual feedback and the displacement sensor information together constitute dual feedback control; the entire robot control is a dual-loop control, the inner loop is a motion control closed loop using displacement sensor information; the outer loop is a closed loop using visual feedback, providing high-precision robot target position information for the inner loop control.

[0087] The entire robot control system is a dual-loop control. The vision closed loop provides high-precision target position information, enabling the robot motion control closed loop to achieve high-precision robot control, forming a vision-motion dual-loop robot control. The motion control closed loop adopts composite active disturbance rejection control to classify and suppress disturbances to the pneumatic robot system: a disturbance observer is designed to estimate time-varying load force disturbances that have a significant impact on the system in real time, such as equivalent driving force disturbances caused by load changes, and perform dynamic compensation; at the same time, other unmodeled dynamics and system uncertainties are regarded as total disturbances, and an extended state observer is used to observe and dynamically compensate for them in real time, realizing disturbance classification observation and cooperative suppression.

[0088] By adopting a vision-based dual-feedback closed-loop control method, the positioning standard deviation of the pneumatic robot end effector is less than 1 mm, achieving good motion control accuracy.

[0089] The above detailed description further illustrates the purpose, technical solution, and beneficial effects of the invention. It should be understood that the above description is only a specific embodiment of the present invention and does not limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A pneumatic robot control method and system based on visual feedback, characterized in that: Includes the following steps, Step 1: Perform image acquisition from a single camera, binarize and preprocess the acquired images using Gaussian filtering, and then obtain the camera image coordinates of the visual markers carried by the robot's end effector using a geometric feature matching method. ; Step 2: Convert the camera image coordinates of the visual markers obtained in Step 1. With three-dimensional spatial position Substitute into equation (1) to calculate the camera's visual projection matrix. To obtain the three-dimensional spatial position With camera image coordinates The mapping relationship; Step 3: For multiple groups and Centroid normalization was performed to obtain multiple sets of normalized camera image coordinates for visual marker points. and the corresponding three-dimensional spatial position Substitute into the objective function based on the Huber loss function Further optimization of the visual projection matrix ; Step 4: Use at least two cameras to observe from different perspectives to obtain the three-dimensional spatial position of the visual marker point. This allows us to further obtain the three-dimensional spatial position of the robot's end effector; Step 5: The three-dimensional spatial position of the robot end effector obtained in Step 4 is used as feedback information, becoming visual feedback; the visual feedback and the displacement sensor information together constitute dual feedback control; the entire robot control is a dual-loop control, the inner loop is a motion control closed loop using displacement sensor information; the outer loop is a closed loop using visual feedback, providing high-precision robot target position information for the inner loop control.

2. The method as described in claim 1, characterized in that: The three-dimensional spatial position described in step two With camera image coordinates The mapping relationship; (1) in, Scale factor; The scale factor in equation (1) Elimination yields (2)。 3. The method as described in claim 1, characterized in that: The third step is to implement the following method: According to equation (2), multiple sets of three-dimensional spatial positions can be selected. and camera image coordinates This constitutes an over-constrained system; To improve the three-dimensional spatial position of visual markers and camera image coordinates Stability, for multiple groups and Perform centroid normalization; , , , in, This represents the total number of visual marker locations across the entire workspace. 3D spatial position of visual markers and camera image coordinates The normalized coordinates are , , , Camera image coordinates after normalization of multiple sets of visual markers and the corresponding three-dimensional spatial position The visual projection matrix is ​​optimized using equation (3). ; (3) in, The objective function is based on the Huber loss function. The Huber loss function is expressed as follows: in, For the threshold, Denotes the Euclidean norm. Indicates the first The visual marker point at the th Normalized coordinates at each position, This represents the total number of visual markers detected at a given location.

4. The method as described in claim 1, characterized in that: The implementation method for step four is as follows: The value of a single camera is obtained through equation (3). Multiple cameras are used to observe from different perspectives, and the projection equations of each camera provide two independent constraints, namely... and Orientation error: The two cameras provide four constraints, which are sufficient to solve for the three-dimensional spatial position of the visual markers. ; The calculated visual projection matrix After performing inverse normalization, we get: Establish the objective function (4) for minimizing the 3D point reprojection under multi-camera observation, and obtain the 3D position of the visual marker points carried by the robot end effector through equation (4): (4) in, The total number of cameras observing simultaneously ( ), For the first The image coordinates of the visual markers carried by the robot's end effector in the camera. For the first The inverse normalized visual projection matrix of the camera; The three-dimensional spatial position of the robot's end effector is then obtained.

5. The method as described in claim 1, characterized in that: Step five is implemented as follows: the entire robot control is a dual-loop control. The vision closed loop provides high-precision target position information, thereby enabling the robot motion control closed loop to achieve high-precision robot control, forming a vision-motion dual-loop robot control. The motion control closed loop adopts composite active disturbance rejection control to classify and suppress disturbances received by the pneumatic robot system: a disturbance observer is designed to estimate time-varying load force disturbances that have a significant impact on the system in real time, such as equivalent driving force disturbances caused by load changes, and to perform dynamic compensation; at the same time, other unmodeled dynamics and system uncertainties are regarded as total disturbances, and an extended state observer is used to observe and dynamically compensate for them in real time, realizing disturbance classification observation and cooperative suppression.

6. The method as described in claim 1, characterized in that: The number of groups mentioned in step three is 6 or more.

7. The method as described in claim 1, characterized in that: The number of cameras mentioned in step four is at least two.

8. The method as described in claim 1, characterized in that: Step 3 optimizes the visual projection matrix The process employs an objective function based on the Huber loss function. .

9. An apparatus for implementing the method as described in claims 1 to 8, characterized in that: include: The robot body 11, air source 1, proportional valve 2, proportional valve 3, proportional valve 4, cylinder 5, cylinder 6, cylinder 7, displacement sensor 8, displacement sensor 9, displacement sensor 10, industrial camera 12, industrial camera 13, controller 14, and PC 15; the displacement sensor is installed on the cylinder, the industrial camera is connected to the PC, both the displacement sensor and the PC are connected to the controller, the output signal of the controller controls the proportional valve to act, and the proportional valve is connected to the cylinder; The gas source provides a gas pressure of 0.6 MPa; the displacement sensor has a range of 200 mm and outputs a voltage signal of 0-10 V; the proportional valve has a range of 0-10 V; the controller has AD and DA functions. The operation process of the robot control system based on vision-based dual feedback control is described as follows: During the movement of the pneumatic robot, the extension and retraction displacement of its actuator cylinder is collected by the displacement sensor and fed back to the controller as cylinder displacement signal; at the same time, the visual marker point image carried by the robot end effector is collected by the dual-camera vision system. After image processing such as image preprocessing and target recognition, the image coordinates of the target are obtained. The three-dimensional spatial coordinates of the end effector are calculated by optimizing the visual projection matrix. This visual feedback information is fed back to the controller. Finally, the pneumatic robot cylinder movement is controlled by the control algorithm that fuses the information from the two sensors, forming a dual feedback control process.

Images