An automatic feeding robot with visual positioning and autonomous obstacle avoidance

Abstract

The application relates to an automatic feeding robot with visual positioning and autonomous obstacle avoidance, and relates to the technical field of industrial robots, in particular to an automatic feeding robot with visual positioning and autonomous obstacle avoidance. In order to achieve the above object, visual identification module transmits the recognized material position and attitude data to a path planning module in real time, the path planning module sends an initial path to a control system, the control system receives environmental map and obstacle information transmitted by a multi-sensor obstacle avoidance module; the control system issues a motion control instruction to a feeding execution mechanism; the sensor of the feeding execution mechanism feeds back to the control system in real time; the control system is responsible for receiving the material pose information output by the visual identification module, the environmental data provided by the multi-sensor obstacle avoidance module and the trajectory generated by the path planning module, and performs real-time scheduling and closed-loop control on the feeding execution mechanism. Through the collaborative design of visual positioning, hierarchical path planning, multi-sensor fusion obstacle avoidance and intelligent execution mechanism, the high precision, flexibility and safety of the automatic feeding process are realized.

Description

Technical Field

[0001] This invention relates to the field of industrial robot technology, and in particular to an automatic loading robot with visual positioning and autonomous obstacle avoidance functions. Background Technology

[0002] Existing automated loading robots mostly operate using fixed paths and fixed workstations. Their running trajectories and gripping positions need to be pre-set before the task begins, making them difficult to adapt to dynamically changing production environments. When the workpiece position shifts or its posture becomes inconsistent, traditional systems often fail to grip accurately, requiring manual intervention and reducing the level of automation. Furthermore, existing obstacle avoidance methods are mostly based on single sensors, which are limited by ranging range, accuracy, or response speed, leading to untimely obstacle recognition or unreasonable obstacle avoidance trajectories, posing safety hazards. In addition, traditional actuators often rely on fixed clamps or simple suction cups, lacking compliant control and gripping criteria, resulting in insufficient gripping reliability, especially prone to failure in complex workpieces or plug-in operation scenarios.

[0003] The obvious shortcomings are: the robot cannot accurately grasp materials when their position or posture is off; the robot is prone to collisions due to moving or temporary obstacles in the production environment, resulting in insufficient safety; and it is difficult to adapt to workpieces of different shapes and sizes, resulting in insufficient flexibility. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings and deficiencies of existing technologies by providing an automated loading robot with visual positioning and autonomous obstacle avoidance. Through visual positioning, hierarchical path planning, multi-sensor fusion obstacle avoidance, and the collaborative design of intelligent actuators, it achieves high precision, flexibility, and safety in the automated loading process.

[0005] To achieve the above objectives, the automatic feeding robot of the present invention, which is equipped with visual positioning and autonomous obstacle avoidance, includes a visual recognition module 1, a path planning module 2, a multi-sensor obstacle avoidance module 3, a feeding execution mechanism 4, and a control system 5. The visual recognition module 1 transmits the identified material position and posture data to the path planning module 2 in real time. The path planning module 2 sends the initial path to the control system 5. The control system 5 receives the environmental map and obstacle information transmitted by the multi-sensor obstacle avoidance module 3. The control system 5 issues motion control commands to the feeding execution mechanism 4. The sensors of the feeding execution mechanism 4 provide real-time feedback to the control system 5. The control system 5 is responsible for receiving the material posture information output by the visual recognition module 1, the environmental data provided by the multi-sensor obstacle avoidance module 3, and the trajectory generated by the path planning module 2, and performing real-time scheduling and closed-loop control of the feeding execution mechanism 4.

[0006] The visual recognition module 1 includes a 2D industrial camera or a 3D depth camera, which, combined with a supplementary light source, is used to acquire images and point cloud data of the loading station. When estimating the material pose, the visual recognition module 1 uses the EPnP combined with the RANSAC method to calculate the initial pose of the material in the camera coordinate system by the correspondence between the key points of the two-dimensional image and the points of the three-dimensional model. After obtaining the initial pose, the iterative nearest point (ICP) algorithm based on point-to-surface constraints is further used to register the material point cloud and the model point cloud to refine the three-dimensional pose of the material.

[0007] The specific method is as follows: the collected data first undergoes image preprocessing, including noise reduction, distortion correction, and depth information completion, to ensure the integrity and stability of the input data;

[0008] The visual recognition module calls a target detection and instance segmentation algorithm based on a convolutional neural network to identify the area where the material is located and obtain the two-dimensional bounding box or mask information of the material. Combining the camera intrinsic and extrinsic parameters and depth data, the PnP algorithm or point cloud ICP registration algorithm is used to calculate the three-dimensional pose of the material and obtain the six-degree-of-freedom pose parameters of the material in the robot base coordinate system.

[0009] The PnP algorithm calculates the 3D pose of the material as follows: A 2D–3D correspondence is established, key points are defined for the model, and n ≥ 6 known object 3D key points are selected. Rather than 2D pixels on the image (n is obtained by direct regression from the detection head or by template projection / contour matching).

[0010] The camera's intrinsic parameters are:

[0011]

[0012] K is derived from the calibration;

[0013] EPnP initial solution, using EPnP to... Represented as four control points Affine combination:

[0014]

[0015] Let the camera coordinates be the lower point Its pixel projection satisfies

[0016] EPNP direct solution Linear least squares, then using Procrustes / SVD to recover R,t (making the world system...) With camera system (Most match)

[0017] After completing the material pose estimation, the visual recognition module 1 outputs a set of candidate grasping poses based on the geometric features of the material point cloud and the grasping candidate generation algorithm, and scores each candidate pose. The scoring criteria include the flatness of the contact surface, the coverage of the gripper / suction cup, and the gaps between surrounding obstacles.

[0018] The visual recognition module 1 outputs the material category, pose, candidate grasping pose set and corresponding confidence information and transmits them to the path planning module 2.

[0019] To ensure the continuity and robustness of the recognition results in complex environments, the visual recognition module also includes a time-series tracking and uncertainty assessment unit. When short-term occlusion, strong reflection, or unstable recognition results occur, this unit maintains the continuity of the output through filtering and multi-frame fusion, and triggers re-recognition or viewpoint switching operations when the uncertainty exceeds the limit.

[0020] The path planning module 2 is used to generate a collision-free trajectory that meets kinematic and dynamic constraints based on the material pose provided by the visual recognition module and the environmental map provided by the multi-sensor obstacle avoidance module. It adopts a hierarchical planning strategy. The global layer searches for the initial path based on the Hybrid A or D Lite algorithm, and the local layer combines the Temporal Elastic Band (TEB) or Model Predictive Control (MPC) to optimize the trajectory and avoid obstacles in real time. To ensure the executability of the trajectory, the module further uses the TOPP-RA time parameterization method to process the speed and acceleration constraints of the geometric path, and realizes dynamic replanning through continuous collision detection and collision time prediction (TTC), thereby ensuring the safety and real-time performance of the robot in the entire process of grasping, handling and loading.

[0021] The global path planning algorithm is either Hybrid A or D Lite. Hybrid A includes orientation / nonholonomic constraints, while D Lite is incremental and suitable for map updates.

[0022] The real-time trajectory optimization algorithm is as follows:

[0023] Question format:

[0024]

[0025] st collision constraints ;

[0026] End-point tracking uses TEB (Timed Elastic Band) or CHOMP / TrajOpt;

[0027] Pure chassis / differential chassis can use MPC (Model Predictive Control) to optimize speed commands (υ,ω) within a rolling window.

[0028] Grab / docking specialization: Add a compliance cone and "approach-insert-retreat" micro-segments to the end coordinate system (velocity ≤ 50 mm / s, attitude fine-tuning ≤ 1°).

[0029] The multi-sensor obstacle avoidance module 3 includes a lidar and an ultrasonic sensor. It fuses data collected by different sensors through extended Kalman filtering to construct an environmental occupancy grid map. When the safe distance between the robot and the obstacle is less than a preset threshold or the predicted collision time is less than a preset value, it triggers path replanning.

[0030] The loading actuator 4 includes an end effector mechanism, which comprises a quick-change interface and a tool identification unit for loading: an adaptive gripper, a vacuum adsorption module, and a customized fixture; during end-effector execution, proximity and micro-motion alignment are controlled using Cartesian space impedance.

[0031]

[0032] Where e represents the TCP error. To measure the external force; reduce the force in the 30–10 mm range near the end. ,improve (Soft compliance, strong damping) to attenuate impact;

[0033] Admittance fine-tuning: When the visual pose covariance is large or there is an insertion / docking requirement (mortise and tenon, positioning hole), a "compliance cone" (±0.5–1.0mm / ±1–2°) is set in the normal direction, while the tangential direction is restricted, and the cone advances in small steps along the structural normal (≤20–50mm / s).

[0034] Estimation of suction cup adsorption pressure and leakage rate during end-effector execution:

[0035]

[0036] Criterion: In Inside or steady state The leak / sealing failure has been identified.

[0037] Effective adsorption area estimation: evaluated by curvature / normal and edge distance; if the effective area is insufficient or the curvature is greater than the threshold, the multi-cavity / dual suction cup scheme is automatically switched and the wrist posture is recalculated.

[0038] Handling monitoring: Periodically perform "micro-unloading" tests (short-term reduced suction → observation) during transportation. (Rebound), failure triggers deceleration and backup trajectory.

[0039] The automatic feeding method of the automatic feeding robot includes the following steps: S1, acquiring workstation images or point clouds through a camera and identifying the material position and posture; S2, generating a feeding path based on the identification results and environmental map; S3, sensing the environment and performing obstacle avoidance and replanning through lidar and / or ultrasonic sensors; S4, driving the robotic arm and end effector to perform grasping and feeding operations; S5, verifying the quality of the results after feeding is completed.

[0040] In step S1, the acquisition and preprocessing of material position and attitude data are performed using 2D+3D (RGB-D / structured light / ToF), HDR / cross-polarized highlight suppression; depth holes are filled using guided filtering or learning-based completion; background removal: RANSAC fits the platform plane and ROI is clipped.

[0041] The pose detection / segmentation algorithm is as follows:

[0042] Preliminary solution to EPNP+RANSAC:

[0043]

[0044] Point-to-surface ICP refinement

[0045]

[0046] Voxel downsampling 2–5 mm, iterations ≤ 20, convergence threshold .

[0047] 8. In step S2, the environment map uses ESDF (or occupied grid + dilation) to give the distance field D(x);

[0048] The trajectory cost is calculated as follows:

[0049] .

[0050] 9. In step S3, obstacle avoidance replanning constructs an ESDF by fusing the EKF of lidar and ultrasonic waves, and uses... Threshold-triggered online replanning.

[0051] The working principle of this invention is as follows: It acquires workstation images and point cloud data through a 2D industrial camera or a 3D depth camera, obtains material bounding boxes or mask information through convolutional neural network detection and segmentation, estimates the initial pose of the material using EPnP combined with RANSAC algorithm, and then performs fine registration through point-to-surface ICP algorithm to obtain the six-degree-of-freedom pose of the material in the robot base coordinate system, and outputs a set of candidate grasping poses and confidence scores.

[0052] Based on the visual recognition results and the environmental map provided by the obstacle avoidance module, a hierarchical planning strategy is adopted. The global layer uses HybridA or DLite to search for coarse paths, and the local layer uses TEB or MPC to optimize the trajectory. Combined with TOPP-RA time parameterization, the trajectory is guaranteed to meet the velocity and acceleration constraints. Dynamic replanning is achieved through continuous collision detection and TTC prediction.

[0053] By integrating data from lidar and ultrasonic sensors, an environmental occupancy map is generated using extended Kalman filtering. When the obstacle distance is less than a threshold or the predicted collision time is less than a set value, path replanning is triggered to achieve redundant safety. The end effector is equipped with a quick-change interface and tool recognition unit, capable of loading adaptive grippers, vacuum suction cups, or custom fixtures. During the approach and docking phases, Cartesian impedance control and admittance micro-motion are employed to achieve compliant alignment and shock resistance. During suction cup adsorption, leakage rate and effective adsorption area are estimated based on pressure-time curves to determine adsorption reliability and automatically switch between methods. The control system receives outputs from the vision, path planning, and obstacle avoidance modules, uniformly performs task scheduling and motion control, issues commands to drive actuators, and achieves closed-loop control and anomaly handling through sensor feedback.

[0054] The beneficial effects of this invention after adopting the above technical solution are as follows: By introducing the collaborative design of visual recognition, path planning, multi-sensor obstacle avoidance, and intelligent actuators, it achieves efficient adaptation to complex dynamic workstation environments, and can still complete stable and accurate material loading operations even in the presence of material position deviations, attitude uncertainties, and obstacle interference. The two-stage attitude estimation method combining EPnP with RANSAC and point-to-surface ICP ensures the accuracy of material six-degree-of-freedom pose recognition; the hierarchical planning strategy of Hybrid A and TEB / MPC, combined with TOPP-RA time parameterization and continuous collision detection, achieves path smoothness, executability, and real-time obstacle avoidance capability; the multi-sensor obstacle avoidance module integrating lidar and ultrasound improves the reliability and safety redundancy of environmental perception; the end effector achieves compliant alignment through impedance / admittance control, and improves the stability and adaptability of grasping by combining leakage rate estimation from the vacuum pressure-time curve; simultaneously, the quality verification process ensures the success rate of material loading through visual verification, force control, or vacuum detection closed-loop. In summary, this invention significantly improves the flexibility, accuracy, and safety of automated feeding robots, reduces human intervention, and is widely applicable to multi-station, multi-material, and dynamic manufacturing environments. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 This is a schematic block diagram of the overall structure of the present invention;

[0057] Figure 2 This is a flowchart of the automatic feeding method of the present invention.

[0058] Figure labeling: 1. Visual recognition module; 2. Path planning module; 3. Multi-sensor obstacle avoidance module; 4. Feeding execution mechanism; 5. Control system. Detailed Implementation

[0059] See Figure 1-2 As shown, the technical solution adopted in this specific embodiment is as follows: it includes a visual recognition module 1, a path planning module 2, a multi-sensor obstacle avoidance module 3, a feeding execution mechanism 4, and a control system 5. The visual recognition module 1 transmits the identified material position and posture data to the path planning module 2 in real time. The path planning module 2 sends the initial path to the control system 5. The control system 5 receives the environmental map and obstacle information transmitted by the multi-sensor obstacle avoidance module 3. The control system 5 issues motion control commands to the feeding execution mechanism 4. The sensors of the feeding execution mechanism 4 provide real-time feedback to the control system 5. The control system 5 is responsible for receiving the material posture information output by the visual recognition module 1, the environmental data provided by the multi-sensor obstacle avoidance module 3, and the trajectory generated by the path planning module 2, and performing real-time scheduling and closed-loop control of the feeding execution mechanism 4.

[0060] The visual recognition module 1 includes a 2D industrial camera or a 3D depth camera, which, combined with a supplementary light source, is used to acquire images and point cloud data of the loading station. When estimating the material pose, the visual recognition module 1 uses the EPnP combined with the RANSAC method to calculate the initial pose of the material in the camera coordinate system by the correspondence between the key points of the two-dimensional image and the points of the three-dimensional model. After obtaining the initial pose, the iterative nearest point (ICP) algorithm based on point-to-surface constraints is further used to register the material point cloud and the model point cloud to refine the three-dimensional pose of the material.

[0061] The specific method is as follows: the collected data first undergoes image preprocessing, including noise reduction, distortion correction, and depth information completion, to ensure the integrity and stability of the input data;

[0062] The visual recognition module calls a target detection and instance segmentation algorithm based on a convolutional neural network to identify the area where the material is located and obtain the two-dimensional bounding box or mask information of the material. Combining the camera intrinsic and extrinsic parameters and depth data, the PnP algorithm or point cloud ICP registration algorithm is used to calculate the three-dimensional pose of the material and obtain the six-degree-of-freedom pose parameters of the material in the robot base coordinate system.

[0063] The PnP algorithm calculates the 3D pose of the material as follows: A 2D–3D correspondence is established, key points are defined for the model, and n ≥ 6 known object 3D key points are selected. Rather than 2D pixels on the image (n is obtained by direct regression from the detection head or by template projection / contour matching).

[0064] The camera's internal parameters are:

[0065]

[0066] K is derived from the calibration;

[0067] EPnP initial solution, using EPnP to... Represented as four control points Affine combination:

[0068]

[0069] Let the camera coordinates be the lower point Its pixel projection satisfies

[0070] EPNP direct solution Linear least squares, then using Procrustes / SVD to recover R,t (making the world system...) With camera system (Most match)

[0071] After completing the material pose estimation, the visual recognition module 1 outputs a set of candidate grasping poses based on the geometric features of the material point cloud and the grasping candidate generation algorithm, and scores each candidate pose. The scoring criteria include the flatness of the contact surface, the coverage of the gripper / suction cup, and the gaps between surrounding obstacles.

[0072] The visual recognition module 1 outputs the material category, pose, candidate grasping pose set and corresponding confidence information and transmits them to the path planning module 2.

[0073] To ensure the continuity and robustness of the recognition results in complex environments, the visual recognition module also includes a time-series tracking and uncertainty assessment unit. When short-term occlusion, strong reflection, or unstable recognition results occur, this unit maintains the continuity of the output through filtering and multi-frame fusion, and triggers re-recognition or viewpoint switching operations when the uncertainty exceeds the limit.

[0074] The path planning module 2 is used to generate a collision-free trajectory that meets kinematic and dynamic constraints based on the material pose provided by the visual recognition module and the environmental map provided by the multi-sensor obstacle avoidance module. It adopts a hierarchical planning strategy. The global layer searches for the initial path based on the Hybrid A or D Lite algorithm, and the local layer combines the Temporal Elastic Band (TEB) or Model Predictive Control (MPC) to optimize the trajectory and avoid obstacles in real time. To ensure the executability of the trajectory, the module further uses the TOPP-RA time parameterization method to process the speed and acceleration constraints of the geometric path, and realizes dynamic replanning through continuous collision detection and collision time prediction (TTC), thereby ensuring the safety and real-time performance of the robot in the entire process of grasping, handling and loading.

[0075] The global path planning algorithm is either Hybrid A or D Lite. Hybrid A includes orientation / nonholonomic constraints, while D Lite is incremental and suitable for map updates.

[0076] The real-time trajectory optimization algorithm is as follows:

[0077] Question format:

[0078]

[0079] st collision constraints ;

[0080] End-point tracking uses TEB (Timed Elastic Band) or CHOMP / TrajOpt;

[0081] Pure chassis / differential chassis can use MPC (Model Predictive Control) to optimize speed commands (υ,ω) within a rolling window.

[0082] Grab / docking specialization: Add a compliance cone and "approach-insert-retreat" micro-segments to the end coordinate system (velocity ≤ 50 mm / s, attitude fine-tuning ≤ 1°).

[0083] The multi-sensor obstacle avoidance module 3 includes a lidar and an ultrasonic sensor. It fuses data collected by different sensors through extended Kalman filtering to construct an environmental occupancy grid map. When the safe distance between the robot and the obstacle is less than a preset threshold or the predicted collision time is less than a preset value, it triggers path replanning.

[0084] The loading actuator 4 includes an end effector mechanism, which comprises a quick-change interface and a tool identification unit for loading: an adaptive gripper, a vacuum adsorption module, and a customized fixture; during end-effector execution, proximity and micro-motion alignment are controlled using Cartesian space impedance.

[0085]

[0086] Where e represents the TCP error. To measure the external force; reduce the force in the 30–10 mm range near the end. ,improve (Soft compliance, strong damping) to attenuate impact;

[0087] Admittance fine-tuning: When the visual pose covariance is large or there is an insertion / docking requirement (mortise and tenon, positioning hole), a "compliance cone" (±0.5–1.0mm / ±1–2°) is set in the normal direction, while the tangential direction is restricted, and the cone advances in small steps along the structural normal (≤20–50mm / s).

[0088] Estimation of suction cup adsorption pressure and leakage rate during end-effector execution:

[0089]

[0090] Criterion: In Inside or steady state The leak / sealing failure has been identified.

[0091] Effective adsorption area estimation: evaluated by curvature / normal and edge distance; if the effective area is insufficient or the curvature is greater than the threshold, the multi-cavity / dual suction cup scheme is automatically switched and the wrist posture is recalculated.

[0092] Handling monitoring: Periodically perform "micro-unloading" tests (short-term reduced suction → observation) during transportation. (Rebound), failure triggers deceleration and backup trajectory.

[0093] The automatic feeding method of the automatic feeding robot includes the following steps: S1, acquiring workstation images or point clouds through a camera and identifying the material position and posture; S2, generating a feeding path based on the identification results and environmental map; S3, sensing the environment and performing obstacle avoidance and replanning through lidar and / or ultrasonic sensors; S4, driving the robotic arm and end effector to perform grasping and feeding operations; S5, verifying the quality of the results after feeding is completed.

[0094] In step S1, the acquisition and preprocessing of material position and attitude data are performed using 2D+3D (RGB-D / structured light / ToF), HDR / cross-polarized highlight suppression; depth holes are filled using guided filtering or learning-based completion; background removal: RANSAC fits the platform plane and ROI is clipped.

[0095] The pose detection / segmentation algorithm is as follows:

[0096] Preliminary solution to EPNP+RANSAC:

[0097]

[0098] Point-to-surface ICP refinement

[0099]

[0100] Voxel downsampling 2–5 mm, iterations ≤ 20, convergence threshold .

[0101] 8. In step S2, the environment map uses ESDF (or occupied grid + dilation) to give the distance field D(x);

[0102] The trajectory cost is calculated as follows:

[0103] .

[0104] 9. In step S3, obstacle avoidance replanning constructs an ESDF by fusing the EKF of lidar and ultrasonic waves, and uses... Threshold-triggered online replanning.

[0105] It acquires workstation images and point cloud data through 2D industrial cameras or 3D depth cameras, obtains material bounding boxes or mask information through convolutional neural network detection and segmentation, estimates the initial pose of the material using EPnP combined with RANSAC algorithm, and then performs fine registration through point-to-surface ICP algorithm to obtain the six-degree-of-freedom pose of the material in the robot base coordinate system, and outputs a set of candidate grasping poses and confidence scores.

[0106] Based on the visual recognition results and the environmental map provided by the obstacle avoidance module, a hierarchical planning strategy is adopted. The global layer uses HybridA or DLite to search for coarse paths, and the local layer uses TEB or MPC to optimize the trajectory. Combined with TOPP-RA time parameterization, the trajectory is guaranteed to meet the velocity and acceleration constraints. Dynamic replanning is achieved through continuous collision detection and TTC prediction.

[0107] By integrating data from lidar and ultrasonic sensors, an environmental occupancy map is generated using extended Kalman filtering. When the obstacle distance is less than a threshold or the predicted collision time is less than a set value, path replanning is triggered to achieve redundant safety. The end effector is equipped with a quick-change interface and tool recognition unit, capable of loading adaptive grippers, vacuum suction cups, or custom fixtures. During the approach and docking phases, Cartesian impedance control and admittance micro-motion are employed to achieve compliant alignment and shock resistance. During suction cup adsorption, leakage rate and effective adsorption area are estimated based on pressure-time curves to determine adsorption reliability and automatically switch between methods. The control system receives outputs from the vision, path planning, and obstacle avoidance modules, uniformly performs task scheduling and motion control, issues commands to drive actuators, and achieves closed-loop control and anomaly handling through sensor feedback.

[0108] By introducing a collaborative design of visual recognition, path planning, multi-sensor obstacle avoidance, and intelligent actuators, this invention achieves efficient adaptation to complex and dynamic workstation environments. It can still complete stable and accurate loading operations even with material position deviations, uncertain postures, and obstacle interference. A two-stage posture estimation method combining EPnP with RANSAC and point-to-surface ICP ensures the accuracy of material six-degree-of-freedom posture recognition. A hierarchical planning strategy using Hybrid A and TEB / MPC, combined with TOPP-RA time parameterization and continuous collision detection, achieves smooth path execution, executability, and real-time obstacle avoidance capabilities. A multi-sensor obstacle avoidance module integrating LiDAR and ultrasound enhances the reliability and safety redundancy of environmental perception. The end effector achieves compliant alignment through impedance / admittance control and improves gripping stability and adaptability by incorporating leakage rate estimation from vacuum pressure-time curves. Simultaneously, a quality verification process using visual verification, force control, or vacuum detection closed-loop ensures a high success rate for loading. In summary, this invention significantly improves the flexibility, accuracy, and safety of automated loading robots.

[0109] The above description is only used to illustrate the technical solution of the present invention and is not intended to limit it. Any other modifications or equivalent substitutions made by those skilled in the art to the technical solution of the present invention, as long as they do not depart from the spirit and scope of the technical solution of the present invention, should be covered within the scope of the claims of the present invention.

Claims

1. An automated loading robot with visual positioning and autonomous obstacle avoidance capabilities, characterized in that: It includes a visual recognition module (1), a path planning module (2), a multi-sensor obstacle avoidance module (3), a feeding execution mechanism (4), and a control system (5). The visual recognition module (1) transmits the identified material position and posture data to the path planning module (2) in real time. The path planning module (2) sends the initial path to the control system (5). The control system (5) receives the environmental map and obstacle information transmitted by the multi-sensor obstacle avoidance module (3). The control system (5) issues motion control commands to the feeding execution mechanism (4). The sensors of the feeding execution mechanism (4) provide real-time feedback to the control system (5). The control system (5) is responsible for receiving the material posture information output by the visual recognition module (1), the environmental data provided by the multi-sensor obstacle avoidance module (3), and the trajectory generated by the path planning module (2), and performs real-time scheduling and closed-loop control of the feeding execution mechanism (4).

2. The automated loading robot with visual positioning and autonomous obstacle avoidance as described in claim 1, characterized in that: The visual recognition module (1) includes a 2D industrial camera or a 3D depth camera, which is combined with a supplementary light source to collect images and point cloud data of the loading station. When estimating the material pose, the visual recognition module (1) uses the EPnP combined with the RANSAC method to calculate the initial pose of the material in the camera coordinate system through the correspondence between the key points of the two-dimensional image and the points of the three-dimensional model. After obtaining the initial pose, the iterative nearest point (ICP) algorithm based on point-to-surface constraints is further used to register the material point cloud and the model point cloud to refine the three-dimensional pose of the material. The specific method is as follows: the collected data first undergoes image preprocessing, including noise reduction, distortion correction, and depth information completion, to ensure the integrity and stability of the input data; The visual recognition module calls a target detection and instance segmentation algorithm based on a convolutional neural network to identify the area where the material is located and obtain the two-dimensional bounding box or mask information of the material. Combining the camera intrinsic and extrinsic parameters and depth data, the PnP algorithm or point cloud ICP registration algorithm is used to calculate the three-dimensional pose of the material and obtain the six-degree-of-freedom pose parameters of the material in the robot base coordinate system. The PnP algorithm calculates the 3D pose of the material as follows: A 2D–3D correspondence is established, key points are defined for the model, and n ≥ 6 known object 3D key points are selected. Rather than 2D pixels on the image (n is obtained by direct regression from the detection head or by template projection / contour matching). The camera's intrinsic parameters are: ； K is derived from the calibration; EPnP initial solution, using EPnP to... Represented as four control points Affine combination: ； Let the camera coordinates be the lower point Its pixel projection satisfies ； EPNP direct solution Linear least squares, then using Procrustes / SVD to recover R,t (making the world system...) With camera system (Most match) After completing the material pose estimation, the visual recognition module 1 outputs a set of candidate grasping poses based on the geometric features of the material point cloud and the grasping candidate generation algorithm, and scores each candidate pose. The scoring criteria include the flatness of the contact surface, the coverage of the gripper / suction cup, and the gaps between surrounding obstacles. The visual recognition module 1 outputs the material category, pose, candidate grasping pose set and corresponding confidence information and transmits them to the path planning module 2. To ensure the continuity and robustness of the recognition results in complex environments, the visual recognition module also includes a time-series tracking and uncertainty assessment unit. When short-term occlusion, strong reflection, or unstable recognition results occur, this unit maintains the continuity of the output through filtering and multi-frame fusion, and triggers re-recognition or viewpoint switching operations when the uncertainty exceeds the limit.

3. The automated loading robot with visual positioning and autonomous obstacle avoidance as described in claim 1, characterized in that: The path planning module 2 is used to generate a collision-free trajectory that meets kinematic and dynamic constraints based on the material pose provided by the visual recognition module and the environmental map provided by the multi-sensor obstacle avoidance module. It adopts a hierarchical planning strategy. The global layer searches for the initial path based on the Hybrid A or D Lite algorithm, and the local layer combines the Temporal Elastic Band (TEB) or Model Predictive Control (MPC) to optimize the trajectory and avoid obstacles in real time. To ensure the executability of the trajectory, the module further uses the TOPP-RA time parameterization method to process the speed and acceleration constraints of the geometric path, and realizes dynamic replanning through continuous collision detection and collision time prediction (TTC), thereby ensuring the safety and real-time performance of the robot in the entire process of grasping, handling and loading. The global path planning algorithm is either Hybrid A or D Lite. Hybrid A includes orientation / nonholonomic constraints, while D Lite is incremental and suitable for map updates. The real-time trajectory optimization algorithm is as follows: Question format: ； st collision constraints ; End-point tracking uses TEB (Timed Elastic Band) or CHOMP / TrajOpt; Pure chassis / differential chassis can use MPC (Model Predictive Control) to optimize speed commands (υ,ω) within a rolling window; Grab / docking specialization: Add a compliance cone and "approach-insert-retreat" micro-segments to the end coordinate system (velocity ≤ 50 mm / s, attitude fine-tuning ≤ 1°).

4. The automated loading robot with visual positioning and autonomous obstacle avoidance as described in claim 1, characterized in that: The multi-sensor obstacle avoidance module (3) includes a lidar and an ultrasonic sensor. It fuses the data collected by different sensors through extended Kalman filtering to construct an environmental occupancy grid map. When the safe distance between the obstacle and the robot is less than a preset threshold or the predicted collision time is less than a preset value, it triggers path replanning.

5. An automated loading robot with visual positioning and autonomous obstacle avoidance as described in claim 1, characterized in that: The loading actuator (4) includes an end effector mechanism, which includes a quick-change interface and a tool identification unit for loading: an adaptive gripper, a vacuum adsorption module, and a custom fixture; during end-effector execution, proximity and micro-motion alignment are controlled by Cartesian space impedance. ； Where e represents the TCP error. To measure the external force; reduce the force in the 30–10 mm range near the end. ,improve (Soft compliance, strong damping) to attenuate impact; Admittance fine-tuning: When the visual pose covariance is large or there is an insertion / docking requirement (mortise and tenon, positioning hole), a "compliance cone" (±0.5–1.0mm / ±1–2°) is set in the normal direction, while the tangential direction is restricted, and the advance is made in small steps along the structural normal (≤20–50mm / s). Estimation of suction cup adsorption pressure and leakage rate during end-effector execution: ； Criterion: In Inside or steady state The leak / sealing failure has been identified. Effective adsorption area estimation: evaluated by curvature / normal and edge distance; if the effective area is insufficient or curvature > threshold, automatically switch to multi-cavity / dual suction cup scheme and recalculate wrist posture; Handling monitoring: Periodically perform "micro-unloading" tests (short-term reduced suction → observation) during transportation. (Rebound), failure triggers deceleration and backup trajectory.

6. The automated loading robot with visual positioning and autonomous obstacle avoidance as described in claim 1, characterized in that: The automatic feeding method of the automatic feeding robot includes the following steps: S1, acquiring workstation images or point clouds through a camera and identifying the material position and posture; S2, generating a feeding path based on the identification results and environmental map; S3, sensing the environment and performing obstacle avoidance and replanning through lidar and / or ultrasonic sensors; S4, driving the robotic arm and end effector to perform grasping and feeding operations; S5, verifying the quality of the results after feeding is completed.

7. The feeding method of an automated feeding robot with visual positioning and autonomous obstacle avoidance according to claim 6, characterized in that: In step S1, the acquisition and preprocessing of material position and attitude data are performed using 2D+3D (RGB-D / structured light / ToF), HDR / cross-polarized highlight suppression; depth holes are filled using guided filtering or learning-based completion; background removal: RANSAC fits the platform plane and ROI is clipped. The pose detection / segmentation algorithm is as follows: Preliminary solution to EPNP+RANSAC: ； Point-to-surface ICP refinement ； Voxel downsampling 2–5 mm, iterations ≤ 20, convergence threshold .

8. The feeding method of an automated feeding robot with visual positioning and autonomous obstacle avoidance according to claim 6, characterized in that: In step S2, the environment map uses ESDF (or occupied grid + dilation) to give the distance field D(x); The trajectory cost is calculated as follows: 。 9. The feeding method of an automated feeding robot with visual positioning and autonomous obstacle avoidance according to claim 6, characterized in that: In step S3, obstacle avoidance replanning constructs an ESDF by fusing the EKF of lidar and ultrasonic waves, and uses... Threshold-triggered online replanning.

Images