A method and system for visual calibration of mobile robots

Abstract

This invention relates to the field of industrial robots, and particularly to a mobile robot visual calibration system, comprising: an image acquisition module: mounted on the body of the mobile robot, used to acquire at least image information of the goods to be picked up and the route image of the mobile robot in the direction of movement; a radar module: acquiring radar information of the mobile robot in real time while it is moving; a shock absorption module: mounted on the body of the mobile robot, used to adjust the shock absorption of the mobile robot; a vibration detection module: used to detect the vibration information of the mobile robot in real time during movement; and a navigation module: used to plan the route of the mobile robot to the delivery location after picking up the goods. It also relates to a mobile robot visual calibration method, including S1: simultaneously acquiring image information of the goods to be picked up through a camera built into the mobile robot body and a camera in the picking area… This can solve the problem of improper obstacle avoidance when the mobile robot is moving with goods.

Description

Technical Field

[0001] This invention relates to the field of industrial robots, and in particular to a method and system for visual calibration of mobile robots. Background Technology

[0002] Robots are essential production equipment in industrial fields such as aviation, aerospace, defense, and machinery manufacturing. The integration and application of robot systems are moving towards intelligentization, which is often reflected in the integration of high-precision vision systems at the robot's end effector. After integrating the vision system, the vision system typically establishes a corresponding pose matrix relationship with the robot's base coordinate system and the tool coordinate system. The calibration of the robot's vision system with the tool coordinate system (commonly known as "hand-eye" calibration) is a crucial step for the robot to guide the end effector to complete tasks. The accuracy of this "hand-eye" calibration determines whether the vision system can accurately guide the robot's spatial position.

[0003] Patent application number 202411370704.8 discloses a mobile robot visual calibration method and system, and an operation method and system. The mobile robot visual calibration method includes the following steps when the robot is in the operation position: Step S1, obtaining the transformation matrix Tb1 from the camera coordinate system to the first calibration board coordinate system and the robot's current photographing posture matrix P1; wherein the first calibration board is located on the robot's robotic arm; Step S2, based on Tx, Tb1, and P1, obtaining the transformation matrix T1 from the camera coordinate system to the robot coordinate system to complete the calibration; wherein Tx is the transformation matrix from the first calibration board coordinate system to the robot coordinate system. This invention is applicable to visual guidance operations of mobile robots where the eyes are outside the hands. It can expand the camera's shooting range, achieve high visual calibration accuracy and good real-time performance, and ensure recognition and positioning accuracy even if the mobile robot cannot accurately reach a fixed position every time, thus improving operation accuracy and reliability.

[0004] The aforementioned technology is applicable to vision-guided operations of mobile robots with eyes outside the hands. It can expand the camera's shooting range, achieve high visual calibration accuracy and good real-time performance. Even if the mobile robot cannot accurately reach the fixed position every time, it can still guarantee recognition and positioning accuracy within ±1mm, effectively overcoming the problem of excessively high positioning accuracy requirements for mobile robots in existing technologies, and improving operational accuracy and reliability. It can accurately segment and smooth the edges of tightly stacked materials inside the bin, achieving accurate material grasping. While accurate positioning is achieved through visual calibration, industrial mobile robots, in addition to accurate picking, also need to transport objects. The transport process mostly follows a preset navigation route. During the movement, the road conditions may change, such as the appearance of potholes or obstacles. If the robot follows the original route, it may encounter obstacles. There is a need for a technology that can adjust the driving state of the cargo robot according to real-time road condition information to adapt to the situation. Summary of the Invention

[0005] This invention provides a visual calibration method and system for mobile robots, which can solve the problem of improper obstacle avoidance when mobile robots are carrying goods and encounter obstacles on the route.

[0006] To address the aforementioned technical problems, this application provides the following technical solution: a mobile robot visual calibration system, comprising:

[0007] Image acquisition module: installed on the body of the mobile robot, used at least to acquire image information of the goods to be picked up and the route image of the mobile robot in the direction of movement;

[0008] Radar module: Acquires real-time radar information maps of the area surrounding the mobile robot as it moves;

[0009] Shock absorption module: Installed on the body of the mobile robot, used to adjust the shock absorption of the mobile robot;

[0010] Vibration detection module: used to monitor the vibration information of the mobile robot in real time during its movement;

[0011] Navigation module: Used to plan the route for the mobile robot to reach the delivery location after picking up the goods;

[0012] The data processing module acquires image information of the goods to be picked up during the picking process, processes the edge information of the goods to obtain the gripping point, and then controls the robotic arm on the mobile robot to grasp the goods and place them in the loading area on the mobile robot. It then continues to grasp goods until a quantity threshold is reached. Once the mobile robot is fully loaded, the data processing module begins planning a navigation route. The mobile robot follows the navigation route, and the data processing module simultaneously acquires radar information and vibration information. It processes the route image and radar information to obtain obstacle information, acquiring the length, width, and height of the obstacles. The data processing module compares this information with the mobile robot's preset passability parameters. If the obstacle is impassable, the navigation route is optimized to avoid it. If the obstacle is passable, the mobile robot's speed is reduced to allow passage. If the vibration information value exceeds the threshold during passage, the vibration damping module is controlled to increase until the vibration information value is reduced below the threshold.

[0013] The basic principles and beneficial effects of this solution are as follows: An image acquisition module captures image information of the goods to be picked up and the robot's route image along its movement direction, providing basic data for subsequent processing. A radar module acquires real-time radar information of the surrounding environment during robot movement to detect and identify obstacles. A shock absorption module adjusts the robot's shock absorption performance to protect the goods and the robot itself, especially on uneven surfaces or when traversing obstacles. A vibration monitoring module monitors the robot's vibration in real-time during movement, ensuring the safety of the goods and the stable operation of the robot. A navigation module plans the optimal route for the robot from the pickup point to the delivery point, ensuring efficient and safe delivery of goods. A data processing module processes image information to identify the edges and gripping points of the goods, controlling the robotic arm to grasp and place the goods. It also processes radar and vibration information to identify and avoid obstacles, optimize the navigation route, and adjust the robot's speed and shock absorption performance as needed.

[0014] This solution enhances the autonomous navigation and cargo handling capabilities of mobile robots in complex environments. By processing data from multiple sensors in real time, the robot can adapt to different environmental conditions, optimize path planning, improve operational efficiency, and ensure the safety of both the cargo and the robot itself. This system has broad application prospects in logistics, warehousing, and manufacturing.

[0015] This solution also adjusts the vibration damping module to regulate the chassis height, thereby reducing vibrations generated when the mobile robot crosses obstacles, ensuring the stability of the transported goods, and preventing excessive vibration that could lead to collisions and damage. The solution uses images from two perspectives obtained by the image acquisition module to determine the position of the goods, improving the accuracy of grasping. Simultaneously, by recognizing vibration signals, it optimizes obstacle passage, making the navigation route more rational.

[0016] Furthermore, the navigation module is also used to detect whether there is a slope on the driving route. When the mobile robot is carrying goods, if the weight exceeds the threshold, when driving to a slope section, the data processing module controls the shock absorption module to lower the chassis height of the mobile robot to the lower limit position.

[0017] Beneficial effects: Lowering the center of gravity ensures that the mobile robot can move smoothly up and down slopes.

[0018] Furthermore, it also includes a speed detection module, which is used to acquire the real-time speed of the mobile robot; the data processing module is also used to collect past road condition information. When the mobile robot passes through an obstacle it has passed before, the data processing module controls the shock absorption module to set the optimal parameters for passing through the obstacle in advance. The optimal parameters for passing through the obstacle are obtained from the statistical data of vibration information of the obstacle passed through before. The vibration information is arranged in ascending order and matched with the vehicle speed under the current vibration information. A threshold for the vibration value is set. The vibration information data within the vibration value threshold range is confirmed as qualified. Before the mobile robot passes through the obstacle again, the data processing module matches the current vehicle speed and finds the parameters of the shock absorption module with the lowest vibration value for adjustment.

[0019] Beneficial effects: By retaining data on the way the robot passes through the same obstacle, the optimal (minimum vibration) condition parameters are selected for the next time the mobile robot passes through the obstacle, so that the mobile robot can maintain a stable state during transport.

[0020] Furthermore, when obstacles appear in uphill areas, the data processing module allows the mobile robot's vibration value to reach the upper limit of the vibration threshold range. By adjusting the movement speed, the mobile robot's chassis is positioned at the minimum height required to pass through the obstacle.

[0021] Beneficial effects: On flat roads, the mobile robot chooses the method with the least vibration to pass through, and on uphill and downhill roads, the mobile robot chooses the method with the lowest center of gravity to pass through, so as to ensure the safety of transportation and prevent overturning.

[0022] Furthermore, the data processing module is also used to acquire monitoring image information on the navigation route. By recognizing the monitoring image information, if an insurmountable obstacle is found on the navigation route, the route is directly replanned and an updated navigation route is pushed to the mobile robot.

[0023] Furthermore, if there are impassable obstacles on the navigation route and there are no other routes to plan, a warning signal will be issued to the staff.

[0024] A method for visual calibration of a mobile robot includes the following steps:

[0025] S1: Simultaneously acquire image information of the goods to be picked up using the camera built into the mobile machine and the camera in the pickup area;

[0026] S2: The data processing module obtains the overall location information of the goods to be picked up through the camera in the pickup area, and obtains the specific location information of the goods to be picked up through the camera built into the machine.

[0027] S3: The robotic arm on the mobile robot grabs the goods and places them in the loading area on the mobile robot, and begins the next goods grabbing until the retrieval is completed;

[0028] S4: Transport along the navigation route.

[0029] Furthermore, it also includes step S5: the data processing module identifies obstacles through the images on the navigation route, and if there are impassable obstacles on the navigation route, it changes the navigation route and updates the information to the mobile robot. Attached Figure Description

[0030] Figure 1 This is a schematic diagram of a mobile robot vision calibration system. Detailed Implementation

[0031] The following detailed description illustrates the specific implementation method:

[0032] Example 1 is attached. Figure 1 As shown,

[0033] A mobile robot vision calibration system, comprising:

[0034] Image acquisition module: installed on the body of the mobile robot, used to acquire at least the image information of the goods to be picked up and the route image of the mobile robot in the direction of movement; the image acquisition module includes a camera installed in the picking area (not installed on the mobile robot) and a camera on the mobile robot, which identifies the goods to be picked up and can acquire accurate images of the goods from multiple angles to facilitate the robotic arm of the mobile robot to grasp them. The mobile robot is an existing robot that conforms to the settings of this solution.

[0035] Radar module: Acquires real-time radar information maps of the surrounding environment of the mobile robot as it moves; can identify obstacles in the surrounding environment when the mobile robot is moving.

[0036] Shock absorption module: Installed on the body of the mobile robot, used to adjust the shock absorption of the mobile robot; this solution uses an electrically controlled shock absorption module, which can control the shock absorption adjustment state according to instructions, and can also control the height adjustment of the chassis.

[0037] Vibration monitoring module: used to detect the vibration information of the mobile robot in real time during the movement process; it uses vibration sensors to monitor the vibration signals in real time.

[0038] Navigation module: Used to plan the route for the mobile robot to travel to the delivery location after picking up the goods; that is, the mobile robot has a built-in GPS module that can locate it and then build a transportation route between it and the delivery location.

[0039] The data processing module acquires image information of the goods to be picked up during the picking process, processes the edge information of the goods to obtain the gripping point, and then controls the robotic arm on the mobile robot to grasp the goods and place them in the loading area on the mobile robot. It then continues to grasp goods until a quantity threshold is reached. Once the mobile robot is fully loaded, the data processing module begins planning a navigation route. The mobile robot follows the navigation route, and the data processing module simultaneously acquires radar information and vibration information. It processes the route image and radar information to obtain obstacle information, acquiring the length, width, and height of the obstacles. The data processing module compares this information with the mobile robot's preset passability parameters. If the obstacle is impassable, the navigation route is optimized to avoid it. If the obstacle is passable, the mobile robot's speed is reduced to allow passage. If the vibration information value exceeds the threshold during passage, the vibration damping module is controlled to raise the chassis until the vibration information value is reduced below the threshold.

[0040] The navigation module is also used to detect whether there is a slope on the driving route. When the mobile robot is carrying cargo, if the weight exceeds a threshold, the data processing module controls the shock absorption module to lower the chassis height of the mobile robot to the lower limit position when traveling on a slope. It also includes a speed detection module, which is used to obtain the real-time speed of the mobile robot. The data processing module is also used to collect past road condition information. When the mobile robot passes a previously passed obstacle again, the data processing module controls the shock absorption module to set the optimal parameters for passing the obstacle in advance. The optimal parameters for passing the obstacle are derived from the statistical data of vibration information from past passages of that obstacle. The vibration information is sorted in ascending order and matched with the vehicle speed under the current vibration information. A threshold value for the vibration value is set, and the vibration information data within the threshold value range is considered acceptable. Before the mobile robot passes the obstacle again, the data processing module matches the current vehicle speed and finds the parameters of the shock absorption module with the lowest vibration value for adjustment. When the obstacle appears in an uphill area, the data processing module allows the vibration value of the mobile robot to reach the upper limit of the vibration threshold range, and adjusts the moving speed so that the chassis of the mobile robot is at the minimum height that can pass the obstacle. By matching vibration values ​​with speed and chassis height, the system prioritizes minimizing vibration at the current speed on flat roads and prioritizing the lowest chassis height on inclines and declines. This balances speed, stability, and safety performance.

[0041] The data processing module is also used to acquire monitoring image information along the navigation route. By identifying the monitoring image information, if an impassable obstacle is found on the navigation route, the module directly replans the route and pushes the updated navigation route to the mobile robot. If there is an impassable obstacle on the navigation route and there are no other routes to plan, a warning signal is issued to the staff.

[0042] It also relates to a mobile robot visual calibration method, applicable to the above system, comprising the following steps:

[0043] S1: Simultaneously acquire image information of the goods to be picked up using the camera built into the mobile machine and the camera in the pickup area;

[0044] S2: The data processing module obtains the overall location information of the goods to be picked up through the camera in the pickup area, and obtains the specific location information of the goods to be picked up through the camera built into the machine.

[0045] S3: The robotic arm on the mobile robot grabs the goods and places them in the loading area on the mobile robot, and begins the next goods grabbing until the retrieval is completed;

[0046] S4: Transport along the navigation route.

[0047] Step S5: The data processing module identifies obstacles through the images on the navigation route. If there are impassable obstacles on the navigation route, the navigation route is changed and updated to the mobile robot.

[0048] Example 2

[0049] The difference between Example 2 and Example 1 is that when the mobile robot is going downhill, if an obstacle appears on the downhill slope, the chassis of the mobile robot is lowered to a height below the maximum vibration threshold before going downhill. During the downhill journey, if the data processing module detects that the real-time speed transmitted from the speed detection module is in an acceleration state, and the speed exceeds 20% of the speed at the beginning of the downhill journey within the threshold, the chassis height is lowered again to the minimum height (e.g., when traveling on uneven roads).

[0050] If the mobile robot stalls, it lowers its chassis again to lower its center of gravity and prevent it from tipping over. At the same time, the increased vibration during the chassis lowering process reduces the robot's speed and lowers the probability of an accident.

[0051] The above are merely embodiments of the present invention. The invention is not limited to the fields covered by these embodiments. Commonly known structures and characteristics in the solutions are not described in detail here. Those skilled in the art are aware of all common technical knowledge in the field prior to the application date or priority date, are able to access all existing technologies in that field, and have the ability to apply conventional experimental methods prior to that date. Those skilled in the art can, under the guidance of this application, improve and implement this solution in combination with their own capabilities. Some typical known structures or methods should not be obstacles for those skilled in the art to implement this application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the structure of the present invention. These should also be considered within the scope of protection of the present invention, and will not affect the effectiveness of the implementation of the present invention or the practicality of the patent. The scope of protection claimed in this application should be determined by the content of its claims, and the specific embodiments described in the specification can be used to interpret the content of the claims.

Claims

1. A mobile robot visual calibration system, characterized in that, include: Image acquisition module: includes a camera installed on the body of the mobile robot, used to acquire at least image information of the goods to be picked up and the route image of the mobile robot in the direction of movement when it moves, and also includes a camera installed in the picking area, used to acquire image information of the goods to be picked up at the same time; Radar module: Acquires real-time radar information maps of the area surrounding the mobile robot as it moves; Shock absorption module: Installed on the body of the mobile robot, it is used to adjust the shock absorption performance of the mobile robot, control the shock absorption adjustment state according to the command, and control the height adjustment of the chassis at the same time. Vibration detection module: used to detect vibration information of the mobile robot in real time during its movement; Navigation module: Used to plan the route for the mobile robot to reach the delivery location after picking up the goods; The data processing module acquires image information of the goods to be picked up during the picking process, processes the edge information of the goods to obtain the gripping point, and then controls the robotic arm on the mobile robot to grasp the goods and place them in the loading area on the mobile robot. It then continues to grasp goods until a quantity threshold is reached. Once the mobile robot is fully loaded, the data processing module begins planning a navigation route. The mobile robot follows the navigation route, and the data processing module simultaneously acquires radar information and vibration information. It processes the route image and radar information to obtain obstacle information, acquiring the length, width, and height of the obstacles. The data processing module compares this information with the mobile robot's preset passability parameters. If the obstacle is impassable, the navigation route is optimized to avoid it. If the obstacle is passable, the mobile robot's speed is reduced to pass through. If the vibration information value exceeds the threshold during the passage, the vibration damping module is controlled to raise the chassis until the vibration information value is reduced below the threshold. The navigation module is also used to detect whether there is a slope on the driving route. When the mobile robot is carrying goods, if the weight exceeds the threshold, when driving to a slope section, the data processing module controls the shock absorption module to lower the chassis height of the mobile robot to the lower limit position. It also includes a speed detection module, which is used to obtain the real-time speed of the mobile robot; the data processing module is also used to collect past road condition information. When the mobile robot passes through an obstacle it has passed before, the data processing module controls the shock absorption module to set the optimal parameters for passing through the obstacle in advance. The optimal parameters for passing through the obstacle are obtained from the statistical data of vibration information of the obstacle passed through before. The vibration information is sorted in ascending order and matched with the vehicle speed under the current vibration information. A threshold for the vibration value is set. The vibration information data within the vibration value threshold range is confirmed as qualified. Before the mobile robot passes through the obstacle again, the data processing module matches the current vehicle speed and finds the parameters of the shock absorption module with the lowest vibration value for adjustment. When an obstacle appears in an uphill area, the data processing module allows the mobile robot's vibration value to reach the upper limit of the vibration threshold range. By adjusting the moving speed, the mobile robot's chassis is positioned at the minimum height required to pass through the obstacle.

2. The mobile robot vision calibration system according to claim 1, characterized in that: The data processing module is also used to acquire monitoring image information on the navigation route. If an insurmountable obstacle is found on the navigation route by recognizing the monitoring image information, the route is directly replanned and an updated navigation route is pushed to the mobile robot.

3. The mobile robot vision calibration system according to claim 2, characterized in that: If there are impassable obstacles on the navigation route and there are no other routes to plan, a warning signal will be sent to the staff.

4. A mobile robot visual calibration method, applicable to the system described in any one of claims 1-3: characterized in that: Includes the following steps: S1: Simultaneously acquire image information of the goods to be picked up using the camera built into the mobile machine and the camera in the pickup area; S2: The data processing module obtains the overall location information of the goods to be picked up through the camera in the pickup area, and obtains the specific location information of the goods to be picked up through the camera built into the machine. S3: The robotic arm on the mobile robot grabs the goods and places them in the loading area on the mobile robot, and begins the next goods grabbing until the retrieval is completed; S4: Transport along the navigation route.

5. A mobile robot visual calibration method according to claim 4, characterized in that: It also includes step S5: The data processing module identifies obstacles through the images on the navigation route. If there are impassable obstacles on the navigation route, the navigation route is changed and updated to the mobile robot.

Images