Visual calibration method and system for mobile robot

Abstract

The invention relates to the field of industrial robots, in particular to a mobile robot vision calibration system, which comprises an image acquisition module arranged on a machine body of a mobile robot and at least used for acquiring image information of goods to be taken and a route image in a moving direction when the mobile robot moves; when the mobile robot moves, the radar module is used for acquiring a Zhou five radar information graph of the mobile robot in real time; the damping module is arranged on a machine body of the mobile robot and used for adjusting damping of the mobile robot; the vibration detection module is used for detecting vibration information of the mobile robot in the moving process in real time; and the navigation module is used for planning a route from the mobile robot to the delivery place after taking the goods. The invention further relates to a visual calibration method of the mobile robot. The visual calibration method comprises the steps that S1, image information of goods to be taken is obtained through the camera of the mobile robot body and the camera of the goods taking area at the same time, and the problem that the mobile robot encounters obstacles on a route during goods carrying movement and improper avoidance can be solved.

Description

Technical Field

[0001] The present invention relates to the field of industrial robots, and in particular to a mobile robot vision calibration method and system. Background Art

[0002] Robots are essential basic production equipment in the fields of aviation, aerospace, national defense, and mechanical manufacturing. The integration and application of robot systems are moving towards intelligence, and the intelligence of robot systems is often reflected in the integration of high-precision vision systems at the end of the robot. After the robot integrates the vision system, the vision system usually establishes a corresponding pose matrix relationship with the robot base coordinate system and the tool coordinate system. The calibration of the robot's vision system and the tool coordinate system (commonly known as "hand-eye" calibration) is an important step for the robot to rely on the vision system to guide the end tool to complete the task. The accuracy of the "hand-eye" calibration determines whether the vision system can accurately complete the spatial position guidance of the robot.

[0003] The invention patent with application number 202411370704.8 discloses a mobile robot visual calibration method and system, an operation method and system, wherein the mobile robot visual calibration method includes: when the robot is in the operating position, performing the following steps: step S1, obtaining the transformation matrix Tb1 from the camera coordinate system to the first calibration plate coordinate system, and the robot's current photo posture matrix P1; wherein the first calibration plate is located on the robot's mechanical arm; step S2, based on Tx, Tb1, and P1, obtaining the transformation matrix T1 from the camera coordinate system to the robot coordinate system, and completing the calibration; wherein Tx is the transformation matrix from the first calibration plate coordinate system to the robot coordinate system. The present invention is suitable for visual guidance operations of mobile robots with eyes outside the hands, can expand the camera shooting range, has high visual calibration accuracy and good real-time performance, and can ensure recognition and positioning accuracy even if the mobile robot cannot accurately reach a fixed position every time, thereby improving the operation accuracy and reliability.

[0004] The above technology is suitable for visual guidance operations of mobile robots with eyes outside hands. It can expand the camera shooting range, and has high visual calibration accuracy and good real-time performance. Even if the mobile robot cannot accurately reach a fixed position every time, it can ensure that the recognition and positioning accuracy is plus or minus 1mm, effectively overcoming the problem of excessively high requirements for the positioning accuracy of mobile robots in the existing technology, and improving the operation accuracy and reliability; it can accurately segment and smooth the edges of tightly stacked materials inside the material box to achieve accurate material grabbing. Precise positioning is achieved through visual calibration, but in addition to accurate picking, industrial mobile robots also need to transport objects, and the transportation process mostly uses a preset navigation route for movement. The road environment may change during the movement. For example, when there are potholes or obstacles, the robot walks along the original route, which may cause obstacles to the robot's movement. Now a technology is needed that can adjust the driving state of the cargo robot according to real-time road conditions information for adaptation. Summary of the invention

[0005] The present invention provides a mobile robot visual calibration method and system, which can solve the problem of improper obstacle avoidance on the route encountered by the mobile robot when carrying cargo.

[0006] In order to solve the above technical problems, the present application provides the following technical solutions: a mobile robot visual calibration system, comprising: Image acquisition module: arranged on the body of the mobile robot, used to acquire at least the image information of the goods to be picked up and the route image in the moving direction of the mobile robot; Radar module: When the mobile robot moves, it obtains radar information maps around the mobile robot in real time; Shock absorption module: arranged on the body of the mobile robot, used to adjust the shock absorption of the mobile robot; Vibration detection module: used to monitor the vibration information of the mobile robot in real time during movement; Navigation module: used to plan the route for the mobile robot to the delivery location after picking up the goods; Data processing module: when picking up goods, it is used to obtain image information of the goods to be picked up, process the edge information of the goods to be picked up, obtain the grabbing points, and then control the manipulator on the mobile robot to grab the goods to be picked up and place them in the loading area on the mobile robot, and then continue to grab the goods until the quantity threshold is reached; when the mobile robot completes loading of goods, the data processing module starts to plan the navigation route, and the mobile robot follows the navigation route. The data processing module simultaneously obtains radar information maps and vibration information. The data processing module processes the route image and radar information to obtain the obstacle information of the route, and obtains the three-dimensional values ​​of the length, width and height of the obstacle on the route. The data processing module compares according to the preset passability parameters of the mobile robot. If the obstacle is an impassable obstacle, the navigation route is optimized to avoid the obstacle; if the obstacle is passable, the moving speed of the mobile robot is reduced to pass through. If the vibration information value exceeds the threshold during the passing process, the shock absorption module is controlled to rise until the vibration information value drops below the threshold.

[0007] Basic principles and beneficial effects of this solution: This solution includes an image acquisition module: responsible for capturing image information of the goods to be picked up and the route image in the direction of robot movement, providing basic data for subsequent processing. Radar module: real-time acquisition of radar information map of the surrounding environment during the movement of the robot, used to detect and identify obstacles. Shock absorption module: used to adjust the shock absorption performance of the robot to protect the goods and the robot itself, especially on uneven roads or when passing obstacles. Vibration monitoring module: real-time monitoring of the vibration of the robot during movement to ensure the safety of the goods and the stable operation of the robot. Navigation module: planning the optimal route of the robot from the pickup point to the delivery point to ensure that the goods can be delivered efficiently and safely. Data processing module: responsible for processing image information to identify the edges and grasping points of the goods, and controlling the manipulator to grasp and place the goods. At the same time, it is also responsible for processing radar information and vibration information to identify and avoid obstacles, optimize the navigation route, and adjust the speed and shock absorption performance of the robot when necessary.

[0008] This solution can improve the autonomous navigation and cargo handling capabilities of mobile robots in complex environments. By processing multiple sensor data in real time, the robot can adapt to different environmental conditions, optimize path planning, improve operating efficiency, and ensure the safety of cargo and the robot itself. This system has broad application prospects in logistics, warehousing, manufacturing and other fields.

[0009] This solution also adjusts the vibration reduction module to adjust the chassis height, thereby reducing the vibration generated when the mobile robot moves over obstacles, ensuring the stability of the goods being transported, and preventing excessive vibration from causing collisions between goods and damage to the goods. This solution determines the position of the goods through the images of two perspectives obtained by the image acquisition module, improving the accuracy of the grasping. At the same time, through the recognition of vibration signals, it optimizes the passage of obstacles and makes the navigation route more reasonable.

[0010] Furthermore, the navigation module is also used to detect whether there is a slope on the driving route. When the mobile robot is carrying cargo, if the weight exceeds a threshold, when driving on a sloped section, the data processing module controls the shock absorption module to lower the chassis height of the mobile robot to the lower limit position.

[0011] Beneficial effect: Lowers the center of gravity, ensuring that the mobile robot can go up and down slopes smoothly.

[0012] Furthermore, it also includes a speed detection module, which is used to obtain the real-time speed of the mobile robot; the data processing module is also used to collect past road condition information, and the next time the mobile robot passes through an obstacle that it has passed before, the data processing module controls the shock absorption module in advance to set the optimal parameters for passing the obstacle; the shock absorption module sets the optimal parameters for passing the obstacle based on the data statistics of vibration information of previous passages through the obstacle, arranges the vibration information in ascending order, and matches the vehicle speed under the current vibration information, sets a threshold for the vibration value, and confirms that the vibration information data is qualified within the vibration value threshold range, and before the mobile robot passes through the obstacle next time, the data processing module matches the current vehicle speed, finds the parameters of the shock absorption module under the lowest vibration value for adjustment.

[0013] Beneficial effect: Data on the way of passing the same obstacle is retained, and the optimal (least vibration) condition parameters are selected for setting before the next time the mobile robot passes the obstacle, so that the mobile robot can move in a stable state during transportation.

[0014] Furthermore, when an obstacle appears in an uphill area, the data processing module allows the vibration value of the mobile robot to reach the upper limit of the vibration threshold range, and by adjusting the moving speed, the chassis of the mobile robot is at the lowest height that can pass the obstacle.

[0015] Beneficial effect: When on a flat road, the mobile robot chooses the method with the least vibration to pass through. When on uphill and downhill sections, the mobile robot chooses the method with the lowest center of gravity to pass through, so as to ensure the safety of transportation and prevent overturning.

[0016] Furthermore, the data processing module is also used to obtain monitoring image information on the navigation route. By identifying the monitoring image information, if it is found that there are impassable obstacles on the navigation route, the route is directly replanned and the updated navigation route is pushed to the mobile robot.

[0017] Furthermore, if there are insurmountable obstacles on the navigation route and there are no other routes that can be planned, a warning signal will be sent to the staff.

[0018] A mobile robot visual calibration method comprises the following steps: S1: The image information of the goods to be picked up is simultaneously obtained by using the camera on the mobile machine body and the camera in the picking area; S2: The data processing module obtains the overall location information of the goods to be picked up through the camera in the pickup area, and obtains the specific location information of the goods to be picked up through the camera on the body; S3: The robot arm on the mobile robot grabs the goods and places them in the cargo loading area of ​​the mobile robot, and starts the next cargo grabbing until the cargo is picked up; S4: Transport along the navigation route.

[0019] Furthermore, the method also includes step S5: the data processing module identifies obstacles through the image screen on the navigation route. If there are impassable obstacles on the navigation route, the navigation route is changed and updated to the mobile robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 Schematic diagram of a mobile robot vision calibration system. DETAILED DESCRIPTION

[0021] The following is further described in detail through specific implementation methods: Embodiment 1 is as attached Figure 1 As shown, A mobile robot visual calibration system, comprising: Image acquisition module: It is set on the body of the mobile robot and is used to obtain at least the image information of the goods to be picked up and the route image of the moving direction of the mobile robot; the image acquisition module includes a camera set in the picking area (not set on the mobile robot) and a camera on the mobile robot, which can identify the goods to be picked up and can obtain accurate images of the goods from multiple angles to facilitate the mobile robot's mechanical arm to grasp. The mobile robot uses an existing robot that meets the settings of this solution.

[0022] Radar module: When the mobile robot moves, it can obtain radar information maps around the mobile robot in real time; it can identify obstacles in the surrounding environment when the mobile robot is moving.

[0023] Shock absorption module: It is installed on the body of the mobile robot and is used to adjust the shock absorption of the mobile robot. This solution uses an electrically controlled shock absorption module, which can control the state of the shock absorption adjustment according to instructions, and can also control the height of the chassis.

[0024] Vibration monitoring module: used to detect the vibration information of the mobile robot in real time during movement; using vibration sensors to monitor vibration signals in real time.

[0025] Navigation module: used to plan the route for the mobile robot to the delivery location after picking up the goods; that is, the mobile robot has a built-in GPS module that can locate it and then build a transportation route between it and the delivery location.

[0026] Data processing module: when picking up goods, it is used to obtain image information of the goods to be picked up, process the edge information of the goods to be picked up, obtain the grabbing points, and then control the manipulator on the mobile robot to grab the goods to be picked up and place them in the loading area on the mobile robot, and then continue to grab the goods until the quantity threshold is reached; when the mobile robot completes loading of goods, the data processing module starts to plan the navigation route, and the mobile robot sets off along the navigation route. The data processing module simultaneously obtains radar information maps and vibration information. The data processing module processes the route image and radar information to obtain the obstacle information on the route, and obtains the three-dimensional values ​​of the length, width and height of the obstacle on the route. The data processing module compares them according to the preset passability parameters of the mobile robot. If the obstacle is an impassable obstacle, the navigation route is optimized to avoid the obstacle; if the obstacle is passable, the moving speed of the mobile robot is reduced to pass through. If the vibration information value exceeds the threshold during the passing process, the shock absorption module is controlled to lift the chassis until the vibration information value drops below the threshold.

[0027] The navigation module is also used to detect whether there is a slope on the driving route. When the mobile robot is carrying cargo, if the weight exceeds the threshold, when driving to the slope section, the data processing module controls the shock absorption module to lower the chassis height of the mobile robot to the lower limit position. It also includes a speed detection module, which is used to obtain the real-time speed of the mobile robot; the data processing module is also used to collect past road condition information. The next time the mobile robot passes through an obstacle that has been passed before, the data processing module controls the shock absorption module in advance to set the optimal parameters for passing the obstacle; the shock absorption module sets the optimal parameters for passing the obstacle from the data statistics of the vibration information of the obstacle that has been passed before, arranges the vibration information in ascending order, and matches the vehicle speed under the current vibration information, sets the threshold of the vibration value, and confirms that the vibration information data is qualified within the vibration value threshold range. Before the mobile robot passes the obstacle next time, the data processing module matches the current vehicle speed and finds the parameters of the shock absorption module under the lowest vibration value for adjustment. When the obstacle appears in the uphill area, the data processing module allows the vibration value of the mobile robot to reach the upper limit of the vibration threshold range, and adjusts the moving speed so that the chassis of the mobile robot is at the lowest height that can pass the obstacle. By matching the vibration value-speed and chassis height, the vehicle is given priority when passing through a flat road with the minimum vibration value within the current speed range, and when passing through uphill and downhill roads with the lowest chassis, which takes into account speed, stability and safety performance.

[0028] The data processing module is also used to obtain monitoring image information on the navigation route. By identifying the monitoring image information, if an impassable obstacle is found on the navigation route, the route is directly re-planned and the updated navigation route is pushed to the mobile robot. If there is an impassable obstacle on the navigation route and there is no other route that can be planned, a warning signal is sent to the staff.

[0029] Also disclosed is a mobile robot visual calibration method, which is applicable to the above system and comprises the following steps: S1: The image information of the goods to be picked up is simultaneously obtained by using the camera on the mobile machine body and the camera in the picking area; S2: The data processing module obtains the overall location information of the goods to be picked up through the camera in the pickup area, and obtains the specific location information of the goods to be picked up through the camera on the body; S3: The robot arm on the mobile robot grabs the goods and places them in the cargo loading area of ​​the mobile robot, and starts the next cargo grabbing until the cargo is picked up; S4: Transport along the navigation route.

[0030] Step S5: The data processing module identifies obstacles through the image screen on the navigation route. If there are impassable obstacles on the navigation route, the navigation route is changed and updated to the mobile robot.

[0031] Embodiment 2 The difference between the second embodiment and the first embodiment is that when the mobile robot is going downhill, if an obstacle appears on the downhill slope, the chassis of the mobile robot is lowered to a height below the maximum limit of the vibration threshold before going downhill. During the downhill driving process, if the data processing module detects that the real-time speed transmitted by the speed detection module is in an accelerated state, and the speed exceeds 20% of the speed just downhill within the threshold, the chassis height is lowered again to the minimum height (for example, when driving on a bumpy road).

[0032] In the case of a stall, the mobile robot lowers the chassis again to lower the center of gravity and prevent the mobile robot from overturning. At the same time, in the process of lowering the chassis, the vibration is increased to reduce the speed of the mobile robot and reduce the probability of accidents.

[0033] The above are only embodiments of the present invention. The invention is not limited to the fields involved in this implementation case. The common sense such as the known specific structures and characteristics in the scheme is not described in detail here. The ordinary technicians in the relevant field know all the common technical knowledge in the technical field to which the invention belongs before the application date or the priority date, can obtain all the existing technologies in the field, and have the ability to apply the conventional experimental means before that date. The ordinary technicians in the relevant field can improve and implement this scheme in combination with their own abilities under the enlightenment given by this application. Some typical known structures or known methods should not become obstacles for ordinary technicians in the relevant field to implement this application. It should be pointed out that for those skilled in the art, without departing from the structure of the present invention, several deformations and improvements can be made, which should also be regarded as the protection scope of the present invention, which will not affect the effect of the implementation of the present invention and the practicality of the patent. The protection scope required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to explain the content of the claims.

Claims

1. A mobile robot visual calibration system, characterized in that: include: Image acquisition module: arranged on the body of the mobile robot, used to acquire at least the image information of the goods to be picked up and the route image in the moving direction of the mobile robot; Radar module: When the mobile robot moves, it obtains radar information maps around the mobile robot in real time; Shock absorption module: arranged on the body of the mobile robot, used to adjust the shock absorption of the mobile robot; Vibration detection module: used to detect the vibration information of the mobile robot in real time during movement; Navigation module: used to plan the route for the mobile robot to the delivery location after picking up the goods; Data processing module: when picking up goods, it is used to obtain image information of the goods to be picked up, process the edge information of the goods to be picked up, obtain the grabbing points, and then control the manipulator on the mobile robot to grab the goods to be picked up and place them in the loading area on the mobile robot, and then continue to grab the goods until the quantity threshold is reached; when the mobile robot completes loading of goods, the data processing module starts to plan the navigation route, and the mobile robot sets off along the navigation route. The data processing module simultaneously obtains radar information maps and vibration information. The data processing module processes the route image and radar information to obtain the obstacle information on the route, and obtains the three-dimensional values ​​of the length, width and height of the obstacle on the route. The data processing module compares them according to the preset passability parameters of the mobile robot. If the obstacle is an impassable obstacle, the navigation route is optimized to avoid the obstacle; if the obstacle is passable, the moving speed of the mobile robot is reduced to pass through. If the vibration information value exceeds the threshold during the passing process, the shock absorption module is controlled to lift the chassis until the vibration information value drops below the threshold.

2. A mobile robot visual calibration system according to claim 1, characterized in that: The navigation module is also used to detect whether there is a slope on the driving route. When the mobile robot is carrying cargo, if the weight exceeds a threshold, when driving on a sloped section, the data processing module controls the shock absorbing module to lower the chassis height of the mobile robot to the lower limit position.

3. A mobile robot visual calibration method and system according to claim 2, characterized in that: It also includes a speed detection module, which is used to obtain the real-time speed of the mobile robot; the data processing module is also used to collect past road condition information, and the next time the mobile robot passes through an obstacle that it has passed before, the data processing module controls the shock absorption module in advance to set the optimal parameters for passing the obstacle; the shock absorption module sets the optimal parameters for passing the obstacle based on data statistics of vibration information of previous passages through the obstacle, arranges the vibration information in ascending order, matches the vehicle speed under the current vibration information, sets a threshold for the vibration value, and confirms that the vibration information data is qualified within the vibration value threshold range, and before the mobile robot passes through the obstacle next time, the data processing module matches the current vehicle speed, finds the parameters of the shock absorption module under the lowest vibration value for adjustment.

4. A mobile robot visual calibration system according to claim 3, characterized in that: When an obstacle appears in an uphill area, the data processing module allows the vibration value of the mobile robot to reach the upper limit of the vibration threshold range, and by adjusting the moving speed, the chassis of the mobile robot is at the lowest height that can pass the obstacle.

5. The mobile robot visual calibration system according to claim 1, characterized in that: The data processing module is also used to obtain monitoring image information on the navigation route. By identifying the monitoring image information, if it is found that there are impassable obstacles on the navigation route, the route is directly replanned and the updated navigation route is pushed to the mobile robot.

6. A mobile robot visual calibration system according to claim 5, characterized in that: If there are insurmountable obstacles on the navigation route and no other routes can be planned, a warning signal will be sent to the staff.

7. A mobile robot visual calibration method, applicable to the system according to any one of claims 1 to 6, characterized in that: The following steps are involved: S1: The image information of the goods to be picked up is simultaneously obtained by using the camera on the mobile machine body and the camera in the picking area; S2: The data processing module obtains the overall location information of the goods to be picked up through the camera in the pickup area, and obtains the specific location information of the goods to be picked up through the camera on the body; S3: The robot grabs the goods through the robot arm on the mobile robot and places them in the cargo loading area of ​​the mobile robot, and starts the next cargo grabbing until the cargo is picked up; S4: Transport along the navigation route.

8. A mobile robot visual calibration method according to claim 7, characterized in that: The method also includes step S5: the data processing module identifies obstacles through the image screen on the navigation route. If there are impassable obstacles on the navigation route, the navigation route is changed and updated to the mobile robot.

Images