A robot and a robot work interfacing method
By acquiring images of the calibration object using a vision sensor, controlling the robot to move to a preset position, and confirming the docking with a limit switch sensor, the problem of low positioning accuracy and high cost of laser sensors is solved, achieving high-precision and low-cost robot docking.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHENZHEN PUDU TECH CO LTD
- Filing Date
- 2021-12-09
- Publication Date
- 2026-08-04
AI Technical Summary
When adding water or charging, existing robots suffer from low positioning accuracy and high cost due to the laser sensors, and docking is prone to failure.
A vision sensor is used to acquire images of the calibration object on the docking target. The vision sensor controls the robot to move to the preset position and dock. The docking is confirmed to be successful by combining the limit switch sensor.
It reduced positioning costs, improved docking accuracy, and reduced docking failure rate.
Smart Images
Figure CN116257047B_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of robotics technology, specifically relating to a robot and a robot operation docking method. Background Technology
[0002] With the development of the internet age, intelligent robots are increasingly entering people's lives to help them accomplish many tasks. Typical examples include cleaning robots (sweepers, mops), service robots (food delivery robots, refueling robots, shopping guide robots), and entertainment robots (chatbots), among others.
[0003] Taking cleaning robots as an example, when a robot encounters a shortage of clean water or power while completing its cleaning work, it will autonomously add water or recharge itself. This process usually requires the robot to connect with a water station or charging station.
[0004] Currently, when robots need to be refilled with water or charged, they mainly use laser sensors installed on the robots to locate the robot's position relative to the water station or charging station. However, laser sensors are expensive, require reflectors to achieve high-precision positioning, and the number of point clouds collected by laser sensors is relatively small and easily affected by strong light, resulting in low positioning accuracy and causing docking failure. Summary of the Invention
[0005] In view of this, this application provides a robot and a robot operation docking method, which can solve the problems of high robot positioning cost and easy docking failure.
[0006] A first aspect of this application provides a robot, including a memory and a processor. The memory stores executable program code, and the processor executes the executable code to perform the following steps:
[0007] When the robot moves to the target position, a first image containing a calibration object is acquired by a vision sensor, wherein the calibration object is set on the docking target of the robot;
[0008] Based on the first image containing the calibration object, control the robot to move from the target position to a first preset position;
[0009] When the robot is determined to have moved to the first preset position, a second image containing the calibration object is acquired through a vision sensor;
[0010] Based on the second image containing the calibration object, the robot obtains a first relative pose relative to the calibration object. Based on the first relative pose, the robot obtains a first direction and then controls the robot to move along the first direction to dock with the docking target.
[0011] In another implementation of the first aspect,
[0012] The robot or the docking target is equipped with a sensor, which is used to send a docking success message when the robot and the docking target successfully dock. The processor is used to execute the executable code to implement the following steps:
[0013] During the process of controlling the robot to move along the first direction to dock with the docking target, when the docking success information is received, the robot is controlled to stop moving.
[0014] In another implementation of the first aspect, a sensor is provided on the robot or the docking target. When the robot and the docking target successfully dock, the sensor emits a docking success message. Controlling the robot to dock with the docking target along the first direction includes:
[0015] Control the robot to move a first distance along the first direction;
[0016] During the process of the robot moving the first distance along the first direction, when the docking success information is received, the robot is controlled to stop moving;
[0017] If the robot does not receive the docking success information after moving the first distance along the first direction, the robot is controlled to stop moving.
[0018] In another implementation of the first aspect, controlling the robot to move from the target position to a first preset position based on the first image containing the calibration object includes:
[0019] Based on the first image containing the calibration object, the second relative pose of the robot with respect to the calibration object is obtained;
[0020] Based on the second relative pose and the preset relative pose, a first target path is obtained from the target position to the first preset position. The preset relative pose is the preset relative pose of the robot relative to the calibration object when the robot is located at the first preset position.
[0021] Control the robot to move from the target location along the first target path to the first preset location.
[0022] In another implementation of the first aspect, controlling the robot to move from the target position to the first preset position along the first target path includes:
[0023] Whenever the visual sensor continuously acquires a preset number of third images containing the calibration object at a preset acquisition frequency, or whenever the robot moves a preset distance, or whenever the robot moves for a preset time interval, the relative pose of the robot with respect to the calibration object is determined when the robot is at its current position.
[0024] Based on the robot's relative pose to the calibration object and the preset relative pose when the robot is in its current position, the path from the robot's current position to the first preset position in the first target path is adjusted to obtain the target path.
[0025] In a first implementation of the first aspect, the relative pose of the robot with respect to the calibration object when the visual sensor continuously acquires a preset number of third images containing the calibration object at a preset acquisition frequency, and determines that the robot is at its current position, includes:
[0026] Based on each of the third images containing the calibration object, a third relative pose of the robot relative to the calibration object corresponding to each of the third images is obtained;
[0027] The obtained third relative pose of the robot relative to the calibration object corresponding to each of the third images is smoothed to obtain a weighted pose, and the weighted pose is used as the relative pose of the robot relative to the calibration object when the robot is in the current position.
[0028] In another implementation of the first aspect, smoothing the obtained third relative pose of the robot relative to the calibration object corresponding to each of the third images to obtain a weighted pose includes:
[0029] The weighted pose is calculated based on the third relative pose of the robot relative to the calibration object corresponding to each of the third images, and the preset weight of the third relative pose of the robot relative to the calibration object corresponding to each of the third images.
[0030] In another implementation of the first aspect, after obtaining the first relative pose of the robot relative to the calibration object and before controlling the robot to move a first distance along a first direction, the processor performs the following steps when executing the executable code:
[0031] A second distance between the robot and the docking target is acquired by a laser sensor mounted on the robot.
[0032] The third distance between the robot and the calibration object is obtained based on the first relative pose.
[0033] Calculate a first difference between the second distance and the third distance. When the first difference meets a first preset range, control the robot to move a first distance along the first direction.
[0034] If the first difference does not meet the first preset range, the robot is controlled to move to the target position.
[0035] In another implementation of the first aspect, after obtaining the first relative pose of the robot relative to the calibration object and before controlling the robot to move a first distance along the first direction, the processor performs the following steps when executing the executable code:
[0036] Calculate a second difference between the first relative pose and the preset relative pose. When the second difference meets a second preset range, control the robot to move the first distance along the first direction.
[0037] If the second difference does not meet the second preset range, the robot is controlled to move to the target position.
[0038] In another implementation of the first aspect, after the robot has moved the first distance along the first direction and has not received the docking success information, and after controlling the robot to stop moving, the processor executes the executable code by performing the following steps:
[0039] Obtain the fourth distance between the robot and the docking target;
[0040] If the fourth distance is greater than the first threshold, then the robot is controlled to move a fifth distance along the robot's heading, wherein the fifth distance is the difference between the fourth distance and the first threshold;
[0041] If the fourth distance is less than or equal to the first threshold, then control the robot to move to the target position.
[0042] In another implementation of the first aspect, the processor performs the following steps when executing the executable code:
[0043] The point cloud data collected by the laser sensor is processed using the SLAM method to obtain a map. The laser sensor is mounted on a robot, and the map is a map of the area where the robot and the docking target are located.
[0044] A second aspect of this application provides a robot docking device, comprising:
[0045] The first acquisition module is used to acquire a first image containing a calibration object through a vision sensor when it is determined that the robot has moved to the target position. The vision sensor is mounted on the robot, and the calibration object is set on the docking target of the robot.
[0046] The first control module is used to control the robot to move from the target position to a first preset position based on the first image containing the calibration object;
[0047] The second acquisition module is used to acquire a second image containing the calibration object through a vision sensor when the robot moves to the first preset position.
[0048] The second control module is used to obtain the first relative pose of the robot relative to the calibration object based on the second image containing the calibration object, and after obtaining the first direction based on the first relative pose, control the robot to move along the first direction to dock with the docking target.
[0049] A third aspect of this application provides a computer-readable storage medium storing a computer program that, when run on a processor, implements the method described in any of the first aspects of this application.
[0050] In the process of docking a robot with a target, this application primarily uses an image of a marker on the target acquired by a vision sensor mounted on the robot to control the robot to reach a first preset position. After reaching the first preset position, the robot is controlled to move to achieve the docking operation between the robot and the target. Because a vision sensor, which is less expensive than a laser sensor, is used, costs are reduced. Furthermore, the information acquired by the vision sensor is an image of the marker, which is less susceptible to interference from strong light, resulting in high positioning accuracy during the positioning process and reducing the docking failure rate.
[0051] It is understood that the beneficial effects of the second and third aspects mentioned above can be found in the relevant descriptions in the first aspect above, and will not be repeated here. Attached Figure Description
[0052] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings, without exceeding the scope of protection claimed by this application.
[0053] Figure 1 This diagram shows a schematic block diagram of the structure of a robot provided in an embodiment of this application;
[0054] Figure 2 This illustration shows a flowchart of a robot operation docking method provided in an embodiment of this application;
[0055] Figure 3 This diagram illustrates a method for establishing a world coordinate system according to an embodiment of this application.
[0056] Figure 4 This illustration shows a flowchart of a robot-based task docking method according to another embodiment of this application;
[0057] Figure 5 This illustration shows a flowchart of a robot-based task docking method according to another embodiment of this application;
[0058] Figure 6 This illustration shows a flowchart of a robot-based task docking method according to another embodiment of this application;
[0059] Figure 7 This illustration shows a flowchart of a robot-based task docking method according to another embodiment of this application;
[0060] Figure 8 This illustration shows a flowchart of a robot-based task docking method according to another embodiment of this application;
[0061] Figure 9 This illustration shows a flowchart of a robot-based task docking method according to another embodiment of this application;
[0062] Figure 10 This illustration shows a flowchart of a robot-based task docking method according to another embodiment of this application;
[0063] Figure 11 This is a schematic block diagram showing the composition of a robot docking device provided in an embodiment of this application. Detailed Implementation
[0064] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0065] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.
[0066] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of exemplary embodiments and may not be to scale. The modules or processes shown in the drawings are not necessarily essential for implementing this application and therefore should not be used to limit the scope of protection of this application.
[0067] See Figure 1 This is a schematic diagram of the robot provided in an embodiment of this application. For ease of explanation, only the parts related to the embodiment of this application are shown. The robot includes:
[0068] The processor 110 and memory 120 are used. The processor 110 is the core of the robot's computation and control, and is the final execution unit for information processing and program execution. The memory 120 can be, for example, a hard disk drive, non-volatile memory (such as flash memory or other electronically programmable limited memory used to form a solid-state drive), volatile memory (such as static or dynamic random access memory), etc., and is not limited in this embodiment. The memory 120 stores executable program code; the processor 110, linked to the memory 120, calls the executable program code stored in the memory 120 to execute the following robot operation docking method.
[0069] See Figure 2 Figure 1 is a flowchart illustrating a robot task docking method provided in an embodiment of this application. As shown in the figure, the task docking method may include the following steps:
[0070] S21, when the robot reaches the target position, a first image containing the calibration object is acquired by a vision sensor, wherein the vision sensor is mounted on the robot and the calibration object is set on the robot's docking target.
[0071] In this embodiment, when the robot needs to dock with a target, it first reaches the target location, which is the location where a marker exists within the field of view of the visual sensor. In one embodiment, the target location can be a location pre-deployed on a map; it is understood that in other embodiments, the target location can also be a location determined by the robot during its movement.
[0072] When the robot reaches the target position, the vision sensor on the robot can acquire an image of the marker located on the docking target. The vision sensor can be a CCD sensor, a CMOS sensor, etc., and this application does not limit it.
[0073] It should be noted that the map includes a map of the area where the robot and the docking target are located. This map is obtained by processing point cloud data collected by the laser sensor using the SLAM method. The laser sensor is mounted on the robot.
[0074] The sensors in this embodiment are all connected to the robot and can transmit data to the robot.
[0075] Of course, the laser sensor + SLAM method is only one method of obtaining maps in this application and cannot constitute the only limitation of this application. In addition, wheel odometry, vision sensor + SLAM and other methods can also be used to obtain maps including the area where the robot and the docking target are located.
[0076] As an example, the robot in this application can be a cleaning robot, and the docking target can be a water station or a charging base station. Before executing the docking method of this application, point cloud data of the area where the cleaning robot and the water station or charging base station are located can be collected first through the laser sensor on the cleaning robot, and the collected point cloud data can be processed using the SLAM method to obtain a map of the area where the cleaning robot and the water station or charging base station are located.
[0077] See Figure 3 In this embodiment of the application, the center point of the calibration object is taken as the origin O of the world coordinate system, the horizontal direction passing through the origin on the plane where the calibration object is located is taken as the X-axis of the world coordinate system, and the direction passing through the origin and perpendicular to the plane where the calibration object is located is taken as the Y-axis of the world coordinate system. Since the world coordinate system is established on the plane of the calibration object, the Z coordinate of the calibration object in the world coordinate system is 0.
[0078] After obtaining the map, such as Figure 3 As shown, a target location can be set on the map. When the cleaning robot needs water or charging, it can be controlled to reach the preset target location first. When the robot reaches the target location, the visual sensor on the robot will collect an image of the marker located at the water station or charging station.
[0079] In addition, the calibration object in this application may be a calibration plate with a checkerboard pattern, an April label pattern, and a solid circle array pattern, which is set on the robot's docking target.
[0080] For ease of description, the image of the calibration object captured by the robot's vision sensor when the robot reaches the target position is defined as the first image.
[0081] S22, based on the first image containing the calibration object, control the robot to move from the target position to the first preset position.
[0082] See Figure 3 In this embodiment, the position reached by the robot based on the first image of the acquired target is defined as the first preset position. This first preset position is also a preset position on the map. Ideally, the robot's pose relative to the target at the first preset position is a preset relative pose that enables successful docking between the robot and the target. However, in actual operation, due to positioning errors inherent in the visual sensor, the robot's pose relative to the target at the first preset position may not be the preset relative pose. This will not be described in detail here, but will be explained in the following description.
[0083] See Figure 4 In another embodiment of this application, controlling a robot to move from a target position to a first preset position based on a first image including a marker includes:
[0084] S2201, Based on the first image containing the calibration object, obtain the second relative pose of the robot with respect to the calibration object.
[0085] Among them, a first pose can be obtained from the first image. This first pose reflects the pose of the visual sensor on the robot relative to the calibration object when the robot is located at the target position. Since the visual sensor is set on the robot, the second relative pose of the robot (the center directly in front of the robot or the center of the robot) relative to the calibration object can be obtained based on the first pose and the position of the visual sensor on the robot.
[0086] Correspondingly, the robot's second relative pose with respect to the calibration object includes the robot's coordinates on the X and Y axes of the world coordinate system, and the angle between the robot's heading and the Y axis.
[0087] As an example, assuming the calibration object in this application is a calibration board with a checkerboard pattern, after the first image is acquired by the vision sensor, the first image can be processed using a preset image detection algorithm to obtain the target region. The target region is the image region in the first image that contains the calibration object. The pixel coordinates (μ, ν) of each pixel in the target region in the pixel coordinate system of the first image can also be obtained. Since the world coordinate system of the checkerboard pattern is predefined and the size of each cell on the checkerboard pattern is also known, the physical coordinates (X, Y) of each pixel in the target region in the world coordinate system can be obtained.
[0088] After obtaining the pixel coordinates (μ, ν) of each pixel in the target region in the pixel coordinate system of the first image and the physical coordinates (X, Y) of each pixel in the target region in the world coordinate system, a preset camera (vision sensor) calibration algorithm can be used to calibrate the camera, thereby obtaining the camera's extrinsic parameter matrices R (rotation matrix) and T (translation matrix). The extrinsic parameter matrices reflect the relative position between the camera coordinate system and the world coordinate system. Therefore, after obtaining the camera's extrinsic parameter matrices, the camera's coordinates in the world coordinate system can be obtained. These coordinates represent the camera's first pose relative to the calibration object. Since the vision sensor is mounted on the robot, the robot's first relative pose relative to the calibration object can be obtained based on the camera's first pose relative to the calibration object and the camera's position on the robot. The first relative pose includes the robot's coordinates on the X and Y axes of the world coordinate system, as well as the angle between the robot's heading and the Y-axis in the world coordinate system.
[0089] The camera calibration algorithm in the embodiments of this application may be Zhang Zhengyou calibration algorithm, etc., and is not particularly limited here.
[0090] S2202, based on the second relative pose and the preset relative pose, obtain the first target path from the target position to the first preset position; wherein the preset relative pose is the preset relative pose of the robot relative to the calibration object when the robot is located at the first preset position.
[0091] In this embodiment of the application, a first target path from the target position to the first preset position is obtained based on the second relative pose of the robot relative to the calibration object when the robot is located at the first preset position obtained in S2201 and the preset relative pose of the robot relative to the calibration object when the robot is at the first preset position under ideal conditions.
[0092] For ease of description, the target path from the target position to the first preset position obtained based on the second relative pose and the preset relative pose is defined as the first target path.
[0093] S2203, control the robot to move from the target position to the first preset position along the first target path.
[0094] In this embodiment of the application, the robot is controlled to start from the target position and move along the first target path to the first preset position by obtaining the first target path through S2202.
[0095] See Figure 5 In another embodiment of this application, the process of controlling the robot to move from the target location to the first preset location along the first target path further includes:
[0096] S51, whenever the vision sensor continuously acquires a preset number of third images containing the calibration object at a preset acquisition frequency, or whenever the robot moves a preset distance, or whenever the robot moves for a preset time interval, determine the relative pose of the robot with respect to the calibration object when the robot is at its current position.
[0097] For ease of description, the preset images containing the calibration objects collected by the vision sensor during the robot's movement along the first target path are defined as the third image.
[0098] In this embodiment, due to the limited stability of the images captured by the vision sensor, the robot may experience significant yaw when moving directly along the first target path obtained from the first image. Therefore, during the robot's movement along the first target path, this solution can also pre-set the image acquisition frequency of the vision sensor. This controls the vision sensor to continuously acquire a preset number of third images containing the calibration object at the preset acquisition frequency as the robot moves towards the first preset position. Based on the acquired preset number of third images containing the calibration object, the robot's relative pose to the calibration object at its current position can be obtained (see [reference]). Figure 3 Point A in the diagram is used to determine the first target path. Based on the robot's relative pose to the calibration object and the preset relative pose when the robot is in its current position, the first target path is adjusted so that when the robot moves along the adjusted path to the first preset position, the robot's pose is more consistent with the preset relative pose.
[0099] S52, based on the robot's relative pose to the calibration object and the preset relative pose when the robot is in its current position, adjust the path from the robot's current position to the first preset position in the first target path to obtain the target path.
[0100] In this embodiment, based on the robot's relative pose to the calibration object when it is at its current position, as determined in S51, and the preset relative pose at the first preset position, the path from the robot's current position to the first preset position in the first target path is adjusted to obtain the final target path. After obtaining the target path, the robot can be controlled to move along the target path to the first preset position.
[0101] See Figure 6 In another embodiment of this application, whenever the vision sensor continuously acquires a preset number of third images containing the calibration object at a preset acquisition frequency, and determines that the robot is at its current position, the robot's relative pose with respect to the calibration object includes:
[0102] S5101, based on each third image containing the calibration object, obtain the third relative pose of the robot relative to the calibration object corresponding to each third image.
[0103] Among them, a second pose can be obtained from the third image. The second pose reflects the pose of the robot's visual sensor relative to the calibration object at the current position of the robot as it moves along the first target path to the first preset position. Since the visual sensor is set on the robot, the third relative pose of the robot (the center directly in front of the robot or the center of the robot) relative to the calibration object can be obtained based on the second pose and the position of the visual sensor on the robot.
[0104] For ease of description, the robot's pose relative to the calibration object obtained from the third image is defined here as the third relative pose.
[0105] As an example, suppose the vision sensor continuously acquires five third images pic1, pic2, pic3, pic4 and pic5 containing the calibration object at a preset acquisition frequency. According to the method described in S2201, these five third images are processed to obtain five third relative poses t1, t2, t3, t4 and t5 corresponding to these five third images.
[0106] S5102, the obtained third relative pose of the robot relative to the calibration object corresponding to each third image is smoothed to obtain a weighted pose, and the weighted pose is used as the relative pose of the robot relative to the calibration object when the robot is in the current position.
[0107] In another embodiment of this application, the obtained third relative pose of the robot relative to the calibration object corresponding to each third image is smoothed to obtain a weighted pose, including:
[0108] The weighted pose is calculated based on the robot's third relative pose relative to the calibration object corresponding to each third image, and the preset weight of the robot's third relative pose relative to the calibration object corresponding to each third image.
[0109] As an example, after obtaining the third relative poses t1, t2, t3, t4, and t5 corresponding to the five third images pic1, pic2, pic3, pic4, and pic5 respectively, the weighted pose t is calculated according to the following formula based on the first preset weight k1 corresponding to the third relative pose t1, the second preset weight k2 corresponding to the third relative pose t2, the third preset weight k3 corresponding to the third relative pose t3, the fourth preset weight k4 corresponding to the third relative pose t4, and the fifth preset weight k5 corresponding to the third relative pose t5:
[0110] t=t1*k1+t2*k2+t3*k3+t4*k4+t5*k5.
[0111] After calculating the weighted pose, the robot's pose relative to the calibration object can be determined when the robot is at its current position. Of course, in practical applications, k1, k2, k3, k4, and k5 can be equal or unequal, and this application does not impose any restrictions on this.
[0112] In another embodiment of this application, when the robot is determined to be at its current position after a preset time interval of robot movement, or after a preset time interval of robot movement, the relative pose of the robot relative to the calibration object includes:
[0113] A fourth image containing the calibration object is acquired via a vision sensor after each preset time interval of robot movement or after each preset time interval of robot movement.
[0114] Based on the fourth image containing the calibration object, the fourth relative pose of the robot relative to the calibration object corresponding to the fourth image is obtained, and this fourth relative pose is taken as the relative pose of the robot relative to the calibration object when the robot is in the current position.
[0115] Accordingly, after determining the robot's fourth relative pose with respect to the calibration object when it is in its current position, the following is also included:
[0116] Based on the robot's fourth relative pose and the preset relative pose at its current position, the path from the robot's current position to the first preset position in the first target path is adjusted to obtain the target path;
[0117] Control the robot to move along the target path to the first preset position.
[0118] Among them, a fourth pose can be obtained from the fourth image. The fourth pose reflects the pose of the robot's visual sensor relative to the calibration object at the current position of the robot as it moves along the first target path to the first preset position. Since the visual sensor is set on the robot, the relative pose of the robot (the center directly in front of the robot or the center of the robot) relative to the calibration object when the robot is at the current position can be obtained based on the fourth pose and the position of the visual sensor on the robot.
[0119] S23, when the robot moves to the first preset position, a second image containing the calibration object is acquired by the vision sensor.
[0120] For ease of description, the image containing the calibration object captured by the vision sensor when the robot moves to the first preset position is defined as the second image.
[0121] S24: Based on the second image containing the calibration object, a first relative pose of the robot relative to the calibration object is obtained. Based on the first relative pose, a first direction is obtained, and the robot is controlled to move along the first direction to dock with the docking target. In this embodiment, the detailed process of obtaining the first relative pose based on the second image can be referred to the relevant description in S2201, and will not be repeated here.
[0122] For ease of description, the robot's relative pose to the calibration object obtained from the second image is defined as the first relative pose. The first relative pose includes the robot's coordinates on the X and Y axes of the world coordinate system and the robot's heading angle, which is the angle between the robot's heading and the Y-axis of the world coordinate system. This heading is referred to as the first direction, which is also the direction from the robot's current position to the docking target.
[0123] In another embodiment of this application, a sensor is provided on the robot or the docking target. The sensor is used to issue a docking success message when the robot and the docking target successfully dock. The processor is used to execute the executable code to implement the following steps:
[0124] During the process of controlling the robot to move along the first direction and dock with the target, the robot stops moving when a docking success message is received.
[0125] In this embodiment, the sensor located on the robot or the docking target can be a limit switch sensor. This limit switch sensor will issue a docking success message when the robot successfully docks with the docking target. After successful docking, the robot can be controlled to stop moving.
[0126] See Figure 7 In another embodiment of this application, a sensor is provided on the robot or the docking target. When the robot and the docking target successfully dock, the sensor sends a docking success message and controls the robot to move along a first direction to dock with the docking target, including:
[0127] S71 controls the robot to move a first distance along a first direction.
[0128] In this embodiment of the application, after determining the first direction based on the first relative pose, the robot can be controlled to move a first distance along the first direction. The first distance can be a distance preset according to the actual situation.
[0129] S72, when the robot is moving a first distance along the first direction, it stops moving when it receives a successful docking message.
[0130] In this embodiment of the application, if the robot's first pair of interfaces successfully docks with the second pair of interfaces of the docking target while moving a first distance along the first direction, a docking success message will be received. The docking success message can also be information sent by a limit switch sensor, which can also be located on the first pair of interfaces or the second pair of interfaces.
[0131] As an example, the first pair of interfaces can be the water inlet of the cleaning robot, and the second pair of interfaces can be the water outlet of the water filling station or the charging port of the charging station. If a docking success message is received during the first distance the cleaning robot moves toward the water filling station, it means that the water inlet of the cleaning robot has successfully docked with the water outlet of the water filling station.
[0132] For ease of description, the information received during the robot's movement of a first distance in the first direction is defined as a successful docking message.
[0133] S73: After the robot has moved a first distance along the first direction, if no docking success information is received, the robot is controlled to stop moving.
[0134] In this embodiment of the application, if the robot does not receive a docking success message from the limit switch sensor on the robot or the docking target while moving a first distance along the first direction, it indicates that the robot has failed to dock with the docking target. At this time, the robot can be controlled to stop moving.
[0135] In another embodiment of this application, after obtaining the first relative pose of the robot relative to the calibration object and before controlling the robot to move a first distance along a first direction, the processor performs the following steps when executing the executable code:
[0136] S81 uses a laser sensor to collect a second distance between the robot and the docking target; the laser sensor is mounted on the robot.
[0137] In this embodiment of the application, after the robot reaches the first preset position, the second distance between the robot and the docking target can be collected by the laser sensor installed on the robot.
[0138] The distance collected by the laser sensor is the distance between the laser sensor and the docking target. Therefore, the distance between the robot (the center directly in front of the robot or the center of the robot) and the docking target can be obtained based on the distance between the laser sensor and the docking target and the position of the laser sensor on the robot.
[0139] For ease of description, the distance between the robot and the docking target, as measured by the laser sensor on the robot after it reaches the first preset position, is defined as the second distance. Furthermore, the laser sensor in this scheme can be a single-point laser sensor.
[0140] S82, based on the first relative pose, obtain the third distance between the robot and the calibration object.
[0141] In this embodiment, the third distance is the robot's coordinate value on the Y-axis of the world coordinate system in the first relative pose. This third distance represents the distance between the robot and the docking target.
[0142] For ease of description, the distance between the robot and the docking target obtained from the first relative pose is defined as the third distance.
[0143] S83, calculate the first difference between the second distance and the third distance. When the first difference meets the first preset range, control the robot to move the first distance along the first direction.
[0144] In this embodiment, the difference between the second distance and the third distance is calculated. If the difference is within a first preset range, it means that the robot's positioning error is within an acceptable range. At this time, after controlling the robot to move a first distance along the first direction, the robot may successfully dock with the docking target.
[0145] For ease of description, the difference between the second distance and the third distance is defined here as the first difference.
[0146] S84, if the first difference does not meet the first preset range, then control the robot to move to the target position.
[0147] In this embodiment of the application, if the first difference is not within the first preset range, it indicates that the robot's positioning error is large. At this time, after controlling the robot to move a first distance along the first direction, the probability of the robot successfully docking with the docking target is extremely small. Therefore, the robot can be directly controlled to move to the target position to restart the positioning and docking work.
[0148] See Figure 9 In another embodiment of this application, after obtaining the first relative pose of the robot relative to the calibration object and before controlling the robot to move a first distance along the first direction, the processor performs the following steps when executing the executable code:
[0149] S91, calculate the second difference between the first relative pose and the preset relative pose. When the second difference meets the second preset range, control the robot to move a first distance along the first direction.
[0150] In the embodiments of this application, see Figure 3Assuming the robot is at the first preset position, its first relative pose with respect to the calibration object is (x2, y2, theta2), and the preset relative pose is (0, y1, 0), the difference C1 between y1 and y2 is calculated to obtain the second difference x2 and C1. It is then determined whether x2 and C1 are both within the second preset range. If both second differences are within the second preset range, the robot is controlled to move a first distance along the first direction obtained in S24 to dock with the target.
[0151] S92, if the second difference does not meet the second preset range, then control the robot to move to the target position.
[0152] In this embodiment of the application, when the second difference does not meet the second preset range, the robot is controlled to move to the target position and restart the positioning and docking work.
[0153] See Figure 10 In another embodiment of this application, after the robot has moved a first distance along a first direction and has not received a successful docking message, the processor, when executing the executable code, performs the following steps after controlling the robot to stop moving:
[0154] S1010: After the robot moves a first distance along the first direction, if no docking success information is received, the robot is controlled to stop moving, and then a fourth distance between the robot and the docking target is obtained.
[0155] In this embodiment of the application, if the robot does not receive docking success information while moving a first distance along the first direction, the distance between the robot and the docking target is collected by a single-point laser sensor on the robot after the robot has moved the first distance.
[0156] The distance collected by the laser sensor is the distance between the laser sensor and the docking target. Therefore, the distance between the robot (the center directly in front of the robot or the center of the robot) and the docking target can be obtained based on the distance between the laser sensor and the docking target and the position of the laser sensor on the robot.
[0157] For ease of description, the distance between the robot and the docking target, as collected by the laser sensor after the robot has moved the first distance, is defined as the fourth distance.
[0158] S1020, if the fourth distance is greater than the first threshold, then control the robot to move a fifth distance along the robot's heading, where the fifth distance is the difference between the fourth distance and the first threshold.
[0159] As an example, suppose the cleaning robot moves a first distance towards the water station but does not receive a successful docking message. Then, the laser sensor on the cleaning robot measures the distance between the robot and the water station. Assuming the first threshold is 4cm, and the fourth distance measured by the laser sensor is 6cm, since the fourth distance 6cm > the first threshold 4cm, the cleaning robot continues moving along its current course towards the water station, moving the difference between the fourth distance and the first threshold by 2cm. During this movement, it docks with the water station.
[0160] In this embodiment, the existence of the first threshold is to ensure that if the robot and the docking target fail to dock successfully but are too close, it may cause damage to the robot or the docking target.
[0161] S1030, if the fourth distance is less than or equal to the first threshold, control the robot to return to the target position.
[0162] As an example, if the fourth distance between the cleaning robot and the water station collected by the laser sensor is 3cm, that is, the fourth distance is less than the preset first threshold of 4cm, then the cleaning robot is controlled to move towards the preset target area, so that after reaching the target area, the docking process is re-executed according to the docking method described in S21 to S245.
[0163] Of course, this application only uses the docking of a cleaning robot with a water station as an example to explain the technical solution of this application. In practical applications, the robot-based operation docking method proposed in this application can also be used for docking of food delivery robots with food counters, docking of refueling robots with the fuel tank opening of vehicles waiting to be refueled, docking of shopping guide robots and chat robots with charging base stations when they need to be charged, etc., which will not be elaborated here.
[0164] See Figure 11 , Figure 11 This diagram illustrates the composition of a robot docking device according to an embodiment of this application. For ease of explanation, only the parts relevant to the embodiment of this application are shown. The docking device 11 includes:
[0165] The first acquisition module 1110 is used to acquire a first image containing a calibration object through a vision sensor after determining that the robot has reached the target position, wherein the vision sensor is disposed on the robot and the calibration object is disposed on the docking target of the robot.
[0166] The first control module 1120 is used to control the robot to move from the target position to a first preset position based on the first image containing the calibration object;
[0167] The second acquisition module 1130 is used to acquire a second image containing a calibration object through a vision sensor when it is determined that the robot has moved to the first preset position.
[0168] The second control module 1140 is used to obtain the first relative pose of the robot relative to the calibration object based on the second image containing the calibration object, and after obtaining the first direction based on the first relative pose, control the robot to move along the first direction to dock with the docking target.
[0169] In another embodiment of this application, the first control module 1120 is further configured to:
[0170] Based on the first image containing the calibration object, the second relative pose of the robot with respect to the calibration object is obtained;
[0171] Based on the second relative pose and the preset relative pose, a first target path is obtained from the target position to the first preset position; wherein, the preset relative pose is the preset relative pose of the robot relative to the calibration object when the robot is located at the first preset position;
[0172] Control the robot to move from the target location along the first target path to the first preset location.
[0173] In another embodiment of this application, the first control module 1120 is further configured to:
[0174] Whenever the visual sensor continuously acquires a preset number of third images containing the calibration object at a preset acquisition frequency, or whenever the robot moves a preset distance, or whenever the robot moves for a preset time interval, the relative pose of the robot with respect to the calibration object is determined when the robot is at its current position.
[0175] Based on the robot's relative pose to the calibration object and the preset relative pose when the robot is in its current position, the path from the robot's current position to the first preset position in the first target path is adjusted to obtain the target path.
[0176] In another embodiment of this application, the first control module 1120 is further configured to:
[0177] Based on each of the third images containing the calibration object, a third relative pose of the robot relative to the calibration object corresponding to each of the third images is obtained;
[0178] The obtained third relative pose of the robot relative to the calibration object corresponding to each of the third images is smoothed to obtain a weighted pose, and the weighted pose is used as the relative pose of the robot relative to the calibration object when the robot is in the current position.
[0179] In another embodiment of this application, the first control module 1120 is further configured to:
[0180] The weighted pose is calculated based on the third relative pose of the robot relative to the calibration object corresponding to each of the third images, and the preset weight of the third relative pose of the robot relative to the calibration object corresponding to each of the third images.
[0181] In another embodiment of this application, the second control module 1140 is further configured to:
[0182] A second distance between the robot and the docking target is acquired by a laser sensor mounted on the robot.
[0183] The third distance between the robot and the calibration object is obtained based on the first relative pose.
[0184] Calculate a first difference between the second distance and the third distance. When the first difference meets a first preset range, control the robot to move a first distance along the first direction.
[0185] If the first difference does not meet the first preset range, the robot is controlled to move to the target position.
[0186] Optionally, in another embodiment of this application, the second control module 1140 may also be used for:
[0187] Calculate a second difference between the first relative pose and the preset relative pose. When the second difference meets a second preset range, control the robot to move the first distance along the first direction.
[0188] If the second difference does not meet the second preset range, the robot is controlled to move to the target position.
[0189] In another embodiment of this application, a sensor is provided on the robot or the docking target. The sensor is used to issue a docking success message when the robot and the docking target successfully dock. The docking device 11 further includes:
[0190] The monitoring module is used to receive docking success information during the process of the robot moving along the first direction and docking with the docking target under the control of the second control module 1140.
[0191] In another embodiment of this application, the second control module 1140 is further configured to control the robot to stop moving when the monitoring module receives the docking success information.
[0192] In another embodiment of this application, a sensor is provided on the robot or the docking target. The sensor is used to send a docking success message when the robot and the docking target successfully dock. The second control module 1140 is further used to: control the robot to move a first distance along the first direction; the monitoring module is further used to receive the docking success message during the process of the second control module 1140 controlling the robot to move the first distance along the first direction; the second control module 1140 is further used to control the robot to stop moving when the monitoring module receives the docking success message; or to control the robot to stop moving when the monitoring module does not receive the docking success message after the robot has moved the first distance along the first direction.
[0193] In another embodiment of this application, the work docking device 11 further includes:
[0194] The third control module is used to control the robot to stop moving after it has moved the first distance along the first direction and has not received the docking success information, and then obtain the fourth distance between the robot and the docking target.
[0195] If the fourth distance is greater than the first threshold, then the robot is controlled to move a fifth distance along the robot's heading, wherein the fifth distance is the difference between the fourth distance and the first threshold;
[0196] If the fourth distance is less than or equal to the first threshold, then control the robot to move to the target position.
[0197] In another embodiment of this application, the work docking device 11 further includes:
[0198] The map building module is used to process point cloud data collected by the laser sensor using the SLAM method to obtain a map, wherein the laser sensor is mounted on the robot, and the map is a map of the area where the robot and the docking target are located.
[0199] It should be noted that the execution process and information interaction between the above-mentioned devices / modules are based on the same concept as the robot-based task docking method provided in the embodiments of this application. For details on its specific functions and technical effects, please refer to the method embodiment section, which will not be repeated here.
[0200] As another embodiment of this application, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, and the computer program, when run on a processor, implements any of the robot docking methods provided in the embodiments of this application.
[0201] If the robot docking method provided in this application is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by one or more processors, it can implement the steps of the various method embodiments described above.
[0202] Similarly, as a computer program product, when the computer program product is run on the robot, it enables the robot to perform the steps in the various method embodiments described above.
[0203] The computer program includes computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable storage medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.
[0204] It should be noted that the content contained in the computer-readable storage medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable storage medium does not include electrical carrier signals and telecommunication signals.
[0205] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.
Claims
1. A robot equipped with a vision sensor, characterized in that, The robot includes a memory and a processor. The memory stores executable program code, and the processor executes the executable program code to perform the following steps: When the robot moves to the target position, a first image containing a calibration object is acquired by the vision sensor, wherein the calibration object is set on the docking target of the robot; Based on the first image containing the calibration object, control the robot to move from the target position to a first preset position; When the robot is determined to have moved to the first preset position, a second image containing the calibration object is acquired by the vision sensor; Based on the second image containing the calibration object, the robot obtains a first relative pose relative to the calibration object. Based on the first relative pose, the robot obtains a first direction and then controls the robot to move along the first direction to dock with the docking target. The robot or the docking target is equipped with a sensor. When the robot and the docking target successfully dock, the sensor sends a docking success message. Controlling the robot to move along the first direction to dock with the docking target includes: Control the robot to move a first distance along the first direction; During the process of the robot moving the first distance along the first direction, when the docking success information is received, the robot is controlled to stop moving; If the robot does not receive the docking success information after moving the first distance along the first direction, the robot is controlled to stop moving. After the robot has moved the first distance along the first direction and has not received the docking success information, the processor, after controlling the robot to stop moving, performs the following steps when executing the executable program code: Obtain the fourth distance between the robot and the docking target; If the fourth distance is greater than the first threshold, the robot is controlled to move a fifth distance along its heading, wherein the fifth distance is the difference between the fourth distance and the first threshold; the first threshold is used to ensure that damage to the robot or the docking target is avoided when the robot fails to dock with the docking target and gets too close. If the fourth distance is less than or equal to the first threshold, then control the robot to move to the target position.
2. The robot as described in claim 1, characterized in that, The step of controlling the robot to move from the target position to a first preset position based on the first image containing the calibration object includes: Based on the first image containing the calibration object, the second relative pose of the robot with respect to the calibration object is obtained; Based on the second relative pose and the preset relative pose, a first target path is obtained from the target position to the first preset position; wherein, the preset relative pose is the preset relative pose of the robot relative to the calibration object when the robot is located at the first preset position; Control the robot to move from the target location along the first target path to the first preset location.
3. The robot as described in claim 2, characterized in that, The control of the robot to move from the target location to the first preset location along the first target path includes: Whenever the vision sensor continuously acquires a preset number of third images containing the calibration object at a preset acquisition frequency, or whenever the robot moves a preset distance, or whenever the robot moves for a preset time interval, the relative pose of the robot with respect to the calibration object is determined when the robot is at its current position. Based on the robot's relative pose to the calibration object and the preset relative pose when the robot is in its current position, the path from the robot's current position to the first preset position in the first target path is adjusted to obtain the target path.
4. The robot as described in claim 3, characterized in that, The relative pose of the robot with respect to the calibration object after the visual sensor continuously acquires a preset number of third images containing the calibration object at a preset acquisition frequency, and determines that the robot is at its current position, includes: Based on each of the third images containing the calibration object, a third relative pose of the robot relative to the calibration object corresponding to each of the third images is obtained; The obtained third relative pose of the robot relative to the calibration object corresponding to each of the third images is smoothed to obtain a weighted pose, and the weighted pose is used as the relative pose of the robot relative to the calibration object when the robot is in the current position.
5. The robot as described in claim 4, characterized in that, The step of smoothing the obtained third relative pose of the robot relative to the calibration object corresponding to each of the third images to obtain a weighted pose includes: The weighted pose is calculated based on the third relative pose of the robot relative to the calibration object corresponding to each of the third images, and the preset weight of the third relative pose of the robot relative to the calibration object corresponding to each of the third images.
6. The robot as described in claim 1, characterized in that, After obtaining the first relative pose of the robot with respect to the calibration object, and before controlling the robot to move a first distance along the first direction, the processor performs the following steps when executing the executable program code: A second distance between the robot and the docking target is acquired by a laser sensor mounted on the robot. The third distance between the robot and the calibration object is obtained based on the first relative pose. Calculate a first difference between the second distance and the third distance. When the first difference meets a first preset range, control the robot to move a first distance along the first direction. If the first difference does not meet the first preset range, the robot is controlled to move to the target position.
7. The robot as described in claim 1, characterized in that, After obtaining the first relative pose of the robot with respect to the calibration object, and before controlling the robot to move a first distance along the first direction, the processor performs the following steps when executing the executable program code: Calculate a second difference between the first relative pose and the preset relative pose. When the second difference meets a second preset range, control the robot to move the first distance along the first direction. If the second difference does not meet the second preset range, the robot is controlled to move to the target position.
8. The robot as described in claim 1, characterized in that, When the processor executes the executable program code, it performs the following steps: The point cloud data collected by the laser sensor is processed using the SLAM method to obtain a map. The laser sensor is mounted on the robot, and the map includes a map of the area where the robot and the docking target are located.
9. A robot operation docking method, characterized in that, The method includes: When the robot moves to the target position, a first image containing a calibration object is acquired by a vision sensor, wherein the calibration object is set on the docking target of the robot; Based on the first image containing the calibration object, control the robot to move from the target position to a first preset position; When the robot is determined to have moved to the first preset position, a second image containing the calibration object is acquired through a vision sensor; Based on the second image containing the calibration object, the robot obtains a first relative pose relative to the calibration object. Based on the first relative pose, the robot obtains a first direction and then controls the robot to move along the first direction to dock with the docking target. The robot or the docking target is equipped with a sensor, which sends a docking success message when the robot and the docking target successfully dock. The control of the robot to move along the first direction to dock with the docking target includes: Control the robot to move a first distance along the first direction; During the process of the robot moving the first distance along the first direction, when the docking success information is received, the robot is controlled to stop moving; If the robot does not receive the docking success information after moving the first distance along the first direction, the robot is controlled to stop moving. After the robot has moved the first distance along the first direction and has not received the docking success information, the method further includes controlling the robot to stop moving after stopping the robot's movement. Obtain the fourth distance between the robot and the docking target; If the fourth distance is greater than the first threshold, the robot is controlled to move a fifth distance along its heading, wherein the fifth distance is the difference between the fourth distance and the first threshold; the first threshold is used to ensure that damage to the robot or the docking target is avoided when the robot fails to dock with the docking target and gets too close. If the fourth distance is less than or equal to the first threshold, then control the robot to move to the target position.