Robot docking method, device, electronic terminal and computer readable storage medium

Abstract

This application provides a robot docking method, apparatus, electronic terminal, and computer-readable storage medium. The robot docking method includes: acquiring the recognition result of a target support leg of a shelf by a laser device, and the odometer data of an odometer device; a preparation point is the position where the robot adjusts its posture before docking with the shelf; a docking point is the position where the robot docks with the shelf; determining the relative posture of the shelf center based on the recognition result of the target support leg of the shelf; determining the current posture of the robot based on the recognition result of the target support leg and the odometer data; determining the docking position of the docking point based on the current posture of the robot, the relative posture of the shelf center, and the calibration docking position of the preparation point; and controlling the robot to move to the docking position of the docking point and dock with the shelf. This application effectively solves the problem of insufficient docking accuracy of a single sensor under complex working conditions by constructing a multi-source fusion perception system of laser device, odometer, and side-view camera.

Description

Technical Field

[0001] This application relates to the field of robotics technology, and in particular to a robot docking method, apparatus, electronic terminal, and computer-readable storage medium. Background Technology

[0002] As robotics technology matures and improves, society is driving the further development of the robotics industry. In various robotic operation scenarios, high-precision docking remains a top priority for all tasks.

[0003] Taking AGV (Automated Guided Vehicle) and AMR (Autonomous Mobile Robot) robots as examples, the traditional method is to set up specific coded targets in the robot's operating environment, and use cameras set on the robot to collect images of the coded targets for recognition and detection, so as to realize a visual docking solution.

[0004] However, current simple visual docking solutions have low accuracy. With the development of robot navigation and positioning technology, centimeter-level terminal-to-point accuracy can no longer meet the requirements of some high-precision processes. Moreover, the target must be observable throughout the entire docking process; otherwise, if there is no real-time feedback (such as robot movement deviation or contamination or damage to the identification strip used for visual navigation), subsequent docking cannot be completed, resulting in a low fault tolerance rate. Summary of the Invention

[0005] The main technical problem addressed by this application is to provide a robot docking method, device, electronic terminal, and computer-readable storage medium to solve the problem of poor docking accuracy between robots and shelves in the prior art.

[0006] To solve the above-mentioned technical problems, the first technical solution adopted in this application is: to provide a robot docking method, wherein the robot is equipped with a laser device, an odometer device, and a side-view camera, the side-view camera being used to identify side-view code strips on the shelf; the robot docking method includes: During the process of the robot moving from the pre-marked preparation point to the pre-marked docking point, in response to the side-view camera not recognizing the side-view code strip, the laser device acquires the recognition result of the target support leg of the shelf, as well as the odometer data of the odometer device; the preparation point is the position where the robot adjusts its posture before docking with the shelf; the docking point is the position where the robot docks with the shelf. Based on the identification results of the target support legs of the shelf, the relative pose of the shelf center with respect to the robot is determined; Based on the identification results of the target support leg and the odometry data, the robot's current pose is determined; Based on the robot's current pose, the relative pose of the shelf center, and the calibration docking position of the preparation point, the docking position of the docking point is determined. Control the robot to move to the docking point and dock with the shelf.

[0007] The target support leg includes the first support leg and the second support leg corresponding to the target connecting edge; the target connecting edge is the connecting edge on the shelf that is parallel to the docking direction of the robot. Obtain the laser equipment's identification results of the target support leg of the shelf, including: Acquire laser point cloud data of the shelf collected by the laser equipment; Clustering is performed on the laser point cloud data to obtain the cluster data corresponding to the first support leg and the second support leg, respectively; Based on the identification results of the target support legs of the shelf, the relative pose of the shelf center with respect to the robot is determined, including: Based on the clustering data corresponding to the first and second supporting legs respectively, the position information of the center point of the first supporting leg and the position information of the center point of the second supporting leg are determined; the position information is in the robot body coordinate system. Based on the position information of the center point of the first support leg, the position information of the center point of the second support leg, and the size information of the shelf, the relative pose of the shelf center with respect to the robot is determined.

[0008] The shelf dimensions include dimensions in the first direction; the first direction is perpendicular to the robot's docking direction. Based on the position information of the center point of the first support leg, the position information of the center point of the second support leg, and the size information of the shelf, the relative pose of the shelf center with respect to the robot is determined, including: Based on the position information of the center point of the first support leg and the position information of the center point of the second support leg, the orientation angle of the target connecting edge relative to the robot is determined; the orientation angle is used as the yaw angle of the shelf relative to the robot. Based on the position information of the center point of the first support leg and the position information of the center point of the second support leg, the relative pose of the target connecting edge with respect to the robot is determined. Based on the relative pose of the target connecting edge to the robot and the dimensional information in the first direction, the relative pose of the shelf center to the robot is determined.

[0009] The robot docking methods also include: When the robot is at the preparation point, the point cloud data of the shelf collected by the laser device is used as the template point cloud data; and the odometer data collected by the odometer device is used as the template odometer data. Based on the identification results of the target support leg and odometry data, the robot's current pose is determined, including: The laser point cloud data of the shelf collected at the current moment is compared with the template point cloud data to determine the robot's first relative pose relative to the preparation point at the current moment. The odometry data collected at the current moment is compared with the template odometry data to determine the robot's second relative pose relative to the preparation point at the current moment; Based on the robot's first relative pose and second relative pose relative to the preparation point at the current moment, and the pose information of the preparation point, residual constraints are constructed to determine the robot's current pose.

[0010] Specifically, based on the robot's first and second relative poses relative to the preparation point at the current moment, and the pose information of the preparation point, residual constraints are constructed to determine the robot's current pose, including: The first result is obtained by multiplying the second relative pose and the pose information of the preparation point; the first residual equation is constructed based on the ratio between the first result and the candidate pose information. The second result is obtained based on the ratio between the candidate pose information and the pose information of the preparation point; the second residual equation is constructed based on the difference between the second result and the first relative pose. Construct a third residual equation based on the difference between the second result and the second relative pose; Based on the sum of at least two of the first residual equation, the second residual equation, and the third residual equation, determine whether the candidate pose information is the robot's current pose.

[0011] Specifically, determining whether candidate pose information should be used as the robot's current pose involves summing at least two of the first, second, and third residual equations, including: Summing at least two of the first, second, and third residual equations yields the final residual value. If the final residual value is less than the preset residual value, the candidate pose information corresponding to the final residual value is determined as the robot's current pose.

[0012] Specifically, determining whether candidate pose information should be used as the robot's current pose involves summing at least two of the first, second, and third residual equations, including: If the final residual value at the current moment is not less than the preset residual value and the cumulative duration is greater than the preset duration, an abnormal prompt message will be generated.

[0013] The docking position is determined based on the robot's current pose, the relative pose of the shelf center, and the calibration docking position of the preparation point, including: The ratio of the product of the robot's current pose and the calibrated docking position at the preparation point to the relative pose of the shelf center is used as the docking position of the docking point.

[0014] The calibration docking position of the preparation point includes the first direction deviation value and the second direction deviation value of the preparation point relative to the docking point, as well as the relative yaw angle between the robot and the shelf when the robot is at the preparation point; the first direction is perpendicular to the second direction, and the second direction is parallel to the docking direction of the robot. The methods for determining the calibration and docking positions of the preparation points include: When the robot is at the preparation point, it acquires the position information of the center point of the first support leg and the position information of the center point of the second support leg based on the laser point cloud data of the shelf collected by the laser device. Based on the position information of the center point of the first support leg and the position information of the center point of the second support leg, determine the relative yaw angle of the robot relative to the shelf when it is at the preparation point; Based on the number of cylinders placed on the shelf along the second direction, the length of the cylinders, the gap between the cylinders, the distance between the outer end face of the cantilever on one side of the robot and the center point of the robot, and the size information of the shelf in the second direction, determine the second direction deviation value of the preparation point relative to the docking point. Based on the dimensions of the shelf in the first direction, determine the first direction deviation value of the preparation point relative to the docking point.

[0015] The robot docking methods also include: During the process of the robot moving from the pre-marked preparation point to the pre-marked docking point, the side-view camera recognizes the detection result of the side-view code strip; Based on the detection results of the side-view code strip, the relative pose of the side-view code strip with respect to the side-view camera is determined; Based on the relative pose of the side-view code tape with respect to the side-view camera and the calibrated docking position of the docking point, the docking position of the docking point at the current moment is determined.

[0016] The methods for determining the docking position of the docking point include: When the robot is at the pre-marked docking point, the detection result of the side-view code band identified by the side-view camera is obtained; Based on the detection results of the side-view camera at the docking point, the initial pose information of the robot at the docking point is determined; the initial pose information is the relative pose of the side-view camera relative to the side-view code strip. As the robot moves from the pre-calibrated preparation point to the pre-calibrated docking point, it responds to the detection result of the side-view code band corresponding to the first frame of the image containing the side-view code band acquired by the side-view camera; Based on the detection results of the side-view code band in the first frame, the actual relative pose of the side-view camera with respect to the side-view code band is determined. The robot's initial pose at the docking point is updated based on the actual relative pose of the side-view camera with respect to the side-view code tape, thus obtaining the calibrated docking position of the docking point.

[0017] The method for determining that the first frame contains the side-view code band includes: Target detection is performed on candidate images acquired by the side-view camera to determine the position information of the side-view code band; Based on the positional information of the side-view code band in the candidate image, the area ratio of the side-view code band to the candidate image is determined; If the area ratio is not less than a preset value and the area ratio corresponding to the previous frame of the candidate image is less than the preset value, then the candidate image is determined to be the first frame containing the side-view code band.

[0018] The method for determining the first frame containing the side-view code band also includes: If the number of identifiers in the side-view code band contained in the candidate image is not less than a preset number, then the candidate image is determined to be the first frame containing the side-view code band.

[0019] Specifically, based on the relative pose of the side-view code strip with respect to the side-view camera and the calibrated docking position of the docking point, the docking position of the docking point at the current moment is determined, including: Obtain the camera extrinsic parameters of the side-view camera and the robot's current pose; Based on the relative pose of the side-view code tape with respect to the side-view camera, the camera extrinsic parameters of the side-view camera, the current pose of the robot, and the calibration docking position corresponding to the docking point, the docking position of the docking point at the current moment is determined.

[0020] The robot docking methods also include: Compare the docking position of the docking point determined by the side-view camera at the current moment with the docking position of the docking point determined by the laser equipment and the odometer equipment; If the deviation between the docking position of the docking point corresponding to the side-view camera and the docking position of the docking point corresponding to the laser device and the odometer device does not exceed a preset range, the robot is controlled to move to the docking position of the docking point corresponding to the side-view camera and dock with the shelf; and / or, if the deviation between the docking position of the docking point corresponding to the side-view camera and the docking position of the docking point corresponding to the laser device and the odometer device exceeds a preset range, an abnormal prompt message is generated.

[0021] Among them, the end of the robot facing the shelf has a cantilever, and the distance between the end face of the cantilever facing the shelf and the center point of the robot is the first distance; Robot docking methods also include: Based on the relative pose of the side-view code strip with respect to the side-view camera at the current moment, and the relative pose of the side-view code strip corresponding to the first frame captured by the side-view camera with respect to the side-view camera, the pose change is determined. The pose change includes the angle change and the depth change. Based on the first distance, the change in angle, and the change in depth, determine the offset of the cantilever toward the end face of the shelf; If the offset exceeds the preset offset, an error message will be generated.

[0022] The side-view camera includes a first camera and a second camera, and the baseline between the first camera and the second camera is perpendicular to the docking direction of the robot. Robot docking methods also include: Based on the relative pose of the side-view code strip with respect to the side-view camera at the current moment and the length of the baseline between the first camera and the second camera, determine the shortest distance between the robot's center point and the side-view code strip. The distance deviation is determined based on the shortest distance between the robot's center point and the side-view code strip, and the shortest distance between the calibrated docking position of the docking point and the side-view code strip; If the distance deviation exceeds the preset distance deviation, an error message will be generated.

[0023] The robot includes a forward-facing camera, which is used to identify shelf markings on the shelves that the robot interacts with. Robot docking methods also include: Control the robot to move to the pre-marked preparation point and obtain the forward-looking recognition result of the shelf markings from the forward-looking camera; The robot is guided to move based on the forward-looking recognition results.

[0024] Among them, the forward recognition results include the two-dimensional coordinate information of each corner point of the shelf label in the forward view image; Guide the robot to move based on the forward-looking recognition results, including: Based on the position information of each corner point corresponding to the shelf label in the front view image and the three-dimensional coordinate information in the coordinates of the label code corresponding to the shelf label, the relative pose information of the robot relative to the shelf label is determined. The distance difference is determined based on the robot's relative pose information relative to the shelf marker and the pre-calibrated relative pose information of the preparation point relative to the shelf marker. The robot is guided to move based on the distance difference.

[0025] To solve the above-mentioned technical problems, the second technical solution adopted in this application is: to provide a robot docking device, the robot docking device comprising: The acquisition module is used to acquire the laser device's recognition result of the target support leg of the shelf and the odometer data of the odometer device in response to the side-view camera not recognizing the side-view code strip during the process of the robot moving from the pre-calibrated preparation point to the pre-calibrated docking point; the preparation point is the position where the robot adjusts its posture before docking with the shelf; the docking point is the position where the robot docks with the shelf. The analysis module is used to determine the relative pose of the shelf center with respect to the robot based on the identification results of the target support legs of the shelf; The processing module is used to determine the robot's current pose based on the identification results of the target support leg and the odometry data. The determination module is used to determine the docking position based on the robot's current pose, the relative pose of the shelf center, and the calibration docking position of the preparation point. The control module is used to control the robot to move to the docking point and dock with the shelf.

[0026] To solve the above-mentioned technical problems, the third technical solution adopted in this application is: to provide an electronic terminal, which includes a memory and a processor coupled to each other, the processor being used to execute program instructions stored in the memory, and the processor being used to execute program data to implement the steps in the robot docking method described above.

[0027] To solve the above-mentioned technical problems, the fourth technical solution adopted in this application is: to provide a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, to implement the steps in the robot docking method described above.

[0028] The beneficial effects of this application are as follows: Unlike existing technologies, this application provides a robot docking method, apparatus, electronic terminal, and computer-readable storage medium. The robot docking method includes: a robot equipped with a laser device, an odometer, and a side-view camera, the side-view camera being used to identify side-view codes on a shelf; the robot docking method includes: during the robot's movement from a pre-calibrated preparation point to a pre-calibrated docking point, in response to the side-view camera not identifying a side-view code, acquiring the laser device's identification result of the target support leg of the shelf, and the odometer data from the odometer device; the preparation point is the position where the robot adjusts its posture before docking with the shelf; the docking point is the position where the robot docks with the shelf; based on the identification result of the target support leg of the shelf, determining the relative posture of the shelf center relative to the robot; based on the identification result of the target support leg and the odometer data, determining the robot's current posture; based on the robot's current posture, the relative posture of the shelf center, and the calibration docking position of the preparation point, determining the docking position of the docking point; and controlling the robot to move to the docking position of the docking point and dock with the shelf. This application effectively solves the problem of insufficient docking accuracy of a single sensor under complex working conditions by constructing a multi-source fusion perception system integrating laser equipment, odometer, and side-view camera. Specifically, when the side-view camera fails to recognize the code tape due to environmental interference, the system immediately switches to a redundant strategy combining laser recognition of the shelf support legs and odometer calculation. This strategy utilizes the geometric features of the support legs to lock the relative pose of the shelf center and relies on odometer data to continuously correct the robot's own motion trajectory, thereby dynamically optimizing the end-effector pose from the preparation point to the docking point by fusing the two types of information. This design not only decouples global navigation from local precise docking, reducing the cumulative error of long-distance positioning, but also significantly improves the docking robustness under adverse conditions such as changes in lighting and code tape contamination through a hardware and software collaborative fault tolerance mechanism. Ultimately, this ensures that the robot can stably and accurately complete automatic docking with the shelf in various warehousing scenarios. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 This is an exemplary single-cantilever robot application scenario diagram of the robot docking method based on multi-source fusion of forward-looking camera, laser device, odometer and side-looking camera provided in this application; Figure 2 This is a flowchart illustrating an embodiment of the robot docking method provided in this application; Figure 3This is a flowchart illustrating a specific embodiment of the robot docking method provided in this application; Figure 4 This is a schematic diagram showing the relative position of the robot and the shelf when the robot is at the docking point, as provided in this application. Figure 5 This is a schematic diagram showing the relative position of the robot and the shelf at the current moment, as provided in this application. Figure 6 This is a schematic diagram of the framework of an embodiment of the robot docking device provided in this application; Figure 7 This is a schematic diagram of the framework of an embodiment of the electronic terminal provided in this application; Figure 8 This is a schematic diagram of a framework of an embodiment of the computer-readable storage medium provided in this application. Detailed Implementation

[0031] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0032] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.

[0033] In this article, the term "and / or" simply describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "more" in this article means two or more objects.

[0034] To enable those skilled in the art to better understand the technical solution of this application, the robot docking method provided in this application will be described in further detail below with reference to the accompanying drawings and specific embodiments.

[0035] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0036] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0037] Common robots typically include, but are not limited to, the following components: Odometry: Provides movement parameters such as linear velocity and angular velocity, enabling a preliminary estimate of the change in robot distance traveled. A common type is the wheel encoder, which estimates the robot's current position based on the position change between consecutive time points and the position at the previous time point.

[0038] Two-dimensional lidar: A sensor used to acquire two-dimensional planar information. It can detect the two-dimensional planar contour information of the surrounding environment and can also detect highly reflective materials.

[0039] Multi-sensor fusion refers to the process in AGV and / or AMR systems that integrates observation data from multiple sensors, uses timestamps and extrinsic parameters between sensors for data synchronization and fusion, and ultimately achieves high-precision observation and calculation results that are consistent in time and space.

[0040] In addition, some artificial visual markers are also needed in application scenarios: artificially designed structured markers for visual recognition, such as QR codes or other graphic codes.

[0041] Traditional methods typically use relative navigation to control the robot to meet and dock with the shelf. This requires ensuring that the robot's vision sensors can observe the target throughout the entire docking process. Otherwise, if there is no real-time feedback, it will be difficult to complete the subsequent docking. This method has a low fault tolerance rate and cannot meet the high precision and high stability requirements of complex scenarios.

[0042] The robot docking method proposed in this application integrates observation information from two-dimensional lidar, forward-looking camera, side-looking camera, and wheeled odometer to create a high-precision docking scheme, thereby ensuring the accuracy and reliability of the robot in performing docking tasks.

[0043] It should be noted that the method in this application is mainly illustrated using a single cantilever robot as an example, but it does not limit the application of the method to other types of robots, which will not be elaborated upon hereafter.

[0044] You can refer to, for example Figure 1 As shown, Figure 1 This is an exemplary application scenario diagram of a single cantilever robot in the robot docking method based on multi-source fusion of a forward-looking camera, laser device, odometer, and side-looking camera provided in this application. In this application, the robot docks with the shelf primarily for picking up or placing materials. In the single cantilever robot's operating scenario, materials are mainly stored in the form of drums or rolls. The robot can retrieve materials by extending its cantilever into the drum. For details, please refer to relevant materials in this field; further elaboration is not provided here.

[0045] For example, in Figure 1In a typical application scenario, the chassis of a single-cantilever AMR robot can be equipped with side-view cameras and laser devices on both sides. The side-view cameras are used to identify side-view code strips on the shelf to obtain relevant pose information; the laser devices are used to detect the pose information of the shelf relative to the robot. The robot is also equipped with an odometry device, which is used to detect the rotation angle (pulse count) of the robot's wheels. It records the number of wheel rotations and the direction of rotation, and combined with the wheel circumference, calculates the theoretical walking distance and turning angle.

[0046] For example, the shelf includes at least two first-direction connecting edges and at least two second-direction connecting edges, which are connected sequentially. The first direction is perpendicular to the robot's docking direction. Side-view code strips are disposed on the inner side of the second-direction connecting edges, with the two first-direction connecting edges having side-view code strips facing each other. A shelf identifier is provided on the side of the shelf opposite the robot. The shelf identifier can have three codes.

[0047] The side-view code band may contain several two-dimensional identification codes. Two side-view code bands contain the same two-dimensional identification codes, and the same two-dimensional identification code is in the same position on the side-view code band.

[0048] For example, the robot's cantilever can be equipped with at least one forward-looking camera. The acquisition direction of the forward-looking camera is parallel to the movement direction of the robot when performing the docking task (or it can be said to be parallel to the axis of the cantilever). The forward-looking camera is used to identify the shelf markings on the shelf that the robot docks with. The cantilever can be adjusted to align with the material cylinder based on the identification results of the shelf markings by the forward-looking camera.

[0049] Before a robot can perform a docking task, it needs to pre-calibrate docking and preparation points. The calibration parameters of the robot at these points can be stored in the form of relative poses between the robot and the shelf (e.g., docking and preparation points). The preparation point is the position where the robot adjusts its pose before docking with the shelf (for example, the robot adjusts its pose at the preparation point to ensure that the center of the cantilever aligns with the center of the material cylinder, so that the cantilever can be correctly inserted into the material cylinder during the subsequent docking process). The docking point is the position where the robot docks with the shelf (for example, after the robot reaches the docking point, that is, after the cantilever has been correctly inserted into the material cylinder, it can control the gripping components on the cantilever to grip the material cylinder on the shelf).

[0050] In subsequent practical applications, whenever the robot's side-view camera successfully identifies the side-view code strip, it can calculate the positions of the preparation point and docking point based on the calibration parameters. Path coverage using the side-view recognition results can eliminate the cumulative error of the odometer equipment and improve docking accuracy.

[0051] The robot docking method provided in this application can be implemented by a server or a terminal alone, or by a server and a terminal working together. In some embodiments, the terminal or server can implement the robot docking method provided in this application by running a computer program. For example, the computer program can be a native program or software module in an operating system; it can be a native application (APP), i.e., a program that needs to be installed in the operating system to run, such as a client that supports virtual scenes, such as a game APP; it can also be a mini-program, i.e., a program that only needs to be downloaded to a browser environment to run; or it can be a mini-program that can be embedded in any APP. In short, the above-mentioned computer program can be any form of application, module, or plugin.

[0052] The following uses a server implementation as an example to illustrate the robot docking method provided in this application.

[0053] Please see Figure 2 , Figure 2 This is a flowchart illustrating an embodiment of the robot docking method provided in this application.

[0054] This embodiment provides a robot docking method, which includes the following steps.

[0055] S1: During the process of the robot moving from the pre-calibrated preparation point to the pre-calibrated docking point, in response to the side-view camera not recognizing the side-view code strip, the laser device acquires the recognition result of the target support leg of the shelf, as well as the odometer data of the odometer device; the preparation point is the position where the robot adjusts its posture before docking with the shelf; the docking point is the position where the robot docks with the shelf.

[0056] S2: Based on the identification results of the target support legs of the shelf, determine the relative pose of the shelf center with respect to the robot.

[0057] S3: Based on the identification results of the target support leg and the odometry data, determine the robot's current pose.

[0058] S4: Based on the robot's current pose, the relative pose of the shelf center, and the calibration docking position of the preparation point, determine the docking position of the docking point.

[0059] S5: Control the robot to move to the docking point and dock with the shelf.

[0060] In one embodiment, a preparation point and a docking point are pre-marked. The side-view camera includes a first camera and a second camera, with the baseline between the first and second cameras perpendicular to the robot's docking direction. For example, the preparation point can be set 15 centimeters from the shelf and located on the shelf's central axis. When the robot is at the preparation point, the robot's cantilever is adjusted to a preset height and angle so that it can smoothly remove the material from the shelf.

[0061] In one specific embodiment, the method for determining the calibration docking position of the preparation point includes the following steps.

[0062] The robot includes a forward-facing camera, which is used to identify shelf markings on the shelves that the robot interacts with.

[0063] Specifically, the robot is controlled to stop at a pre-marked preparation point. A front-view image is obtained by capturing images of the shelf markings using a front-view camera. The shelf markings include multiple QR codes, for example, three. The image coordinates [u, v] of each QR code corner point in the front-view image are then used to... i The 3D coordinates [X, Y, 0] of the corner point of the QR code in the shelf label. i The matching process is performed to obtain the correspondence between the image coordinate system and the three-dimensional coordinate system corresponding to the preparation point.

[0064] In one specific embodiment, the image coordinates [u, v] of each QR code corner point can be determined using the ArUco recognition algorithm. i The 3D coordinates [X, Y, 0] of the corner point of the QR code in the shelf label. i The matching is performed to obtain the correspondence between the image coordinate system and the three-dimensional coordinate system. The value of i ranges from [1, 12].

[0065] After obtaining at least three sets (usually four or more) of corresponding 3D-2D corner points, the system will invoke the PnP solution algorithm. The algorithm will determine the rotation matrix R and translation vector t of the front-view camera relative to the shelf marking coordinate system (i.e., the shelf coordinate system). This (R, t) directly gives the robot's position (X, Y) and attitude (yaw angle) in the shelf coordinate system.

[0066] In one embodiment, the calibration docking position of the preparation point includes a first directional deviation value and a second directional deviation value of the preparation point relative to the docking point, as well as the relative yaw angle between the robot and the shelf when the robot is at the preparation point; the first direction is perpendicular to the second direction, and the second direction is parallel to the docking direction of the robot.

[0067] Please see Figure 3 , Figure 3 This is a flowchart illustrating a specific embodiment of the robot docking method provided in this application.

[0068] In one specific embodiment, the method for determining the calibration docking position of the preparation point includes the following steps.

[0069] S101: When the robot is at the preparation point, it acquires the position information of the center point of the first support leg and the position information of the center point of the second support leg based on the laser point cloud data of the shelf collected by the laser device.

[0070] In one embodiment, after the AMR (Autonomous Mobile Robot) arrives at the preparation point and completes vehicle centering, the chassis's laser equipment is triggered to collect laser point cloud data of the shelf area.

[0071] The original laser point cloud is preprocessed to filter out ground points and outlier noise points, and the remaining point cloud is segmented by Euclidean clustering to obtain several independent point cloud clusters, each of which corresponds to a potential shelf support leg.

[0072] Calculate the projected profile of each point cloud cluster on the horizontal plane and its minimum bounding rectangle. Based on preset shelf leg size thresholds and aspect ratio thresholds, select candidate point cloud clusters that meet the geometric characteristics of the shelf legs.

[0073] Two target support legs located on the same long side of the shelf and distributed along the shelf's extension direction are selected. The center coordinates of all laser points in their respective point cloud clusters are calculated and denoted as the center point P1 of the first support leg and the center point P2 of the second support leg. The long side is parallel to the robot's docking direction. The center points P1(x1, y1) and P2(x2, y2) of the first and second support legs are both positional information in the laser coordinate system.

[0074] S102: Based on the position information of the center point of the first support leg and the position information of the center point of the second support leg, determine the relative yaw angle of the robot relative to the shelf when it is at the preparation point.

[0075] Specifically, construct a target vector P that points from P1 to P2. 12 That is, P 12 =P2-P1. Calculate the robot's relative yaw angle θ with respect to the shelf based on the direction angle of this vector in the horizontal plane. L The calculation formula is: (Formula 1) Among them, P 12x This refers to the component of the target vector in the second direction; P 12y It refers to the component of the target vector in the first direction.

[0076] In one specific embodiment, before calculation, the coordinates of P1 and P2 are transformed to a coordinate system based on the robot's center using a LiDAR. Simultaneously, if more than two support legs are identified, a least squares method is used to fit a straight line to multiple center points, and the endpoints at both ends of the line are designated as P1 and P2 to improve measurement robustness. If the calculated support leg spacing deviates from the nominal dimensions of the shelf beyond a preset error threshold, the identification is deemed invalid, triggering a re-identification or an alarm.

[0077] S103: Based on the number of material cylinders placed on the shelf along the second direction, the length of the material cylinders, the gap between the material cylinders, the distance between the outer end face of the cantilever on one side of the robot and the center point of the robot, and the size information of the shelf in the second direction, determine the second direction deviation value of the preparation point relative to the docking point.

[0078] In one embodiment, the second directional deviation value of the preparation point relative to the docking point can be determined based on the following formula.

[0079] Lx=(L2*N+L3*(N-1)+L4+L1-La)-L / 2+F L / 2 (Formula 2) In the formula: Lx represents the second-direction deviation value of the preparation point relative to the docking point; L1 represents the distance from the first cylinder to the front end of the shelf; L2 represents the length of the cylinder; L3 represents the interval between adjacent cylinders; L4 represents the length of the cantilever protruding from the last cylinder; N represents the number of cylinders arranged along the second direction on the shelf; La represents the distance between the end face of the cantilever away from the robot and the center point of the robot.

[0080] S104: Based on the dimensions of the shelf in the first direction, determine the first direction deviation value of the preparation point relative to the docking point.

[0081] In one embodiment, the position information X of the shelf center in the laser coordinate system is determined based on the center point P1 (x1, y1) of the first support leg and the center point P2 (x2, y2) of the second support leg. 12 The position information of the shelf center in the laser coordinate system is obtained by using the position information of the shelf center in the laser coordinate system and the relative yaw angle of the robot with respect to the shelf. .

[0082] In one specific embodiment, the pose information of the shelf center in the robot coordinate system is determined based on the following formula. Here, the robot coordinate system is a coordinate system constructed with the robot center as the origin.

[0083] (Formula 3) In the formula: Where W represents the width of the shelf; F wThis indicates the length of the shelf leg in the first direction; This indicates the pose information of the shelf center in the robot coordinate system.

[0084] The component of the shelf center's pose information in the robot coordinate system in the second direction ( ) y As a docking parameter for the robot in the second direction.

[0085] Specifically, when ( ) y When the position is positive, control the robot to move forward along the first direction until the deviation value converges within the preset allowable error range, thus completing the lateral position adjustment before docking.

[0086] Specifically, when ( ) y When the value is negative, the robot is controlled to move in the opposite direction along the first direction until the deviation value converges within the preset allowable error range, thus completing the lateral position adjustment before docking.

[0087] In one embodiment, the method for determining the calibration and docking position of the docking point includes the following steps.

[0088] When the robot is at the pre-calibrated docking point, the detection results of the side-view code strip identified by the side-view camera are obtained.

[0089] Based on the detection results of the side-view camera corresponding to the docking point, the initial pose information of the robot at the docking point is determined; the initial pose information is the relative pose of the side-view camera relative to the side-view code strip.

[0090] Please see Figure 4 , Figure 4 This is a schematic diagram showing the relative position of the robot and the shelf when the robot is at the docking point, as provided in this application.

[0091] In one specific embodiment, the robot's cantilever is inserted into the hollow area of ​​the material cylinder on the shelf, causing the robot to stop at a pre-marked docking point. Images of the side-view code strip on the shelf are acquired by a first camera and a second camera, respectively. The relative pose of the side-view camera relative to the side-view code strip is determined based on the recognition results of the first and second cameras, denoted as […]. .

[0092] In one embodiment, the recognition result of the first camera includes a first offset of the target two-dimensional identifier in the image captured by the first camera; the recognition result of the second camera includes a second offset of the target two-dimensional identifier in the image captured by the second camera.

[0093] Based on the recognition results of the first camera and the second camera, as well as the baseline length between the first and second cameras, the yaw angle of the side-view camera relative to the side-view code strip is determined. The yaw angle of the side-view camera relative to the side-view code strip is the yaw angle of the robot relative to the shelf.

[0094] In one specific embodiment, the yaw angle θ0 of the side-view camera relative to the side-view code band is determined by Formula 1.

[0095] (Formula 4) In the formula: d1 is the offset recognized by the first camera; d2 is the offset recognized by the second camera; B is the baseline length between the first camera and the second camera.

[0096] In one embodiment, the relative position information of the robot center relative to the shelf center is determined by the recognition results of the first camera and the recognition results of the second camera.

[0097] The above steps can determine the robot's yaw angle relative to the shelf and the relative position information of the robot's center relative to the shelf's center. The robot's yaw angle relative to the shelf and the relative position information of the robot's center relative to the shelf's center, determined at the docking point, are used as the pose information of the docking point.

[0098] In one embodiment, the robot is controlled to move toward a pre-marked preparation point, which is the position where the robot adjusts its posture before docking with the shelf.

[0099] For example, during the execution of a complete docking task, the robot typically receives the docking task at a certain location and then navigates to the preparation point corresponding to the docking task using LiDAR and odometry. This process is a common navigation method in the field and will not be elaborated upon here.

[0100] However, traditional navigation methods may cause the robot to deviate from the preparation point due to accumulated odometry errors. Therefore, in the method of this application, a pre-calibrated preparation point position can be used as a reference value when the robot begins to move towards the preparation point. During the movement to the preparation point, the forward-looking camera acquires images of the shelf markings, determines the pose information of the preparation point based on the forward-looking recognition results corresponding to the forward-looking camera, and directly performs path overlay (e.g., using the preparation point position determined by the real-time forward-looking recognition results to replace the reference value) to eliminate the accumulated odometry error.

[0101] It should be noted that, since the robot may still reach the preparation point when navigating based on the reference value (with or without deviation), this application does not require the robot's forward-facing camera to recognize the shelf markings during its movement to the preparation point.

[0102] In one embodiment, the robot docking method further includes the following steps.

[0103] Control the robot to move to a pre-marked preparation point and obtain the forward-view recognition result of the shelf markings from the forward-view camera; guide the robot to move according to the forward-view recognition result.

[0104] The forward recognition results include the two-dimensional coordinate information of each corner point of the shelf label in the forward view image.

[0105] In one specific embodiment, the relative pose information of the robot relative to the shelf label is determined based on the position information of each corner point corresponding to the shelf label in the front view image and the three-dimensional coordinate information in the label code coordinates corresponding to the shelf label; the distance difference is determined based on the relative pose information of the robot relative to the shelf label and the relative pose information of the pre-calibrated preparation point relative to the shelf label; and the robot is guided to move according to the distance difference.

[0106] In one embodiment, the target support leg includes a first support leg and a second support leg corresponding to the target connecting edge; the target connecting edge is a connecting edge on the shelf parallel to the docking direction of the robot. In one embodiment, laser point cloud data of the shelf collected by a laser device is acquired; the laser point cloud data is clustered to obtain cluster data corresponding to the first support leg and the second support leg, respectively.

[0107] Specifically, the specific implementation method for determining the relative pose of the shelf center with respect to the robot based on the identification result of the target support leg of the shelf in step S2 is as follows.

[0108] In one embodiment, the position information of the center point of the first support leg and the center point of the second support leg are determined based on the clustering data corresponding to the first support leg and the second support leg, respectively; the position information is in the robot body coordinate system. Based on the position information of the center point of the first support leg, the position information of the center point of the second support leg, and the size information of the shelf, the relative pose of the shelf center with respect to the robot is determined.

[0109] The shelf dimensions include dimensions in the first direction; the first direction is perpendicular to the robot's docking direction.

[0110] In one specific embodiment, the orientation angle of the target connecting edge relative to the robot is determined based on the position information of the center point of the first supporting leg and the position information of the center point of the second supporting leg; the orientation angle is used as the yaw angle of the shelf relative to the robot.

[0111] Based on the position information of the center point of the first support leg and the position information of the center point of the second support leg, the relative pose of the target connecting edge with respect to the robot is determined.

[0112] Based on the relative pose of the target connecting edge to the robot and the dimensional information in the first direction, the relative pose of the shelf center to the robot is determined.

[0113] In one specific implementation, the relative pose of the shelf center with respect to the robot can be determined according to the above steps S101 to S104.

[0114] Specifically, the specific implementation method for determining the robot's current pose based on the identification result of the target support leg and the odometry data in step S3 is as follows.

[0115] In one embodiment, point cloud data of the shelf collected by a laser device is used as template point cloud data Lt0; and odometer data collected by an odometer device is used as template odometer data. .

[0116] The laser point cloud data Lt of the shelf collected at the current moment is compared with the template point cloud data Lt0 to determine the robot's first relative pose relative to the preparation point at the current moment. .

[0117] Odometer data collected at the current moment With template odometer data By comparison, the second relative pose of the robot relative to the preparation point at the current moment is determined. .

[0118] Based on the robot's first relative pose and second relative pose relative to the preparation point at the current moment, and the pose information of the preparation point, residual constraints are constructed to determine the robot's current pose.

[0119] In one specific embodiment, a first result is obtained by multiplying the second relative pose and the pose information of the preparation point; a first residual equation is constructed based on the ratio between the first result and the candidate pose information, as shown below.

[0120] (Formula 5) The second result is obtained based on the ratio between the candidate pose information and the pose information of the preparation point; the second residual equation is constructed based on the difference between the second result and the first relative pose, as shown below.

[0121] (Formula 6) The third residual equation is constructed based on the difference between the second result and the second relative pose; the third residual equation is shown below.

[0122] (Formula 7) Based on the sum of at least two of the first residual equation, the second residual equation, and the third residual equation, determine whether the candidate pose information is the robot's current pose.

[0123] Specifically, at least two of the first residual equation, the second residual equation, and the third residual equation are summed to obtain the final residual value.

[0124] (Formula 8) In one specific embodiment, in response to the final residual value being less than a preset residual value, it is determined that the candidate pose information T corresponding to the final residual value will be... t As the robot's current pose .

[0125] In one specific embodiment, in response to the final residual value corresponding to the current moment being not less than a preset residual value and the cumulative duration being greater than a preset duration, an abnormal prompt message is generated so as to promptly detect situations such as robot slipping, robot hitting an obstacle, or laser damage.

[0126] Specifically, the specific implementation method for determining the docking position data of the docking point based on the robot's current pose, the relative pose of the shelf center, and the calibration docking position of the preparation point in step S4 is as follows.

[0127] In one embodiment, the ratio of the product of the robot's current pose and the calibrated docking position of the preparation point to the relative pose of the shelf center is used as the docking position of the docking point.

[0128] In one specific embodiment, the docking position relative to the docking point when the robot is in its current position is determined based on the following formula. .

[0129] (Formula 9) In the formula: This is the calibration and docking position at the preparation point; Relative position of the shelf center; This is the robot's current pose.

[0130] In one embodiment, during the process of the robot moving from a pre-calibrated preparation point to a pre-calibrated docking point, the detection result of the side-view code strip is identified by the side-view camera.

[0131] In one specific embodiment, the method for determining the first frame containing the side-view code band includes the following steps.

[0132] Specifically, target detection is performed on the candidate images acquired by the side-view camera to determine the position information of the side-view code band; based on the position information of the side-view code band in the candidate image, the area ratio of the side-view code band to the candidate image is determined.

[0133] If the area ratio is not less than a preset value and the area ratio corresponding to the previous frame of the candidate image is less than the preset value, then the candidate image is determined to be the first frame containing the side-view code band.

[0134] If the number of identifiers in the side-view code band contained in the candidate image is not less than a preset number, then the candidate image is determined to be the first frame containing the side-view code band.

[0135] In one specific embodiment, after the robot enters the buffer rack to a preset depth, the side-view camera is triggered to acquire images. First, the ArUco algorithm is used to extract the corner points of all QR codes within the current field of view. Then, the u-axis and v-axis coordinates of each corner point in the pixel coordinate system are traversed. Corner points with a pixel coordinate of less than 50 or greater than the image width minus 50 in the u-axis direction are removed to eliminate the influence of lens edge distortion. Subsequently, the maximum value u of the u-axis coordinate among the remaining valid corner points is calculated. max With minimum value u min , when u max with u min If the difference exceeds 1 / 3 of the image width and the number of QR codes involved is greater than 2, the code band range within the current field of view is determined to meet the valid recognition conditions, and all corner points within this range are input as valid data into the subsequent pose calculation algorithm; otherwise, if the difference is insufficient or the number of QR codes is insufficient, the current frame is discarded and the robot is controlled to fine-tune its position until the conditions are met. Preferably, the four corner points of the same QR code are counted only once, and the edge culling threshold can be dynamically adjusted according to the camera distortion parameters.

[0136] In one embodiment, based on the detection result of the side-view code band, the relative pose of the side-view code band with respect to the side-view camera is determined; based on the relative pose of the side-view code band with respect to the side-view camera and the calibrated docking position of the docking point, the docking position of the docking point at the current moment is determined; when the robot moves from the pre-calibrated preparation point to the pre-calibrated docking point, in response to the detection result of the side-view code band corresponding to the first frame of the image containing the side-view code band acquired by the side-view camera.

[0137] Based on the detection results of the side-view code strip in the first frame, the actual relative pose of the side-view camera with respect to the side-view code strip is determined; based on the actual relative pose of the side-view camera with respect to the side-view code strip, the initial pose of the robot at the docking point is updated to obtain the calibrated docking position of the docking point.

[0138] Specifically, it acquires the camera extrinsic parameters of the side-view camera and the robot's current pose.

[0139] Based on the relative pose of the side-view code tape with respect to the side-view camera, the camera extrinsic parameters of the side-view camera, the current pose of the robot, and the calibration docking position corresponding to the docking point, the docking position of the docking point at the current moment is determined.

[0140] In one specific embodiment, the docking position of the docking point at the current moment is determined according to the following formula based on the relative pose of the side-view code tape with respect to the side-view camera, the camera extrinsic parameters of the side-view camera, the current pose of the robot, and the calibration docking position corresponding to the docking point.

[0141] (Formula 10) In one specific embodiment, the robot docking method further includes the following steps.

[0142] Specifically, the docking position at the current moment, determined by the side-view camera, is compared with the docking position determined by the laser device and the odometer device. The docking position includes coordinate data and yaw angle.

[0143] If the deviation between the docking position of the docking point corresponding to the side-view camera and the docking position of the docking points corresponding to the laser device and the odometer device does not exceed the preset range, the robot is controlled to move to the docking position of the docking point corresponding to the side-view camera and dock with the shelf.

[0144] If the deviation between the docking position of the docking point corresponding to the side-view camera and the docking position of the docking points corresponding to the laser device and the odometer device exceeds a preset range, an abnormal prompt message will be generated.

[0145] The preset range includes a preset distance and a preset angle. The preset distance can be 2cm and the preset angle can be 2°, which can be set according to the actual situation.

[0146] If the distance deviation between the coordinate data of the docking point corresponding to the side-view camera and the coordinate data of the docking points corresponding to the laser device and the odometer device is greater than a preset distance, or if the angle deviation between the yaw angle of the docking point corresponding to the side-view camera and the yaw angle of the docking point corresponding to the laser device and the odometer device is greater than a preset angle, an abnormal prompt message will be generated.

[0147] During the docking process, the difference between laser equipment recognition and side-view camera recognition is used to detect invalid values ​​in the side-view camera recognition results, so as to avoid accidents caused by incorrect side-view camera recognition results; during the docking process, the displacement change at the end of the cantilever and the absolute distance from the robot's center point to the lateral code tape are calculated to prevent the vehicle from deviating and scraping the inner wall of the material cylinder.

[0148] The robot has a cantilever at the end facing the shelf, and the distance between the end face of the cantilever facing the shelf and the center point of the robot is the first distance La.

[0149] In one specific embodiment, the robot docking method further includes the following steps.

[0150] Based on the relative pose of the side-view code strip with respect to the side-view camera at the current moment, and the relative pose of the side-view code strip corresponding to the first frame captured by the side-view camera with respect to the side-view camera, the pose change is determined. The pose change includes the angle change and the depth change. Based on the first distance, the change in angle, and the change in depth, the offset of the cantilever toward the end face of the shelf is determined.

[0151] In one embodiment, the offset of the cantilever toward the end face of the shelf is determined according to the following formula based on a first distance, an angle change, and a depth change.

[0152] ΔL=La*sin(Δα)+Δz (Formula 11) In the formula: Δα represents the change in the robot's angle between the current time and the acquisition time of the first frame; Δz represents the change in depth.

[0153] If the offset exceeds a preset offset, an error message will be generated. The preset offset can be 0.03 meters, or it can be set according to the actual situation.

[0154] In one specific embodiment, the robot docking method further includes the following steps.

[0155] Please see Figure 5 , Figure 5 This is a schematic diagram showing the relative positions of the robot and the shelf at the current moment, as provided in this application.

[0156] Based on the relative pose of the side-view code strip with respect to the side-view camera at the current moment and the length of the baseline between the first camera and the second camera, determine the shortest distance between the robot's center point and the side-view code strip.

[0157] Specifically, DL1 represents the distance between the robot's center point and the side-view code band at the current moment. The offset corresponding to the side-view code band can be determined by the first camera as d1, and by the second camera as d2. The angle between the baseline between the first and second cameras and the shortest distance line connecting the robot's center point and the side-view code band is b. Furthermore, d+c and L can both be calculated using the extrinsic parameters of the side-view camera. Due to the principle of triangle similarity, b and c are equal.

[0158] Specifically, the shortest distance between the robot's center point and the side-view code strip can be determined using the following formula.

[0159] (Formula 12) The distance deviation is determined based on the shortest distance between the robot's center point and the side-view code strip, and the shortest distance between the calibrated docking position of the docking point and the side-view code strip.

[0160] If the distance deviation exceeds the preset distance deviation, an error message will be generated.

[0161] This embodiment quantifies and evaluates docking deviation by comparing the shortest distance between the robot's center and the side-view code strip in real time with the shortest distance between the pre-calibrated docking position and the same code strip. When the deviation exceeds a preset threshold, an abnormality prompt is automatically triggered. This allows for timely detection of deviations caused by positioning errors or environmental changes, preventing the docking operation from continuing under excessive deviation conditions. This improves the safety and reliability of the docking process, while also reducing computational load and providing rapid response, thus offering an effective safety barrier for autonomous robot docking tasks.

[0162] Please see Figure 6 , Figure 6 This is a schematic diagram of the framework of an embodiment of the robot docking device provided in this application.

[0163] This embodiment provides a robot docking device 60, which includes an acquisition module 61, an analysis module 62, a processing module 63, a determination module 64, and a control module 65.

[0164] The acquisition module 61 is used to acquire the laser device's recognition result of the target support leg of the shelf and the odometer data of the odometer device in response to the side-view camera not recognizing the side-view code strip during the process of the robot moving from the pre-calibrated preparation point to the pre-calibrated docking point; the preparation point is the position where the robot adjusts its posture before docking with the shelf; the docking point is the position where the robot docks with the shelf.

[0165] Analysis module 62 is used to determine the relative pose of the shelf center with respect to the robot based on the identification results of the target support legs of the shelf.

[0166] The processing module 63 is used to determine the robot's current pose based on the identification results of the target support leg and the odometry data.

[0167] The determination module 64 is used to determine the docking position of the docking point based on the robot's current pose, the relative pose of the shelf center, and the calibration docking position of the preparation point.

[0168] The control module 65 is used to control the robot to move to the docking position of the docking point and dock with the shelf.

[0169] The robot docking device 60 provided in this embodiment effectively solves the problem of insufficient docking accuracy of a single sensor under complex working conditions by constructing a multi-source fusion perception system of laser equipment, odometer, and side-view camera. Specifically, when the side-view camera cannot recognize the code tape due to environmental interference, the system immediately switches to a redundant strategy that combines laser recognition of the shelf support legs with odometer calculation. This strategy utilizes the geometric features of the support legs to lock the relative pose of the shelf center and relies on odometer data to continuously correct the robot's own motion trajectory, thereby dynamically optimizing the end pose from the preparation point to the docking point by fusing the two types of information. This design not only decouples global navigation from local precise docking, reducing the cumulative error of long-distance positioning, but also significantly improves the docking robustness under adverse conditions such as changes in lighting and code tape contamination through a hardware and software collaborative fault tolerance mechanism. Ultimately, this ensures that the robot can stably and accurately complete automatic docking with the shelf in various warehousing scenarios.

[0170] Please see Figure 7 , Figure 7 This is a schematic diagram of a framework of an embodiment of the electronic terminal provided in this application. The electronic terminal 80 includes a memory 81 and a processor 82 coupled to each other. The processor 82 is used to execute program instructions stored in the memory 81 to implement the steps of any of the robot docking method embodiments described above. In a specific implementation scenario, the electronic terminal 80 may include, but is not limited to, a microcomputer or a server. In addition, the electronic terminal 80 may also include mobile devices such as laptops and tablets, which are not limited here.

[0171] Specifically, processor 82 controls itself and memory 81 to implement the steps of any of the robot docking method embodiments described above. Processor 82 can also be referred to as a CPU (Central Processing Unit). Processor 82 may be an integrated circuit chip with signal processing capabilities. Processor 82 can also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor. Furthermore, processor 82 can be implemented using integrated circuit chips.

[0172] Please see Figure 8 , Figure 8This is a schematic diagram of a framework of an embodiment of the computer-readable storage medium provided in this application. The computer-readable storage medium 90 stores program instructions 901 that can be executed by a processor. The program instructions 901 are used to implement the steps of any of the robot docking method embodiments described above.

[0173] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0174] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0175] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0176] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0177] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0178] The above are merely embodiments of this application and do not limit the scope of patent protection of this application. Any equivalent structural or procedural changes made using the content of this application’s specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of this application.

Claims

1. A robot docking method, characterized in that, The robot is equipped with a laser device, an odometer, and a side-view camera, which is used to identify the side-view code tape on the shelf; The robot docking method includes: During the process of the robot moving from a pre-marked preparation point to a pre-marked docking point, in response to the side-view camera not recognizing the side-view code strip, the identification result of the target support leg of the shelf by the laser device and the odometer data of the odometer device are acquired; the preparation point is the position where the robot adjusts its posture before docking with the shelf; the docking point is the position where the robot docks with the shelf; Based on the identification results of the target support legs of the shelf, the relative pose of the shelf center with respect to the robot is determined; Based on the identification results of the target support leg and the odometry data, the current pose of the robot is determined; Based on the robot's current pose, the relative pose of the shelf center, and the calibrated docking position of the preparation point, the docking position of the docking point is determined. The robot is controlled to move to the docking point and dock with the shelf.

2. The robot docking method according to claim 1, characterized in that, The target support leg includes a first support leg and a second support leg corresponding to the target connecting edge; the target connecting edge is a connecting edge on the shelf that is parallel to the docking direction of the robot; The step of obtaining the laser device's recognition result of the target support leg of the shelf includes: Acquire the laser point cloud data of the shelf collected by the laser device; Clustering is performed on the laser point cloud data to obtain clustering data corresponding to the first support leg and the second support leg, respectively. Determining the relative pose of the shelf center with respect to the robot based on the identification result of the target support leg of the shelf includes: Based on the clustering data corresponding to the first support leg and the second support leg respectively, the position information of the center point of the first support leg and the position information of the center point of the second support leg are determined; the position information is in the robot body coordinate system. Based on the position information of the center point of the first support leg, the position information of the center point of the second support leg, and the size information of the shelf, the relative pose of the shelf center with respect to the robot is determined.

3. The robot docking method according to claim 2, characterized in that, The shelf's dimensions include dimensions in a first direction; this first direction is perpendicular to the robot's docking direction. Determining the relative pose of the shelf center with respect to the robot based on the position information of the center point of the first support leg, the position information of the center point of the second support leg, and the size information of the shelf includes: Based on the position information of the center point of the first support leg and the position information of the center point of the second support leg, the orientation angle of the target connecting edge relative to the robot is determined; the orientation angle is used as the yaw angle of the shelf relative to the robot. Based on the position information of the center point of the first support leg and the position information of the center point of the second support leg, the relative pose of the target connecting edge with respect to the robot is determined; The relative pose of the shelf center relative to the robot is determined based on the relative pose of the target connecting edge relative to the robot and the size information in the first direction.

4. The robot docking method according to claim 2, characterized in that, The method further includes: When the robot is at the preparation point, the point cloud data of the shelf collected by the laser device is acquired as template point cloud data; and the odometer data collected by the odometer device is acquired as template odometer data. Determining the robot's current pose based on the identification result of the target support leg and the odometry data includes: The laser point cloud data of the shelf collected at the current moment is compared with the template point cloud data to determine the first relative pose of the robot relative to the preparation point at the current moment. The odometer data collected at the current moment is compared with the template odometer data to determine the second relative pose of the robot relative to the preparation point at the current moment; Based on the first relative pose and the second relative pose of the robot relative to the preparation point at the current moment, and the pose information of the preparation point, residual constraints are constructed to determine the current pose of the robot.

5. The robot docking method according to claim 4, characterized in that, The step of constructing residual constraints based on the first and second relative poses of the robot relative to the preparation point at the current time, and the pose information of the preparation point, to determine the current pose of the robot includes: A first result is obtained by multiplying the second relative pose and the pose information of the preparation point; a first residual equation is constructed based on the ratio between the first result and the candidate pose information. A second result is obtained based on the ratio between the candidate pose information and the pose information of the preparation point; a second residual equation is constructed based on the difference between the second result and the first relative pose. A third residual equation is constructed based on the difference between the second result and the second relative pose; Based on the sum of at least two of the first residual equation, the second residual equation, and the third residual equation, it is determined whether the candidate pose information is the current pose of the robot.

6. The robot docking method according to claim 5, characterized in that, Determining whether the candidate pose information is the current pose of the robot based on the sum of at least two of the first residual equation, the second residual equation, and the third residual equation includes: At least two of the first residual equation, the second residual equation, and the third residual equation are summed to obtain the final residual value; If the final residual value is less than a preset residual value, then the candidate pose information corresponding to the final residual value is determined as the current pose of the robot.

7. The robot docking method according to claim 6, characterized in that, Determining whether the candidate pose information is the current pose of the robot based on the sum of at least two of the first residual equation, the second residual equation, and the third residual equation includes: If the final residual value at the current time is not less than the preset residual value and the cumulative duration is greater than the preset duration, an abnormal prompt message is generated.

8. The robot docking method according to claim 1, characterized in that, The process of determining the docking position of the docking point based on the robot's current pose, the relative pose of the shelf center, and the calibrated docking position of the preparation point includes: The ratio of the product of the robot's current pose and the calibrated docking position of the preparation point to the relative pose of the shelf center is taken as the docking position of the docking point.

9. The robot docking method according to claim 8, characterized in that, The calibration docking position of the preparation point includes a first directional deviation value and a second directional deviation value of the preparation point relative to the docking point, as well as the relative yaw angle between the robot and the shelf when the robot is at the preparation point; The first direction is perpendicular to the second direction, and the second direction is parallel to the docking direction of the robot; The method for determining the calibration docking position of the preparation point includes: When the robot is at the preparation point, it acquires the position information of the center point of the first support leg and the position information of the center point of the second support leg, obtained from the laser point cloud data of the shelf collected by the laser device. Based on the position information of the center point of the first support leg and the position information of the center point of the second support leg, the relative yaw angle of the robot relative to the shelf when it is at the preparation point is determined; Based on the number of material cylinders placed on the shelf along the second direction, the length of the material cylinders, the gap between the material cylinders, the distance between the outer end face of the cantilever on one side of the robot and the center point of the robot, and the size information of the shelf in the second direction, the second direction deviation value of the preparation point relative to the docking point is determined. Based on the dimensions of the shelf in the first direction, the first directional deviation value of the preparation point relative to the docking point is determined.

10. The robot docking method according to claim 1, characterized in that, The robot docking method also includes: During the process of the robot moving from the pre-calibrated preparation point to the pre-calibrated docking point, in response to the detection result of the side-view code strip identified by the side-view camera; Based on the detection results of the side-view code band, the relative pose of the side-view code band with respect to the side-view camera is determined; Based on the relative pose of the side-view code strip with respect to the side-view camera and the calibrated docking position of the docking point, the docking position of the docking point at the current moment is determined.

11. The robot docking method according to claim 10, characterized in that, The method for determining the calibration docking position of the docking point includes: When the robot is at the pre-calibrated docking point, the detection result of the side-view code band identified by the side-view camera is obtained; Based on the detection results detected by the side-view camera corresponding to the docking point, the initial pose information of the robot at the docking point is determined; the initial pose information is the relative pose of the side-view camera relative to the side-view code strip. As the robot moves from the pre-calibrated preparation point to the pre-calibrated docking point, it responds to the detection result of the side-view code band corresponding to the first frame of the image containing the side-view code band acquired by the side-view camera; Based on the detection results of the side-view code band in the first frame, the actual relative pose of the side-view camera with respect to the side-view code band is determined; The robot's initial pose at the docking point is updated based on the actual relative pose of the side-view camera with respect to the side-view code strip, thereby obtaining the calibrated docking position of the docking point.

12. The robot docking method according to claim 11, characterized in that, The method for determining that the first frame contains the side-view code band includes: Target detection is performed on the candidate images acquired by the side-view camera to determine the position information of the side-view code band; Based on the position information of the side-view code band in the candidate image, the area ratio of the side-view code band to the candidate image is determined; In response to the fact that the area ratio is not less than a preset value and the area ratio corresponding to the previous frame of the candidate image is less than the preset value, the candidate image is determined to be the first frame containing the side-view code band.

13. The robot docking method according to claim 12, characterized in that, The method for determining the image in the first frame that contains the side-view code band further includes: If the number of identifiers in the side-view code band contained in the candidate image is not less than a preset number, then the candidate image is determined to be the first frame containing the side-view code band.

14. The robot docking method according to claim 10, characterized in that, The step of determining the docking position of the docking point at the current moment based on the relative pose of the side-view code band with respect to the side-view camera and the calibrated docking position of the docking point includes: Obtain the camera extrinsic parameters of the side-view camera and the current pose of the robot; Based on the relative pose of the side-view code strip with respect to the side-view camera, the camera extrinsic parameters of the side-view camera, the current pose of the robot, and the calibrated docking position corresponding to the docking point, the docking position of the docking point at the current moment is determined.

15. The robot docking method according to claim 10, characterized in that, The robot docking method also includes: The docking position of the docking point determined by the side-view camera at the current moment is compared with the docking position of the docking point determined by the laser device and the odometer device. If the deviation between the docking position of the docking point corresponding to the side-view camera and the docking position of the docking point corresponding to the laser device and the odometer device does not exceed a preset range, the robot is controlled to move to the docking position of the docking point corresponding to the side-view camera and dock with the shelf; and / or, if the deviation between the docking position of the docking point corresponding to the side-view camera and the docking position of the docking point corresponding to the laser device and the odometer device exceeds a preset range, an abnormal prompt message is generated.

16. The robot docking method according to claim 10, characterized in that, The robot has a cantilever at one end facing the shelf, and the distance between the end face of the cantilever facing the shelf and the center point of the robot is a first distance; The robot docking method also includes: Based on the relative pose of the side-view code band with respect to the side-view camera at the current moment, and the relative pose of the side-view code band with respect to the side-view camera corresponding to the first frame captured by the side-view camera, the pose change amount is determined, and the pose change amount includes the angle change amount and the depth change amount. Based on the first distance, the angle change, and the depth change, the offset of the cantilever toward the end face of the shelf is determined; If the offset exceeds a preset offset, an error message is generated.

17. The robot docking method according to claim 10, characterized in that, The side-view camera includes a first camera and a second camera, and the baseline between the first camera and the second camera is perpendicular to the docking direction of the robot. The robot docking method also includes: Based on the relative pose of the side-view code strip with respect to the side-view camera at the current moment and the length of the baseline between the first camera and the second camera, determine the shortest distance between the center point of the robot and the side-view code strip; The distance deviation is determined based on the shortest distance between the center point of the robot and the side-view code strip and the shortest distance between the calibrated docking position of the docking point and the side-view code strip; If the distance deviation is greater than a preset distance deviation, an error message is generated.

18. The robot docking method according to claim 1, characterized in that, The robot includes a forward-facing camera, which is used to identify shelf markings on the shelf that the robot docks with. The robot docking method also includes: Control the robot to move towards the pre-marked preparation point and obtain the forward-looking recognition result of the shelf label by the forward-looking camera; The robot is guided to move based on the forward-looking recognition results.

19. The robot docking method according to claim 18, characterized in that, The forward recognition result includes the two-dimensional coordinate information of each corner point of the shelf label in the forward view image; The step of guiding the robot to move based on the forward-looking recognition result includes: Based on the position information of each corner point corresponding to the shelf label in the front view image and the three-dimensional coordinate information in the coordinates of the identification code corresponding to the shelf label, the relative pose information of the robot relative to the shelf label is determined. Based on the robot's relative pose information relative to the shelf label and the pre-calibrated relative pose information of the preparation point relative to the shelf label, the distance difference is determined; The robot is guided to move based on the distance difference.

20. A robot docking device, characterized in that, The docking device of the robot includes: The acquisition module is used to acquire the laser device's recognition result of the target support leg of the shelf and the odometer data of the odometer device in response to the side-view camera not recognizing the side-view code strip during the process of the robot moving from the pre-marked preparation point to the pre-marked docking point; the preparation point is the position where the robot adjusts its posture before docking with the shelf; the docking point is the position where the robot docks with the shelf. An analysis module is used to determine the relative pose of the shelf center with respect to the robot based on the identification results of the target support legs of the shelf; The processing module is used to determine the current pose of the robot based on the identification result of the target support leg and the odometry data; The determination module is used to determine the docking position of the docking point based on the robot's current pose, the relative pose of the shelf center, and the calibration docking position of the preparation point. The control module is used to control the robot to move to the docking position of the docking point and dock with the shelf.

21. An electronic terminal, characterized in that, The electronic terminal includes a memory and a processor coupled to each other. The processor is used to execute program instructions stored in the memory and to execute program data to implement the steps in the robot docking method as described in any one of claims 1 to 19.

22. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the robot docking method as described in any one of claims 1 to 19.

Images