Robot docking method, electronic equipment and computer readable storage medium

Abstract

The invention discloses a robot docking method, electronic equipment and a computer readable storage medium, and the method comprises the steps: in the moving process of a robot, in response to the detection of camera identification information for each identification code, selecting at least two target identification codes from each identification code according to the camera identification information, the spacing distance between adjacent target identification codes is greater than the length of the identification code; determining the current pose of the robot according to each target identification code; and controlling the robot to move towards the target docking position according to the current pose until the robot reaches the target docking position. Therefore, the butt joint precision of the robot can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robots, in particular to a robot docking method, an electronic device and a computer readable storage medium. BACKGROUND

[0002] With the rapid development of robot technology, the scenarios in which mobile robots are applied are also increasing. In most application scenarios, robots often need to dock with a station so that the robot can perform a corresponding task.

[0003] When docking, the robot can paste an identification code in a specific area, and the robot determines its position by recognizing the identification code. When the position of the robot is determined by the identification code, the relative pose between the identification code and the robot needs to be determined, and then the current pose of the robot is determined based on the relative pose between the identification code and the robot and the preset position of the identification code. At present, when the identification code is recognized, the pose is mainly recognized by a single identification code collected. The distance between the corner points of the single identification code is too short, and the relative pose between the identification code and the robot calculated is easily affected by the fluctuation of the pixel value of the corner point, thereby reducing the pose accuracy of the robot. SUMMARY

[0004] The technical problem solved by the present application is to provide a robot docking method, an electronic device and a computer readable storage medium, which can improve the accuracy of robot docking.

[0005] To solve the above technical problem, one technical solution adopted by the present application is to provide a robot docking method, which comprises: during movement of the robot, in response to detection of camera recognition information for each identification code, selecting at least two target identification codes from each identification code according to the camera recognition information, and the interval distance between adjacent target identification codes is greater than the length of the identification code; determining the current pose of the robot according to each target identification code; and controlling the robot to move to a target docking position according to the current pose until the target docking position is reached.

[0006] To solve the above technical problem, another technical solution adopted by the present application is to provide an electronic device comprising a memory and a processor, the memory storing program instructions, and the processor fetching the program instructions from the memory to execute the above robot docking method.

[0007] To solve the above technical problem, another technical solution adopted by the present application is to provide a computer readable storage medium comprising program data stored therein, the program data being executed by a processor to implement the above robot docking method.

[0008] The robot docking method of the present application selects at least two target identification codes from the identification codes according to the camera identification information in response to detecting the camera identification information for each identification code during the movement of the robot, and the interval distance between adjacent target identification codes is greater than the length of the identification code; determines the current pose of the robot according to each target identification code; and controls the robot to move to the target docking position according to the current pose until the target docking position is reached. Thus, by selecting target identification codes with a relatively long interval distance from the identification codes, the corner point baseline width of the pose recognition can be improved, and the problem of unstable pose recognition accuracy caused by a too short corner point baseline width can be avoided. BRIEF DESCRIPTION OF DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0010] Figure 1 is a flowchart of an exemplary embodiment of the robot docking method shown by the present application;

[0011] Figure 2 is a schematic diagram of an exemplary embodiment of the first marker shown by the present application;

[0012] Figure 3 is an application scenario schematic diagram of an exemplary embodiment of the target docking position of the present application;

[0013] Figure 4 is a flowchart of an exemplary embodiment of the robot docking process shown by the present application;

[0014] Figure 5 is a structural schematic diagram of an exemplary embodiment of the graph optimization model shown by the present application;

[0015] Figure 6 is a structural schematic diagram of another exemplary embodiment of the graph optimization model shown by the present application;

[0016] Figure 7 is a specific flowchart of an exemplary embodiment of the robot docking method shown by the present application;

[0017] Figure 8 is a structural schematic diagram of an exemplary embodiment of the robot docking device shown by the present application;

[0018] Figure 9 is a structural schematic diagram of an embodiment of the electronic device provided by the present application;

[0019] Figure 10 FIG. 1 is a structural schematic diagram of an embodiment of the computer readable storage medium provided in the present application. DETAILED DESCRIPTION

[0020] The technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the drawings in the embodiments of the present application. It can be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application. In addition, it should be noted that, for the convenience of description, only parts related to the present application are shown in the drawings, but not all structures. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0021] First of all, it should be noted that a robot is a kind of semi-autonomous or fully autonomous automatic control machine. The robot in the present embodiment can be a carrying robot, such as AGV (Automated Guided Vehicle), AMR (Autonomous Mobile Robot) and the like. The running process of the robot is mainly to connect the workbench, materials and the like, so as to realize the carrying, picking and the like of the materials. Therefore, high-precision connection is the key to the whole running process of the robot.

[0022] Based on this, the present application provides a robot connection method, an electronic device and a computer readable storage medium, which can improve the robot connection accuracy. For details, please refer to Figure 1 , Figure 1 FIG. 1 is a flow schematic diagram of an exemplary embodiment of the robot connection method shown in the present application.

[0023] The execution subject of the robot connection method can be a terminal device or a server or other processing device, wherein the terminal device can be a user equipment (User Equipment, UE), a computer, a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (Personal Digital Assistant, PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device and the like. The execution subject of the robot connection method can also be a robot connection device. In some possible implementation ways, the robot connection method can be realized by the processor calling the computer readable instructions stored in the memory.

[0024] Specifically, the robot connection method of the present embodiment includes the following steps:

[0025] S110: In the process of moving the robot, in response to detecting the camera recognition information for each identification code, selecting at least two target identification codes from each identification code according to the camera recognition information, and the interval distance between adjacent target identification codes is greater than the length of the identification code.

[0026] The identification code is a feature structured mark for visual recognition. Exemplarily, the identification code can be an Aruco code (two-dimensional code). The Aruco code is a square mark composed of a wide black frame and an internal binary matrix that determines its identifier (id). In this embodiment, a plurality of identification codes constitute the first marker.

[0027] The camera recognition information refers to the image information obtained by using the camera to collect the current environment. It should be noted that the camera recognition information collected does not require complete inclusion of all identification codes, which is determined according to the collection range of the camera. The first marker can refer to Figure 2 , Figure 2 is a schematic diagram of an exemplary embodiment of the first marker shown in this application. As can be seen from Figure 2 , the first marker is composed of a plurality of identification codes, which are arranged at a fixed interval according to a fixed ID order, and a code band coordinate system is established with the center point of an identification code with a certain fixed ID, so as to facilitate the calculation of the pose of the camera relative to the code band coordinate system through the ID during identification. When the camera is collecting, it can only collect part of the identification codes, or it can collect all the identification codes.

[0028] The target identification code is used for pose solving to obtain the current pose of the robot. Exemplarily, after the robot docking device obtains the camera recognition information for each identification code, it selects the identification codes with an interval distance greater than the length of the identification code as target identification codes, so as to obtain identification codes with a wider corner baseline for pose recognition. Exemplarily, the robot docking device can select the identification codes at the left and right ends as target identification codes, and of course it can also select the identification codes in the middle as target identification codes, as long as the interval distance between the selected target identification codes is greater than the length of the identification code. It should be noted that the interval distance between adjacent target identification codes refers to the interval distance between the corresponding corner points in adjacent target identification codes. Limiting the interval distance between adjacent target identification codes to be greater than the length of the identification code can ensure that the length of the corner baseline formed by the selected target identification codes is at least greater than the length of a single target identification code. The corner baseline width can be the distance between the minimum and maximum values of the horizontal coordinates of the corners in each target identification code.

[0029] In some embodiments, the robot docking device can detect the image captured by the camera to determine whether the first marker is detected. In other embodiments, the robot docking device can also measure the distance between the target point and the robot, and when the distance is less than a preset distance value, it is considered that the robot has reached the area of the first marker, and the camera is started to identify the first marker. S120: determining the current pose of the robot according to each target identification code.

[0030] The current pose refers to the pose information of the robot relative to the target docking position at the current time. In some embodiments, the robot docking device can determine the current pose of the robot by pose solving based on each target identification code. For example, the pose solving method can be PnP (Perspective-n-Point). In other embodiments, the robot docking device can also determine the current pose of the robot by laser recognition information. In other embodiments, the robot docking device can also determine the current pose of the robot by camera recognition information and laser recognition information, or camera recognition information and odometry recognition information, or laser recognition information and odometry recognition information. In some embodiments, the robot docking device can also determine the current pose of the robot by camera recognition information, laser recognition information and odometry recognition information.

[0031] S130: controlling the robot to move to the target docking position according to the current pose until the target docking position is reached.

[0032] The target docking position refers to the position where the robot needs to dock. For example, the target docking position can be measured and determined in advance, so that the robot can perform related tasks after reaching the target docking position. For details, please refer to Figure 3 , Figure 3 is an application scenario diagram of an exemplary embodiment of the target docking position of the present application. It includes the docking mechanism of the robot, and the upper part of the docking mechanism is prevented from having a roll. The first marker is arranged on the lower part of both sides of the docking structure, which guides the robot to reach the target docking position. When the docking is combined with laser recognition, a reflector plate can also be arranged below the docking structure. The reflector plate can adjust the installation position by itself in combination with the space below the docking mechanism and the main road space. The width of the reflector plate is generally greater than or equal to 40 cm.

[0033] It can be seen that, in the robot docking method of the embodiments of the present application, in the process of moving the robot, at least two target identification codes are selected from the identification codes according to the camera recognition information in response to the detection of the camera recognition information for each identification code, the interval distance between adjacent target identification codes is greater than the length of the identification code, the current pose of the robot is determined according to each target identification code, and the robot is controlled to move to the target docking position according to the current pose until the target docking position is reached. Thus, the corner baseline length of the pose recognition can be improved by selecting the identification codes with a relatively large interval distance as the target identification codes, and the problem of unstable pose recognition accuracy caused by a too short corner baseline length can be avoided.

[0034] In some embodiments, the process of selecting at least two target identification codes from the identification codes according to the camera recognition information in step S110 can further include: determining the coordinate information of each identification code according to the camera recognition information; and performing screening processing on each identification code according to the coordinate information of each identification code to obtain each target identification code.

[0035] After obtaining the camera recognition information of the identification codes, in order to improve the recognition accuracy and reduce error interference, the robot docking device can further perform screening processing on each identification code according to the coordinate information of each identification code to obtain at least one target identification code. In some embodiments, the robot docking device can detect the corner integrity of each identification code, and if the identification code represents integrity, the corresponding identification code is retained as a target identification code, and if the identification code represents incompleteness, the corresponding identification code is removed. In other embodiments, the identification codes located at the edge of the captured image of the camera are more severely distorted, so the robot docking device can remove the identification codes located at the edge of the captured image to obtain the target identification codes. On this basis, the robot docking device determines the pixel region where each identification code is located according to the coordinate information of each identification code; and in response to the pixel region of the identification code being located in a target pixel region, the corresponding identification code is determined as a target identification code. Thus, the identification codes located in the target pixel region are retained, and the identification codes not located in the target pixel region are removed, which can reduce the error caused by camera distortion. The coordinate information of each identification code can be coordinate information in the camera coordinate system. In some embodiments, after obtaining the captured image of the camera, the robot docking device converts four corner points in the detected identification codes to the camera coordinate system using the camera intrinsic parameters to obtain the coordinate information of each identification code, and the coordinate information of each identification code includes the coordinates of the four corner points. The target pixel region can be a region with less distortion in the image. For example, the target pixel region can be determined in advance according to the camera properties. For example, the central region of the image can be used as the target pixel region, and the area of the central region is determined according to the actual situation. After determining the target pixel region, each identification code is screened through the target pixel region.

[0036] In some embodiments, the robot docking device calculates the intersection-over-union of the pixel region of each identification code with the target pixel region, and determines the intersection-over-union value of each identification code; if the intersection-over-union value is greater than a preset intersection-over-union value, it is determined that the pixel region of the identification code is located in the target pixel region; if the intersection-over-union value is less than or equal to the preset intersection-over-union value, it is determined that the pixel region of the identification code is not located in the target pixel region.

[0037] In other embodiments, the robot docking device can also determine whether each corner point is located in the target pixel region according to the coordinates of each corner point, to determine whether the pixel region of the identification code is located in the target pixel region. Specifically, the horizontal coordinates of the upper-left corner point and the lower-left corner point in each identification code are obtained; the scale of each identification code is calculated using the horizontal coordinates of the upper-left corner point and the upper-right corner point, and the calculation formula is as follows:

[0038] s = |x2-x1| / d

[0039] wherein s represents the scale of the identification code, x2 represents the horizontal coordinate of the upper-left corner point of the identification code, x1 represents the horizontal coordinate of the lower-left corner point of the identification code, and d represents the actual size of the identification code.

[0040] When the scale s of the identification code is less than a preset scale, the vertical coordinates of the four corner points in each identification code are traversed, and the identification code with a vertical coordinate less than a first preset value or greater than a second preset value is removed. The preset scale can be any value between 1-1.5, the first preset value and the second preset value are determined according to the camera installation height, the first preset value can be the camera installation height multiplied by a first preset multiple, the first preset multiple can be 1 / 5, the second preset value can be the camera installation height multiplied by a second preset multiple, and the second preset multiple can be 4 / 5.

[0041] In other embodiments, the robot docking device determines the size of the center identification code and the size of the other identification codes except the center identification code according to the coordinate information of each identification code; in response to the deviation between the size of the other identification codes and the size of the center identification code being greater than a preset deviation, the corresponding other identification codes are removed to obtain removed identification codes; the number of identification codes of each removed identification code is counted; and in response to the number of identification codes being greater than or equal to a preset number of identification codes, the removed coordinate information is determined as the target identification code. In this way, the other identification codes are screened through the center identification code with smaller distortion, which can improve the quality of the identification code and thus improve the pose recognition accuracy.

[0042] The center identification code can be the identification code closest to the center of the camera among the identification codes. Illustratively, the center identification code can be determined according to the coordinate information of the identification codes. Since the farther the distance from the center of the camera, the more serious the distortion, the embodiment selects the center identification code to screen the other identification codes detected to obtain the target identification code meeting the requirements. Specifically, the robot docking device determines the size of each identification code according to the coordinate information of each identification code, and determines whether the other identification codes meet the requirements according to the size deviation of the center identification code and the size of the other identification codes. The size can include the length and width of each identification code, and the length deviation and width deviation between the center identification code and the other identification codes are compared respectively. If the length deviation is greater than a preset length deviation and / or the width deviation is greater than a preset width deviation, the corresponding other identification code is removed. The size can also include the area of each identification code, and the area deviation between the center identification code and the other identification codes is compared respectively. If the area deviation is greater than a preset area deviation, the corresponding other identification code is removed.

[0043] After removing the other identification codes that do not meet the requirements, the number of identification codes of each removed identification code is counted. If the number of identification codes of the removed identification codes is greater than or equal to a preset number of identification codes, the removed identification codes are taken as the target identification codes. If the number of identification codes is less than the preset number of identification codes, the center identification code is removed from each identification code, and a new center identification code is determined from the identification codes from which the center identification code is removed, and the above screening steps are repeated.

[0044] In other embodiments, the robot docking device can first perform a first screening process on each identification code according to the pixel area of each identification code and the target pixel area to obtain the identification codes after the first screening process. Then, the center identification code is selected from the identification codes after the first screening process, and a second screening process is performed on the other identification codes according to the size of the center identification code to obtain the target identification codes. Of course, in other embodiments, the robot docking device can first select the center identification code from each identification code, and perform a first screening process on the other identification codes according to the size of the center identification code to obtain the identification codes after the first screening process. Then, a second screening process is performed on the identification codes after the first screening process according to the pixel area of each identification code after the first screening process and the target pixel area to obtain the target identification codes.

[0045] After obtaining the at least one target identification code, the robot docking device selects, from the corner point horizontal coordinates of the target identification codes, a corner point with a corner point horizontal coordinate less than a first preset horizontal coordinate and a corner point horizontal coordinate greater than a second preset horizontal coordinate as an initial corner point; selects, from the corner point vertical coordinates of the initial corner points, a corner point with a corner point vertical coordinate less than a first preset vertical coordinate and a corner point vertical coordinate greater than a second preset vertical coordinate as a target corner point; and in response to the target corner points meeting a preset requirement, performs pose solving according to the target corner points to obtain the current pose of the robot. In this way, corner points at two ends can be selected, and calculation errors caused by a too short corner point baseline can be reduced.

[0046] The first preset horizontal coordinate and the second preset horizontal coordinate can be set according to actual requirements. For example, the corner point baseline width can be determined according to the detection accuracy, and then the first preset horizontal coordinate and the second preset horizontal coordinate can be determined according to the corner point baseline width. The corner point baseline width refers to the absolute value of the difference between the horizontal coordinates of two adjacent corner points. In other embodiments, the initial corner points can also be selected from the corner point horizontal coordinates of the target identification codes, and the initial corner points correspond to the minimum corner point horizontal coordinate and the maximum corner point horizontal coordinate.

[0047] The first preset vertical coordinate and the second preset vertical coordinate can also be set according to actual requirements. For example, the corner point baseline height can be determined according to the detection accuracy, and then the first preset vertical coordinate and the second preset vertical coordinate can be determined according to the corner point baseline height. The corner point baseline height refers to the absolute value of the difference between the vertical coordinates of two adjacent corner points. In other embodiments, the target corner points can also be selected from the corner point vertical coordinates of the initial corner points, and the target corner points correspond to the minimum corner point vertical coordinate and the maximum corner point vertical coordinate.

[0048] Of course, in other embodiments, the corner point horizontal coordinate screening process and the corner point vertical coordinate screening process can be interchanged, that is, the initial corner points are obtained by screening the corner point vertical coordinates first, and then the target corner points are obtained by screening the corner point horizontal coordinates.

[0049] Further, after obtaining the target corner points, the robot docking device determines whether the target corner points meet preset requirements. If yes, the current pose of the robot is obtained according to the target corner points. The preset requirements can be that the target corner points are not in the distortion area of the collected image. For example, it can be determined whether the corner point horizontal coordinate and / or the corner point vertical coordinate of each target corner point is less than a first preset distortion coordinate, and / or whether the corner point horizontal coordinate and / or the corner point vertical coordinate of each target corner point is greater than a second preset distortion coordinate. If yes, the corresponding target corner point is removed, and the target corner points are re-determined based on the removed target corner points. Specifically, the following cases are included: first, it is determined whether the corner point horizontal coordinate of each target corner point is less than the first preset distortion coordinate. If yes, the corresponding target corner point is removed. Second, it is determined whether the corner point vertical coordinate of each target corner point is less than the first preset distortion coordinate. If yes, the corresponding target corner point is removed. Third, it is determined whether the corner point horizontal coordinate of each target corner point is less than the first preset distortion coordinate and whether the corner point vertical coordinate of each target corner point is less than the first preset distortion coordinate. If yes, the corresponding target corner point is removed. Fourth, it is determined whether the corner point horizontal coordinate of each target corner point is less than the second preset distortion coordinate. If yes, the corresponding target corner point is removed. Fifth, it is determined whether the corner point vertical coordinate of each target corner point is less than the second preset distortion coordinate. If yes, the corresponding target corner point is removed. Sixth, it is determined whether the corner point horizontal coordinate of each target corner point is less than the second preset distortion coordinate and whether the corner point vertical coordinate of each target corner point is less than the second preset distortion coordinate. If yes, the corresponding target corner point is removed. Seventh, it is determined whether the corner point horizontal coordinate of each target corner point is less than the first preset distortion coordinate, whether the corner point vertical coordinate of each target corner point is less than the first preset distortion coordinate, whether the corner point horizontal coordinate of each target corner point is less than the second preset distortion coordinate, and whether the corner point vertical coordinate of each target corner point is less than the second preset distortion coordinate. If yes, the corresponding target corner point is removed.

[0050] Further, the robot docking device obtains the code band coordinate information of each target corner point in the code band coordinate system. In order to facilitate the distinction, the coordinate information of each target corner point in the camera coordinate system is determined as the camera coordinate information. The camera coordinate information and the code band coordinate information of each target corner point in the camera coordinate system and the code band coordinate system are used for pose solving to obtain the current pose of the robot.

[0051] The process of pose solving is as follows:

[0052] The position of the first marker in the world coordinate system is known as The value is fixed and can be expressed by the following equation at any time:

[0053]

[0054] wherein, represents the pose of the robot in the world coordinate system at the corresponding time, represents the extrinsic parameter of the camera in the robot body coordinate system, which is a constant, represents the pose of the first marker relative to the camera coordinate system, since the size and spacing of each marker code in the first marker are known, and the camera intrinsic parameter is known, the pose of the target marker code in the camera coordinate system can be calculated by detecting the ID and corner point of the marker code using the PnP algorithm.

[0055] At any docking time t1 and target time t2, the target time refers to the time point when the robot reaches the target docking position, and the following equation group can be obtained:

[0056]

[0057] and are equal, and the pose of the robot in the world coordinate system at the target time can be obtained by combining equation (1) and equation (2):

[0058]

[0059] Simplifying equation (3) gives the following equation:

[0060]

[0061] wherein, represents the pose of the first marker in the world coordinate system at time t1, represents the pose of the robot in the world coordinate system at time t1, which is a known quantity, represents the pose of the camera in the robot body coordinate system, represents the pose of the first marker in the camera coordinate system at time t1, represents the pose of the first marker in the world coordinate system at time t2, represents the pose of the robot in the world coordinate system at time t2, which is a known quantity, represents the pose of the first marker in the camera coordinate system at time t2.

[0062] By setting the pose of the camera relative to the code band coordinate system at the target docking position the current pose can be calculated. Wherein, the pose of the camera relative to the code band coordinate system at the target docking position can be set in advance in the manner that the camera is directly opposite the origin of the code band coordinate system when the robot docking is completed

[0063] In some embodiments, the present application also proposes a method for identifying the pose of a robot by combining a laser radar, a camera and an odometer. Specifically, laser identification information and odometer identification information of the robot are obtained; a first initial pose of the robot is determined according to each target identification code; a second initial pose of the robot is determined according to the laser identification information; a third initial pose of the robot is determined according to the odometer identification information; and a current pose of the robot is determined according to the first initial pose, the second initial pose and the third initial pose. Thus, the pose information of the robot is determined by fusing various sensors, and the accuracy is higher than that of a single sensor.

[0064] The laser identification information refers to the information obtained by scanning the surrounding environment by the laser radar. The laser radar is a sensor for detecting two-dimensional plane information, which can be installed at a specific position of the robot to obtain the two-dimensional plane contour information of the surrounding environment of the robot, and is also used for detecting high-reflective materials. Exemplarily, the laser radar can be installed at the front left and lower right of the robot, respectively. In some embodiments, a second marker can be set in the relevant area of the target docking position in advance, the laser radar scans the second marker to obtain the point cloud information of the second marker, and the point cloud information of the second marker is processed, so that the pose of the robot under the laser identification information can be obtained. The second marker can be a reflective plate with a laser reflective sticker.

[0065] The odometer identification information refers to the information obtained by detecting the movement of the robot by the odometer. The odometer is a device for measuring the travel and speed of the robot, which is usually installed in the wheel of the robot and uses the relative movement data of the robot within a period of time. For example, the odometer identification information shows that the robot has moved forward by 9 m and rotated to the right by 45° within a period of time.

[0066] During the movement of the robot, the laser identification information detected by the laser radar and the odometer identification information detected by the odometer are continuously obtained, and when the camera identification information for the first marker is not detected, the current pose of the robot is determined by the laser identification information and the odometer identification information or the current pose of the robot is determined by the laser identification information alone; when the camera identification information for the first marker is detected, the current pose of the robot is determined by combining the camera identification information, the laser identification information and the odometer identification information.

[0067] Reference can be made to Figure 4The laser radars are respectively located at front and back of the robot; the camera is installed on the side of the robot to identify the first marker; the first marker is a black solid line on the side, and a code band composed of two-dimensional codes. When the robot starts docking, the robot docking device opens the laser radars to scan the second marker and the odometer to determine the current pose of the robot; when the distance between the robot and the target docking position is detected to be less than a first preset distance value, the camera is opened to detect the camera recognition information for the first marker until the target docking position is reached. The first preset distance value can be the distance between the first point of the code band in the direction of approaching the target docking position and the projection point of the target docking position in the extension direction of the code band.

[0068] The first initial pose is determined according to the camera recognition information. After the robot docking device detects the camera recognition information for the first marker, the camera recognition information is processed to obtain the first initial pose of the robot. In some embodiments, the first marker can be a feature point in the image captured by the camera, and the feature point is identified and processed to obtain the first initial pose of the robot. The selection method of the feature point includes but is not limited to Harris Corner detection algorithm, non-maximum suppression, etc. In other embodiments, the first marker can include at least one identification code, and the robot docking device filters the identification codes according to the coordinate information of each identification code to obtain each target identification code; and performs pose recognition processing on each target identification code to obtain the first initial pose of the robot.

[0069] The second initial pose is determined based on the laser recognition information detected by the robot. In some embodiments, the robot docking device detects the second marker in the laser recognition information to obtain point cloud data of the second marker, and performs pose recognition on the point cloud data of the second marker to obtain the second initial pose of the robot. In other embodiments, the robot docking device sets a sliding window in the laser data queue, reads multiple frames of original point clouds through the sliding window when there are a preset number of original point clouds in the laser data queue; detects the second marker based on the obtained multiple frames of original point clouds to obtain the second initial pose of the robot. In this way, the number of point clouds is enriched by multiple frames of original point clouds, solving the problem that it is difficult to accurately detect the pose of the second marker in the laser coordinate system when the laser radar is far away from the second marker.

[0070] The third initial pose is determined based on mileage recognition information of the robot. Illustratively, the distance change and the angle change between a historical time and a current time are detected by the mileage meter, and the historical time can include any historical time; and the third initial pose of the robot is determined according to the distance change and the angle change of the robot. It should be noted that the mileage recognition information needs to be determined based on two detection results, so when detecting for the first time, the current pose of the robot is determined based on the laser recognition information, or the current pose of the robot is determined based on the laser recognition information and the camera recognition information.

[0071] Before fusing the first initial pose, the second initial pose and the third initial pose, the recognition results of different sensors need to be time-stamped and controlled to ensure fusion accuracy. Since the frequency of the camera is different from the frequency of the mileage meter and the laser radar, the time stamps need to be interpolated and synchronized. Specifically, the image acquisition time stamp of the camera with a lower frequency is taken as the reference, and the front and back two frames of data near the image acquisition time stamp are found in the buffered mileage meter queue with a higher frequency, and the mileage recognition information at the image acquisition time stamp is fitted according to the interpolation principle to complete the synchronization.

[0072] After the robot docking device obtains the synchronized first initial pose, the second initial pose and the third initial pose, the first initial pose, the second initial pose and the third initial pose are jointly optimized by using a graph optimization model to obtain the current pose of the robot.

[0073] In order to reduce errors, the robot docking device can also use the second initial pose corresponding to the laser recognition information to verify the first initial pose corresponding to the camera recognition information. If the verification is successful, the current pose of the robot is determined by using the first initial pose, the second initial pose and the third initial pose; if the verification fails, the first initial pose is rejected, and the current pose of the robot is determined by using the second initial pose and the third initial pose. Illustratively, the difference value between the first initial pose and the second initial pose is obtained; in response to the difference value being less than a preset difference value, the first initial pose, the second initial pose and the third initial pose are input into the graph optimization model for optimization to obtain the current pose of the robot; in response to the difference value being greater than or equal to the preset difference value, the second initial pose and the third initial pose are input into the graph optimization model for optimization to obtain the current pose of the robot. Since the second initial pose corresponding to the laser recognition information is more accurate than the first initial pose corresponding to the camera recognition information, the first initial pose is verified by using the second initial pose, which can prevent the first initial pose with a large error from affecting the accuracy of the current pose of the robot.

[0074] The pose includes an angle of the robot. The robot docking device obtains a difference value between the first initial pose and the second initial pose, which can be a difference value between a first angle in the first initial pose and a second angle in the second initial pose. In some embodiments, since there is a certain fluctuation in the initial angle of the robot calculated by the detected laser point cloud during the docking process, if the angle is directly used, the angle of the route planned by the robot autonomously may fluctuate greatly, which is not conducive to the trajectory tracking of motion control. Therefore, the initial angle can be smoothed to obtain the first angle of the first initial pose.

[0075] The preset difference value is set according to actual requirements. Exemplarily, the preset difference value can be any one of 1°-5°. When the difference value is less than the preset difference value, it is considered that the first initial pose meets the requirements, and the first initial pose, the second initial pose and the third initial pose are input into the graph optimization model for joint optimization to obtain the current pose of the robot. When the difference value is greater than or equal to the preset difference value, it is considered that the first initial pose does not meet the requirements, and the second initial pose and the third initial pose are input into the graph optimization model for joint optimization to obtain the current pose of the robot.

[0076] The graph optimization model refers to expressing an optimization problem in the form of a graph. A graph is a structure composed of vertices and edges, and an edge connects several vertices, representing the relationship between the vertices. The graph optimization model can refer to Figure 5 , Figure 5 is a structural schematic diagram of an exemplary embodiment of the graph optimization model shown in the present application. Figure 5 The circle represents the angle to be optimized, the square represents the second angle recognized by the laser, the triangle represents the third angle recognized by the odometer, and the diamond represents the first angle recognized by the camera. For one of the angles to be optimized, the first angle, the second angle and the third angle can be used as constraint information of the angle to be optimized to jointly optimize the angle to be optimized to obtain a more accurate current angle. The third angle recognized by the odometer can include one or more.

[0077] Refer to Figure 5The pose optimization process can include: constructing residual values of the first initial pose, residual values of the second initial pose and residual values of the third initial pose according to the first preset optimization variable respectively; performing weighted sum processing on the residual values of the first initial pose, the residual values of the second initial pose and the residual values of the third initial pose according to the first preset weight, the second preset weight and the third preset weight respectively to obtain a target residual value, the first preset weight being proportional to the number of the identification codes; and optimizing for the purpose of reducing the target residual value to obtain the current pose of the robot. Thus, the current pose of the robot is determined by optimizing the residual values of the multiple sensors, which can improve the optimization effect, and the weight of the first initial pose is adjusted according to the number of the identification codes, the more the number of the identification codes, the more accurate the first initial pose, and the increase of the corresponding weight can correspondingly improve the accuracy of the current pose.

[0078] The pose optimization includes angle optimization, residual values of the first angle are constructed according to the first preset optimization variable, and the residual values of the first angle are weighted according to the first preset weight to obtain the residual values of the first angle after weighted processing. Exemplarily, the calculation formula of the residual values of the first angle after weighted processing is as follows:

[0079] E Mci =Q c *(M ci -B i )

[0080] Wherein, Q c represents the first preset weight, M ci represents the first angle, and B i represents the first preset optimization variable. The first preset weight increases with the increase of the number of the detected identification codes, but does not exceed the second preset weight. Exemplarily, the first preset weight is less than the second preset weight of the first preset multiple, and the first preset multiple can be any value in 1 / 2-3 / 4.

[0081] Residual values of the second angle are constructed according to the first preset optimization variable, and the residual values of the second angle are weighted according to the second preset weight to obtain the residual values of the second angle after weighted processing. Exemplarily, the calculation formula of the residual values of the second angle after weighted processing is as follows:

[0082] E Mai =Q p *(M ai -B i )

[0083] Wherein, Q p represents the second preset weight, M ai represents the second angle, and B i represents the first preset optimization variable.

[0084] The residual values ​​for the third angle are constructed based on the first preset optimization variables. Then, the residual values ​​for the third angle are weighted according to the third preset weights to obtain the weighted residual values ​​for the third angle. For example, the formula for calculating the weighted residual values ​​for the third angle is as follows:

[0085] E βij =Q r *(Δβ-(B j -B i ))

[0086] Among them, Q r B represents the third preset weight, Δβ represents the third angle for mileage recognition, determined by the change in angle between the angle at the historical moment and the angle at the current moment. j B represents the historical optimization variable corresponding to a historical moment. i This represents the first preset optimization variable at the current moment.

[0087] After obtaining the residual values ​​after weighted processing of the first angle, the second angle, and the third angle, joint optimization is performed using these residual values ​​to obtain the robot's current angle. The joint optimization process is essentially obtaining the min(∑) i E Mαi +∑ i E Mci +∑ i,j E βij ).

[0088] Furthermore, this application also proposes a displacement optimization method. Generally speaking, the displacement recognized by the camera is more accurate than the displacement recognized by the laser. Therefore, the first displacement in the first initial pose recognized by the camera and the second displacement in the second initial pose recognized by the laser can be used for joint optimization to obtain the robot's current displacement. Similarly, displacement optimization can be achieved through a graph optimization model, such as... Figure 6 As shown, Figure 6 This is a schematic diagram of another exemplary embodiment of the graph optimization model shown in this application. Figure 6 In the diagram, circles represent the displacement to be optimized, squares represent the second displacement recognized by laser, and rhombuses represent the first displacement recognized by the camera. The first and second displacements are used as constraint information to optimize the displacement to obtain the robot's current displacement.

[0089] The residual value of the first displacement is constructed based on the second preset optimization variable. Then, the residual value of the first displacement is weighted according to the fourth preset weight to obtain the weighted residual value of the first displacement. For example, the formula for calculating the weighted residual value of the first displacement is as follows:

[0090] E Gci =Qd *(G ci -T i )

[0091] wherein, Q d represents a fourth preset weight, G ci represents a first displacement recognized by the camera, T i represents a second preset optimization variable. The second preset optimization variable is a variable of a laser timestamp detection closest to a timestamp corresponding to the first displacement. The fourth preset weight increases with an increase in the number of detected identification codes, but does not exceed a fifth preset weight corresponding to laser recognition. Exemplarily, the fourth preset weight is less than the fifth preset weight by a second preset multiple, which can be any one of 3-5. It can also be set that the fourth preset weight is greater than the second preset weight by a third preset multiple, which can be 3 times.

[0092] A residual value of a second displacement is constructed according to the second preset optimization variable, and the residual value of the second displacement is weighted according to a fifth preset weight to obtain a residual value of the second displacement after weighting. Exemplarily, the residual value of the second displacement after weighting is calculated according to the following formula:

[0093] E Gai = Q f *(G ai -T i )

[0094] wherein, Q f represents a fifth preset weight, G ai represents a second displacement recognized by laser, T i represents a second preset optimization variable. It should be noted that the second preset optimization variable needs to be timestamp-aligned with the second displacement.

[0095] After obtaining the residual value of the first displacement after weighting and the residual value of the second displacement after weighting, joint optimization is performed using the residual values, so that the current angle of the robot can be obtained. The process of joint optimization is to obtain min(∑ i E Gαi +∑ i E Gci ).

[0096] Further, in the docking process of the robot, the distance between the robot and the target docking position is detected in real time; if the distance between the robot and the target docking position is less than a second preset distance value, it indicates that the robot enters a dangerous area, and a line following accuracy supervision module is started; if the left and right displacement coordinates exceed a preset threshold or the angle exceeds a preset angle threshold in the dangerous area, the robot is controlled to retreat to a laser recognition preparation point along the current angle, and the fusion recognition is restarted; if the same point position appears multiple times in succession, the robot is stopped and an alarm is given.

[0097] To elaborate the robot docking method of the present application, the process is further illustrated by the flow chart shown below: Figure 7

[0098] The robot docking device controls the robot to reach the laser recognition preparation point, turns on the laser radar to scan the second marker, obtains the laser recognition information, and at the same time turns on the odometer to collect the odometer recognition information of the robot; in the first docking stage, the current pose of the robot is determined by using the laser recognition information and the odometer recognition information, and the robot is controlled to approach the target docking position according to the current pose of the robot.

[0099] In the second docking stage, when it is detected that the distance between the robot and the target docking position is less than the first preset distance value, the camera detection of the camera recognition information for the first marker is turned on. At the same time, the odometer recognition information and the laser recognition information are obtained, the odometer recognition information is saved to the odometer queue, the laser recognition information is saved to the laser data queue, and the camera recognition information is saved to the camera data queue.

[0100] The multiple frames of laser recognition information are obtained from the laser data queue by using the sliding window, the linear difference is performed in the odometer queue, the third initial pose synchronized with each timestamp in the sliding window is calculated and stored in the timestamp synchronization queue; by using the synchronized poses of each timestamp, the laser recognition information is projected into the laser coordinate system of the latest frame in the sliding window, the screening of the point cloud data is performed, and the screened point cloud data is obtained; the motion distortion removal is performed on the screened point cloud data, and the target point cloud data is obtained; the pose recognition processing is performed on the target point cloud data, the second initial pose is obtained, and the second initial pose is stored in the feature center queue.

[0101] In the process of determining the pose of the robot by using the camera recognition information, the present embodiment first performs screening processing on each marker code by using the camera installation height and the pixel area where the marker code is located, to obtain the target marker code; then, the target corner points are selected from each target marker code, and the baseline length of the corner points is increased; the pose recognition is performed based on the target corner points, and the first initial pose is determined.

[0102] ​The robot docking device judges whether the difference value between the first initial pose and the second initial pose is less than a preset difference value, if not, joint optimization is performed on the synchronized first initial pose, second initial pose and third initial pose to obtain the current pose of the robot; if the difference value is greater than or equal to the preset difference value, it is judged whether the number of laser recognition information in the feature center queue is greater than or equal to 2, if yes, joint optimization is performed based on the time stamp synchronization queue and the feature center queue to obtain the initial pose of the robot, and then it is judged whether the number of nodes of joint optimization is greater than a preset optimization number threshold, if yes, sliding window optimization is performed to ensure that the number of nodes of joint optimization is less than a certain range, if not, the initial pose of the robot is determined as the current pose of the robot; if the number of laser recognition information in the feature center queue is less than 2, the current pose of the robot is determined in combination with the second initial pose and the third initial pose.

[0103] Please refer to Figure 8 , Figure 8 is a structural schematic diagram of an exemplary embodiment of a robot docking device shown in the present application. The robot docking device 800 comprises a selection module 810, a determination module 820 and a control module 830. The acquisition module 810 is configured to select at least two target identification codes from the identification codes in response to detecting camera recognition information for each identification code during movement of the robot, according to the camera recognition information, the interval distance between adjacent target identification codes being greater than the length of the identification code; the determination module 820 is configured to determine the current pose of the robot according to each target identification code; and the control module 830 is configured to control the robot to move to a target docking position according to the current pose until the target docking position is reached.

[0104] The above scheme, the robot docking device selects at least two target identification codes from the identification codes in response to detecting camera recognition information for each identification code during movement of the robot, according to the camera recognition information, the interval distance between adjacent target identification codes being greater than the length of the identification code; determines the current pose of the robot according to each target identification code; and controls the robot to move to a target docking position according to the current pose until the target docking position is reached. Thus, the target identification codes with a relatively long interval distance are selected from the identification codes, which can improve the corner baseline width of pose recognition and avoid the problem of unstable pose recognition accuracy caused by too short corner baseline width.

[0105] The functions of each module can be referred to the robot docking method embodiments, which will not be described here.

[0106] To implement the robot docking method of the above-mentioned embodiments, the present application proposes another electronic device, please refer to Figure 9 , Figure 9 is a structural schematic diagram of an embodiment of an electronic device provided by the present application.

[0107] The electronic device 900 includes a memory 910 and a processor 920, wherein the memory 910 and the processor 920 are coupled.

[0108] The memory 910 is configured to store program data, and the processor 920 is configured to execute the program data to implement the robot docking method in the above-described embodiments.

[0109] In this embodiment, the processor 920 can also be referred to as a CPU (Central Processing Unit). The processor 920 can be an integrated circuit chip having a processing capability of signals. The processor 920 can also be a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application-Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor 920 can also be any conventional processor.

[0110] The present application also provides a computer readable storage medium, such as Figure 10 As shown in the figure, the computer readable storage medium 1000 is configured to store program data 1010, and the program data 1010, when executed by a processor, is configured to implement the robot docking method in the method embodiments of the present application.

[0111] The method involved in the robot docking method embodiments of the present application exists in the form of a software functional unit when implemented and sold or used as an independent product, and can be stored in a device, such as a computer readable storage medium. Based on this understanding, the technical solutions of the present application, essentially or the part that contributes to the prior art, or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a ROM (Read-Only Memory), a RAM (Random Access Memory), a magnetic disk or an optical disk, and various other media capable of storing program codes.

[0112] The above merely describes the embodiments of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation, or direct or indirect application in other related technical fields, which is made according to the content of the present application specification and drawings, is also included in the patent protection scope of the present application.

Claims

1. A method of robotic docking, the method comprising: The robot docking method comprises: During the movement of the robot, in response to detecting camera recognition information for each identification code, at least two target identification codes are selected from each identification code according to the camera recognition information, and the interval distance between adjacent target identification codes is greater than the length of the identification code; The current pose of the robot is determined according to each target identification code; The robot is controlled to move to a target docking position according to the current pose until the target docking position is reached.

2. The robotic docking method of claim 1, wherein, The step of selecting at least two target identification codes from each identification code according to the camera recognition information comprises: The coordinate information of each identification code is determined according to the camera recognition information; Each target identification code is obtained by screening each identification code according to the coordinate information of each identification code.

3. The method of docking a robot of claim 2, wherein, The step of screening each identification code according to the coordinate information of each identification code to obtain each target identification code comprises: The pixel region in which each identification code is located is determined according to the coordinate information of each identification code; In response to the pixel region of the identification code being located in a target pixel region, the corresponding identification code is determined as the target identification code.

4. The method of docking a robot of claim 2, wherein, The coordinate information of each target identification code comprises the horizontal coordinate and the vertical coordinate of the corner point of each target identification code, and after the step of screening each identification code according to the coordinate information of each identification code to obtain each target identification code, the method further comprises: An initial corner point is selected from the horizontal coordinates of the corner points of each target identification code, the horizontal coordinate of which is less than a first preset horizontal coordinate and the horizontal coordinate of which is greater than a second preset horizontal coordinate; A target corner point is selected from the vertical coordinates of the corner points of each initial corner point, the vertical coordinate of which is less than a first preset vertical coordinate and the vertical coordinate of which is greater than a second preset vertical coordinate. The step of determining the current pose of the robot according to each target identification code comprises: In response to each target corner point meeting a preset requirement, the current pose of the robot is obtained by solving the pose according to each target corner point.

5. The method of docking a robot of claim 2, wherein, The step of screening each identification code according to the coordinate information of each identification code to obtain each target identification code comprises: The size of a central identification code and the size of other identification codes except the central identification code are determined according to the coordinate information of each identification code; In response to the deviation between the size of the other identification codes and the size of the central identification code being greater than a preset deviation, the corresponding other identification codes are removed to obtain removed identification codes; The number of identification codes of each removed identification code is counted; In response to the number of identification codes being greater than or equal to a preset number of identification codes, the removed coordinate information is determined as the target identification code.

6. The method of docking a robot of claim 1, wherein, The step of determining the current pose of the robot according to each target identification code comprises: Laser recognition information and odometry recognition information of the robot are obtained; A first initial pose of the robot is determined according to each target identification code; A second initial pose of the robot is determined according to the laser recognition information; A third initial pose of the robot is determined according to the odometry recognition information; The current pose of the robot is determined according to the first initial pose, the second initial pose and the third initial pose.

7. The method of docking a robot of claim 6, wherein, The step of determining the current pose of the robot according to the first initial pose, the second initial pose and the third initial pose comprises: obtaining a difference value between the first initial pose and the second initial pose; in response to the difference value being less than a preset difference value, inputting the first initial pose, the second initial pose and the third initial pose into a graph optimization model for optimization to obtain the current pose of the robot; in response to the difference value being greater than or equal to the preset difference value, inputting the second initial pose and the third initial pose into the graph optimization model for optimization to obtain the current pose of the robot.

8. The method of docking a robot of claim 6, wherein, The step of determining the current pose of the robot according to the first initial pose, the second initial pose and the third initial pose comprises: constructing a residual value of the first initial pose, a residual value of the second initial pose and a residual value of the third initial pose according to a first preset optimization variable respectively; performing weighted sum processing on the residual value of the first initial pose, the residual value of the second initial pose and the residual value of the third initial pose according to a first preset weight, a second preset weight and a third preset weight respectively to obtain a target residual value, the first preset weight being proportional to the number of the identification codes; optimizing for the purpose of reducing the target residual value to obtain the current pose of the robot.

9. An electronic device, comprising: comprise: a memory and a processor, wherein the memory stores program instructions, and the processor retrieves the program instructions from the memory to execute the method according to any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, comprise: program data stored therein, which, when executed by a processor, is used to implement the method according to any one of claims 1-8.

Images