Robot, robot initial positioning method and device and medium

Abstract

The invention belongs to the technical field of autonomous navigation, and discloses a robot, a robot initial positioning method and device and a medium, and the method comprises the steps: matching a first top image collected under the current pose of the robot with a pre-established visual map to obtain a matched pose, and each frame of second top image corresponds to a global pose. And further, based on the matched pose, the first laser point cloud collected under the current pose is matched with a pre-established laser map to determine an initial positioning pose, and the laser map is a point cloud map obtained by scanning the surrounding environment by the robot. Therefore, visual rough positioning is carried out firstly, then laser accurate positioning is carried out, and the initial positioning accuracy is guaranteed. In addition, positioning can be carried out without the help of external equipment or tools such as labels and charging piles, and usability of the robot is improved.

Description

Technical Field

[0001] The present application belongs to the field of autonomous navigation technology, and in particular, relates to a robot, a robot initial positioning method, a device and a medium. Background Art

[0002] Autonomous navigation is a key technology for robots to achieve autonomous movement. Specifically, the robot needs to be able to accurately locate its own position and plan the movement path according to the target position (movement end point). The key to accurate positioning of autonomous navigation lies in the initial positioning of the robot when it is turned on.

[0003] One feasible initial positioning method is visual label positioning, which is to pre-attach visual labels on the ceiling of the work scene for initial positioning, calibrate the global position of the label when building the map, and capture the label through the camera when the robot is turned on for initial positioning, and calculate its own position relative to the label, and then calculate the global position of the robot in combination with the global position of the label, thereby achieving initial positioning when it is turned on. However, this method requires pre-attaching visual labels on the ceiling of the work scene, which is time-consuming and labor-intensive. In addition, the initial positioning when the robot is turned on must be performed under the label, which is inconvenient to use and has limited usage scenarios.

[0004] Another feasible way is to position the charging pile. By setting a fixed charging pile position in the working scene, the global position of the charging pile is known. When the robot is pushed onto the charging pile, the global position of the robot can be obtained to achieve the initial positioning when it is turned on. However, this requires setting a fixed charging pile position in the working scene, which will occupy the site, and the initial positioning when it is turned on must be performed at the charging pile, which is inconvenient to use. In addition, when the position of the charging pile is moved, the initial positioning will be deviated.

[0005] Another feasible method is laser radar environment matching positioning. The laser radar scans the surrounding environment to obtain the first laser point cloud, matches the first laser point cloud with the laser map, and then calculates the global position of the robot to achieve initial positioning at startup. However, laser environment matching is greatly affected by the initial value, the positioning success rate is unstable, it is easy to match incorrectly, and it is easy to fail when the working scene changes.

[0006] It can be seen that how to accurately locate the initial positioning posture of the robot, improve the accuracy of the robot's autonomous navigation, and then improve the user experience is an urgent problem to be solved by technical personnel in this field. Summary of the invention

[0007] The purpose of the present application is to provide a robot, a robot initial positioning method, a device and a medium, which are used to accurately locate the initial positioning posture of the robot, improve the accuracy of the robot's autonomous navigation, and thereby improve the user experience.

[0008] In a first aspect, an embodiment of the present application provides a robot initial positioning method, comprising:

[0009] Matching the first top image collected at the current posture of the robot with a pre-established visual map to obtain a matching posture; wherein the visual map includes multiple frames of second top images, and each frame of the second top image corresponds to a global posture;

[0010] Based on the visual matching posture, the first laser point cloud collected at the current posture is matched with a pre-established laser map to determine the initial positioning posture of the robot; wherein the laser map is a point cloud map obtained by the robot scanning the surrounding environment.

[0011] Therefore, the technical solution of the embodiment of the present application first obtains a rough pose by matching the first top image with the visual map. Compared with the first laser point cloud data of the surrounding environment, the top image is relatively more fixed and will not change easily, that is, it is not easily affected by the surrounding environment, thereby ensuring the reliability of positioning. Further, on the basis of the rough pose, the first laser point cloud is matched with the laser map to achieve the initial positioning of the robot at any unknown position. Thus, rough visual positioning is performed first, and then precise laser positioning is performed, thereby ensuring the accuracy of the initial positioning of the robot. In addition, there is no need to use external equipment or tools (for example, tags and charging piles, etc.), which improves the usability of the robot.

[0012] In a possible implementation manner of the first aspect, matching the first top image acquired at the current posture of the robot with a pre-established visual map to obtain a matching posture includes:

[0013] Determine a target image in the second top image whose similarity with the first top image reaches a threshold;

[0014] calculating a relative angle between the first top image and the target image;

[0015] The matching pose is obtained according to the global pose of the target image and the relative angle.

[0016] Therefore, based on a large number of historical second top images and the visual map established by the global posture corresponding to each second top image, the first top image collected at the current posture of the robot is matched to obtain the rough initial positioning posture of the current robot. The lightweight matching design avoids a large amount of data calculation and improves the positioning speed. In addition, there is no need to use additional equipment or tools such as tags and charging piles. It is easy to use and ensures that the robot can locate the current initial positioning posture when it is turned on at any position, and provides a positioning basis for subsequent laser precise positioning, thereby improving the positioning accuracy of the robot's initial positioning posture.

[0017] In a possible implementation manner of the first aspect, determining a target image in the second top image whose similarity with the first top image reaches a threshold includes:

[0018] Extracting first visual feature points of the first top image and second visual feature points of each of the second top images in the visual map;

[0019] Calculating a first descriptor of the first visual feature point and a second descriptor of the second visual feature point;

[0020] Inputting each of the first descriptors into a matching model to obtain a first visual word bag, and inputting each of the second descriptors into the matching model to obtain a second visual word bag, wherein the matching model is a natural language model obtained through sample training;

[0021] Matching each of the first visual word bags with each of the second visual word bags to obtain a first matching result, and matching each of the first descriptors with each of the second descriptors to obtain a second matching result;

[0022] The target image is determined according to the first matching result and the second matching result.

[0023] Therefore, based on the extraction and description of the visual feature points of the first top image and the second top image, fast image matching is achieved, providing a basis for roughly locating the initial positioning posture of the current robot according to the target image, and under rough positioning, the visual positioning is less affected by environmental changes and has strong robustness. In addition, the image matching method avoids a large amount of data calculation, thereby lowering the performance requirements of the robot data processing chip and achieving fast positioning speed.

[0024] In a possible implementation manner of the first aspect, based on the matching posture, matching the first laser point cloud collected at the current posture with a pre-established laser map to determine the initial positioning posture of the robot includes:

[0025] Setting an initial matching position of the first laser point cloud in the laser map according to the matching posture;

[0026] Taking the matching initial position as the center, matching the first laser point cloud with a pre-established laser map within a preset matching range; wherein the preset matching range includes a matching distance and a matching angle;

[0027] The posture corresponding to the successful matching of the first laser point cloud and the laser map is used as the initial positioning posture of the robot.

[0028] Therefore, laser positioning is further performed on the basis of the rough posture, thereby improving the accuracy of the robot's initial positioning posture. In addition, laser positioning is based on matching the laser map with the first laser point cloud under the robot's current posture. This matching method also does not require a large amount of data calculation, further improving the positioning speed.

[0029] In a possible implementation manner of the first aspect, when it is determined that the first laser point cloud fails to match the laser map, the method further includes:

[0030] After controlling the robot to rotate a first preset angle, a second laser point cloud is collected;

[0031] Determining whether the second laser point cloud matches the laser map successfully;

[0032] If the match is successful, the pose corresponding to the successful match between the second laser point cloud and the laser map is used as the initial positioning pose of the robot;

[0033] If the matching fails, a prompt signal is sent, and when it is determined that the robot has moved, the process returns to the step of matching the top image captured at the current posture of the robot with the pre-established visual map to obtain a matching posture, and subsequent steps are performed until the initial positioning posture of the robot is determined.

[0034] Therefore, when laser positioning fails, the robot will be triggered to rotate and continue positioning. If the rotation still fails, positioning can continue during the movement of the robot. Therefore, even if part of the environment changes significantly, the positioning accuracy of the robot can be guaranteed.

[0035] In a possible implementation manner of the first aspect, the visual map is formed by collecting a frame of the second top image each time the robot moves a preset distance and / or rotates a second preset angle.

[0036] Therefore, each time the robot moves a preset distance and / or rotates a second preset angle, a visual map consisting of a frame of the second top image is collected to ensure the uniformity of the visual map, thereby ensuring the matching success rate of the first top image at any position, while avoiding the storage pressure of the robot caused by collecting a large number of repeated second top images when the robot is stationary in the same place.

[0037] In a possible implementation manner of the first aspect, determining the target image according to the first matching result and the second matching result includes:

[0038] Determining similar feature points in each of the second visual feature points that match each of the first visual feature points according to the first matching result and the second matching result;

[0039] The second top image in which the number of similar feature points reaches a preset value is used as the target image.

[0040] In a second aspect, an embodiment of the present application provides a robot initial positioning device, comprising:

[0041] A first matching module is used to match the first top image collected at the current posture of the robot with a pre-established visual map to obtain a matching posture; wherein the visual map includes multiple frames of second top images, and each frame of the second top image corresponds to a global posture;

[0042] The second matching module is used to obtain the matching posture based on the first matching module, and match the first laser point cloud collected at the current posture with a pre-established laser map to determine the initial positioning posture of the robot; wherein the laser map is a point cloud map obtained by the robot scanning the surrounding environment.

[0043] In a third aspect, an embodiment of the present application provides a robot, comprising: a camera, a laser radar, a memory, a processor, and a computer program stored in the memory and executable on the processor;

[0044] The camera is connected to the processor and is used to collect a first top image of the robot in its current position; wherein the top image is an image directly above the robot;

[0045] The laser radar is connected to the processor and is used to collect a first laser point cloud in the current posture;

[0046] The processor implements the robot initial positioning method when executing the computer program.

[0047] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the robot initial positioning method described in any one of the first aspects above is implemented.

[0048] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on a terminal device, enables the terminal device to execute the robot initial positioning method described in any one of the above-mentioned first aspects.

[0049] It should be noted that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0051] Figure 1 A schematic diagram of a flow chart of a robot initial positioning method provided in an embodiment of the present application;

[0052] Figure 2 A schematic diagram of the structure of a robot provided in an embodiment of the present application;

[0053] Figure 3 A schematic diagram of a flow chart of a robot initial positioning method provided by another embodiment of the present application;

[0054] Figure 4 A schematic flow chart of another robot initial positioning method provided by another embodiment of the present application;

[0055] Figure 5 A schematic diagram of a structure diagram of a robot initial positioning device provided in an embodiment of the present application;

[0056] Figure 6 A schematic diagram of the structure of a robot provided by another embodiment of the present invention;

[0057] The figure numbers are as follows: 20 is a robot, 21 is a camera, 22 is a laser radar, 23 is an industrial computer, 24 is a first top image, 25 is a ceiling, 60 is a camera, 61 is a preset laser radar, 62 is a processor, 63 is a memory, and 64 is a computer program. DETAILED DESCRIPTION

[0058] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0059] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0060] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0061] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0062] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0063] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0064] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0065] Figure 1 A schematic diagram of a process flow of a robot initial positioning method provided in an embodiment of the present application, such as Figure 1 As shown, the method includes:

[0066] S10: Matching the first top image collected at the current posture of the robot with a pre-established visual map to obtain a matching posture; wherein the visual map includes multiple frames of second top images, and each frame of the second top image corresponds to a global posture;

[0067] In a specific embodiment, after the robot is turned on, the first top image of the robot's current posture is obtained, wherein the first top image may be an image directly above the robot, which may be obtained by photographing the ceiling with a camera disposed on the top of the robot. After obtaining the first top image, the first top image is matched with a pre-established visual map to obtain a matching posture, wherein the visual map is composed of multiple frames of the second top image, and each frame of the second top image may correspond to a corresponding global posture. Optionally, when the robot is mapping, the global posture of the visual map image may be determined by laser positioning to improve the efficiency of mapping.

[0068] In fact, the second top image is similar to the first top image. It is also an image captured directly above the robot. It is a top image captured during the robot's historical driving process, especially during the mapping process. Each time a second top image is captured, the corresponding global pose is stored, that is, the correspondence between multiple poses during the robot's driving process and the second top image is recorded.

[0069] Thus, a visual map is constructed through the correspondence between the historical multi-frame second top image and the global pose. After the robot is turned on and initially positioned, the first top image of the current pose can be matched with the visual map to obtain a matching pose. It is worth noting that the matching pose obtained is a rough pose for determining the current pose of the robot, that is, the current pose of the robot is roughly positioned based on the visual map.

[0070] S11: Based on the matching posture, the first laser point cloud collected at the current posture is matched with a pre-established laser map to determine the initial positioning posture of the robot; wherein the laser map is a point cloud map obtained by the robot scanning the surrounding environment.

[0071] Furthermore, after obtaining the matching posture through step S10, on the basis of the matching posture, the first laser point cloud collected under the current posture of the robot is matched with the pre-established laser map to obtain a matching result, and the final robot initial positioning posture is determined according to the matching result, wherein the laser map is the point cloud map obtained by scanning the surrounding environment by laser radar during the historical driving process of the robot, especially including the process of mapping.

[0072] Specifically, based on the rough posture of the robot (matching posture), the current surrounding environment of the robot is further matched with the historical surrounding environment to determine the precise posture of the robot, thereby completing the initial positioning of the robot when it is turned on.

[0073] It should be noted that since the first top image is the image directly above the robot, for indoor service robots, such as the restaurant scene of a food delivery robot, the first top image collected is mostly an image of the ceiling, which is not easy to change, and the top environment rarely changes. The first laser point cloud is the data of the robot's surrounding environment, which is easily affected by environmental changes. Therefore, in order to ensure the accuracy of the robot's initial positioning, step S10 should be executed first, and then step S11, that is, after obtaining a rough posture based on the first top image, precise positioning is performed based on the first laser point cloud data.

[0074] Figure 2 This is a schematic diagram of the structure of a robot provided in an embodiment of the present application. In order to make the technical solution provided by the present application more clear to those skilled in the art, the following will be combined with Figure 2 The hardware environment used in the robot initial positioning method provided in this application is introduced in detail. It should be noted that: Figure 2 This is an exemplary hardware architecture, and this application does not limit this in the specific embodiment.

[0075] like Figure 2 As shown, the robot 20 provided in the present application may include a camera 21, a laser radar 22 and an industrial computer 23, wherein the camera 21 is used to collect a first top image 24 and a second top image of the robot 20, and the top image is as shown in FIG. Figure 2 The image of the top of the robot 20 shown is, for example, a top image in the direction of the ceiling 25 collected by the food delivery robot 20 in a restaurant application scenario. The laser radar 22 is used to collect the first laser point cloud of the robot 20 in real time, that is, the laser radar 22 is used to collect the laser point cloud of the environment around the robot 20, and the industrial computer 23 is used to execute the above steps S10 and S11.

[0076] In a specific implementation, a monocular or multi-camera 21 can be installed on the top of the robot 20, a laser radar 22 can be installed on the side of the robot 20, an industrial computer 23 can be set inside the robot 20, and the camera 21 and the laser radar 22 are both connected to the industrial computer 23.

[0077] When building a map, the second top image captured by the camera 21 is transmitted to the industrial computer 23. The industrial computer 23 constructs the second top image of each frame and the corresponding global posture into a visual map. When performing initial positioning, the camera 21 transmits the captured first top image 24 to the industrial computer 23. The industrial computer 23 matches the first top image 24 with the visual map to obtain the rough posture (matching posture) of the robot 20.

[0078] The laser radar 22 can scan the surrounding environment, collect laser point clouds and transmit them to the industrial computer 23. When building a map, the robot 20 moves, and the laser radar 22 scans the surrounding environment to obtain historical laser point clouds, which are transmitted to the industrial computer 23. The industrial computer 23 builds a laser map based on the historical laser point clouds. When performing initial positioning, the first laser point cloud is matched with the laser map in combination with the rough pose (visual matching pose), and then the initial positioning pose of the robot 20 is determined. Among them, the first laser point cloud is the laser point cloud data at the initial position when the robot 20 is turned on.

[0079] It should be noted that the first top image 24 and the first laser point cloud at the current position of the robot 20 can be collected at the same time, or the first top image 24 can be collected first and then the first laser point cloud image can be collected. Preferably, simultaneous collection can reduce positioning errors. However, since visual positioning is less affected by the environment, in order to ensure positioning accuracy, visual positioning must be performed before laser positioning, that is, the matching of the first top image 24 should be performed before the matching of the first laser point cloud.

[0080] Therefore, the technical solution of the embodiment of the present application first matches the first top image with the visual map to obtain a rough pose. Compared with the first laser point cloud data of the surrounding environment, the top image is relatively more fixed and will not change easily, that is, the top environment is not easy to change, which ensures the reliability of rough positioning. Further on the basis of the rough pose, the first laser point cloud is matched with the laser map to achieve accurate positioning within the rough positioning range, and realize the initial positioning of the robot at any unknown position. Thus, the visual rough positioning is performed first, and then the laser precise positioning is performed, which ensures the accuracy and efficiency of the initial positioning. In addition, without the help of external equipment or tools (for example, tags and charging piles, etc.), the robot can be initially positioned at any unknown position by relying only on the laser radar and camera carried by the robot itself. The robot is easy to deploy and use, which improves the usability of the robot. If the laser point cloud is used for positioning directly, it is easy to lose positioning or have a large deviation in scenes where the surrounding environment is changeable, such as restaurants, or similar surrounding environments (such as long corridors) are likely to appear. If only the top image is used for positioning, a very large number of images need to be collected to ensure positioning accuracy, and mapping is difficult and computationally intensive. At the same time, visual positioning is easily affected by factors such as the richness of the environment's texture features, changes in illumination, and differences in viewing angles, resulting in poor positioning accuracy and stability. Laser matching in laser positioning is prone to mismatching in the absence of prior information due to factors such as scene layout similarity and scene changes. The errors of the two independent positioning may be large, so even if they are fused in parallel, the accuracy of positioning cannot be guaranteed. This solution is carried out in series through visual positioning and laser positioning, that is, rough visual positioning is performed first, and then laser matching is performed based on the prior conditions. In other words, the visual positioning result is used as the input of laser positioning to further improve the accuracy and ensure the accuracy of positioning.

[0081] Figure 3 A schematic flow chart of a robot initial positioning method provided in another embodiment of the present application is provided as a preferred embodiment. Figure 3 As shown, matching the first top image collected at the current posture of the robot with the pre-established visual map to obtain a matching posture includes:

[0082] S30: determining a target image in the second top image whose similarity with the first top image reaches a threshold value;

[0083] When the first top image is matched with the visual map, specifically, a target image whose similarity with the first top image reaches a threshold is selected from the second top image, for example, the second top image that is most similar and has a similarity of more than 80% can be used as the target image. Specifically, specific position points (for example, corners) and / or specific texture features in the image and / or image color and other features can be determined in the second top image to match with the first top image.

[0084] S31: Calculate the relative angle between the first top image and the target image;

[0085] When the first top image is successfully matched with the visual map, that is, after the target image whose similarity with the first top image reaches the threshold value is determined in the visual map, the relative angle δ between the first top image and the target image is calculated i Specifically, the first top image and the target image are matched with each other for feature points, and the relative angle δ between the first top image and the target image is obtained according to the angle difference of the matched feature points. i .

[0086] S32: Obtain a matching pose according to the global pose and relative angle of the target image.

[0087] It can be understood that the second top image i of each frame corresponds to a global pose. Therefore, when the first top image is successfully matched with the visual map, the matched target image corresponds to a global pose 9x i ,y i ,θ i ).

[0088] Furthermore, the angle θ corresponding to the target image i Superposition relative angle δ i Get the initial angle θ of the current robot i +δ i , and the global position (x i ,y i ) as the initial position of the current robot, so as to obtain the matching posture of the robot (x i ,y i ,θ i +δ i ), that is, to obtain the rough initial positioning pose of the robot.

[0089] It is worth noting that when there is no target image in the second top image whose similarity with the first top image reaches a threshold, the robot can be controlled to move a preset distance to a preset angle, or the robot can send a prompt signal for manually moving the robot. During the movement of the robot, the robot collects the first top image in real time, and matches the real-time collected first top image with the visual map until the target image is matched.

[0090] If the robot matches the target image during the movement, a corresponding matching success signal can be sent in time to prompt the user to stop moving the robot. Of course, after obtaining the target image, the subsequent steps are continued until the initial positioning posture of the robot is determined.

[0091] Therefore, by setting the similarity threshold, the situation where the positioning anomaly uses an image with too low similarity as the target image can be obviously excluded. By calculating the relative angle between the first top image and the target image and then obtaining the matching pose, instead of directly using the global pose of the target image as the matching pose, the accuracy of the obtained matching pose can be improved, thereby improving the success rate and efficiency of subsequent matching, realizing a lightweight matching design, avoiding a large amount of data calculation, and improving the positioning speed.

[0092] Figure 4 A schematic flow chart of another robot initial positioning method provided in another embodiment of the present application, in some optional embodiments, such as Figure 4 As shown, determining a target image in the second top image whose similarity with the first top image reaches a threshold includes:

[0093] S40: extracting first visual feature points of the first top image and second visual feature points of each second top image in the visual map;

[0094] S41: Calculate a first descriptor of a first visual feature point and a second descriptor of a second visual feature point;

[0095] In a specific embodiment, the first top image corresponds to a global pose, a first visual feature point of the first top image is extracted, and a first descriptor of the first visual feature point is calculated. It is worth noting that each (frame) visual image generally contains multiple visual feature points, and each feature point corresponds to a descriptor.

[0096] It can be understood that each second top image i in the visual map corresponds to a global pose (x i ,y i ,θ i ), the second visual feature points can be extracted from the second top image of each frame when building the image, and the second descriptor of the second visual feature points can be calculated.

[0097] S42: Input each first descriptor into a matching model to obtain a first visual word bag, and input each second descriptor into a matching model to obtain a second visual word bag, wherein the matching model is a natural language model obtained through sample training;

[0098] All first descriptors are input into the matching model to obtain a first visual word bag, and all second descriptors are input into the matching model to obtain a second visual word bag. It is worth noting that the matching model is a natural language model obtained through sample training. In some optional embodiments, the matching model can be a bag-of-words model.

[0099] S43: matching the first visual word bag with the second visual word bag to obtain a first matching result, and matching each first descriptor with each second descriptor to obtain a second matching result;

[0100] S44: Determine the target image according to the first matching result and the second matching result.

[0101] Furthermore, based on the first matching result, multiple candidate target images that successfully match the first top image in the visual map can be determined, and based on the second matching result, the number of similar feature points that successfully match the first top image and the candidate target images can be determined, and then the final matching target image can be determined from the candidate target images based on the number of similar feature points.

[0102] In some optional embodiments, determining the target image according to the first matching result and the second matching result includes:

[0103] Determining similar feature points in each second visual feature point that match each first visual feature point according to the first matching result and the second matching result;

[0104] The second top image in which the number of similar feature points reaches a preset value may be used as a target image.

[0105] Specifically, the second top image in which the number of similar feature points in the second visual feature points reaches a preset value is taken as the target image, wherein the similar feature points refer to similarities in specific position points (for example, corners) in the image, and / or similarities in specific textures in the image, and / or similarities in specific colors in the image, and / or similarities in specific shapes in the image (for example, circles, triangles, etc.), etc.

[0106] Furthermore, after the target image is successfully matched, the rough initial positioning posture of the current robot can be determined according to the target image. Specifically, after the image matching is successful, the first top image and the target image are matched for feature points, and the relative angle δ between the first top image and the target image is obtained according to the angle difference of the matched feature points. i ; The global position (x i ,y i ) as the current robot position, and the angle θ of the target image i Superimpose the relative angles to get the current robot angle θ i +δ i The above current robot position and angle are the rough position and posture of the current robot initial positioning (x i ,y i ,θ i +δ i ).

[0107] Therefore, compared with the prior art which requires additional inter-frame motion estimation and calculation of the 3D coordinates of feature points, although the positioning accuracy is higher, the robustness is poor. This solution has a small amount of calculation, low requirements on the robot's chip computing power, fast positioning speed, and because it is a rough positioning, it is more robust to changes in the visual environment. In addition, based on the extraction of visual feature points from the first top image and the second top image, the descriptor and the visual word bag are matched separately, which can achieve fast and accurate matching of images, providing a basis for locating the rough initial positioning posture of the current robot according to the target image.

[0108] As a preferred embodiment, based on the matching posture, matching the first laser point cloud collected at the current posture with the pre-established laser map to determine the initial positioning posture of the robot includes:

[0109] Setting an initial matching position of the first laser point cloud in the laser map according to the matching posture;

[0110] Taking the matching initial position as the center, matching the first laser point cloud with the pre-established laser map within a preset matching range; wherein the preset matching range includes a matching distance and a matching angle;

[0111] The position and posture corresponding to the successful matching of the first laser point cloud and the laser map is used as the initial positioning position and posture of the robot.

[0112] It is worth noting that the laser map is a point cloud map established by laser SLAM technology, and laser matching refers to the matching of the first laser point cloud scanned by the laser radar with the laser map under the current posture of the robot. Specifically, on the basis of the matching posture (rough posture), the matching initial position in the laser map is set according to the matching posture. In some optional embodiments, it can be understood that the matching posture is found in the laser map and the posture is set as the matching initial position.

[0113] Furthermore, with the initial matching position as the center, the search distance range and the search angle range (i.e., the pre-set matching range) are defined, and the optimal matching result is searched within the matching range by a preset algorithm, that is, the first laser point cloud is matched with the laser map, and if the match is successful, the posture corresponding to the successful match is used as the initial positioning posture. The preset algorithm may be, but is not limited to, a branch and bound algorithm, and an exhaustive matching method of the branch and bound algorithm may be used.

[0114] Therefore, the laser matching method of this scheme is an exhaustive matching within a local range, that is, with the visual positioning result as the center, a matching area is delineated, and an exhaustive matching based on the branch and bound algorithm is performed within the area to ensure the best matching result. Compared with the prior art that performs global matching based on laser features, which is very prone to mismatching due to laser similarity, this application further performs laser positioning based on the rough posture, thereby improving the accuracy of the robot's initial positioning posture positioning. In addition, laser positioning is based on matching the laser map with the first laser point cloud under the robot's current posture. This matching method also does not require a large amount of data calculation, further improving the positioning speed.

[0115] On the basis of the above embodiment, if the first laser point cloud fails to match the laser map, the method further includes:

[0116] After controlling the robot to rotate a first preset angle, a second laser point cloud is collected;

[0117] Determine whether the second laser point cloud matches the laser map successfully;

[0118] If the match is successful, the position corresponding to the successful match between the second laser point cloud and the laser map is used as the initial positioning position of the robot;

[0119] If the matching fails, a prompt signal is sent, and when it is determined that the robot has moved, the process returns to the step of matching the top image captured at the current posture of the robot with the pre-established visual map to obtain a matching posture, and subsequent steps are executed until the initial positioning posture of the robot is determined.

[0120] Since laser positioning is easily affected by the environment, for example, temporary obstacles or crowds in front of the robot may cause laser matching failure. In order to avoid the above problems, after determining that the first laser point cloud fails to match the laser map, the robot is controlled to rotate a first preset angle, where the first preset angle is less than 360°. In fact, it is sufficient to control the device that collects the first laser point cloud to rotate at a certain angle.

[0121] By rotating the preset angle, the robot can solve the problem of partial obstruction of the viewing angle or partial changes in the surrounding environment causing matching failure. After rotating the first preset angle, continue to collect the second laser point cloud, and match the second laser point cloud with the laser map. If the match is successful, the position corresponding to the successful matching of the second laser point cloud is used as the initial positioning position of the robot. It can be understood that the second laser point cloud also belongs to the preset matching range.

[0122] Furthermore, if the second laser point cloud fails to match the laser map, the laser point cloud data can be collected again after rotating a certain angle to match the laser map until the initial positioning posture is determined. It should be noted that when the rotation angle is greater than or equal to 360° after the matching fails, the robot is no longer controlled to rotate, that is, the surrounding environment of the robot's current position may have changed significantly. At this time, the robot can be controlled to move, or a signal can be sent to prompt the user to move the robot.

[0123] During the movement of the robot, visual positioning and laser positioning are continuously performed, that is, the step of matching the top image collected at the current posture of the robot with the pre-established visual map to obtain a matching posture is returned, and subsequent steps are executed until the initial positioning posture is determined.

[0124] Therefore, when laser positioning fails, the robot will be triggered to rotate and continue positioning, so as to cope with changes in part of the surrounding environment. If positioning still fails after rotation, positioning can continue during the process of moving the robot, thereby quickly achieving positioning. Even if the environment changes significantly, the robot's positioning accuracy can be guaranteed.

[0125] In a specific embodiment, the visual map and the laser map are established simultaneously, so that each frame of the second top image in the visual map can be bound to a global posture, wherein the generation rule of the visual map can be that the robot collects a frame of the second top image every time it moves a preset distance and / or rotates a second preset angle. Alternatively, the robot collects a frame of the second top image every fixed time during the movement process.

[0126] It can be understood that the visual map obtained by collecting a frame of the second top image after each movement of a preset distance and / or rotation of the second preset angle is relatively uniform, and the image density is not affected by the robot's mapping speed. It can ensure the success rate of image matching with the first top image at any position. At the same time, it can also avoid generating a large number of repeated images in the same place when the robot is stationary. Therefore, the generation rule of the visual map of the present application is preferably composed of collecting a frame of the second top image after each movement of the preset distance and / or rotation of the second preset angle.

[0127] Therefore, when establishing a visual map, this solution only needs to collect image information, extract features from the image, and calculate feature descriptors. A visual word bag is generated based on the feature descriptors of the image, which can be used to match the target image. In addition, after each movement of the robot by a preset distance and / or rotation by a second preset angle, a visual map consisting of a frame of the second top image is collected to ensure the uniformity of the visual map, thereby ensuring the matching success rate of the first top image at any position, while avoiding the storage pressure of the robot caused by collecting a large number of repeated second top images when the robot is stationary in the same place.

[0128] It should be noted that the serial numbers of the steps in the embodiments of the present application do not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0129] In the above embodiments, the robot initial positioning method is described in detail, and the present application also provides an embodiment corresponding to a robot initial positioning device. It should be noted that the present application describes the embodiments of the device part from two perspectives, one is based on the functional module perspective, and the other is based on the hardware structure perspective.

[0130] Figure 5 A schematic diagram of a structure diagram of a robot initial positioning device provided in an embodiment of the present application is shown in FIG. Figure 5 As shown, the device comprises:

[0131] A first matching module 50 is used to match the first top image collected at the current posture of the robot with a pre-established visual map to obtain a matching posture; wherein the visual map includes multiple frames of second top images, and each frame of the second top image corresponds to a global posture;

[0132] The second matching module 51 is used to match the first laser point cloud collected at the current posture with a pre-established laser map based on the matching posture obtained by the first matching module to determine the initial positioning posture of the robot; wherein the laser map is a point cloud map obtained by the robot scanning the surrounding environment.

[0133] Since the robot initial positioning device provided in the embodiment of the present application corresponds to the robot initial positioning method in the above embodiment, the beneficial effects produced are the same.

[0134] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0135] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0136] Figure 6 A schematic diagram of the structure of a robot provided in another embodiment of the present invention is shown in FIG. Figure 6 As shown, the robot includes: a camera 60, a preset laser radar 61, at least one processor 62, a memory 63, and a computer program 64 stored in the memory and executable on at least one processor, wherein the camera 60 is connected to the processor 62 to collect the first top image of the robot in the current position, and the top image refers to the image directly above the robot. The preset laser radar 61 is connected to the processor 62 to collect the first laser point cloud in the current position, and the processor 62 implements the steps in any of the above-mentioned method embodiments when executing the computer program 64.

[0137] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0138] An embodiment of the present application provides a computer program product. When the computer program product runs on a mobile terminal, the mobile terminal can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0139] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the camera / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0140] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0141] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0142] In the embodiments provided in the present application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0143] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0144] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A robot initial positioning method, characterized in that: include: Matching the first top image collected at the current posture of the robot with a pre-established visual map to obtain a matching posture; wherein the visual map includes multiple frames of second top images, and each frame of the second top image corresponds to a global posture; Based on the matching posture, the first laser point cloud collected at the current posture is matched with a pre-established laser map to determine the initial positioning posture of the robot; wherein the laser map is a point cloud map obtained by the robot scanning the surrounding environment.

2. The robot initial positioning method according to claim 1, characterized in that: The step of matching the first top image acquired at the current posture of the robot with the pre-established visual map to obtain a matching posture comprises: Determine a target image in the second top image whose similarity with the first top image reaches a threshold; calculating a relative angle between the first top image and the target image; The matching pose is obtained according to the global pose of the target image and the relative angle.

3. The robot initial positioning method according to claim 2, characterized in that: The step of determining a target image in the second top image whose similarity with the first top image reaches a threshold value comprises: Extracting first visual feature points of the first top image and second visual feature points of each of the second top images in the visual map; Calculating a first descriptor of the first visual feature point and a second descriptor of the second visual feature point; Inputting each of the first descriptors into a matching model to obtain a first visual word bag, and inputting each of the second descriptors into the matching model to obtain a second visual word bag, wherein the matching model is a natural language model obtained through sample training; Matching each of the first visual word bags with each of the second visual word bags to obtain a first matching result, and matching each of the first descriptors with each of the second descriptors to obtain a second matching result; The target image is determined according to the first matching result and the second matching result.

4. The robot initial positioning method according to claim 1, characterized in that: The step of matching the first laser point cloud collected at the current posture with a pre-established laser map based on the matching posture to determine the initial positioning posture of the robot includes: Setting an initial matching position of the first laser point cloud in the laser map according to the matching posture; Taking the matching initial position as the center, matching the first laser point cloud with a pre-established laser map within a preset matching range; wherein the preset matching range includes a matching distance and a matching angle; The posture corresponding to the successful matching of the first laser point cloud and the laser map is used as the initial positioning posture of the robot.

5. The robot initial positioning method according to claim 4, characterized in that: When it is determined that the first laser point cloud fails to match the laser map, the method further includes: After controlling the robot to rotate a first preset angle, a second laser point cloud is collected; Determining whether the second laser point cloud matches the laser map successfully; If the match is successful, the pose corresponding to the successful match between the second laser point cloud and the laser map is used as the initial positioning pose of the robot; If the matching fails, a prompt signal is sent, and when it is determined that the robot has moved, the process returns to the step of matching the top image captured at the current posture of the robot with the pre-established visual map to obtain a matching posture, and subsequent steps are performed until the initial positioning posture of the robot is determined.

6. The robot initial positioning method according to any one of claims 1 to 3, characterized in that: The visual map is formed by collecting a frame of the second top image each time the robot moves a preset distance and / or rotates a second preset angle.

7. The robot initial positioning method according to claim 3, characterized in that: Determining the target image according to the first matching result and the second matching result includes: Determining similar feature points in each of the second visual feature points that match each of the first visual feature points according to the first matching results and the second matching results; The second top image in which the number of similar feature points reaches a preset value is used as the target image.

8. A robot initial positioning device, characterized in that: include: A first matching module is used to match the first top image collected at the current posture of the robot with a pre-established visual map to obtain a matching posture; wherein the visual map includes multiple frames of second top images, and each frame of the second top image corresponds to a global posture; The second matching module is used to match the first laser point cloud collected at the current posture with a pre-established laser map based on the matching posture obtained by the first matching module to determine the initial positioning posture of the robot; wherein the laser map is a point cloud map obtained by the robot scanning the surrounding environment.

9. A robot, characterized in that: include: A camera, a lidar, a memory, a processor, and a computer program stored in the memory and executable on the processor; The camera is connected to the processor and is used to collect a first top image of the robot in its current position; wherein the top image is an image directly above the robot; The laser radar is connected to the processor and is used to collect a first laser point cloud in the current posture; When the processor executes the computer program, the robot initial positioning method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the robot initial positioning method according to any one of claims 1 to 7 is implemented.

Images