Mapping method, positioning method, device, robot and storage medium

Abstract

The present application relates to a mapping method, positioning method, device, robot, and storage medium. The mapping method includes: obtaining a laser point cloud image of the local environment area of the robot's space captured by a laser sensor, and determining the target outline of obstacles included in the local environment area based on the laser point cloud image; obtaining the obstacle type of the obstacle, and based on the obstacle type and target outline, determining the confidence information of each position point included in the target outline, where the obstacle type includes static obstacle type and dynamic obstacle type; and establishing a confidence map of the local environment area based on the position information and confidence information of each position point. This method can improve the robot's positioning accuracy.

Description

Technical Field

[0001] The present application relates to the field of robotics technology, and in particular to a mapping method, a positioning method, a device, a robot, and a storage medium. Background Art

[0002] With the continuous development of science and technology, robots are gradually becoming part of people's lives. As the number of robot application scenarios continues to increase, the requirements for robot mapping and positioning technologies are also becoming increasingly higher. Among them, laser-based SLAM technology is a commonly used technology for robot mapping and positioning.

[0003] Currently, laser-based SLAM technology involves two steps: mapping and localization. Mapping involves deriving the laser device's pose information for each frame from laser data collected from previous and subsequent frames, thereby generating a probabilistic map of the robot's environment. Localization involves matching the laser data from each frame with the probabilistic map to determine the robot's pose information.

[0004] However, when the scene layout changes and there are a large number of dynamic obstacles, the robot's posture information obtained by the above method has low reliability and low positioning accuracy. Summary of the Invention

[0005] Based on this, it is necessary to provide a mapping method, positioning method, device, robot and storage medium that can improve the positioning accuracy of the robot in response to the above technical problems.

[0006] In a first aspect, the present application provides a mapping method, the method comprising:

[0007] Obtaining a laser point cloud image captured by a laser sensor of a local environment area in the space where the robot is located, and determining target contours of obstacles included in the local environment area based on the laser point cloud image;

[0008] Obtaining an obstacle type of the obstacle, and determining confidence information of each position point included in the target outline based on the obstacle type and the target outline, where the obstacle type includes a static obstacle type and a dynamic obstacle type;

[0009] Based on the location information and confidence information of each location point, a confidence map of the local environmental area is established.

[0010] In one embodiment, the determining of confidence information of each position point included in the target contour based on the obstacle type of the obstacle and the target contour includes:

[0011] If the obstacle type is a static obstacle type, the confidence information of the position point corresponding to the obstacle in the target outline is set to the first value;

[0012] If the obstacle type is a dynamic obstacle type, the confidence information of the position point corresponding to the obstacle in the target outline is set to a second value;

[0013] The first value is greater than the second value.

[0014] In one embodiment, the laser point cloud image includes multiple frames of temporally continuous images; determining the target contour of obstacles included in the local environment area based on the laser point cloud image includes:

[0015] Determine the initial outline of each obstacle based on the first frame image in the multiple frame images;

[0016] For each obstacle, the initial contour is expanded based on the initial contour and the laser point cloud image to obtain the target contour of the obstacle.

[0017] In one embodiment, the initial contour is expanded based on the initial contour and the laser point cloud image to obtain the target contour of the obstacle, including:

[0018] For each contour pixel in the initial contour, determine whether the neighboring pixel points corresponding to the contour pixel point have connected pixels in the candidate image. The candidate image is the image other than the first frame image in the laser point cloud image.

[0019] If the neighborhood pixels corresponding to the contour pixels have connected pixels in the candidate image, the initial contour is expanded using the connected pixels to obtain the target contour.

[0020] In one embodiment, the initial contour is expanded based on the initial contour and the laser point cloud image to obtain the target contour of the obstacle, including:

[0021] Obtain the first outline of the obstacle corresponding to the reference image, where the reference image is a frame image other than the first frame image in the laser point cloud image;

[0022] Obtaining a fitting curve corresponding to a second contour, where the second contour is a contour of the obstacle obtained based on an image that is temporally preceding the reference image in the laser point cloud image;

[0023] It is determined whether there are intersecting pixels in the first contour that intersect with the fitting curve. If there are intersecting pixels in the first contour that intersect with the fitting curve, the initial contour is extended using the intersecting pixels to obtain the target contour.

[0024] In one embodiment, the method further comprises:

[0025] Based on the laser point cloud image, a probability grid map corresponding to the local environmental area is established. The probability grid map includes occupancy probability information of each location point in the local environmental area.

[0026] In a second aspect, the present application also provides a positioning method. The method includes:

[0027] Obtain a current laser point cloud image obtained by a laser sensor capturing a local environment area of the space where the robot is located, and determine obstacles included in the current laser point cloud image based on the current laser point cloud image;

[0028] Based on the confidence map and probability grid map of the local environment area, the confidence information and occupancy probability information of each position point in the target outline of the obstacle are determined. The confidence information is determined based on the obstacle type of the obstacle, which includes static obstacle type and dynamic obstacle type.

[0029] The current posture information of the robot is determined based on the confidence information, occupancy probability information and the current laser point cloud image.

[0030] In one embodiment, the confidence information corresponding to the static obstacle type is greater than the confidence information corresponding to the dynamic obstacle type. The current posture information of the robot is determined based on the confidence information, the occupancy probability information, and the current laser point cloud image, including:

[0031] For each position point in the target contour, the product of the occupancy probability information of the position point and the confidence information of the position point is used as the matching score of the position point;

[0032] Based on the matching score and the current laser point cloud image, the current posture information of the robot is determined.

[0033] In a third aspect, the present application also provides a mapping device. The device includes:

[0034] An acquisition module is used to acquire a laser point cloud image captured by a laser sensor of a local environment area in the space where the robot is located, and determine the target outline of obstacles included in the local environment area based on the laser point cloud image;

[0035] a determination module, configured to obtain an obstacle type of the obstacle and, based on the obstacle type and the target profile, determine confidence information of each position point included in the target profile, wherein the obstacle type includes a static obstacle type and a dynamic obstacle type;

[0036] The mapping module is used to build a confidence map of the local environment area based on the location information and confidence information of each location point.

[0037] In a fourth aspect, the present application further provides a positioning device. The device includes:

[0038] An acquisition module is used to acquire a current laser point cloud image obtained by a laser sensor photographing a local environment area in the space where the robot is located, and based on the current laser point cloud image, determine obstacles included in the current laser point cloud image;

[0039] A determination module is used to determine the confidence information and occupancy probability information of each position point in the target outline of the obstacle based on the confidence map and the probability grid map of the local environment area. The confidence information is determined based on the obstacle type of the obstacle, which includes a static obstacle type and a dynamic obstacle type.

[0040] The positioning module is used to determine the current posture information of the robot based on the confidence information, occupancy probability information and the current laser point cloud image.

[0041] In a fifth aspect, the present application further provides a robot. The robot includes a laser sensor, a memory, and a processor. The laser sensor is used to capture a local area in a location where the robot is located to obtain a laser point cloud image. The memory stores a computer program executable on the processor, and the processor is used to implement the steps of the mapping method described in the first aspect or the steps of the positioning method described in the second aspect when executing the computer program.

[0042] In a sixth aspect, the present application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, and when the computer program is executed by a processor, it implements the steps of the mapping method described in the first aspect or the steps of the positioning method described in the second aspect. The mapping method, positioning method, device, robot, and storage medium described above determine the target outline and obstacle type of obstacles included in the local environment area based on the laser point cloud image obtained by the laser sensor of the local environment area in the space where the robot is located. The obstacle type includes a static obstacle type and a dynamic obstacle type; based on the obstacle type and target outline of the obstacle, the confidence information of each position point included in the target outline is determined to establish a confidence map for the local environment area. Since different confidence information is set for different obstacle outlines according to different obstacle types, the distinction between dynamic obstacles and static obstacles in the local environment area is achieved. This enables the robot to understand the scene distribution of obstacles in the local environment area based on the established confidence map when performing positioning, thereby improving the accuracy of the positioning results and the robustness in dynamic environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A schematic structural diagram of a robot in one embodiment;

[0044] Figure 21 is a flow chart of a mapping method according to an embodiment;

[0045] Figure 3 is a schematic diagram of a spatial scene in which a robot is located in one embodiment;

[0046] Figure 4 101 is a flow chart of step 101 in one embodiment;

[0047] Figure 5 202 is a flow chart of step 202 in one embodiment;

[0048] Figure 6 A schematic diagram of eight areas of pixels in one embodiment;

[0049] Figure 7 is a schematic diagram of a spatial scene in which a robot is located in another embodiment;

[0050] Figure 8 is a flow chart of step 202 in another embodiment;

[0051] Figure 9 is a schematic diagram of a contour expansion result in another embodiment;

[0052] Figure 10 Schematic diagram of a flowchart of a mapping method in another embodiment;

[0053] Figure 11 1 is a flow chart of a positioning method according to an embodiment;

[0054] Figure 12 603 in one embodiment;

[0055] Figure 13 is a flowchart of a positioning method in another embodiment;

[0056] Figure 14 is a structural block diagram of a mapping device in one embodiment;

[0057] Figure 15 is a structural block diagram of a positioning device in one embodiment;

[0058] Figure 16 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0060] The mapping method provided in the embodiment of the present application may be executed by a mapping device, which is arranged in a Figure 1 The robot shown can be implemented as part or all of the robot's terminal through software, hardware, or a combination of software and hardware. The terminal can be a personal computer, laptop, media player, smart TV, smartphone, tablet computer, portable wearable device, etc.

[0061] like Figure 1 As shown, the robot is provided with a laser sensor 1. Optionally, the number of the laser sensor 1 can be one or more. The laser sensor can be provided at the top, middle, bottom, left or right of the robot. When there are multiple laser sensors, the installation positions of the laser sensors are different. Figure 1 This is just an example of one robot implementation. In actual applications, the number and installation locations of laser sensors can be determined based on the specific usage scenario and the shape and structure of the robot, and are not limited here.

[0062] Please refer to Figure 2 , which shows a flow chart of a mapping method provided by an embodiment of the present application. This embodiment takes the method applied to a terminal as an example. It is understandable that the method can also be applied to a system including a terminal and a server, and implemented through the interaction between the terminal and the server. Figure 2 As shown, the interaction method of the mobile robot may include the following steps:

[0063] Step 101: obtain a laser point cloud image obtained by a laser sensor photographing a local environment area in the space where the robot is located, and determine the target contours of obstacles included in the local environment area based on the laser point cloud image.

[0064] Optionally, the space where the robot is located is a closed space, such as indoors. The robot moves in the closed space while photographing the space using the laser sensor 1. After taking a certain number of images, a small map submap is constructed based on the captured images.

[0065] The number of obstacles included in the local environment area may be one or more.

[0066] Optionally, the laser point cloud image is processed using a relevant image processing algorithm to obtain the target outline of each obstacle. Specifically, the image processing algorithm is an image segmentation algorithm, an image edge extraction algorithm, a machine learning algorithm, etc.

[0067] Step 102: Obtain the obstacle type of the obstacle, and determine the confidence information of each position point included in the target contour based on the obstacle type and the target contour.

[0068] Among them, obstacle types include static obstacle types and dynamic obstacle types.

[0069] Among them, the static obstacles are obstacles that cannot move autonomously or are difficult to move. For example, corridors, walls, load-bearing columns and tables can be identified as static obstacles; the dynamic obstacles can be objects that can move automatically (such as robots) or objects that can be moved (such as chairs with wheels, suitcases, etc.) and pedestrians.

[0070] As Figure 3 As an example of the spatial scene shown in the figure, the robot ( Figure 3 When using laser sensors to capture laser point cloud images, the obstacles that can be collected are the objects numbered 1-7 in the figure, where 1 and 5 are tables, 2 is a load-bearing column, 3 and 4 are chairs, and 6 is a wall. After obstacle type classification, static obstacles are Figure 3 The obstacles numbered 1, 2, 5, and 6 are the objects marked with multiple lines in the figure. The dynamic obstacles are Figure 3 Obstacles numbered 3 and 4.

[0071] Optionally, a classification algorithm is used to classify obstacles in the laser point cloud image. The classification algorithm may be a support vector machine (SVM) algorithm, a random forest algorithm, a deep learning algorithm, etc. To improve the accuracy of the classification results, the classification results obtained by the classification algorithm may be sent to staff for manual verification and confirmation.

[0072] Optionally, the confidence information is a confidence value. Different confidence values are set for different obstacles based on the obstacle type and obstacle volume.

[0073] Step 103: Create a confidence map of the local environment area based on the location information and confidence information of each location point.

[0074] Optionally, a map coordinate system is established, and based on the map coordinate system, the position information of each location point is converted to obtain the map coordinate point corresponding to each location point; based on the map coordinate point and the corresponding confidence information, a confidence map of the local environmental area is established.

[0075] Optionally, the robot's location is divided into grids, and for each grid corresponding to an obstacle, the confidence information corresponding to the obstacle is set as the value of the grid.

[0076] In this embodiment, based on a laser point cloud image captured by a laser sensor of the local environment region within the robot's space, the target outlines and obstacle types of obstacles within the local environment region are determined. The obstacle types include static and dynamic obstacle types. Based on the obstacle types and target outlines, confidence information for each location point within the target outline is determined to create a confidence map for the local environment region. By assigning different confidence levels to the target outlines of different obstacles based on their types, dynamic and static obstacles in the local environment region are distinguished. This enables the robot to understand the scene distribution of obstacles within the local environment region based on the established confidence map during positioning, improving the accuracy of positioning results and their robustness in dynamic environments.

[0077] In the embodiment of this application, based on Figure 2 The embodiment shown in FIG. 1 is related to the implementation process of determining the confidence information of each position point included in the target contour based on the obstacle type and the target contour in step 102. The implementation process includes:

[0078] If the obstacle type is a static obstacle type, the confidence information of the position point corresponding to the obstacle in the target outline is set to a first value; if the obstacle type is a dynamic obstacle type, the confidence information of the position point corresponding to the obstacle in the target outline is set to a second value, wherein the first value is greater than the second value.

[0079] Optionally, the first value and the second value may be determined manually based on multiple experimental results.

[0080] Optionally, a mapping table between obstacle types and confidence information is stored in the terminal. For each obstacle, after obtaining the obstacle type of the obstacle, the processor calls the mapping table to obtain the corresponding confidence information.

[0081] Optionally, the difference between the first value and the second value is not less than a preset threshold value. For example, the preset threshold value is not less than 1.

[0082] In this embodiment, by setting a higher confidence value for static obstacles and a lower confidence value for dynamic obstacles, the robot is more likely to use static obstacle information when performing positioning, thereby improving the robustness of the method and the reliability of the robot's positioning results.

[0083] In one embodiment, the laser point cloud image includes multiple frames of temporally continuous images. Figure 4As shown, based on any of the above embodiments, this embodiment relates to the implementation process of determining the target contour of the obstacle included in the local area based on the laser point cloud image in step 101, and the implementation process includes the following steps:

[0084] Step 201 : determining the initial outline of each obstacle based on the first frame of the multi-frame image.

[0085] Optionally, the number of image frames included in the laser point cloud image is 40-60 frames.

[0086] Optionally, to reduce the processor's computational workload, after obtaining the first frame, a deep learning algorithm is used to extract and classify the obstacles in the first frame. The processed results are then sent to staff for review and confirmation. Subsequent frames only need to use image processing algorithms to expand the outlines based on the results of the first frame.

[0087] Step 202 : For each obstacle, based on the initial contour and the laser point cloud image, the initial contour is expanded to obtain the target contour of the obstacle.

[0088] Optionally, a region growing algorithm or a clustering algorithm is used to expand the initial contour to obtain a target contour of the obstacle.

[0089] In this embodiment, the target contour of the obstacle is obtained by expanding the initial contour based on the initial contour and the laser point cloud image, thereby achieving acquisition of the complete contour area of the obstacle and further improving the robustness of the method.

[0090] Furthermore, in one implementation, Figure 5 As shown, the above step 202 expands the initial contour of each obstacle based on the initial contour and the laser point cloud image to obtain the target contour of the obstacle, including steps 301 and 302:

[0091] Step 301 : for each contour pixel in the initial contour, determine whether there is a connected pixel in the candidate image to the neighboring pixel points corresponding to the contour pixel.

[0092] The candidate images are images other than the first frame image in the laser point cloud image.

[0093] Optionally, the neighborhood pixel points are the pixel points corresponding to the four neighborhoods of the contour pixel point, that is, when the contour pixel point is (x, y), the pixel points corresponding to its four neighborhoods include: (x-1, y), (x+1, y), (x, y-1) and (x, y+1).

[0094] Optionally, the neighborhood pixel points are the pixel points corresponding to the 8 neighborhoods of the contour pixel point, that is, when the contour pixel point is (x, y), the pixel points corresponding to its 8 neighborhoods include: (x-1, y-1), (x-1, y), (x-1, y+1), (x, y-1), (x, y+1), (x+1, y-1), (x+1, y) and (x+1, y+1), such as Figure 6 As shown, for pixel point P, its corresponding neighborhood pixel point is labeled P i (i=1,2,…,8) pixels.

[0095] Optionally, for each contour pixel point included in the initial contour, determine whether there are connected pixel points in the subsequent frames for each neighboring pixel point of the contour pixel point; if the distance between the pixel point corresponding to a certain position point in the subsequent frame and the neighboring pixel point is less than a preset threshold, the pixel point corresponding to the position point is used as the connected pixel point.

[0096] Furthermore, for the outline pixel points corresponding to the obstacle in the candidate image, it is determined whether the neighboring pixels of the outline pixel points corresponding to the obstacle in the candidate image are connected to pixels in a subsequent image of the candidate image, where the subsequent image is an image that is located after the candidate image in time sequence.

[0097] Optionally, based on all connected pixel points obtained in the above process, a connected pixel point set is obtained.

[0098] Step 302: If there are connected pixels in the candidate image for the neighboring pixels corresponding to the contour pixel, the initial contour is expanded using the connected pixels to obtain the target contour.

[0099] Optionally, based on the obtained set of connected pixel points and the outline pixel points of the obstacle corresponding to the multiple frames of images, a region generation algorithm is used to obtain the target outline.

[0100] Optionally, the initial contour is connected to the laser point cloud image including the connected pixel points corresponding to each frame to expand the initial contour and obtain the target contour.

[0101] In this embodiment, for each contour pixel point in the initial contour, it is determined whether there is a connected pixel point in the candidate image for the neighborhood pixel point corresponding to the contour pixel point. If there is a connected pixel point in the candidate image for the neighborhood pixel point corresponding to the contour pixel point, the initial contour is expanded using the connected pixel point to obtain the target contour. By traversing each contour pixel point to obtain the connected pixel point corresponding to each contour pixel point, the integrity of the obstacle contour extension is improved, thereby improving the reliability of the confidence map.

[0102] Furthermore, some obstacles in the laser point cloud image may be blocked by other objects. Figure 7 Taking the scene where the robot is shown as an example, for obstacle b, when the robot (represented by a triangle in the figure) is driving, object c will block the area where obstacle b is located, thereby causing the outline of obstacle b to be broken.

[0103] Therefore, in view of the above situation, in one implementation, based on any of the above embodiments, as Figure 8 As shown, in the above step 202, for each obstacle, the initial contour is expanded based on the initial contour and the laser point cloud image to obtain the target contour of the obstacle, including steps 401, 402 and 403:

[0104] Step 401: Acquire a first outline of an obstacle corresponding to a reference image.

[0105] The reference image is a frame image other than the first frame image in the laser point cloud image.

[0106] Optionally, based on the time sequence of each frame image, each frame image except the first frame image in the laser point cloud image is sequentially used as a reference image, and the time sequence is the shooting order corresponding to each frame image.

[0107] Optionally, for each obstacle in the reference image, a point cloud clustering algorithm is used to obtain a first outline of each obstacle.

[0108] Step 402: Obtain a fitting curve corresponding to the second contour.

[0109] The second contour is the contour of the obstacle obtained based on an image that is temporally preceding the reference image in the laser point cloud image.

[0110] Optionally, the second contour is a contour obtained based on steps 301 and 302.

[0111] Optionally, a curve fitting algorithm is used to fit the contour pixel points included in the second contour to obtain the fitting curve, wherein the curve fitting algorithm includes straight line fitting, univariate multi-function fitting and Bezier curve fitting.

[0112] Step 403 , determining whether there are intersecting pixels in the first contour that intersect with the fitting curve; if there are intersecting pixels in the first contour that intersect with the fitting curve, the initial contour is extended using the intersecting pixels to obtain the target contour.

[0113] Optionally, if the first contour intersects the fitting curve, the pixel point corresponding to the intersection position is used as the intersection pixel point; or the pixel point within a preset range from the intersection position is used as the intersection pixel point.

[0114] Optionally, the first contour is connected to the intersecting pixel points based on the fitting curve to expand the initial contour to obtain the target contour.

[0115] like Figure 9 As shown, for Figure 7 In the scenario shown, the method provided in this embodiment, that is, the linear fitting method, is used to expand the outline of the obstacle b, thereby connecting the broken left and right parts, and finally obtaining the target outline of the obstacle b.

[0116] In this embodiment, in order to solve the problem that the obstacle contour is broken due to occlusion, the obstacle contour information is expanded and supplemented by adopting a curve fitting method, thereby improving the reliability of the confidence map.

[0117] In one embodiment, based on Figure 2 In the embodiment shown, the mapping method further includes the following steps:

[0118] Based on the laser point cloud image, a probability grid map corresponding to the local environmental area is established. The probability grid map includes occupancy probability information of each location point in the local environmental area.

[0119] The occupancy probability information is obtained based on a probability grid map corresponding to the local area. This probability grid map is a submap created based on the laser point cloud image captured by the laser sensor. This probability grid map is an important map format that divides the robot's location into grids, each of which has only two states: occupied or idle. The value in the grid is the occupancy probability value (i.e., occupancy probability information), which is used to indicate the probability that the grid is occupied.

[0120] In this embodiment, the establishment of a probability grid map is achieved, which facilitates the subsequent positioning of the robot based on the probability grid map.

[0121] In one embodiment, Figure 10 As shown, a mapping method is provided, including:

[0122] Step 501: Acquire a laser point cloud image obtained by capturing a local environment area of the space where the robot is located by a laser sensor. The laser point cloud image includes multiple frames of continuous images in time sequence.

[0123] Step 502: Determine the initial outline of each obstacle based on the first frame of the multiple frames.

[0124] Step 503 , for each contour pixel point in the initial contour, determine whether there is a connected pixel point in the candidate image corresponding to the area pixel point of the contour pixel point. The candidate image is the image other than the first frame image in the laser point cloud image.

[0125] Step 504: If there are connected pixels in the candidate image for the area pixels corresponding to the contour pixels, the initial contour is expanded using the connected pixels to obtain the target contour.

[0126] Step 505 : obtaining a first outline of the obstacle corresponding to a reference image, where the reference image is a frame image other than the first frame image in the laser point cloud image.

[0127] Step 506 : Obtain a fitting curve corresponding to a second contour, where the second contour is a contour of the obstacle obtained based on an image that is temporally preceding the reference image in the laser point cloud image.

[0128] Step 507 , determining whether there are intersecting pixels in the first contour that intersect with the fitting curve; if there are intersecting pixels in the first contour that intersect with the fitting curve, the initial contour is expanded using the intersecting pixels to obtain the target contour.

[0129] Step 508: Obtain the obstacle type of the obstacle, which includes a static obstacle type and a dynamic obstacle type.

[0130] Step 509: If the obstacle type is a static obstacle type, the confidence information of the position point corresponding to the obstacle in the target outline is set to a first value; if the obstacle type is a dynamic obstacle type, the confidence information of the position point corresponding to the obstacle in the target outline is set to a second value.

[0131] The first value is greater than the second value.

[0132] Step 510: Create a confidence map of the local environment area based on the location information and confidence information of each location point.

[0133] Step 511: Create a probability grid map corresponding to the local environment area based on the laser point cloud image.

[0134] The probability grid map includes occupancy probability information of each location point in the local environment area.

[0135] This embodiment distinguishes between dynamic and static obstacles in a local area by assigning different confidence levels to the outlines of different obstacle types. This achieves the goal of assigning higher confidence levels to static obstacles and lower confidence levels to dynamic obstacles. This allows the robot to understand the distribution of scenes in the local environment during positioning, improving the accuracy of positioning results and robustness in dynamic environments. Furthermore, by expanding the outlines of obstacles, the reliability of this method is further enhanced.

[0136] Laser-based SLAM technology also includes a positioning process, which is to match the laser data of each frame with the constructed map to determine the corresponding position information of the robot. Therefore, for this positioning process, please refer to Figure 11 .like Figure 11 As shown, the embodiment of the present application provides a positioning method, including the following steps:

[0137] Step 601: obtain a current laser point cloud image obtained by a laser sensor shooting a local environment area of the space where the robot is located, and determine obstacles included in the current laser point cloud image based on the current laser point cloud image.

[0138] Step 602: Determine the confidence information and occupancy probability information of each position point in the target outline of the obstacle based on the confidence map and the probability grid map of the local environment area.

[0139] The confidence information is determined based on the obstacle type of the obstacle, and the obstacle type includes a static obstacle type and a dynamic obstacle type.

[0140] Specifically, based on the confidence map of the local environment area, the confidence information of each position point in the target outline of the obstacle included in the local environment area is determined; based on the probability grid map of the local environment area, the occupancy probability information of each position point in the target outline of the obstacle included in the local environment area is determined.

[0141] Step 603: Determine the current posture information of the robot based on the confidence information, the occupancy probability information, and the current laser point cloud image.

[0142] The posture information includes the position information and orientation information of the robot.

[0143] Optionally, the target contour area of the obstacle corresponding to the area with higher confidence information is selected as a reference object, and then based on the occupancy probability information and the current laser point cloud image, the previous laser point cloud image is matched with the probability grid map to obtain the current posture information of the robot.

[0144] In this embodiment, based on the confidence map and probability grid map of the local environment area, the confidence information and occupancy probability information of each position point in the target contour of the obstacle are determined, and the confidence information, occupancy probability information and the current laser point cloud image are fully considered in the current posture information of the robot, thereby realizing the determination of the scene distribution of obstacles in the local environment area based on the confidence information in the local environment area, thereby improving the accuracy of the positioning results and the robustness in a dynamic environment.

[0145] In one embodiment, the confidence information corresponding to the static obstacle type is greater than the confidence information corresponding to the dynamic obstacle type. Figure 12 As shown, based on Figure 11 The embodiment shown in FIG. 6 is related to determining the current posture information of the robot according to the confidence information, the occupancy probability information and the current laser point cloud image in step 603, including steps 701 and 702:

[0146] Step 701 : For each position point in the target contour, the product of the occupancy probability information of the position point and the confidence information of the position point is used as the matching score of the position point.

[0147] Specifically, the expression corresponding to the matching score is:

[0148] score(x,y)=p(x,y)*w(x,y),

[0149] Among them, score(x,y) represents the matching score corresponding to the location point (x,y), p(x,y) represents the occupancy probability information corresponding to the location point (x,y), and w(x,y) represents the confidence information corresponding to the location point (x,y).

[0150] Step 702: Determine the current posture information of the robot based on the matching score and the current laser point cloud image.

[0151] Optionally, for each obstacle, the matching score corresponding to the obstacle is calculated using the following formula:

[0152]

[0153] Among them, Sco re represents the matching score corresponding to the obstacle, j represents the number of location points included in the obstacle, and score(x,y) represents the matching score corresponding to the location points (x,y) included in the obstacle.

[0154] Optionally, the obstacles are arranged in descending order according to their matching scores, and several obstacles with the highest ranking positions are selected as reference objects. The robot's position and posture are determined based on the matching scores and the current laser point cloud image.

[0155] In this embodiment, by taking the product of the occupancy probability information of the location point and the confidence information of the location point as the matching score of the location point, the purpose of assigning a higher matching score to static obstacles and a lower matching score to dynamic obstacles is achieved, thereby improving the probability of locating the robot based on static obstacles. This method is simple and has a small amount of calculation.

[0156] In one embodiment, Figure 13 As shown, a positioning method is provided, including:

[0157] Step 801: obtain a current laser point cloud image obtained by a laser sensor shooting a local environment area of the space where the robot is located, and determine obstacles included in the current laser point cloud image based on the current laser point cloud image.

[0158] Step 802 : Based on the confidence map and the probability grid map of the local environment area, confidence information and occupancy probability information of each position point in the target outline of the obstacle are determined.

[0159] The confidence information is determined based on the obstacle type of the obstacle, which includes a static obstacle type and a dynamic obstacle type. The confidence information corresponding to the static obstacle type is greater than the confidence information corresponding to the dynamic obstacle type.

[0160] Step 803 : For each position point in the target contour, the product of the occupancy probability information of the position point and the confidence information of the position point is used as the matching score of the position point.

[0161] Step 804: Determine the current posture information of the robot based on the matching score and the current laser point cloud image.

[0162] In this embodiment, the purpose of assigning a higher matching score to static obstacles and a lower matching score to dynamic obstacles is achieved, thereby improving the probability of positioning the robot based on static obstacles, thereby improving the accuracy of the positioning results and the robustness in dynamic environments.

[0163] It should be understood that, although the steps in the flowcharts of the above embodiments are shown in sequence as indicated by the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be performed in other orders. Moreover, at least a portion of the steps in the flowcharts of the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily to be performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0164] Based on the same inventive concept, the present application also provides a mapping device for implementing the aforementioned mapping method. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations of one or more positioning device embodiments provided below can be found in the above-mentioned limitations of the mapping method and will not be further elaborated here.

[0165] In one embodiment, Figure 14 As shown, a mapping device is provided, comprising: an acquisition module 100, a determination module 200 and a mapping module 300, wherein:

[0166] An acquisition module 100 is configured to acquire a laser point cloud image captured by a laser sensor of a local environment region in the space where the robot is located, and determine target contours of obstacles included in the local environment region based on the laser point cloud image;

[0167] A determination module 200 is configured to obtain an obstacle type of the obstacle and determine confidence information of each position point included in the target profile based on the obstacle type and the target profile. The obstacle type includes a static obstacle type and a dynamic obstacle type.

[0168] The mapping module 300 is used to build a confidence map of the local environment area based on the location information and confidence information of each location point.

[0169] In one embodiment, the determining module 200 is specifically configured to:

[0170] If the obstacle type is a static obstacle type, the confidence information of the position point corresponding to the obstacle in the target outline is set to a first value; if the obstacle type is a dynamic obstacle type, the confidence information of the position point corresponding to the obstacle in the target outline is set to a second value; wherein the first value is greater than the second value.

[0171] In one embodiment, the laser point cloud image includes multiple frames of temporally continuous images; the acquisition module 100 is specifically configured to:

[0172] Determine the initial outline of each obstacle based on the first frame image in the multiple frame images;

[0173] For each obstacle, the initial contour is expanded based on the initial contour and the laser point cloud image to obtain the target contour of the obstacle.

[0174] In one embodiment, the acquisition module 100 is further configured to:

[0175] For each contour pixel in the initial contour, determine whether the neighboring pixel points corresponding to the contour pixel point have connected pixels in the candidate image. The candidate image is the image other than the first frame image in the laser point cloud image.

[0176] If the neighborhood pixels corresponding to the contour pixels have connected pixels in the candidate image, the initial contour is expanded using the connected pixels to obtain the target contour.

[0177] In one embodiment, the acquisition module 100 is further configured to:

[0178] Obtain the first outline of the obstacle corresponding to the reference image, where the reference image is a frame image other than the first frame image in the laser point cloud image;

[0179] Obtaining a fitting curve corresponding to a second contour, where the second contour is a contour of the obstacle obtained based on an image that is temporally preceding the reference image in the laser point cloud image;

[0180] It is determined whether there are intersecting pixels in the first contour that intersect with the fitting curve. If there are intersecting pixels in the first contour that intersect with the fitting curve, the initial contour is extended using the intersecting pixels to obtain the target contour.

[0181] In one embodiment, the mapping device is further configured to:

[0182] Based on the laser point cloud image, a probability grid map corresponding to the local environmental area is established. The probability grid map includes occupancy probability information of each location point in the local environmental area.

[0183] Each module in the aforementioned mapping device may be implemented in whole or in part through software, hardware, or a combination thereof. Each module may be embedded in or independent of a processor in a computer device in the form of hardware, or may be stored in a computer device memory in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0184] Based on the same inventive concept, embodiments of the present application also provide a positioning device for implementing the aforementioned positioning method. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations of one or more positioning device embodiments provided below can be found in the above-described limitations of the positioning method and will not be further elaborated here.

[0185] In one embodiment, Figure 15 As shown, a positioning device is provided, including: an acquisition module 400, a determination module 500 and a positioning module 600, wherein:

[0186] An acquisition module 400 is configured to acquire a current laser point cloud image captured by a laser sensor of a local environment area in the space where the robot is located, and determine obstacles included in the current laser point cloud image based on the current laser point cloud image;

[0187] A determination module 500 is configured to determine confidence information and occupancy probability information of each location point in a target outline of an obstacle based on a confidence map and a probability grid map of a local environment area, wherein the confidence information is determined based on an obstacle type, which includes a static obstacle type and a dynamic obstacle type.

[0188] The positioning module 600 is used to determine the current posture information of the robot based on the confidence information, the occupancy probability information and the current laser point cloud image.

[0189] In one embodiment, the confidence information corresponding to the static obstacle type is greater than the confidence information corresponding to the dynamic obstacle type. The positioning module 600 is specifically configured to:

[0190] For each position point in the target contour, the product of the occupancy probability information of the position point and the confidence information of the position point is used as the matching score of the position point;

[0191] Based on the matching score and the current laser point cloud image, the current posture information of the robot is determined.

[0192] Each module in the positioning device described above can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in hardware form, or can be stored in a memory in a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0193] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 16As shown. The computer device includes a processor, a memory and a communication interface connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a positioning method is implemented.

[0194] Those skilled in the art will understand that Figure 16 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0195] In one embodiment, a robot is provided. The robot includes a laser sensor, a memory, and a processor. The laser sensor is configured to capture a local area in a location where the robot is located to obtain a laser point cloud image. The memory stores a computer program executable on the processor. The processor is configured to implement the following steps when executing the computer program:

[0196] Obtaining a laser point cloud image captured by a laser sensor of a local environment area in the space where the robot is located, and determining target contours of obstacles included in the local environment area based on the laser point cloud image;

[0197] Obtaining an obstacle type of the obstacle, and determining confidence information of each position point included in the target outline based on the obstacle type and the target outline, where the obstacle type includes a static obstacle type and a dynamic obstacle type;

[0198] Based on the location information and confidence information of each location point, a confidence map of the local environmental area is established.

[0199] In one embodiment, the determination of confidence information of each position point included in the target outline based on the obstacle type and the target outline includes:

[0200] If the obstacle type is a static obstacle type, the confidence information of the position point corresponding to the obstacle in the target outline is set to a first value; if the obstacle type is a dynamic obstacle type, the confidence information of the position point corresponding to the obstacle in the target outline is set to a second value; wherein the first value is greater than the second value.

[0201] In one embodiment, the laser point cloud image includes multiple frames of temporally continuous images; determining the target contour of obstacles included in the local environment area based on the laser point cloud image includes:

[0202] Determine the initial outline of each obstacle based on the first frame image in the multiple frame images;

[0203] For each obstacle, the initial contour is expanded based on the initial contour and the laser point cloud image to obtain the target contour of the obstacle.

[0204] In one embodiment, the initial contour is expanded based on the initial contour and the laser point cloud image to obtain the target contour of the obstacle, including:

[0205] For each contour pixel point in the initial contour, determine whether the neighborhood pixel points corresponding to the contour pixel point have connected pixel points in the candidate image. The candidate image is the image other than the first frame image in the laser point cloud image; if the neighborhood pixel points corresponding to the contour pixel point have connected pixel points in the candidate image, the initial contour is expanded using the connected pixel points to obtain the target contour.

[0206] In one embodiment, the initial contour is expanded based on the initial contour and the laser point cloud image to obtain the target contour of the obstacle, including:

[0207] The first outline of the obstacle corresponding to the reference image is obtained, where the reference image is a frame image other than the first frame image in the laser point cloud image; the fitting curve corresponding to the second outline is obtained, where the second outline is the outline of the obstacle obtained based on the image that is temporally before the reference image in the laser point cloud image; and it is determined whether there are intersecting pixels in the first outline that intersect with the fitting curve. If there are intersecting pixels in the first outline that intersect with the fitting curve, the initial outline is extended using the intersecting pixels to obtain the target outline.

[0208] In one embodiment, the processor further implements the following steps when executing the computer program:

[0209] Based on the laser point cloud image, a probability grid map corresponding to the local environmental area is established. The probability grid map includes occupancy probability information of each location point in the local environmental area.

[0210] In one embodiment, a robot is provided. The robot includes a laser sensor, a memory, and a processor. The laser sensor is configured to capture a local area in a location where the robot is located to obtain a laser point cloud image. The memory stores a computer program executable on the processor. The processor is configured to implement the following steps when executing the computer program:

[0211] Obtain a current laser point cloud image obtained by a laser sensor capturing a local environment area of the space where the robot is located, and determine obstacles included in the current laser point cloud image based on the current laser point cloud image;

[0212] Based on the confidence map and probability grid map of the local environment area, the confidence information and occupancy probability information of each position point in the target outline of the obstacle are determined. The confidence information is determined based on the obstacle type of the obstacle, which includes static obstacle type and dynamic obstacle type.

[0213] The current posture information of the robot is determined based on the confidence information, occupancy probability information and the current laser point cloud image.

[0214] In one embodiment, the confidence information corresponding to the static obstacle type is greater than the confidence information corresponding to the dynamic obstacle type. The current posture information of the robot is determined based on the confidence information, the occupancy probability information, and the current laser point cloud image, including:

[0215] For each position point in the target contour, the product of the occupancy probability information of the position point and the confidence information of the position point is used as the matching score of the position point; based on the matching score and the current laser point cloud image, the current position information of the robot is determined.

[0216] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned mapping method embodiments or the steps of any of the above-mentioned positioning method embodiments are implemented.

[0217] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0218] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0219] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A mapping method, characterized in that: The method comprises: Obtaining a laser point cloud image of a local environment region of the space where the robot is located, obtained by photographing the image with a laser sensor, and determining target contours of obstacles included in the local environment region based on the laser point cloud image; and establishing a probability grid map corresponding to the local environment region based on the laser point cloud image; Obtaining an obstacle type of the obstacle, and determining confidence information of each position point included in the target outline based on the obstacle type of the obstacle and the target outline, wherein the obstacle type includes a static obstacle type and a dynamic obstacle type; Based on the position information of each position point and the confidence information, a confidence map of the local environment area is established to match the collected multi-frame laser point cloud images in the local environment area of the space where the robot is located with the constructed map to determine the corresponding posture information of the robot; wherein, the process of determining the corresponding posture information of the robot includes: determining the occupancy probability information of each position point in the target contour according to the confidence map and the probability grid map; taking the product of the occupancy probability information of each position point in the target contour and the confidence information of each position point as the matching score of the corresponding position point; and determining the posture information of the robot at the moment of collecting each laser point cloud image based on the matching score of each position point and the multi-frame laser point cloud images in the local environment area.

2. The method according to claim 1, characterized in that The determining, based on the obstacle type of the obstacle and the target outline, confidence information of each position point included in the target outline includes: If the obstacle type is a static obstacle type, setting the confidence information of the position point corresponding to the obstacle in the target outline to a first value; If the obstacle type is a dynamic obstacle type, setting the confidence information of the position point corresponding to the obstacle in the target outline to a second value; The first value is greater than the second value.

3. The method according to any one of claims 1 or 2, characterized in that The laser point cloud image includes a plurality of frames of temporally continuous images; and determining the target contour of the obstacle included in the local environment area based on the laser point cloud image includes: Determine the initial outline of each obstacle based on the first frame image in the multiple frame images; For each obstacle, based on the initial contour and the laser point cloud image, the initial contour is expanded to obtain a target contour of the obstacle.

4. The method according to claim 3, characterized in that The step of expanding the initial contour based on the initial contour and the laser point cloud image to obtain a target contour of the obstacle includes: For each contour pixel point in the initial contour, determine whether a neighboring pixel point corresponding to the contour pixel point has a connected pixel point in a candidate image, where the candidate image is an image in the laser point cloud image other than the first frame image; If there are connected pixels in the candidate image for the neighboring pixels corresponding to the contour pixels, the initial contour is expanded using the connected pixels to obtain the target contour.

5. The method according to claim 3, characterized in that The step of expanding the initial contour based on the initial contour and the laser point cloud image to obtain a target contour of the obstacle includes: Acquire a first outline of the obstacle corresponding to a reference image, where the reference image is a frame image other than the first frame image in the laser point cloud image; Obtaining a fitting curve corresponding to a second contour, where the second contour is a contour of the obstacle obtained based on an image that is temporally preceding the reference image in the laser point cloud image; Determine whether there are intersecting pixel points in the first contour that intersect with the fitting curve. If there are intersecting pixel points in the first contour that intersect with the fitting curve, expand the initial contour using the intersecting pixel points to obtain the target contour.

6. The method according to claim 1, characterized in that The probability grid map includes occupancy probability information of each location point in the local environment area.

7. A positioning method, characterized in that: The method comprises: Obtaining a current laser point cloud image obtained by a laser sensor capturing a local environment area of the space where the robot is located, and determining obstacles included in the current laser point cloud image based on the current laser point cloud image; Determining occupancy probability information of each position point in the target outline of the obstacle based on the confidence map and the probability grid map of the local environment area; and determining confidence information of each position point in the target outline of the obstacle based on the obstacle type; the obstacle type includes a static obstacle type and a dynamic obstacle type; For each position point in the target outline, taking the product of the occupancy probability information of the position point and the confidence information of the position point as the matching score of the position point; Based on the matching scores and the current laser point cloud image, the current posture information of the robot is determined.

8. A mapping device, characterized in that: The device comprises: an acquisition module, configured to acquire a laser point cloud image captured by a laser sensor of a local environment region in the space where the robot is located, and determine target contours of obstacles included in the local environment region based on the laser point cloud image; and, based on the laser point cloud image, establish a probability grid map corresponding to the local environment region; a determination module, configured to obtain an obstacle type of the obstacle and, based on the obstacle type of the obstacle and the target profile, determine confidence information of each position point included in the target profile, wherein the obstacle type includes a static obstacle type and a dynamic obstacle type; A mapping module is used to establish a confidence map of the local environment area based on the position information of each position point and the confidence information, so as to match the collected multi-frame laser point cloud images in the local environment area of the space where the robot is located with the constructed map to determine the corresponding posture information of the robot; wherein, the process of determining the corresponding posture information of the robot includes: determining the occupancy probability information of each position point in the target contour according to the confidence map and the probability grid map; taking the product of the occupancy probability information of each position point in the target contour and the confidence information of each position point as the matching score of the corresponding position point; and determining the posture information of the robot at the moment of collecting each laser point cloud image based on the matching score of each position point and the multi-frame laser point cloud images in the local environment area.

9. A positioning device, characterized in that: The device comprises: an acquisition module, configured to acquire a current laser point cloud image obtained by a laser sensor capturing a local environment area of the space where the robot is located, and determine obstacles included in the current laser point cloud image based on the current laser point cloud image; a determination module, configured to determine, based on the confidence map and the probability grid map of the local environment area, confidence information and occupancy probability information of each position point in the target outline of the obstacle, wherein the confidence information is determined based on the obstacle type of the obstacle, and the obstacle type includes a static obstacle type and a dynamic obstacle type; A positioning module is used to, for each position point in the target contour, use the product of the occupancy probability information of the position point and the confidence information of the position point as the matching score of the position point; and determine the current posture information of the robot based on each matching score and the current laser point cloud image.

10. A robot, characterized in that: The robot includes a laser sensor, a memory, and a processor. The laser sensor is used to capture a local area in the robot's location to obtain a laser point cloud image. The memory stores a computer program that can be run on the processor. The processor is used to implement the steps of the mapping method according to any one of claims 1 to 6, or the steps of the positioning method according to claim 7 when executing the computer program.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the mapping method according to any one of claims 1 to 6 or the steps of the positioning method according to claim 7 are implemented.

Images