Robot positioning method, device, equipment, medium and program product

Abstract

The embodiment of the invention provides a robot positioning method and device, equipment, a medium and a program product, and relates to the technical field of robos.The method comprises the steps that based on space description information of a working environment where a robot is located, a first static point cloud located in a static space in the working environment is extracted from environment point clouds, the environment point cloud is a point cloud acquired by point cloud acquisition equipment arranged on the robot, and the expected point cloud structure of the static space tends to be consistent with the point cloud structure of a point cloud map of the working environment; and point cloud registration is carried out on the first static point cloud and the point cloud map, and the pose of the robot is obtained according to a registration result. By applying the scheme provided by the embodiment of the invention, the positioning accuracy and stability of the robot can be improved.

Description

Technical Field

[0001] This invention relates to the field of robotics, and in particular to a robot positioning method, apparatus, equipment, medium, and program product. Background Technology

[0002] With the rapid development of robotics technology, robots with navigation capabilities have been widely used in many scenarios such as indoor warehousing, logistics distribution, and commercial services. One of the core technologies for achieving autonomous robot navigation is Simultaneous Localization and Mapping (SLAM). Using this technology, robots can construct point cloud maps (mapping) based on point cloud data collected by point cloud acquisition devices and determine their own pose (localization) within the point cloud map.

[0003] The localization process is performed under the assumption that the point cloud structure remains largely unchanged; that is, it is assumed that the point cloud structure of the real-time point cloud collected by the robot is essentially consistent with the point cloud structure of the point cloud map. Under this assumption, by registering the real-time point cloud and the point cloud map, the robot's pose in the point cloud map can be obtained, thus achieving robot localization.

[0004] However, once the point cloud map is constructed, it generally remains fixed in the short term, while the robot's surrounding environment may constantly change dynamically. For example, in public places such as shopping malls, there are often constantly changing crowds near the robot. This can easily cause the point cloud structure of the real-time point cloud collected by the robot to differ significantly from the point cloud structure of the point cloud map, thus rendering the registration results invalid and resulting in a significant reduction in the robot's positioning accuracy and stability. Summary of the Invention

[0005] The purpose of this invention is to provide a robot positioning method, apparatus, device, medium, and program product to improve the positioning accuracy of robots. The specific technical solution is as follows:

[0006] In a first aspect, embodiments of the present invention provide a robot localization method, comprising:

[0007] Based on the spatial description information of the robot's working environment, the first static point cloud located in the static space of the working environment is extracted from the environmental point cloud. The environmental point cloud is the point cloud collected by the point cloud acquisition device set on the robot. The expected point cloud structure of the static space is consistent with the point cloud structure of the point cloud map of the working environment.

[0008] Point cloud registration is performed between the first static point cloud and the point cloud map, and the robot's pose is obtained based on the registration result.

[0009] In a second aspect, embodiments of the present invention provide a robot positioning device, comprising:

[0010] The static point cloud extraction module is used to extract the first static point cloud located in the static space of the working environment from the environmental point cloud based on the spatial description information of the working environment of the robot. The environmental point cloud is the point cloud collected by the point cloud acquisition device set on the robot. The expected point cloud structure of the static space is consistent with the point cloud structure of the point cloud map of the working environment.

[0011] The localization module is used to perform point cloud registration between the first static point cloud and the point cloud map, and obtain the robot's pose based on the registration result.

[0012] Thirdly, embodiments of the present invention provide an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0013] Memory, used to store computer programs;

[0014] When a processor executes a program stored in memory, it implements the steps of the method described in the first aspect.

[0015] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0016] Fifthly, embodiments of the present invention provide a computer program product containing instructions that, when run on a computer, cause the computer to perform the steps of the method described in the first aspect.

[0017] As can be seen from the above, when using the solution provided in this embodiment of the invention for robot localization, the robot pose is not obtained by directly configuring the environmental point cloud and point cloud map collected by the robot. Instead, it utilizes the spatial description information of the robot's working environment to remove dynamic point clouds from the environmental point cloud that are inconsistent with the point cloud structure of the point cloud map, thereby selecting static point clouds that are consistent with the point cloud structure of the point cloud map. Then, only static point clouds are used for point cloud registration with the point cloud map, eliminating the interference caused by dynamic point clouds to point cloud registration, thereby reducing registration errors, improving the accuracy of registration results, and thus improving the robot's localization accuracy and stability.

[0018] In highly dynamic scenarios where environmental changes are frequent and point cloud features are severely lacking, the solution provided by the embodiments of the present invention can still maintain stable and reliable pose estimation, ensure the positioning accuracy of the robot, fundamentally improve the positioning resilience and operational safety of the robot in highly dynamic scenarios, and provide a guarantee for the reliable deployment and application of the robot in highly dynamic environments.

[0019] Of course, implementing any product or method of the present invention does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0021] Figure 1 A flowchart illustrating the first robot localization method provided in an embodiment of the present invention;

[0022] Figure 2 A schematic diagram of a static region provided in an embodiment of the present invention;

[0023] Figure 3 This is a flowchart illustrating the second robot localization method provided in an embodiment of the present invention;

[0024] Figure 4 A schematic diagram of a robot positioning process provided in an embodiment of the present invention;

[0025] Figure 5 This is a schematic diagram of the structure of a robot positioning device provided in an embodiment of the present invention;

[0026] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art based on the present invention are within the scope of protection of the present invention.

[0028] First, the application scenarios of the solutions provided in the embodiments of the present invention will be introduced.

[0029] The application scenarios of the solution provided in this embodiment of the invention are: various scenarios that require robot positioning and task execution based on positioning results.

[0030] In scenarios where robots perform tasks, robot localization is often the foundation for subsequent processes. For example, the robot first locates its own position, then plans a path from its current position to the task target based on the localization result, and then follows the planned path to reach the task target and perform the set operations for the task target.

[0031] The following examples will provide a more intuitive explanation of the above application scenarios.

[0032] Example 1: Product recommendation scenario

[0033] In this scenario, the robot is deployed in a shopping mall. During the initial deployment, the robot uses a point cloud acquisition device to scan the surrounding environment and constructs a point cloud map of the shopping mall based on SLAM technology.

[0034] In the application phase, customers can input their shopping needs into the robot via voice or touchscreen. After recognizing the customer's needs based on the input, the robot can query the location of the target product according to the product database, and locate its own position by registering with the real-time collected environmental point cloud and point cloud map. Then, it can plan a path based on its own position and the location of the target product, and guide the customer to the location of the target product according to the planned path, or autonomously grab the target product and deliver it to the customer.

[0035] Example 2: Home companionship scenario

[0036] In this scenario, the robot is deployed in the user's living room, bedroom, and other locations. Similarly, during the initial deployment, the robot uses a point cloud acquisition device to scan the surrounding environment and constructs a point cloud map of the aforementioned locations based on SLAM technology.

[0037] In the application phase, the robot can perform operations according to its set companion functions. For example, when the user sets a medication reminder time, the robot locates its own position by registering with the real-time collected environmental point cloud and point cloud map, and plans a path based on the pre-acquired medicine box location; then, it retrieves the medicine from the medicine box location according to the planned path and delivers it to the user's bedroom, reminding the user to take the medicine via voice.

[0038] The localization process is performed under the assumption that the point cloud structure remains largely unchanged; that is, it is assumed that the point cloud structure of the real-time point cloud acquired by the robot is essentially consistent with the point cloud structure of the point cloud map. Under this assumption, by registering the real-time point cloud and the point cloud map, the robot's pose in the point cloud map can be obtained, thus achieving robot localization. It is evident that the stability and accuracy of the registration process heavily depend on the effective correspondence between the feature structures of the real-time acquired point cloud and the point cloud map.

[0039] However, point cloud maps are generally fixed in the short term after construction, while the robot's surrounding environment may constantly change dynamically. For example, in public places such as shopping malls, exhibition halls, and science parks, robots easily attract crowds due to their novelty. These dense and dynamically changing pedestrians can severely obstruct the robot's scanning of the surrounding static environment (such as walls, pillars, and fixed furniture), causing significant changes in the point cloud structure of the real-time point cloud collected by the robot compared to the point cloud map. This leads to the blurring or even complete invalidation of the effective correspondence between the feature structures of the real-time point cloud and the point cloud map. Consequently, the accuracy of the localization results obtained by registering the real-time point cloud with the point cloud map will decrease significantly, making it difficult for the robot to know its exact position in the environment. This can lead to task interruption and navigation stagnation, or even more serious collisions and safety accidents.

[0040] It is evident that the aforementioned problems severely restrict the reliable deployment and application of robots in complex, highly dynamic human traffic environments.

[0041] In view of the above, embodiments of the present invention provide a robot positioning scheme to improve the positioning accuracy of the robot.

[0042] The robot positioning scheme provided in the embodiments of the present invention will be described in detail below with reference to the flowchart.

[0043] See Figure 1 The above is a flowchart illustrating the first robot localization method provided in an embodiment of the present invention. The method includes the following steps S101 to S102.

[0044] Step S101: Based on the spatial description information of the robot's working environment, extract the first static point cloud located in the static space of the working environment from the environmental point cloud.

[0045] The working environment of a robot, that is, the environment in which the robot performs its tasks, varies depending on the specific application scenario. For example, in the aforementioned product guidance scenario, the working environment could be a shopping mall or other shopping venue; in the aforementioned home service scenario, the working environment could be a living room, bedroom, or other similar space.

[0046] The environmental point cloud refers to the point cloud currently collected by the point cloud acquisition device installed on the robot. The aforementioned point cloud acquisition device may specifically be a lidar, a binocular camera, a depth camera, etc., but this embodiment of the invention does not limit it.

[0047] The following describes the static space in the work environment.

[0048] Static space does not refer to a space in the working environment where objects are completely stationary, but rather to a space where the positions of objects have not changed significantly compared to the working environment at the time of point cloud map construction. In the solution provided by this embodiment, objects within a space are considered stationary if their positions have not changed significantly. Furthermore, since the objects within this space remain stationary from the time the point cloud map is constructed, the expected point cloud structure of the point cloud in this space will tend to be consistent with the point cloud structure of the point cloud map.

[0049] The embodiments of the present invention do not limit the specific setting of the static space described above, but only require that the expected point cloud structure of the static space is consistent with the point cloud structure of the point cloud map of the working environment.

[0050] Specifically, "converging with consistency" means that the expected point cloud structure in the static space has a high degree of similarity to the corresponding point cloud structure in the point cloud map, such as exceeding a set threshold.

[0051] Based on the above explanation, this step essentially involves extracting static point clouds from the environmental point cloud collected at the current moment, representing objects whose positions have not changed significantly. The extracted static point clouds are those whose expected point cloud structure is consistent with the point cloud structure of the point cloud map. In practice, this extraction requires the use of spatial description information of the working environment.

[0052] The spatial description information described above is used to describe the location of the static space in the robot's working environment. It can take many forms, which will be introduced below with examples.

[0053] In one implementation, the spatial description information can be a horizontal projection map of the working environment, which includes a defined static area, namely the horizontal projection area of ​​the static space.

[0054] As can be seen, a horizontal projection map is a projection of the work environment onto a horizontal plane, which includes the outlines of objects in the work environment.

[0055] Based on the actual scenario, staff can estimate the space in the work environment where objects are unlikely to move and set the horizontal projection area of ​​such space in the horizontal projection map as a static area. For example, when the work environment is an exhibition, the area inside the warning line in the horizontal projection map can be set as a static area; when the work environment is a factory, the area outside the material storage area in the horizontal projection map can be set as a static area.

[0056] In another implementation, the spatial description information can be a three-dimensional heatmap. This heatmap is constructed based on real point cloud data collected by the robot, where each element corresponds one-to-one with a pre-defined voxel in the working environment. The value of each element represents the probability that the corresponding voxel belongs to the static space. These voxels are multiple spatial regions of uniform size obtained by spatially dividing the working environment. Thus, voxels corresponding to elements in the three-dimensional heatmap with a probability greater than a threshold can be considered as static spaces.

[0057] The specific construction methods of the above-mentioned horizontal projection map and three-dimensional heat map will be detailed later, and will not be described in detail here.

[0058] Of course, depending on the spatial description information, the methods for extracting the first static point cloud from the environmental point cloud will also differ, and will be introduced separately below.

[0059] In one implementation, points whose horizontal coordinates lie in a static region of a horizontally projected map can be directly extracted from the environmental point cloud to obtain the first static point cloud. It is evident that, by utilizing the static region in the horizontal projection, the first static point cloud can be quickly and efficiently determined from the environmental point cloud based solely on the two-dimensional coordinates of points within the environmental point cloud.

[0060] In one possible implementation, in order to reduce the amount of data processing required to extract static point clouds, the number of intersections between the ray corresponding to each point in the environmental point cloud and the static region can be determined, and points in the environmental point cloud with an odd number of corresponding intersections can be identified as points whose horizontal coordinates are located in the static region.

[0061] In this system, the ray corresponding to each point extends from the horizontal coordinate of that point as its origin in a first direction, which is either horizontal to the right or horizontal to the left. The following will illustrate this... Figure 2 The examples shown provide a clear illustration.

[0062] See Figure 2 The irregular closed shapes represent static areas in the horizontally projected map, and A, B, C, and D represent the projection points of points in the environmental point cloud onto the horizontal plane. It can be seen that the ray extending to the right from A has one intersection point with the projected area: a1; the ray extending to the right from B has two intersection points: b1 and b2; the ray extending to the right from C has three intersection points: c1, c2, and c3; and the ray extending to the right from D has two intersection points: d1 and d2. The number of intersection points for A and C is odd, therefore A and C are points whose horizontal coordinates are located within the static area; the number of intersection points for B and D is even, therefore B and D are points whose horizontal coordinates are not located within the static area.

[0063] In another implementation, points located at static voxels can be extracted from the environmental point cloud to obtain the first static point cloud. The aforementioned static voxels are the voxels corresponding to elements in the 3D heatmap that have a probability greater than a certain threshold.

[0064] The 3D heatmap is constructed based on real point cloud data collected by the robot. Each element's value represents the probability that the corresponding voxel belongs to the static space. Therefore, elements with a probability greater than a threshold have a higher confidence level in belonging to the static space. It is evident that the 3D heatmap can more accurately represent the static space in the working environment, and using it helps improve the accuracy of the extracted first static point cloud.

[0065] After this step is completed, it is equivalent to removing dynamic point clouds in the environment point cloud that are inconsistent with the point cloud structure of the point cloud map, and only retaining static point clouds whose point cloud structure is consistent with the point cloud map.

[0066] Of course, those skilled in the art will understand that when the coordinate systems used by the environmental point cloud and the point cloud map are inconsistent, the robot's pose can be estimated by first using data collected by sensors such as the inertial measurement unit (IMU), wheeled odometer, and positioning unit set up on the robot. Based on the obtained pose, the environmental point cloud can be transformed to the coordinate system used by the point cloud map, and then the first static point cloud mentioned above can be extracted from the environmental point cloud. This will not be elaborated further here.

[0067] Step S102: Perform point cloud registration between the first static point cloud and the point cloud map, and obtain the robot's pose based on the registration result.

[0068] Specifically, the above registration operation can be performed using algorithms such as Iterative Closest Point (ICP), Normal Distributions Transform (NDT), and Feature Sampling Consensus Registration to obtain the robot's pose and thus achieve robot localization. This embodiment of the invention does not limit the specific methods used.

[0069] For example, the robot's pose can be obtained by registration according to the following expression:

[0070] ;

[0071] in, The registered robot pose is essentially a pose transformation matrix that maximizes the matching degree between the first static point cloud and the points in the point cloud map; N represents the total number of points in the first static point cloud, M represents the total number of points in the point cloud map; T represents the transformation matrix, which represents rotation and translation in three-dimensional space. This represents the coordinates of the i-th point in the first static point cloud. This represents the coordinates of the j-th point in the point cloud map.

[0072] As can be seen from the above, when using the solution provided in this embodiment of the invention for robot localization, the robot pose is not obtained by directly configuring the environmental point cloud and point cloud map collected by the robot. Instead, it utilizes the spatial description information of the robot's working environment to remove dynamic point clouds from the environmental point cloud that are inconsistent with the point cloud structure of the point cloud map, thereby selecting static point clouds that are consistent with the point cloud structure of the point cloud map. Then, only static point clouds are used for point cloud registration with the point cloud map, eliminating the interference caused by dynamic point clouds to point cloud registration, thereby reducing registration errors, improving the accuracy of registration results, and thus improving the robot's localization accuracy and stability.

[0073] In highly dynamic scenarios where environmental changes are frequent and point cloud features are severely lacking, the solution provided by the embodiments of the present invention can still maintain stable and reliable pose estimation, ensure the positioning accuracy of the robot, fundamentally improve the positioning resilience and operational safety of the robot in highly dynamic scenarios, and provide a guarantee for the reliable deployment and application of the robot in highly dynamic environments.

[0074] The following is an example illustrating the construction method of the horizontal projection map mentioned above.

[0075] Specifically, a method can be used to directly project the points in the point cloud map onto a horizontal plane to generate a horizontally projected map.

[0076] To improve the focus on spatial information of interest within the working environment and enhance data processing efficiency, a possible implementation method for constructing a horizontally projected map is as follows:

[0077] The process involves extracting target points whose height values ​​fall within a set height range from a point cloud map; then, projecting these target points onto a horizontal plane to obtain a two-dimensional point cloud; finally, downsampling the two-dimensional point cloud and connecting the points in the downsampled two-dimensional point cloud to obtain a horizontally projected map. This method of generating a horizontally projected map can also be called an Open Street Map (OSM) map.

[0078] Among them, extracting target points whose height values ​​are within a set height range from the point cloud map can retain only the information of the space of interest in the working environment, reduce unnecessary data interference, and improve data processing efficiency; downsampling and then connecting points can further reduce the data to be processed while retaining the object outline information, thereby improving data processing efficiency.

[0079] The construction method of the three-dimensional heat map mentioned above will be illustrated by an example.

[0080] Specifically, a first static point cloud can be extracted based on a horizontal projection map. Then, effective static point clouds whose actual point cloud structure is consistent with the point cloud structure of the point cloud map can be extracted from the first static point cloud. The three-dimensional heat map can be initialized based on the effective static point cloud.

[0081] In other words, based on the horizontal projection map, the first static point cloud is extracted using the method described above. After each extraction of the static point cloud, the effective static point cloud is extracted from the first static point cloud, and the elements in the 3D heat map are updated according to the effective point cloud until the construction completion condition is met, and finally the heat map that has been constructed (i.e. initialized) is obtained.

[0082] Considering that initializing a 3D heatmap takes time, in one possible implementation, to avoid affecting the robot's task execution and to achieve rapid task deployment and response, while the robot is performing its task, a static point cloud can be extracted using a horizontal projection map to achieve robot localization, and a 3D heatmap can be constructed simultaneously using the aforementioned method. After the 3D heatmap is constructed, all subsequent extractions of static point clouds and robot localization will be performed using the 3D heatmap. The following section will refer to the appendix... Figure 3 Please provide a detailed explanation.

[0083] See Figure 3 The above is a flowchart illustrating the second robot positioning method provided in the embodiment of the present invention. The method includes the following steps S301 to S305.

[0084] Step S301: Based on the static area in the horizontal projection map, extract the static point cloud from the environmental point cloud, perform point cloud registration between the static point cloud and the point cloud map, and obtain the robot's pose based on the registration result.

[0085] In this step, the method described above is used to extract static point clouds using a horizontal projection map to achieve robot localization, which will not be detailed here.

[0086] Step S302: Based on the obtained pose, perform pose transformation on the static point cloud, and extract the effective static point cloud from the transformed static point cloud whose actual point cloud structure is consistent with the point cloud structure of the point cloud map.

[0087] The transformed static point cloud can be called the second static point cloud. Specifically, effective static point clouds can be extracted from the second static point cloud using methods such as point cloud feature matching.

[0088] To improve the accuracy of the extracted effective static point cloud, one possible implementation is to determine the minimum distance between each point in the second static point cloud and the points on the point cloud map; then, extract the points whose minimum distance is less than a distance threshold from the second static point cloud to obtain the effective static point cloud.

[0089] For each point in the second static point cloud, if its corresponding minimum distance is less than the distance threshold, it means that there is a point in the point cloud map that is close to that point, that is, there is a matching point in the point cloud map, indicating that the point is likely to be a point corresponding to a static object.

[0090] The above minimum distance It can be represented by the following expression:

[0091] ;

[0092] in, This represents the second static point cloud obtained after transformation.

[0093] After obtaining the minimum distance corresponding to each point in the second static point cloud, it can be expressed as follows: This point is determined to belong to a valid static point cloud. Still an invalid static point cloud (Point clouds other than the valid static point clouds in the first static point cloud):

[0094] .

[0095] in, This represents the aforementioned distance threshold. In other words, if the minimum distance corresponding to a point is less than or equal to the distance threshold, the point is considered a valid static point cloud; otherwise, the point is considered an invalid static point cloud.

[0096] Step S303: Initialize a three-dimensional heatmap based on the positions of points in the effective static point cloud.

[0097] As mentioned above, each element of the 3D heatmap corresponds one-to-one with each voxel pre-divided in the working environment. Each element is initially 0, and its value represents the probability that the voxel corresponding to that element belongs to the static space.

[0098] For each voxel, the more points it contains in the effective static point cloud, the greater the probability that the voxel belongs to the static space.

[0099] Therefore, to improve the rationality and accuracy of the determined probability accumulation value, one possible implementation is to determine the probability accumulation value of the element corresponding to each voxel in the 3D heatmap based on the number of points located in each voxel in the effective static point cloud. Then, the probability accumulation value corresponding to each element is added to the probability accumulation value of that element to obtain the updated element. The aforementioned quantity is positively correlated with the probability accumulation value.

[0100] Preferably, the cumulative probability value of a voxel can be determined using the following expression. :

[0101] ;

[0102] Where e is the natural base, and x represents the number of points in the effective static point cloud that are located at that voxel, that is, the number of times that voxel is hit by points in the effective static point cloud.

[0103] Step S304: Determine whether the updated 3D heat map meets the initialization completion condition. If not, return to step S301; if yes, proceed to step S305.

[0104] The initialization completion condition, also known as the construction completion condition mentioned above, can specifically be that the effective point cloud that has participated in the initialization of the heatmap is greater than a set number of frames, or the initialization duration is greater than a set duration, etc. This embodiment of the invention does not limit this.

[0105] To ensure the heatmap is fully initialized and improve the accuracy of the static regions described by the constructed heatmap, one possible implementation could be that the completion condition is: the total number of voxels with a hit count greater than a first number is greater than a second number. Here, the hit count is the cumulative number of points confirmed to be located within a voxel.

[0106] The first and second quantities mentioned above can be set by staff based on actual scenario needs and / or experience. For example, the first quantity can be 100 and the second quantity can be 5000.

[0107] After this step is completed, if the updated 3D heat map does not meet the initialization completion conditions, return to step S301 to continue initializing the heat map; if the updated 3D heat map has met the initialization completion conditions, then proceed to step S305 to achieve robot localization using a more accurate 3D heat map.

[0108] Step S305: Based on the 3D heat map, extract the static point cloud from the environmental point cloud, perform point cloud registration between the static point cloud and the point cloud map, and obtain the robot's pose based on the registration result.

[0109] In this step, the method described above is used to extract static point clouds using a 3D heatmap and achieve robot localization, which will not be detailed here.

[0110] In this way, on the one hand, before the heat map is completed, the robot can be located using a horizontal projection map without affecting the robot's task execution, enabling rapid task deployment and response; on the other hand, the heat map is built through long-term data accumulation, and after the heat map is completed, the robot can be located using a more accurate 3D heat map, achieving higher precision robot positioning.

[0111] When the robot's position changes little or remains unchanged, if the heatmap is initialized based on the static point cloud collected by the robot in each frame, the initialized heatmap can only reflect the probability that a small area of ​​space in the working environment belongs to the static space, and cannot reflect the complete information of the action environment.

[0112] In view of this, in one possible implementation, for each frame of the currently determined static point cloud, if the currently determined static point cloud is not the first point cloud to participate in the 3D heatmap update, it is determined whether the difference representation value between the static point cloud and the previous point cloud that participated in the 3D heatmap update is greater than or equal to the difference threshold. If yes, step S302 is executed to initialize the heatmap with the static point cloud; otherwise, step S302 is not executed.

[0113] The aforementioned difference characterization values ​​may include differences in acquisition time and / or acquisition location between point clouds.

[0114] When the difference characterization value includes both of the above two difference values, the difference threshold can include sub-thresholds corresponding to the difference value at the time of acquisition and the difference value at the location of acquisition, respectively. In this way, if both the difference value at the time of acquisition and the difference value at the location of acquisition are greater than their corresponding sub-thresholds, it is determined that the static point cloud is used to initialize the heat map.

[0115] Next, we will proceed through... Figure 4 ,for Figure 3 The robot localization process shown will be explained more intuitively.

[0116] like Figure 4 As shown, the robot first constructs a map using the fused scanned environmental point cloud, obtaining a point cloud map. Then, it projects this point cloud map to obtain a projected map. During task execution, the robot first uses the projected map to segment the point cloud (i.e., extracting static point clouds from the environmental point cloud). The segmented static point cloud is then registered with the point cloud map to obtain the robot's real-time pose. Simultaneously, a point cloud heatmap (i.e., a 3D heatmap) is constructed based on the segmented static point cloud and the point cloud registration results. After the point cloud heatmap is constructed, it is used instead of the projected map for subsequent point cloud segmentation. That is, all subsequent point cloud segmentation is performed using the point cloud heatmap, and the segmented static point cloud is registered with the point cloud map to obtain the robot's real-time pose.

[0117] In this way, heatmaps can be initialized based on static point clouds in different spaces within the corresponding working environment, so that the initialized heatmaps can more comprehensively and accurately reflect the probability that each voxel in the working environment belongs to the static space.

[0118] Corresponding to the robot positioning method described above, this embodiment of the invention also provides a robot positioning device.

[0119] See Figure 5 This is a schematic diagram of a robot positioning device provided in an embodiment of the present invention. The device includes:

[0120] The static point cloud extraction module 501 is used to extract the first static point cloud located in the static space of the working environment from the environmental point cloud based on the spatial description information of the working environment where the robot is located. The environmental point cloud is the point cloud collected by the point cloud acquisition device set on the robot. The expected point cloud structure of the static space is consistent with the point cloud structure of the point cloud map of the working environment.

[0121] The positioning module 502 is used to perform point cloud registration between the first static point cloud and the point cloud map, and obtain the robot's pose based on the registration result.

[0122] As can be seen from the above, when using the solution provided in this embodiment of the invention for robot localization, the robot pose is not obtained by directly configuring the environmental point cloud and point cloud map collected by the robot. Instead, it utilizes the spatial description information of the robot's working environment to remove dynamic point clouds from the environmental point cloud that are inconsistent with the point cloud structure of the point cloud map, thereby selecting static point clouds that are consistent with the point cloud structure of the point cloud map. Then, only static point clouds are used for point cloud registration with the point cloud map, eliminating the interference caused by dynamic point clouds to point cloud registration, thereby reducing registration errors, improving the accuracy and stability of registration results, and thus improving the robot's localization accuracy.

[0123] In highly dynamic scenarios where environmental changes are frequent and point cloud features are severely lacking, the solution provided by the embodiments of the present invention can still maintain stable and reliable pose estimation, ensure the positioning accuracy of the robot, fundamentally improve the positioning resilience and operational safety of the robot in highly dynamic scenarios, and provide a guarantee for the reliable deployment and application of the robot in highly dynamic environments.

[0124] In one possible implementation,

[0125] The spatial description information includes: a horizontal projection map of the working environment, which includes a defined static area. The static area includes: a horizontal projection area of ​​the static space, and a static point cloud extraction module, which is specifically used to extract points with horizontal coordinates located in the static area from the environmental point cloud to obtain the first static point cloud.

[0126] By utilizing the static region in the horizontal projection, the first static point cloud can be quickly and efficiently determined from the environmental point cloud based solely on the two-dimensional coordinates of points in the environmental point cloud.

[0127] In one possible implementation,

[0128] The spatial description information includes: a three-dimensional heat map, in which each element corresponds one-to-one with each voxel in the working environment, and the value of each element represents the probability that the voxel corresponding to that element belongs to the static space; and a static point cloud extraction module, which is specifically used to extract points located in static voxels from the environmental point cloud to obtain the first static point cloud, wherein the static voxel is the voxel corresponding to the element in the three-dimensional heat map that is greater than the probability threshold.

[0129] The 3D heatmap is constructed based on real point cloud data collected by the robot. Each element's value represents the probability that the corresponding voxel belongs to the static space. Therefore, elements with a probability greater than a threshold have a higher confidence level in belonging to the static space. It is evident that the 3D heatmap can more accurately represent the static space in the working environment, and using it helps improve the accuracy of the extracted first static point cloud.

[0130] In one possible implementation, the spatial description information is initially a horizontally projected map, and the device further includes:

[0131] The effective point cloud extraction module is used to extract effective static point clouds from the first static point cloud whose actual point cloud structure is consistent with the point cloud structure of the point cloud map.

[0132] The element update module is used to update each element in the 3D heatmap based on the position of points in the effective static point cloud;

[0133] The information replacement module is used to replace the spatial description information with the updated 3D heatmap in response to the updated 3D heatmap meeting the construction completion conditions.

[0134] In this way, on the one hand, before the heat map is completed, the robot can be located using a horizontal projection map without affecting the robot's task execution, enabling rapid task deployment and response; on the other hand, the heat map is built through long-term data accumulation, and after the heat map is completed, the robot can be located using a more accurate 3D heat map, achieving higher precision robot positioning.

[0135] In one possible implementation,

[0136] The effective point cloud extraction module is specifically used to perform pose transformation on a first static point cloud based on pose to obtain a second static point cloud; determine the minimum distance between each point in the second static point cloud and a point on the point cloud map; and extract points from the second static point cloud whose minimum distance is less than a distance threshold to obtain the effective static point cloud. This can improve the accuracy of the extracted effective static point cloud.

[0137] In one possible implementation,

[0138] The element update module is specifically used to determine the cumulative probability value of the element corresponding to each voxel in the 3D heatmap based on the number of points located at each voxel in the effective static point cloud. The number of points is positively correlated with the cumulative probability value. The updated element is obtained by summing the cumulative probability value of each element. This improves the rationality and accuracy of the determined cumulative probability values.

[0139] In one possible implementation,

[0140] The conditions for successful construction include: the total number of voxels that have been hit more times than the first number is greater than the second number.

[0141] The number of hits is the cumulative number of points that have been identified as being located within a voxel.

[0142] This allows the heatmap to be fully initialized and improves the accuracy of the static regions described by the constructed heatmap.

[0143] In one possible implementation, the device further includes:

[0144] The difference determination module is used to determine the difference characterization value between the first static point cloud and the previous point cloud that participated in the 3D heatmap update if the first static point cloud is not the first point cloud to participate in the 3D heatmap update. The difference characterization value includes: the difference value at the time of acquisition and / or the difference value at the acquisition location.

[0145] The trigger module is used to trigger the effective point cloud extraction module in response to a difference characterization value being greater than or equal to a difference threshold.

[0146] In this way, heatmaps can be initialized based on static point clouds in different spaces within the corresponding working environment, so that the initialized heatmaps can more comprehensively and accurately reflect the probability that each voxel in the working environment belongs to the static space.

[0147] In one possible implementation, when the spatial description information includes a horizontally projected map, the following module is used to determine points in the environmental point cloud whose horizontal coordinates lie in a static region:

[0148] The intersection point number determination module is used to determine the number of intersection points between the ray corresponding to each point in the environmental point cloud and the static region. The ray corresponding to each point takes the horizontal coordinate of the point as the origin and extends in a first direction, which is either a horizontal rightward direction or a horizontal leftward direction.

[0149] The determination module identifies points in the environmental point cloud whose number of intersections is odd as points located in the static region on the horizontal coordinate. This reduces the amount of data processing required when extracting static point clouds.

[0150] In one possible implementation, the following modules are used to construct a horizontally projected map:

[0151] The target point extraction module is used to extract target points whose height values ​​are within a set height range from the point cloud map.

[0152] The projection module is used to project target points onto a horizontal plane to obtain a two-dimensional point cloud.

[0153] The generation module is used to downsample the 2D point cloud and connect the points in the downsampled 2D point cloud to obtain a horizontally projected map.

[0154] Extracting target points whose height values ​​are within a set height range from a point cloud map allows us to retain only the information of the space of interest in the working environment, reducing unnecessary data interference and improving data processing efficiency. Downsampling and then connecting the points can further reduce the amount of data to be processed while retaining the object's outline information, thus improving data processing efficiency.

[0155] Corresponding to the robot positioning method described above, embodiments of the present invention also provide an electronic device, a storage medium, and a computer program product.

[0156] This invention also provides an electronic device, such as... Figure 6 As shown, it includes a processor 601, a communication interface 602, a memory 603, and a communication bus 604, wherein the processor 601, the communication interface 602, and the memory 603 communicate with each other through the communication bus 604.

[0157] Memory 603 is used to store computer programs;

[0158] The processor 601 is used to execute the program stored in the memory 603 to implement any of the aforementioned robot positioning methods.

[0159] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0160] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0161] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0162] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0163] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the robot localization methods described above.

[0164] The present invention also provides a computer program product containing instructions that, when run on a computer, cause the computer to perform any of the robot positioning methods described above.

[0165] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0166] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0167] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, electronic devices, and storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0168] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A robot localization method, characterized in that, The method includes: Based on the spatial description information of the robot's working environment, a first static point cloud located in the static space of the working environment is extracted from the environmental point cloud. The environmental point cloud is a point cloud collected by a point cloud acquisition device installed on the robot. The expected point cloud structure of the static space is consistent with the point cloud structure of the point cloud map of the working environment. The first static point cloud and the point cloud map are registered together, and the pose of the robot is obtained based on the registration result.

2. The method according to claim 1, characterized in that, The spatial description information includes: a horizontal projection map of the working environment, the horizontal projection map including a defined static region, the static region including: a horizontal projection region of the static space, and the step of extracting a first static point cloud located in the static space of the working environment from the environmental point cloud including: Points with horizontal coordinates located in the static region are extracted from the environmental point cloud to obtain the first static point cloud; or, The spatial description information includes: a three-dimensional heatmap, where each element in the three-dimensional heatmap corresponds one-to-one with each voxel in the working environment, and the value of each element represents the probability that the voxel corresponding to that element belongs to the static space; the extraction of the first static point cloud located in the static space of the working environment from the environmental point cloud includes: Points located in static voxels are extracted from the environmental point cloud to obtain the first static point cloud, wherein the static voxels are the voxels corresponding to elements in the three-dimensional heat map that are greater than a probability threshold.

3. The method according to claim 2, characterized in that, The spatial description information is initially the horizontal projection map, and the method further includes: Extract effective static point clouds from the first static point cloud whose actual point cloud structure is consistent with the point cloud structure of the point cloud map. Based on the positions of the points in the effective static point cloud, update each element in the three-dimensional heat map; In response to the updated 3D heatmap meeting the construction completion conditions, the spatial description information is replaced with the updated 3D heatmap.

4. The method according to claim 3, characterized in that, The extraction of effective static point clouds from the first static point cloud, whose actual point cloud structure is consistent with the point cloud structure of the working environment, includes: Based on the pose, the first static point cloud is transformed to obtain a second static point cloud; the minimum distance between each point in the second static point cloud and the points in the point cloud map is determined; points whose minimum distance is less than a distance threshold are extracted from the second static point cloud to obtain a valid static point cloud. And / or, The step of updating elements in the 3D heatmap based on the positions of points in the effective static point cloud includes: Based on the number of points located at each voxel in the effective static point cloud, the cumulative probability value of the element corresponding to each voxel in the three-dimensional heat map is determined, wherein the number is positively correlated with the cumulative probability value; the cumulative probability value corresponding to each element is added to the cumulative probability value of each element to obtain the updated element; The conditions for completing the construction include: the total number of voxels with a hit count greater than the first number is greater than the second number, wherein the hit count is the cumulative number of points that have been determined to be located within a voxel.

5. The method according to claim 3 or 4, characterized in that, If the first static point cloud is not the first point cloud to participate in the 3D heatmap update, before extracting the effective static point cloud whose actual point cloud structure is consistent with the point cloud structure of the working environment from the first static point cloud, the process further includes: Determine the difference characterization value between the first static point cloud and the previous point cloud that participated in the 3D heatmap update. The difference characterization value includes: the difference value at the time of acquisition and / or the difference value at the location of acquisition. In response to the difference characterization value being greater than or equal to the difference threshold, the step of extracting an effective static point cloud from the first static point cloud whose actual point cloud structure tends to be consistent with the point cloud structure of the working environment is performed.

6. The method according to any one of claims 2 to 5, characterized in that, When the spatial description information includes the horizontal projection map, the points in the environmental point cloud whose horizontal coordinates are located in the static area are determined as follows: Determine the number of intersections between the ray corresponding to each point in the environmental point cloud and the static region, wherein the ray corresponding to each point takes the horizontal coordinate of that point as the origin and extends in a first direction, the first direction being either a horizontal rightward direction or a horizontal leftward direction. Points in the environmental point cloud with an odd number of intersections are defined as points whose horizontal coordinates are located in the static region. And / or, The horizontal projection map is constructed in the following manner: Extract target points whose height values ​​are within a set height range from the point cloud map; project the target points onto a horizontal plane to obtain a two-dimensional point cloud; downsample the two-dimensional point cloud and connect the points in the downsampled two-dimensional point cloud to obtain the horizontal projection map.

7. A robot positioning device, characterized in that, The device includes: The static point cloud extraction module is used to extract a first static point cloud located in the static space of the working environment from the environmental point cloud based on the spatial description information of the working environment in which the robot is located. The environmental point cloud is a point cloud collected by a point cloud acquisition device set on the robot, and the expected point cloud structure of the static space is consistent with the point cloud structure of the point cloud map of the working environment. The positioning module is used to perform point cloud registration between the first static point cloud and the point cloud map, and obtain the pose of the robot based on the registration result.

8. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the steps of the method described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method described in any one of claims 1 to 6.

10. A computer program product containing instructions, characterized in that, When the instructions are executed on a computer, the computer causes the computer to perform the method of any one of claims 1 to 6.

Images