Multi-mobile-robot large-area environment map establishing method based on implicit neural points

Abstract

The invention discloses a multi-mobile robot large-area environment map establishing method based on implicit neural points, which is characterized in that 3D point clouds acquired by mobile robots are preprocessed, and neural point sets of local maps of the mobile robots and corresponding neural point features are constructed; secondly, a signed distance field of the local map is predicted based on the neural point features corresponding to the neural points, sub-maps of geometric representation displayed by the mobile robots are constructed through the generated signed distance field, then the sub-maps of the mobile robots are uploaded to a central processing unit to be associated, overlapping areas are screened, and the local map is obtained. The method comprises the following steps of: overlapping a plurality of sub-maps, constructing a local feature descriptor of each neural point in an overlapping region, matching and clustering the neural points of each sub-map by utilizing the constructed local feature descriptor to obtain a clustering point cluster matched and corresponding to each sub-map, and finally fusing each local map according to the clustering point cluster matched and corresponding to each sub-map to obtain a fused local map. And a complete large-area environment map is established.

Description

Technical Field

[0001] The present invention relates to the field of positioning and mapping, and in particular to a method for establishing a large-area environment map of multiple mobile robots based on implicit neural points. Background Art

[0002] As an important component of intelligent mobile robots, large-scale high-precision maps can provide accurate road information for mobile robots to perform autonomous positioning, path planning and other downstream tasks. Although existing mapping and positioning technologies can meet the needs of stable operation of robots in small environments, they still face challenges such as large positioning errors, slow mapping speed and large map memory usage when mapping and positioning in large unknown environments.

[0003] As implicit neural representations have shown encouraging results in various fields, they have also made good progress in the field of simultaneous localization and mapping (SLAM). The simultaneous localization and mapping technology that combines implicit representations such as signed distance functions, occupancy fields, and neural radiation fields provides many solutions to large-scale scene localization and mapping problems.

[0004] SLAM based on implicit neural representation aims to implicitly understand irregular environmental information in a data-driven way in map construction, and use the large-scale parameters of deep learning networks to predict specific environmental information from a given perspective or point cloud data. At the same time, SLAM based on implicit neural representation can optimize camera poses and implicit scene network parameters through inverse rendering in tracking, or combine visual odometry (VO) and implicit neural map construction to achieve mobile robot positioning and map construction.

[0005] The SHINE-Mapping algorithm uses a learnable octree-based hierarchical feature grid and a globally shared MLP decoder to represent the continuous signed distance field (SDF) of the environment, thereby generating an implicit representation of the map. The local feature vector is optimized online, and the signed distance value of the MLP output is supervised by using direct measurements to capture the local geometry. Based on this implicit map representation, an explicit geometric representation of the map is generated in the form of a triangular mesh through the Marching cubes algorithm. However, the SDF construction method based on the octree hierarchical feature grid stores the original lidar 3D point cloud features in the octree node corners. Although it reduces the number of points to be processed, it will cause large map errors, and this implicit function is not easy to analyze and operate. Summary of the invention

[0006] A brief summary of one or more aspects is given below to provide a basic understanding of these aspects. This summary is not an exhaustive overview of all conceived aspects, and is neither intended to identify the key or critical elements of all aspects nor to define the scope of any or all aspects. Its only purpose is to give some concepts of one or more aspects in a simplified form as a prelude to a more detailed description that will be given later.

[0007] The purpose of the present invention is to solve the above problems and provide a method for establishing a large-area environment map of multiple mobile robots based on implicit neural points. Multiple mobile robots are used to construct implicit neural expressions of different local environments, and then the established Mesh sub-maps are uploaded to the central processing unit for association and fusion, so as to establish a complete large-area environment map.

[0008] The technical solution of the present invention is:

[0009] The present invention provides a method for establishing a large-area environment map of multiple mobile robots based on implicit neural points, comprising the following steps:

[0010] Step S1: Based on the built-in sensors of each mobile robot, 3D point cloud data of the surrounding environment is collected, and each frame of the collected 3D point cloud data is preprocessed to construct a neural point set of the local map of each mobile robot and the neural point features of each neural point;

[0011] Step S2: predicting the signed distance field of each mobile robot's local map based on the neural point features corresponding to each neural point, and generating an implicit neural expression of each mobile robot's local map;

[0012] Step S3: constructing a sub-map of the display geometry representation of each mobile robot based on the signed distance field of the local map of each mobile robot;

[0013] Step S4: uploading the sub-maps of each mobile robot to the central processor for association and screening the overlapping areas;

[0014] Step S5: constructing a local feature descriptor for each neural point in the overlapping area, and using the constructed local feature descriptor to match the neural points of each sub-map to obtain a neural point matching pair;

[0015] Step S6: clustering the neural point matching pairs to obtain the clustering point clusters corresponding to each sub-map matching;

[0016] Step S7: The local maps are merged according to the corresponding clustering point clusters matched by each sub-map to establish a complete large-area environment map.

[0017] According to an embodiment of the method for establishing a large-area environmental map of multiple mobile robots based on implicit neural points of the present invention, when the method for establishing a large-area environmental map of multiple mobile robots based on implicit neural points preprocesses the collected 3D point cloud data, a voxel grid is established for the collected 3D point cloud data, and then the 3D point cloud data is down-sampled using the constructed voxel grid to obtain a set of neural points corresponding to the mobile robot; wherein, after the method for establishing a large-area environmental map of multiple mobile robots based on implicit neural points completes the down-sampling of the 3D point cloud data, the corresponding neural point features are obtained based on the voxel grid where each neural point is located, and the neural point features based on the voxel grid are stored in a hash table for subsequent positioning and mapping queries.

[0018] According to an embodiment of the method for establishing a large-area environmental map of multiple mobile robots based on implicit neural points of the present invention, the neural point features include the three-dimensional coordinates of the neural points in the world coordinate system, quaternions, and geometric feature vectors representing the local geometric features of the neural points; wherein, after the method for establishing a large-area environmental map of multiple mobile robots based on implicit neural points completes the downsampling of 3D point cloud data, an interpolation method is used to interpolate between each neural point, and then the signed distance field of each interpolation position is predicted by combining the neural point features of multiple neighboring neural points in the field of each interpolation position and the relative coordinate difference from the interpolation position, thereby generating an implicit neural expression of the local map of each mobile robot.

[0019] According to an embodiment of the method for establishing a large-area environmental map of multiple mobile robots based on implicit neural points of the present invention, the method for establishing a large-area environmental map of multiple mobile robots based on implicit neural points adopts a shallow neural network to predict the signed distance field of each interpolation position; wherein, after the method for establishing a large-area environmental map of multiple mobile robots based on implicit neural points obtains the neural point features of each adjacent neural point and the relative coordinate difference from the interpolation position according to a hash table, the neural point features of each adjacent neural point and the corresponding relative coordinate difference are cascaded, and then the cascaded three-dimensional coordinates, quaternions and geometric feature vectors are input into the shallow neural network for decoding, thereby outputting a signed distance field for representing the local geometric features of the three-dimensional space.

[0020] According to an embodiment of the method for establishing a large-area environment map of multiple mobile robots based on implicit neural points of the present invention, after the method for establishing a large-area environment map of multiple mobile robots based on implicit neural points generates the implicit neural expression of the local map of each mobile robot, the least square method based on the multi-weighted transformation matrix is ​​used to optimize the signed distance field prediction value error of any neural point in the current frame, thereby optimizing the position and posture of the sensor, and updating the local map of the implicit neural expression and the corresponding neural point features based on the signed distance field of the optimized neural point; wherein, the method for establishing a large-area environment map of multiple mobile robots based on implicit neural points constructs the optimal transformation matrix through the following formula:

[0021]

[0022] Among them, T * represents the optimal transformation matrix,

[0023] p r Represents the current frame neural point set collected by the sensor,

[0024] P represents any neural point in the neural point set of the current frame,

[0025] S(T p ) represents the signed distance field value of the neural point p under the multi-weighted transformation matrix T.

[0026] According to an embodiment of the method for establishing a large-area environmental map of multiple mobile robots based on implicit neural points of the present invention, after the method for establishing a large-area environmental map of multiple mobile robots based on implicit neural points predicts the signed distance field of the local map of each mobile robot, the voxel-level reconstruction algorithm MarchingCubes is used to construct a Mesh sub-map of the displayed geometric representation of each mobile robot; wherein, when the method for establishing a large-area environmental map of multiple mobile robots based on implicit neural points uses the Marching cubes algorithm to construct the Mesh sub-map of the mobile robot, the position correspondence between the vertices of each voxel in the local map based on implicit neural expression and the geometric obstacles is determined, and then the signed distance field of each neural point is converted into triangular mesh information, thereby generating a Mesh sub-map of the displayed geometric representation.

[0027] According to an embodiment of the method for establishing a large-area environment map for multiple mobile robots based on implicit neural points of the present invention, when constructing a Mesh sub-map of each mobile robot, the method for establishing a large-area environment map for multiple mobile robots based on implicit neural points determines the distance from each vertex to the surface of a geometric obstacle according to the signed distance field of each voxel vertex, and judges the position correspondence between each vertex and the geometric obstacle according to the distance from each vertex to the surface of the geometric obstacle, thereby establishing a Mesh sub-map showing geometric representation; wherein, the method for establishing a large-area environment map for multiple mobile robots based on implicit neural points judges the position correspondence between the vertex of any cell in the three-dimensional space and the geometric obstacle by the formula shown below:

[0028]

[0029] Where x represents the three-dimensional space R 3 The point in

[0030] SDF(x) represents the distance field of point x.

[0031] s represents the distance from point x to the surface of the geometric obstacle.

[0032] According to an embodiment of the method for establishing a large-area environment map for multiple mobile robots based on implicit neural points of the present invention, when constructing a sub-map of a geometric representation of a display of each mobile robot, the method for establishing a large-area environment map for multiple mobile robots based on implicit neural points performs loop detection based on a point cloud map of a historical frame, thereby optimizing the positioning error and the map; wherein, when performing loop detection, the method for establishing a large-area environment map for multiple mobile robots based on implicit neural points identifies the loop area by comparing the global descriptors of the point cloud maps of the current frame and the historical frame, and optimizes the positioning error and the map of the current frame through the identified loop area.

[0033] According to an embodiment of the method for establishing a large-area environmental map of multiple mobile robots based on implicit neural points of the present invention, after uploading the sub-map of each mobile robot to the central processor, the local coordinate system of the sub-map of each mobile robot is converted into a global coordinate system, thereby associating each sub-map to the same coordinate system and filtering the overlapping areas.

[0034] According to an embodiment of the method for establishing a large-area environment map of multiple mobile robots based on implicit neural points of the present invention, when screening the overlapping areas of each sub-map, the method for establishing a large-area environment map of multiple mobile robots based on implicit neural points constructs a corresponding octree for each sub-map located in the same coordinate system, and then judges in turn based on the constructed octree grid whether each neural point in each sub-map is located in the same voxel grid, thereby screening out the neural points located in the same voxel grid as the overlapping areas.

[0035] According to an embodiment of the method for establishing a large-area environment map of multiple mobile robots based on implicit neural points of the present invention, after the method for establishing a large-area environment map of multiple mobile robots based on implicit neural points filters out the overlapping areas of each sub-map, the local feature descriptor corresponding to each neural point component in the overlapping area is saved, and the index number of each neural point of each sub-map in the overlapping area and the corresponding local feature descriptor are saved in the corresponding hash table.

[0036] According to an embodiment of the method for establishing a large-area environment map for multiple mobile robots based on implicit neural points of the present invention, after the method for establishing a large-area environment map for multiple mobile robots based on implicit neural points constructs a local feature descriptor for each neural point in the overlapping area, a KD-Tree is constructed for the constructed local feature descriptor to search for similar neural point matching pairs in each Mesh submap, thereby matching the neural points in each Mesh submap; wherein, when the method for establishing a large-area environment map for multiple mobile robots based on implicit neural points searches for similar neural point matching pairs in each Mesh submap, the eigenvalue similarity and relative position distance between the local feature vectors of each neural point are calculated by searching a hash table, and then the similarity between each neural point is judged based on a preset eigenvalue similarity threshold and a relative position distance threshold, thereby screening out similar neural point matching pairs.

[0037] According to an embodiment of the method for establishing a large-area environment map of multiple mobile robots based on implicit neural points of the present invention, after completing the neural point matching of each sub-map, the method for establishing a large-area environment map of multiple mobile robots based on implicit neural points uses a density-based clustering algorithm DBSCAN to cluster the neural point matching pairs, thereby obtaining the cluster point clusters corresponding to the matching of each sub-map; wherein, after obtaining the cluster point clusters corresponding to the matching of each sub-map, the method for establishing a large-area environment map of multiple mobile robots based on implicit neural points constructs a transformation matrix by calculating the offset of the center of the outer bounding box of each cluster point cluster, and then performs a preliminary transformation on each sub-map through the offset transformation matrix, thereby fusing each local map.

[0038] According to an embodiment of the method for establishing a large-area environment map of multiple mobile robots based on implicit neural points of the present invention, the method for establishing a large-area environment map of multiple mobile robots based on implicit neural points calculates the offset of the center of the outer bounding box of each cluster point cluster by the following formula:

[0039]

[0040] in, Respectively represent the center of the outer bounding box of Mesh submap A and B, R 3 Represents three-dimensional space.

[0041] According to an embodiment of the method for establishing a large-area environment map for multiple mobile robots based on implicit neural points of the present invention, the method for establishing a large-area environment map for multiple mobile robots based on implicit neural points performs an initial transformation on each sub-map through an offset transformation matrix, and then uses an improved Fast-GICP registration algorithm to perform secondary matching on each transformed sub-map, and finally applies the two-stage point cloud transformation to the corresponding Mesh sub-map, thereby obtaining each Mesh sub-map after fine fusion; wherein, when the method for establishing a large-area environment map for multiple mobile robots based on implicit neural points performs secondary matching on the sub-map, KD-tree is first used to find the nearest neighbor point cloud pairs between different sub-maps, and then the distance error between the point cloud pairs is minimized through continuous optimization, thereby achieving secondary matching.

[0042] According to an embodiment of the method for establishing a large-area environment map for multiple mobile robots based on implicit neural points of the present invention, the method for establishing a large-area environment map for multiple mobile robots based on implicit neural points respectively sets a local environment positioning and mapping module based on implicit neural points in the local server of each mobile robot, so that each mobile robot can perform positioning and mapping based on implicit neural points in its own coordinate system; wherein, after each mobile robot executes steps S1 to S3 through the local environment positioning and mapping module based on implicit neural points of the local server to construct a sub-map in its own coordinate system, the constructed sub-map is uploaded to the central processing unit for fusion, thereby establishing a large-area environment map.

[0043] The present invention also provides a computer readable medium storing a computer program code, wherein the computer program code implements the method as described above when executed by a processor.

[0044] The present invention also provides a map fusion and positioning device for multiple mobile robots based on implicit neural points, comprising:

[0045] a memory for storing instructions executable by a processor; and

[0046] A processor is used to execute the instructions to implement the method as described above.

[0047] Compared with the prior art, the present invention has the following beneficial effects: for the positioning and mapping of a large-scale unknown environment, the present invention deploys a local environment positioning and mapping module in the local servers of multiple mobile robots respectively, so that each mobile robot can be positioned and mapped in its own local coordinate system, and the constructed sub-map is uploaded to the central processor for collaborative fusion, thereby establishing a large-area environmental map. Compared with the prior art, the present invention changes the SDF construction method based on the octree hierarchical feature grid in the SHINE-Mapping algorithm into an SDF construction method based on down-sampled grid neural points, which increases the map flexibility and operability on the basis of ensuring the reduction of the map memory size, making the constructed map transformation more flexible and more convenient for subsequent map fusion. At the same time, when fusing multiple sub-maps, the present invention converts the local coordinate system of each sub-map into a global coordinate system, and constructs the local shape descriptor of each sub-map neural point in the overlapping area to match each sub-map, thereby improving the sub-map matching efficiency and avoiding excessive reliance on good initial transformations. In addition, based on the initial transformation, the present invention adopts the Fast-GICP algorithm combined with the Huber weight function and the M estimate to perform fine fusion of the map. This secondary fusion method can avoid the problem of over-reliance on initial changes in traditional map matching algorithms, improve the sub-map fusion accuracy, and achieve fine fusion even when the sub-map shapes are not completely consistent. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The above features and advantages of the present invention can be better understood after reading the detailed description of the embodiments of the present disclosure in conjunction with the following drawings. In the drawings, the components are not necessarily drawn to scale, and components with similar related properties or features may have the same or similar reference numerals.

[0049] Figure 1 It is a flow chart showing an embodiment of a method for establishing a large-area environment map of multiple mobile robots based on implicit neural points of the present invention.

[0050] Figure 2 It is a schematic diagram showing an embodiment of a distributed architecture for establishing a large-area environment map by multiple mobile robots of the present invention.

[0051] Figure 3 It is a data flow diagram showing an embodiment of a local environment positioning and mapping module based on implicit neural points of the present invention.

[0052] Figure 4 A single robot structure of the present invention is shown in FIG. Figure 1 Schematic diagram for comparing the effects of the embodiments.

[0053] Figure 5 This is a diagram showing the collaborative fusion of multiple mobile robots of the present invention. Figure 1Schematic diagram for comparing the effects of the embodiments. DETAILED DESCRIPTION

[0054] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for the description of the embodiments. Obviously, the drawings described below are only some examples or embodiments of the present application. For ordinary technicians in this field, the present application can also be applied to other similar scenarios based on these drawings without creative work. Unless it is obvious from the language environment or otherwise explained, the same reference numerals in the figures represent the same structure or operation.

[0055] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0056] Unless otherwise specifically stated, the relative arrangement, numerical expressions and numerical values ​​of the parts and steps set forth in these embodiments do not limit the scope of the present application. At the same time, it should be understood that, for ease of description, the sizes of the various parts shown in the accompanying drawings are not drawn according to the actual proportional relationship. The technology, method and equipment known to those of ordinary skill in the relevant field may not be discussed in detail, but in appropriate cases, the technology, method and equipment should be considered as a part of the authorization specification. In all examples shown and discussed here, any specific value should be interpreted as being merely exemplary, rather than as a limitation. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so that once a certain item is defined in an accompanying drawing, it does not need to be further discussed in subsequent drawings.

[0057] When describing the embodiments of the present invention in detail, for the sake of convenience, the cross-sectional views showing the device structure will not be partially enlarged according to the general scale, and the schematic views are only examples, which should not limit the scope of protection of the present invention. In addition, in actual production, the three-dimensional dimensions of length, width and depth should be included.

[0058] In the description of the present application, it should be understood that the directions or positional relationships indicated by directional words such as "front, back, up, down, left, right", "lateral, vertical, perpendicular, horizontal" and "top, bottom" are usually based on the directions or positional relationships shown in the drawings, and are only for the convenience of describing the present application and simplifying the description. Unless otherwise specified, these directional words do not indicate or imply that the device or element referred to must have a specific direction or be constructed and operated in a specific direction, and therefore cannot be understood as limiting the scope of protection of the present application; the directional words "inside and outside" refer to the inside and outside relative to the contours of each component itself.

[0059] Disclosed herein is an embodiment of a method for establishing a large-area environment map of multiple mobile robots based on implicit neural points. Figure 1 is a flow chart showing an embodiment of a method for establishing a large-area environment map of multiple mobile robots based on implicit neural points of the present invention. Figure 1 ,The following is a detailed description of the steps of the ,method for building a large area environment map for multiple mobile robots based on ,implicit neural points.

[0060] Step S1: Based on the built-in sensors of each mobile robot, 3D point cloud data of the surrounding environment is collected, and each frame of the collected 3D point cloud data is preprocessed to construct a neural point set of the local map of each mobile robot and the neural point features of each neural point.

[0061] In this embodiment, in order to obtain the environmental information of a large area with wide scenes and unknown changes, a local environment positioning and mapping module based on implicit neural points is set in the local servers of multiple mobile robots to form the following Figure 2 The distributed architecture of the large-area environment map of multiple mobile robots shown in the figure enables each mobile robot to locate and build a map based on implicit neural points in its own coordinate system, and upload the established sub-map to the central processor for fusion to establish a complete large-area environment map. Each mobile robot performs steps S1 to S3 on the collected 3D point cloud data through the local environment positioning and mapping module based on implicit neural points of the local server to define the neural point structure. Then, the implicit neural expression of the local map of the mobile robot is established by interpolating and predicting the signed distance field (hereinafter sometimes referred to as SDF), and a Mesh sub-map showing the geometric representation is constructed based on the SDF of the local map, so that each mobile robot can locate and build a map of the local environment locally, and upload the established sub-map to the central processor for fusion to establish a complete large-area environment map.

[0062] Among them, when collecting local environmental information, each mobile robot uses built-in sensors (such as lidar) to collect 3D point cloud data of the surrounding environment, and then pre-processes the collected 3D point cloud data to obtain a set of neural points of the local map and the corresponding neural point features. Figure 3 The present invention is a local environment positioning and mapping module based on implicit neural points. Figure 3 , this embodiment is described in detail.

[0063] like Figure 3 As shown, in this embodiment, after the mobile robot collects 3D point cloud data of the surrounding environment through the built-in sensor (such as Velodyne VLP-16 laser radar), each frame of the collected 3D point cloud data is transmitted to the local environment positioning and mapping module based on implicit neural points for positioning and mapping. The local environment positioning and mapping module based on implicit neural points establishes a corresponding voxel grid for downsampling for each frame of input 3D point cloud data, thereby obtaining the neural points of the voxelized local map. Among them, one cell corresponds to one neural point. If any cell has multiple points, the center point of the cell is used to represent the neural point of the cell, thereby voxelizing the collected 3D point cloud data to obtain a set of neural points of the local map, and obtains the neural point features of the neural points in each cell, and then stores them in a hash table for query and use in subsequent positioning and mapping.

[0064] The neural point features include the three-dimensional coordinates of the neural point in the world coordinate system, quaternions, and optimizable local feature vectors representing the local geometric features of the neural point. The optimizable local feature vectors are composed of the geometric features represented by the normal vector and curvature of the neural point, and the relationship features between the points. These neural point features are stored in a hash table to facilitate subsequent positioning and query operations in mapping.

[0065] Step S2: predicting the signed distance field of each mobile robot's local map based on the neural point features corresponding to each neural point, and generating an implicit neural expression of each mobile robot's local map.

[0066] In this embodiment, after completing the voxel gridding of the 3D point cloud data collected by each mobile robot through the above steps, the interpolation method is used to interpolate between each neural point, and then the signed distance field SDF of each interpolation position is obtained by combining the neural point features of multiple neighboring neural points in the field of each interpolation position and the relative coordinate difference from the interpolation position, thereby generating an implicit neural expression of the local map of each mobile robot.

[0067] Specifically, in this embodiment, when predicting the signed distance field SDF of any interpolation position, the relative coordinate differences of the six nearest neighboring neural points in the interpolation position range from the interpolation position are counted, and then the neural point features of the six neighboring neural points themselves and the corresponding relative coordinate differences are cascaded and input into a shallow neural network for decoding, and the signed distance field SDF of the interpolation position is output.

[0068] In one embodiment, a multi-layer perceptron MLP is used as a shallow neural network for predicting a signed distance field SDF. The MLP network consists of an 18-dimensional input layer, 2 hidden layers, 64 neurons, and a 1-dimensional output layer, and each layer is connected by a fully connected layer. The 18-dimensional input layer includes 3-dimensional position input neurons, 4-dimensional quaternion input neurons, 8-dimensional feature vector neurons, and 3-dimensional relative position coordinate difference input neurons. After the three-dimensional coordinates, quaternions, and geometric feature vectors of the neighboring neural points at any interpolation position are cascaded, they are input into the MLP through the 18-dimensional input layer for decoding, thereby obtaining the signed distance field SDF of the interpolation position. When performing the decoding operation, the MLP combines the binary cross entropy loss function and the Leaky ReLU excitation function, and weights the prediction to the final predicted SDF value according to the distance from the neighboring neural point to the difference position.

[0069] In addition, in this embodiment, after the implicit neural expression of the local map of each mobile robot is generated through the above steps, the least squares method based on the multi-weighted transformation matrix can be used to optimize the SDF prediction value error of any neural point in the current frame, so as to achieve accurate positioning of the sensor posture, and update the local map based on the implicit neural point and the corresponding neural point features. Among them, when optimizing the SDF error value, the Levenberg-Marquardt algorithm is used to solve the multi-weighted optimal transformation matrix, and the least squares error of the SDF prediction on the transformed neural point is minimized by finding the optimal transformation matrix, so that the neural point at the time step t fits the obstacle surface. Among them, the optimal transformation matrix construction formula is as follows:

[0070]

[0071] Among them, T * represents the optimal transformation matrix; p r represents the current frame point set collected by the sensor, P represents any neural point in the current frame neural point set, S(T p ) represents the signed distance field value SDF of the neural point p under the multi-weighted transformation matrix T.

[0072] Step S3: constructing a sub-map of each mobile robot based on the signed distance field in the local map of each mobile robot.

[0073] In this embodiment, after the signed distance field of each mobile robot's local map is predicted through the above steps, the voxel-level reconstruction algorithm MarchingCubes is used to construct a Mesh sub-map of each mobile robot's display geometry representation. When constructing the Mesh sub-map of the mobile robot, firstly, the position correspondence of the neural point of each cell in the local map based on implicit neural expression relative to the geometric obstacle is determined according to the voxel grid constructed above, and then the signed distance field of each neural point is converted into triangular mesh information, and then the Mesh sub-map of the display geometry representation is generated using the Marching cube algorithm.

[0074] Specifically, in this embodiment, when determining the position correspondence between each vertex of any voxel and a geometric obstacle, the distance from the vertex to the surface of the geometric obstacle is determined based on the SDF of the vertex, and then the position correspondence between the vertex and the geometric obstacle is determined based on the distance, and then the Mesh sub-map displayed in the form of a triangular mesh is generated using the Marching cube algorithm. Among them, SDF is the correspondence between the distance value between the neural point in the three-dimensional space and the surface of the geometric obstacle, and its mapping relationship and the position correspondence with the geometric obstacle are as follows:

[0075]

[0076] Where x represents the three-dimensional space R 3 For the points in , SDF(x) represents the distance field of point x, and s represents the distance from point x to the surface of the geometric obstacle. Therefore, the position correspondence between each point and the geometric obstacle can be determined through this formula, so that the Marching cube algorithm generates a Mesh sub-map that displays the geometric representation.

[0077] In addition, in this embodiment, when constructing the sub-maps of the geometric representation of each mobile robot display, the point cloud map of the historical frame can be used for loop detection to optimize the positioning error and the map. When performing loop detection, any single robot can identify the loop area by comparing the global descriptors of the point cloud map of the current frame and the historical frame, and then optimize the positioning error and the map of the current frame through the identified loop area.

[0078] Specifically, the neural points in each frame of the map are mapped to two-dimensional polar coordinates through bird's-eye view projection, and the annular resolution is set to 20 and the fan resolution is set to 60. Then, the average neural point features of each bin are counted as the global descriptor of the point cloud of the frame. The loop is detected by comparing the global descriptors of the current frame and the historical frame, and then the corresponding Mesh sub-map is generated. Figure 4 This is a diagram showing a single robot structure of the present invention. Figure 1 Schematic diagram of the effect comparison of the embodiment. Figure 4(1) A dense mesh map built for a single robot. Figure 4 (2) is the comparison error diagram between the robot trajectory diagram and the true value. Figure 4 As shown, a single robot in the present invention adopts an implicit neural point-based autonomous positioning and mapping method to be tested in an open source data set, and the test results show that it has high positioning accuracy and map integrity.

[0079] Step S4: Upload the sub-maps of each mobile robot to the central processor for association and filter the overlapping areas.

[0080] In this embodiment, after each mobile robot completes the construction of the sub-map displaying the geometric representation through the above steps, the established sub-map is uploaded to the central processor for fusion, thereby establishing a complete large-area environment map. Before the map fusion is performed, the local coordinate system of the sub-map of each mobile robot is first converted to the global coordinate system, and then the overlapping area is screened according to each sub-map associated with the same coordinate system.

[0081] Specifically, in this embodiment, when screening the overlapping areas of each submap, a corresponding octree is constructed for each submap located in the same coordinate system, and then based on the constructed octree grid, it is determined in turn whether each neural point in each submap is located in the same voxel grid, thereby screening the neural points located in the same voxel grid as the overlapping area. The maximum depth of the constructed octree is 4, and the expanded leaf node size is 0.1.

[0082] Step S5: construct a local feature descriptor for each neural point in the overlapping area, and use the constructed local feature descriptor to match the neural points of each sub-map to obtain a neural point matching pair.

[0083] In this embodiment, the overlapping areas of the sub-maps of each mobile robot are obtained through the above steps, and the corresponding local feature descriptors are calculated for each neural point in the overlapping area, and the index numbers of each neural point in each sub-map in the overlapping area and the corresponding local feature descriptors are saved in their respective hash tables for subsequent query matching.

[0084] Specifically, in this embodiment, when constructing the local feature descriptor of the neural point in each overlapping area, for each neural point in the overlapping area, a sphere with a radius of 10 is established with the neural point as the center, and then the radius of the sphere is segmented logarithmically to establish 4 concentric spheres, and the sphere area is divided into 6 sectors along the circumferential direction, so as to count the number of point distributions in each sector to represent the descriptor of the point. Among them, the radius of the sphere R j The segmentation formula is as follows:

[0085]

[0086] Among them, R j represents the radius of the concentric sphere, r min represents the minimum radius threshold, r max Indicates the maximum radius threshold. min =1 to prevent points from being distributed in an area with a radius that is too small. j∈[1,5], J=5, r max =10.

[0087] In this embodiment, after the local feature descriptors of each neural point in the overlapping area are constructed through the above steps, a KD-Tree is constructed for the constructed local feature descriptors to search for similar neural point matching pairs in each submap, thereby matching the neural points in each submap. Among them, when constructing the KD-Tree to search for similar neural point matching pairs in each submap, the eigenvalue similarity and relative position distance between the local feature vectors of each neural point are calculated by searching the hash table, and then the similarity between each neural point is judged based on the preset eigenvalue similarity threshold (for example, 0.2) and relative position distance threshold (for example, 8), so as to screen out similar neural point matching pairs. This method of constructing a KD-Tree in the form of a hash table can prevent the problem of multiple neural points in submap A matching with one neural point in submap B, thereby ensuring that the number of matching corresponding neural points in each submap is consistent.

[0088] Step S6: Cluster the neural point matching pairs to obtain the cluster point clusters corresponding to each sub-map matching.

[0089] Step S7: The local maps are merged according to the corresponding clustering point clusters matched by each sub-map to establish a complete large-area environment map.

[0090] In this embodiment, after completing the neural point matching of each sub-map through the above steps, the density-based clustering algorithm DBSCAN is used to cluster the neural point matching pairs, thereby obtaining the cluster point clusters corresponding to the matching of each sub-map. After obtaining the cluster point clusters corresponding to the matching of each sub-map through the DBSCAN algorithm, a transformation matrix is ​​constructed according to the offset of the center of the outer bounding box of each cluster point cluster in different sub-maps, and then each sub-map is transformed through the offset transformation matrix, thereby realizing the initial transformation of the sub-map. The transformation formula is as follows:

[0091]

[0092] Among them, T1 is the initial transformation matrix, Respectively represent the center of the outer bounding box of the 3D point cloud submap A and B, R 3 Represents three-dimensional space.

[0093] In addition, in this embodiment, after completing the initial transformation of each sub-map, the improved Fast-GICP registration algorithm can also be used to perform secondary matching on the transformed Mesh sub-maps, so as to achieve fine fusion of the map using the Fast-GICP algorithm combined with the Huber weight function and the M estimate. Among them, when performing secondary matching on the Mesh sub-map, a sphere with a radius of 20 and a farthest correlation distance of 5 is established with the center of the outer bounding box of each point cluster as the center of the sphere, and the neural points in the sphere are processed by the improved Fast-GICP. The specific operation is: first use the KD tree data structure to improve the efficiency of the nearest neighbor search, find the nearest neighbor point cloud pairs between different point cloud sub-maps, and then continuously optimize through Levenberg-Marquardt to minimize the distance error between the point cloud pairs to achieve matching. In this way, the sub-maps are subjected to secondary fine fusion, and the global trajectory and the fused map are optimized, so as to obtain the final large-area environment map.

[0094] Figure 5 This is a diagram showing the collaborative fusion of multiple mobile robots of the present invention. Figure 1 Effect comparison diagram of the embodiment. Figure 5 (1) is the original Mesh dense submap, Figure 5 (2) is the fused global Mesh map. Figure 5 It can be seen that in actual large-scale environmental tests, the multi-mobile robot collaborative mapping method based on geometric descriptors proposed in the present invention has higher map fusion accuracy and robustness.

[0095] The present specification also provides a computer-readable medium storing a computer program code, which, when executed by a processor, implements the method for establishing a large-area environment map for multiple mobile robots based on implicit neural points as described above.

[0096] This specification also provides a map fusion and positioning device for multiple mobile robots based on implicit neural points, including an instruction memory for storing instructions executable by a processor, and a processor for executing instructions in the instruction memory to implement the method for establishing a large-area environment map for multiple mobile robots based on implicit neural points as described above.

[0097] The previous description of the disclosure is provided to enable any person skilled in the art to make or use the disclosure. Various modifications to the disclosure will be apparent to those skilled in the art, and the general principles defined herein may be applied to other variations without departing from the spirit or scope of the disclosure. Thus, the disclosure is not intended to be limited to the examples and designs described herein, but should be granted the widest scope consistent with the principles and novel features disclosed herein.

[0098] Those skilled in the art will further appreciate that the various illustrative logic blocks, modules, circuits, and algorithm steps described in conjunction with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or a combination of the two. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps are generally described above in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. The technician may implement the described functionality in different ways for each specific application, but such implementation decisions should not be interpreted as resulting in a departure from the scope of the present invention.

[0099] The various illustrative logic blocks, modules, and circuits described in conjunction with the embodiments disclosed herein may be implemented or performed with a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in cooperation with a DSP core, or any other such configuration.

[0100] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. The software module may reside in a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read and write information from / to the storage medium. In an alternative, a storage medium may be integrated into a processor. The processor and the storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In an alternative, the processor and the storage medium may reside in a user terminal as discrete components.

[0101] In one or more exemplary embodiments, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented as a computer program product in software, each function may be stored on or transmitted by a computer-readable medium as one or more instructions or codes. Computer-readable media include both computer storage media and communication media, including any medium that facilitates the transfer of a computer program from one place to another. Storage media may be any available medium that can be accessed by a computer. As an example and not limitation, such a computer-readable medium may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of an instruction or data structure and can be accessed by a computer. Any connection is also properly referred to as a computer-readable medium. For example, if the software is transmitted from a website, a server, or other remote source using a coaxial cable, a fiber optic cable, a twisted pair, a digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwaves, the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwaves are included in the definition of the medium. Disk and disc as used herein include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc, wherein disk often reproduces data magnetically, while disc reproduces data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

Claims

1. A method for establishing a large-area environment map for multiple mobile robots based on implicit neural points, characterized in that: The following steps are involved: Step S1: Based on the built-in sensors of each mobile robot, 3D point cloud data of the surrounding environment is collected, and each frame of the collected 3D point cloud data is preprocessed to construct a neural point set of the local map of each mobile robot and the neural point features of each neural point; Step S2: predicting the signed distance field of each mobile robot's local map based on the neural point features corresponding to each neural point, and generating an implicit neural expression of each mobile robot's local map; Step S3: constructing a sub-map of the display geometry representation of each mobile robot based on the signed distance field of the local map of each mobile robot; Step S4: uploading the sub-maps of each mobile robot to the central processor for association and screening the overlapping areas; Step S5: constructing a local feature descriptor for each neural point in the overlapping area, and using the constructed local feature descriptor to match the neural points of each sub-map to obtain a neural point matching pair; Step S6: clustering the neural point matching pairs to obtain the clustering point clusters corresponding to each sub-map matching; Step S7: The local maps are merged according to the corresponding clustering point clusters matched by each sub-map to establish a complete large-area environment map.

2. The method for establishing a large-area environment map of multiple mobile robots based on implicit neural points according to claim 1 is characterized in that: When the method for establishing a large-area environmental map of multiple mobile robots based on implicit neural points preprocesses the collected 3D point cloud data, a voxel grid is established for the collected 3D point cloud data, and then the 3D point cloud data is downsampled using the constructed voxel grid to obtain a set of neural points corresponding to the mobile robot; wherein, after the method for establishing a large-area environmental map of multiple mobile robots based on implicit neural points completes the downsampling of the 3D point cloud data, the corresponding neural point features are obtained based on the voxel grid where each neural point is located, and the neural point features based on the voxel grid are stored in a hash table for subsequent positioning and mapping queries.

3. The method for establishing a large-area environment map of multiple mobile robots based on implicit neural points according to claim 2 is characterized in that: The neural point features include the three-dimensional coordinates of the neural point in the world coordinate system, quaternions, and geometric feature vectors representing the local geometric features of the neural point; wherein, after the method for establishing a large-area environmental map of multiple mobile robots based on implicit neural points completes the downsampling of the 3D point cloud data, an interpolation method is used to interpolate between each neural point, and then the neural point features of multiple neighboring neural points in the field of each interpolation position and the relative coordinate difference from the interpolation position are combined to predict the signed distance field of each interpolation position, thereby generating an implicit neural expression of the local map of each mobile robot.

4. The method for establishing a large-area environment map of multiple mobile robots based on implicit neural points according to claim 3 is characterized in that: The method for establishing a large-area environmental map of multiple mobile robots based on implicit neural points adopts a shallow neural network to predict the signed distance field of each interpolation position; wherein, after obtaining the neural point features of each adjacent neural point and the relative coordinate difference from the interpolation position according to the hash table, the neural point features of each adjacent neural point and the corresponding relative coordinate difference are cascaded, and then the cascaded three-dimensional coordinates, quaternions and geometric feature vectors are input into the shallow neural network for decoding, thereby outputting a signed distance field for representing the local geometric features of the three-dimensional space.

5. The method for establishing a large-area environment map of multiple mobile robots based on implicit neural points according to claim 1 is characterized in that: After the implicit neural point-based method for establishing a large-area environment map for multiple mobile robots generates the implicit neural expression of the local map of each mobile robot, the least square method based on the multi-weighted transformation matrix is ​​used to optimize the signed distance field prediction value error of any neural point in the current frame, thereby optimizing the position and posture of the sensor, and updating the local map of the implicit neural expression and the corresponding neural point features based on the signed distance field of the optimized neural point; wherein, the method for establishing a large-area environment map for multiple mobile robots based on implicit neural points constructs the optimal transformation matrix through the following formula: Among them, T * represents the optimal transformation matrix, p r Represents the current frame neural point set collected by the sensor, P represents any neural point in the neural point set of the current frame, S(T p ) represents the signed distance field value of the neural point p under the multi-weighted transformation matrix T.

6. The method for establishing a large-area environment map of multiple mobile robots based on implicit neural points according to claim 2 is characterized in that: The method for establishing a large-area environmental map of multiple mobile robots based on implicit neural points predicts the signed distance field of the local map of each mobile robot, and then uses the voxel-level reconstruction algorithm MarchingCubes to construct a Mesh sub-map of each mobile robot's displayed geometric representation; wherein, when the method for establishing a large-area environmental map of multiple mobile robots based on implicit neural points uses the Marching cubes algorithm to construct the Mesh sub-map of the mobile robot, the position correspondence between the vertices of each voxel in the local map based on implicit neural expression and the geometric obstacles is determined, and then the signed distance field of each neural point is converted into triangular mesh information, thereby generating a Mesh sub-map of the displayed geometric representation.

7. The method for establishing a large-area environment map of multiple mobile robots based on implicit neural points according to claim 6 is characterized in that: When constructing the Mesh sub-map of each mobile robot, the method for establishing a large-area environment map for multiple mobile robots based on implicit neural points determines the distance from each vertex to the surface of a geometric obstacle according to the signed distance field of each voxel vertex, and judges the position correspondence between each vertex and the geometric obstacle according to the distance from each vertex to the surface of the geometric obstacle, thereby establishing a Mesh sub-map showing geometric representation; wherein, the method for establishing a large-area environment map for multiple mobile robots based on implicit neural points judges the position correspondence between the vertex of any cell in the three-dimensional space and the geometric obstacle by the formula shown below: Where x represents the three-dimensional space R 3 The point in SDF(x) represents the distance field of point x. s represents the distance from point x to the surface of the geometric obstacle.

8. The method for establishing a large-area environment map of multiple mobile robots based on implicit neural points according to claim 1 is characterized in that: The method for establishing a large-area environment map for multiple mobile robots based on implicit neural points performs loop detection based on the point cloud map of the historical frame when constructing a sub-map of the geometric representation of each mobile robot display, thereby optimizing the positioning error and the map; wherein, when performing loop detection, the method for establishing a large-area environment map for multiple mobile robots based on implicit neural points identifies the loop area by comparing the global descriptors of the point cloud maps of the current frame and the historical frame, and optimizes the positioning error and the map of the current frame through the identified loop area.

9. The method for establishing a large-area environment map of multiple mobile robots based on implicit neural points according to claim 1 is characterized in that: The method for establishing a large-area environment map of multiple mobile robots based on implicit neural points uploads the sub-maps of each mobile robot to a central processor, and then transforms the local coordinate system of the sub-map of each mobile robot into a global coordinate system, thereby associating each sub-map into the same coordinate system and screening the overlapping areas.

10. The method for establishing a large-area environment map of multiple mobile robots based on implicit neural points according to claim 9, characterized in that: When screening the overlapping areas of each sub-map, the method for establishing a large-area environment map for multiple mobile robots based on implicit neural points constructs a corresponding octree for each sub-map located in the same coordinate system, and then judges whether each neural point in each sub-map is located in the same voxel grid based on the constructed octree grid, thereby screening out the neural points located in the same voxel grid as the overlapping areas.

11. The method for establishing a large-area environment map of multiple mobile robots based on implicit neural points according to claim 10, characterized in that: The method for establishing a large-area environment map for multiple mobile robots based on implicit neural points screens out the overlapping areas of each sub-map, and then stores the index number of each neural point of each sub-map in the overlapping area and the corresponding local feature descriptor in the corresponding hash table for the local feature descriptor corresponding to each neural point component in the overlapping area.

12. The method for establishing a large-area environment map of multiple mobile robots based on implicit neural points according to claim 11, characterized in that: After the method for establishing a large-area environment map for multiple mobile robots based on implicit neural points constructs the local feature descriptors of each neural point in the overlapping area, a KD-Tree is constructed for the constructed local feature descriptors to search for similar neural point matching pairs in each Mesh submap, thereby matching the neural points in each Mesh submap; wherein, when searching for similar neural point matching pairs in each Mesh submap, the method for establishing a large-area environment map for multiple mobile robots based on implicit neural points calculates the eigenvalue similarity and relative position distance between the local feature vectors of each neural point by searching a hash table, and then judges the similarity between each neural point based on a preset eigenvalue similarity threshold and relative position distance threshold, thereby screening out similar neural point matching pairs.

13. The method for establishing a large-area environment map of multiple mobile robots based on implicit neural points according to claim 1, characterized in that: After completing the neural point matching of each sub-map, the method for establishing a large-area environment map of multiple mobile robots based on implicit neural points uses the density-based clustering algorithm DBSCAN to cluster the neural point matching pairs, thereby obtaining the cluster point clusters corresponding to the matching of each sub-map; wherein, after obtaining the cluster point clusters corresponding to the matching of each sub-map, the method for establishing a large-area environment map of multiple mobile robots based on implicit neural points constructs a transformation matrix by calculating the offset of the center of the outer bounding box of each cluster point cluster, and then performs a preliminary transformation on each sub-map through the offset transformation matrix, thereby fusing each local map.

14. The method for establishing a large-area environment map of multiple mobile robots based on implicit neural points according to claim 13, characterized in that: The method for establishing a large-area environment map of multiple mobile robots based on implicit neural points calculates the offset of the center of the outer bounding box of each cluster point cluster by the following formula: in, Respectively represent the center of the outer bounding box of Mesh submap A and B, R 3 Represents three-dimensional space.

15. The method for establishing a large-area environment map of multiple mobile robots based on implicit neural points according to claim 13, characterized in that: The method for establishing a large-area environmental map for multiple mobile robots based on implicit neural points performs an initial transformation on each sub-map through an offset transformation matrix, and then uses an improved Fast-GICP registration algorithm to perform secondary matching on each transformed sub-map. Finally, the two-stage point cloud transformation is applied to the corresponding Mesh sub-map to obtain each Mesh sub-map after fine fusion. Among them, when the method for establishing a large-area environmental map for multiple mobile robots based on implicit neural points performs secondary matching on the sub-map, KD-tree is first used to find the nearest neighbor point cloud pairs between different sub-maps, and then the distance error between the point cloud pairs is minimized through continuous optimization, thereby achieving secondary matching.

16. According to the method for establishing a large-area environment map of multiple mobile robots based on implicit neural points in claim 1, the method for establishing a large-area environment map of multiple mobile robots based on implicit neural points respectively sets a local environment positioning and mapping module based on implicit neural points in the local server of each mobile robot, so that each mobile robot can perform positioning and mapping based on implicit neural points in its own coordinate system; wherein, After each mobile robot executes steps S1 to S3 to construct a sub-map in its own coordinate system through the local environment positioning and mapping module based on implicit neural points of the local server, the constructed sub-map is uploaded to the central processor for fusion, thereby establishing a large-area environment map.

17. A computer readable medium storing computer program code, characterized in that: The computer program code implements the method according to any one of claims 1 to 16 when executed by a processor.

18. A map fusion and positioning device for multiple mobile robots based on implicit neural points, characterized in that: include: a memory for storing instructions executable by a processor; as well as A processor, configured to execute the instructions to implement the method according to any one of claims 1 to 16.

Images