Map alignment method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a map alignment method and device, electronic equipment and a storage medium, and relates to the technical field of navigation positioning, and the specific scheme is as follows: performing slicing and rasterization processing on a first 3D point cloud map at a first height of a first 2D raster map to obtain a second 2D raster map; obtaining a first conversion relation based on the positions of the salient points in the first 2D grid map and the second 2D grid map; updating the first 3D point cloud map by using the first conversion relation to obtain a third 2D grid map; generating a map constraint relationship between the first 2D grid map and the third 2D grid map; and based on the map constraint relationship, performing map optimization on the pose of the key frame of the third 2D grid map, and based on the optimized pose, reconstructing the first 3D point cloud map to obtain an aligned 3D map. By applying the scheme provided by the embodiment of the invention, quick alignment of the 2D grid map and the 3D point cloud map of the operation scene can be realized.

Description

Technical Field

[0001] The present application relates to the field of navigation and positioning technology, and in particular to a map alignment method, device, electronic device and storage medium. Background Art

[0002] With the development of robot technology, mobile robots are introduced into more and more scenarios to complete various tasks. For example, in warehousing scenarios, mobile robots are used to complete material handling tasks. In order to ensure that the mobile robot successfully completes the task, it is necessary to build a map of the mobile robot's working scene and a topological map corresponding to the map of the working scene, so that the mobile robot can operate based on the map and topological map of the working scene. The map of the working scene may be a 2D (two-dimensional) grid map or a 3D (three-dimensional) point cloud map. The above 2D grid map and 3D point cloud map Figure 1 They are generally constructed in their own corresponding coordinate systems. When both a 2D grid map and a 3D point cloud map are constructed in the operation scene, the coordinate systems of these two maps are often inconsistent, that is, the 2D grid map and the 3D point cloud map are not aligned, resulting in the 2D grid map and the 3D point cloud map being unable to share the same topological map, making it difficult to deploy mobile robots in the operation scene.

[0003] Therefore, it is necessary to provide a map alignment solution to quickly align the 2D grid map and 3D point cloud map of the work scene. Summary of the Invention

[0004] The purpose of the embodiments of the present application is to provide a map alignment method, device, electronic device, and storage medium to achieve rapid alignment of a 2D grid map and a 3D point cloud map of a work scene. The specific technical solution is as follows:

[0005] According to one aspect of an embodiment of the present application, a map alignment method is provided, the method comprising:

[0006] Slicing and rasterizing a first 3D point cloud map of the work scene at a first height corresponding to a first 2D raster map of the work scene to obtain a second 2D raster map, wherein the first height is the height of a device that collects map data for the first 2D raster map;

[0007] Matching the first 2D grid map and the second 2D grid map based on positions of salient points in the work scene in the first 2D grid map and the second 2D grid map to obtain a first conversion relationship between a coordinate system of the first 2D grid map and a coordinate system of the first 3D point cloud map;

[0008] Using the first conversion relationship, updating the first 3D point cloud map, obtaining an updated 2D grid map corresponding to the second 2D grid map based on the updated first 3D point cloud map as a third 2D grid map, and generating a map constraint relationship between the first 2D grid map and the third 2D grid map;

[0009] performing graph optimization on the poses of the key frames of the third 2D grid map based on the poses of the key frames of the first 2D grid map, the first pose constraint relationship between the key frames of the first 2D grid map, the second pose constraint relationship between the key frames of the third 2D grid map, and the map constraint relationship to obtain optimized poses;

[0010] The first 3D point cloud map is reconstructed based on the optimized pose to obtain an aligned 3D map aligned with the first 2D grid map.

[0011] In one embodiment of the present application, slicing and rasterizing the first 3D point cloud map of the work scene at a first height corresponding to the first 2D grid map of the work scene to obtain a second 2D grid map includes:

[0012] performing layered processing on the first 3D point cloud map of the operation scene according to the coordinate extreme values ​​of the point clouds in the first 3D point cloud map and a preset number of slice layers;

[0013] Determining a grid in the grid map based on a resolution of the first 2D grid map and a coordinate extreme value of a point cloud in a first layer, wherein the first layer is the layer in which the first height is located in the layered result of the first 3D point cloud map;

[0014] Based on the point cloud in the first layer, the determined probability of each grid being occupied is obtained, and a second 2D grid map generated based on the obtained probability is obtained.

[0015] In one embodiment of the present application, obtaining the determined probability of each grid being occupied based on the point cloud in the first layer includes:

[0016] For each point in the point cloud in the first layer, determine the grid occupied by the point as the target grid hit by the point. If the target grid is hit for the first time, determine the probability of the target grid being occupied to be a preset probability value. If the target grid is hit more than once, update the probability of the target grid being occupied according to the following expression:

[0017]

[0018] Among them, l j (m i) is the probability of the target grid being occupied after being hit for the jth time, is the updated value of the probability that the preset grid is occupied, p is the probability value of the preset grid being occupied, l j-1 (m i ) is the probability of the target grid being occupied after being hit for the j-1th time.

[0019] In one embodiment of the present application, matching the first 2D grid map and the second 2D grid map based on positions of salient points in the work scene in the first 2D grid map and the second 2D grid map to obtain a first conversion relationship between the coordinate system of the first 2D grid map and the coordinate system of the first 3D point cloud map includes:

[0020] Obtaining first pixel coordinates of a salient point in the work scene on the first 2D grid map and second pixel coordinates of the salient point on the second 2D grid map;

[0021] Converting the first pixel coordinates into first real coordinates in the world coordinate system based on a coordinate value in which the origin of the coordinate system to which the first 2D grid map belongs is located in the world coordinate system;

[0022] Based on the coordinate value of the origin of the coordinate system to which the second 2D grid map belongs being located in the world coordinate system, converting the second pixel coordinate into a second real coordinate in the world coordinate system;

[0023] The first real coordinates and the second real coordinates are matched to determine a first conversion relationship between a coordinate system corresponding to the first 2D grid map and a coordinate system corresponding to the first 3D point cloud map.

[0024] In one embodiment of the present application, generating a map constraint relationship between the first 2D grid map and the third 2D grid map includes:

[0025] For each key frame in the third 2D grid map, determine the position of the key frame in the pose at the third pixel coordinate of the first 2D grid map and the fourth pixel coordinate of the third 2D grid map, respectively; sample the first 2D grid map with the third pixel coordinate as the center and according to a preset sampling radius to obtain a first virtual sampling frame; sample the third 2D grid map with the fourth pixel coordinate as the center and according to the preset sampling radius to obtain a second virtual sampling frame;

[0026] A map constraint relationship between the first 2D grid map and the third 2D grid map is generated based on the real coordinates of the third pixel coordinates, the real coordinates of the fourth pixel coordinates, the first virtual sampling frame, and the second virtual sampling frame.

[0027] In one embodiment of the present application, reconstructing the first 3D point cloud map based on the optimized pose to obtain an aligned 3D map aligned with the first 2D grid map includes:

[0028] For each point of the point cloud data in the first 3D point cloud map, determine the key frame in the first 3D point cloud map that is closest to the point as the neighboring key frame, determine the target optimized pose corresponding to the neighboring key frame in the optimized pose, and use the target optimized pose to transform the coordinates of the point to obtain an aligned 3D map aligned with the first 2D grid map.

[0029] In one embodiment of the present application, after reconstructing the second 3D point cloud map based on the optimized pose to obtain an aligned 3D map aligned with the first 2D grid map, the method further includes:

[0030] Performing layered processing on the aligned 3D map according to the coordinate extreme values ​​of the point cloud in the aligned 3D map and a preset number of slice layers;

[0031] Determining a grid in the grid map based on a preset resolution and a coordinate extreme value of a second layered point cloud, wherein the second layer is: a layer where the second height is located in the layered result of the aligned 3D map;

[0032] Based on the point cloud of the second layer, the determined probability of each grid being occupied is obtained, and a fourth 2D grid map generated based on the obtained probability is obtained.

[0033] According to another aspect of an embodiment of the present application, a map alignment device is provided, the device comprising:

[0034] a slicing and rasterization processing module, configured to slice and rasterize a first 3D point cloud map of the work scene at a first height corresponding to a first 2D raster map of the work scene to obtain a second 2D raster map, wherein the first height is the height of a device that collects map data for the first 2D raster map;

[0035] a map matching module, configured to match the first 2D grid map and the second 2D grid map based on positions of salient points in the work scene in the first 2D grid map and the second 2D grid map, and obtain a first conversion relationship between the coordinate system of the first 2D grid map and the coordinate system of the first 3D point cloud map;

[0036] a map updating module, configured to update the first 3D point cloud map using the first conversion relationship, obtain an updated 2D grid map corresponding to the second 2D grid map based on the updated first 3D point cloud map as a third 2D grid map, and generate a map constraint relationship between the first 2D grid map and the third 2D grid map;

[0037] a graph optimization module, configured to perform graph optimization on the poses of the key frames of the third 2D grid map based on the poses of the key frames of the first 2D grid map, the first pose constraint relationship between the key frames of the first 2D grid map, the second pose constraint relationship between the key frames of the third 2D grid map, and the map constraint relationship, to obtain optimized poses;

[0038] A map reconstruction module is used to reconstruct the first 3D point cloud map based on the optimized posture to obtain an aligned 3D map aligned with the first 2D grid map.

[0039] In one embodiment of the present application, the slicing and rasterization processing module includes: a layered processing unit, used to perform layered processing on the first 3D point cloud map of the work scene according to the coordinate extreme values ​​of the point cloud in the first 3D point cloud map and a preset number of slicing layers; a grid determination unit, used to determine the grids in the grid map based on the resolution of the first 2D grid map and the coordinate extreme values ​​of the point cloud in the first layer, wherein the first layer is: the layer where the first height is located in the layered result of the first 3D point cloud map; a probability acquisition unit, used to obtain the probability of each determined grid being occupied based on the point cloud in the first layer, and obtain a second 2D grid map generated based on the obtained probability.

[0040] In one embodiment of the present application, the probability obtaining unit is specifically configured to: for each point in the point cloud in the first layer, determine the grid occupied by the point as the target grid hit by the point; if the target grid is hit for the first time, determine the probability of the target grid being occupied to be a preset probability value; if the target grid is hit more than once, update the probability of the target grid being occupied according to the following expression: Among them, l j (m i ) is the probability of the target grid being occupied after being hit for the jth time, is the updated value of the probability that the preset grid is occupied, p is the probability value of the preset grid being occupied, l j-1 (m i ) is the probability of the target grid being occupied after being hit for the j-1th time.

[0041] In one embodiment of the present application, the map matching module is specifically used to: obtain the first pixel coordinates of the significant point in the work scene in the first 2D grid map, and the second pixel coordinates of the significant point in the second 2D grid map; based on the coordinate value of the origin of the coordinate system to which the first 2D grid map belongs is located in the world coordinate system, convert the first pixel coordinates into the first real coordinates in the world coordinate system; based on the coordinate value of the origin of the coordinate system to which the second 2D grid map belongs is located in the world coordinate system, convert the second pixel coordinates into the second real coordinates in the world coordinate system; match the first real coordinates and the second real coordinates to determine the first conversion relationship between the coordinate system corresponding to the first 2D grid map and the coordinate system corresponding to the first 3D point cloud map.

[0042] In one embodiment of the present application, the map update module is specifically configured to: for each key frame in the third 2D grid map, determine the position of the key frame in the pose at the third pixel coordinate of the first 2D grid map and the fourth pixel coordinate of the third 2D grid map, respectively; sample the first 2D grid map with the third pixel coordinate as the center and according to a preset sampling radius to obtain a first virtual sampling frame; sample the third 2D grid map with the fourth pixel coordinate as the center and according to a preset sampling radius to obtain a second virtual sampling frame; and generate a map constraint relationship between the first 2D grid map and the third 2D grid map based on the real coordinates of the third pixel coordinate, the real coordinates of the fourth pixel coordinate, the first virtual sampling frame, and the second virtual sampling frame.

[0043] In one embodiment of the present application, the map reconstruction module is specifically used to: for each point of the point cloud data in the first 3D point cloud map, determine the key frame in the first 3D point cloud map that is closest to the point as the neighboring key frame, and in the optimized pose, determine the target optimized pose corresponding to the neighboring key frame, and use the target optimized pose to transform the coordinates of the point to obtain an aligned 3D map aligned with the first 2D grid map.

[0044] In one embodiment of the present application, the device also includes: a layered processing module, used to perform layered processing on the aligned 3D map according to the coordinate extreme values ​​of the point cloud in the aligned 3D map and a preset number of slice layers; a grid determination module, used to determine the grids in the grid map based on the preset resolution and the coordinate extreme values ​​of the second layered point cloud, wherein the second layer is: the layer where the second height is located in the layered result of the aligned 3D map; a probability acquisition module, used to obtain the determined probability of each grid being occupied based on the point cloud of the second layer, and obtain a fourth 2D grid map generated based on the obtained probability.

[0045] According to another aspect of the embodiments of the present application, an electronic device is provided, including:

[0046] Memory for storing computer programs;

[0047] The processor is configured to implement any of the above-mentioned map alignment methods when executing a program stored in the memory.

[0048] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, any of the above-mentioned map alignment methods is implemented.

[0049] According to another aspect of the embodiments of the present application, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute any of the above-mentioned map alignment methods.

[0050] Beneficial effects of the embodiments of the present application:

[0051] As can be seen from the above, by applying the map alignment method provided in the embodiment of the present application, the first 3D point cloud map of the work scene is sliced ​​and rasterized to obtain a second 2D raster map of the first height belonging to the work scene. The first 2D raster map and the second 2D raster map contain the same area in the work scene. By pre-determining the significant points in the same area contained in the two 2D raster maps, the positions of the significant points can be determined in the first 2D raster map and the second 2D raster map respectively. Since the second 2D raster map is obtained by slicing and rasterizing the first 3D point cloud map, the coordinate system to which the second 2D raster map actually belongs is the coordinate system of the first 3D point cloud map. Then, the positions of the significant points in the first 2D raster map and the second 2D raster map are actually: the position of the real position of the significant point in the coordinate system of the first 2D raster map, and the position of the real position of the significant point in the coordinate system of the first 3D point cloud map. Then, based on the positions of the significant points in different coordinate systems, the first 2D raster map can be constructed. A first transformation relationship between the coordinate system of a 2D grid map and the coordinate system of a first 3D point cloud map is determined. Based on the first transformation relationship, the first 3D point cloud map is updated. Then, based on the updated first 3D point cloud map, a third 2D grid map is obtained. Then, a map constraint relationship is generated between the first 2D grid map and the third 2D grid map. Based on the map constraint relationship, the poses of the key frames of the first 2D grid map, the first pose constraint relationship between the key frames of the first 2D grid map, the second pose constraint relationship between the key frames of the third 2D grid map, and the map constraint relationship, the poses of the key frames of the third 2D grid map are graph optimized, so that the poses of the key frames of the third 2D grid map can be further aligned with the coordinate system of the first 2D grid map. Then, the first 3D point cloud map is reconstructed based on the optimized pose. The aligned 3D map obtained in this way can be aligned with the first 2D grid map, thereby realizing rapid alignment of the 2D grid map and the 3D point cloud map of the work scene.

[0052] Of course, it is not necessary to achieve all the advantages described above at the same time when implementing any product or method of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0054] Figure 1 A flowchart of a map alignment method provided in an embodiment of the present application;

[0055] Figure 2aA schematic diagram of the location of a significant point on a map provided in an embodiment of the present application;

[0056] Figure 2b A schematic diagram of another significant point location on a map provided in an embodiment of the present application;

[0057] Figure 3 A schematic diagram of optimized edges and optimized nodes for graph optimization provided in an embodiment of the present application;

[0058] Figure 4 A schematic diagram of a process flow of a slicing and rasterization processing method provided in an embodiment of the present application;

[0059] Figure 5 A schematic flow chart of another slicing and rasterization processing method provided in an embodiment of the present application;

[0060] Figure 6 A schematic structural diagram of a map alignment device provided in an embodiment of the present application;

[0061] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0062] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field based on this application are within the scope of protection of this application.

[0063] The application scenarios of the embodiments of the present application are described below.

[0064] Scene 1

[0065] If there is currently a 2D grid map of the work scene and a topological map corresponding to the 2D grid map, and a 2D navigation mobile robot is using the 2D grid map and the topological map corresponding to the 2D grid map to perform work, wherein the topological map corresponding to the 2D grid map records the positions of the work nodes on the 2D grid map where the mobile robot is performing work and the movement route. In this case, if a 3D navigation robot needs to be added to the work scene, then a 3D point cloud map of the work scene needs to be constructed. Then, based on the position of each topological point in the topological map corresponding to the 2D grid map, the topological points of the topological map corresponding to the 3D point cloud map are mapped one by one. In addition, a mapping relationship is constructed between the topological map corresponding to the 2D grid map and the topological map corresponding to the 3D point cloud map, wherein the topological map corresponding to the 3D point cloud map records the positions of the work nodes on the 3D point cloud map where the mobile robot is performing work and the movement route. In this way, the 3D navigation robot can perform work in the work scene based on the 3D point cloud map and the topological map corresponding to the 3D point cloud map.

[0066] Using the map alignment method provided in the embodiment of the present application, the 3D point cloud map of the working scene can be aligned to the 2D grid map, and the coordinates of the midpoints of the point cloud of the obtained aligned 3D map can be aligned with the coordinate system of the 2D grid map, so that the global consistency of the position information in the 3D point cloud map and the position information in the 2D grid map can be achieved. Furthermore, the coordinates of the real target position of the topological point in the topological map corresponding to the 2D grid map and the coordinates of the real target position in the aligned 3D map are the same and aligned. That is to say, the 3D navigation robot can use the aligned 3D map and the topological map corresponding to the 2D grid map to perform operations in the working scene, and the two can share the same set of topological maps.

[0067] In the map alignment method provided in the embodiment of the present application, if the first 2D grid map has a corresponding topological map, then the topological map can be used as the topological map corresponding to the aligned 3D map aligned with the first 2D grid map.

[0068] Scene 2

[0069] If a 2D grid map of a work scene at an altitude of α and a corresponding topological map exist, and a 2D navigation mobile robot equipped with an α-altitude sensor is currently operating using the 2D grid map at an altitude of α and the corresponding topological map at an altitude of α, then if a 2D navigation mobile robot equipped with a β-altitude sensor is to be added to the work scene, a 2D grid map of the work scene at an altitude of β must be constructed. However, the map data for the 2D grid map at an altitude of α and the map data for the 2D grid map at an altitude of β are collected using different devices, and therefore the coordinate systems of the 2D grid map at an altitude of α and the 2D grid map at an altitude of β are also different. Therefore, it is necessary to map the topological points of the topological map corresponding to the 2D grid map at an altitude of β one by one based on the position of each topological point in the topological map corresponding to the 2D grid map at an altitude of α, and then construct a mapping relationship between the topological map corresponding to the 2D grid map at an altitude of α and the topological map corresponding to the 2D grid map at an altitude of β. In this way, the 2D navigation mobile robot equipped with the β height sensor can operate in the operation scene based on the 2D grid map of the β height and the topological map corresponding to the 2D grid map of the β height.

[0070] Using the map alignment method provided in the embodiment of the present application, the first 3D point cloud map of the work scene can be collected first, and then the 3D point cloud map of the work scene can be aligned with the 2D grid map at an α height. The coordinates of the midpoints of the point cloud of the obtained aligned 3D map can be aligned with the coordinate system of the 2D grid map at an α height, and global consistency of the position information in the 3D point cloud map and the position information in the 2D grid map at an α height can be achieved. Furthermore, the coordinates of the real target position of the topological point in the topological map corresponding to the 2D grid map at an α height and the coordinates of the real target position in the aligned 3D map are the same and aligned. Furthermore, the aligned 3D map can be sliced ​​and rasterized at the β height of the working scene to obtain a 2D grid map at the β height. The coordinates of the positions in the obtained 2D grid map at the β height can be aligned with the coordinate system of the 2D grid map at the α height, thereby achieving global consistency between the position information in the 2D grid map at the β height and the position information in the 2D grid map at the α height. Similarly, the 2D grid map at the β height is sliced ​​from the aligned 3D map, and the coordinates of the real target position of the topological point in the topological map corresponding to the 2D grid map at the α height are the same and aligned with the coordinates of the real target position in the 2D grid map at the β height. That is to say, a 2D navigation mobile robot equipped with a β height sensor can use the 2D grid map at the β height and the topological map corresponding to the 2D grid map at the α height to operate in the working scene, and the two can share the same set of topological maps.

[0071] In the map alignment method provided in an embodiment of the present application, if a corresponding topological map exists for the first 2D grid map at height α of the work scene, then the above-mentioned topological map can be used as a topological map corresponding to the aligned 3D map aligned with the first 2D grid map, or the above-mentioned topological map can be used as a topological map corresponding to the 2D grid map at height β.

[0072] It can be seen that the map alignment method provided in the embodiment of the present application can greatly reduce the difficulty of deploying mobile robots that navigate based on different types of sensors in the same work scene.

[0073] The following describes the execution subject of the embodiment of the present application.

[0074] The solution provided in the embodiments of the present application can be applied to electronic devices such as desktop computers, laptops, tablet computers, and servers. For the sake of convenience, the execution entities of the map alignment methods provided in the embodiments of the present application are collectively referred to as alignment devices.

[0075] The concepts involved in the embodiments of the present application are explained below.

[0076] 1. Keyframes

[0077] A key frame is a data frame collected by a collection device containing point cloud data in the working environment. When building a map, a data frame can be selected from the continuous data frames collected by the collection device that collects map data for the working environment.

[0078] The key frame includes the point cloud data corresponding to the key frame collected by the acquisition device. For example, if the data frame collected by the acquisition device at position A is selected as the key frame, the key frame includes the point cloud data collected by the acquisition device at position A.

[0079] 2. Keyframe pose

[0080] The pose of the key frame is: the pose of the acquisition device when the acquisition device acquires the key frame.

[0081] The acquisition device can record the relative position information of the point cloud relative to the acquisition device as the position information of the point cloud in the key frame acquired. If the pose of the key frame belongs to the coordinate system of the 3D point cloud map, then based on the relative position information of the point cloud and the pose of the key frame, the position of the point cloud in the key frame in the 3D point cloud map can be determined. The pose of the key frame can also describe the overall pose of the point cloud of the key frame in the map coordinate system. Based on this, the 3D point cloud map can also be expressed as: 3DMap=T p1 P1∪T p2 P2∪…∪T pn P n , where T p1is the first key frame in the 3D point cloud map, P1 is the pose of the first key frame in the 3D point cloud map, and n is the number of key frames in the 3D point cloud map.

[0082] 3. Pose Constraint Relationships between Keyframes

[0083] The pose constraint relationship between keyframes is the relative pose between adjacent keyframes. The pose constraint relationship between keyframes can be calculated by using point cloud registration between two adjacent keyframes, or it can be obtained based on the pose of two adjacent keyframes.

[0084] 4. Map Loopback Pose Constraints

[0085] The map loop pose constraint relationship is: the relative pose relationship of the first key frame with respect to the second key frame, where the second key frame is: another key frame in the map key frame that has a different acquisition time from the first key frame and exists in the same area as the first key frame.

[0086] The keyframes, their poses, the pose constraints between them, and the map loop pose constraints can be obtained when building the map. For 2D raster maps, the 2D keyframes, their poses, the pose constraints between them, and the map loop pose constraints can be obtained. For 3D point cloud maps, the 3D keyframes, their poses, the pose constraints between them, and the map loop pose constraints can be obtained.

[0087] The map alignment method provided in the embodiment of the present application is described in detail below.

[0088] In one embodiment of the present application, see Figure 1 A flowchart of a map alignment method is provided, wherein the method includes the following steps S101-S105.

[0089] Step S101: Slice and rasterize a first 3D point cloud map of the work scene at a first height corresponding to a first 2D grid map of the work scene to obtain a second 2D grid map.

[0090] The first height is: the height of a device for collecting map data of the first 2D grid map.

[0091] The implementation of step S101 is described in the following embodiments and will not be described in detail here.

[0092] Step S102: Based on the positions of the salient points in the work scene in the first 2D grid map and the second 2D grid map, the first 2D grid map and the second 2D grid map are matched to obtain a first conversion relationship between the coordinate system of the first 2D grid map and the coordinate system of the first 3D point cloud map.

[0093] Among them, the salient points in the work scene are real location points in the work scene pre-selected by personnel, the preset number of salient points is greater than or equal to 3, and the preset number of salient points cannot be located on the same straight line.

[0094] The position of the salient point in the first 2D grid map and the position of the salient point in the second 2D grid map, and the position of the same salient point in different maps are corresponding. If the number of salient points is 3, the position of the salient point can be determined in the first 2D grid map and the second 2D grid map based on each salient point, respectively, to obtain 3 pairs of positions of the salient points in the first 2D grid map and the second 2D grid map.

[0095] The salient points can be easily identifiable position points in the work scene. For example, the salient points can be the position points of markers placed in advance by personnel, or they can be the position points of the scene itself in the work scene, such as the position points at the corners of the wall in the work scene, the position points at the side edges of the prismatic support columns, etc.

[0096] For example, see Figure 2a A schematic diagram of the location of a significant point on a map is provided. Figure 2a In the example, taking 4 significant points as an example, the first 2D grid map M on the left 2D A shown in 2D 、B 2D 、C 2D 、D 2D The four significant points are respectively determined at the four locations of the first 2D grid map. Similarly, the four significant points are determined at the four locations of the second 2D grid map M on the right. 3D A shown in 3D 、B 3D 、C 3D 、D 3D The four significant points are determined at four locations of the second 2D grid map. 2D With A 3D are the positions of the salient point A in the first 2D grid map and the second 2D grid map, respectively. 2D With A 3D is corresponding. Similarly, B 2D With B 3D are the positions of the salient point B in the first 2D grid map and the second 2D grid map, respectively. 2D With B 3D Correspondingly, C 2D with C 3D are the positions of the salient point C in the first 2D grid map and the second 2D grid map, respectively. 2D with C 3D It corresponds to D2D With D 3D are the position of the salient point D in the first 2D grid map, the position of the salient point D in the second 2D grid map, and D 2D With D 3D Are corresponding.

[0097] The first conversion relationship may include a 2D pose conversion relationship between the coordinate system of the second 2D grid map and the coordinate system of the first 2D grid map, and may also include a 3D pose conversion relationship between the coordinate system of the second 2D grid map and the coordinate system of the first 3D point cloud map. After obtaining the 2D pose conversion relationship, the 2D pose conversion relationship may be converted to obtain a 3D pose conversion relationship. For example, if the obtained 2D pose conversion relationship is converted into the Euler angle form: {x T ,y T ,yaw T}, the 3D pose transformation relationship obtained by converting the 2D pose transformation relationship in the form of Euler angles is: {x T ,y T ,0,0,0,yaw T The 2D pose conversion relationship can convert the pose of the second 2D grid map keyframe into a 2D pose in the coordinate system of the first 2D grid map, and the 3D pose conversion relationship can convert the pose of the first 3D point cloud map keyframe into a 3D pose in the coordinate system of the first 2D grid map.

[0098] The following describes the method of obtaining the first conversion relationship in step S102.

[0099] In one implementation, the alignment device may use a least squares parameter fitting method to match the first 2D grid map and the second 2D grid map based on the position of the salient point in the first 2D grid map and the second 2D grid map to obtain a first conversion relationship. For example, the least squares parameter fitting method may be a four-parameter method, etc.

[0100] In another implementation, the alignment device can calculate the transformation relationship between the position of the salient point in the first 2D grid map and the position of the salient point in the second 2D grid map by projection transformation or other means, achieve matching between the first 2D grid map and the second 2D grid map, and obtain a first conversion relationship.

[0101] Other methods of obtaining the first conversion relationship in step S102 are described in the following embodiments and are not described in detail here.

[0102] Step S103: Using the first conversion relationship, the first 3D point cloud map is updated, and based on the updated first 3D point cloud map, an updated 2D grid map corresponding to the second 2D grid map is obtained as the third 2D grid map, and a map constraint relationship is generated between the first 2D grid map and the third 2D grid map.

[0103] The following describes a method for updating the first 3D point cloud map using the first conversion relationship in step S103.

[0104] In one implementation, the alignment device can use the first transformation relationship to update the pose of the key frame of the first 3D point cloud map to obtain the pose of the key frame of the second 3D point cloud map, and then use the pose of the key frame of the second 3D point cloud map to update the point cloud data in the key frame of the first 3D point cloud map to obtain the second 3D point cloud map.

[0105] For example, the following expression can be used to update the pose of the keyframe of the first 3D point cloud map:

[0106] Pose 3d =T 3d ×Pose 3do ;

[0107] Among them, Pose 3d is the pose of the key frame of the second 3D point cloud map, T 3d Pose is the 3D pose transformation relationship included in the first transformation relationship. 3do is the pose of the key frame of the first 3D point cloud map.

[0108] In addition, the alignment device can calculate the pose constraints between the key frames of the second 3D point cloud map based on the poses of the key frames of the second 3D point cloud map. If the first 3D point cloud map has a map closure pose constraint, the map closure pose constraint of the second 3D point cloud map can also be calculated based on the poses of the key frames of the second 3D point cloud map.

[0109] Among them, if the first transformation relationship is used to update the key frame pose of the first 3D point cloud map, the key frame pose of the second 3D point cloud map is in the form of rotation matrix R and translation vector t: {R, t}. Then the key frame pose can be converted into Euler angle form according to the following expression: {x t ,y t ,z t ,roll R ,pitch R ,yaw R}, where x t 、y t and z tare the three components of the translation vector t.

[0110] Assume that the rotation matrix R is expressed as follows:

[0111]

[0112] Then, the rotation matrix can be converted to the pose of the keyframe in Euler angle form according to the following expression:

[0113] roll R =atan2(r 32 ,r 33 );

[0114]

[0115] yaw R =atan2(r 21 ,r 11 ).

[0116] The following describes a method for obtaining the updated 2D grid map corresponding to the second 2D grid map based on the updated first 3D point cloud map in step S103 as the third 2D grid map.

[0117] In one implementation, the updated first 3D point cloud map is used as the second 3D point cloud map, and the second 3D point cloud map is sliced ​​and rasterized at a first height to obtain a third 2D raster map.

[0118] The above method of obtaining the third 2D grid map is similar to the slicing and rasterization method in step S101. The difference lies in the difference between the "first 3D point cloud map" in step S101 and the "second 3D point cloud map" in the method of obtaining the third 2D grid map, and the difference between the "second 2D grid map" in step S101 and the "third 2D grid map" in the method of obtaining the third 2D grid map.

[0119] In addition, when a 2D grid map is obtained by slicing and rasterizing a 3D point cloud map, the pose of the key frame in the 3D point cloud map can be split into the 2D pose of the key frame in the 2D grid map. For example, if the pose of the key frame in the 3D point cloud map is expressed in the form of Euler angles: {x 3D ,y 3D ,z 3D ,roll 3D ,pitch 3D ,yaw 3D}, then the 2D pose of the key frame in the corresponding 2D grid map obtained by splitting is: {x 3D ,y 3D ,yaw 3DSimilarly, the alignment device can split the pose constraints between the key frames of the second 3D point cloud map in a similar manner to splitting the poses of the key frames in the 3D point cloud map into 2D poses of the key frames in the 2D grid map, thereby obtaining the pose constraints between the key frames of the third 2D grid map. If the second 3D point cloud map has a map closure pose constraint, the map closure pose constraint of the second 3D point cloud map can also be split to obtain the map closure pose constraint of the third 2D grid map.

[0120] The following combination Figure 2b The third 2D grid map will be described.

[0121] After updating the first 3D point cloud map using the first transformation relationship, the resulting second 3D point cloud map is still difficult to align with the first 2D grid map. The coordinates of the specified position in the third 2D grid map obtained by slicing and rasterizing the second 3D point cloud map are still different from the coordinates of the specified position in the first 2D grid map. For example, see Figure 2b Another schematic diagram of the location of a significant point on a map is provided. Figure 2a Corresponding to the example provided, taking the same four significant points as the example provided in 2a as an example, then if Figure 2b As shown, in the first 2D grid map M 2D The positions of the four significant points are determined as follows: A 2D 、B 2D 、C 2D 、D 2D Similarly, in the third 2D grid map M' 3D The positions corresponding to the four significant points are determined as: A' 3D , B' 3D , C' 3D 、D' 3D .

[0122] The following describes a method for generating the map constraint relationship between the first 2D grid map and the third 2D grid map in step S103.

[0123] The map constraint relationship is a 2D posture conversion relationship between the third 2D grid map coordinate system and the coordinate system of the first 2D grid map.

[0124] In one implementation, the alignment device can determine the position of the point cloud in the third 2D grid map based on the point cloud data and pose of the key frames of the third 2D grid map, and determine the position of the point cloud in the first 2D grid map based on the point cloud data and pose of the key frames of the first 2D grid map. Based on the positions of the point cloud in the third 2D grid map and the positions of the point cloud in the first 2D grid map, an association matrix is ​​constructed to represent the association relationship between the coordinate system of the first 2D grid map and the coordinate system of the third 2D grid map. The association matrix is ​​decomposed using a singular value decomposition method to calculate the map constraint relationship between the first 2D grid map and the third 2D grid map.

[0125] Other methods of generating map constraint relationships in step S103 will be described in the following embodiments and will not be described in detail here.

[0126] Step S104: Based on the pose of the key frame of the first 2D grid map, the first pose constraint relationship between the key frames of the first 2D grid map, the second pose constraint relationship between the key frames of the third 2D grid map, and the map constraint relationship, the pose of the key frame of the third 2D grid map is optimized to obtain an optimized pose.

[0127] In one implementation, the alignment device uses the first pose constraint relationship between the key frames of the first 2D grid map, the second pose constraint relationship between the key frames of the third 2D grid map, and the map constraint relationship as the optimization edges in the graph optimization, uses the poses of the key frames of the first 2D grid map and the poses of the key frames of the third 2D grid map as the optimization nodes, fixes the poses of the key frames of the first 2D grid map, and performs graph optimization on the poses of the key frames of the third 2D grid map to obtain the optimized poses.

[0128] Furthermore, if there is a map loop pose constraint relationship in the first 2D grid map and a map loop pose constraint relationship in the third 2D grid map, the alignment device uses the first pose constraint relationship between the key frames of the first 2D grid map, the second pose constraint relationship between the key frames of the third 2D grid map, the map loop pose constraint relationship of the first 2D grid map, the map loop pose constraint relationship of the third 2D grid map and the map constraint relationship as the optimization edges in the graph optimization, and uses the pose of the key frame of the first 2D grid map and the pose of the key frame of the third 2D grid map as the optimization nodes, fixes the pose of the key frame of the first 2D grid map, and performs graph optimization on the pose of the key frame of the third 2D grid map to obtain the optimized pose.

[0129] For example, see Figure 3 A schematic diagram of the optimized edges and optimized nodes of a graph optimization is provided. Figure 3 The blank circles in the graph represent the optimization nodes in the graph optimization. Figure 3The black line segments in the graph represent the optimized edges in graph optimization, M 2D The blank circle in the dotted box represents: the pose of the key frame of the first 2D grid map, M 2D The black diamond line segment in the dotted box represents: the first pose constraint relationship between the key frames of the first 2D grid map, M 2D The black triangle line segment in the dotted box represents the loop pose constraint relationship of the first 2D grid map. 3D The blank circle in the dashed box represents the pose of the key frame of the third 2D grid map, M' 3D The black diamond line in the dotted box represents the second pose constraint relationship between the key frames of the third 2D grid map, M' 3D The black triangle segments in the dotted box represent the loop pose constraints of the third 2D grid map. The black square segments in the figure represent the map constraints.

[0130] In addition, the obtained optimized pose is the optimized pose of the key frame of the third 2D grid map, that is, the obtained optimized pose belongs to a 2D pose. The alignment device can restore the optimized pose to a 3D pose based on the pose of the key frame in the second 3D point cloud map corresponding to the pose of the key frame of the third 2D grid map. For example, if the optimized pose is expressed in the form of Euler angles: {x opt ,y opt ,yaw opt}, the pose of the key frame in the second 3D point cloud map is expressed in the form of Euler angles: {x 3Dα ,y 3Dα ,z 3Dα ,roll 3Dα ,pitch 3Dα ,yaw 3Dα}, then the optimized pose restored to 3D pose is: {x opt ,y opt ,z 3Dα ,roll 3Dα ,pitch 3Dα ,yaw opt}.

[0131] Step S105: reconstructing the first 3D point cloud map based on the optimized pose to obtain an aligned 3D map aligned with the first 2D grid map.

[0132] In one implementation, for each point of the point cloud data in the first 3D point cloud map, a keyframe in the first 3D point cloud map that is closest to the point is determined as a neighboring keyframe. In the optimized pose, a target optimized pose corresponding to the neighboring keyframe is determined, and the coordinates of the point are transformed using the target optimized pose to obtain an aligned 3D map aligned with the first 2D grid map.

[0133] The following describes a method for determining the key frame in the first 3D point cloud map that is closest to the point.

[0134] Specifically, the alignment device can construct a keyframe of a kd-tree (k-dimensional tree, a tree-like data structure) structure based on the position of the keyframe in the first 3D point cloud map in the horizontal plane. In the keyframe of the kd-tree structure, for each point of the point cloud data in the first 3D point cloud map, the keyframe in the first 3D point cloud map that is closest to the point is determined. In this way, by searching for neighboring keyframes, the neighboring keyframes closest to the points in the point cloud data are searched, and the coordinates of the points are transformed using the target optimized pose corresponding to the neighboring keyframes. This can more accurately transform the coordinates of each point in the first 3D point cloud map, so that each point in the first 3D point cloud map is aligned with the coordinate system of the first 2D grid map, thereby obtaining an aligned 3D map.

[0135] The key frames in the first 3D point cloud map and the corresponding optimized poses are described below.

[0136] The second 2D grid map is obtained by slicing and rasterizing the first 3D point cloud map. During the slicing and rasterizing process, the key frames of the second 2D grid map can be determined based on the key frames in the first 3D point cloud map. Then, each key frame in the obtained second 2D grid map corresponds one-to-one to each key frame in the first 3D point cloud map.

[0137] Similarly, after the first 3D point cloud map is updated, the key frames in the third 2D grid map obtained based on the updated first 3D point cloud map correspond one-to-one to the key frames in the updated first 3D point cloud map, and after the poses of the key frames of the first 3D point cloud map are updated, the number of key frames of the first 3D point cloud map does not change, and the key frames after the update still correspond one-to-one to the key frames before the update. Therefore, the key frames in the third 2D grid map also correspond one-to-one to the key frames in the first 3D point cloud map.

[0138] Furthermore, when optimizing the poses of the keyframes in the third 2D grid map, the optimized keyframes actually still correspond one-to-one with the keyframes before optimization. Therefore, the keyframes in the optimized third 2D grid map also correspond one-to-one with the keyframes in the first 3D point cloud map. It can be seen that during the map update, map keyframe pose optimization, map slicing, and rasterization processes, the correspondence between the keyframes before and after processing can be determined. Therefore, there is a correspondence between the keyframes in each of the above maps. Based on this, for each optimized keyframe pose, the keyframe in the first 3D point cloud map corresponding to the optimized pose can be determined in the first 3D point cloud map.

[0139] As can be seen from the above, by applying the map alignment method provided in the embodiment of the present application, the first 3D point cloud map of the work scene is sliced ​​and rasterized to obtain a first 2D raster map and a second 2D raster map belonging to the first height of the work scene. The first 2D raster map and the second 2D raster map contain the same area in the work scene. By pre-determining significant points in the same area contained in the two 2D raster maps, the positions of the significant points can be determined in the first 2D raster map and the second 2D raster map, respectively. Since the second 2D raster map is obtained by slicing and rasterizing the first 3D point cloud map, the coordinate system to which the second 2D raster map actually belongs is the coordinate system of the first 3D point cloud map. Then, the positions of the significant points in the first 2D raster map and the second 2D raster map are actually: the position of the real position of the significant point in the coordinate system of the first 2D raster map, and the position of the real position of the significant point in the coordinate system of the first 3D point cloud map. Then, based on the position of the significant point in different coordinate systems, , construct a first transformation relationship between the coordinate system of the first 2D grid map and the coordinate system of the first 3D point cloud map, based on such a first transformation relationship, update the first 3D point cloud map to obtain a first 3D point cloud map, and then obtain a third 2D grid map based on the updated first 3D point cloud map. Then, generate a map constraint relationship between the first 2D grid map and the third 2D grid map, and based on such a map constraint relationship, the pose of the key frame of the first 2D grid map, the first pose constraint relationship between the key frames of the first 2D grid map, the second pose constraint relationship between the key frames of the third 2D grid map, and the map constraint relationship, perform graph optimization on the pose of the key frame of the third 2D grid map, so that the pose of the key frame of the third 2D grid map can be further aligned with the coordinate system of the first 2D grid map, and then reconstruct the first 3D point cloud map based on the optimized pose. The aligned 3D map obtained in this way can be aligned with the first 2D grid map, thereby realizing rapid alignment of the 2D grid map and the 3D point cloud map of the work scene.

[0140] Corresponding to scenario 1 above, using the map alignment method provided in the embodiment of the present application, the first 3D point cloud map is aligned with the first 2D grid map to obtain an aligned 3D map. A 3D navigation robot is then used to test the aligned 3D map and the topological map corresponding to the first 2D grid map. The 3D navigation robot is controlled to move to six preset points. The errors of the 3D navigation robot relative to the 2D navigation mobile robot are 3.12cm, 2.23cm, 2.00cm, 3.00cm, 3.16cm, and 0.15cm, respectively. It can be seen that the 3D navigation robot has a high accuracy in navigating using the aligned 3D map obtained using the map alignment method provided in the embodiment of the present application.

[0141] The following describes the manner in which the slicing and rasterization process is performed in step S101.

[0142] In one embodiment of the present application, see Figure 4 A schematic flow chart of a slicing and rasterization processing method is provided, wherein the method includes the following steps S401-S403.

[0143] Step S401: performing layer processing on the first 3D point cloud map of the operation scene according to the coordinate extreme values ​​of the point cloud in the first 3D point cloud map and the preset number of slice layers.

[0144] In one implementation, the alignment device determines the height of each layer of the first 3D point cloud map based on the coordinate extreme values ​​of the point cloud in the height direction in the first 3D point cloud map of the work scene and the preset number of slice layers: Among them, z max is the maximum coordinate extreme value of the point cloud in the height direction in the first 3D point cloud map, z min is the minimum coordinate extreme value of the point cloud in the height direction in the first 3D point cloud map, N chip The preset number of slice layers.

[0145] The following describes a method for determining the first layer.

[0146] The first 3D point cloud map is layered according to the height of each layer to obtain the layered results of each layer of the first 3D point cloud map. Then, the layer number of the first layer where the first height is located is calculated according to the following expression: Among them, h chip is the first height. If the calculated value is not an integer, the calculated value is rounded up to obtain the number of the first layer where the first height is located. Then, based on the calculated number of layers, the first layer is determined in the layering result.

[0147] Step S402: Determine the grids in the grid map based on the resolution of the first 2D grid map and the coordinate extremes of the point cloud in the first layer.

[0148] The first layer is the layer where the first height is located in the layered result of the first 3D point cloud map. The resolution of the first 2D grid map is the same as that of the second 2D grid map, for example, the resolution may be 0.02 m.

[0149] In one implementation, the alignment device may determine the number of grids in the grid map and the overall size of the grids according to the following expression:

[0150]

[0151] Where W is the number of grids divided in the grid map, It can represent the overall size of the grid, and δ is the resolution of the first 2D grid map.

[0152] Step S403: Based on the point cloud in the first layer, the determined probability of each grid being occupied is obtained, and a second 2D grid map generated based on the obtained probability is obtained.

[0153] In one implementation, for each point in the point cloud in the first layer, the grid occupied by the point is determined as the target grid hit by the point. If the target grid is hit for the first time, the probability of the target grid being occupied is determined to be a preset probability value. If the target grid is hit more than once, the probability of the target grid being occupied is updated according to the following expression:

[0154]

[0155] Among them, l j (m i ) is the probability of the target grid being occupied after being hit for the jth time, is the updated value of the probability that the preset grid is occupied, p is the probability value of the preset grid being occupied, l j-1 (m i ) is the probability of the target grid being occupied after being hit for the j-1th time.

[0156] After traversing each point in the point cloud in the first layer in the above manner, the final probability of each grid being occupied is obtained, and the second 2D grid map is obtained.

[0157] For example, if the probability of grid occupancy is obtained for the first point in the point cloud in the first layer, and the target grid where the first point is located is hit for the first time, the preset probability value of the probability of target grid occupancy can be: If the probability of grid occupation is obtained for the second point in the point cloud in the first layer, and if the target grid where the second point is located is the same as the target grid where the first point is located, then when the target grid is hit for the second time, the probability of the target grid being occupied is updated to: By analogy, the probability of each grid being occupied is finally obtained, and a second 2D grid map generated based on the obtained probabilities is obtained.

[0158] Furthermore, a maximum probability threshold for the probability of grid occupation can be set, and the probability of grid occupation greater than the probability threshold among the obtained grid occupation probabilities is adjusted to the maximum probability threshold. In this way, the above-mentioned process of processing the probability of grid occupation can be understood as the process of filtering the probability of grid occupation according to a preset filtering method. The above-mentioned preset filtering method refers to a filtering method that retains information at high frequencies and suppresses information at low frequencies. After such processing, the probability of grid occupation greater than the above-mentioned probability threshold is not only adjusted but also retained, and the probability of grid occupation less than or equal to the above-mentioned probability threshold is suppressed, so that a stable contour grid can be retained in the generated second 2D grid map.

[0159] In this way, the probability of occupancy of each grid in the grid map can be accurately updated, thereby improving the accuracy of determining the probability of occupancy of each grid.

[0160] From the above, it can be seen that through the coordinate extreme values ​​of the point cloud in the first 3D point cloud map and the preset number of slicing layers, the 3D point cloud map can be accurately sliced ​​and rasterized, and the grids in the grid map can be accurately determined. Then, based on the position of the determined grid and the position of the point cloud in the first layer, the probability of each grid being occupied can be accurately determined, and the second 2D grid map can be accurately generated.

[0161] In addition, when a first 3D point cloud map of the operation scene and a topological map corresponding to the first 3D point cloud map have been constructed, the first 3D point cloud map can be processed based on the slicing and rasterization processing method of steps S401-S403 above to obtain a 2D raster map at a first height, wherein the first height can be set according to the height of the sensor mounted on the 2D navigation mobile robot deployed in the operation scene. In this way, by setting different first heights, the first 3D point cloud map can be sliced ​​and rasterized multiple times to obtain 2D raster maps at different heights. In this way, the time cost, labor cost, equipment resource cost, etc. required for re-collecting 2D map data, reconstructing the 2D raster map, and reconstructing the topological map can be avoided.

[0162] In one embodiment of the present application, see Figure 5A flowchart of another slicing and rasterization processing method is provided. The method further includes the following steps S501-S503 after step S105.

[0163] Step S101: Slice and rasterize a first 3D point cloud map of the work scene at a first height corresponding to a first 2D grid map of the work scene to obtain a second 2D grid map.

[0164] Step S102: Based on the positions of the salient points in the work scene in the first 2D grid map and the second 2D grid map, the first 2D grid map and the second 2D grid map are matched to obtain a first conversion relationship between the coordinate system of the first 2D grid map and the coordinate system of the first 3D point cloud map.

[0165] Step S103: Using the first conversion relationship, the first 3D point cloud map is updated, and based on the updated first 3D point cloud map, an updated 2D grid map corresponding to the second 2D grid map is obtained as the third 2D grid map, and a map constraint relationship is generated between the first 2D grid map and the third 2D grid map.

[0166] Step S104: Based on the pose of the key frame of the first 2D grid map, the first pose constraint relationship between the key frames of the first 2D grid map, the second pose constraint relationship between the key frames of the third 2D grid map, and the map constraint relationship, the pose of the key frame of the third 2D grid map is optimized to obtain an optimized pose.

[0167] Step S105: reconstructing the first 3D point cloud map based on the optimized pose to obtain an aligned 3D map aligned with the first 2D grid map.

[0168] Step S501: performing layer processing on the aligned 3D map according to the coordinate extreme values ​​of the point cloud in the aligned 3D map and the preset number of slice layers.

[0169] Step S502: Determine the grids in the grid map based on the preset resolution and the coordinate extremes of the second layered point cloud.

[0170] The second layer is the layer at the second highest level in the layered result of the aligned 3D map.

[0171] Step S503: Based on the point cloud of the second layer, the determined probability of each grid being occupied is obtained, and a fourth 2D grid map generated based on the obtained probability is obtained.

[0172] The implementation method of steps S501-S503 is similar to that of steps S401-S403. The differences are the differences between the "first 3D point cloud map" in steps S401-S403 and the "aligned 3D map" in steps S501-S503, the "first layer" in steps S401-S403 and the "second layer" in steps S501-S503, and the "second 2D grid map" in steps S401-S403 and the "fourth 2D grid map" in steps S501-S503. They will not be described in detail here.

[0173] In this way, after obtaining the aligned 3D map, the fourth 2D grid map obtained by layering and slicing and rasterizing the aligned 3D map is also aligned with the coordinate system of the first 2D grid map, and multiple 2D grid maps belonging to the same coordinate system at different heights in the work scene can be obtained.

[0174] Corresponding to the second scenario above, when the topological map corresponding to the first 2D grid map at the first height has been constructed, the fourth 2D grid map at the second height can be obtained in the above manner. In this way, 2D grid maps at different heights can share the topological map, reducing the time cost, manpower cost, equipment resource cost, etc. required to reconstruct the topological map.

[0175] In the case where a 2D grid map at a height of 295 mm and a topological map corresponding to the first 2D grid map at a height of 295 mm have been constructed for the operation scene, and a 2D navigation mobile robot equipped with a 295 mm height sensor is using the first 2D grid map at a height of 295 mm and the topological map corresponding to the first 2D grid map at a height of 295 mm to perform an operation, after obtaining a fourth 2D grid map at a height of 2395 mm using the map alignment method provided in an embodiment of the present application, the 2D navigation mobile robot equipped with a 2395 mm height sensor is used to obtain a fourth 2D grid map at a height of 2395 mm. The navigation robot was tested for navigation based on the fourth 2D grid map at a height of 2395mm and the topological map corresponding to the first 2D grid map at a height of 295mm. The 2D navigation robot equipped with a 2395mm height sensor was controlled to go to six preset points. The errors of the 2D navigation robot equipped with a 2395mm height sensor relative to the 2D navigation mobile robot equipped with a 295mm height sensor were 1.42cm, 1.25cm, 2.00cm, 0.65cm, 1.16cm, and 0.45cm, respectively. It can be seen that the 2D navigation robot equipped with different height sensors has a high accuracy in navigating the 2D grid map obtained by rasterizing the aligned 3D map obtained using the map alignment method provided in the embodiment of the present application.

[0176] The following describes the method of obtaining the first conversion relationship in step S102.

[0177] In one embodiment of the present application, the alignment device may obtain the first conversion relationship by following the steps AC:

[0178] Step A: obtaining first pixel coordinates of a salient point in a work scene on a first 2D grid map and second pixel coordinates of the salient point on a second 2D grid map.

[0179] Specifically, a person may manually select a first pixel coordinate of a salient point in the first 2D grid map and a second pixel coordinate of a salient point in the second 2D grid map. The alignment device may obtain the first pixel coordinate and the second pixel coordinate input by the person.

[0180] Step B: Based on the coordinate value of the origin of the coordinate system of the first 2D grid map in the world coordinate system, the first pixel coordinate is converted into a first real coordinate in the world coordinate system. Based on the coordinate value of the origin of the coordinate system of the second 2D grid map in the world coordinate system, the second pixel coordinate is converted into a second real coordinate in the world coordinate system.

[0181] When constructing the first 2D grid map and the second 2D grid map, the coordinate value of the origin of the coordinate system of the first 2D grid map in the world coordinate system can be recorded, and the coordinate value of the origin of the coordinate system of the second 2D grid map in the world coordinate system can be recorded.

[0182] In one approach, the alignment device can determine, based on the resolution of the first 2D grid map and the first pixel coordinate, the real x-axis distance and the real y-axis distance of the first pixel coordinate relative to the origin of the coordinate system of the first 2D grid map, and then determine the first real coordinate based on the real x-axis distance and the real y-axis distance of the first pixel coordinate and the coordinate value of the origin in the world coordinate system. The x-axis and y-axis are two coordinate axes in the world coordinate system that are parallel to the horizontal plane.

[0183] Specifically, the alignment device may determine the first real coordinate of the first pixel coordinate according to the following expression:

[0184] x1=pix x1 ×δ+x 2dorg ;

[0185] y1=pix y1 ×δ+y 2dorg ;

[0186] Among them, x1 is the coordinate value on the x-axis of the first real coordinate, y1 is the coordinate value on the y-axis of the first real coordinate, and pix x1 is the coordinate value of the first pixel on the x-axis, pix y1 is the coordinate value of the first pixel coordinate on the y-axis, δ is the resolution of the first 2D grid map, x2dorg is the coordinate value of the origin of the first 2D grid map on the x-axis, y 2dorg is the coordinate value of the origin of the first 2D grid map on the y-axis.

[0187] Similarly, the alignment device can determine the second real coordinate of the second pixel coordinate according to the following expression:

[0188] x2=pix x2 ×δ+x 3dorg ;

[0189] y2=pix y2 ×δ+y 3dorg ;

[0190] Among them, x2 is the coordinate value on the x-axis of the second real coordinate, y2 is the coordinate value on the y-axis of the second real coordinate, and pix x is the coordinate value of the second pixel on the x-axis, pix y is the coordinate value of the second pixel on the y-axis, x 3dorg is the coordinate value of the origin of the second 2D grid map on the x-axis, y 3dorg The coordinate value of the origin of the second 2D grid map on the y-axis.

[0191] Step C: Match the first real coordinates and the second real coordinates to determine a first conversion relationship between the coordinate system corresponding to the first 2D grid map and the coordinate system corresponding to the first 3D point cloud map.

[0192] In one implementation, the alignment device may use 2D-ICP (2D-Iterative Closest Point) to match the first real coordinate and the second real coordinate, and calculate a first transformation relationship between the coordinate system corresponding to the first 2D grid map and the coordinate system corresponding to the first 3D point cloud map.

[0193] In addition, based on 2D-ICP calculations

[0194] From the above, it can be seen that by determining the first pixel coordinates of the salient point in the first 2D grid map and the second pixel coordinates of the salient point in the second 2D grid map, and then converting the pixel coordinates into real coordinates, the real coordinates of the salient point in the coordinate system of the first 2D grid map and the real coordinates of the real position of the salient point in the coordinate system of the first 3D point cloud map can be accurately determined. Based on the real coordinates of the same salient point in the two coordinate systems, the first conversion relationship between the two coordinate systems can be more accurately determined.

[0195] The following describes how to implement step S103 of generating map constraint relationships.

[0196] In one embodiment of the present application, see Figure 5 A flowchart of a method for generating map constraint relationships is provided. The method includes the following steps S501-S502.

[0197] Step S501: For each key frame in the third 2D grid map, determine the position of the key frame in the pose at the third pixel coordinate of the first 2D grid map and the fourth pixel coordinate of the third 2D grid map, respectively. With the third pixel coordinate as the center and according to the preset sampling radius, sample the first 2D grid map to obtain a first virtual sampling frame. With the fourth pixel coordinate as the center and according to the preset sampling radius, sample the third 2D grid map to obtain a second virtual sampling frame.

[0198] The following describes a method for determining the position of the key frame in the pose at the third pixel coordinate of the first 2D grid map and the fourth pixel coordinate of the third 2D grid map.

[0199] For example, if the position in the keyframe pose is {x k ,y k}, then the third pixel coordinate {row3,col3} of the position in the first 2D grid map can be determined according to the following expression:

[0200] col3=(x k -x 2dorg ) / δ;

[0201] row3=(y k -y 2dorg ) / δ.

[0202] The fourth pixel coordinate {row4,col4} of the position in the second 2D grid map is determined by the following expression:

[0203] col4=(x k -x 3dorg ) / δ;

[0204] row4=(y k -y 3dorg ) / δ.

[0205] The sampling method is described below.

[0206] When sampling the first 2D grid map, the alignment device records point cloud data within a circle centered at the third pixel coordinate and having a preset sampling radius, thereby obtaining a first virtual sampling frame. Similarly, when sampling the third 2D grid map, the alignment device records point cloud data within a circle centered at the fourth pixel coordinate and having a preset sampling radius, thereby obtaining a second virtual sampling frame.

[0207] Step S502: Generate a map constraint relationship between the first 2D grid map and the third 2D grid map based on the real coordinates of the third pixel coordinates, the real coordinates of the fourth pixel coordinates, the first virtual sampling frame, and the second virtual sampling frame.

[0208] In one implementation, the alignment device determines the real coordinates of the point cloud data in the first virtual sampling frame in the world coordinate system based on the real coordinates of the third pixel coordinates and the first virtual sampling frame, determines the real coordinates of the point cloud data in the second virtual sampling frame in the world coordinate system based on the real coordinates of the fourth pixel coordinates and the second virtual sampling frame, uses 2D-ICP to match the real coordinates of the point cloud data in the first virtual sampling frame in the world coordinate system and the real coordinates of the point cloud data in the second virtual sampling frame in the world coordinate system, and calculates the map constraint relationship between the first 2D grid map and the third 2D grid map.

[0209] As can be seen from the above, the first 2D grid map and the third 2D grid map are sampled respectively using a virtual frame sampling method to obtain a first virtual sampling frame of the first 2D grid map and a second virtual sampling frame of the third 2D grid map. The virtual sampling frame includes point cloud data. In this way, not only the real coordinates of the same position are used for matching, but also the point cloud data included in the virtual sampling frame is used for matching. In this way, in the process of determining the conversion relationship between the coordinate systems of the two maps, not only the coordinates of the same position in different coordinate systems are considered, but also the point cloud data included in the virtual sampling frame is considered. The conversion relationship between the coordinate system of the first 2D grid map and the third 2D grid map can be determined more accurately, and then the map constraint relationship between the first 2D grid map and the third 2D grid map can be generated more accurately.

[0210] Corresponding to the above-mentioned map alignment method, an embodiment of the present application also provides a map alignment device.

[0211] In one embodiment of the present application, see Figure 6 A schematic diagram of the structure of a map alignment device is provided, the device comprising:

[0212] The slicing and rasterization processing module 601 is configured to slice and rasterize a first 3D point cloud map of the work scene at a first height corresponding to a first 2D raster map of the work scene to obtain a second 2D raster map, wherein the first height is the height of a device that collected map data for the first 2D raster map.

[0213] A map matching module 602 is configured to match the first 2D grid map and the second 2D grid map based on positions of salient points in the work scene in the first 2D grid map and the second 2D grid map, to obtain a first conversion relationship between the coordinate system of the first 2D grid map and the coordinate system of the first 3D point cloud map;

[0214] A map updating module 603 is configured to update the first 3D point cloud map using the first conversion relationship, obtain an updated 2D grid map corresponding to the second 2D grid map based on the updated first 3D point cloud map as a third 2D grid map, and generate a map constraint relationship between the first 2D grid map and the third 2D grid map;

[0215] A graph optimization module 604 is configured to perform graph optimization on the poses of the key frames of the third 2D grid map based on the poses of the key frames of the first 2D grid map, the first pose constraint relationship between the key frames of the first 2D grid map, the second pose constraint relationship between the key frames of the third 2D grid map, and the map constraint relationship, to obtain optimized poses;

[0216] The map reconstruction module 605 is configured to reconstruct the first 3D point cloud map based on the optimized posture to obtain an aligned 3D map aligned with the first 2D grid map.

[0217] As can be seen from the above, by applying the map alignment method provided in the embodiment of the present application, the first 3D point cloud map of the work scene is sliced ​​and rasterized to obtain a first 2D raster map and a second 2D raster map belonging to the first height of the work scene. The first 2D raster map and the second 2D raster map contain the same area in the work scene. By pre-determining significant points in the same area contained in the two 2D raster maps, the positions of the significant points can be determined in the first 2D raster map and the second 2D raster map, respectively. Since the second 2D raster map is obtained by slicing and rasterizing the first 3D point cloud map, the coordinate system to which the second 2D raster map actually belongs is the coordinate system of the first 3D point cloud map. Then, the positions of the significant points in the first 2D raster map and the second 2D raster map are actually: the position of the real position of the significant point in the coordinate system of the first 2D raster map, and the position of the real position of the significant point in the coordinate system of the first 3D point cloud map. Then, based on the position of the significant point in different coordinate systems, , construct a first transformation relationship between the coordinate system of the first 2D grid map and the coordinate system of the first 3D point cloud map, based on such a first transformation relationship, update the first 3D point cloud map to obtain a first 3D point cloud map, and then obtain a third 2D grid map based on the updated first 3D point cloud map. Then, generate a map constraint relationship between the first 2D grid map and the third 2D grid map, and based on such a map constraint relationship, the pose of the key frame of the first 2D grid map, the first pose constraint relationship between the key frames of the first 2D grid map, the second pose constraint relationship between the key frames of the third 2D grid map, and the map constraint relationship, perform graph optimization on the pose of the key frame of the third 2D grid map, so that the pose of the key frame of the third 2D grid map can be further aligned with the coordinate system of the first 2D grid map, and then reconstruct the first 3D point cloud map based on the optimized pose. The aligned 3D map obtained in this way can be aligned with the first 2D grid map, thereby realizing rapid alignment of the 2D grid map and the 3D point cloud map of the work scene.

[0218] In one embodiment of the present application, the slicing and rasterization processing module includes: a layered processing unit, used to perform layered processing on the first 3D point cloud map of the work scene according to the coordinate extreme values ​​of the point cloud in the first 3D point cloud map and a preset number of slicing layers; a grid determination unit, used to determine the grids in the grid map based on the resolution of the first 2D grid map and the coordinate extreme values ​​of the point cloud in the first layer, wherein the first layer is: the layer where the first height is located in the layered result of the first 3D point cloud map; a probability acquisition unit, used to obtain the probability of each determined grid being occupied based on the point cloud in the first layer, and obtain a second 2D grid map generated based on the obtained probability.

[0219] From the above, it can be seen that through the coordinate extreme values ​​of the point cloud in the first 3D point cloud map and the preset number of slicing layers, the 3D point cloud map can be accurately sliced ​​and rasterized, and the grids in the grid map can be accurately determined. Then, based on the position of the determined grid and the position of the point cloud in the first layer, the probability of each grid being occupied can be accurately determined, and the second 2D grid map can be accurately generated.

[0220] In one embodiment of the present application, the probability obtaining unit is specifically configured to: for each point in the point cloud in the first layer, determine the grid occupied by the point as the target grid hit by the point; if the target grid is hit for the first time, determine the probability of the target grid being occupied to be a preset probability value; if the target grid is hit more than once, update the probability of the target grid being occupied according to the following expression: Among them, l j (m i ) is the probability of the target grid being occupied after being hit for the jth time, p is the preset probability value of the grid being occupied, is the updated value of the probability that the preset grid is occupied, l j-1 (m i ) is the probability of the target grid being occupied after being hit for the j-1th time.

[0221] In this way, the probability of occupancy of each grid in the grid map can be accurately updated, thereby improving the accuracy of determining the probability of occupancy of each grid.

[0222] In one embodiment of the present application, the map matching module is specifically used to: obtain the first pixel coordinates of the significant point in the work scene in the first 2D grid map, and the second pixel coordinates of the significant point in the second 2D grid map; based on the coordinate value of the origin of the coordinate system to which the first 2D grid map belongs is located in the world coordinate system, convert the first pixel coordinates into the first real coordinates in the world coordinate system; based on the coordinate value of the origin of the coordinate system to which the second 2D grid map belongs is located in the world coordinate system, convert the second pixel coordinates into the second real coordinates in the world coordinate system; match the first real coordinates and the second real coordinates to determine the first conversion relationship between the coordinate system corresponding to the first 2D grid map and the coordinate system corresponding to the first 3D point cloud map.

[0223] From the above, it can be seen that by determining the first pixel coordinates of the salient point in the first 2D grid map and the second pixel coordinates of the salient point in the second 2D grid map, and then converting the pixel coordinates into real coordinates, the real coordinates of the salient point in the coordinate system of the first 2D grid map and the real coordinates of the real position of the salient point in the coordinate system of the first 3D point cloud map can be accurately determined. Based on the real coordinates of the same salient point in the two coordinate systems, the first conversion relationship between the two coordinate systems can be more accurately determined.

[0224] In one embodiment of the present application, the map update module is specifically configured to: for each key frame in the third 2D grid map, determine the position of the key frame in the pose at the third pixel coordinate of the first 2D grid map and the fourth pixel coordinate of the third 2D grid map, respectively; sample the first 2D grid map with the third pixel coordinate as the center and according to a preset sampling radius to obtain a first virtual sampling frame; sample the third 2D grid map with the fourth pixel coordinate as the center and according to a preset sampling radius to obtain a second virtual sampling frame; and generate a map constraint relationship between the first 2D grid map and the third 2D grid map based on the real coordinates of the third pixel coordinate, the real coordinates of the fourth pixel coordinate, the first virtual sampling frame, and the second virtual sampling frame.

[0225] As can be seen from the above, the first 2D grid map and the third 2D grid map are sampled respectively using a virtual frame sampling method to obtain a first virtual sampling frame of the first 2D grid map and a second virtual sampling frame of the third 2D grid map. The virtual sampling frame includes point cloud data. In this way, not only the real coordinates of the same position are used for matching, but also the point cloud data included in the virtual sampling frame is used for matching. In this way, in the process of determining the conversion relationship between the coordinate systems of the two maps, not only the coordinates of the same position in different coordinate systems are considered, but also the point cloud data included in the virtual sampling frame is considered. The conversion relationship between the coordinate system of the first 2D grid map and the third 2D grid map can be determined more accurately, and then the map constraint relationship between the first 2D grid map and the third 2D grid map can be generated more accurately.

[0226] In one embodiment of the present application, the map reconstruction module is specifically used to: for each point of the point cloud data in the first 3D point cloud map, determine the key frame in the first 3D point cloud map that is closest to the point as the neighboring key frame, and in the optimized pose, determine the target optimized pose corresponding to the neighboring key frame, and use the target optimized pose to transform the coordinates of the point to obtain an aligned 3D map aligned with the first 2D grid map.

[0227] In this way, by searching for neighboring key frames, searching for the neighboring key frames closest to the points in the point cloud data, and using the target optimized pose corresponding to the neighboring key frames to transform the coordinates of the points, the coordinate transformation of each point in the first 3D point cloud map can be performed more accurately, so that each point of the first 3D point cloud map is aligned with the coordinate system of the first 2D grid map to obtain an aligned 3D map.

[0228] In one embodiment of the present application, the device also includes: a layered processing module, used to perform layered processing on the aligned 3D map according to the coordinate extreme values ​​of the point cloud in the aligned 3D map and a preset number of slice layers; a grid determination module, used to determine the grids in the grid map based on the preset resolution and the coordinate extreme values ​​of the second layered point cloud, wherein the second layer is: the layer where the second height is located in the layered result of the aligned 3D map; a probability acquisition module, used to obtain the determined probability of each grid being occupied based on the point cloud of the second layer, and obtain a fourth 2D grid map generated based on the obtained probability.

[0229] In this way, after obtaining the aligned 3D map, the fourth 2D grid map obtained by layering and slicing and rasterizing the aligned 3D map is also aligned with the coordinate system of the first 2D grid map, and multiple 2D grid maps belonging to the same coordinate system at different heights in the work scene can be obtained.

[0230] The present application also provides an electronic device, such as Figure 7 Shown, including:

[0231] Memory 701, used for storing computer programs;

[0232] The processor 702 is configured to implement any of the aforementioned map alignment methods when executing the program stored in the memory 701. The electronic device may further include a communication bus and / or a communication interface, and the processor 702, the communication interface, and the memory 701 communicate with each other via the communication bus.

[0233] The communication bus mentioned in the electronic device mentioned above may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not mean that there is only one bus or only one type of bus.

[0234] The communication interface is used for communication between the above electronic device and other devices.

[0235] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage. Alternatively, the memory may be at least one storage device located away from the processor.

[0236] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0237] In another embodiment provided by the present application, a computer-readable storage medium is further provided, wherein a computer program is stored in the computer-readable storage medium. When the computer program is executed by a processor, the steps of any of the above-mentioned map alignment methods are implemented.

[0238] In another embodiment provided by the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any one of the map alignment methods in the above embodiments.

[0239] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a solid-state drive (SSD).

[0240] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0241] Each embodiment in this specification is described in a related manner. Similar portions between the various embodiments can be referenced to each other. Each embodiment focuses on the differences between the other embodiments. In particular, the device, electronic device, storage medium, and computer program product embodiments are generally similar to the method embodiments, so their descriptions are relatively simple. For related portions, reference can be made to the descriptions of the method embodiments.

[0242] The above description is only a preferred embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application are included in the scope of protection of the present application.

Claims

1. A map alignment method, characterized in that: The method comprises: Slicing and rasterizing a first 3D point cloud map of the work scene at a first height corresponding to a first 2D raster map of the work scene to obtain a second 2D raster map, wherein the first height is the height of a device that collects map data for the first 2D raster map; Matching the first 2D grid map and the second 2D grid map based on positions of salient points in the work scene in the first 2D grid map and the second 2D grid map to obtain a first conversion relationship between a coordinate system of the first 2D grid map and a coordinate system of the first 3D point cloud map; Using the first conversion relationship, updating the first 3D point cloud map, obtaining an updated 2D grid map corresponding to the second 2D grid map based on the updated first 3D point cloud map as a third 2D grid map, and generating a map constraint relationship between the first 2D grid map and the third 2D grid map; performing graph optimization on the poses of the key frames of the third 2D grid map based on the poses of the key frames of the first 2D grid map, the first pose constraint relationship between the key frames of the first 2D grid map, the second pose constraint relationship between the key frames of the third 2D grid map, and the map constraint relationship to obtain optimized poses; The first 3D point cloud map is reconstructed based on the optimized pose to obtain an aligned 3D map aligned with the first 2D grid map.

2. The method according to claim 1, characterized in that The first 3D point cloud map of the operation scene is sliced ​​and rasterized at a first height corresponding to the first 2D grid map of the operation scene to obtain a second 2D grid map, including: performing layered processing on the first 3D point cloud map of the operation scene according to the coordinate extreme values ​​of the point clouds in the first 3D point cloud map and a preset number of slice layers; Determining a grid in the grid map based on a resolution of the first 2D grid map and a coordinate extreme value of a point cloud in a first layer, wherein the first layer is the layer in which the first height is located in the layered result of the first 3D point cloud map; Based on the point cloud in the first layer, the determined probability of each grid being occupied is obtained, and a second 2D grid map generated based on the obtained probability is obtained.

3. The method according to claim 2, characterized in that The obtaining, based on the point cloud in the first layer, the determined probability of each grid being occupied, includes: For each point in the point cloud in the first layer, determine the grid occupied by the point as the target grid hit by the point. If the target grid is hit for the first time, determine the probability of the target grid being occupied to be a preset probability value. If the target grid is hit more than once, update the probability of the target grid being occupied according to the following expression: Among them, l j (m i ) is the probability of the target grid being occupied after being hit for the jth time, is the updated value of the probability that the preset grid is occupied, p is the probability value of the preset grid being occupied, l j-1 (m i ) is the probability of the target grid being occupied after being hit for the j-1th time.

4. The method according to claim 1, wherein The matching of the first 2D grid map and the second 2D grid map based on positions of salient points in the work scene in the first 2D grid map and the second 2D grid map to obtain a first conversion relationship between a coordinate system of the first 2D grid map and a coordinate system of the first 3D point cloud map includes: Obtaining first pixel coordinates of a salient point in the work scene on the first 2D grid map and second pixel coordinates of the salient point on the second 2D grid map; Converting the first pixel coordinates into first real coordinates in the world coordinate system based on a coordinate value in which the origin of the coordinate system to which the first 2D grid map belongs is located in the world coordinate system; Based on the coordinate value of the origin of the coordinate system to which the second 2D grid map belongs being located in the world coordinate system, converting the second pixel coordinate into a second real coordinate in the world coordinate system; The first real coordinates and the second real coordinates are matched to determine a first conversion relationship between a coordinate system corresponding to the first 2D grid map and a coordinate system corresponding to the first 3D point cloud map.

5. The method according to claim 1, wherein The generating of a map constraint relationship between the first 2D grid map and the third 2D grid map includes: For each key frame in the third 2D grid map, determine the position of the key frame in the pose at the third pixel coordinate of the first 2D grid map and the fourth pixel coordinate of the third 2D grid map, respectively; sample the first 2D grid map with the third pixel coordinate as the center and according to a preset sampling radius to obtain a first virtual sampling frame; sample the third 2D grid map with the fourth pixel coordinate as the center and according to the preset sampling radius to obtain a second virtual sampling frame; A map constraint relationship between the first 2D grid map and the third 2D grid map is generated based on the real coordinates of the third pixel coordinates, the real coordinates of the fourth pixel coordinates, the first virtual sampling frame, and the second virtual sampling frame.

6. The method according to claim 1, characterized in that The reconstructing the first 3D point cloud map based on the optimized pose to obtain an aligned 3D map aligned with the first 2D grid map includes: For each point of the point cloud data in the first 3D point cloud map, determine the key frame in the first 3D point cloud map that is closest to the point as the neighboring key frame, determine the target optimized pose corresponding to the neighboring key frame in the optimized pose, and use the target optimized pose to transform the coordinates of the point to obtain an aligned 3D map aligned with the first 2D grid map.

7. The method according to any one of claims 1 to 6, characterized in that After reconstructing the first 3D point cloud map based on the optimized pose to obtain an aligned 3D map aligned with the first 2D grid map, the method further includes: Performing layered processing on the aligned 3D map according to the coordinate extreme values ​​of the point cloud in the aligned 3D map and a preset number of slice layers; Determining a grid in the grid map based on a preset resolution and a coordinate extreme value of a second layered point cloud, wherein the second layer is: a layer where the second height is located in the layered result of the aligned 3D map; Based on the point cloud of the second layer, the determined probability of each grid being occupied is obtained, and a fourth 2D grid map generated based on the obtained probability is obtained.

8. A map alignment device, characterized in that: The device comprises: a slicing and rasterization processing module, configured to slice and rasterize a first 3D point cloud map of the work scene at a first height corresponding to a first 2D raster map of the work scene to obtain a second 2D raster map, wherein the first height is the height of a device that collects map data for the first 2D raster map; a map matching module, configured to match the first 2D grid map and the second 2D grid map based on positions of salient points in the work scene in the first 2D grid map and the second 2D grid map, and obtain a first conversion relationship between the coordinate system of the first 2D grid map and the coordinate system of the first 3D point cloud map; a map updating module, configured to update the first 3D point cloud map using the first conversion relationship, obtain an updated 2D grid map corresponding to the second 2D grid map based on the updated first 3D point cloud map as a third 2D grid map, and generate a map constraint relationship between the first 2D grid map and the third 2D grid map; a graph optimization module, configured to perform graph optimization on the poses of the key frames of the third 2D grid map based on the poses of the key frames of the first 2D grid map, the first pose constraint relationship between the key frames of the first 2D grid map, the second pose constraint relationship between the key frames of the third 2D grid map, and the map constraint relationship, to obtain optimized poses; A map reconstruction module is used to reconstruct the first 3D point cloud map based on the optimized posture to obtain an aligned 3D map aligned with the first 2D grid map.

9. The device according to claim 8, characterized in that The slicing and rasterization processing module includes: a layer processing unit for performing layer processing on the first 3D point cloud map of the operation scene according to the coordinate extreme values ​​of the point cloud in the first 3D point cloud map and a preset number of slicing layers; a grid determination unit for determining the grids in the grid map based on the resolution of the first 2D grid map and the coordinate extreme values ​​of the point cloud in the first layer, wherein the first layer is the layer where the first height is located in the layered result of the first 3D point cloud map; a probability acquisition unit for obtaining the probability of each determined grid being occupied based on the point cloud in the first layer, and obtaining a second 2D grid map generated based on the obtained probability; and / or The probability obtaining unit is specifically configured to: for each point in the point cloud in the first layer, determine the grid occupied by the point as the target grid hit by the point; if the target grid is hit for the first time, determine the probability of the target grid being occupied to be a preset probability value; if the target grid is hit more than once, update the probability of the target grid being occupied according to the following expression: Among them, l j (m i ) is the probability of the target grid being occupied after being hit for the jth time, is the updated value of the probability that the preset grid is occupied, p is the probability value of the preset grid being occupied, l j-1 (m i ) is the probability of the target grid being occupied after being hit for the j-1th time; and / or The map matching module is specifically configured to: obtain first pixel coordinates of a salient point in the work scene in the first 2D grid map, and second pixel coordinates of the salient point in the second 2D grid map; convert the first pixel coordinates into first real coordinates in the world coordinate system based on the coordinate value of the origin of the coordinate system to which the first 2D grid map belongs located in the world coordinate system; convert the second pixel coordinates into second real coordinates in the world coordinate system based on the coordinate value of the origin of the coordinate system to which the second 2D grid map belongs located in the world coordinate system; match the first real coordinates and the second real coordinates to determine a first conversion relationship between the coordinate system corresponding to the first 2D grid map and the coordinate system corresponding to the first 3D point cloud map; and / or The map updating module is specifically configured to: for each key frame in the third 2D grid map, determine the position of the key frame in the pose at a third pixel coordinate of the first 2D grid map and a fourth pixel coordinate of the third 2D grid map, respectively; sample the first 2D grid map with the third pixel coordinate as the center and according to a preset sampling radius to obtain a first virtual sampling frame; sample the third 2D grid map with the fourth pixel coordinate as the center and according to a preset sampling radius to obtain a second virtual sampling frame; and generate a map constraint relationship between the first 2D grid map and the third 2D grid map based on the real coordinates of the third pixel coordinate, the real coordinates of the fourth pixel coordinate, the first virtual sampling frame, and the second virtual sampling frame; and / or The map reconstruction module is specifically configured to: for each point of the point cloud data in the first 3D point cloud map, determine a keyframe in the first 3D point cloud map that is closest to the point as a neighboring keyframe; determine a target optimized pose corresponding to the neighboring keyframe in the optimized pose; and transform the coordinates of the point using the target optimized pose to obtain an aligned 3D map aligned with the first 2D grid map; and / or The device also includes: a layer processing module for performing layer processing on the aligned 3D map based on the coordinate extreme values ​​of the point cloud in the aligned 3D map and a preset number of slice layers; a grid determination module for determining the grids in the grid map based on a preset resolution and the coordinate extreme values ​​of the second layer point cloud, wherein the second layer is the layer at the second height in the layered result of the aligned 3D map; and a probability acquisition module for obtaining, based on the point cloud of the second layer, the probability of each determined grid being occupied, to obtain a fourth 2D grid map generated based on the obtained probability.

10. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the method according to any one of claims 1 to 7 when executing a program stored in a memory.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images