Map generation method, device, equipment and storage medium

Abstract

The present application provides a map generation method, apparatus, device and storage medium, which are applied to a cloud server, including: obtaining a plurality of visual point cloud maps and a plurality of laser grid maps generated by a mobile device corresponding to a plurality of different areas in a target scene. According to the visual images corresponding to the plurality of visual point cloud maps, a first coordinate transformation relationship between the plurality of visual point cloud maps is determined. According to the first coordinate transformation relationship, the plurality of visual point cloud maps and the plurality of laser grid maps are converted to the same coordinate system to obtain a plurality of target visual point cloud images and a plurality of target laser grid maps corresponding to the same coordinate system. According to the laser point cloud data corresponding to the plurality of target laser grid maps, a second coordinate transformation relationship between the plurality of target laser grid maps is determined to correct the coordinates corresponding to the plurality of target visual point cloud images and the plurality of target laser grid maps, and generate a visual point cloud map and a laser grid map of the target scene.

Description

Technical Field

[0001] The present invention relates to the field of Internet technology, and in particular to a map generation method, device, equipment and storage medium. Background Art

[0002] When performing tasks indoors, intelligent robots typically first scan the surroundings using sensors (such as cameras and laser sensors) to obtain scene information and construct a map of the environment. Based on this map, the robot can locate its specific location within the indoor environment and, further, plan a path based on the specific mission destination, enabling autonomous movement.

[0003] However, when the indoor environment is large, such as a large gymnasium, exhibition hall or temporary hospital, it is difficult to scan the scene information of the entire indoor environment at one time, and errors are prone to occur when building the map, requiring re-building the map, resulting in low mapping efficiency. In addition, the map built based on the scene information obtained by a single sensing device cannot adapt to a variety of actual scenarios, and the map error is large. Summary of the Invention

[0004] Embodiments of the present invention provide a map generation method, apparatus, device, and storage medium for improving the accuracy of a target scene map.

[0005] In a first aspect, an embodiment of the present invention provides a map generation method, which is applied to a cloud server, wherein the cloud server is in communication with a mobile device, and the method includes:

[0006] Obtaining a plurality of visual point cloud maps and a plurality of laser grid maps generated by the mobile device for a plurality of different areas in a target scene; wherein the visual point cloud map is generated by the mobile device based on visual images captured by a camera, and the laser grid map is generated by the mobile device based on laser point cloud data captured by a single-line laser radar, and the coordinate system of the visual point cloud map and the laser grid map corresponding to the same area is the same;

[0007] determining a first coordinate transformation relationship between the plurality of visual point cloud maps according to the visual images respectively corresponding to the plurality of visual point cloud maps;

[0008] According to the first coordinate transformation relationship, the multiple visual point cloud maps and the multiple laser grid maps are converted into the same coordinate system to obtain multiple target visual point cloud images and multiple target laser grid maps corresponding to the same coordinate system;

[0009] Determining a second coordinate transformation relationship between the multiple target laser grid maps according to the laser point cloud data respectively corresponding to the multiple target laser grid maps;

[0010] According to the second coordinate transformation relationship, the coordinates corresponding to the multiple target visual point cloud images and the multiple target laser grid maps are corrected to splice the corrected multiple target visual point cloud images and the multiple target laser grid maps to generate a visual point cloud map and a laser grid map of the target scene.

[0011] In a second aspect, an embodiment of the present invention provides a map generation device, which is applied to a cloud server, wherein the cloud server is communicatively connected to a mobile device, and the device includes:

[0012] an acquisition module, configured to acquire a plurality of visual point cloud maps and a plurality of laser grid maps generated by the mobile device for a plurality of different areas in a target scene; wherein the visual point cloud map is generated by the mobile device based on visual images captured by a camera, and the laser grid map is generated by the mobile device based on laser point cloud data captured by a single-line laser radar, and the coordinate system of the visual point cloud map and the laser grid map corresponding to the same area is the same;

[0013] A processing module is used to determine a first coordinate transformation relationship between the multiple visual point cloud maps based on the visual images respectively corresponding to the multiple visual point cloud maps; based on the first coordinate transformation relationship, convert the multiple visual point cloud maps and the multiple laser grid maps into the same coordinate system to obtain multiple target visual point cloud images and multiple target laser grid maps corresponding to the same coordinate system; based on the laser point cloud data respectively corresponding to the multiple target laser grid maps, determine a second coordinate transformation relationship between the multiple target laser grid maps; based on the second coordinate transformation relationship, correct the coordinates corresponding to the multiple target visual point cloud images and the multiple target laser grid maps, so as to splice the corrected multiple target visual point cloud images and the multiple target laser grid maps to generate a visual point cloud map and a laser grid map of the target scene.

[0014] In a third aspect, an embodiment of the present invention provides an electronic device comprising: an inertial measurement unit, a camera, a laser sensor, a memory, a processor, and a communication interface; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor executes the map generation method as described in the first aspect.

[0015] In a fourth aspect, an embodiment of the present invention provides a non-transitory machine-readable storage medium having executable code stored thereon. When the executable code is executed by a processor of an electronic device, the processor can at least implement the map generation method described in the first aspect.

[0016] In an embodiment of the present invention, a map of a target scene is generated based on a mapping architecture in which a cloud server and a mobile device located on the edge side collaborate with each other. Specifically, the target scene is first divided into different areas, and then the visual image and laser point cloud data of each area are respectively collected by a camera and a single-line laser sensor configured on the mobile device located on the edge side to generate a visual point cloud map and a laser grid map corresponding to each area. Afterwards, the cloud server obtains a plurality of visual point cloud maps and a plurality of laser grid maps generated by the mobile device for a plurality of different areas in the target scene, and determines a first coordinate transformation relationship based on the visual point cloud images corresponding to the different areas, so as to convert the visual point cloud maps and laser grid maps corresponding to different coordinate systems into the same coordinate system, thereby obtaining a plurality of target visual point cloud images and a plurality of target laser grid maps; then, based on the target laser grid maps corresponding to different areas in the same coordinate system, a second coordinate transformation relationship is determined to correct the coordinates corresponding to the plurality of target visual point cloud images and the plurality of target laser grid maps. Since visual point cloud maps and laser raster maps are generated based on different types of scene information, and based on the information complementarity between visual images and laser point cloud data, the coordinates of multiple visual point cloud maps and laser raster maps are transformed and corrected, which can better combine the visual point cloud maps and laser raster maps corresponding to different areas. In addition, due to the sufficient computing power of the cloud server, accurate visual point cloud maps and laser raster maps of the target scene can be obtained efficiently. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 A schematic diagram of a map generation system provided by an embodiment of the present invention;

[0019] Figure 2 A flowchart of a map generation method provided by an embodiment of the present invention;

[0020] Figure 3 A schematic diagram of a constraint relationship provided by an embodiment of the present invention;

[0021] Figure 4 A schematic diagram of laser grid maps corresponding to different areas provided by an embodiment of the present invention;

[0022] Figure 5 A schematic diagram of a laser grid map of a target scene provided by an embodiment of the present invention;

[0023] Figure 6 A schematic structural diagram of a map generating device provided by an embodiment of the present invention;

[0024] Figure 7 This is a schematic structural diagram of an electronic device provided in this embodiment. DETAILED DESCRIPTION

[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0026] In addition, the step sequence in the following method embodiments is only an example and not a strict limitation.

[0027] In this embodiment, to improve map generation efficiency and accuracy, a map of the target scene is generated based on a collaborative mapping architecture between a cloud server and mobile devices located at the edge. The mobile devices are equipped with cameras and single-line laser radars, and are used to move within different areas of the target scene, collecting scene information for each area and generating visual point cloud maps and laser grid maps for each area. Optionally, the mobile devices include robots.

[0028] Taking a mobile device as a robot as an example, the map generation system composed of the robot at the edge and the cloud server is as follows: Figure 1 As shown, Figure 1 This diagram illustrates a map generation system provided by an embodiment of the present invention. The system includes several edge-side robots (Robot 1, Robot 2, ..., Robot N) and a cloud server. Robots 1, 2, ..., and N are each connected to the cloud server for information and command exchange. When generating a target scene map, the edge-side robots and the cloud server collaborate to generate a visual point cloud map and a laser grid map of the target scene.

[0029] In this embodiment, the target scene includes M different areas. There is an overlapping area between two adjacent areas, which can also be called a common view area. Figure 1 As shown in the gray area in the figure, robots 1, 2, ..., and N are placed in different areas and are configured to use their cameras to capture visual images of the current area and generate a visual point cloud map of the current area based on the visual images. They are also configured to use their single-line laser sensors to capture laser point cloud data of the current area and generate a laser grid map of the current area based on the laser point cloud data.

[0030] Among them, generating a visual point cloud map of the current area based on the visual image includes: extracting two-dimensional feature points of the visual image; performing three-dimensional reconstruction of the two-dimensional feature points through methods such as triangulation to obtain corresponding three-dimensional visual point clouds; and generating a visual point cloud map of the current area based on the three-dimensional visual point clouds. Generating a laser raster map of the current area based on laser point cloud data includes: determining nodes based on the laser point cloud data, constructing submaps based on the nodes, and generating multiple laser raster maps of the current area based on the submaps, wherein a node corresponds to at least one frame of laser point cloud data, and a submap corresponds to multiple nodes. In this embodiment, the specific generation process of the visual point cloud map and the laser raster map can refer to related technologies, such as the Cartographer algorithm.

[0031] Optionally, the number of robots can be the same as the number of areas included in the target scene, so that multiple robots can synchronously generate visual point cloud maps and laser grid maps for different areas respectively; the number of robots can also be less than the number of areas included in the target scene, and the same robot can generate visual point cloud maps and laser grid maps for different areas respectively in different time periods.

[0032] The cloud server is used to obtain the visual point cloud maps and laser grid maps of different areas generated by each robot and stitch them together to generate the visual point cloud map and laser grid map of the target scene. Given the computing power advantage of the cloud server, the visual point cloud map and laser grid map of the target scene can be stitched together accurately and efficiently.

[0033] In an alternative embodiment, Figure 1 The cloud server in the illustrated map generation system can be replaced with a target robot among multiple robots. Specifically, a robot with sufficient computing power can be configured as the target robot. This target robot communicates with the other robots, exchanging information and instructions. It can obtain the visual point cloud maps and laser grid maps of different areas generated by other robots and stitch them together to generate a visual point cloud map and laser grid map of the target scene.

[0034] Among them, the cloud server and the target robot are also used to send the visual point cloud map and laser grid map of the target scene to the robot performing the task in the target scene.

[0035] The following combination Figure 2 The map generation method provided by the embodiment of the present invention is described by taking the cloud server as the execution subject as an example. In actual application, the map generation method can also be executed by the target robot mentioned above.

[0036] Figure 2 A flowchart of a map generation method provided in an embodiment of the present invention is applied to a cloud server, such as Figure 2 As shown, the method includes the following steps:

[0037] 201. Obtain multiple visual point cloud maps and multiple laser grid maps generated by a mobile device for multiple different areas in a target scene; wherein the visual point cloud map is generated by the mobile device based on visual images captured by a camera, and the laser grid map is generated by the mobile device based on laser point cloud data captured by a single-line laser radar, and the coordinate systems of the visual point cloud map and the laser grid map corresponding to the same area are the same.

[0038] For multiple visual point cloud maps and multiple laser grid maps corresponding to multiple different areas in the target environment, since they are generated synchronously by different mobile devices (such as robots) or generated by the same mobile device in different time periods, the multiple visual point cloud maps correspond to different coordinate systems. Similarly, the multiple laser grid maps also correspond to different coordinate systems.

[0039] When generating a visual point cloud map of the target scene, if you want to stitch together the visual point cloud maps corresponding to multiple regions, you need to convert the multiple visual point cloud maps to the same coordinate system. Similarly, when generating a laser grid map of the target scene, if you want to stitch together the laser grid maps corresponding to multiple regions, you also need to convert the multiple laser grid maps to the same coordinate system.

[0040] In practical applications, visual images are easily disturbed by factors such as ambient light, and laser point cloud data cannot well describe the scene information of degraded scenes such as long corridors. Therefore, performing coordinate transformation on the visual point cloud map based only on the visual information corresponding to the visual point cloud image, or performing coordinate transformation on the laser grid map based only on the laser information corresponding to the laser grid map, will result in inaccurate visual point cloud map and laser grid map of the target scene finally spliced together.

[0041] To improve the accuracy of the generated visual point cloud map and laser grid map of the target scene, this embodiment leverages the complementary information provided by cameras and single-line laser sensors. In summary, multiple visual point cloud maps and laser grid maps are initially aligned based on the scene recognition and positioning capabilities of the visual information corresponding to the visual point cloud map. Then, coordinate corrections are performed on these preliminarily aligned visual point cloud maps and laser grid maps using the laser information corresponding to the laser grid map.

[0042] It should be noted that in this embodiment, for the same area, based on the pre-calibrated relative positional relationship between the camera and the single-line laser sensor, the visual image and laser point cloud data synchronously collected by the camera and the single-line laser sensor can be converted to the same coordinate system, so that the generated visual point cloud map and laser grid map also correspond to the same coordinate system. Since the coordinate system of the visual point cloud map and laser grid map corresponding to the same area is the same, the coordinate transformation relationship determined based on multiple visual point cloud maps can be used to adjust the coordinates of multiple laser grid maps. Similarly, the coordinate transformation relationship determined based on multiple laser grid maps can also be used to adjust the coordinates of multiple visual point cloud maps.

[0043] The coordinate transformation process of multiple visual point cloud maps and multiple laser grid maps is described in detail below through steps 202 to 205.

[0044] 202. Determine a first coordinate transformation relationship between the multiple visual point cloud maps based on the visual images corresponding to the multiple visual point cloud maps.

[0045] In this embodiment, overlapping areas exist between different regions. For example, there is an overlapping area ab between region a and region b. It is understood that if there is an overlapping area ab between region a and region b, then both the visual point cloud map Ma corresponding to region a and the visual point cloud map Mb corresponding to region b contain a portion of the visual point cloud map corresponding to the overlapping area ab. Based on the portion of the visual point cloud map corresponding to the overlapping area ab, the coordinate transformation relationship between the visual point cloud map Ma and the visual point cloud map Mb is determined.

[0046] During implementation, similar two-dimensional feature points between each of the multiple visual point cloud maps can be determined based on the two-dimensional feature points of the visual images corresponding to the multiple visual point cloud maps. For example, a distance calculation, such as Euclidean distance or Hamming distance, can be performed based on the descriptors corresponding to the two-dimensional feature points of the visual point cloud map Ma and the descriptors corresponding to the two-dimensional feature points of the visual point cloud map Mb. If the distance between the two descriptors is less than a set threshold, the two-dimensional feature points corresponding to the two descriptors are determined to be similar two-dimensional feature points between the visual point cloud map Ma and the visual point cloud map Mb.

[0047] Then, based on the similar 2D feature points between the multiple visual point cloud maps, the target visual images that match each other are determined. It is understood that a frame of visual image corresponds to multiple 2D feature points. If two frames of visual image both contain images corresponding to a certain object, then the two frames of image must correspond to a certain number of similar 2D feature points.

[0048] In this embodiment, in order to avoid interference from noise or other factors in determining the target visual image, a threshold value can be pre-set. The set threshold value is used to determine whether two frames of images containing similar two-dimensional feature points can be used as target visual images. Specifically, if the number of similar two-dimensional feature points between the first visual image corresponding to the first visual point cloud map and the second visual image corresponding to the second visual point cloud map is greater than or equal to the set threshold value, the first visual image and the second visual image are determined to be target visual images that match between the first visual point cloud map and the second visual point cloud map. The first visual point cloud map and the second visual point cloud map are any two visual point cloud maps from a plurality of visual point cloud maps.

[0049] Afterwards, the pose transformation relationship between each pair of the multiple visual point cloud maps is determined based on the pose information corresponding to the target visual image. The pose information includes rotation information and translation information. When generating a visual point cloud map of a certain area based on the visual images of that area, the pose information of each visual image is predetermined. Furthermore, it should be noted that the target visual images in this embodiment exist in pairs. During the specific implementation process, the pose transformation relationship between the two visual point cloud maps is determined based on the pose information corresponding to each visual image in the paired target visual images.

[0050] For ease of understanding, let's assume, for example, that the number of similar 2D feature points between visual image a1 corresponding to visual point cloud map Ma and visual image b1 corresponding to visual point cloud map Mb exceeds a set threshold. Then, visual image a1 and visual image b1 are determined to be target visual images that match between visual point cloud map Ma and visual point cloud map Mb. Then, based on the pose information corresponding to visual image a1 and the pose information corresponding to visual image b1, the pose transformation relationship between visual point cloud map Ma and visual point cloud map Mb is determined.

[0051] During the specific implementation process, there may be more than one pair of matching target visual images between the two visual point cloud maps, wherein each pair of matching target visual images can determine a pose transformation relationship. Ideally, the pose transformation relationships determined by at least one pair of matching target visual images are the same, for example: the pose transformation relationship T(a1-b1) between the visual image a1 corresponding to the visual point cloud map Ma and the visual image b1 corresponding to the visual point cloud map Mb is the same as the pose transformation relationship T(a2-b2) between the visual image a2 corresponding to the visual point cloud map Ma and the visual image b2 corresponding to the visual point cloud map Mb. However, in actual applications, due to factors such as equipment measurement errors, the pose transformation relationships determined by at least one pair of matching target visual images are often not the same.

[0052] Therefore, if there is at least one pair of matching target visual images between the first visual point cloud map and the second visual point cloud map, the initial pose transformation relationship corresponding to each pair of matching target visual images is determined based on the pose information corresponding to each pair of matching target visual images. Subsequently, the pose transformation relationship between the first visual point cloud map and the second visual point cloud map is determined based on the at least one initial pose transformation relationship.

[0053] As an optional implementation method, an optimization function can be generated with the target pose transformation relationship as the quantity to be optimized, wherein the optimization function takes the deviation between the optimized target pose transformation relationship and at least one initial pose transformation relationship as the optimization target; and the target pose transformation relationship that meets the optimization target is determined to be the pose transformation relationship between the first visual point cloud map and the second visual point cloud map.

[0054] For example, assuming that there are three pairs of matching target visual images between the visual point cloud map Ma and the visual point cloud map Mb, and three initial pose transformation relationships are determined correspondingly, namely T1, T2 and T3, then the target pose transformation relationship x can be used as the quantity to be optimized, and the optimization function is generated: Min(x-T1)+Min(x-T1)+Min(x-T3), with minimizing the deviation between x and T1, T2 and T3 as the optimization goal, and determining the target pose transformation relationship that meets the optimization goal: x=T0. Among them, the initial value of x can be taken from any one of T1, T2 and T3, and Min means minimization. Afterwards, T0 is determined as the pose transformation relationship between the visual point cloud map Ma and the visual point cloud map Mb.

[0055] Finally, according to the pose transformation relationship between the multiple visual point cloud maps, the first coordinate transformation relationship between the multiple visual point cloud maps is determined. The first coordinate transformation relationship includes: the pose transformation relationship between the multiple visual point cloud maps and the target visual point cloud map 5, and the target visual point cloud map is any one of the multiple visual point cloud maps. In layman's terms,

[0056] It is to select a coordinate system and determine the transformation relationship between other coordinate systems and the selected coordinate system.

[0057] For example, assuming that the target scene is divided into four areas, namely area a, area b, area c and area d, the corresponding generated visual point cloud maps are visual point cloud map Ma, visual point cloud map Mb, visual point cloud map Mc and visual point cloud map D.

[0058] There is a pose transformation relationship Tab between the visual point cloud map Ma and the visual point cloud map Mb, and there is a pose transformation relationship Tbc between the visual point cloud map Mb and the visual point cloud map Mc.

[0059] There is a pose transformation relationship Tad between the visual point cloud map Md. If the visual point cloud map Ma is selected as the target visual point cloud map, since there is no direct pose transformation relationship between the visual point cloud map Mc and the visual point cloud map Ma, the pose transformation relationship Tac between the visual point cloud map Mc and the visual point cloud map Ma is determined based on Tab and Tbc, thereby determining the first pose transformation relationship corresponding to the visual point cloud map Ma, the visual point cloud map Mb, the visual point cloud map Mc and the visual point cloud map Md. The first pose transformation relationship includes: Tab, Tac and Tad.

[0060] 203. According to the first coordinate transformation relationship, the multiple visual point cloud maps and the multiple laser grid maps are transformed into the same coordinate system to obtain multiple target visual point cloud images and multiple target laser grid maps corresponding to the same coordinate system.

[0061] Based on the above assumptions, the visual point cloud map Mb, the visual point cloud map Mc and the visual point cloud map Md are all converted to visual point

[0062] The coordinate system of the cloud map Ma. Similar to the visual point cloud map, the laser grid maps corresponding to 0 in area a, area b, area c, and area d are also converted to the same coordinate system based on Tab, Tac, and Tad.

[0063] In practical applications, there may still be gaps or ghosting between multiple target visual maps or multiple target laser grid maps that are initially aligned through the first coordinate transformation relationship. In this embodiment, the first coordinate transformation relationship is further optimized based on the multiple target laser grid maps.

[0064] 204. Determine the laser point cloud data corresponding to the laser grid maps of the multiple targets. Figure 5 The second coordinate transformation relationship between them.

[0065] For ease of understanding, first combine Figure 3 Explain the two concepts of intra-regional constraint relationship and inter-regional constraint relationship. Figure 3 A schematic diagram of a constraint relationship provided by an embodiment of the present invention is shown in FIG. Figure 3 As shown, the laser grid map of each area corresponds to multiple submaps, each submap corresponds to multiple nodes, and each node corresponds to multiple frames of laser point cloud data ( Figure 3 not shown).

[0066] The constraint relationship within the 0 area is used to describe the relative position between the node node and the submap submap corresponding to the same laser grid map. Figure 3The solid line represents the constraint relationship between the submap submap-a1 and the node node-a1.

[0067] The inter-region constraint relationship is used to describe the relative position between the node and submap corresponding to different target laser grid maps. Figure 3 The dotted line represents the constraint relationship between the submap submap-a1 and the node node-b1.

[0068] In this embodiment, the intra-regional constraints between the multiple target laser grid maps are predetermined when the laser grid maps for each region are generated. In this embodiment, the coordinates corresponding to the multiple target visual maps and the multiple target laser grid maps are corrected by determining the inter-regional constraints between the different target laser grid maps.

[0069] First, according to the nodes and submaps corresponding to the multiple target laser grid maps, the inter-region constraint relationships between the multiple target laser grid maps are determined.

[0070] It can be understood that if there is an overlapping area ab between area a and area b, then there is a partial laser grid map corresponding to the overlapping area ab in both the laser grid map Ma' corresponding to area a and the laser grid map Mb' corresponding to area b, and the node node and submap submap corresponding to the overlapping area ab in the laser grid map Ma' match the node node and submap submap corresponding to the overlapping area ab in the laser grid map Mb'.

[0071] Therefore, when determining the inter-region constraint relationship between multiple target laser grid maps, if the laser point cloud data in the node corresponding to the first target laser grid map matches the submap corresponding to the second target laser grid map, a first inter-region constraint relationship is established between the first target laser grid map and the second target laser grid map; if the laser point cloud data in the node corresponding to the second target laser grid map matches the submap corresponding to the first target laser grid map, a second inter-region constraint relationship is established between the first target laser grid map and the second target laser grid map; based on the first inter-region constraint relationship and the second inter-region constraint relationship, the inter-region constraint relationship between the first target laser grid map and the second target laser grid map is determined.

[0072] The first target laser grid map and the second target laser grid map are any two target laser grid maps among the plurality of target laser grid maps. The first inter-region constraint relationship and the second inter-region constraint relationship are actually a cross constraint.

[0073] For ease of understanding, for example, suppose Figure 3 The laser point cloud data in the node node-a1 corresponding to the laser grid map Ma' is matched with the submap submap-b1 corresponding to the laser grid map Mb', then a constraint relationship between the node node-a1 and the submap submap-b1 is established as the first inter-region constraint relationship between the laser grid map Ma' and the laser grid map Mb'. Similarly, if Figure 3 The laser point cloud data in node-b1, corresponding to laser grid map Mb', matches the submap-a1 corresponding to laser grid map Ma'. A constraint relationship is established between node-b1 and submap-a1, serving as the second inter-region constraint relationship between laser grid map Ma' and laser grid map Mb'. Finally, the inter-region constraint relationships between laser grid map Ma' and laser grid map Mb' are determined as follows: a constraint relationship between node-a1 and submap-b1, and a constraint relationship between node-b1 and submap-a1.

[0074] In practical applications, the number of the first inter-region constraint relationship and the second inter-region constraint relationship between the two laser grid maps is more than one.

[0075] Afterwards, the poses corresponding to the nodes and submaps of the multiple target laser grid maps are used as the quantities to be optimized, and the intra-region constraint relationships and inter-region constraint relationships corresponding to the multiple target laser grid maps are used as the constraint quantities to determine the optimized poses corresponding to the nodes and submaps of the multiple target laser grid maps.

[0076] Finally, the second coordinate transformation relationship between multiple target laser grid maps is determined based on the optimized posture.

[0077] 205. According to the second coordinate transformation relationship, the coordinates corresponding to the multiple target visual point cloud images and the multiple target laser grid maps are corrected to splice the corrected multiple target visual point cloud images and the multiple target laser grid maps to generate a visual point cloud map and a laser grid map of the target scene.

[0078] Figure 4 Schematic diagram of laser grid maps corresponding to different areas provided by the embodiment of the present invention. Assume that the target scene is divided into 4 areas, namely: area a, area b, area c and area d, and the corresponding generated laser grid maps are as follows: Figure 4As shown, they are laser grid map Ma', laser grid map Mb', laser grid map Mc' and laser grid map Md' respectively. Based on the first coordinate transformation relationship and the second coordinate transformation relationship, the laser grid map Ma', laser grid map Mb', laser grid map Mc' and laser grid map Md' are transformed, and the laser grid map of the target scene obtained by splicing is shown as follows Figure 5 As shown, Figure 5 A schematic diagram of a laser grid map of a target scene provided by an embodiment of the present invention. The splicing of the visual point cloud map and the laser grid map is similar and will not be repeated here.

[0079] In an embodiment of the present invention, when generating a map of a target scene, the target scene is first divided into different areas. Then, a camera and a single-line laser sensor configured on an edge-side mobile device collect visual images and laser point cloud data of each area to generate a visual point cloud map and a laser grid map corresponding to each area. Afterwards, the cloud server obtains multiple visual point cloud maps and multiple laser grid maps generated by the mobile device for multiple different areas in the target scene, and determines a first coordinate transformation relationship based on the visual point cloud images corresponding to the different areas, so as to convert the visual point cloud maps and laser grid maps corresponding to different coordinate systems into the same coordinate system, thereby obtaining multiple target visual point cloud images and multiple target laser grid maps. Then, based on the target laser grid maps corresponding to different areas in the same coordinate system, a second coordinate transformation relationship is determined to correct the coordinates corresponding to the multiple target visual point cloud images and the multiple target laser grid maps. Since visual point cloud maps and laser grid maps are generated based on different types of scene information, and based on the information complementarity between visual images and laser point cloud data, the coordinates of multiple visual point cloud maps and laser grid maps are transformed and corrected, which can better combine the visual point cloud maps and laser grid maps corresponding to different areas. Combined with the computing power advantage of cloud servers, accurate visual point cloud maps and laser grid maps of the target scene can be efficiently obtained, and mobile devices can use visual positioning and laser positioning at the same time to achieve better positioning effects.

[0080] The map generation device of one or more embodiments of the present invention will be described in detail below. Those skilled in the art will appreciate that these devices can be constructed using commercially available hardware components and configured according to the steps taught in this solution.

[0081] Figure 6 A structural diagram of a map generating device provided by an embodiment of the present invention is shown in FIG. Figure 6 As shown, the device is applied to a cloud server, which is in communication with a mobile device and includes: an acquisition module 11 and a processing module 12.

[0082] An acquisition module 11 is used to acquire multiple visual point cloud maps and multiple laser grid maps generated by the mobile device for multiple different areas in the target scene; wherein the visual point cloud map is generated by the mobile device based on visual images captured by a camera, and the laser grid map is generated by the mobile device based on laser point cloud data captured by a single-line laser radar, and the coordinate systems of the visual point cloud map and the laser grid map corresponding to the same area are the same.

[0083] The processing module 12 is used to determine a first coordinate transformation relationship between the multiple visual point cloud maps based on the visual images respectively corresponding to the multiple visual point cloud maps; based on the first coordinate transformation relationship, convert the multiple visual point cloud maps and the multiple laser grid maps into the same coordinate system to obtain multiple target visual point cloud images and multiple target laser grid maps corresponding to the same coordinate system; based on the laser point cloud data respectively corresponding to the multiple target laser grid maps, determine a second coordinate transformation relationship between the multiple target laser grid maps; based on the second coordinate transformation relationship, correct the coordinates corresponding to the multiple target visual point cloud images and the multiple target laser grid maps, so as to splice the corrected multiple target visual point cloud images and the multiple target laser grid maps to generate a visual point cloud map and a laser grid map of the target scene.

[0084] Optionally, the processing module 12 is specifically used to determine similar two-dimensional feature points between the multiple visual point cloud maps based on the two-dimensional feature points of the visual images corresponding to the multiple visual point cloud maps respectively; determine target visual images that match between the multiple visual point cloud maps based on the similar two-dimensional feature points; determine the posture transformation relationship between the multiple visual point cloud maps based on the posture information corresponding to the target visual image; and determine the first coordinate transformation relationship between the multiple visual point cloud maps based on the posture transformation relationship.

[0085] Optionally, the processing module 12 is further specifically used to determine that the first visual image and the second visual image are target visual images that match between the first visual point cloud map and the second visual point cloud map if the number of similar two-dimensional feature points between the first visual image corresponding to the first visual point cloud map and the second visual image corresponding to the second visual point cloud map is greater than or equal to a set threshold; wherein the first visual point cloud map and the second visual point cloud map are any two visual point cloud maps among the multiple visual point cloud maps.

[0086] Optionally, the processing module 12 is further specifically used to determine the initial posture transformation relationship corresponding to each pair of matching target visual images according to the posture information corresponding to each pair of matching target visual images if there is at least one pair of matching target visual images between the first visual point cloud map and the second visual point cloud map; and determine the posture transformation relationship between the first visual point cloud map and the second visual point cloud map according to at least one initial posture transformation relationship.

[0087] Optionally, the processing module 12 is further specifically used to generate an optimization function with the target pose transformation relationship as the quantity to be optimized, wherein the optimization function takes the deviation between the optimized target pose transformation relationship and at least one initial pose transformation relationship as the optimization target; and determines that the target pose transformation relationship that meets the optimization target is the pose transformation relationship between the first visual point cloud map and the second visual point cloud map.

[0088] Optionally, the processing module 12 is further specifically configured to determine, based on the nodes and submaps corresponding to the multiple target laser grid maps, an inter-region constraint relationship between each of the multiple target laser grid maps, the inter-region constraint relationship being used to describe the relative poses between the nodes and submaps corresponding to different target laser grid maps; wherein each target laser grid map corresponds to multiple submaps, each submap corresponds to multiple nodes, and each node corresponds to multiple frames of laser point cloud data; using the poses corresponding to the nodes and submaps of the multiple target laser grid maps as quantities to be optimized, and the intra-region constraint relationships and the inter-region constraint relationships corresponding to the multiple target laser grid maps as constraint quantities, determine the optimized poses corresponding to the nodes and submaps of the multiple target laser grid maps; wherein the intra-region constraint relationship is used to describe the relative poses between the nodes and submaps corresponding to the same target laser grid map; and determining the second coordinate transformation relationship between the multiple target laser grid maps based on the optimized poses.

[0089] Optionally, the processing module 12 is further specifically configured to establish a first inter-region constraint relationship between the first target laser grid map and the second target laser grid map if the laser point cloud data in the node corresponding to the first target laser grid map matches the submap corresponding to the second target laser grid map; wherein the first target laser grid map and the second target laser grid map are any two target laser grid maps among the multiple target laser grid maps; establish a second inter-region constraint relationship between the first target laser grid map and the second target laser grid map if the laser point cloud data in the node corresponding to the second target laser grid map matches the submap corresponding to the first target laser grid map; and determine the inter-region constraint relationship between the first target laser grid map and the second target laser grid map based on the first inter-region constraint relationship and the second inter-region constraint relationship.

[0090] Optionally, the mobile device comprises a robot.

[0091] Figure 6 The device shown can execute the steps in the aforementioned embodiments. For detailed execution process and technical effects, please refer to the description in the aforementioned embodiments and will not be repeated here.

[0092] In one possible design, the above Figure 6 The structure of the map generating device shown can be implemented as an electronic device. Figure 7 As shown, the electronic device may include: a memory 21, a processor 22, and a communication interface 23. The memory 21 stores executable code, and when the executable code is executed by the processor 22, the processor 22 can at least implement the map generation method provided in the above embodiments.

[0093] In addition, an embodiment of the present invention provides a non-transitory machine-readable storage medium, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor can at least implement the map generation method provided in the aforementioned embodiment.

[0094] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Those skilled in the art can understand and implement the present invention without inventive effort.

[0095] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by adding a necessary general hardware platform, and of course can also be implemented by a combination of hardware and software. Based on this understanding, the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a computer product. The present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0096] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A map generation method, characterized in that: Applied to a cloud server, the cloud server being communicatively connected to a mobile device, the method includes: Obtaining a plurality of visual point cloud maps and a plurality of laser grid maps generated by the mobile device for a plurality of different areas in a target scene; wherein the visual point cloud map is generated by the mobile device based on visual images captured by a camera, and the laser grid map is generated by the mobile device based on laser point cloud data captured by a single-line laser radar, and the coordinate system of the visual point cloud map and the laser grid map corresponding to the same area is the same; determining a first coordinate transformation relationship between the plurality of visual point cloud maps according to the visual images respectively corresponding to the plurality of visual point cloud maps; According to the first coordinate transformation relationship, the multiple visual point cloud maps and the multiple laser grid maps are converted into the same coordinate system to obtain multiple target visual point cloud images and multiple target laser grid maps corresponding to the same coordinate system; Determining a second coordinate transformation relationship between the multiple target laser grid maps according to the laser point cloud data respectively corresponding to the multiple target laser grid maps; According to the second coordinate transformation relationship, the coordinates corresponding to the multiple target visual point cloud images and the multiple target laser grid maps are corrected to splice the multiple corrected target visual point cloud images to generate a visual point cloud map of the target scene, and the multiple corrected target laser grid maps are spliced to generate a laser grid map of the target scene.

2. The method according to claim 1, characterized in that The determining, based on the visual images respectively corresponding to the multiple visual point cloud maps, a first coordinate transformation relationship between the multiple visual point cloud maps includes: Determining similar two-dimensional feature points between two of the multiple visual point cloud maps based on the two-dimensional feature points of the visual image corresponding to each of the multiple visual point cloud maps; Determining target visual images that match between any two of the plurality of visual point cloud maps based on the similar two-dimensional feature points; Determining a pose transformation relationship between each of the plurality of visual point cloud maps according to pose information corresponding to the target visual image; According to the posture transformation relationship, a first coordinate transformation relationship between the multiple visual point cloud maps is determined.

3. The method according to claim 2, characterized in that The step of determining target visual images that match between two of the plurality of visual point cloud maps based on the similar two-dimensional feature points includes: If the number of similar two-dimensional feature points between the first visual image corresponding to the first visual point cloud map and the second visual image corresponding to the second visual point cloud map is greater than or equal to a set threshold, the first visual image and the second visual image are determined to be target visual images that match between the first visual point cloud map and the second visual point cloud map; wherein the first visual point cloud map and the second visual point cloud map are any two visual point cloud maps among the multiple visual point cloud maps.

4. The method according to claim 3, characterized in that The determining of the pose transformation relationship between each of the plurality of visual point cloud maps according to the pose information corresponding to the target visual image includes: If there is at least one pair of matching target visual images between the first visual point cloud map and the second visual point cloud map, determining an initial pose transformation relationship corresponding to each pair of matching target visual images according to the pose information corresponding to each pair of matching target visual images; According to at least one initial pose transformation relationship, a pose transformation relationship between the first visual point cloud map and the second visual point cloud map is determined.

5. The method according to claim 4, characterized in that The determining, based on at least one initial pose transformation relationship, a pose transformation relationship between the first visual point cloud map and the second visual point cloud map includes: An optimization function is generated with the target posture transformation relationship as the quantity to be optimized, wherein the optimization function takes the deviation between the optimized target posture transformation relationship and at least one initial posture transformation relationship as the optimization target; Determine a target pose transformation relationship that meets the optimization goal as a pose transformation relationship between the first visual point cloud map and the second visual point cloud map.

6. The method according to claim 1, characterized in that The determining of a second coordinate transformation relationship between the plurality of target laser grid maps according to the laser point cloud data respectively corresponding to the plurality of target laser grid maps includes: Determining, based on the nodes and submaps corresponding to the multiple target laser grid maps, inter-region constraint relationships between each of the multiple target laser grid maps, wherein the inter-region constraint relationships are used to describe the relative positions between the nodes and submaps corresponding to different target laser grid maps; wherein each target laser grid map corresponds to multiple submaps, each submap corresponds to multiple nodes, and each node corresponds to multiple frames of laser point cloud data; The optimized poses corresponding to the nodes and submaps of the multiple target laser grid maps are determined by using the poses corresponding to the nodes and submaps of the multiple target laser grid maps as quantities to be optimized, and the intra-region constraint relationships and the inter-region constraint relationships corresponding to the multiple target laser grid maps as constraint relationships; wherein the intra-region constraint relationships are used to describe the relative poses between the nodes and submaps corresponding to the same target laser grid map; According to the optimized position and posture, a second coordinate transformation relationship between the multiple target laser grid maps is determined.

7. The method according to claim 6, characterized in that The determining of the inter-region constraint relationship between each of the plurality of target laser grid maps according to the nodes and sub-maps respectively corresponding to the plurality of target laser grid maps includes: If the laser point cloud data in the node corresponding to the first target laser grid map matches the submap corresponding to the second target laser grid map, a first inter-region constraint relationship is established between the first target laser grid map and the second target laser grid map; wherein the first target laser grid map and the second target laser grid map are any two target laser grid maps among the multiple target laser grid maps; If the laser point cloud data in the node corresponding to the second target laser grid map matches the submap corresponding to the first target laser grid map, then establishing a second inter-region constraint relationship between the first target laser grid map and the second target laser grid map; An inter-region constraint relationship between the first target laser grid map and the second target laser grid map is determined according to the first inter-region constraint relationship and the second inter-region constraint relationship.

8. The method according to any one of claims 1 to 7, characterized in that The mobile device includes a robot.

9. A map generating device, characterized in that: Applied to a cloud server, the cloud server is communicatively connected to a mobile device, and the device includes: an acquisition module, configured to acquire a plurality of visual point cloud maps and a plurality of laser grid maps generated by the mobile device for a plurality of different areas in a target scene; wherein the visual point cloud map is generated by the mobile device based on visual images captured by a camera, and the laser grid map is generated by the mobile device based on laser point cloud data captured by a single-line laser radar, and the coordinate system of the visual point cloud map and the laser grid map corresponding to the same area is the same; A processing module is used to determine a first coordinate transformation relationship between the multiple visual point cloud maps based on the visual images respectively corresponding to the multiple visual point cloud maps; according to the first coordinate transformation relationship, convert the multiple visual point cloud maps and the multiple laser grid maps into the same coordinate system to obtain multiple target visual point cloud images and multiple target laser grid maps corresponding to the same coordinate system; according to the laser point cloud data respectively corresponding to the multiple target laser grid maps, determine a second coordinate transformation relationship between the multiple target laser grid maps; according to the second coordinate transformation relationship, correct the coordinates corresponding to the multiple target visual point cloud images and the multiple target laser grid maps, so as to splice the corrected multiple target visual point cloud images to generate a visual point cloud map of the target scene, and splice the corrected multiple target laser grid maps to generate a laser grid map of the target scene.

10. An electronic device, characterized in that: include: A memory, a processor, and a communication interface; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor executes the map generation method according to any one of claims 1 to 8.

11. A non-transitory machine-readable storage medium, characterized in that The non-transitory machine-readable storage medium stores executable code, and when the executable code is executed by a processor of an electronic device, the processor is caused to execute the map generation method according to any one of claims 1 to 8.

Images