A Map Alignment Method, Device, Electronic Device, and Storage Medium

Abstract

An embodiment of the present application provides a map alignment method, apparatus, electronic device, and storage medium. The method includes: obtaining an original map, where the original map is a QR code map or a map constructed based on first environmental features; for each point to be collected, obtaining mapping data collected by a mobile collection device at the point to be collected, and obtaining a first pose of the mobile collection device when it is located at the point to be collected; for the mapping data corresponding to each first pose, matching the second environmental features of the mapping data with the second environmental features of adjacent mapping data, and taking the feature points with successful matching as the second environmental features corresponding to the first pose; according to each first pose and the second environmental features corresponding to each first pose, generating a map for recording the second environmental features corresponding to each first pose as the target map of the area to be collected. By applying the technical solution provided by the embodiment of the present application, the constructed visual map can be aligned with the old map.

Description

Technical Field

[0001] This application relates to the technical field of navigation and positioning, and particularly to a map alignment method, apparatus, electronic device, and storage medium. Background Art

[0002] Before the full development of visual navigation technology, for most scenarios, robots mainly achieved navigation based on QR code maps, laser maps, and texture maps (subsequently collectively referred to as old maps). However, with the development of visual navigation technology, its advantage of low cost has attracted more and more attention. Subsequently, more and more robots have been set to navigate based on vision. Therefore, after introducing robots that navigate based on vision (subsequently referred to as new robots) into the scenario, it is necessary to construct a visual map of the scenario for the new robots to move in the scenario. However, since there are still robots that move based on old maps (subsequently referred to as old robots) in the scenario, in order to make the positioning of new robots and old robots consistent in the same scenario, it is also necessary to align the visual map with the old map when constructing the visual map. Summary of the Invention

[0003] The purpose of the embodiments of this application is to provide a map alignment method, apparatus, electronic device, and storage medium to align the constructed visual map with the old map. The specific technical solutions are as follows:

[0004] In a first aspect, the embodiments of this application provide a map alignment method, and the method includes:

[0005] Obtain the original map of the area to be collected, where the original map is a QR code map or a map constructed based on the first environmental features;

[0006] According to the original map, control the mobile acquisition device to move to each point to be collected in the area to be collected;

[0007] For each point to be collected, obtain the mapping data collected by the mobile acquisition device at the point to be collected, and obtain the first pose of the mobile acquisition device when it is at the point to be collected, and use the obtained mapping data as the mapping data corresponding to the obtained first pose;

[0008] For the mapping data corresponding to each first pose, match the second environmental features of the mapping data with the second environmental features of adjacent mapping data, and use the successfully matched feature points as the second environmental features corresponding to the first pose, where the types of the second environmental features and the first environmental features are different;

[0009] Generate a map for recording the second environmental features corresponding to each of the first poses, as the target map of the area to be collected, according to each of the first poses and the second environmental features corresponding to each of the first poses.

[0010] In some embodiments, obtaining the mapping data collected by the mobile collection device at the point to be collected, and obtaining the first pose of the mobile collection device when it is located at the point to be collected, and using the obtained mapping data as the mapping data corresponding to the obtained first pose, includes:

[0011] Obtain each piece of mapping data, first pose, the timestamp of each piece of mapping data, and the timestamp of each first pose sent by the mobile collection device, where the mobile collection device is configured to collect mapping data and obtain its own first pose, the timestamp of the mapping data, and the timestamp of the first pose each time it reaches the point to be collected; and is also configured to send the collected mapping data, the obtained first pose, the timestamp of the mapping data, and the timestamp of the first pose.

[0012] For each of the first poses, determine the mapping data whose timestamp matches the timestamp of the first pose as the mapping data corresponding to the first pose.

[0013] In some embodiments, the original map is a QR code map, and the second environmental feature is a laser feature, or a texture feature, or a visual feature;

[0014] The first environmental feature is a laser feature, and the second environmental feature is a texture feature, or a visual feature;

[0015] The first environmental feature is a texture feature, and the second environmental feature is a laser feature, or a visual feature.

[0016] In some embodiments, the original map is a QR code map, and the mobile collection device is equipped with an odometer.

[0017] The method further includes: for each of the points to be collected, obtaining the odometer data of the odometer when the mobile collection device is located at the point to be collected.

[0018] The generating a map for recording the second environmental features corresponding to each of the first poses, as the target map of the area to be collected, according to each of the first poses and the second environmental features corresponding to each of the first poses, includes:

[0019] Generate a map for recording the second environmental features corresponding to each of the first poses, as the map to be optimized, according to each of the first poses and the second environmental features corresponding to each of the first poses.

[0020] Based on the odometer data corresponding to each of the first poses and the mapping data corresponding to each of the first poses, optimize the map to be optimized based on odometer constraints and visual observation constraints to obtain the target map.

[0021] In some embodiments, optimizing the map to be optimized and using the optimized map to be optimized as the target map of the area to be collected includes:

[0022] Optimize the position and orientation in the second pose involved in the map to be optimized, and optimize the orientation in the third pose involved in the map to be optimized, and use the optimized map to be optimized as the target map of the area to be collected, where the second pose is the pose calculated by the mobile collection device based on the odometer, and the third pose is the pose obtained by the mobile collection device scanning the QR code.

[0023] In some embodiments, each of the first poses, the odometer data corresponding to each of the first poses, the mapping data corresponding to each of the first poses, and the target map are stored correspondingly, and the method further includes:

[0024] In response to a deletion instruction for the target map, read each of the first poses stored correspondingly and the mapping data corresponding to each of the first poses; from each of the read first poses and the mapping data corresponding to each of the first poses, delete the first pose and the mapping data indicated by the deletion instruction to obtain a target deleted map; and / or,

[0025] In response to an addition instruction for the target map, read each of the first poses stored correspondingly, the odometer data corresponding to each of the first poses, and the mapping data corresponding to each of the first poses, and obtain the pose to be added indicated by the addition instruction and the mapping data to be added corresponding to each of the poses to be added; add the pose to be added indicated by the addition instruction and the mapping data to be added corresponding to each of the poses to be added to each of the read first poses and the mapping data corresponding to each of the first poses to obtain an initial amplified map; based on the odometer constraints and visual constraints, optimize the area outside the first area in the initial amplified map according to each of the read first poses, the odometer data corresponding to each of the first poses, the mapping data corresponding to each of the first poses, and the obtained pose to be added and the mapping data to be added corresponding to each of the poses to be added; and, based on the odometer constraints, visual constraints, and co-visibility constraints, optimize the first area in the initial amplified map to obtain a target amplified map; where the first area is the area where the mapping data to be added coincides with the mapping data corresponding to each of the first poses.

[0026] In a second aspect, an embodiment of the present application further provides a map alignment device, and the device includes:

[0027] A map acquisition module, configured to acquire an original map of an area to be collected, where the original map is a QR code map or a map constructed based on first environmental features;

[0028] A motion control module, configured to control a mobile acquisition device to move to each point to be collected in the area to be collected according to the original map;

[0029] A data acquisition module, configured to, for each point to be collected, acquire mapping data collected by the mobile acquisition device at the point to be collected, and acquire a first pose of the mobile acquisition device when it is at the point to be collected, and use the acquired mapping data as the mapping data corresponding to the acquired first pose;

[0030] A feature matching module, configured to, for the mapping data corresponding to each first pose, match the second environmental features of the mapping data with the second environmental features of adjacent mapping data, and use the successfully matched feature points as the second environmental features corresponding to the first pose, where the types of the second environmental features are different from those of the first environmental features;

[0031] A map generation module, configured to generate a map for recording the second environmental features corresponding to each first pose according to each first pose and the second environmental features corresponding to each first pose, as the target map of the area to be collected.

[0032] In some embodiments, the data acquisition module is specifically configured to:

[0033] Acquire each piece of mapping data, first pose, time stamp of each piece of mapping data, and time stamp of each first pose sent by the mobile acquisition device, where the mobile acquisition device is configured to, every time it reaches a point to be collected, collect mapping data and acquire its own first pose, time stamp of the mapping data, and time stamp of the first pose; and is further configured to send the collected mapping data, acquired first pose, time stamp of the mapping data, and time stamp of the first pose;

[0034] For each first pose, determine the mapping data whose time stamp matches the time stamp of the first pose as the mapping data corresponding to the first pose.

[0035] In some embodiments, the original map is a QR code map, and the second environmental features are laser features, or texture features, or visual features;

[0036] The first environmental features are laser features, and the second environmental features are texture features, or visual features;

[0037] The first environmental feature is a texture feature, and the second environmental feature is a laser feature or a visual feature.

[0038] In some embodiments, the original map is a QR code map, and the mobile acquisition device is equipped with an odometer.

[0039] The data acquisition module is further configured to:

[0040] For each of the points to be acquired, obtain the odometer data of the odometer when the mobile acquisition device is located at the point to be acquired.

[0041] The map generation module is specifically configured to:

[0042] Generate a map for recording the second environmental features corresponding to each of the first poses based on each of the first poses and the second environmental features corresponding to each of the first poses, as the map to be optimized.

[0043] Optimize the map to be optimized based on the odometer data corresponding to each of the first poses and the mapping data corresponding to each of the first poses, and based on the odometer constraint and the visual observation constraint, to obtain the target map.

[0044] In some embodiments, optimizing the map to be optimized to obtain the target map includes:

[0045] Optimize the position and orientation in the second pose involved in the map to be optimized, and optimize the orientation in the third pose involved in the map to be optimized, and use the optimized map to be optimized as the target map of the area to be acquired, where the second pose is the pose calculated by the mobile acquisition device based on the odometer, and the third pose is the pose obtained by the mobile acquisition device scanning the QR code.

[0046] In some embodiments, each of the first poses, the odometer data corresponding to each of the first poses, the mapping data corresponding to each of the first poses, and the target map are stored correspondingly, and the device further includes:

[0047] A map deletion module, configured to, in response to a deletion instruction for the target map, read each of the first poses and the mapping data corresponding to each of the first poses stored correspondingly; delete the first pose and the mapping data indicated by the deletion instruction from the read first poses and the mapping data corresponding to each of the first poses, to obtain a target deleted map; and / or

[0048] A map augmentation module, configured to, in response to an addition instruction for the target map, read each of the stored first poses, the odometry data corresponding to each of the first poses, and the mapping data corresponding to each of the first poses, and obtain the poses to be added indicated by the addition instruction, and the mapping data to be added corresponding to each of the poses to be added; add the poses to be added indicated by the addition instruction, and the mapping data to be added corresponding to each of the poses to be added, to each of the read first poses and the mapping data corresponding to each of the first poses, to obtain an initial augmented map; optimize, based on odometry constraints and visual constraints, the area outside the first area in the initial augmented map according to each of the read first poses, the odometry data corresponding to each of the first poses, the mapping data corresponding to each of the first poses, and the obtained poses to be added and the mapping data to be added corresponding to each of the poses to be added; and optimize the first area in the initial augmented map based on odometry constraints, visual constraints, and co-visibility constraints, to obtain a target augmented map; wherein the first area is the area where the mapping data to be added overlaps with the mapping data corresponding to each of the first poses.

[0049] In a third aspect, an embodiment of the present application provides an electronic device, including:

[0050] A memory, configured to store a computer program;

[0051] A processor, configured to implement any one of the above map alignment methods when executing the program stored on the memory.

[0052] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, any one of the above map alignment methods is implemented.

[0053] In a fifth aspect, an embodiment of the present application further provides a computer program product containing instructions, which when running on a computer, causes the computer to execute any one of the above map alignment methods.

[0054] Advantageous effects of the embodiments of the present application:

[0055] In the technical solution provided by the embodiments of the present application, the original map is a two-dimensional code map or a map constructed based on the first environmental features. After obtaining the original map of the area to be collected, according to the original map, the mobile collection device is controlled to move to each point to be collected in the area to be collected. For each point to be collected, the mapping data collected by the mobile collection device at the point to be collected is obtained, and the first pose of the mobile collection device when it is located at the point to be collected is obtained. The obtained mapping data is used as the mapping data corresponding to the obtained first pose. Furthermore, for the mapping data corresponding to each first pose, the second environmental features of the mapping data are matched with the second environmental features of the adjacent mapping data, and the feature points with successful matching are used as the second environmental features corresponding to the first pose. Thus, according to each first pose and the second environmental features corresponding to each first pose, a map for recording the second environmental features corresponding to each first pose can be generated as the target map of the area to be collected.

[0056] It can be understood that in the embodiments of the present application, the target map is constructed based on the first pose and the second environmental features corresponding to the first pose. The first pose is the pose of the mobile collection device itself obtained when it is located at each point to be collected. The second environmental features corresponding to the first pose are obtained by feature matching between the second environmental features of the mapping data corresponding to the first pose and the second environmental features of the adjacent mapping data of the mapping data. Moreover, the mobile collection device moves to each point to be collected according to the original map. Therefore, the target map determined based on the first pose and the second environmental features corresponding to the first pose is a map aligned with the original map. Thus, when the target map to be constructed is a visual map, the visual map aligned with the original map can be directly constructed according to the technical solution provided by the embodiments of the present application.

[0057] Of course, when implementing any product or method of the present application, it is not necessarily required to achieve all the above-mentioned advantages simultaneously. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present application, and those of ordinary skill in the art can also obtain other embodiments according to these drawings.

[0059] Figure 1 It is the first flow schematic diagram of the map alignment method provided by the embodiments of the present application;

[0060] Figure 2 It is a detailed schematic diagram of the above step S13;

[0061] Figure 3A schematic diagram for navigation based on a QR code map provided by an embodiment of the present application;

[0062] Figure 4 The second process schematic diagram of the map alignment method provided by an embodiment of the present application;

[0063] Figure 5 The third process schematic diagram of the map alignment method provided by an embodiment of the present application;

[0064] Figure 6 The fourth process schematic diagram of the map alignment method provided by an embodiment of the present application;

[0065] Figure 7 The fifth process schematic diagram of the map alignment method provided by an embodiment of the present application;

[0066] Figure 8 The sixth process schematic diagram of the map alignment method provided by an embodiment of the present application;

[0067] Figure 9 A structural schematic diagram of a map alignment device provided by an embodiment of the present application;

[0068] Figure 10 A structural schematic diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0069] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art based on the present application belong to the scope of protection of the present application.

[0070] Before the visual navigation technology was fully developed, for most scenarios, robots mainly achieved navigation based on QR code maps, laser maps, and texture maps (collectively referred to as old maps) later. However, with the development of visual navigation technology, its advantage of low cost has attracted more and more attention. Subsequently, more and more robots began to be set to navigate based on vision. Therefore, after introducing robots that navigate based on vision (subsequently referred to as new robots) into the scenario, a visual map of the scenario needs to be constructed for the new robots to move in the scenario. However, since there are still robots that move based on old maps (subsequently referred to as old robots) in the scenario, in order to make the positioning of the new robots and the old robots consistent in the same scenario, the visual map and the old map need to be aligned when constructing the visual map.

[0071] In the related art, there is a method for aligning a 3D map of a robot. In this method, it is necessary to first obtain a two-dimensional (2D) laser map of the robot's operating environment and a three-dimensional (3D) laser map to be aligned with the 2D laser map. Then, through calculation steps such as pre-alignment, horizontal restoration, point cloud registration, rotation and translation, the 3D laser map is aligned with the 2D laser map. Based on this method, although the alignment of the 2D laser map and the 3D laser map can be achieved, it has the following defects:

[0072] (1) This method is for aligning a 3D laser map with an existing 2D laser map and is not applicable to aligning a QR code map with a visual map;

[0073] (2) This method requires obtaining a 3D laser map in advance and then aligning the 3D laser map with an existing 2D laser map, and it is impossible to directly obtain a 3D laser map aligned with the 2D laser map, and the implementation is relatively complex;

[0074] (3) This method requires calculation steps such as pre-alignment, horizontal restoration, point cloud registration, rotation and translation to align the 3D laser map with an existing 2D laser map, and the calculation process is relatively complex.

[0075] In view of the above problems, the present application provides a map alignment method. Based on the technical solution provided by the embodiments of the present application, a target map aligned with the original map can be directly constructed, and moreover, the technical solution provided by the present application does not require complex alignment calculations, translation and rotation transformation steps, etc., and the implementation is relatively simple, and the calculation process is also relatively simple.

[0076] The map alignment method provided by the present application is applied to an electronic device, and the electronic device can be a desktop computer, a portable computer, a smart mobile terminal, etc. It can be understood that the map alignment method provided by the present application can be understood as a map construction method for the purpose of alignment.

[0077] Next, the map alignment method provided by the embodiments of the present application will be described in detail through specific examples:

[0078] See Figure 1 , Figure 1 which is the first flow schematic diagram of the map alignment method provided by the embodiments of the present application, and includes the following steps:

[0079] Step S11: Obtain the original map of the area to be collected;

[0080] Among them, the original map is a QR code map or a map constructed based on the first environmental features;

[0081] Step S12: According to the original map, control the mobile acquisition device to move to each point to be acquired in the area to be acquired;

[0082] Step S13: For each point to be acquired, obtain the mapping data collected by the mobile acquisition device at the point to be acquired, and obtain the first pose of the mobile acquisition device when it is located at the point to be acquired. Take the obtained mapping data as the mapping data corresponding to the obtained first pose;

[0083] Step S14: For the mapping data corresponding to each first pose, match the second environmental feature of the mapping data with the second environmental feature of the adjacent mapping data, and take the successfully matched feature points as the second environmental feature corresponding to the first pose;

[0084] Among them, the types of the second environmental feature and the first environmental feature are different;

[0085] Step S15: According to each first pose and the map of the second environmental feature corresponding to each first pose, use it as the target map of the area to be acquired.

[0086] In the technical solution provided by the embodiment of the present application, the original map is a QR code map or a map constructed based on the first environmental feature. After obtaining the original map of the area to be acquired, according to the original map, control the mobile acquisition device to move to each point to be acquired in the area to be acquired. For each point to be acquired, obtain the mapping data collected by the mobile acquisition device at the point to be acquired, and obtain the first pose of the mobile acquisition device when it is located at the point to be acquired. Take the obtained mapping data as the mapping data corresponding to the obtained first pose; furthermore, for the mapping data corresponding to each first pose, match the second environmental feature of the mapping data with the second environmental feature of the adjacent mapping data, and take the successfully matched feature points as the second environmental feature corresponding to the first pose; thus, according to each first pose and the second environmental feature corresponding to each first pose, a map for recording the second environmental feature corresponding to each first pose can be generated, which is used as the target map of the area to be acquired.

[0087] It can be understood that in the embodiments of the present application, the target map is constructed based on the first pose and the second environmental feature corresponding to the first pose. The first pose is the pose of the mobile acquisition device obtained when it is located at each point to be acquired, and the second environmental feature corresponding to the first pose is obtained by feature matching of the second environmental feature of the mapping data corresponding to the first pose and the second environmental feature of the adjacent mapping data of the mapping data. Moreover, the mobile acquisition device moves to each point to be acquired according to the original map. Therefore, the target map determined based on the first pose and the second environmental feature corresponding to the first pose is a map aligned with the original map. And the original map is a QR code map or a map constructed based on the first environmental feature, and the type of the second environmental feature is different from that of the first environmental feature.

[0088] It can be seen that when applying the technical solution provided by the embodiments of the present application, when the original map is a QR code map or a map constructed based on the first environmental feature, the target map aligned with the original map can be directly constructed; moreover, the technical solution provided by the embodiments of the present application does not require complex alignment calculations, translations, rotations, transformations and other steps, and the implementation is relatively simple, and the calculation process is also relatively simple. Thus, when the target map to be constructed is a visual map, the visual map aligned with the original map can be directly constructed according to the technical solution provided by the embodiments of the present application, and the implementation is relatively simple, and the calculation process is also relatively simple.

[0089] In the above step S11, the area to be acquired is the area where the target map needs to be constructed; the first environmental feature can be various types of features such as a laser feature and a texture feature. Among them, the laser feature can also be divided into a two-dimensional laser feature and a three-dimensional laser feature. Taking the laser feature and the texture feature as examples, when the first environmental feature is a laser feature, the map constructed based on the first environmental feature can be called a laser map; when the first environmental feature is a downward-looking texture feature, the map constructed based on the first environmental feature can be called a texture map.

[0090] It can be understood that the map constructed based on the first environmental feature can actually be understood as a map for realizing navigation based on the first environmental feature. For example, for the laser map composed of laser features, the principle of its navigation is that a lidar is installed on the robot. When the robot moves to a certain pose, the point cloud data of the surrounding environment is collected by the lidar. Then, according to the collected point cloud data, the pose where the robot is currently located is determined, and in this way, the autonomous navigation and positioning of the robot are realized; for another example, for the texture map constructed based on the texture feature, the principle of its navigation is that a visual sensor is installed on the robot. When the robot moves to a certain pose, the image texture information around is obtained through the visual sensor. Then, according to the obtained image texture information, the pose where the robot is currently located is determined, and in this way, the autonomous navigation and positioning of the robot are realized.

[0091] Here, the navigation principle of the map constructed based on other first environmental features is the same as that of the laser map and the texture map. The only difference lies in the feature data used for positioning the robot. Specifically, the feature data used for positioning the robot is actually the first environmental features used for constructing the map. For example, the feature data used for positioning the robot in the laser map is the point cloud data collected based on the laser emitted by the laser sensor, and the feature data used for positioning the robot in the texture map is the image texture information. Therefore, the navigation principle of the map constructed based on other first environmental features will not be elaborated here. In addition, for the principle of autonomous navigation and positioning based on the QR code map, reference can be made to the description in the following Figure 3 and will not be expanded here.

[0092] In this application, the reason for obtaining the original map is that the target map to be constructed needs to be aligned with the original map, that is, the coordinate of the target map to be constructed needs to be aligned with the original map to ensure that the positioning results of the robot in the target map and the original map are consistent. Therefore, in this application, the original map (which can also be called the old map) needs to be obtained first, and then the mobile acquisition device is controlled to move to each point to be collected according to the original map (see step S12 below), and the first pose and the second environmental feature corresponding to the first pose are collected at each point to be collected (see steps S13 - S14 below). Furthermore, according to each first pose and the second environmental feature corresponding to each first pose, the target map is obtained (see step S25 below). In this way, in the constructed target map, the second environmental feature corresponding to the robot when it is in a certain pose is actually the second environmental feature corresponding to the robot when it is in this pose in the original map.

[0093] It can be seen that in this application, the role of the original map is actually to control the mobile acquisition device to move to each point to be collected. Therefore, in this application, as long as the original map can be used to realize the navigation and positioning of the mobile acquisition device in the area to be collected and is the map that the target map needs to be aligned with, the specific type of the original map is not important. That is, it is not important which specific type of feature the first environmental feature used for constructing the original map is. Therefore, in this application, no specific limitation is made on the first environmental feature. The original map can be any type of map. However, since there is no need to construct a map identical to the original map (for example, there is no problem of constructing a 2D laser map based on a 2D laser map), in this application, the types of the first environmental feature and the second environmental feature are different. In this article, for the convenience of description and understanding, the examples in the following content mainly take the original map as the QR code map, the laser map, and the texture map; and the target map as the visual map for illustration.

[0094] In the above step S12, the mobile acquisition device can be a device that can autonomously navigate and position according to the original map, or a device that can move to the position specified by the instruction issued by the electronic device in response to the instruction; it can be understood that in order to collect the subsequent mapping data and obtain its own pose, the mobile acquisition device is equipped with a device that can be used to collect mapping data and a device that can be used to determine its own pose. For specific content, reference can be made to the embodiments shown after the following step S15.

[0095] The points to be collected are points set by the user according to the actual situation. In some embodiments, the user can directly specify which positions in the area to be collected are the points to be collected. Then, according to the original map, the mobile acquisition device can move to these points to be collected one by one; or, the user can also plan the movement path of the mobile acquisition device in the area to be collected and preset a preset time in advance. Then, according to the original map, the mobile acquisition device moves along the pre-planned movement path. Every time the preset time is reached, the position where the mobile acquisition device is located is used as a point to be collected. For example, the pre-planned path is to move from position A to position B, and the preset time is 2 seconds (s). The mobile acquisition device starts to move along the pre-planned path from position A. After moving for 2 s, it reaches position C. Then position C is a point to be collected; then, continue to move along the pre-planned path from position C. After moving for 2 s, it reaches position D. Then position D is a point to be collected, and so on, until it moves to position B.

[0096] It can be understood that the purpose of setting the points to be collected in this application is to generate a target map based on the first pose of the mobile acquisition device at the point to be collected and the mapping data corresponding to the first pose. Therefore, in order to improve the accuracy and precision of the target map, the number of points to be collected can be designed to be relatively large and distributed as evenly as possible in the area to be collected.

[0097] In the above step S13, the mapping data is the data for providing the second environmental feature obtained by the mobile acquisition device at the point to be collected, and the second environmental feature is the feature for constructing the target map. That is to say, the specific content of the mapping data collected by the mobile acquisition device depends on the type of the target map to be constructed. The first pose is the pose of the mobile acquisition device itself obtained when it is located at the point to be collected. Specifically, in a specific example, when the original map is a texture map, the mobile acquisition device can determine the pose where it is located, that is, the first pose, according to the image texture information collected by its own visual sensor.

[0098] In the embodiments of the present application, for each point to be collected, the mobile collection device collects mapping data, obtains its own first pose, and uses the obtained mapping data as the mapping data corresponding to the first pose. For example, when the mobile collection device is located at the point to be collected A, the collected mapping data is Data 1, and the obtained first pose is Pose 1, then the mapping data 1 is the mapping data corresponding to Pose 1; when the mobile collection device is located at the point to be collected B, the collected mapping data is Data 2, and the obtained first pose is Pose 2, then the mapping data 2 is the mapping data corresponding to Pose 2; where Data 1 and Data 2 are both mapping data; Pose 1 and Pose 2 are both first poses.

[0099] Among them, for the specific method of obtaining the mapping data corresponding to the first pose, reference can be made to the description in the following Figure 2 below.

[0100] In the above step S14, the electronic device can extract the features of each mapping data as the second environmental features of each mapping data; thus, each first pose corresponds to a mapping data, and each mapping data corresponds to a second environmental feature. Based on this, for the mapping data corresponding to each first pose, the second environmental feature of the mapping data is matched with the second environmental features of adjacent mapping data, and then the feature points with successful matching are used as the second environmental features corresponding to the first pose. Among them, the types of the first environmental features and the second environmental features are different.

[0101] In the above step S15, according to each first pose and the second environmental features corresponding to each first pose, the electronic device can generate a map for recording the second environmental features corresponding to each first pose as the target map of the area to be collected.

[0102] In some embodiments, when the original map in the above step S11 is a QR code map, the second environmental features can be laser features, texture features, or visual features; when the first environmental feature in the above step S11 is a laser feature, the second environmental features can be texture features or visual features, or, in some embodiments, when the first environmental feature is specifically a two-dimensional laser feature, the second environmental features can also be three-dimensional laser features; when the first environmental feature in the above step S11 is a texture feature, the second environmental features can be laser features or visual features.

[0103] Among them, when the second environmental feature is a laser feature, the type of the target map in the above step S15 is a laser map, the device for collecting mapping data carried by the mobile acquisition device in the above step S13 can be a laser sensor, and the collected mapping data can be point cloud data; when the second environmental feature is a texture feature, the type of the target map in the above step S15 is a texture map, the device for collecting mapping data carried by the mobile acquisition device in the above step S13 can be a vision sensor, and the collected mapping data can be image texture data; when the second environmental feature is a vision feature, the type of the target map in the above step S15 is a vision map (i.e., a three-dimensional map that can represent environmental features and poses), the device for collecting mapping data carried by the mobile acquisition device in the above step S13 can be a vision sensor, and the collected mapping data can be image frames. Moreover, in the above step S14, the extraction of the second environmental feature of the mapping data (such as image frames) can be achieved based on the traditional Scale Invariant Feature Transform (SIFT) algorithm, or the extraction of the second environmental feature of the mapping data (such as image frames) can also be achieved based on the deep learning algorithm SuperPoint (i.e., superpoint feature).

[0104] In the above step S15, the specific way to extract the second environmental feature of the mapping data can be determined by the user according to actual needs, and this application does not make any limitations.

[0105] See Figure 2 , Figure 2 which is a refined schematic diagram of the above step S13 and can include the following steps:

[0106] Step S21: Obtain each piece of mapping data, the first pose, the timestamp of each piece of mapping data, and the timestamp of each first pose sent by the mobile acquisition device;

[0107] Among them, the mobile acquisition device is used to collect mapping data and obtain its own first pose, the timestamp of the mapping data, and the timestamp of the first pose every time it reaches a point to be collected; it is also used to send the collected mapping data, the obtained first pose, the timestamp of the mapping data, and the timestamp of the first pose;

[0108] Step S22: For each first pose, determine the mapping data whose timestamp matches the timestamp of the first pose as the mapping data corresponding to the first pose.

[0109] In the technical solution provided by the embodiments of the present application, the electronic device obtains each mapping data, the first pose, the timestamps of each mapping data, and the timestamps of each first pose from the mobile acquisition device. Thus, based on the timestamps of each mapping data and the timestamps of each first pose, the mapping data with the timestamp matching the timestamp of the first pose can be determined, and then the mapping data corresponding to each first pose can be determined. This provides a specific implementation manner for determining the mapping data corresponding to the first pose in step S13 above.

[0110] In the above step S21, when each point to be acquired is reached, the mobile acquisition device acquires mapping data, and obtains its own first pose, the timestamp of the mapping data, and the timestamp of the first pose. Moreover, the acquired mapping data, the obtained first pose, the timestamp of the mapping data, and the timestamp of the first pose are also sent. <##

[0111] Among them, the mobile acquisition device can send these data to the electronic device every time it obtains the mapping data, the first pose, and the timestamps of the mapping data and the first pose of a point to be acquired, or it can also send all the data to the electronic device together after acquiring the mapping data, the first pose, and the timestamps of the mapping data and the first pose of all points to be acquired. In addition, in addition to the mobile acquisition device actively sending the mapping data, the first pose, and the timestamps of the mapping data and the first pose to the electronic device, the electronic device can also actively obtain the mapping data, the first pose, and the timestamps of the mapping data and the first pose from the mobile acquisition device.

[0112] In the above step S22, for each first pose, the electronic device determines the mapping data with the timestamp matching the timestamp of the first pose as the mapping data corresponding to the first pose.

[0113] Specifically, in some embodiments, the electronic device can determine the mapping data corresponding to each first pose according to an interpolation method. Among them, the interpolation method adopted can specifically be a linear interpolation method, a Lagrange interpolation method, a Newton interpolation method, etc.

[0114] Alternatively, in some other embodiments, when the acquisition frequency of the first pose and the mapping data is high, or when the mobile acquisition device moves slowly during the acquisition process, the interpolation method may not be used, and the mapping data with the same timestamp as the timestamp of the first pose can be directly determined as the mapping data corresponding to the first pose. That is, in this embodiment, the timestamp matching in the above step S22 can be understood as the same timestamp, and the same timestamp means the same within the allowable error range, or it can be understood that when the timestamp of the mapping data is less than the preset threshold compared with the timestamp of the first pose, they can be considered consistent.

[0115] In some embodiments, for the above step S12, after the mobile acquisition device obtains the first pose and mapping data at each point to be acquired, it can also store the first pose and mapping data correspondingly, and then directly send the stored first poses and mapping data to the electronic device. In this way, the electronic device obtains the mapping data corresponding to each first pose, and the mobile acquisition device does not need to obtain the timestamps of the mapping data and the first pose.

[0116] For the case where the original map is a QR code map, since the navigation method of the QR code map has its particularity, therefore, for the case where the original map is a QR code map, in order to make the generated target map more accurate, the embodiments of the present application also provide a map alignment method. Below, for the convenience of subsequent description, the navigation method of the QR code map will be described first.

[0117] See Figure 3 , Figure 3 is a schematic diagram of navigation based on a QR code map provided by the embodiments of the present application. As can be seen from Figure 3 , a QR code array is pre-laid in the area to be acquired (such as Figure 3 the array composed of QR codes 31 in the figure. It can be understood that Figure 3 is only for illustrative purposes. Therefore, to make the drawings concise and clear, Figure 3 not all the QR codes constituting the QR code array are shown in the figure, and only one QR code is marked). A camera 33 is carried on the mobile acquisition device 32. When the mobile acquisition device 32 moves in the area to be acquired, if there is a QR code (such as QR code 31) at the position where the mobile acquisition device 32 is located, the mobile acquisition device 32 can scan the QR code through the camera 33, and then determine the current pose of the mobile acquisition device 32 based on the scanning result; if there is no QR code at the position where the mobile acquisition device 32 is located, the mobile acquisition device 32 needs to calculate its own current pose based on the odometer data carried by itself. In this way, the position of the mobile acquisition device is located in real time.

[0118] Combined with the above Figure 3 shown content, it can be understood that since the QR code itself needs to occupy a certain area, the QR codes in the area to be acquired should not be too close to each other. Therefore, there is a certain distance between each QR code. Thus, when the mobile acquisition device is not in the area where the QR code is located, it is impossible to determine its own pose by identifying the QR code, and it can only calculate the current pose through the odometer data. In this regard, due to the cumulative offset error of the odometer, there may be a pose jump, so there may be a contradiction between some first poses obtained by the mobile acquisition device, which will affect the accuracy of the target map.

[0119] Thus, in some embodiments, when the original map is a QR code map, the mobile acquisition device is equipped with an odometer. For each point to be acquired, the electronic device can also obtain the odometer data of the odometer when the mobile acquisition device is at the point to be acquired. Furthermore, based on the first pose, the odometer data, and the mapping data, the map generated in step S15 for recording the second environmental features corresponding to each first pose is optimized, and then the optimized map is used as the target map. In this way, the situation where the accuracy of the target map is relatively low due to odometer errors can be avoided as much as possible.

[0120] Specifically, taking the following Figure 4 and Figure 5 shown process as an example, the implementation process of the map alignment method provided in the embodiments of the present application when the original map is a QR code map is exemplarily described.

[0121] Refer to Figure 4 , Figure 4 , which is the second process schematic diagram of the map alignment method provided in the embodiments of the present application, and may include the following steps:

[0122] Step S41: Obtain the original map of the area to be acquired;

[0123] Wherein, the original map is a QR code map or a map constructed based on the first environmental features;

[0124] Step S42: According to the original map, control the mobile acquisition device to move to each point to be acquired in the area to be acquired;

[0125] Step S43: For each point to be acquired, obtain the mapping data collected by the mobile acquisition device at the point to be acquired, and obtain the first pose of the mobile acquisition device when it is at the point to be acquired, and the odometer data of the odometer carried by the mobile acquisition device when it is at the point to be acquired, and use the obtained mapping data as the mapping data corresponding to the obtained first pose;

[0126] Step S44: For the mapping data corresponding to each first pose, match the second environmental features of the mapping data with the second environmental features of the adjacent mapping data, and use the successfully matched feature points as the second environmental features corresponding to the first pose;

[0127] Wherein, the types of the second environmental features and the first environmental features are different;

[0128] Step S45: Generate a map for recording the second environmental features corresponding to each first pose according to each first pose and the second environmental features corresponding to each first pose, and use it as the map to be optimized;

[0129] Step S46: Based on the odometer data corresponding to each first pose and the mapping data corresponding to each first pose, the map to be optimized is optimized based on the odometer constraints and the visual constraints to obtain the target map.

[0130] In the technical solution provided in the embodiment of the present application, in addition to obtaining mapping data and the first pose, the electronic device also obtains odometer data. Furthermore, based on the odometer data and mapping data, the electronic device can optimize the map to be optimized that is directly generated based on the first pose and the second environmental features corresponding to the first pose, and thus obtain the target map, thereby avoiding as much as possible the low accuracy of the target map caused by odometer errors.

[0131] It can be understood that the above steps S41-S42 are Figure 1 Steps S11 to S12 are the same, and step S44 is the same as the above-mentioned step S14, so they are not repeated here.

[0132] In the above step S43, for each point to be collected, the electronic device also obtains the odometer data of the odometer carried by the mobile collection device when it is located at the point to be collected. The process of the electronic device obtaining the odometer data can refer to the above Figure 2 The electronic device shown in the figure obtains the mapping data, the first pose and other data. For other descriptions of step S43, please refer to the above step S13 and the above step S44. Figure 2 Description in .

[0133] In the above step S45, the map to be optimized is a map generated by the electronic device according to each first posture and the second environmental feature corresponding to each first posture, and used to record the second environmental feature corresponding to each first posture.

[0134] In the above step S46, based on the odometer data corresponding to the first pose and the mapping data corresponding to each first pose, the electronic device can optimize the map to be optimized based on the odometer constraints and the visual constraints to obtain the target map.

[0135] See also Figure 5 , Figure 5 The third flowchart of the map alignment method provided in the embodiment of the present application may include the following steps:

[0136] Step S51: obtaining the original map of the area to be collected;

[0137] The original map is a QR code map or a map constructed based on the first environmental feature;

[0138] Step S52: According to the original map, the mobile collection device is controlled to move to each point to be collected in the area to be collected;

[0139] Step S53: For each point to be collected, obtain the mapping data collected by the mobile collection device at the point to be collected, and obtain the first pose of the mobile collection device when it is at the point to be collected, as well as the odometer data of the odometer carried by the mobile collection device when it is at the point to be collected. Take the obtained mapping data as the mapping data corresponding to the obtained first pose;

[0140] Step S54: For the mapping data corresponding to each first pose, match the second environmental features of the mapping data with the second environmental features of the adjacent mapping data, and take the successfully matched feature points as the second environmental features corresponding to the first pose;

[0141] Among them, the types of the second environmental features and the first environmental features are different;

[0142] Step S55: Generate a map for recording the second environmental features corresponding to each first pose according to each first pose and the second environmental features corresponding to each first pose, as the map to be optimized;

[0143] Step S56: Based on the odometer data corresponding to each first pose and the mapping data corresponding to each first pose, optimize the position and pose in the second pose involved in the map to be optimized and optimize the pose in the third pose involved in the map to be optimized based on the odometer constraint and the visual constraint to obtain the target map;

[0144] Among them, the second pose is the pose calculated by the mobile collection device based on the odometer, and the third pose is the pose obtained by the mobile collection device scanning the QR code.

[0145] In the technical solution provided by the embodiment of the present application, it can be understood that for the case where the original map is a QR code map, if the first pose is deduced based on the odometer data, then due to the error of the odometer, it is possible that both the position and the pose in the first pose are inaccurate. Therefore, when performing odometer optimization, for this type of first pose, the position and pose in the first pose need to be optimized based on the odometer constraint and the visual constraint; and if the first pose is obtained based on the QR code, the position provided by the QR code for the first pose is accurate, but due to possible tilting and pasting of the QR code, etc., the pose in the first pose obtained based on the QR code may be inaccurate. Therefore, for this type of first pose, only the pose in the first pose needs to be optimized based on the odometer constraint and the visual constraint. In this way, the process of optimizing the target map is simplified, the calculation amount is reduced, and the operation efficiency is improved.

[0146] It can be understood that the above Step S51 - Step S52 is the same as Figure 1 Steps S11 - S12 in, Step S54 is the same as the above Step S14, and Step S53 is the same as Figure 4Same as step S43 in [reference], step S55 is the same as step S45 in [reference], so it will not be elaborated here. Figure 4 Same as step S45 in [reference], so it will not be elaborated here.

[0147] In the above step S56, the second pose and the third pose are actually both the first pose. Among them, the second pose is the first pose calculated by the mobile acquisition device based on the odometer, and the third pose is the pose obtained by the mobile acquisition device scanning the QR code.

[0148] In the embodiment of the present application, when optimizing the map to be optimized according to the odometer constraint and the visual constraint, for the second pose involved in the map to be optimized, both the position and the pose in the second pose are optimized; while for the third pose involved in the map to be optimized, only the pose in the third pose can be optimized.

[0149] In some embodiments, after the electronic device obtains the target map, it can store the first poses, the odometer data corresponding to the first poses, the mapping data corresponding to the first poses, and the target map in a corresponding manner. Refer to Figure 6 , Figure 6 is the fourth process schematic diagram of the map alignment method provided by the embodiment of the present application, and may include the following steps:

[0150] Step S61: Obtain the original map of the area to be collected;

[0151] Wherein, the original map is a QR code map or a map constructed based on the first environmental features;

[0152] Step S62: According to the original map, control the mobile acquisition device to move to each point to be collected in the area to be collected;

[0153] Step S63: For each point to be collected, obtain the mapping data collected by the mobile acquisition device at the point to be collected, and obtain the first pose of the mobile acquisition device when it is at the point to be collected, and the odometer data of the odometer carried by the mobile acquisition device when it is at the point to be collected, and use the obtained mapping data as the mapping data corresponding to the obtained first pose;

[0154] Step S64: For the mapping data corresponding to each first pose, match the second environmental features of the mapping data with the second environmental features of the adjacent mapping data, and use the successfully matched feature points as the second environmental features corresponding to the first pose;

[0155] Wherein, the type of the second environmental feature is different from that of the first environmental feature;

[0156] Step S65: Generate a map for recording the second environmental features corresponding to each first pose according to each first pose and the second environmental features corresponding to each first pose, as the map to be optimized;

[0157] Step S66: Based on the odometry data corresponding to each first pose and the mapping data corresponding to each first pose, and based on the odometry constraint and the visual constraint, optimize the map to be optimized to obtain the target map.

[0158] The above Step S61 - Step S66 is the same as Figure 4 Steps S41 - S46 in [reference], so it will not be elaborated here.

[0159] Step S67: Correspondingly store each first pose, the odometry data corresponding to each first pose, the mapping data corresponding to each first pose, and the target map.

[0160] In the embodiment of the present application, after obtaining the target map, the electronic device can correspondingly store the target map, the odometry data corresponding to each first pose when constructing the target map, each first pose, and the mapping data corresponding to each first pose. Among them, the corresponding storage relationship includes the corresponding relationship between the target map and the first pose, the odometry data, and the mapping data, and also includes the corresponding relationship between the first pose and the odometry data, and the corresponding relationship between the first pose and the mapping data.

[0161] In the technical solution provided by the embodiment of the present application, after obtaining the target map, the odometry data corresponding to each first pose, the mapping data corresponding to each first pose, and the target map are correspondingly stored. In this way, when the target map is subsequently maintained, such as operations like deletion, modification, and splicing, the map after maintenance can still be optimized based on these data to facilitate improving the accuracy of the map.

[0162] For example, in some embodiments, in response to a deletion instruction for the target map, the electronic device reads the correspondingly stored first poses, the odometry data corresponding to each first pose, and the mapping data corresponding to each first pose. Then, from the mapping data corresponding to each first pose read, delete the first pose and the mapping data indicated by the deletion instruction, and use the map formed by the remaining first poses and mapping data as the target deleted map. Among them, the deletion instruction can be understood as the user selecting a certain area in the target map and clicking the corresponding deletion button to indicate the deletion of that area.

[0163] Or, in some embodiments, in response to an addition instruction for the target map, the electronic device reads the correspondingly stored first poses, the odometry data corresponding to each first pose, and the mapping data corresponding to each first pose, and obtains the pose to be added indicated by the addition instruction and the mapping data to be added corresponding to each pose to be added. Then, the electronic device adds the pose to be added indicated by the addition instruction and the mapping data to be added corresponding to each pose to be added to the read first poses and the mapping data corresponding to each first pose, thereby obtaining the initial amplified map.

[0164] After that, based on the odometer data corresponding to each first pose, the mapping data corresponding to each first pose, the odometer data corresponding to each first pose, the mapping data corresponding to each first pose, and the to-be-added poses obtained, and the to-be-added mapping data corresponding to each to-be-added pose, the electronic device optimizes the area outside the first area in the initial augmented map based on odometer constraints and visual constraints; and, based on the odometer data corresponding to each first pose, the mapping data corresponding to each first pose, the odometer data corresponding to each first pose, the mapping data corresponding to each first pose, and the to-be-added poses obtained, and the to-be-added mapping data corresponding to each to-be-added pose, optimizes the first area in the initial augmented map based on odometer constraints, visual constraints, and co-visibility constraints. Thus, the optimized initial augmented map is used as the target augmented map. Alternatively, the electronic device may also perform odometer constraints and visual constraints on the initial augmented map only based on the odometer data corresponding to each first pose read, the mapping data corresponding to each first pose, and the mapping data corresponding to each first pose, to obtain the target augmented map. Or, in some embodiments, when the to-be-added poses and the to-be-added mapping data corresponding to each to-be-added pose are poses and mapping data obtained based on a QR code map, the electronic device may also perform odometer constraints, visual constraints, and co-visibility constraints on the initial augmented map according to the odometer data corresponding to each first pose read, the mapping data corresponding to each first pose, the to-be-added poses obtained, the to-be-added mapping data corresponding to each to-be-added pose, and the odometer data corresponding to each to-be-added pose, and then use the optimized initial augmented map as the target augmented map. Among them, the first area is the area where the to-be-added mapping data coincides with the mapping data corresponding to each first pose.

[0165] In the embodiments of the present application, the to-be-added poses and the mapping data corresponding to each to-be-added pose can be understood as the poses and mapping data collected and obtained by the mobile acquisition device in other areas except the to-be-acquired area. Understandably, in order to enable the to-be-added poses and the mapping data corresponding to the to-be-added poses to jointly construct a map with the first poses and the mapping data corresponding to the first poses, therefore, the type of the mapping data corresponding to the to-be-added data should be the same as the type of the mapping data corresponding to the first pose, and the position of the sensor for collecting the mapping data on the mobile acquisition device should be the same. That is, if the mapping data corresponding to the first pose is an image frame, and the visual sensor is located directly in front of the mobile acquisition device when collecting the mapping data corresponding to the first pose, then the mapping data corresponding to the to-be-added pose should also be an image frame, and the visual sensor should also be located directly in front of the mobile acquisition device when collecting the mapping data corresponding to the to-be-added pose.

[0166] The addition instruction can be understood as the user selecting the pose to be added to the target map and the mapping data corresponding to the pose to be added, and clicking the corresponding addition button, thereby indicating to add the pose to be added and the mapping data corresponding to the pose to be added to the target map. In some embodiments, when each pose to be added and the mapping data to be added corresponding to each pose to be added are the poses and mapping data obtained based on the QR code map, the addition instruction can also be understood as the user selecting the data to be added to the target map and clicking the corresponding addition button, where the data to be added includes the pose to be added, the mapping data corresponding to the pose to be added, and the odometer data corresponding to the pose to be added.

[0167] Alternatively, in some embodiments, the electronic device can also first respond to the deletion instruction to implement the deletion of the target map, determine the first pose remaining after deletion and the mapping data corresponding to the first pose, and then, in response to the addition instruction, construct an initial modified map according to the pose to be added indicated by the addition instruction, the mapping data corresponding to the pose to be added, and the first pose remaining after deletion and the mapping data corresponding to the first pose. Then, based on the odometer constraint, visual constraint, and co-visibility constraint, the initial modified map is optimized according to each first pose, the odometer data corresponding to each first pose, the mapping data corresponding to each first pose, the pose to be added, and the mapping data to be added corresponding to each pose to be added to obtain the target modified map. It can be understood that based on this example, it is possible to replace a certain area in the target map.

[0168] Or, in some embodiments, the electronic device can also first respond to the addition instruction to implement the addition of the target map to obtain a target augmented map. Furthermore, for the target augmented map, the poses in the target augmented map, the odometer data corresponding to the poses, the mapping data corresponding to the poses, and the target augmented map are stored correspondingly; then, when the electronic device receives a deletion instruction for the target augmented map, the electronic device can implement the deletion of the target map according to the data stored for the target augmented map, and the specific steps can refer to the above example of obtaining the target deleted map.

[0169] In another embodiment, after obtaining the target map, if the target map is only used for positioning, only the target map and the mapping data corresponding to each first pose in the target map can be stored correspondingly.

[0170] Combined with the above Figure 2 and Figure 6 , for the case where the original map is a QR code map, the map alignment method provided by the embodiments of the present application can also be implemented in another process. Refer to Figure 7 , Figure 7 which is the fifth process schematic diagram of the map alignment method provided by the embodiments of the present application, and can include the following steps:

[0171] Step S71: Obtain the original map of the area to be collected;

[0172] Among them, the original map is a QR code map or a map constructed based on the first environmental features; the same as the above step S61.

[0173] Step S72: According to the original map, control the mobile collection device to move to each point to be collected in the area to be collected; the same as the above step S62.

[0174] Step S73: Obtain each mapping data, the first pose sent by the mobile collection device, as well as the time stamps of each mapping data, the time stamps of each first pose, and the odometer data corresponding to each first pose;

[0175] Among them, the mobile collection device is used to collect mapping data and obtain its own first pose, the time stamp of the mapping data, and the time stamp of the first pose every time it reaches a point to be collected; it is also used to send the collected mapping data, the obtained first pose, the time stamp of the mapping data, and the time stamp of the first pose; refer to the above steps S63 and S21.

[0176] Step S74: For each first pose, determine the mapping data whose time stamp matches the time stamp of the first pose as the mapping data corresponding to the first pose; refer to the above steps S63 and S22.

[0177] Step S75: For the mapping data corresponding to each first pose, match the second environmental features of the mapping data with the second environmental features of the adjacent mapping data, and use the successfully matched feature points as the second environmental features corresponding to the first pose;

[0178] Among them, the type of the second environmental features is different from that of the first environmental features; the same as the above step S64.

[0179] Step S76: According to each first pose and the second environmental features corresponding to each first pose, generate a map for recording the second environmental features corresponding to each first pose as the map to be optimized; the same as the above step S65.

[0180] Step S77: According to the odometer data corresponding to each first pose and the mapping data corresponding to each first pose, optimize the map to be optimized based on odometer constraints and visual constraints to obtain the target map. The same as the above step S66.

[0181] Step S78, correspondingly store the odometer data corresponding to each first pose, the mapping data corresponding to each first pose, and the target map. The same as the above step S67.

[0182] Next, taking the original map as maps of types such as QR code map, laser map, texture map, etc., and the target map as a visual map as an example, based on Figure 8 , the map alignment method provided in the embodiments of the present application will be described again. Figure 8 FIG. Figure 8 is a sixth process schematic diagram of the map alignment method provided in the embodiments of the present application, which may include the following steps:

[0183] Step S81: Make the robot move along the existing map, and collect camera data and vehicle body positioning pose data;

[0184] In the embodiments of the present application, the robot can be understood as the above-mentioned mobile acquisition device; the existing map can be understood as the above-mentioned original map; the camera data can be understood as the above-mentioned mapping data; the vehicle body positioning pose data can be understood as the above-mentioned first pose. For the specific content of this step, reference can be made to the descriptions in the above steps S12 - S13.

[0185] Step S82: Align the image data with the vehicle body pose data, and extract the feature points and descriptors of the image;

[0186] In the embodiments of the present application, aligning the image data with the vehicle body pose data can be understood as determining the mapping data corresponding to each first pose; extracting the feature points and descriptors of the image can be understood as extracting the second environmental features of each mapping data. For the specific content of this step, reference can be made to the descriptions in the above steps S13 and Figure 2 in the description.

[0187] Step S83: Perform inter-frame feature matching to construct a visual 3D map, and the coordinates of this map have been aligned with the existing map;

[0188] In the embodiments of the present application, performing inter-frame feature matching can be understood as, for the mapping data corresponding to each first pose, matching the second environmental features of the mapping data with the second environmental features of the adjacent mapping data, and taking the successfully matched feature points as the second environmental features corresponding to the first pose; specifically, reference can be made to the description in the above step S14. For the process of constructing the visual 3D map, reference can be made to the description in the above step S15.

[0189] Step S84: For the QR code map, add odometer pose constraints to the positioning data between adjacent QR codes, and obtain more accurate poses and mapping data through graph optimization;

[0190] Specifically, reference can be made to the descriptions in the above steps S45 - S46, or reference can also be made to the description in the above step S56.

[0191] Step S85: Structurally streamline and organize the mapping data to obtain a mapping map for later editing and a positioning map for positioning. Reference can be made to the description in the above step S67.

[0192] Based on the technical solution provided by the embodiments of the present application, the following beneficial effects can be achieved:

[0193] (1) Based on the technical solution provided by the embodiments of the present application, a visual map that is aligned with maps of types such as laser maps (including 2D lasers and 3D lasers), QR code maps, and texture maps can be obtained, which can be used for visual navigation and positioning;

[0194] (2) Based on the technical solution provided by the embodiments of the present application, only by controlling the robot to move according to the existing map and collecting image frames and vehicle body positioning pose data, the visual map can be aligned with the existing map, without the need for cumbersome alignment calculations, translation, rotation transformation and other processes, and it is relatively simple and convenient to implement;

[0195] (3) In the technical solution provided by the embodiments of the present application, for the case where the existing map is a QR code map, a graph optimization problem is constructed using the inter-frame constraint of the original odometer, so that the optimized visual map can not only ensure the position alignment on the QR code, but also make the map accuracy between QR codes consistent;

[0196] (4) In the technical solution provided by the embodiments of the present application, since the positioning result of the existing map is directly used in the process of constructing the visual map, the obtained visual map can be strictly aligned with the existing map without introducing errors caused by manual alignment.

[0197] Corresponding to the above map alignment method, the embodiments of the present application also provide a map alignment device, as Figure 9 shown, the device includes:

[0198] A map acquisition module 91, configured to acquire an original map of the area to be collected, where the original map is a QR code map or a map constructed based on the first environmental feature;

[0199] A motion control module 92, configured to control the mobile acquisition device to move to each point to be collected in the area to be collected according to the original map;

[0200] A data acquisition module 93, configured to, for each point to be collected, acquire the mapping data collected by the mobile acquisition device at the point to be collected, and acquire the first pose of the mobile acquisition device when it is located at the point to be collected, and use the acquired mapping data as the mapping data corresponding to the acquired first pose;

[0201] A feature matching module 94, configured to, for the mapping data corresponding to each first pose, match the second environmental feature of the mapping data with the second environmental feature of the adjacent mapping data, and use the successfully matched feature points as the second environmental feature corresponding to the first pose, where the type of the second environmental feature is different from that of the first environmental feature;

[0202] A map generation module 95, configured to generate a map for recording the second environmental features corresponding to each first pose based on each first pose and the second environmental features corresponding to each first pose, as the target map of the area to be collected.

[0203] In the technical solution provided by the embodiments of the present application, the original map is a two-dimensional code map or a map constructed based on the first environmental features. After obtaining the original map of the area to be collected, according to the original map, the mobile collection device is controlled to move to each point to be collected in the area to be collected. For each point to be collected, the mapping data collected by the mobile collection device at the point to be collected is obtained, and the first pose of the mobile collection device when it is located at the point to be collected is obtained, and the obtained mapping data is used as the mapping data corresponding to the obtained first pose. Furthermore, for the mapping data corresponding to each first pose, the second environmental features of the mapping data are matched with the second environmental features of the adjacent mapping data, and the successfully matched feature points are used as the second environmental features corresponding to the first pose. Thus, according to each first pose and the second environmental features corresponding to each first pose, a map for recording the second environmental features corresponding to each first pose can be generated as the target map of the area to be collected.

[0204] It can be understood that in the embodiments of the present application, the target map is constructed based on the first pose and the second environmental features corresponding to the first pose. The first pose is the pose of the mobile collection device itself obtained when it is located at each point to be collected. The second environmental features corresponding to the first pose are obtained by feature matching of the second environmental features of the mapping data corresponding to the first pose and the second environmental features of the adjacent mapping data of the mapping data. Moreover, the mobile collection device moves to each point to be collected according to the original map. Therefore, the target map determined based on the first pose and the second environmental features corresponding to the first pose is a map aligned with the original map. And the original map is a two-dimensional code map or a map constructed based on the first environmental features, and the types of the second environmental features and the first environmental features are different.

[0205] It can be seen that when the original map is a two-dimensional code map or a map constructed based on the first environmental features in the technical solution provided by the embodiments of the present application, a target map aligned with the original map can be directly constructed. And the technical solution provided by the embodiments of the present application does not require complex alignment calculations, translation and rotation transformation steps, etc., and the implementation is relatively simple, and the calculation process is also relatively simple. Thus, when the target map to be constructed is a visual map, the visual map aligned with the original map can be directly constructed according to the technical solution provided by the embodiments of the present application, and the implementation is relatively simple, and the calculation process is also relatively simple.

[0206] In some embodiments, the data acquisition module 93 is specifically configured to acquire each mapping data, the first pose, the timestamps of each of the mapping data, and the timestamps of each of the first poses sent by the mobile acquisition device, where the mobile acquisition device is configured to, upon reaching each point to be acquired, acquire mapping data and obtain its own first pose, the timestamp of the mapping data, and the timestamp of the first pose; and is further configured to send the acquired mapping data, the obtained first pose, the timestamp of the mapping data, and the timestamp of the first pose.

[0207] For each of the first poses, determine the mapping data whose timestamp matches the timestamp of the first pose as the mapping data corresponding to the first pose.

[0208] In some embodiments, the original map is a QR code map, and the second environmental feature is a laser feature, or a texture feature, or a visual feature.

[0209] The first environmental feature is a laser feature, and the second environmental feature is a texture feature, or a visual feature.

[0210] The first environmental feature is a texture feature, and the second environmental feature is a laser feature, or a visual feature.

[0211] In some embodiments, the original map is a QR code map, and the mobile acquisition device is equipped with an odometer.

[0212] The data acquisition module 93 is further configured to, for each point to be acquired, acquire the odometer data of the odometer when the mobile acquisition device is at the point to be acquired.

[0213] The map generation module 95 is specifically configured to generate a map for recording the second environmental features corresponding to each of the first poses based on each of the first poses and the second environmental features corresponding to each of the first poses as the map to be optimized.

[0214] Optimize the map to be optimized based on the odometer data corresponding to each of the first poses and the mapping data corresponding to each of the first poses under the odometer constraint and the visual observation constraint to obtain the target map.

[0215] In some embodiments, optimizing the map to be optimized to obtain the target map includes:

[0216] Optimize the position and orientation in the second pose involved in the map to be optimized, and optimize the orientation in the third pose involved in the map to be optimized, and use the optimized map to be optimized as the target map of the area to be acquired, where the second pose is the pose calculated by the mobile acquisition device based on the odometer, and the third pose is the pose obtained by the mobile acquisition device scanning the QR code.

[0217] In some embodiments, each of the first poses, the odometry data corresponding to each of the first poses, the mapping data corresponding to each of the first poses, and the target map are stored correspondingly. The apparatus further includes:

[0218] A map deletion module, configured to, in response to a deletion instruction for the target map, read each of the first poses stored correspondingly and the mapping data corresponding to each of the first poses; delete the first pose and the mapping data indicated by the deletion instruction from the read first poses and the mapping data corresponding to each of the first poses, to obtain a target deleted map; and / or,

[0219] A map augmentation module, configured to, in response to an addition instruction for the target map, read each of the first poses stored correspondingly, the odometry data corresponding to each of the first poses, and the mapping data corresponding to each of the first poses, and obtain the pose to be added indicated by the addition instruction and the mapping data to be added corresponding to each of the poses to be added; add the pose to be added indicated by the addition instruction and the mapping data to be added corresponding to each of the poses to be added to the read first poses and the mapping data corresponding to each of the first poses, to obtain an initial augmented map; optimize the area outside the first area in the initial augmented map based on odometry constraints and visual constraints according to the read first poses, the odometry data corresponding to each of the first poses, the mapping data corresponding to each of the first poses, and the obtained pose to be added and the mapping data to be added corresponding to each of the poses to be added; and optimize the first area in the initial augmented map based on odometry constraints, visual constraints, and co-visibility constraints, to obtain a target augmented map; wherein the first area is the area where the mapping data to be added overlaps with the mapping data corresponding to each of the first poses.

[0220] An embodiment of the present application further provides an electronic device, as Figure 10 shown, including:

[0221] A memory 101, configured to store a computer program;

[0222] A processor 102, configured to implement any of the above-mentioned map alignment methods when executing the program stored on the memory 101.

[0223] And the above-mentioned electronic device may further include a communication bus and / or a communication interface. The processor 102, the communication interface, and the memory 101 complete communication with each other through the communication bus.

[0224] The communication bus mentioned in the above electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of simplicity, only a thick line is used to represent it in the figure, but it does not mean that there is only one bus or one type of bus.

[0225] The communication interface is used for communication between the above electronic device and other devices.

[0226] The memory may include a Random Access Memory (RAM), or may also include a Non-Volatile Memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located far from the aforementioned processor.

[0227] The above-mentioned processor may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0228] In another embodiment provided by the present application, a computer-readable storage medium is also provided. A computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the steps of any of the above map alignment methods are implemented.

[0229] In another embodiment provided by the present application, a computer program product containing instructions is also provided. When it runs on a computer, the computer is caused to execute any of the map alignment methods in the above embodiments.

[0230] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from a website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, fiber optic, digital subscriber line (DSL)) or wirelessly (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a solid-state disk (SSD), etc.

[0231] It should be noted that in this document, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variation thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising a..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.

[0232] Each embodiment in this specification is described in a related manner. For the same or similar parts among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for devices, electronic devices, storage media, and computer program products, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiments.

[0233] The foregoing are only the preferred embodiments of the present application and are not intended to limit the protection scope of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application are all included in the protection scope of the present application.

Claims

1. A map alignment method, characterized in that, The method includes: Obtaining an original map of the area to be collected, where the original map is a QR code map or a map constructed based on first environmental features; the area to be collected is the area where a target map needs to be constructed; Controlling a mobile acquisition device to move to each point to be collected in the area to be collected according to the original map; For each point to be collected, obtaining the mapping data collected by the mobile acquisition device at the point to be collected, and obtaining the first pose of the mobile acquisition device when it is at the point to be collected, and taking the obtained mapping data as the mapping data corresponding to the obtained first pose; For the mapping data corresponding to each first pose, matching the second environmental features of the mapping data with the second environmental features of adjacent mapping data, and taking the feature points with successful matching as the second environmental features corresponding to the first pose, where the types of the second environmental features are different from those of the first environmental features; Generating a map for recording the second environmental features corresponding to each first pose according to each first pose and the second environmental features corresponding to each first pose, as the target map of the area to be collected.

2. The method according to claim 1, wherein The step of obtaining the mapping data collected by the mobile acquisition device at the point to be collected, and obtaining the first pose of the mobile acquisition device when it is at the point to be collected, and taking the obtained mapping data as the mapping data corresponding to the obtained first pose includes: Obtaining each mapping data, first pose, the time stamp of each mapping data, and the time stamp of each first pose sent by the mobile acquisition device, where the mobile acquisition device is used to collect mapping data and obtain its own first pose, the time stamp of the mapping data, and the time stamp of the first pose every time it reaches the point to be collected; and is also used to send the collected mapping data, the obtained first pose, the time stamp of the mapping data, and the time stamp of the first pose; For each first pose, determining the mapping data whose time stamp matches the time stamp of the first pose as the mapping data corresponding to the first pose.

3. The method according to claim 1, wherein The original map is a QR code map, and the second environmental features are laser features, or texture features, or visual features; The first environmental features are laser features, and the second environmental features are texture features, or visual features; The first environmental features are texture features, and the second environmental features are laser features, or visual features.

4. The method according to claim 1, wherein The original map is a QR code map, and the mobile acquisition device is equipped with an odometer. The method further includes: for each point to be collected, obtaining the odometer data of the odometer of the mobile acquisition device when it is at the point to be collected; The step of generating a map for recording the second environmental features corresponding to each first pose according to each first pose and the second environmental features corresponding to each first pose, as the target map of the area to be collected includes: Generating a map for recording the second environmental features corresponding to each first pose according to each first pose and the second environmental features corresponding to each first pose, as a map to be optimized; Based on the odometry data corresponding to each of the first poses and the mapping data corresponding to each of the first poses, the map to be optimized is optimized based on odometry constraints and visual observation constraints to obtain the target map.

5. The method according to claim 4, wherein The optimizing the map to be optimized to obtain the target map includes: Optimizing the position and attitude in the second pose involved in the map to be optimized, and optimizing the attitude in the third pose involved in the map to be optimized, and taking the optimized map to be optimized as the target map of the area to be collected, where the second pose is the pose calculated by the mobile collection device based on odometry, and the third pose is the pose obtained by the mobile collection device scanning a QR code.

6. The method according to claim 4, characterized in that, Each of the first poses, the odometry data corresponding to each of the first poses, the mapping data corresponding to each of the first poses, and the target map are stored correspondingly, and the method further includes: In response to a deletion instruction for the target map, reading each of the first poses stored correspondingly and the mapping data corresponding to each of the first poses; deleting the first pose and the mapping data indicated by the deletion instruction from the read first poses and the mapping data corresponding to each of the first poses to obtain a target deleted map; and / or, In response to an addition instruction for the target map, reading each of the first poses stored correspondingly, the odometry data corresponding to each of the first poses, and the mapping data corresponding to each of the first poses, and obtaining the pose to be added indicated by the addition instruction and the mapping data to be added corresponding to each of the poses to be added; adding the pose to be added indicated by the addition instruction and the mapping data to be added corresponding to each of the poses to be added to the read first poses and the mapping data corresponding to each of the first poses to obtain an initial augmented map; based on the read first poses, the odometry data corresponding to each of the first poses, the mapping data corresponding to each of the first poses, and the obtained pose to be added and the mapping data to be added corresponding to each of the poses to be added, optimizing the area outside the first area in the initial augmented map based on odometry constraints and visual constraints; and, optimizing the first area in the initial augmented map based on odometry constraints, visual constraints, and co-visibility constraints to obtain a target augmented map; where the first area is the area where the mapping data to be added coincides with the mapping data corresponding to each of the first poses.

7. A map alignment device, characterized in that, The device includes: A map acquisition module, configured to acquire an original map of the area to be collected, where the original map is a QR code map or a map constructed based on first environmental features; the area to be collected is the area where a target map needs to be constructed; A motion control module, configured to control the mobile collection device to move to each point to be collected in the area to be collected according to the original map; A data acquisition module, configured to obtain mapping data collected by the mobile acquisition device at each of the to-be-acquired points, and obtain a first pose of the mobile acquisition device when it is at the to-be-acquired point, and use the obtained mapping data as the mapping data corresponding to the obtained first pose; A feature matching module, configured to match the second environmental features of the mapping data corresponding to each first pose with the second environmental features of adjacent mapping data, and use the successfully matched feature points as the second environmental features corresponding to the first pose, where the types of the second environmental features and the first environmental features are different; A map generation module, configured to generate a map for recording the second environmental features corresponding to each first pose according to each first pose and the second environmental features corresponding to each first pose, as the target map of the to-be-acquired area.

8. The apparatus according to claim 7, wherein The data acquisition module is specifically configured to: obtain each piece of mapping data, first pose, timestamp of each piece of mapping data, and timestamp of each first pose sent by the mobile acquisition device, where the mobile acquisition device is configured to collect mapping data and obtain its own first pose, timestamp of the mapping data, and timestamp of the first pose each time it reaches the to-be-acquired point; and is further configured to send the collected mapping data, obtained first pose, timestamp of the mapping data, and timestamp of the first pose; for each first pose, determine the mapping data whose timestamp matches the timestamp of the first pose as the mapping data corresponding to the first pose; and / or, The original map is a QR code map, and the second environmental features are laser features, or texture features, or visual features; the first environmental features are laser features, and the second environmental features are texture features, or visual features; the first environmental features are texture features, and the second environmental features are laser features, or visual features; and / or, The original map is a QR code map, the mobile acquisition device is equipped with an odometer, and the data acquisition module is further configured to obtain the odometer data of the mobile acquisition device when it is at each of the to-be-acquired points for each of the to-be-acquired points; the map generation module is further configured to: generate a map for recording the second environmental features corresponding to each first pose according to each first pose and the second environmental features corresponding to each first pose, as a map to be optimized; optimize the map to be optimized based on odometer constraints and visual observation constraints according to the odometer data corresponding to each first pose and the mapping data corresponding to each first pose, to obtain the target map; and / or, Optimizing the to-be-optimized map to obtain the target map includes: optimizing the position and attitude in the second pose involved in the to-be-optimized map, and optimizing the attitude in the third pose involved in the to-be-optimized map, and taking the optimized to-be-optimized map as the target map of the to-be-acquired area, where the second pose is the pose calculated by the mobile acquisition device based on the odometer, and the third pose is the pose obtained by the mobile acquisition device scanning the QR code; and / or, Each of the first poses, the odometer data corresponding to each of the first poses, the mapping data corresponding to each of the first poses, and the target map are stored correspondingly, and the device further includes: A map deletion module, configured to respond to a deletion instruction for the target map, read each of the first poses stored correspondingly and the mapping data corresponding to each of the first poses; delete the first pose and the mapping data indicated by the deletion instruction from the read first poses and the mapping data corresponding to each of the first poses to obtain a target deleted map; and / or, A map augmentation module, configured to respond to an addition instruction for the target map, read each of the first poses stored correspondingly, the odometer data corresponding to each of the first poses, and the mapping data corresponding to each of the first poses, and obtain the to-be-added poses indicated by the addition instruction and the to-be-added mapping data corresponding to each of the to-be-added poses; add the to-be-added poses indicated by the addition instruction and the to-be-added mapping data corresponding to each of the to-be-added poses to the read first poses and the mapping data corresponding to each of the first poses to obtain an initial augmented map; optimize the area outside the first area in the initial augmented map based on odometer constraints and visual constraints according to the read first poses, the odometer data corresponding to each of the first poses, the mapping data corresponding to each of the first poses, and the obtained to-be-added poses and the to-be-added mapping data corresponding to each of the to-be-added poses; and optimize the first area in the initial augmented map based on odometer constraints, visual constraints, and co-visibility constraints to obtain a target augmented map; where the first area is the area where the to-be-added mapping data overlaps with the mapping data corresponding to each of the first poses.

9. An electronic device, characterized in that, Including: A memory for storing a computer program; A processor, configured to implement the method according to any one of claims 1-6 when executing the program stored on the memory.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, it implements the method according to any one of claims 1-6.

Images