Map alignment method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a map alignment method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining an original map which is a two-dimensional code map or a map constructed based on a first environment feature; aiming at each to-be-acquired point location, acquiring mapping data acquired by the mobile acquisition equipment at the to-be-acquired point location, and acquiring a first pose of the mobile acquisition equipment when the mobile acquisition equipment is located at the to-be-acquired point location; for the mapping data corresponding to each first pose, matching the second environment feature of the mapping data with the second environment feature of the adjacent mapping data, and taking the successfully matched feature point as the second environment feature corresponding to the first pose; and according to the first poses and the second environment features corresponding to the first poses, generating a map for recording the second environment features corresponding to the first poses, and taking the map as a target map of the to-be-collected area. By applying the technical scheme provided by the embodiment of the invention, the constructed visual map can be aligned with the old map.

Description

Technical Field

[0001] The present application relates to the field of navigation and positioning technology, and in particular to a map alignment method, device, electronic device and storage medium. Background Art

[0002] Before the comprehensive development of visual navigation technology, for most scenarios, robots achieved navigation mainly based on QR code maps, laser maps, and texture maps (hereinafter collectively referred to as old maps). However, with the development of visual navigation technology, its low-cost advantage has attracted more and more attention. As a result, more and more robots have begun to be set up for vision-based navigation. Therefore, after introducing a robot that navigates based on vision into a scene (hereinafter referred to as a new robot), it is necessary to build a visual map of the scene for the new robot to move in the scene. However, since there are still robots that move based on old maps in the scene (hereinafter referred to as old robots), in order to ensure that the new robot and the old robot have consistent positioning in the same scene, the visual map needs to be aligned with the old map when building the visual map. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide a map alignment method, device, electronic device and storage medium to align the constructed visual map with the old map. The specific technical solution is as follows:

[0004] In a first aspect, an embodiment of the present application provides a map alignment method, the method comprising:

[0005] Obtaining an original map of the area to be collected, wherein the original map is a QR code map or a map constructed based on the first environmental feature;

[0006] According to the original map, the mobile collection device is controlled to move to each point to be collected in the area to be collected;

[0007] For each of the points to be collected, the mapping data collected by the mobile collection device at the point to be collected is obtained, and the first position of the mobile collection device when it is located at the point to be collected is obtained, and the obtained mapping data is used as the mapping data corresponding to the obtained first position;

[0008] For the mapping data corresponding to each first posture, matching the second environmental feature of the mapping data with the second environmental feature of the adjacent mapping data, and taking the successfully matched feature points as the second environmental feature corresponding to the first posture, wherein the second environmental feature is of a different type from the first environmental feature;

[0009] According to each of the first postures and the second environmental features corresponding to each of the first postures, a map for recording the second environmental features corresponding to each of the first postures is generated as a target map of the area to be collected.

[0010] In some embodiments, the step of acquiring the mapping data collected by the mobile acquisition device at the point to be collected, acquiring the first pose of the mobile acquisition device when the mobile acquisition device is located at the point to be collected, and using the acquired mapping data as the mapping data corresponding to the acquired first pose includes:

[0011] Acquire each mapping data, the first pose, and the timestamp of each mapping data and the timestamp of each first pose sent by the mobile acquisition device, wherein the mobile acquisition device is used to collect mapping data and obtain its own first pose, the timestamp of the mapping data, and the timestamp of the first pose every time it arrives at the point to be collected; and is also used to send the collected mapping data, the obtained first pose, the timestamp of the mapping data, and the timestamp of the first pose;

[0012] For each of the first postures, mapping data having a timestamp that matches the timestamp of the first posture is determined as the mapping data corresponding to the first posture.

[0013] In some embodiments, the original map is a QR code map, and the second environmental feature is a laser feature, or a texture feature, or a visual feature;

[0014] The first environmental feature is a laser feature, and the second environmental feature is a texture feature or a visual feature;

[0015] The first environmental feature is a texture feature, and the second environmental feature is a laser feature or a visual feature.

[0016] In some embodiments, the original map is a QR code map, and the mobile acquisition device carries an odometer.

[0017] The method further includes: for each of the points to be collected, obtaining the odometer data of the odometer when the mobile collection device is located at the point to be collected;

[0018] The step of generating a map for recording the second environmental features corresponding to each of the first postures according to each of the first postures and the second environmental features corresponding to each of the first postures as a target map of the area to be collected includes:

[0019] According to each of the first postures and the second environment features corresponding to each of the first postures, generating a map for recording the second environment features corresponding to each of the first postures as a map to be optimized;

[0020] According to the odometer data corresponding to each of the first postures and the mapping data corresponding to each of the first postures, the map to be optimized is optimized based on odometer constraints and visual observation constraints to obtain the target map.

[0021] In some embodiments, the optimizing the map to be optimized and using the optimized map to be optimized as the target map of the area to be collected includes:

[0022] The position and posture of the second posture involved in the map to be optimized are optimized, and the posture of the third posture involved in the map to be optimized is optimized, and the optimized map to be optimized is used as the target map of the area to be collected, wherein the second posture is the posture calculated by the mobile collection device based on the odometer, and the third posture is the posture obtained by the mobile collection device scanning the QR code.

[0023] In some embodiments, each of the first postures, the odometer data corresponding to each of the first postures, the mapping data corresponding to each of the first postures, and the target map are stored accordingly, and the method further includes:

[0024] In response to a deletion instruction for the target map, reading the corresponding first postures and the mapping data corresponding to the first postures; deleting the first postures and the mapping data indicated by the deletion instruction from the read first postures and the mapping data corresponding to the first postures, to obtain a target deleted map; and / or,

[0025] In response to an add instruction for the target map, the corresponding first postures, the odometer data corresponding to the first postures, and the mapping data corresponding to the first postures are read, and the postures to be added indicated by the add instruction and the mapping data to be added corresponding to the postures to be added are obtained; the postures to be added indicated by the add instruction and the mapping data to be added corresponding to the postures to be added are added to the first postures read and the mapping data corresponding to the first postures, so as to obtain an initial augmented map; according to the first postures read, the odometer data corresponding to the first postures, the mapping data corresponding to the first postures, and the acquired postures to be added and the mapping data to be added corresponding to the postures to be added, based on the odometer constraints and the visual constraints, the area outside the first area in the initial augmented map is optimized; and the first area in the initial augmented map is optimized based on the odometer constraints, the visual constraints and the common vision constraints to obtain the target augmented map; wherein the first area is an area where the mapping data to be added overlaps with the mapping data corresponding to the first postures.

[0026] In a second aspect, an embodiment of the present application further provides a map alignment device, the device comprising:

[0027] A map acquisition module, used to acquire an original map of the area to be collected, wherein the original map is a QR code map or a map constructed based on the first environmental feature;

[0028] A motion control module, used to control the mobile acquisition device to move to each point to be collected in the area to be collected according to the original map;

[0029] A data acquisition module is used to acquire, for each of the points to be collected, the mapping data collected by the mobile acquisition device at the point to be collected, and to acquire the first position when the mobile acquisition device is located at the point to be collected, and to use the acquired mapping data as the mapping data corresponding to the acquired first position;

[0030] A feature matching module is used to match the second environmental feature of the mapping data corresponding to each first posture with the second environmental feature of the adjacent mapping data, and use the successfully matched feature points as the second environmental feature corresponding to the first posture, wherein the second environmental feature is of a different type from the first environmental feature;

[0031] A map generation module is used to generate a map for recording the second environmental features corresponding to each of the first postures according to each of the first postures and the second environmental features corresponding to each of the first postures, as a target map of the area to be collected.

[0032] In some embodiments, the data acquisition module is specifically used to:

[0033] Acquire each mapping data, the first pose, and the timestamp of each mapping data and the timestamp of each first pose sent by the mobile acquisition device, wherein the mobile acquisition device is used to collect mapping data and obtain its own first pose, the timestamp of the mapping data, and the timestamp of the first pose every time it arrives at the point to be collected; and is also used to send the collected mapping data, the obtained first pose, the timestamp of the mapping data, and the timestamp of the first pose;

[0034] For each of the first postures, mapping data having a timestamp that matches the timestamp of the first posture is determined as the mapping data corresponding to the first posture.

[0035] In some embodiments, the original map is a QR code map, and the second environmental feature is a laser feature, or a texture feature, or a visual feature;

[0036] The first environmental feature is a laser feature, and the second environmental feature is a texture feature or a visual feature;

[0037] The first environmental feature is a texture feature, and the second environmental feature is a laser feature or a visual feature.

[0038] In some embodiments, the original map is a QR code map, and the mobile acquisition device carries an odometer.

[0039] The data acquisition module is further used for:

[0040] For each of the points to be collected, obtaining the odometer data of the odometer when the mobile collection device is located at the point to be collected;

[0041] The map generation module is specifically used for:

[0042] According to each of the first postures and the second environment features corresponding to each of the first postures, generating a map for recording the second environment features corresponding to each of the first postures as a map to be optimized;

[0043] According to the odometer data corresponding to each of the first postures and the mapping data corresponding to each of the first postures, the map to be optimized is optimized based on odometer constraints and visual observation constraints to obtain the target map.

[0044] In some embodiments, optimizing the map to be optimized to obtain the target map includes:

[0045] The position and posture of the second posture involved in the map to be optimized are optimized, and the posture of the third posture involved in the map to be optimized is optimized, and the optimized map to be optimized is used as the target map of the area to be collected, wherein the second posture is the posture calculated by the mobile collection device based on the odometer, and the third posture is the posture obtained by the mobile collection device scanning the QR code.

[0046] In some embodiments, each of the first postures, the odometer data corresponding to each of the first postures, the mapping data corresponding to each of the first postures, and the target map are stored accordingly, and the device further includes:

[0047] A map deletion module, configured to respond to a deletion instruction for the target map, read the corresponding first postures and the mapping data corresponding to the first postures; delete the first postures and the mapping data indicated by the deletion instruction from the read first postures and the mapping data corresponding to the first postures, to obtain a target deleted map; and / or,

[0048] A map augmentation module, for responding to an add instruction for the target map, reading the corresponding stored first postures, the odometer data corresponding to the first postures, and the mapping data corresponding to the first postures, and obtaining the postures to be added indicated by the add instruction and the mapping data to be added corresponding to the postures to be added; adding the postures to be added indicated by the add instruction and the mapping data to be added corresponding to the postures to be added to the read first postures and the mapping data corresponding to the first postures to obtain an initial augmented map; optimizing the area outside the first area in the initial augmented map based on the odometer constraint and the visual constraint according to the read first postures, the odometer data corresponding to the first postures, the mapping data corresponding to the first postures, and the acquired postures to be added and the mapping data to be added corresponding to the postures to be added; and optimizing the first area in the initial augmented map based on the odometer constraint, the visual constraint and the common view constraint to obtain a target augmented map; wherein the first area is an area where the mapping data to be added overlaps with the mapping data corresponding to the first postures.

[0049] In a third aspect, an embodiment of the present application provides an electronic device, including:

[0050] Memory, used to store computer programs;

[0051] The processor is used to implement any of the above-mentioned map alignment methods when executing the program stored in the memory.

[0052] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the map alignment method described above is implemented.

[0053] In a fifth aspect, an embodiment of the present application further provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute any of the map alignment methods described above.

[0054] Beneficial effects of the embodiments of the present application:

[0055] In the technical solution provided in the embodiment of the present application, the original map is a QR code map or a map constructed based on the first environmental feature. After the original map of the area to be collected is obtained, the mobile collection device is controlled to move to each point to be collected in the area to be collected according to the original map. For each point to be collected, the mapping data collected by the mobile collection device at the point to be collected is obtained, and the first position of the mobile collection device when it is located at the point to be collected is obtained, and the obtained mapping data is used as the mapping data corresponding to the obtained first position; further, for the mapping data corresponding to each first position, the second environmental feature of the mapping data is matched with the second environmental feature of the adjacent mapping data, and the feature point with successful matching is used as the second environmental feature corresponding to the first position; thus, according to each first position and the second environmental feature corresponding to each first position, a map for recording the second environmental feature corresponding to each first position can be generated as the target map of the area to be collected.

[0056] It can be understood that in the embodiment of the present application, the target map is constructed based on the first posture and the second environmental features corresponding to the first posture, and the first posture is the posture of the mobile acquisition device itself when it is located at each point to be collected, and the second environmental features corresponding to the first posture are obtained by feature matching the second environmental features of the mapping data corresponding to the first posture and the second environmental features of the adjacent mapping data of the mapping data, and the mobile acquisition device moves to each point to be collected according to the original map. Therefore, the target map determined based on the first posture and the second environmental features corresponding to the first posture is a map aligned with the original map. Therefore, when the target map to be constructed is a visual map, a visual map aligned with the original map can be directly constructed according to the technical solution provided in the embodiment of the present application.

[0057] Of course, implementing any product or method of the present application does not necessarily require achieving all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0059] Figure 1 A schematic diagram of a first flow chart of a map alignment method provided in an embodiment of the present application;

[0060] Figure 2 A detailed schematic diagram of the above step S13;

[0061] Figure 3A schematic diagram of navigation based on a QR code map provided in an embodiment of the present application;

[0062] Figure 4 A second flow chart of the map alignment method provided in the embodiment of the present application;

[0063] Figure 5 A third flow chart of the map alignment method provided in the embodiment of the present application;

[0064] Figure 6 A fourth flow chart of the map alignment method provided in the embodiment of the present application;

[0065] Figure 7 A fifth flow chart of the map alignment method provided in the embodiment of the present application;

[0066] Figure 8 A sixth flow chart of the map alignment method provided in the embodiment of the present application;

[0067] Fig. 9 A schematic diagram of the structure of a map alignment device provided in an embodiment of the present application;

[0068] Fig.10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0069] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field based on the present application belong to the scope of protection of the present application.

[0070] Before the comprehensive development of visual navigation technology, for most scenarios, robots achieved navigation mainly based on QR code maps, laser maps, and texture maps (hereinafter collectively referred to as old maps). However, with the development of visual navigation technology, its low-cost advantage has attracted more and more attention. As a result, more and more robots have begun to be set up for vision-based navigation. Therefore, after introducing a robot that navigates based on vision into a scene (hereinafter referred to as a new robot), it is necessary to build a visual map of the scene for the new robot to move in the scene. However, since there are still robots that move based on old maps in the scene (hereinafter referred to as old robots), in order to ensure that the new robot and the old robot have consistent positioning in the same scene, the visual map needs to be aligned with the old map when building the visual map.

[0071] In the related art, there is a robot 3D map alignment method, in which it is necessary to first obtain a two-dimensional (2D) laser map of the robot's operating environment and a three-dimensional (3D) laser map to be aligned with the 2D laser map. After that, the 3D laser map is aligned with the 2D laser map through calculation steps such as pre-alignment, horizontal recovery, point cloud registration, rotation and translation. Based on this method, although the 2D laser map can be aligned with the 3D laser map, it has the following defects:

[0072] (1) This method is aimed at aligning a 3D laser map with an existing 2D laser map, and is not suitable for aligning a QR code map with a visual map;

[0073] (2) This method requires obtaining a 3D laser map in advance and then aligning the 3D laser map with the existing 2D laser map. It is impossible to directly obtain a 3D laser map aligned with a 2D laser map, and the implementation is relatively complicated.

[0074] (3) This method requires calculation steps such as pre-alignment, horizontal recovery, point cloud registration, rotation and translation to align the 3D laser map with the existing 2D laser map, and the calculation process is relatively complicated.

[0075] In response to the above problems, the present application provides a map alignment method. Based on the technical solution provided in the embodiments of the present application, a target map aligned with the original map can be directly constructed. Moreover, the technical solution provided by the present application does not require complex alignment calculations and translation and rotation transformations, and is relatively simple to implement and has a relatively simple calculation process.

[0076] The map alignment method provided in the present application is applied to an electronic device, which may be a desktop computer, a portable computer, an intelligent mobile terminal, etc. It can be understood that the map alignment method provided in the present application can be understood as a map construction method for the purpose of alignment.

[0077] The following is a detailed description of the map alignment method provided in the embodiment of the present application through a specific example:

[0078] See also Figure 1 , Figure 1 The first flow chart of the map alignment method provided in the embodiment of the present application includes the following steps:

[0079] Step S11: Obtaining the original map of the area to be collected;

[0080] The original map is a QR code map or a map constructed based on the first environmental feature;

[0081] Step S12: According to the original map, the mobile collection device is controlled to move to each point to be collected in the area to be collected;

[0082] Step S13: for each point to be collected, obtain the mapping data collected by the mobile collection device at the point to be collected, and obtain the first position when the mobile collection device is located at the point to be collected, and use the obtained mapping data as the mapping data corresponding to the obtained first position;

[0083] Step S14: for the mapping data corresponding to each first pose, matching the second environment feature of the mapping data with the second environment feature of the adjacent mapping data, and taking the successfully matched feature points as the second environment feature corresponding to the first pose;

[0084] wherein the second environmental feature is of a different type than the first environmental feature;

[0085] Step S15: using each first posture and a map of the second environmental features corresponding to each first posture as a target map of the area to be collected.

[0086] In the technical solution provided in the embodiment of the present application, the original map is a QR code map or a map constructed based on the first environmental feature. After the original map of the area to be collected is obtained, the mobile collection device is controlled to move to each point to be collected in the area to be collected according to the original map. For each point to be collected, the mapping data collected by the mobile collection device at the point to be collected is obtained, and the first position of the mobile collection device when it is located at the point to be collected is obtained, and the obtained mapping data is used as the mapping data corresponding to the obtained first position; further, for the mapping data corresponding to each first position, the second environmental feature of the mapping data is matched with the second environmental feature of the adjacent mapping data, and the feature point with successful matching is used as the second environmental feature corresponding to the first position; thus, according to each first position and the second environmental feature corresponding to each first position, a map for recording the second environmental feature corresponding to each first position can be generated as the target map of the area to be collected.

[0087] It can be understood that in the embodiment of the present application, the target map is constructed based on the first posture and the second environmental feature corresponding to the first posture, and the first posture is the posture of the mobile acquisition device itself acquired when it is located at each point to be collected, and the second environmental feature corresponding to the first posture is obtained by feature matching the second environmental feature of the mapping data corresponding to the first posture and the second environmental feature of the adjacent mapping data of the mapping data, and the mobile acquisition device moves to each point to be collected according to the original map. Therefore, the target map determined based on the first posture and the second environmental feature corresponding to the first posture is a map aligned with the original map, and the original map is a QR code map or a map constructed based on the first environmental feature, and the second environmental feature is of a different type from the first environmental feature.

[0088] It can be seen that by applying the technical solution provided in the embodiment of the present application, when the original map is a QR code map or a map constructed based on the first environmental feature, a target map aligned with the original map can be directly constructed; and the technical solution provided in the embodiment of the present application does not require complex alignment calculations, translations, rotations, transformations and other steps, and is relatively simple to implement and the calculation process is relatively simple. Therefore, when the target map to be constructed is a visual map, a visual map aligned with the original map can be directly constructed according to the technical solution provided in the embodiment of the present application, and the implementation is relatively simple and the calculation process is relatively simple.

[0089] In the above step S11, the area to be collected is the area where the target map needs to be constructed; the first environmental feature can be various types of features such as laser features, texture features, etc., among which laser features can also be divided into two-dimensional laser features and three-dimensional laser features; taking laser features and texture features as examples, when the first environmental feature is a laser feature, the map constructed based on the first environmental feature can be called a laser map; when the first environmental feature is a downward-looking texture feature, the map constructed based on the first environmental feature can be called a texture map.

[0090] It can be understood that the map constructed based on the first environmental feature can actually be understood as a map for realizing navigation based on the first environmental feature. For example, the laser map constructed based on laser features, the principle of navigation is that a laser radar is installed on the robot, and when the robot moves to a certain position, the laser radar collects point cloud data of the surrounding environment, and then, the current position of the robot is determined according to the collected point cloud data, thereby realizing autonomous navigation and positioning of the robot; for another example, the texture map constructed based on texture features, the principle of navigation is that a visual sensor is installed on the robot, and when the robot moves to a certain position, the visual sensor obtains the surrounding image texture information, and then, the current position of the robot is determined according to the obtained image texture information, thereby realizing autonomous navigation and positioning of the robot.

[0091] Here, the navigation principle of the map constructed based on other first environmental features is the same as that of the laser map and the texture map. The only difference is that the feature data used to locate the robot is different. Specifically, the feature data used to locate the robot is actually the first environmental feature used to construct the map. For example, the feature data used to locate the robot in the laser map is the point cloud data collected based on the laser emitted by the laser sensor, and the feature data used to locate the robot in the texture map is the image texture information. Therefore, the navigation principle of the map constructed based on other first environmental features will not be described here. In addition, the principle of autonomous navigation and positioning based on the QR code map can be found in the following Figure 3 The description in will not be elaborated here.

[0092] In this application, the reason for obtaining the original map is that the target map to be constructed must be aligned with the original map, that is, the target map to be constructed must be aligned with the coordinates of the original map to ensure that the positioning results of the robot in the target map and the original map are consistent. Therefore, this application needs to first obtain the original map (also called the old map), and then control the mobile acquisition device to move to each point to be collected according to the original map (see step S12 below), and collect the first posture and the second environmental features corresponding to the first posture at each point to be collected (see steps S13-S14 below), and then, according to each first posture and the second environmental features corresponding to each first posture, obtain the target map (see step S25 below). In this way, in the constructed target map, the second environmental features corresponding to the robot when it is in a certain posture are actually the second environmental features corresponding to the robot when it is in the posture in the original map.

[0093] It can be seen that in this application, the role of the original map is actually to control the mobile collection device to move to each point to be collected. Therefore, in this application, the original map can be used as long as it can be used to realize the navigation and positioning of the mobile collection device in the area to be collected, and it is a map that the target map needs to be aligned with. The specific type of the original map is not important, that is, it is not important what type of feature the first environmental feature used to construct the original map is. Therefore, in this application, there is no specific limitation on the first environmental feature. The original map can be any type of map. However, because there is no need to construct a map identical to the original map (for example, there is no problem of reconstructing a 2D laser map based on a 2D laser map), in this application, the types of the first environmental feature and the second environmental feature are different. In this article, for the sake of ease of description and understanding, the examples in the subsequent content mainly take the original map as a QR code map, a laser map, and a texture map; the target map is a visual map as an example for explanation.

[0094] In the above step S12, the mobile acquisition device can be a device that can autonomously perform navigation and positioning according to the original map, or it can be a device that can move to the position specified by the instruction in response to the instruction issued by the electronic device; it can be understood that in order to realize the collection of subsequent mapping data and the acquisition of its own posture, the mobile acquisition device carries a device that can be used to collect mapping data, and a device that can be used to determine its own posture. For details, please refer to the content of the embodiment shown after the following step S15.

[0095] The points to be collected are points set by the user according to actual conditions. In some embodiments, the user can directly specify which positions in the area to be collected are the points to be collected. Then, according to the original map, the mobile collection device can move to these points to be collected one by one; or, the user can also plan the movement path of the mobile collection device in the area to be collected, and pre-set the preset time. Then, according to the original map, the mobile collection device moves along the pre-planned movement path. Every time the preset time is reached, the position of the mobile collection device is taken as a point to be collected. For example, the pre-planned path is from position A to position B, and the preset time is 2 seconds (S). The mobile collection device starts to move from position A along the pre-planned path, and arrives at position C after 2S. Then position C is a point to be collected; then, it continues to move from position C along the pre-planned path, and arrives at position D after 2S. Then position D is a point to be collected, and so on, until it moves to position B.

[0096] It can be understood that the purpose of setting the points to be collected in the present application is to generate a target map based on the first pose of the mobile collection device at the points to be collected and the mapping data corresponding to the first pose. Therefore, in order to improve the accuracy and precision of the target map, the number of points to be collected can be designed to be larger and distributed as evenly as possible in the area to be collected.

[0097] In the above step S13, the mapping data is data obtained by the mobile acquisition device at the point to be collected for providing the second environmental feature, and the second environmental feature is a feature used to construct the target map, that is, the specific content of the mapping data collected by the mobile acquisition device depends on the type of target map to be constructed. The first pose is the pose of the mobile acquisition device itself when it is located at the point to be collected. Specifically, in a specific example, when the original map is a texture map, the mobile acquisition device can determine the pose of the mobile acquisition device based on the image texture information collected by the visual sensor carried by the mobile acquisition device, that is, the first pose.

[0098] In the embodiment of the present application, for each point to be collected, the mobile collection device collects mapping data, obtains its own first pose, and uses the obtained mapping data as the mapping data of the first pose. For example, when the mobile collection device is located at point A to be collected, the mapping data collected is data 1, and the first pose obtained is pose 1, then mapping data 1 is the mapping data corresponding to pose 1; when the mobile collection device is located at point B to be collected, the mapping data collected is data 2, and the first pose obtained is pose 2, then mapping data 2 is the mapping data corresponding to pose 2; wherein, data 1 and data 2 are both mapping data; pose 1 and pose 2 are both first poses.

[0099] For the specific method of obtaining the mapping data corresponding to the first pose, please refer to the following Figure 2 Description in .

[0100] In the above step S14, the electronic device can extract the features of each mapping data as the second environmental features of each mapping data; thus, each first pose corresponds to one mapping data, and each mapping data corresponds to one second environmental feature. Based on this, for the mapping data corresponding to each first pose, the second environmental features of the mapping data are matched with the second environmental features of the adjacent mapping data, and then the successfully matched feature points are used as the second environmental features corresponding to the first pose. Wherein, the types of the first environmental features and the second environmental features are different.

[0101] In the above step S15, according to each first posture and the second environmental feature corresponding to each first posture, the electronic device can generate a map for recording the second environmental feature corresponding to each first posture as a target map of the area to be collected.

[0102] In some embodiments, when the original map in the above step S11 is a two-dimensional code map, the second environmental feature may be a laser feature, a texture feature, or a visual feature; when the first environmental feature in the above step S11 is a laser feature, the second environmental feature may be a texture feature, or a visual feature, or, in some embodiments, when the first environmental feature is specifically a two-dimensional laser feature, the second environmental feature may also be a three-dimensional laser feature; when the first environmental feature in the above step S11 is a texture feature, the second environmental feature may be a laser feature, or a visual feature.

[0103] Among them, when the second environmental feature is a laser feature, the type of the target map in the above step S15 is a laser map, the device carried by the mobile acquisition device in the above step S13 for collecting mapping data may be a laser sensor, and the collected mapping data may be point cloud data; when the second environmental feature is a texture feature, the type of the target map in the above step S15 is a texture map, the device carried by the mobile acquisition device in the above step S13 for collecting mapping data may be a visual sensor, and the collected mapping data may be image texture data; when the second environmental feature is a visual feature, the type of the target map in the above step S15 is a visual map (i.e., a three-dimensional map capable of representing environmental features and posture), the device carried by the mobile acquisition device in the above step S13 for collecting mapping data may be a visual sensor, and the collected mapping data may be an image frame, and in the above step S14, based on the traditional scale-invariant feature change (ScaleInvariant Feature The second environmental features of the mapping data (such as image frames) can be extracted by using the SIFT (Single Integer Transform) algorithm, or by using the SuperPoint (i.e., super-point feature) deep learning algorithm.

[0104] In the above step S15, the specific method for extracting the second environmental feature of the mapping data can be determined by the user according to actual needs, and this application does not limit it.

[0105] See also Figure 2 , Figure 2 This is a detailed schematic diagram of the above step S13, which may include the following steps:

[0106] Step S21: Acquire each mapping data, the first pose, and the timestamp of each mapping data and the timestamp of each first pose sent by the mobile acquisition device;

[0107] The mobile acquisition device is used to collect mapping data and obtain its own first pose, the timestamp of the mapping data, and the timestamp of the first pose every time it arrives at the point to be collected; it is also used to send the collected mapping data, the obtained first pose, the timestamp of the mapping data, and the timestamp of the first pose;

[0108] Step S22: For each first pose, determine the mapping data whose timestamp matches the timestamp of the first pose as the mapping data corresponding to the first pose.

[0109] In the technical solution provided in the embodiment of the present application, the electronic device obtains each mapping data, the first pose, and the timestamp of each mapping data and the timestamp of each first pose from the mobile acquisition device, thereby, based on the timestamp of each mapping data and the timestamp of each first pose, the mapping data whose timestamp matches the timestamp of the first pose can be determined, and then the mapping data corresponding to each first pose can be determined. A specific implementation method is provided for determining the mapping data corresponding to the first pose in the above step S13.

[0110] In the above step S21, each time a point to be collected is reached, the mobile collection device collects mapping data, and obtains its own first pose, the timestamp of the mapping data, and the timestamp of the first pose, and also sends the collected mapping data, the obtained first pose, the timestamp of the mapping data, and the timestamp of the first pose.

[0111] The mobile acquisition device may send the mapping data, the first position, and the timestamp of the mapping data and the timestamp of the first position of each point to be collected to the electronic device after obtaining the mapping data, the first position, and the timestamp of the mapping data and the timestamp of the first position of each point to be collected, or may send all the data to the electronic device together after collecting the mapping data, the first position, and the timestamp of the mapping data and the timestamp of the first position of each point to be collected. In addition, in addition to the mobile acquisition device actively sending the mapping data, the first position, and the timestamp of the mapping data and the timestamp of the first position of each point to the electronic device, the electronic device may also actively obtain the mapping data, the first position, and the timestamp of the mapping data and the timestamp of the first position of each point from the mobile acquisition device.

[0112] In the above step S22, for each first pose, the electronic device determines the mapping data whose timestamp matches the timestamp of the first pose as the mapping data corresponding to the first pose.

[0113] Specifically, in some embodiments, the electronic device may determine the mapping data corresponding to each first pose according to an interpolation method, wherein the interpolation method used may specifically be a linear interpolation method, a Lagrange interpolation method, a Newton interpolation method, and the like.

[0114] Alternatively, in other embodiments, when the acquisition frequency of the first pose and the mapping data is high, or the mobile acquisition device moves slowly during the acquisition process, the interpolation method may not be used, and the mapping data with the same timestamp as the timestamp of the first pose may be directly determined as the mapping data corresponding to the first pose. That is, in this embodiment, the matching of the timestamps in the above step S22 can be understood as the same timestamps, and the same timestamps means the same within the allowable error range, or it can be understood that when the timestamp of the mapping data and the timestamp of the first pose are less than a preset threshold, they can be considered to be consistent.

[0115] In some embodiments, for the above step S12, the mobile acquisition device may also store the first pose and mapping data correspondingly after acquiring the first pose and mapping data at each point to be acquired, and then directly send the correspondingly stored first poses and mapping data to the electronic device, so that the electronic device obtains the mapping data corresponding to each first pose, and the mobile acquisition device does not need to obtain the timestamp of the mapping data and the timestamp of the first pose.

[0116] In the case where the original map is a QR code map, since the navigation method of the QR code map has its particularity, in order to make the generated target map more accurate, the embodiment of the present application also provides a map alignment method. Below, for the convenience of subsequent description, the navigation method of the QR code map is first described.

[0117] See also Figure 3 , Figure 3 A schematic diagram of navigation based on a QR code map provided in an embodiment of the present application, comprising Figure 3 It can be seen that a QR code array (such as Figure 3 The array of two-dimensional codes 31 in FIG. 1 can be understood as follows: Figure 3 This is for illustrative purposes only, so to make the drawings concise and clear, Figure 3 Not all the two-dimensional codes constituting the two-dimensional code array are shown, and only one two-dimensional code is marked). The mobile acquisition device 32 carries a camera 33. When the mobile acquisition device 32 moves in the area to be collected, if there is a two-dimensional code (such as the two-dimensional code 31) at the position of the mobile acquisition device 32, the mobile acquisition device 32 can scan the two-dimensional code through the camera 33, and then determine the current position of the mobile acquisition device 32 based on the scanning result; if there is no two-dimensional code at the position of the mobile acquisition device 32, the mobile acquisition device 32 needs to calculate its current position based on the odometer data in the odometer carried by itself, so as to locate the position of the mobile acquisition device in real time.

[0118] Combined with the above Figure 3 The content shown is understandable. Since the QR code itself needs to occupy a certain area, the intervals between each QR code in the area to be collected should not be too small. Therefore, there is a certain distance between each QR code. Therefore, when the mobile collection device is not in the area where the QR code is located, it is impossible to determine its own position by identifying the QR code, and can only calculate the current position through the odometer data. In this regard, due to the cumulative offset error of the odometer, the position jump will occur. Therefore, there may be contradictions between some of the first position obtained by the mobile collection device, which in turn affects the accuracy of the target map.

[0119] Therefore, in some embodiments, when the original map is a QR code map, the mobile acquisition device carries an odometer. For each point to be collected, the electronic device can also obtain the odometer data of the odometer when the mobile acquisition device is located at the point to be collected, and then optimize the map generated in the above step S15 for recording the second environmental features corresponding to each first posture based on the first posture, odometer data, and map construction data, and then use the optimized map as the target map. In this way, the low accuracy of the target map caused by the odometer error can be avoided as much as possible.

[0120] Specifically, the following Figure 4 and Figure 5 The process shown is used as an example to illustrate the implementation process of the map alignment method provided in the embodiment of the present application when the original map is a QR code map.

[0121] See also Figure 4 , Figure 4 The second flow chart of the map alignment method provided in the embodiment of the present application may include the following steps:

[0122] Step S41: obtaining the original map of the area to be collected;

[0123] The original map is a QR code map or a map constructed based on the first environmental feature;

[0124] Step S42: According to the original map, the mobile collection device is controlled to move to each point to be collected in the area to be collected;

[0125] Step S43: for each point to be collected, obtain the mapping data collected by the mobile collection device at the point to be collected, obtain the first position when the mobile collection device is located at the point to be collected, and obtain the odometer data of the odometer carried by the mobile collection device when it is located at the point to be collected, and use the obtained mapping data as the mapping data corresponding to the obtained first position;

[0126] Step S44: for the mapping data corresponding to each first pose, matching the second environment feature of the mapping data with the second environment feature of the adjacent mapping data, and taking the successfully matched feature points as the second environment feature corresponding to the first pose;

[0127] wherein the second environmental feature is of a different type than the first environmental feature;

[0128] Step S45: generating a map for recording the second environmental features corresponding to each first posture according to each first posture and the second environmental features corresponding to each first posture as a map to be optimized;

[0129] Step S46: According to the odometer data corresponding to each first pose and the mapping data corresponding to each first pose, the map to be optimized is optimized based on the odometer constraint and the visual constraint to obtain the target map.

[0130] In the technical solution provided in the embodiment of the present application, in addition to obtaining mapping data and the first pose, the electronic device also obtains odometer data. Furthermore, based on the odometer data and mapping data, the electronic device can optimize the map to be optimized that is directly generated based on the first pose and the second environmental features corresponding to the first pose, and then obtain the target map, thereby avoiding as much as possible the situation where the target map has low accuracy due to odometer errors.

[0131] It can be understood that the above steps S41 to S42 are Figure 1 Step S11-step S12 are the same, and step S44 is the same as the above-mentioned step S14, so they are not repeated here.

[0132] In the above step S43, for each point to be collected, the electronic device also obtains the odometer data of the odometer carried by the mobile collection device when it is located at the point to be collected. The process of the electronic device obtaining the odometer data can refer to the above Figure 2 The electronic device shown in the figure obtains the mapping data, the first pose and other data. For other descriptions of step S43, please refer to the above step S13, as well as the above step S44. Figure 2 Description in .

[0133] In the above step S45, the map to be optimized is a map generated by the electronic device according to each first posture and the second environmental features corresponding to each first posture for recording the second environmental features corresponding to each first posture.

[0134] In the above step S46, according to the odometer data corresponding to the first pose and the mapping data corresponding to each first pose, the electronic device can optimize the map to be optimized based on the odometer constraints and the visual constraints to obtain the target map.

[0135] See also Figure 5 , Figure 5 The third flow chart of the map alignment method provided in the embodiment of the present application may include the following steps:

[0136] Step S51: obtaining the original map of the area to be collected;

[0137] The original map is a QR code map or a map constructed based on the first environmental feature;

[0138] Step S52: According to the original map, the mobile collection device is controlled to move to each point to be collected in the area to be collected;

[0139] Step S53: for each point to be collected, obtain the mapping data collected by the mobile collection device at the point to be collected, obtain the first position when the mobile collection device is located at the point to be collected, and obtain the odometer data of the odometer carried by the mobile collection device when it is located at the point to be collected, and use the obtained mapping data as the mapping data corresponding to the obtained first position;

[0140] Step S54: for the mapping data corresponding to each first pose, matching the second environment feature of the mapping data with the second environment feature of the adjacent mapping data, and taking the successfully matched feature points as the second environment feature corresponding to the first pose;

[0141] wherein the second environmental feature is of a different type than the first environmental feature;

[0142] Step S55: generating a map for recording the second environmental features corresponding to each first posture according to each first posture and the second environmental features corresponding to each first posture as a map to be optimized;

[0143] Step S56: According to the odometer data corresponding to each first posture and the mapping data corresponding to each first posture, based on the odometer constraint and the visual constraint, the position and posture in the second posture involved in the map to be optimized are optimized, and the posture in the third posture involved in the map to be optimized is optimized to obtain a target map;

[0144] Among them, the second posture is the posture calculated by the mobile acquisition device based on the odometer, and the third posture is the posture obtained by the mobile acquisition device scanning the QR code.

[0145] In the technical solution provided in the embodiment of the present application, it can be understood that, for the case where the original map is a QR code map, if the first pose is calculated based on the odometer data, then due to the error of the odometer, the position and attitude in the first pose may be inaccurate. Therefore, when optimizing the odometer, for this type of first pose, the position and attitude in the first pose need to be optimized according to the odometer constraint and the visual constraint; and if the first pose is obtained based on the QR code, the position provided by the QR code for the first pose is accurate, but since the QR code may be tilted when pasted, the attitude in the first pose obtained based on the QR code may be inaccurate. Therefore, for this type of first pose, it is only necessary to optimize the attitude in the first pose based on the odometer constraint and the visual constraint. In this way, the process of optimizing the target map is simplified, the amount of calculation is reduced, and the operation efficiency is improved.

[0146] It can be understood that the above steps S51 to S52 are Figure 1 Step S11-step S12 are the same as above, step S54 is the same as above step S14, and step S53 is the same as above step S54. Figure 4The same as step S43 in step S55. Figure 4 The step S45 is the same as that in the above step and will not be described again here.

[0147] In the above step S56, the second posture and the third posture are actually the first posture, wherein the second posture is the first posture calculated by the mobile acquisition device based on the odometer, and the third posture is the posture obtained by the mobile acquisition device scanning the QR code.

[0148] In an embodiment of the present application, when optimizing the map to be optimized according to the odometer constraints and the visual constraints, for the second posture involved in the map to be optimized, both the position and the posture in the second posture are optimized; and for the third posture involved in the map to be optimized, only the posture in the third posture can be optimized.

[0149] In some embodiments, after the electronic device obtains the target map, it can store each first pose, the odometer data corresponding to each first pose, the mapping data corresponding to each first pose, and the target map accordingly. Figure 6 , Figure 6 The fourth flow chart of the map alignment method provided in the embodiment of the present application may include the following steps:

[0150] Step S61: obtaining the original map of the area to be collected;

[0151] The original map is a QR code map or a map constructed based on the first environmental feature;

[0152] Step S62: According to the original map, the mobile collection device is controlled to move to each point to be collected in the area to be collected;

[0153] Step S63: for each point to be collected, obtain the mapping data collected by the mobile collection device at the point to be collected, obtain the first position when the mobile collection device is located at the point to be collected, and obtain the odometer data of the odometer carried by the mobile collection device when it is located at the point to be collected, and use the obtained mapping data as the mapping data corresponding to the obtained first position;

[0154] Step S64: for the mapping data corresponding to each first pose, matching the second environment feature of the mapping data with the second environment feature of the adjacent mapping data, and taking the successfully matched feature points as the second environment feature corresponding to the first pose;

[0155] wherein the second environmental feature is of a different type than the first environmental feature;

[0156] Step S65: generating a map for recording the second environmental features corresponding to each first posture according to each first posture and the second environmental features corresponding to each first posture as a map to be optimized;

[0157] Step S66: According to the odometer data corresponding to each first pose and the mapping data corresponding to each first pose, the map to be optimized is optimized based on the odometer constraint and the visual constraint to obtain the target map.

[0158] The above steps S61 to S66 are Figure 4 Steps S41 to S46 are the same, so they are not repeated here.

[0159] Step S67: Correspondingly storing each first position, the odometer data corresponding to each first position, the mapping data corresponding to each first position, and the target map.

[0160] In an embodiment of the present application, after obtaining the target map, the electronic device can correspondingly store the target map, the odometer data corresponding to each first posture when constructing the target map, each first posture, and the mapping data corresponding to each first posture, wherein the corresponding storage relationship includes the correspondence between the target map and the first posture, the odometer data, and the mapping data, and also includes the correspondence between the first posture and the odometer data, and the correspondence between the first posture and the mapping data.

[0161] In the technical solution provided in the embodiment of the present application, after obtaining the target map, the odometer data corresponding to each first posture, the mapping data corresponding to each first posture, and the target map are stored accordingly. In this way, when the target map is subsequently maintained, such as deletion, modification, splicing, etc., the maintained map can still be optimized based on these data to improve the accuracy of the map.

[0162] For example, in some embodiments, in response to a delete instruction for a target map, the electronic device reads the corresponding stored first pose, the odometer data corresponding to each first pose, and the mapping data corresponding to each first pose. Afterwards, the first pose and mapping data indicated by the delete instruction are deleted from the mapping data corresponding to each first pose read, and the map consisting of the remaining first pose and mapping data is used as the target deletion map. The delete instruction can be understood as the user selecting an area in the target map and clicking the corresponding delete button to indicate the deletion of the area.

[0163] Alternatively, in some embodiments, in response to an add instruction for the target map, the electronic device reads each corresponding first pose, the odometer data corresponding to each first pose, and the mapping data corresponding to each first pose, and obtains the pose to be added indicated by the add instruction, and the mapping data to be added corresponding to each pose to be added. Afterwards, the electronic device adds the pose to be added indicated by the add instruction, and the mapping data to be added corresponding to each pose to be added to the read first poses and the mapping data corresponding to each first pose, thereby obtaining an initial augmented map.

[0164] Afterwards, according to the read first poses, the odometer data corresponding to the first poses, the mapping data corresponding to the first poses, and the acquired poses to be added and the mapping data to be added corresponding to the poses to be added, the electronic device optimizes the area outside the first area in the initial augmented map based on the odometer constraint and the visual constraint; and, according to the read first poses, the odometer data corresponding to the first poses, the mapping data corresponding to the first poses, and the acquired poses to be added and the mapping data to be added corresponding to the poses to be added, the first area in the initial augmented map is optimized based on the odometer constraint, the visual constraint and the common vision constraint, so that the optimized initial augmented map is used as the target augmented map. Alternatively, the electronic device can also perform odometer constraints and visual constraints on the initial augmented map based only on the read first poses, the odometer data corresponding to the first poses, and the mapping data corresponding to the first poses to obtain the target augmented map. Alternatively, in some embodiments, when each posture to be added and the mapping data to be added corresponding to each posture to be added are postures and mapping data obtained based on a two-dimensional code map, the electronic device can also perform odometer constraints, visual constraints, and common vision constraints on the initial augmented map based on the read first poses, the odometer data corresponding to each first pose, the mapping data corresponding to each first pose, and the acquired postures to be added, the mapping data to be added corresponding to each posture to be added, and the odometer data corresponding to each posture to be added, and then use the optimized initial augmented map as the target augmented map. Among them, the first area is the area where the mapping data to be added overlaps with the mapping data corresponding to each first pose.

[0165] In the embodiment of the present application, the posture to be added and the mapping data corresponding to each posture to be added can be understood as the posture and mapping data collected and acquired by the mobile acquisition device in other areas except the area to be collected. It can be understood that in order to enable the posture to be added and the mapping data corresponding to the posture to be added to construct a map together with the first posture and the mapping data corresponding to the first posture, the type of mapping data corresponding to the data to be added should be the same as the type of mapping data corresponding to the first posture, and the position of the sensor for collecting the mapping data on the mobile acquisition device should be the same, that is, if the mapping data corresponding to the first posture is an image frame, and the visual sensor is located directly in front of the mobile acquisition device when collecting the mapping data corresponding to the first posture, then the mapping data corresponding to the posture to be added should also be an image frame, and when collecting the mapping data corresponding to the posture to be added, the visual sensor should also be located directly in front of the mobile acquisition device.

[0166] The add instruction can be understood as the user selecting the pose to be added to the target map and the mapping data corresponding to the pose to be added, and clicking the corresponding add button, thereby indicating that the pose to be added and the mapping data corresponding to the pose to be added are added to the target map. In some embodiments, when each pose to be added and the mapping data to be added corresponding to each pose to be added are poses and mapping data obtained based on a QR code map, the add instruction can also be understood as the user selecting the data to be added to the target map and clicking the corresponding add button, wherein the data to be added includes the pose to be added, the mapping data corresponding to the pose to be added, and the odometer data corresponding to the pose to be added.

[0167] Alternatively, in some embodiments, the electronic device may first respond to a delete instruction to delete the target map, determine the first pose remaining after the deletion and the mapping data corresponding to the first pose, and then respond to an add instruction to construct an initial modified map based on the pose to be added indicated by the add instruction, the mapping data corresponding to the pose to be added, and the first pose remaining after the deletion and the mapping data corresponding to the first pose, and then optimize the initial modified map based on the odometer constraint, visual constraint and common vision constraint according to each first pose, the odometer data corresponding to each first pose, the mapping data corresponding to each first pose, the pose to be added, and the mapping data to be added corresponding to each pose to be added, to obtain a target modified map. It can be understood that based on this example, it is possible to replace a certain area in the target map.

[0168] Alternatively, in some embodiments, the electronic device may first respond to the add instruction to implement the addition of the target map to obtain the target augmented map, and then, for the target augmented map, store the pose in the target augmented map, the odometer data corresponding to the pose, the mapping data corresponding to the pose, and the target augmented map; thereafter, when the electronic device receives a delete instruction for the target augmented map, the electronic device can delete the target map according to the data stored for the target augmented map. For the specific steps, refer to the above-mentioned example of obtaining the target deleted map.

[0169] In another embodiment, after obtaining the target map, if the target map is only used for positioning, then only the target map and the mapping data corresponding to each first pose in the target map may be stored.

[0170] Combined with the above Figure 2 and Figure 6 For the case where the original map is a QR code map, the map alignment method provided in the embodiment of the present application can also be implemented in another process, see Figure 7 , Figure 7 A fifth flow chart of the map alignment method provided in the embodiment of the present application may include the following steps:

[0171] Step S71: obtaining the original map of the area to be collected;

[0172] The original map is a QR code map or a map constructed based on the first environmental feature; the same as the above step S61.

[0173] Step S72: According to the original map, control the mobile acquisition device to move to each point to be collected in the area to be collected; the same as the above step S62.

[0174] Step S73: Acquire each mapping data, the first pose, the timestamp of each mapping data, the timestamp of each first pose, and the odometer data corresponding to each first pose sent by the mobile acquisition device;

[0175] Among them, the mobile acquisition device is used to collect mapping data and obtain its own first pose, the timestamp of the mapping data, and the timestamp of the first pose every time it arrives at the point to be collected; it is also used to send the collected mapping data, the acquired first pose, the timestamp of the mapping data, and the timestamp of the first pose; please refer to the above steps S63 and S21.

[0176] Step S74: for each first pose, determine the mapping data whose timestamp matches the timestamp of the first pose as the mapping data corresponding to the first pose; see the above steps S63 and S22.

[0177] Step S75: for the mapping data corresponding to each first pose, matching the second environment feature of the mapping data with the second environment feature of the adjacent mapping data, and taking the successfully matched feature points as the second environment feature corresponding to the first pose;

[0178] The second environmental feature is of a different type from the first environmental feature; this is the same as the above step S64.

[0179] Step S76: Based on each first posture and the second environmental feature corresponding to each first posture, generate a map for recording the second environmental feature corresponding to each first posture as the map to be optimized; the same as the above step S65.

[0180] Step S77: According to the odometer data corresponding to each first pose and the mapping data corresponding to each first pose, the map to be optimized is optimized based on the odometer constraint and the visual constraint to obtain the target map. This is the same as the above step S66.

[0181] Step S78, storing the odometer data corresponding to each first pose, the mapping data corresponding to each first pose, and the target map, which is the same as the above step S67.

[0182] Below, we take the original map as a QR code map, laser map, texture map and other types of maps, and the target map as a visual map as an example. Figure 8 , the map alignment method provided in the embodiment of the present application is explained again. Figure 8 The sixth flow chart of the map alignment method provided in the embodiment of the present application may include the following steps:

[0183] Step S81: Make the robot move along the existing map to collect camera data and vehicle positioning posture data;

[0184] In the embodiment of the present application, the robot can be understood as the mobile acquisition device mentioned above; the existing map can be understood as the original map mentioned above; the camera data can be understood as the map data mentioned above; and the vehicle positioning posture data can be understood as the first posture mentioned above. The specific content of this step can be referred to the description in the above steps S12 and S13.

[0185] Step S82: aligning the image data with the vehicle posture data, and extracting feature points and descriptors of the image;

[0186] In the embodiment of the present application, aligning the image data with the vehicle posture data can be understood as determining the mapping data corresponding to each first posture; extracting the feature points and descriptors of the image can be understood as extracting the second environmental features of each mapping data. The specific content of this step can be referred to the above steps S13 and Figure 2 Description in .

[0187] Step S83: performing inter-frame feature matching to construct a visual 3D map, the coordinates of which have been aligned with the existing map;

[0188] In the embodiment of the present application, performing inter-frame feature matching can be understood as matching the second environmental features of the mapping data with the second environmental features of the adjacent mapping data for the mapping data corresponding to each first pose, and using the successfully matched feature points as the second environmental features corresponding to the first pose; for details, see the description in the above step S14. For the process of constructing a visual 3D map, see the description in the above step S15.

[0189] Step S84: For the QR code map, add odometer pose constraints to the positioning data between adjacent QR codes, and obtain more accurate pose and map data through graph optimization;

[0190] For details, please refer to the description in the above steps S45 and S46, or refer to the description in the above step S56.

[0191] Step S85: Simplify and organize the mapping data structure to obtain a mapping map for later editing and a positioning map for positioning. Please refer to the description in the above step S67.

[0192] Based on the technical solution provided in the embodiments of the present application, the following beneficial effects can be achieved:

[0193] (1) Based on the technical solution provided in the embodiments of the present application, a visual map aligned with laser maps (including two-dimensional laser, three-dimensional laser), two-dimensional code maps, texture maps and other types of maps can be obtained, which can be used for visual navigation and positioning;

[0194] (2) Based on the technical solution provided in the embodiment of the present application, the visual map can be aligned with the existing map by simply controlling the robot to move according to the existing map and collecting image frames and vehicle positioning posture data. There is no need to load cumbersome alignment calculations, translation, rotation transformation and other processes, and the implementation is relatively simple and convenient;

[0195] (3) In the technical solution provided in the embodiment of the present application, when the existing map is a QR code map, the original odometer inter-frame constraints are used to construct a graph optimization problem, so that the optimized visual map can not only ensure the position alignment on the QR code, but also make the map accuracy between the QR codes consistent;

[0196] (4) In the technical solution provided in the embodiment of the present application, since the positioning results of the existing map are directly used in the process of constructing the visual map, the obtained visual map can be strictly aligned with the existing map without introducing errors caused by human alignment.

[0197] Corresponding to the above-mentioned map alignment method, the embodiment of the present application also provides a map alignment device, such as Fig. 9 As shown, the device comprises:

[0198] A map acquisition module 91 is used to acquire an original map of the area to be collected, wherein the original map is a QR code map or a map constructed based on the first environmental feature;

[0199] The motion control module 92 is used to control the mobile acquisition device to move to each point to be collected in the area to be collected according to the original map;

[0200] The data acquisition module 93 is used to acquire the mapping data collected by the mobile acquisition device at each point to be collected, and to acquire the first position when the mobile acquisition device is located at the point to be collected, and to use the acquired mapping data as the mapping data corresponding to the acquired first position;

[0201] A feature matching module 94 is used to match the second environmental feature of the mapping data with the second environmental feature of the adjacent mapping data for the mapping data corresponding to each first pose, and use the successfully matched feature points as the second environmental feature corresponding to the first pose, wherein the second environmental feature is of a different type from the first environmental feature;

[0202] The map generation module 95 is used to generate a map for recording the second environmental features corresponding to each first posture according to each first posture and the second environmental features corresponding to each first posture, as a target map of the area to be collected.

[0203] In the technical solution provided in the embodiment of the present application, the original map is a QR code map or a map constructed based on the first environmental feature. After the original map of the area to be collected is obtained, the mobile collection device is controlled to move to each point to be collected in the area to be collected according to the original map. For each point to be collected, the mapping data collected by the mobile collection device at the point to be collected is obtained, and the first position of the mobile collection device when it is located at the point to be collected is obtained, and the obtained mapping data is used as the mapping data corresponding to the obtained first position; further, for the mapping data corresponding to each first position, the second environmental feature of the mapping data is matched with the second environmental feature of the adjacent mapping data, and the feature point with successful matching is used as the second environmental feature corresponding to the first position; thus, according to each first position and the second environmental feature corresponding to each first position, a map for recording the second environmental feature corresponding to each first position can be generated as the target map of the area to be collected.

[0204] It can be understood that in the embodiment of the present application, the target map is constructed based on the first posture and the second environmental feature corresponding to the first posture, and the first posture is the posture of the mobile acquisition device itself acquired when it is located at each point to be collected, and the second environmental feature corresponding to the first posture is obtained by feature matching the second environmental feature of the mapping data corresponding to the first posture and the second environmental feature of the adjacent mapping data of the mapping data, and the mobile acquisition device moves to each point to be collected according to the original map. Therefore, the target map determined based on the first posture and the second environmental feature corresponding to the first posture is a map aligned with the original map, and the original map is a QR code map or a map constructed based on the first environmental feature, and the second environmental feature is of a different type from the first environmental feature.

[0205] It can be seen that by applying the technical solution provided in the embodiment of the present application, when the original map is a QR code map or a map constructed based on the first environmental feature, a target map aligned with the original map can be directly constructed; and the technical solution provided in the embodiment of the present application does not require complex alignment calculations and translation and rotation transformations, and is relatively simple to implement and the calculation process is relatively simple. Therefore, when the target map to be constructed is a visual map, a visual map aligned with the original map can be directly constructed according to the technical solution provided in the embodiment of the present application, and the implementation is relatively simple and the calculation process is relatively simple.

[0206] In some embodiments, the data acquisition module 93 is specifically used to acquire each mapping data, the first pose, and the timestamp of each mapping data and the timestamp of each first pose sent by the mobile acquisition device, wherein the mobile acquisition device is used to acquire mapping data and acquire its own first pose, the timestamp of the mapping data, and the timestamp of the first pose every time it arrives at the point to be acquired; and is also used to send the acquired mapping data, the acquired first pose, the timestamp of the mapping data, and the timestamp of the first pose;

[0207] For each of the first postures, mapping data having a timestamp that matches the timestamp of the first posture is determined as the mapping data corresponding to the first posture.

[0208] In some embodiments, the original map is a QR code map, and the second environmental feature is a laser feature, or a texture feature, or a visual feature;

[0209] The first environmental feature is a laser feature, and the second environmental feature is a texture feature or a visual feature;

[0210] The first environmental feature is a texture feature, and the second environmental feature is a laser feature or a visual feature.

[0211] In some embodiments, the original map is a QR code map, and the mobile acquisition device carries an odometer.

[0212] The data acquisition module 93 is also used to acquire the odometer data of the odometer when the mobile collection device is located at the point to be collected for each point to be collected;

[0213] A map generation module 95 is specifically used to generate a map for recording the second environmental features corresponding to each of the first postures according to each of the first postures and the second environmental features corresponding to each of the first postures as a map to be optimized;

[0214] According to the odometer data corresponding to each of the first postures and the mapping data corresponding to each of the first postures, the map to be optimized is optimized based on odometer constraints and visual observation constraints to obtain the target map.

[0215] In some embodiments, optimizing the map to be optimized to obtain a target map includes:

[0216] The position and posture of the second posture involved in the map to be optimized are optimized, and the posture of the third posture involved in the map to be optimized is optimized, and the optimized map to be optimized is used as the target map of the area to be collected, wherein the second posture is the posture calculated by the mobile collection device based on the odometer, and the third posture is the posture obtained by the mobile collection device scanning the QR code.

[0217] In some embodiments, each of the first postures, the odometer data corresponding to each of the first postures, the mapping data corresponding to each of the first postures, and the target map are stored accordingly, and the device further includes:

[0218] A map deletion module, configured to respond to a deletion instruction for the target map, read the corresponding first postures and the mapping data corresponding to the first postures; delete the first postures and the mapping data indicated by the deletion instruction from the read first postures and the mapping data corresponding to the first postures, to obtain a target deleted map; and / or,

[0219] A map augmentation module, for responding to an add instruction for the target map, reading the corresponding stored first postures, the odometer data corresponding to the first postures, and the mapping data corresponding to the first postures, and obtaining the postures to be added indicated by the add instruction and the mapping data to be added corresponding to the postures to be added; adding the postures to be added indicated by the add instruction and the mapping data to be added corresponding to the postures to be added to the read first postures and the mapping data corresponding to the first postures to obtain an initial augmented map; optimizing the area outside the first area in the initial augmented map based on the odometer constraint and the visual constraint according to the read first postures, the odometer data corresponding to the first postures, the mapping data corresponding to the first postures, and the acquired postures to be added and the mapping data to be added corresponding to the postures to be added; and optimizing the first area in the initial augmented map based on the odometer constraint, the visual constraint and the common view constraint to obtain a target augmented map; wherein the first area is an area where the mapping data to be added overlaps with the mapping data corresponding to the first postures.

[0220] The present application also provides an electronic device, such as Fig.10 As shown, including:

[0221] Memory 101, used for storing computer programs;

[0222] The processor 102 is used to implement any of the above-mentioned map alignment methods when executing the program stored in the memory 101.

[0223] Furthermore, the electronic device may further include a communication bus and / or a communication interface, and the processor 102, the communication interface, and the memory 101 communicate with each other via the communication bus.

[0224] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0225] The communication interface is used for communication between the above electronic device and other devices.

[0226] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0227] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0228] In another embodiment provided in the present application, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned map alignment methods are implemented.

[0229] In another embodiment provided by the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any map alignment method in the above embodiments.

[0230] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a solid-state hard disk (SSD), etc.

[0231] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0232] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for devices, electronic devices, storage media, and computer program products, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0233] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. A map alignment method, characterized in that: The method comprises: Obtaining an original map of the area to be collected, wherein the original map is a QR code map or a map constructed based on the first environmental feature; According to the original map, the mobile collection device is controlled to move to each point to be collected in the area to be collected; For each of the points to be collected, the mapping data collected by the mobile collection device at the point to be collected is obtained, and the first position of the mobile collection device when it is located at the point to be collected is obtained, and the obtained mapping data is used as the mapping data corresponding to the obtained first position; For the mapping data corresponding to each first posture, matching the second environmental feature of the mapping data with the second environmental feature of the adjacent mapping data, and taking the successfully matched feature points as the second environmental feature corresponding to the first posture, wherein the second environmental feature is of a different type from the first environmental feature; According to each of the first postures and the second environmental features corresponding to each of the first postures, a map for recording the second environmental features corresponding to each of the first postures is generated as a target map of the area to be collected.

2. The method according to claim 1, characterized in that The step of obtaining the mapping data collected by the mobile acquisition device at the point to be collected, and obtaining the first pose of the mobile acquisition device when the mobile acquisition device is located at the point to be collected, and using the obtained mapping data as mapping data corresponding to the obtained first pose, includes: Acquire each mapping data, the first pose, and the timestamp of each mapping data and the timestamp of each first pose sent by the mobile acquisition device, wherein the mobile acquisition device is used to collect mapping data and obtain its own first pose, the timestamp of the mapping data, and the timestamp of the first pose every time it arrives at the point to be collected; and is also used to send the collected mapping data, the obtained first pose, the timestamp of the mapping data, and the timestamp of the first pose; For each of the first postures, mapping data having a timestamp that matches the timestamp of the first posture is determined as the mapping data corresponding to the first posture.

3. The method according to claim 1, characterized in that The original map is a two-dimensional code map, and the second environmental feature is a laser feature, or a texture feature, or a visual feature; The first environmental feature is a laser feature, and the second environmental feature is a texture feature or a visual feature; The first environmental feature is a texture feature, and the second environmental feature is a laser feature or a visual feature.

4. The method according to claim 1, characterized in that The original map is a QR code map, and the mobile acquisition device carries an odometer. The method further includes: for each of the points to be collected, obtaining the odometer data of the odometer when the mobile collection device is located at the point to be collected; The step of generating a map for recording the second environmental features corresponding to each of the first postures according to each of the first postures and the second environmental features corresponding to each of the first postures as a target map of the area to be collected includes: According to each of the first postures and the second environment features corresponding to each of the first postures, generating a map for recording the second environment features corresponding to each of the first postures as a map to be optimized; According to the odometer data corresponding to each of the first postures and the mapping data corresponding to each of the first postures, the map to be optimized is optimized based on odometer constraints and visual observation constraints to obtain the target map.

5. The method according to claim 4, characterized in that The step of optimizing the map to be optimized to obtain the target map includes: The position and posture of the second posture involved in the map to be optimized are optimized, and the posture of the third posture involved in the map to be optimized is optimized, and the optimized map to be optimized is used as the target map of the area to be collected, wherein the second posture is the posture calculated by the mobile collection device based on the odometer, and the third posture is the posture obtained by the mobile collection device scanning the QR code.

6. The method according to claim 4, characterized in that Each of the first postures, the odometer data corresponding to each of the first postures, the mapping data corresponding to each of the first postures, and the target map are stored accordingly, and the method further includes: In response to a deletion instruction for the target map, reading the corresponding first postures and the mapping data corresponding to the first postures; deleting the first postures and the mapping data indicated by the deletion instruction from the read first postures and the mapping data corresponding to the first postures, to obtain a target deleted map; and / or, In response to an add instruction for the target map, the corresponding first postures, the odometer data corresponding to the first postures, and the mapping data corresponding to the first postures are read, and the postures to be added indicated by the add instruction and the mapping data to be added corresponding to the postures to be added are obtained; the postures to be added indicated by the add instruction and the mapping data to be added corresponding to the postures to be added are added to the first postures read and the mapping data corresponding to the first postures, so as to obtain an initial augmented map; according to the first postures read, the odometer data corresponding to the first postures, the mapping data corresponding to the first postures, and the acquired postures to be added and the mapping data to be added corresponding to the postures to be added, based on the odometer constraints and the visual constraints, the area outside the first area in the initial augmented map is optimized; and the first area in the initial augmented map is optimized based on the odometer constraints, the visual constraints and the common vision constraints to obtain the target augmented map; wherein the first area is an area where the mapping data to be added overlaps with the mapping data corresponding to the first postures.

7. A map alignment device, characterized in that: The device comprises: A map acquisition module, used to acquire an original map of the area to be collected, wherein the original map is a QR code map or a map constructed based on the first environmental feature; A motion control module, used to control the mobile acquisition device to move to each point to be collected in the area to be collected according to the original map; A data acquisition module is used to acquire, for each of the points to be collected, the mapping data collected by the mobile acquisition device at the point to be collected, and to acquire the first position when the mobile acquisition device is located at the point to be collected, and to use the acquired mapping data as the mapping data corresponding to the acquired first position; A feature matching module is used to match the second environmental feature of the mapping data corresponding to each first posture with the second environmental feature of the adjacent mapping data, and use the successfully matched feature points as the second environmental feature corresponding to the first posture, wherein the second environmental feature is of a different type from the first environmental feature; A map generation module is used to generate a map for recording the second environmental features corresponding to each of the first postures according to each of the first postures and the second environmental features corresponding to each of the first postures, as a target map of the area to be collected.

8. The device according to claim 7, characterized in that The data acquisition module is specifically used to: acquire each mapping data, the first pose, and the timestamp of each mapping data and the timestamp of each first pose sent by the mobile acquisition device, wherein the mobile acquisition device is used to acquire mapping data and acquire its own first pose, the timestamp of the mapping data, and the timestamp of the first pose every time it arrives at the point to be acquired; it is also used to send the acquired mapping data, the acquired first pose, the timestamp of the mapping data, and the timestamp of the first pose; for each first pose, determine the mapping data whose timestamp matches the timestamp of the first pose as the mapping data corresponding to the first pose; and / or The original map is a two-dimensional code map, the second environmental feature is a laser feature, or a texture feature, or a visual feature; the first environmental feature is a laser feature, and the second environmental feature is a texture feature, or a visual feature; the first environmental feature is a texture feature, and the second environmental feature is a laser feature, or a visual feature; and / or, The original map is a QR code map, the mobile acquisition device carries an odometer, and the data acquisition module is further used to obtain the odometer data of the odometer when the mobile acquisition device is located at the point to be collected for each point to be collected; the map generation module is further used to: generate a map for recording the second environmental features corresponding to each first posture according to each first posture and the second environmental features corresponding to each first posture, as a map to be optimized; optimize the map to be optimized based on the odometer constraints and visual observation constraints according to the odometer data corresponding to each first posture and the mapping data corresponding to each first posture, to obtain the target map; and / or, The optimizing the map to be optimized to obtain the target map includes: optimizing the position and posture in the second posture involved in the map to be optimized, optimizing the posture in the third posture involved in the map to be optimized, and using the optimized map to be optimized as the target map of the area to be collected, wherein the second posture is the posture calculated by the mobile collection device based on the odometer, and the third posture is the posture obtained by the mobile collection device scanning the QR code; and / or, Each of the first postures, the odometer data corresponding to each of the first postures, the mapping data corresponding to each of the first postures, and the target map are stored accordingly, and the device further includes: A map deletion module, configured to respond to a deletion instruction for the target map, read the corresponding first postures and the mapping data corresponding to the first postures; delete the first postures and the mapping data indicated by the deletion instruction from the read first postures and the mapping data corresponding to the first postures, to obtain a target deleted map; and / or, A map augmentation module, for responding to an add instruction for the target map, reading the corresponding stored first postures, the odometer data corresponding to the first postures, and the mapping data corresponding to the first postures, and obtaining the postures to be added indicated by the add instruction and the mapping data to be added corresponding to the postures to be added; adding the postures to be added indicated by the add instruction and the mapping data to be added corresponding to the postures to be added to the read first postures and the mapping data corresponding to the first postures to obtain an initial augmented map; optimizing the area outside the first area in the initial augmented map based on the odometer constraint and the visual constraint according to the read first postures, the odometer data corresponding to the first postures, the mapping data corresponding to the first postures, and the acquired postures to be added and the mapping data to be added corresponding to the postures to be added; and optimizing the first area in the initial augmented map based on the odometer constraint, the visual constraint and the common view constraint to obtain a target augmented map; wherein the first area is an area where the mapping data to be added overlaps with the mapping data corresponding to the first postures.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, for implementing any of the methods described in claims 1-6 when executing a program stored in a memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images