A method and system for mapping and positioning of visual fusion laser

Abstract

This invention provides a method and system for mapping and localization of laser systems using visual fusion. The method includes: S1, acquiring first pose data from a laser device and first visual data from a vision device; constructing and updating a visual map based on the first scan data and the first visual data; S2, acquiring second scan data from the laser device and second visual data from the vision device, and performing joint localization based on the second pose data, the second visual data, and the visual map. The solution of this invention requires significantly less computational power than existing methods, is easier to deploy, and can improve the success rate of existing laser localization methods.

Description

Technical Field

[0001] This invention relates to the field of robotics, and more specifically, to a method, system, electronic device, and computer storage medium for vision and laser fusion mapping and localization. Background Technology

[0002] Existing visual mapping and localization technologies require the construction of 3D point cloud models online or offline, which demands high computing power and involves complex deployment. Existing laser mapping and localization technologies are prone to localization failures under conditions such as large initial positional deviations, high pedestrian traffic, and changes in the structural environment, making them difficult to meet practical needs. Summary of the Invention

[0003] In order to at least solve the technical problems existing in the background art, the present invention provides a method, system, electronic device and computer storage medium for vision and laser fusion mapping and positioning, so as to reduce computing power requirements and deployment difficulty, while improving the success rate of existing laser positioning.

[0004] The first aspect of the present invention provides a method for mapping and localization using visual fusion lasers, applied to robots, the method comprising the following steps:

[0005] S1, acquire the first pose data of the laser device and the first visual data of the vision device; construct and update the visual map based on the first pose data and the first visual data;

[0006] S2, acquire the second pose data of the laser device and the second visual data of the vision device, and perform joint positioning based on the second pose data, the second visual data and the visual map.

[0007] Further, in step S1, constructing and updating the visual map based on the first pose data and the first visual data includes:

[0008] S11, associate the first pose data and the first vision data;

[0009] S12, determine the current grid cell of the robot based on the first pose data, and store the first vision data in the current grid cell.

[0010] Further, in step S12, the current grid cell of the robot is determined based on the first pose data, and the first visual data is stored in the current grid cell, including:

[0011] S121, determine whether there is already visual data in the current grid cell; if yes, go to S122, otherwise go to S123;

[0012] S122, Update the visual data using the first visual data;

[0013] S123, add the first visual data to the raster.

[0014] Furthermore, the raster stores the global feature point descriptor of the image, the local feature point descriptor of the image, and the first pose data associated with the first visual data.

[0015] Further, in step S122, updating the visual data using the first visual data includes:

[0016] S1221, determine if the covariance of the first pose data is less than 1. If so, proceed to S1222.

[0017] Otherwise, proceed to S1226;

[0018] S1222, determine whether the current first visual data detects a loop closure; if yes, go to S1223, otherwise go to S1227;

[0019] S1223, Determine whether the difference between the current first pose data and the third pose data of the loop closure is greater than the threshold. If yes, go to S1224; otherwise, go to S1225.

[0020] S1224, determine that the current first visual data and the loopback third visual data are not in the same position, and store the first visual data in the grid;

[0021] S1225, determine that the current first visual data loopback is successful, and update the timestamp of the first visual data in the grid to the current time;

[0022] S1226, determine that the current first pose data is unreliable, and do not add the first vision data to the grid;

[0023] S1227, Determine that the current first visual data is an image that is not in the raster, and add it to the raster.

[0024] Further, in step S2, joint localization is performed based on the second pose data, the second visual data, and the visual map, including:

[0025] S21, extract the visual pose from the second visual data and calculate the confidence level of the visual pose;

[0026] S22, if the confidence level meets the threshold condition, use the visual pose as the initial value to perform matching calculation between the second pose data and the visual pose.

[0027] S23. When the matching calculation result meets the threshold condition, perform backend graph optimization.

[0028] Further, in step S21, the visual pose is extracted from the second visual data, and the confidence level of the visual pose is calculated, including:

[0029] S211, extract the global feature point descriptor and local feature point descriptor of the second visual data, determine several candidate image frames based on the global feature point descriptor, and calculate the feature point matching rate of the local feature point descriptor for each candidate image frame.

[0030] S212, If the feature point matching rate meets the threshold condition, then calculate the Essential matrix and the epipolar point rate.

[0031] S213, decompose the Essential matrix to recover the true rotation of the visual device;

[0032] S214, when there are three-dimensional feature points in the current grid map, calculate the visual pose based on the three-dimensional information of the three-dimensional feature points and PnP, and calculate the confidence level of the visual pose based on the feature point matching rate, epipolar in-point rate and global feature point descriptor distance.

[0033] A second aspect of the present invention provides a mapping and positioning system for visual fusion laser, including a mapping module and a positioning module;

[0034] The mapping module is used to acquire the first pose data of the laser device and the first visual data of the vision device; and to construct and update the visual map based on the first pose data and the first visual data.

[0035] The positioning module is used to acquire the second pose data of the laser device and the second visual data of the vision device, and to perform joint positioning based on the second pose data, the second visual data and the visual map.

[0036] A third aspect of the present invention provides an electronic device, comprising: a memory storing executable program code; a processor coupled to the memory; the processor calling the executable program code stored in the memory to perform the method as described in any of the preceding claims.

[0037] A fourth aspect of the present invention provides a computer storage medium storing a computer program that, when executed by a processor, performs the method described in any of the preceding claims.

[0038] The beneficial effects of this invention are as follows:

[0039] 1) Existing visual relocalization systems require the pre-establishment of a complete visual map, and the process involves significant computational resources for visual pose optimization, map feature point optimization, and overall image optimization. In contrast, this system utilizes laser-derived visual poses and optimizes the visual map along with surrounding images, thus limiting resource consumption and enabling efficient operation on robots.

[0040] 2) Existing visual relocalization methods use static, unchanging map information. This system's visual map is updated in real-time, enabling it to handle dynamic scenarios and thus ensuring continued effectiveness over extended periods. Attached Figure Description

[0041] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a schematic diagram of the main process of a visual fusion laser mapping and positioning method disclosed in an embodiment of the present invention;

[0043] Figure 2-4 This is a schematic diagram of the process of constructing and updating a visual map as disclosed in an embodiment of the present invention;

[0044] Figure 5-6 This is a schematic diagram of the joint positioning process disclosed in an embodiment of the present invention;

[0045] Figure 7 This is a schematic diagram of the structure of a vision fusion laser mapping and positioning system disclosed in an embodiment of the present invention;

[0046] Figure 8 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of the present invention. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0048] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the application. The singular forms "a," "the," and "the" as used in the embodiments of this application and the appended claims are also intended to include the plural forms, and "multiple" generally includes at least two unless the context clearly indicates otherwise.

[0049] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0050] It should be understood that although the terms first, second, third, etc., may be used to describe ... in the embodiments of this application, these ... should not be limited to these terms. These terms are only used to distinguish .... For example, without departing from the scope of the embodiments of this application, first ... can also be referred to as second ..., and similarly, second ... can also be referred to as first ....

[0051] Depending on the context, the words “if” or “suppose” as used here can be interpreted as “when” or “in response to determination” or “in response to detection.” Similarly, depending on the context, the phrases “if determination” or “if detection (of the stated condition or event)” can be interpreted as “when determination” or “in response to determination” or “when detection (of the stated condition or event)” or “in response to detection (of the stated condition or event).”

[0052] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a product or system comprising a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a product or system. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the product or system that includes that element.

[0053] The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0054] Please see Figure 1 , Figure 1 This is a schematic flowchart of a visual fusion laser mapping and positioning method disclosed in an embodiment of the present invention. Figure 1 As shown in the figure, a visual fusion laser mapping and localization method according to an embodiment of the present invention is applied to a robot, and the method includes the following steps:

[0055] S1, acquire the first pose data of the laser device and the first visual data of the vision device; construct and update the visual map based on the first pose data and the first visual data;

[0056] S2, acquire the second pose data of the laser device and the second visual data of the vision device, and perform joint positioning based on the second pose data, the second visual data and the visual map.

[0057] In this embodiment of the invention, a combined visual and laser mapping and localization approach is employed to improve the success rate of laser localization. Specifically, under laser scanning or normal localization conditions, a vision-based image database and map file are constructed and optimized from scratch. Then, with an existing visual map, combined visual and laser localization is performed to obtain the robot's pose information within the grid map, enabling localization initialization or combined pose optimization, thereby improving the success rate of the original laser localization. The visual mapping process of this invention is completed in real-time during robot task execution and the map is updated during subsequent repetitive tasks to enhance adaptability to changes in lighting and environment.

[0058] The application of this invention is to robots, such as AGVs, intelligent vehicles, and legged dogs (e.g., bipedal and quadrupedal), which are electromechanical devices capable of self-movement or semi-self-movement. The robot is equipped with laser and vision devices, supplemented by hardware modules such as drive devices, steering control devices, and communication devices. This enables the visual fusion laser mapping and positioning of the robot, thereby achieving safe movement within a given environment.

[0059] Further, see Figure 2 As shown, in step S1, constructing and updating the visual map based on the first pose data and the first visual data includes:

[0060] S11, associate the first pose data and the first vision data;

[0061] S12, determine the current grid cell of the robot based on the first pose data, and store the first vision data in the current grid cell.

[0062] In this embodiment of the invention, the robot's pose (x, y, yaw) can be pre-divided into grids with a fixed resolution. Based on the pose provided by the laser, the current grid position of the robot is determined, and the corresponding visual information is stored in the corresponding grid. By repeating this process, a visual map can be constructed.

[0063] The robot's pose is determined based on the first pose data. Since this is a mature existing technology, the specific determination method will not be elaborated further.

[0064] Furthermore, referring to Figure 3 As shown, in step S12, the current grid cell of the robot is determined based on the first pose data, and the first visual data is stored in the current grid cell, including:

[0065] S121, determine whether there is already visual data in the current grid cell; if yes, go to S122, otherwise go to S123;

[0066] S122, Update the visual data using the first visual data;

[0067] S123, add the first visual data to the raster.

[0068] In this embodiment of the invention, during the robot's movement in the scene, it can also determine whether visual data has already been stored in the current grid. If it has, the old data is updated using the currently acquired first visual data; if not, the currently acquired first visual data can be directly added to the grid. By repeating this process, a visual map can be constructed and continuously updated.

[0069] Furthermore, the raster stores the global feature point descriptor of the image, the local feature point descriptor of the image, and the first pose data associated with the first visual data.

[0070] In this embodiment of the invention, the grid stores global and local feature point descriptors of the image extracted from visual data. In addition, the first pose data associated with the visual data is also stored in the corresponding grid.

[0071] Furthermore, referring to Figure 4 As shown, in step S122, updating the visual data using the first visual data includes:

[0072] S1221, determine whether the covariance of the first pose data is less than 1. If yes, go to S1222; otherwise, go to S1226.

[0073] S1222, determine whether the current first visual data detects a loop closure; if yes, go to S1223, otherwise go to S1227;

[0074] S1223, Determine whether the difference between the current first pose data and the loop closure third pose data is greater than the threshold (displacement 10m or rotation 180°). If yes, go to S1224; otherwise, go to S1225.

[0075] S1224, determine that the current first visual data and the loopback third visual data are not in the same position, and store the first visual data in the grid;

[0076] S1225, determine that the current first visual data loopback is successful, and update the timestamp of the first visual data in the grid to the current time;

[0077] S1226, determine that the current first pose data is unreliable, and do not add the first vision data to the grid;

[0078] S1227, Determine that the current first visual data is an image that is not in the raster, and add it to the raster.

[0079] In this embodiment of the invention, the visual data stored in the grid is preferably an image. Of course, the visual data can also be stored using a feature matrix, and this invention does not exclude this method.

[0080] Furthermore, referring to Figure 5 As shown, in step S2, joint localization is performed based on the second pose data, the second visual data, and the visual map, including:

[0081] S21, extract the visual pose from the second visual data and calculate the confidence level of the visual pose;

[0082] S22, if the confidence level meets the threshold condition, use the visual pose as the initial value to perform matching calculation between the second pose data and the visual pose.

[0083] S23. When the matching calculation result meets the threshold condition, perform backend graph optimization.

[0084] Furthermore, referring to Figure 6 As shown, in step S21, the visual pose is extracted from the second visual data, and the confidence level of the visual pose is calculated, including:

[0085] S211, extract the global feature point descriptor and local feature point descriptor of the second visual data (which can be extracted through a deep neural network), determine several (e.g., 5) candidate image frames based on the global feature point descriptor, and calculate the feature point matching rate of the local feature point descriptor for each candidate image frame.

[0086] S212, if the feature point matching rate meets the threshold condition (e.g., 0.2), then calculate the Essential matrix (calculated based on the local feature point descriptor matching results and camera intrinsic parameters), and calculate the epipolar in-point rate;

[0087] S213, Decompose the Essential matrix to recover the true rotation of the visual device (decomposing the Essential matrix can yield a relative rotation, which, when multiplied by the rotation in the map, gives the true rotation of the camera).

[0088] S214, when there are three-dimensional feature points in the current grid map, calculate the visual pose based on the three-dimensional information of the three-dimensional feature points (X, Y, Z coordinates of the feature points) and PnP, and calculate the confidence level of the visual pose based on the feature point matching rate, the epipolar in-point rate and the global feature point descriptor distance.

[0089] In this embodiment of the invention, 2D-2D constraints are constructed using epipolar geometry to solve for camera rotation, and image positions in the map database are used as the current camera displacement. Given the existence of adjacent images in the map, 3D-2D constraints are constructed using binocular feature point triangulation, and rotation and displacement are solved simultaneously using PNP. Finally, the confidence score for visual localization is calculated based on the obtained pose. To accelerate computation, feature point descriptors are quantized using int8, and descriptor matching is accelerated using the SSE / NEON instruction set.

[0090] The confidence score for visual pose is composed of the feature point matching rate, the epipolar in-line rate, and the global feature point descriptor distance of the image. The calculation method is as follows, with values ​​in the formula adjusted empirically. The final output confidence score ranges from 0 to 1; the closer to 1, the more reliable the result.

[0091]

[0092] In the formula, confidence is the confidence level of the visual pose; match_point is the feature point matching rate; inlier_point is the inlier rate; and global_distance_point is the global feature point descriptor distance of the image.

[0093] The values ​​of match_ratio and inlier_ratio are between 0 and 1, with values ​​closer to 1 being better. The global_distance is between 0 and 2, with values ​​closer to 0 indicating that the two images are more similar.

[0094] It should be noted that the epipolar in-line ratio is calculated based on the Essential matrix. After obtaining the Essential matrix, the pixel distance can be used to calculate which points in the feature point matching results are in-line points and which are out-line points. Epipolar in-line ratio = number of epipolar in-line points matching feature points / total number of all matching feature points.

[0095] Please see Figure 7 , Figure 7 This is a schematic diagram of the structure of a visual fusion laser mapping and positioning system disclosed in an embodiment of the present invention. Figure 7 As shown, a visual fusion laser mapping and positioning system according to an embodiment of the present invention includes a mapping module (101) and a positioning module (102);

[0096] The mapping module (101) is used to acquire the first pose data of the laser device and the first visual data of the vision device; and to construct and update the visual map based on the first pose data and the first visual data.

[0097] The positioning module (102) is used to acquire the second pose data of the laser device and the second visual data of the vision device, and to perform joint positioning based on the second pose data, the second visual data and the visual map.

[0098] The specific functions of the visual fusion laser mapping and positioning system in this embodiment are the same as those in Embodiment 1 above. Since the system in this embodiment adopts all the technical solutions of the above embodiments, it has at least all the beneficial effects brought about by the technical solutions of the above embodiments, which will not be repeated here.

[0099] Please see Figure 8 , Figure 8 This invention discloses an electronic device comprising: a memory storing executable program code; a processor coupled to the memory; and the processor calling the executable program code stored in the memory to execute the method as described in any of the foregoing embodiments.

[0100] This invention also discloses a computer storage medium storing a computer program, which is executed by a processor to perform the method as described in any of the foregoing embodiments.

[0101] The technical solutions of this disclosure can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes one or more instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of this disclosure. The aforementioned storage medium can be a non-transitory storage medium, including: a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and other media capable of storing program code; it can also be a transient storage medium.

[0102] The foregoing description and accompanying drawings fully illustrate embodiments of this disclosure to enable those skilled in the art to practice them. Other embodiments may include structural, logical, electrical, procedural, and other changes. The embodiments represent only possible variations. Individual components and functions are optional unless explicitly required, and the order of operation may vary. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. Moreover, the terminology used in this application is for describing embodiments only and is not intended to limit the claims. As used in the description of embodiments and claims, the singular forms “a,” “an,” and “the” are intended to equally include the plural forms unless the context clearly indicates otherwise. Similarly, the term “and / or” as used in this application means including one or more of the associated listed items and all possible combinations thereof. Additionally, when used in this application, the term "comprise" and its variations "comprises" and / or "comprising" refer to the presence of stated features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof. Without further limitations, an element defined by the phrase "comprises a..." does not exclude the presence of other identical elements in the process, method, or apparatus that includes the element. In this document, each embodiment may focus on the differences from other embodiments, and similar or identical parts between embodiments can be referred to mutually. For methods, products, etc., disclosed in the embodiments, if they correspond to the method section disclosed in the embodiments, the relevant parts can be referred to the description of the method section.

[0103] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this disclosure. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0104] The methods and products (including but not limited to devices and equipment) disclosed in the embodiments herein can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units may be merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms. Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to implement this embodiment according to actual needs. Furthermore, the functional units in the embodiments of this disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0105] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than that shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. In the descriptions corresponding to the flowcharts and block diagrams in the accompanying drawings, the operations or steps corresponding to different blocks may also occur in a different order than disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. Each block in a block diagram and / or flowchart, and combinations of blocks in a block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

Claims

1. A visual-fusion laser mapping and localization method, applied to robots, characterized in that, The method includes the following steps: S1, acquire the first pose data of the laser device and the first visual data of the vision device; construct and update the visual map based on the first pose data and the first visual data; S2, acquire the second pose data of the laser device and the second visual data of the vision device, and perform joint positioning based on the second pose data, the second visual data and the visual map; Step S1 includes: S11, associate the first pose data and the first visual data; S12, determine the current grid of the robot based on the first pose data, and store the first visual data in the current grid; Step S12 includes: S121, determine whether there is already visual data in the current grid; if yes, proceed to S122, otherwise proceed to S123; S122, Update the visual data using the first visual data; S123, add the first visual data to the grid; Step S122 includes: S1221, Determine whether the covariance of the first pose data is less than 1. If yes, proceed to S1222; otherwise, proceed to S1226. S1222, determine whether the current first visual data detects a loop closure; if yes, proceed to S1223, otherwise proceed to S1227; S1223, Determine whether the difference between the current first pose data and the loop closure third pose data is greater than the threshold. If yes, proceed to S1224; otherwise, proceed to S1225. S1224, determine that the current first visual data and the loopback third visual data are not in the same position, and store the first visual data in the grid; S1225, determine that the current first visual data loopback is successful, and update the timestamp of the first visual data in the grid to the current time; S1226, determine that the current first pose data is unreliable, and do not add the first visual data to the grid; S1227, determine that the current first visual data is an image that is not in the grid, and add it to the grid.

2. The method for mapping and positioning lasers based on visual fusion according to claim 1, characterized in that: The grid stores global feature point descriptors, local feature point descriptors, and the first pose data associated with the first visual data.

3. The method for mapping and positioning lasers based on visual fusion according to claim 1, characterized in that: In step S2, the joint localization based on the second pose data, the second visual data, and the visual map includes: S21, extract the visual pose from the second visual data and calculate the confidence level of the visual pose; S22, when the confidence level meets the threshold condition, the visual pose is used as the initial value to perform matching calculation between the second pose data and the visual pose; S23, when the matching calculation result meets the threshold condition, perform backend graph optimization.

4. The method for mapping and positioning lasers based on visual fusion according to claim 3, characterized in that: In step S21, the visual pose is extracted from the second visual data, and the confidence level of the visual pose is calculated, including: S211, extract the global feature point descriptor and local feature point descriptor of the second visual data, determine several candidate image frames based on the global feature point descriptor, and calculate the feature point matching rate of the local feature point descriptor for each candidate image frame. S212, If the feature point matching rate meets the threshold condition, then calculate the Essential matrix and the epipolar point rate. S213, Decompose the Essential matrix to recover the true rotation; S214, when there are three-dimensional feature points in the current grid map, calculate the visual pose based on the three-dimensional information of the three-dimensional feature points and PnP, and calculate the confidence level of the visual pose based on the feature point matching rate, the epipolar in-point rate and the global feature point descriptor distance.

5. A visual fusion laser mapping and positioning system, comprising a mapping module and a positioning module; The mapping module is used to acquire the first pose data of the laser device and the first visual data of the vision device; Construct and update a visual map based on the first pose data and the first visual data; The positioning module is used to acquire the second pose data of the laser device and the second visual data of the vision device, and perform joint positioning based on the second pose data, the second visual data and the visual map; The mapping module includes a data association submodule and a data storage submodule; The data association submodule is used to associate the first pose data and the first visual data; The data storage submodule is used to determine the current grid of the robot based on the first pose data, and store the first visual data in the current grid. The data storage submodule includes a visual data judgment unit, a visual data update unit, and a visual data addition unit. The visual data judgment unit is used to determine whether there is already visual data in the current grid; if so, it switches to the visual data update unit; otherwise, it switches to the visual data addition unit. The visual data update unit is used to update the visual data using the first visual data; The visual data adding unit is used to add the first visual data to the grid; The visual data update unit includes a first judgment subunit, a second judgment subunit, a third judgment subunit, a visual data storage subunit, a timestamp update subunit, a visual data exclusion subunit, and a visual data addition subunit. The first judgment subunit is used to determine whether the covariance of the first pose data is less than 1. If it is, it is transferred to the second judgment subunit; otherwise, it is transferred to the visual data exclusion subunit. The second judgment subunit is used to determine whether the current first visual data detects a loop closure; if so, it proceeds to the third judgment subunit; otherwise, it proceeds to the visual data addition subunit. The third judgment subunit is used to determine whether the difference between the current first pose data and the loop-closed third pose data is greater than a threshold. If so, it is transferred to the visual data storage subunit; otherwise, it is transferred to the timestamp update subunit. A visual data storage subunit is used to determine that the current first visual data and the loopback third visual data are not in the same position, and to store the first visual data in the grid. The timestamp update subunit is used to determine that the current first visual data loopback is successful and to update the timestamp of the first visual data in the grid to the current time. The visual data exclusion subunit is used to determine that the current first pose data is unreliable and not to add the first visual data to the grid. A visual data addition subunit is used to determine if the current first visual data is an image that is not in the grid, and to add it to the grid.

6. An electronic device, comprising: Memory containing executable program code; A processor coupled to the memory; The characteristic feature is that the processor calls the executable program code stored in the memory to execute the method as described in any one of claims 1-4.

7. A computer storage medium storing a computer program, characterized in that: The computer program is executed by the processor to perform the method as described in any one of claims 1-4.

Images