Lidar-based mapping method and apparatus, intelligent device, and storage medium

Abstract

The application relates to the technical field of map construction, in particular to a mapping method and device based on a laser radar, an intelligent device and a storage medium. The mapping method comprises the following steps: acquiring point cloud data, wherein the point cloud data comprises first point cloud data, the first point cloud data is acquired by a first laser radar arranged at a first position of an intelligent device; the first point cloud data is screened, and a first key frame is determined according to a screening result, the first key frame is first point cloud data meeting a preset condition; loop matching is performed according to the first key frame and a historical key frame library, wherein the historical key frame library comprises historical key frames corresponding to the first laser radar and historical key frames corresponding to a second laser radar arranged at a second position of the intelligent device; and a map is reconstructed according to loop matching results of a plurality of frames of the first key frame.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of map reconstruction, and in particular to a mapping method and device based on a laser radar, an intelligent device, and a storage medium. BACKGROUND

[0002] Constructing a three-dimensional point cloud map of an accurate scene is a prerequisite for automatic driving, surveying, and three-dimensional reconstruction technology. A good map can make the positioning of a robot more accurate, the user's senses more realistic, and the reconstructed scene free of disordered problems.

[0003] In order to adapt to the complex changes of indoor and outdoor scenes, a backpack-type laser radar device can be used to scan and collect data to construct a three-dimensional point cloud map of an accurate scene. Ghosting generally occurs when a repeated path is traveled. For the same environment, the laser radar can scan twice. If the maps scanned before and after cannot establish a correct position constraint relationship, the superposition of the two maps will produce serious ghosting.

[0004] How to reduce the ghosting problem as much as possible and improve the accuracy and effectiveness of map reconstruction is a problem that needs to be considered. SUMMARY

[0005] Therefore, the embodiments of the present application provide a mapping method and device based on a laser radar, an intelligent device, and a storage medium, which can reduce the ghosting problem and improve the accuracy and effectiveness of map reconstruction.

[0006] A first aspect of the embodiments of the present application provides a mapping method based on a laser radar, which comprises:

[0007] Obtaining point cloud data, wherein the point cloud data comprises first point cloud data, and the first point cloud data is obtained by a first laser radar arranged at a first position of an intelligent device;

[0008] Filtering the first point cloud data, and determining a first key frame according to a filtering result, wherein the first key frame is first point cloud data satisfying a preset condition;

[0009] Performing loop matching according to the first key frame and a historical key frame library, wherein the historical key frame library comprises historical key frames corresponding to the first laser radar and historical key frames corresponding to a second laser radar arranged at a second position of the intelligent device;

[0010] Reconstructing a map according to loop matching results of a plurality of frames of the first key frame.

[0011] In a possible implementation manner of the first aspect, the filtering of the first point cloud data and the determination of the first key frame according to the filtering result comprises:

[0012] comparing a pose of the first point cloud data of the current frame with a pose of first point cloud data of a neighboring frame of the current frame to obtain pose variable data;

[0013] if the pose variable data satisfies a preset threshold, determining the first point cloud data of the current frame as a first key frame.

[0014] In a possible implementation manner of the first aspect, before the first point cloud data is screened and the first key frame is determined according to a screening result, the method comprises:

[0015] de-distorting the point cloud data by using a preset algorithm to obtain de-distorted first point cloud data and second point cloud data;

[0016] the screening of the first point cloud data and the determination of the first key frame according to the screening result, comprising:

[0017] screening the de-distorted first point cloud data and determining the first key frame according to a screening result.

[0018] In a possible implementation manner of the first aspect, the loop closure matching according to the first key frame and the historical key frame library comprises:

[0019] constructing a historical key frame cluster corresponding to the first key frame, the historical key frame cluster being composed of a plurality of frame historical key frames extracted from the historical key frame library;

[0020] performing loop closure matching on the first key frame and the historical key frame cluster corresponding to the first key frame.

[0021] In a possible implementation manner of the first aspect, the constructing of the historical key frame cluster corresponding to the first key frame comprises:

[0022] searching for a target historical key frame in the historical key frame library, the target historical key frame being a historical key frame in the historical key frame library closest to a corresponding position of the first key frame;

[0023] extracting historical key frames of a specified number of frames adjacent to the target historical key frame;

[0024] constructing the historical key frame cluster corresponding to the first key frame according to the target historical key frame and the historical key frames of the specified number of frames adjacent to the target historical key frame.

[0025] In a possible implementation manner of the first aspect, the loop closure matching result comprises a matching score;

[0026] the reconstructing of the map according to the loop closure matching results of the plurality of frames of the first key frame, comprising:

[0027] If the matching score of the first key frame and the historical key frame cluster thereof is less than a preset matching threshold, it is determined that the key frame is successfully matched with the historical key frame cluster thereof.

[0028] 0Based on the pose of the first key frame matched successfully, the pose of each historical key frame in the historical key frame cluster corresponding to the first key frame is corrected.

[0029] According to the correction result, a map is reconstructed.

[0030] In a possible implementation manner of the first aspect, the point cloud data further includes second point cloud data, the second point cloud data being acquired by a second laser radar arranged at a second position of the intelligent device; and the mapping method further includes:

[0031] Acquiring a calibration relationship, the calibration relationship being used to calibrate the positional relationship between the first laser radar and the second laser radar;

[0032] According to the pose of the first key frame and the calibration relationship, a second key frame is determined, the second key frame being second point cloud data corresponding to the pose of the first key frame;

[0033] 0The first key frame and the second key frame are added to the historical key frame library.

[0034] A second aspect of the embodiment of the application provides a mapping device based on a laser radar, the mapping device including:

[0035] A point cloud data acquisition unit is configured to acquire point cloud data, the point cloud data including first point cloud data, the first point cloud data being acquired by a first laser radar arranged at a first position of an intelligent device; and a first key frame determination unit is configured to filter the first point cloud data and determine a first key frame according to a filtering result, the first key frame being first point cloud data meeting a preset condition.

[0036] A matching unit is configured to perform loop matching according to the first key frame and a historical key frame library, wherein the historical key frame library includes historical key frames corresponding to the first laser radar and historical key frames corresponding to a second laser radar arranged at a second position of the intelligent device.

[0037] A map reconstruction unit is configured to reconstruct a map according to loop matching results of a plurality of frames of the first key frame.

[0038] In a possible implementation manner of the second aspect, the first key frame determination unit includes:

[0039] The pose comparison module is configured to compare a pose of the first point cloud data of the current frame with a pose of first point cloud data of a neighboring frame of the current frame to obtain pose variable data.

[0040] The first key frame determination module is configured to determine the first point cloud data of the current frame as a first key frame if the pose variable data satisfies a preset threshold.

[0041] In a possible implementation of the second aspect, the mapping device further includes:

[0042] The distortion processing unit is configured to de-distort the point cloud data by using a preset algorithm to obtain de-distorted first point cloud data and second point cloud data.

[0043] The first key frame determination unit is further configured to filter the de-distorted first point cloud data and determine a first key frame according to a filtering result.

[0044] In a possible implementation of the second aspect, the matching unit includes:

[0045] The historical key frame cluster construction module is configured to construct a historical key frame cluster corresponding to the first key frame, the historical key frame cluster being composed of a plurality of frame historical key frames extracted from the historical key frame library.

[0046] The loop matching module is configured to perform loop matching on the first key frame and the historical key frame cluster corresponding to the first key frame.

[0047] In a possible implementation of the second aspect, the historical key frame cluster construction module includes:

[0048] The target historical key frame finding submodule is configured to find a target historical key frame in the historical key frame library, the target historical key frame being a historical key frame closest in position to the first key frame in the historical key frame library.

[0049] The historical key frame extraction submodule is configured to extract a specified number of historical key frames adjacent to the target historical key frame.

[0050] The historical key frame cluster construction submodule is configured to construct a historical key frame cluster corresponding to the first key frame according to the target historical key frame and the specified number of historical key frames adjacent to the target historical key frame.

[0051] In a possible implementation of the second aspect, the loop matching result includes a matching score, and the map reconstruction unit includes:

[0052] The matching module is configured to determine that the key frame is successfully matched with the historical key frame cluster if a matching score of the first key frame with the historical key frame cluster is less than a preset matching threshold.

[0053] The loop correction module is configured to correct a pose of each historical key frame in the historical key frame cluster corresponding to the first key frame based on the pose of the successfully matched first key frame.

[0054] The map reconstruction reconstructs a map according to the correction result.

[0055] In a possible implementation of the second aspect, the point cloud data further includes second point cloud data, and the second point cloud data is acquired by a second laser radar arranged at a second position of the intelligent device; and the mapping device further includes:

[0056] The position calibration unit is configured to acquire a calibration relationship, and the calibration relationship is used to calibrate a position relationship between the first laser radar and the second laser radar.

[0057] The second key frame determination unit is configured to determine a second key frame according to the pose of the first key frame and the calibration relationship, and the second key frame is second point cloud data corresponding to the pose of the first key frame.

[0058] The historical key frame library updating unit is configured to add the first key frame and the second key frame to the historical key frame library.

[0059] The third aspect of the embodiments of the present application provides an intelligent device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the laser radar based mapping method provided in the first aspect of the embodiments of the present application when executing the computer program.

[0060] The fourth aspect of the embodiments of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the laser radar based mapping method provided in the first aspect of the embodiments of the present application.

[0061] The fifth aspect of the embodiments of the present application provides a computer program product, which, when executed on a terminal device, causes the terminal device to perform the steps of the laser radar based mapping method described in the first aspect of the embodiments of the present application.

[0062] In the embodiments of the present application, by acquiring point cloud data, the point cloud data includes first point cloud data, the first point cloud data is filtered, and the first key frame is determined according to the filtering result, the first key frame is the first point cloud data meeting the preset condition, then the loop matching is performed according to the first key frame and the historical key frame library, wherein the historical key frame library includes the historical key frame corresponding to the first laser radar and the historical key frame corresponding to the second laser radar arranged at the second position of the intelligent device, and then the map is reconstructed according to the loop matching result of a plurality of frames of the first key frame. The present application can improve the accuracy of loop matching, thereby reducing ghosting, improving the accuracy and effectiveness of map reconstruction. BRIEF DESCRIPTION OF DRAWINGS

[0063] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0064] Figure 1.1 is a schematic diagram of a backpack type laser radar device;

[0065] Figure 1.2 is a schematic diagram of a backpack type laser radar device provided by the embodiments of the present application;

[0066] Figure 2 is an implementation flowchart of the mapping method based on laser radar provided by the embodiments of the present application;

[0067] Figure 3 is a specific implementation flowchart of step S102 in the mapping method based on laser radar provided by the embodiments of the present application;

[0068] Figure 4 is a specific implementation flowchart of step S103 in the mapping method based on laser radar provided by the embodiments of the present application;

[0069] Figure 5 is a specific implementation flowchart of constructing the historical key frame cluster corresponding to the first key frame in the mapping method based on laser radar provided by the embodiments of the present application;

[0070] Figure 6 is a specific implementation flowchart of step S104 in the mapping method based on laser radar provided by the embodiments of the present application;

[0071] Figure 7 is a specific implementation flowchart of updating the historical key frame library in the mapping method based on laser radar provided by the embodiments of the present application;

[0072] Figure 8 This is a structural block diagram of the lidar-based mapping device provided in the embodiments of this application;

[0073] Figure 9 This is a schematic diagram of a smart device provided in an embodiment of this application. Detailed Implementation

[0074] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0075] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0076] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0077] To adapt to the complex changes in indoor and outdoor scenes, backpack-mounted LiDAR devices are typically used to scan and collect data, constructing accurate 3D point cloud maps of the scene. The biggest advantage of backpack-mounted LiDAR devices is their adaptability; mapping can be completed in any area accessible to a person. Figure 1.1 As shown, a person carrying a single LiDAR device walks a designated route and returns to the starting point. To ensure the constructed map reflects more scene details or other conditions, sometimes it's necessary to walk in reverse for another loop or section. In this case, the weight of the person causes them to tilt involuntarily, which in turn causes the LiDAR device to tilt as well. For the same location, data scanned at different times may not match, potentially resulting in ghosting and affecting the accuracy of the map construction.

[0078] To solve the above problems, the embodiment of the present application provides a mapping method based on a laser radar, a device, an intelligent device and a storage medium, please refer to the following.

[0079] The various method embodiments of the present application provide a mapping method based on a laser radar, which is suitable for various types of intelligent devices that need to perform three-dimensional point cloud mapping.

[0080] The mapping method based on a laser radar provided by the present application will be exemplarily described below in combination with specific embodiments.

[0081] Figure 1.2 A schematic diagram of a backpack-type laser radar device provided by the embodiment of the present application is shown. As shown in the figure, Figure 1.2 two laser radars are included in the embodiment of the present application, i.e. a first laser radar and a second laser radar, and one laser radar cannot block the scanning ray of the other laser radar. Among them, the first laser radar is horizontally arranged at a first position of the intelligent device, and the second laser radar is vertically arranged at a second position of the intelligent device. The first laser radar and the second laser radar work simultaneously. That is, when a person walks with the intelligent device provided with the first laser radar and the second laser radar, the first laser radar and the second laser radar simultaneously scan and collect point cloud data.

[0082] Figure 2 The implementation process of the mapping method based on a laser radar provided by the embodiment of the present application is shown, and the execution end of the embodiment of the present application can be an intelligent device. The method process can include the following steps S101 to S104.

[0083] Step S101: acquiring point cloud data, the point cloud data including first point cloud data, the first point cloud data being acquired by a first laser radar arranged at a first position of an intelligent device.

[0084] In the embodiment of the present application, the first point cloud data is scanned and acquired by the first laser radar horizontally arranged at the first position of the intelligent device. The point cloud data further includes second point cloud data, and the second point cloud data is scanned and acquired by a second laser radar vertically arranged at a second position of the intelligent device.

[0085] Step S102: screening the first point cloud data, and determining a first key frame according to the screening result, the first key frame being the first point cloud data meeting a preset condition.

[0086] In the embodiment of the present application, the first point cloud data is screened to determine the first key frame, and then subsequent processing is performed based on the first key frame, which can improve the mapping efficiency.

[0087] The point cloud data collected at different times has different poses of the lidar. That is, different point cloud data corresponds to different poses. As a possible implementation of the present application, Figure 3 A specific implementation process of filtering the first point cloud data and determining a first key frame according to the filtering result in the method provided by the embodiment of the present application is shown, and is described in detail as follows:

[0088] A1: comparing the pose of the first point cloud data of the current frame with the pose of the first point cloud data of the adjacent frame to obtain pose variable data.

[0089] In the embodiment, the feature points of the first point cloud data are extracted, inter-frame matching is performed based on the feature point set, and the pose of the first lidar is obtained. The feature points include feature corner points, feature plane points, etc. The pose of the first point cloud data of the current frame refers to the pose of the first lidar when the first point cloud data of the current frame is collected. The adjacent frame first point cloud data is the previous frame of the first point cloud data of the current frame. The pose of the adjacent frame first point cloud data refers to the pose of the first lidar when the first point cloud data of the previous frame of the current frame is collected.

[0090] The pose includes a travel distance and / or an angle. The travel distance and / or the angle of the first point cloud data of the current frame is compared with the travel distance and / or the angle of the first point cloud data of the adjacent frame to obtain change data of the travel distance and / or the angle, and it is determined whether the change data meets a preset threshold.

[0091] Exemplarily, it is determined whether the change data of the travel distance is greater than a preset distance threshold. It is determined whether the change data of the travel angle is greater than a preset angle threshold.

[0092] A2: if the pose variable data meets the preset threshold, the first point cloud data of the current frame is determined as a first key frame.

[0093] In the embodiment, when the change data of the travel distance is greater than the preset distance threshold, or when the change data of the travel angle is greater than the preset angle threshold, it is determined that the pose variable data meets the preset threshold, and the first point cloud data of the current frame is determined as the first key frame.

[0094] In the embodiment of the present application, the point cloud data is filtered through the determination of the key frame, interference caused by invalid point cloud data to subsequent processing is avoided, and the efficiency and accuracy of mapping can be improved.

[0095] As a possible implementation of the present application, before the first point cloud data is filtered and the first key frame is determined according to the filtering result, the point cloud data is de-distorted by using a preset algorithm to obtain de-distorted first point cloud data and second point cloud data. Then, the de-distorted first point cloud data is filtered, and the first key frame is determined according to the filtering result.

[0096] In a possible implementation, the acquired point cloud data is de-distorted by using IMU (Inertial Measurement Unit) pre-integration, and the motion distortion of the first lidar and the second lidar in the running process is corrected by using high-frequency IMU data.

[0097] In the embodiments of the present application, the accuracy of the point cloud data can be improved by de-distorting the acquired point cloud data by using a preset algorithm, which is beneficial to improving the accuracy of mapping.

[0098] Step S103: performing loop matching according to the first key frame and a historical key frame library, wherein the historical key frame library includes historical key frames corresponding to the first lidar and historical key frames corresponding to a second lidar arranged at a second position of the intelligent device.

[0099] The historical key frame library stores historical key frames collected by the first lidar and the second lidar along a human walking path. The historical key frames corresponding to the first lidar are first historical point cloud data collected by the first lidar along the human walking path and satisfying the preset condition, and the historical key frames corresponding to the second lidar are second historical point cloud data collected by the second lidar along the human walking path and satisfying another preset condition.

[0100] In the historical key frame library, the historical key frames corresponding to the first lidar and the historical key frames corresponding to the second lidar at the same time are associated based on poses, specifically, associated based on poses of the historical key frames corresponding to the first lidar.

[0101] As a possible implementation of the present application, Figure 4 The specific implementation process of performing loop matching according to the first key frame and the historical key frame library in the method embodiment provided by the embodiments of the present application is shown, and is described in detail as follows:

[0102] B1: constructing a historical key frame cluster corresponding to the first key frame, wherein the historical key frame cluster is composed of a plurality of frame historical key frames extracted from the historical key frame library.

[0103] As a possible implementation of the present application, as shown in Figure 5 The constructing the historical key frame cluster corresponding to the first key frame includes:

[0104] B11: Search for the target historical keyframe in the historical keyframe library. The target historical keyframe is the historical keyframe in the library that is closest to the location corresponding to the first keyframe. The location distance refers to the distance between the LiDAR scanning locations.

[0105] B12: Extract a specified number of historical keyframes that are adjacent to the target historical keyframe.

[0106] In this embodiment, historical keyframes that are consecutively adjacent to the target historical keyframe for a specified number of frames are extracted, as well as historical keyframes that are consecutively adjacent to the target historical keyframe for a specified number of frames. The specified number of frames can be set by the user.

[0107] B13: Based on the target historical keyframe and a specified number of historical keyframes adjacent to the target historical keyframe, construct a historical keyframe cluster corresponding to the first keyframe.

[0108] The historical keyframe cluster includes the target historical keyframe and a specified number of historical keyframes adjacent to the target historical keyframe.

[0109] For example, the historical keyframe closest to the position corresponding to the first keyframe is found in the historical keyframe library. K Extract PKF K The previous 5 historical keyframes, and PKF K The next 5 keyframes. According to PKF K 5 keyframes prior to PKFK and PKF K The next 5 historical keyframes are used to construct the historical keyframe library corresponding to the first keyframe.

[0110] B2: Perform a loop-based matching between the first keyframe and its corresponding historical keyframe cluster.

[0111] In this embodiment, the first point cloud data of the first key frame is transformed to the coordinate system of the historical key frame cluster, and then matched with the historical key frame cluster to calculate the matching score.

[0112] Commonly used matching algorithms include the iterative nearest point algorithm. In this embodiment, the specific algorithm for loop closure matching can be referred to the existing technology, and will not be elaborated here.

[0113] In this embodiment of the application, by constructing a historical keyframe cluster corresponding to the first keyframe, and then performing a loop-matching between the first keyframe and its corresponding historical keyframe cluster, the effectiveness of the matching can be effectively improved.

[0114] Step S104: reconstructing the map according to the loop matching results of the first key frames.

[0115] In the embodiments of the present application, the steps S101 to S103 are repeatedly executed, and the map is reconstructed according to the loop matching results of the first key frames and the corresponding historical key frame clusters.

[0116] In the embodiments of the present application, the first laser radar and the second laser radar are carried by a person to walk around a designated route, and the first point cloud data and the second point cloud data are scanned and collected in multiple frames. The first key frames are obtained by screening the multiple first point cloud data. The loop matching is performed on each first key frame, and the map is reconstructed by combining the loop matching results of the first key frames and the corresponding historical key frame clusters.

[0117] As a possible implementation manner of the present application, the loop matching result includes a matching score, Figure 6 An implementation process of reconstructing the map according to the loop matching results of the first key frames in the embodiments of the present application is shown, and is described in detail as follows:

[0118] C1: if the matching score of the first key frame and the historical key frame cluster thereof is less than a preset matching threshold, it is determined that the first key frame and the historical key frame cluster thereof are matched successfully.

[0119] The preset matching threshold can be an average distance of each point in the first key frame point cloud to the corresponding point in the historical key frame cluster.

[0120] For example, the preset matching threshold can be 0.1, and when the matching score of the first key frame and the historical key frame cluster thereof is less than 0.1, it is determined that the matching is successful.

[0121] C2: based on the pose of the first key frame matched successfully, the pose of each historical key frame in the historical key frame cluster corresponding to the first key frame is corrected.

[0122] C3: according to the correction result, the map is reconstructed. The correction result includes the correction results of the first key frames matched successfully.

[0123] In the embodiments of the present application, by pose correction, more loops are added in the mapping process, which can effectively reduce ghosting and improve the effectiveness and accuracy of mapping.

[0124] As a possible implementation manner of the present application, the point cloud data further includes second point cloud data, the second point cloud data is obtained by a second laser radar arranged at a second position of the intelligent device, Figure 7 An implementation process of updating the historical key frame library in the method embodiments provided by the embodiments of the present application is shown, and is described in detail as follows:

[0125] D1: Obtain the calibration relationship, which is used to calibrate the positional relationship between the first lidar and the second lidar.

[0126] The positional relationship between the first and second lidars can be determined using a special calibration board or calibration room. In this embodiment, the historical keyframe library includes the calibration relationship between the first and second lidars.

[0127] D2: Based on the pose of the first keyframe and the calibration relationship, determine the second keyframe, whereby the second keyframe is the second point cloud data corresponding to the pose of the first keyframe.

[0128] For example, the first keyframe PKF n{ PPose n PPC n}, second keyframe SKF n {PPose n PPC n}, PPose n For the first keyframe PKF n pose, SPose n For the second keyframe SKF n The pose of the first and second lidars is given by T. sp SPose n =T sp* PPose n .

[0129] D3: Add the first keyframe and the second keyframe to the historical keyframe library.

[0130] Since the pose variable data of the second lidar at the second position is negligible, it is neither possible nor necessary to determine the second keyframe by filtering the second point cloud data. In this embodiment, the second point cloud data corresponding to the pose of the first keyframe is determined as the second keyframe. By combining the pose of the first keyframe with the calibration relationship between the first and second lidars, the pose of the second keyframe can be effectively determined. The determined first and second keyframes are then added to the historical keyframe library for storage.

[0131] In the embodiment of the present application, by acquiring point cloud data, the point cloud data includes first point cloud data, the first point cloud data is filtered, and a first key frame is determined according to the filtering result, the first key frame is the first point cloud data meeting a preset condition, then loop matching is performed according to the first key frame and a historical key frame library, the historical key frame library includes historical key frames corresponding to the first lidar and historical key frames corresponding to a second lidar arranged at a second position of the intelligent device, and a map is reconstructed according to loop matching results of a plurality of frames of the first key frames. The present application can improve the accuracy of loop matching, thereby reducing ghosting, improving the accuracy and effectiveness of map reconstruction.

[0132] It should be understood that the size of the serial number of each step in the above-mentioned various embodiments does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0133] According to the laser radar-based mapping method described in the above embodiments, Figure 8 The structure block diagram of the laser radar-based mapping device provided by the embodiments of the present application is shown, and only the parts related to the embodiments of the present application are shown for ease of illustration.

[0134] Referring to Figure 8 The mapping device is applied to an intelligent device, and the mapping device comprises a point cloud data acquisition unit 81, a first key frame determination unit 82, a matching unit 83, and a map reconstruction unit 84, wherein:

[0135] The point cloud data acquisition unit 81 is configured to acquire point cloud data, the point cloud data including first point cloud data, the first point cloud data being acquired by a first lidar arranged at a first position of the intelligent device;

[0136] The first key frame determination unit 82 is configured to filter the first point cloud data, and determine a first key frame according to the filtering result, the first key frame being the first point cloud data meeting a preset condition;

[0137] The matching unit 83 is configured to perform loop matching according to the first key frame and a historical key frame library, wherein the historical key frame library includes historical key frames corresponding to the first lidar and historical key frames corresponding to a second lidar arranged at a second position of the intelligent device;

[0138] The map reconstruction unit 84 is configured to reconstruct a map according to loop matching results of a plurality of frames of the first key frames.

[0139] As a possible implementation manner of the present application, the first key frame determination unit 82 comprises:

[0140] a pose comparison module, configured to compare a pose of the first point cloud data of the current frame with a pose of first point cloud data of a neighboring frame of the current frame to obtain pose variable data;

[0141] a first key frame determination module, configured to determine the first point cloud data of the current frame as a first key frame if the pose variable data satisfies a preset threshold.

[0142] As a possible implementation of the present application, the mapping device further comprises:

[0143] a distortion processing unit, configured to de-distort the point cloud data by using a preset algorithm to obtain de-distorted first point cloud data and second point cloud data;

[0144] The first key frame determination unit is further configured to screen the de-distorted first point cloud data and determine a first key frame according to a screening result.

[0145] As a possible implementation of the present application, the matching unit 83 comprises:

[0146] a historical key frame cluster construction module, configured to construct a historical key frame cluster corresponding to the first key frame, the historical key frame cluster being composed of a plurality of frame historical key frames extracted from the historical key frame library;

[0147] a loop matching module, configured to perform loop matching on the first key frame and the historical key frame cluster corresponding thereto.

[0148] As a possible implementation of the present application, the historical key frame cluster construction module comprises:

[0149] a target historical key frame searching submodule, configured to search for a target historical key frame in the historical key frame library, the target historical key frame being a historical key frame closest to a corresponding position of the first key frame in the historical key frame library;

[0150] a historical key frame extraction submodule, configured to extract a specified number of historical key frames adjacent to the target historical key frame;

[0151] a historical key frame cluster construction submodule, configured to construct the historical key frame cluster corresponding to the first key frame according to the target historical key frame and the specified number of historical key frames adjacent to the target historical key frame.

[0152] As a possible implementation of the present application, the loop matching result comprises a matching score, and the map reconstruction unit 84 comprises:

[0153] The matching module is configured to determine that the key frame is successfully matched with the historical key frame cluster if a matching score of the first key frame and the historical key frame cluster thereof is less than a preset matching threshold.

[0154] The loop correction module is configured to correct a pose of each historical key frame in the historical key frame cluster corresponding to the first key frame based on the pose of the successfully matched first key frame.

[0155] The map reconstruction reconstructs a map according to the correction result.

[0156] As a possible implementation of the present application, the point cloud data further includes second point cloud data acquired by a second laser radar arranged at a second position of the intelligent device; and the mapping device further includes:

[0157] The position calibration unit is configured to acquire a calibration relationship, which is used to calibrate a position relationship between the first laser radar and the second laser radar.

[0158] The second key frame determination unit is configured to determine a second key frame according to the pose of the first key frame and the calibration relationship, the second key frame being second point cloud data corresponding to the pose of the first key frame.

[0159] The historical key frame library updating unit is configured to add the first key frame and the second key frame to the historical key frame library.

[0160] In the embodiments of the present application, point cloud data is acquired, the point cloud data including first point cloud data, the first point cloud data is filtered, and a first key frame is determined according to a filtering result, the first key frame being first point cloud data satisfying a preset condition, then loop matching is performed according to the first key frame and a historical key frame library, the historical key frame library including historical key frames corresponding to the first laser radar and historical key frames corresponding to a second laser radar arranged at a second position of the intelligent device, and a map is reconstructed according to loop matching results of a plurality of frames of the first key frame. The present application can improve the accuracy of loop matching, thereby reducing ghosting, and improving the accuracy and effectiveness of map reconstruction.

[0161] The embodiments of the present application also provide a computer readable storage medium storing a computer program, the computer program being executed by a processor to implement steps of any one of the mapping methods based on laser radars shown in the embodiments of the present application. Figures 2 to 7 The embodiments of the present application also provide an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor implementing the computer program to implement steps of any one of the mapping methods based on laser radars shown in the embodiments of the present application.

[0162] The embodiments of the present application also provide an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor implementing the computer program to implement steps of any one of the mapping methods based on laser radars shown in the embodiments of the present application.Figures 2 to 7 steps of any one of the laser radar based mapping methods represented by the equations.

[0163] The embodiments of the present application further provide a computer program product, which, when running on a terminal device, causes the terminal device to perform the steps of any one of the laser radar based mapping methods represented by the equations. Figures 2 to 7 steps of any one of the laser radar based mapping methods represented by the equations.

[0164] Figure 9 is a schematic diagram of the intelligent device provided by an embodiment of the present application. As shown in the equation Figure 9 The intelligent device 9 of this embodiment includes a processor 90, a memory 91, and a computer program 92 stored in the memory 91 and capable of running on the processor 90. The processor 90 implements the steps in the above-mentioned various embodiments of the laser radar based mapping methods when executing the computer program 92, such as the steps S101-S104 shown in the equation Figure 2 Alternatively, the processor 90 implements the functions of the modules / units in the above-mentioned various device embodiments when executing the computer program 92, such as the functions of the units 81-84 shown in the equation Figure 8

[0165] The computer program 92 can be divided into one or more modules / units, which are stored in the memory 91 and executed by the processor 90 to complete the present application. The one or more modules / units can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program 92 in the intelligent device 9.

[0166] The processor 90 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0167] ​The memory 91 can be an internal storage unit of the smart device 9, for example, a hard disk or a memory of the smart device 9. The memory 91 can also be an external storage device of the smart device 9, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the smart device 9. Further, the memory 91 can also include both the internal storage unit and the external storage device of the smart device 9. The memory 91 is used to store the computer program and other programs and data required by the smart device. The memory 91 can also be used to temporarily store data that has been output or is to be output.

[0168] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional units and modules is exemplified, and in actual application, the above functions can be completed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of software functional unit. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction, and do not limit the protection scope of the present application. The specific working process of the units and modules in the apparatus can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0169] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system, apparatus and unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0170] In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described or recorded in detail in a certain embodiment can refer to the relevant description of other embodiments.

[0171] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0172] In the embodiments of the present application, it should be understood that the disclosed apparatus and method can be implemented in other manners. For example, the described apparatus embodiments are merely schematic. The division of the modules or units is merely logical function division. There can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another apparatus, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between different units, can be indirect couplings or communication connections through some interfaces, apparatuses or units, and can be electrical, mechanical or in other forms.

[0173] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0174] In addition, each functional unit in the various embodiments of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0175] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, all or part of the flow of the above-mentioned embodiment methods can also be completed by a computer program instructing related hardware, and the computer program can be stored in a computer readable storage medium. When the processor executes the computer program, the steps of each method embodiment described above can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable file or some intermediate form. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the computer readable medium can include appropriate contents according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.

[0176] The above-described embodiments are only used to illustrate the technical solutions of the present application, but not limit them; although the present application is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A mapping method based on lidar, characterized in that, The mapping method includes: Acquire point cloud data, which includes first point cloud data and second point cloud data. The first point cloud data is acquired by a first lidar located at a first position of the smart device, and the second point cloud data is acquired by a second lidar located at a second position of the smart device. The first point cloud data is filtered, and the first key frame is determined based on the filtering results. The first key frame is the first point cloud data that meets the preset conditions. A historical keyframe cluster corresponding to the first keyframe is constructed, and a loop-closure match is performed between the first keyframe and its corresponding historical keyframe cluster. The historical keyframe cluster consists of several historical keyframes extracted from a historical keyframe library, which includes historical keyframes corresponding to the first lidar and historical keyframes corresponding to the second lidar located at the second position of the smart device. A calibration relationship is obtained, which is used to calibrate the positional relationship between the first lidar and the second lidar. Based on the pose of the first keyframe and the calibration relationship, a second keyframe is determined. The second keyframe is the second point cloud data corresponding to the pose of the first keyframe. The first keyframe and the second keyframe are added to the historical keyframe library. The map is reconstructed based on the loop closure matching results of the first keyframe across several frames.

2. The method as described in claim 1, characterized in that, The step of filtering the first point cloud data and determining the first keyframe based on the filtering results includes: Compare the pose of the first point cloud data in the current frame with the pose of the first point cloud data in the adjacent frames to obtain pose variable data. If the pose variable data meets the preset threshold, then the first point cloud data of the current frame is determined as the first keyframe.

3. The method as described in claim 1, characterized in that, Before filtering the first point cloud data and determining the first keyframe based on the filtering results, the process includes: The point cloud data is distorted using a preset algorithm to obtain the first and second point cloud data after distortion removal. The step of filtering the first point cloud data and determining the first keyframe based on the filtering results includes: The first point cloud data after distortion correction is filtered, and the first keyframe is determined based on the filtering results.

4. The method as described in claim 1, characterized in that, The construction of the historical keyframe cluster corresponding to the first keyframe includes: Search for a target historical keyframe in the historical keyframe library. The target historical keyframe is the historical keyframe in the historical keyframe library that is closest to the position corresponding to the first keyframe. Extract a specified number of historical keyframes that are adjacent to the target historical keyframe; Based on the target historical keyframe and a specified number of historical keyframes adjacent to the target historical keyframe, construct a historical keyframe cluster corresponding to the first keyframe.

5. The method as described in claim 1, characterized in that, The loop closure matching result includes a matching score; the process of reconstructing the map based on the loop closure matching results of the first keyframe across several frames includes: If the matching score of the first keyframe and its historical keyframe cluster is less than the preset matching threshold, then the keyframe is determined to be successfully matched with its historical keyframe cluster. Based on the pose of the first keyframe that was successfully matched, the pose of each historical keyframe in the historical keyframe cluster corresponding to the first keyframe is corrected. Based on the correction results, the map is reconstructed.

6. A mapping device based on lidar, characterized in that, The mapping device includes: A point cloud data acquisition unit is used to acquire point cloud data, which includes first point cloud data and second point cloud data. The first point cloud data is acquired by a first lidar located at a first position of the smart device, and the second point cloud data is acquired by a second lidar located at a second position of the smart device. The first keyframe determination unit is used to filter the first point cloud data and determine the first keyframe according to the filtering result. The first keyframe is the first point cloud data that meets the preset conditions. The matching unit is used to construct a historical key frame cluster corresponding to the first key frame and perform loop matching between the first key frame and the corresponding historical key frame cluster. The historical key frame cluster consists of several historical key frames extracted from the historical key frame library. The historical key frame library includes historical key frames corresponding to the first lidar and historical key frames corresponding to the second lidar set at the second position of the smart device. A position calibration unit is used to obtain a calibration relationship, which is used to calibrate the positional relationship between the first lidar and the second lidar; The second keyframe determination unit is used to determine the second keyframe based on the pose of the first keyframe and the calibration relationship. The second keyframe is the second point cloud data corresponding to the pose of the first keyframe. The historical keyframe library update unit is used to add the first keyframe and the second keyframe to the historical keyframe library. The map reconstruction unit is used to reconstruct the map based on the loop closure matching results of the first keyframe in several frames.

7. A smart device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the lidar-based mapping method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the lidar-based mapping method as described in any one of claims 1 to 5.

Images