A scene recognition-oriented lightweight topology map construction method
By extracting descriptors from accumulated point cloud data and generating topological nodes, the problem of node sparsity and uneven recognition performance in existing topological map construction is solved, and lightweight and efficient scene recognition topological map construction is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- DUT ARTIFICIAL INTELLIGENCE INST DALIAN
- Filing Date
- 2023-02-02
- Publication Date
- 2026-07-24
AI Technical Summary
Existing topology map construction methods cannot simultaneously achieve lightweight and efficient recognition in scene recognition tasks. Density-based methods result in overly dense nodes, while appearance-based methods result in overly sparse nodes, failing to satisfy the balance between the sparsity of the topology map and the performance of scene recognition.
By collecting and accumulating multi-frame point cloud data of the robot's running trajectory, scene descriptors are extracted, and topological nodes are generated by calculating the similarity between descriptors. The topological nodes are represented by equally spaced curved grids and real number matrices, and the optimal topological map is generated by adjusting the similarity threshold.
It achieves lightweight topology map construction, reducing the number of nodes without compromising scene recognition performance, making it particularly suitable for large-scale scene construction and significantly reducing the number of nodes compared to appearance-based methods.
Smart Images
Figure CN115930952B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of robotics, and in particular to a lightweight topology map construction method for scene recognition. Background Technology
[0002] With the rapid development of mobile robot systems, 3D laser sensors, as active sensors, offer advantages over other sensors, such as being unaffected by ambient light, being stable and reliable, and providing accurate measurement results. Therefore, they play an increasingly important role in map building tasks. In mobile robot map building, topological maps are commonly used, abstracting the environment into several topologically related nodes. This effectively reduces the storage burden of map data, and the topological structure allows robots to directly utilize the map for path planning and scene recognition tasks.
[0003] In topology map construction algorithms, most existing topology maps generate topology nodes based on density or appearance. Density-based topology maps cannot achieve accurate scene recognition tasks as the environment expands. On the other hand, appearance-based construction methods result in overly dense topology nodes, failing to fully utilize scene information in the robot's motion trajectory, leading to inefficient scene recognition tasks and a loss of the sparsity of the topology map.
[0004] The paper (Ravankar AA, Ravankar A, Emmaru T, et al. A hybrid topological mapping and navigation method for large area robot mapping[C] / / 2017 56th Annual Conference of the Society of Instrument and Control Engineers of Japan (SICE).IEEE,2017:1104-1107.) proposes a method for generating topological nodes at equal intervals along the robot's trajectory, which is a classic density-based topological map construction method. During topological map construction, if the selected interval is too small, the topological points will be too dense, failing to meet the requirement of lightweight mapping; if the interval is too large, it will not reflect the characteristics of scene changes, and the scene recognition performance of the topological map will deteriorate.
[0005] The literature (Tang L, Wang Y, Ding X, et al. Topological local-metric framework for mobile robots navigation: a long term perspective[J]. Autonomous Robots, 2019, 43(1): 197-211.) proposes a keyframe selection strategy. A keyframe is retained when the similarity between the point cloud descriptor of a pose point and the point cloud descriptor of a retained keyframe changes significantly. This is a classic appearance-based keyframe selection method. However, this method only uses point cloud data from a single frame, which is insufficient for describing the topological nodes in a large-scale map, resulting in excessively dense topological nodes that cannot meet the need for lightweight topological maps.
[0006] In summary, how to design a lightweight topology map construction method for scene recognition that ensures a sufficiently sparse number of nodes in the topology map while maintaining good scene recognition performance has become a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention
[0007] The technical problem to be solved by the present invention is to overcome the shortcomings of the existing technology and provide a lightweight topology map construction method for scene recognition.
[0008] This invention is achieved through the following technical solution: a lightweight topology map construction method for scene recognition, comprising the following steps:
[0009] S1: Collect data and process the data to obtain the cumulative point cloud; specifically, after the data is collected, process the robot's running trajectory and laser point cloud data, accumulate each frame of laser point cloud data and its neighboring laser point cloud data, and use the accumulated point cloud data to replace the laser point cloud data.
[0010] S2: Extract scene descriptors; specifically including: based on the point cloud data obtained in S1, calculate the orientation angle and pitch angle of the points in the point cloud in the local coordinate system, establish an equally spaced curved surface grid, assign each point in the point cloud data to the equally spaced curved surface grid, use a real number matrix as the descriptor, and map the maximum height of the points in the equally spaced curved surface grid to the real number matrix, with each element of the real number matrix corresponding to the other.
[0011] S3: Calculate the similarity between descriptors; specifically, it includes: extracting a pair of descriptors based on S2, using the average of the cosine similarity of the corresponding column vectors in the two real matrices of the pair of descriptors to represent the cosine similarity between the pair of descriptors, and using column offsets to calculate and find the minimum distance in the real matrix as the final similarity.
[0012] S4: Generate topology nodes; specifically, this includes: extracting descriptors from S2 to generate a descriptor set C, establishing an empty topology map set TopoMap, traversing set C, searching for the descriptor with the highest similarity for descriptor s in the topology map set TopoMap, with its similarity being Sim, and adding descriptors with similarity Sim lower than the similarity threshold Threshold to the topology map.
[0013] S5: Determine the optimal topology map; specifically, this includes: exhaustively enumerating similarity thresholds to generate several different topology maps, then calculating and plotting the precision and recall curves of each topology map, and finding the topology map with the optimal precision-recall curve as the final result.
[0014] According to the above technical solution, preferably, the accumulation method in S1 is as follows: for the laser point cloud data of the m-th frame, a voxel map is established based on its point cloud data, and the laser point cloud data after the m-th frame is traversed from front to back in chronological order. The traversed laser point cloud data is rotated and translated to the local coordinate system of the m-th frame, and the voxel map is updated with the rotated and translated laser point cloud data. When the voxel map has no change, the traversal is stopped, and the point cloud data accumulated by the voxel map is used to replace the single frame laser point cloud data of the m-th frame.
[0015] According to the above technical solution, preferably, the coordinate mapping formula for assigning each point in the point cloud data to the equally spaced curved surface grid in S2 is as follows:
[0016]
[0017] Where, N r N represents the number of grid rings. s Let r be the number of sectors, and r be the effective distance from the laser point to the center of the lidar sensor. max r is the maximum effective distance from the laser point to the center of the lidar sensor. min Let be the minimum effective distance from the laser point p to the center of the lidar sensor. Let x represent the value of laser point p on the x-axis in three-dimensional space, y represent the value of laser point p on the y-axis in three-dimensional space, z represent the value of laser point p on the z-axis in three-dimensional space, i be the horizontal index of laser point p in the equally spaced curved grid, and j be the horizontal and radial index of laser point p in the equally spaced curved grid. For integer manipulation;
[0018] In S2, the maximum height of the midpoint of the equally spaced curved grid is mapped to a real matrix, with a one-to-one correspondence between the elements of the real matrix. Specifically, this includes using an N... r ×N s The real matrix I is used as the descriptor;
[0019]
[0020] Among them, a i1,j1 Let P be the element in the i1th row and j1st column of the real matrix I. i1,j1 Let z(p) be the set of laser points in the i1th row and j1th column of the equally spaced curved grid, and z(p) be the value of point p on the z-axis.
[0021] According to the above technical solution, preferably, a pair of descriptors in S3 is I. q and I c Cosine similarity is expressed as:
[0022]
[0023] Among them, S(I q ,I c ) is the descriptor I q and descriptor I c Cosine similarity between them, N s For sectors
[0024] quantity, For descriptor I q The column vector of the j1-th column in a real matrix. For descriptor I c The column vector of the j1-th column in a real matrix;
[0025] The final similarity is expressed as:
[0026]
[0027] Among them, Similarity(I q , The final similarity score is denoted as . For descriptor I c The column offset matrix obtained by offsetting n columns of the real matrix has a final value range of [0,1].
[0028] According to the above technical solution, preferably, the specific implementation of S5 is as follows:
[0029] S51: Change the similarity threshold Threshold in S4 from 0 to 1 to generate topology maps with different similarity thresholds.
[0030] S52: For each map generated in S51, iterate through the descriptor subset C generated in S4. For each descriptor s in the descriptor subset C, search for the descriptor s with the highest similarity to s in the established topological map. t ;
[0031] S53: Regarding s and s in S52 tPerform the test: If s and s t The similarity is greater than the test threshold C thres And the distance is less than the test distance C d If s and s t The similarity is less than the test threshold C thres And the distance is less than the test distance C d If s and s t The similarity is greater than the test threshold C thres And the distance is greater than the test distance C d If the false positive count is positive, then increment the false positive count by one.
[0032] Among them, the test distance C d = 5 meters, when the test threshold is C thres At that time, the corresponding precision and recall are:
[0033]
[0034]
[0035] S54: Change the test threshold C in S53 thres By uniformly distributing these values within the interval [0,1], a set of precision and recall rates can be obtained. Plotting this set of precision and recall rates yields a precision-recall curve, which is the precision-recall curve of the topology map. By plotting the precision-recall curves of the topology map corresponding to different similarity thresholds, the topology map with the largest area enclosed by the precision-recall curve is selected as the final result, thus completing the construction of the topology map.
[0036] The beneficial effects of this invention are: this invention describes the scene based on the multi-frame point cloud data accumulated from the robot's running trajectory, extracts descriptors from the point cloud, and then determines whether to generate topological nodes based on the similarity of the descriptors; this invention can ensure the effective construction of lightweight topological maps, and is particularly suitable for the needs of building topological maps for large-scale scenes; compared with appearance-based topological map construction methods, this invention greatly reduces the number of topological map nodes, but still ensures reliable scene recognition performance. Attached Figure Description
[0037] Figure 1 This is a flowchart of the present invention;
[0038] Figure 2 A schematic diagram for creating a voxel map at the corresponding location of the trajectory using laser point cloud data;
[0039] Figure 3A schematic diagram illustrating the updating of the voxel map using traversed laser point cloud data;
[0040] Figure 4 A schematic diagram of a point cloud descriptor subsurface raster;
[0041] Figure 5 This is a schematic diagram illustrating the generation and discrimination of topological nodes;
[0042] Figure 6 This is a single-frame point cloud data map;
[0043] Figure 7 For cumulative point cloud data;
[0044] Figure 8 This is a diagram showing the proportion of single-frame point cloud data in the cumulative point cloud data.
[0045] Figure 9 To describe the sub-effects;
[0046] Figure 10 Precision-recall curves for topological maps with different similarity thresholds;
[0047] Figure 11 Create a rendering of the topology map;
[0048] Figure 12 This is a comparison and verification chart of topology map performance;
[0049] Figure 13 Satellite map of the experimental scene. Detailed Implementation
[0050] To enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and preferred embodiments.
[0051] like Figure 1 As shown, this invention provides a lightweight topology map construction method for scene recognition. This invention uses a 32-line 3D LiDAR as the main sensor, acquiring environmental 3D point clouds at a frequency of 10Hz, with approximately 64,000 points per frame. The horizontal scanning range is 360 degrees with a resolution of 0.18 degrees, and the vertical scanning range is 45 degrees with a resolution of 1.4516 degrees. The data is processed using an Intel I58400 to construct the topology map, specifically including the following steps:
[0052] S1: Collect data and process the data to obtain the cumulative point cloud; specifically, after the data is collected, process the robot's running trajectory and laser point cloud data. Each frame of laser point cloud data in the data has a corresponding pose. Accumulate each frame of laser point cloud data and its neighboring laser point cloud data, and use the accumulated point cloud data to replace the laser point cloud data.
[0053] S2: Extract scene descriptors; specifically including: based on the point cloud data obtained in S1, calculate the orientation angle and pitch angle of the points in the point cloud in the local coordinate system, establish an equally spaced curved surface grid, assign each point in the point cloud data to the equally spaced curved surface grid, use a real number matrix as the descriptor, and map the maximum height of the points in the equally spaced curved surface grid to the real number matrix, with each element of the real number matrix corresponding to the other.
[0054] S3: Calculate the similarity between descriptors; specifically, it includes: extracting a pair of descriptors based on S2, and using the average of the cosine similarity of the corresponding column vectors in the two real matrices of the pair of descriptors to represent the cosine similarity between the pair of descriptors. Since the viewpoints of the LiDAR are different, there will be a rotation relationship between the two descriptors. Therefore, column offset is used to calculate the real matrix and find the minimum distance as the final similarity.
[0055] S4: Generate topology nodes; specifically including: such as Figure 5 As shown, descriptors are extracted based on S2 to generate a descriptor set C. An empty topology map set TopoMap is established. The set C is traversed, and for each descriptor s, the descriptor with the highest similarity is searched in the topology map set TopoMap. Its similarity is Sim. Descriptors with similarity Sim lower than the similarity threshold Threshold are added to the topology map.
[0056] S5: Determine the optimal topology map; specifically, this includes: exhaustively enumerating similarity thresholds to generate several different topology maps, then calculating and plotting the precision and recall curves of each topology map, and finding the topology map with the optimal precision-recall curve as the final result.
[0057] like Figures 2-3 as well as Figures 6-8 As shown, according to the above embodiment, preferably, the accumulation method in S1 is as follows: for the laser point cloud data of the m-th frame, a voxel map is established based on its point cloud data. The laser point cloud data after the m-th frame is traversed from front to back in chronological order. The traversed laser point cloud data is rotated and translated to the local coordinate system of the m-th frame. The voxel map is updated with the rotated and translated laser point cloud data. The shaded part is a schematic diagram of the updated voxel map. When the voxel map does not change, the traversal stops, and the point cloud data accumulated by the voxel map is used to replace the single frame laser point cloud data of the m-th frame.
[0058] like Figure 4 , Figure 9 As shown, according to the above embodiment, preferably, the coordinate mapping formula for assigning each point in the point cloud data to the equally spaced curved surface grid in S2 is:
[0059]
[0060] Where, N r N represents the number of grid rings. s Let r be the number of sectors, and r be the effective distance from the laser point to the center of the lidar sensor. max r is the maximum effective distance from the laser point to the center of the lidar sensor. min Let be the minimum effective distance from the laser point p to the center of the lidar sensor. Let x represent the value of laser point p on the x-axis in three-dimensional space, y represent the value of laser point p on the y-axis in three-dimensional space, z represent the value of laser point p on the z-axis in three-dimensional space, i represent the lateral index of laser point p in the equally spaced curved grid, increasing clockwise, and j represent the radial index of laser point p in the equally spaced curved grid, increasing from the center to the outer edge. For integer manipulation;
[0061] In S2, the maximum height of the midpoint of the equally spaced curved grid is mapped to a real matrix, with a one-to-one correspondence between the elements of the real matrix. Specifically, this includes using an N... r ×N s The real matrix I is used as the descriptor;
[0062]
[0063]
[0064] The set of laser points in an equally spaced curved grid, where z(p) is the value of point p on the z-axis.
[0065] According to the above embodiment, preferably, the pair of descriptors in S3 is I. q and I c Cosine similarity is expressed as:
[0066]
[0067] Among them, S(I q ,I c ) is the descriptor I q and descriptor I c Cosine similarity between them, N s For the number of sectors, For descriptor I q The column vector of the j1-th column in a real matrix. For descriptor I c The column vector of the j1-th column in a real matrix;
[0068] The final similarity is expressed as:
[0069]
[0070] Among them, Similarity(I q , The final similarity score is denoted as . For descriptor I c The column offset matrix obtained by offsetting n columns of the real matrix has a final value range of [0,1].
[0071] According to the above embodiments, preferably, S5 is implemented as follows:
[0072] S51: Change the similarity threshold Threshold in S4 from 0 to 1 to generate topology maps with different similarity thresholds.
[0073] S52: For each map generated in S51, iterate through the descriptor subset C generated in S4. For each descriptor s in the descriptor subset C, search for the descriptor s with the highest similarity to s in the established topological map. t ;
[0074] S53: Regarding s and s in S52 t Perform the test: If s and s t The similarity is greater than the test threshold C thres And the distance is less than the test distance C d If s and s t The similarity is less than the test threshold C thres And the distance is less than the test distance C d If s and s t The similarity is greater than the test threshold C thres And the distance is greater than the test distance C d If the false positive count is positive, then increment the false positive count by one.
[0075] Among them, the test distance C d = 5 meters, when the test threshold is C thres At that time, the corresponding precision and recall are:
[0076]
[0077]
[0078] S54: Change the test threshold C in S53 thresBy uniformly distributing these values within the interval [0,1], a set of precision and recall rates can be obtained. Plotting this set of precision and recall rates yields a precision-recall curve, which is the precision-recall curve of the topology map. By plotting the precision-recall curves of the topology map corresponding to different similarity thresholds, the topology map with the largest area enclosed by the precision-recall curve is selected as the final result, thus completing the construction of the topology map.
[0079] like Figure 10 As shown, the precision-recall curves for topology maps with similarity thresholds of 0.7, 0.75, 0.78, 0.83, 0.85, 0.87, 0.90, and 0.95 are displayed. When the similarity threshold is 0.90, the area enclosed by the precision-recall curve reaches its maximum, and the number of topology nodes is relatively small. Therefore, this value is selected as the final generated topology map. The topology map construction is now complete. Figure 11 To create effects for the topology map.
[0080] To verify the effectiveness and superiority of the method of this invention, based on the topology map generated from existing scene data, experimental equipment was used to perform secondary data acquisition on the same location, obtaining two additional scene data packets. The scene data packets acquired in this secondary acquisition were used to verify scene recognition against the topology map generated by the method of this invention and the topology map generated by the appearance-based topology map construction method, and the precision-recall curves were plotted. The topology map performance comparison verification graph is shown below. Figure 12 As shown. The topology map generated by the method of this invention achieves the same scene recognition performance as the topology map generated by the appearance-based topology map construction method, while reducing the number of topology nodes from 19278 to 3396. The satellite map of the experimental scene is shown below. Figure 13 As shown.
[0081] The beneficial effects of this invention are: this invention describes the scene based on the multi-frame point cloud data accumulated from the robot's running trajectory, extracts descriptors from the point cloud, and then determines whether to generate topological nodes based on the similarity of the descriptors; this invention can ensure the effective construction of lightweight topological maps, and is particularly suitable for the needs of building topological maps for large-scale scenes; compared with appearance-based topological map methods, this invention greatly reduces the number of nodes required to build topological maps, but still ensures reliable scene recognition performance.
[0082] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.
Claims
1. A lightweight topology map construction method for scene recognition, characterized in that, Includes the following steps: S1: Collect data and process the data to obtain the cumulative point cloud; specifically, after the data is collected, process the robot's running trajectory and laser point cloud data, accumulate each frame of laser point cloud data and its neighboring laser point cloud data, and use the accumulated point cloud data to replace the laser point cloud data. S2: Extract scene descriptors; specifically including: based on the point cloud data obtained in S1, calculate the orientation angle and pitch angle of the points in the point cloud in the local coordinate system, establish an equally spaced curved surface grid, assign each point in the point cloud data to the equally spaced curved surface grid, use a real number matrix as the descriptor, and map the maximum height of the points in the equally spaced curved surface grid to the real number matrix, with each element of the real number matrix corresponding to the other. S3: Calculate the similarity between descriptors; specifically, it includes: extracting a pair of descriptors based on S2, using the average of the cosine similarity of the corresponding column vectors in the two real matrices of the pair of descriptors to represent the cosine similarity between the pair of descriptors, and using column offsets to calculate and find the minimum distance in the real matrix as the final similarity. S4: Generate topology nodes; specifically, this includes: extracting descriptors from S2 to generate a descriptor set C, establishing an empty topology map set TopoMap, traversing set C, searching for the descriptor with the highest similarity for descriptor s in the topology map set TopoMap, with its similarity being Sim, and adding descriptors with similarity Sim lower than the similarity threshold Threshold to the topology map. S5: Determine the optimal topology map; specifically, this includes: exhaustively enumerating similarity thresholds to generate several different topology maps, then calculating and plotting the precision and recall curves of each topology map, and finding the topology map with the optimal precision-recall curve as the final result.
2. The lightweight topology map construction method for scene recognition according to claim 1, characterized in that, The accumulation method in S1 is as follows: For the laser point cloud data of the m-th frame, a voxel map is built based on its point cloud data. The laser point cloud data after the m-th frame is traversed from front to back in chronological order. The traversed laser point cloud data is rotated and translated to the local coordinate system of the m-th frame. The voxel map is updated with the rotated and translated laser point cloud data. When the voxel map has no change, the traversal is stopped, and the point cloud data accumulated by the voxel map is used to replace the single frame laser point cloud data of the m-th frame.
3. The lightweight topology map construction method for scene recognition according to claim 1, characterized in that, The formula for assigning each point in the point cloud data to the coordinate mapping of the equally spaced curved surface grid in S2 is as follows: Where, N r N represents the number of grid rings. s Let r be the number of sectors, and r be the effective distance from the laser point to the center of the lidar sensor. max r is the maximum effective distance from the laser point to the center of the lidar sensor. min Let be the minimum effective distance from the laser point p to the center of the lidar sensor. Let x represent the value of laser point p on the x-axis in three-dimensional space, y represent the value of laser point p on the y-axis in three-dimensional space, z represent the value of laser point p on the z-axis in three-dimensional space, i be the horizontal index of laser point p in the equally spaced curved grid, and j be the horizontal and radial index of laser point p in the equally spaced curved grid. For integer manipulation; In step S2, mapping the maximum height of the midpoints of the equally spaced curved grid to a real matrix and corresponding one-to-one with the elements of the real matrix specifically includes: using an N r ×N s The real matrix I is used as the descriptor; in, Let be the element in the i1-th row and j1-th column of the real matrix I. Let z(p) be the set of laser points in the i1th row and j1th column of the equally spaced curved grid, and z(p) be the value of point p on the z-axis.
4. The lightweight topology map construction method for scene recognition according to claim 1, characterized in that, The pair of descriptors in S3 is I q and I c Cosine similarity is expressed as: Among them, S(I q ,I c ) is the descriptor I q and descriptor I c Cosine similarity between them, N s For the number of sectors, For descriptor I q The column vector of the j1-th column in a real matrix. For descriptor I c The column vector of the j1-th column in a real matrix; The final similarity is expressed as: in, For the final similarity, For descriptor I c The column offset matrix obtained by offsetting n columns of the real matrix has a final value range of [0,1].
5. The lightweight topology map construction method for scene recognition according to claim 1, characterized in that, The specific steps of S5 are as follows: S51: Change the similarity threshold Threshold in S4 from 0 to 1 to generate topology maps with different similarity thresholds. S52: For each map generated in S51, iterate through the descriptor subset C generated in S4. For each descriptor s in the descriptor subset C, search for the descriptor s with the highest similarity to s in the established topological map. t ; S53: Regarding s and s in S52 t Perform the test: If s and s t The similarity is greater than the test threshold C thres And the distance is less than the test distance C d If s and s t The similarity is less than the test threshold C thres And the distance is less than the test distance C d If s and s t The similarity is greater than the test threshold C thres And the distance is greater than the test distance C d If the false positive count is positive, then increment the false positive count by one. Among them, the test distance C d = 5 meters, when the test threshold is C thres At that time, the corresponding precision and recall are: S54: Change the test threshold C in S53 thres By uniformly distributing these values within the interval [0,1], a set of precision and recall rates can be obtained. Plotting this set of precision and recall rates yields a precision-recall curve, which is the precision-recall curve of the topology map. By plotting the precision-recall curves of the topology map corresponding to different similarity thresholds, the topology map with the largest area enclosed by the precision-recall curve is selected as the final result, thus completing the construction of the topology map.