Rapid SLAM feature point extraction and processing method based on Voronoi graph dynamic division

Abstract

The invention provides a rapid SLAM feature point extraction and processing method based on Voronoi graph dynamic division, and the method comprises the steps: constructing an image pyramid based on image data, carrying out the reasonable initial seed point selection of a Voronoi graph, carrying out the relatively uniform Voronoi graph division, carrying out the feature point detection in a divided sub-region through employing a FAST algorithm, carrying out the initial screening of feature points and sub-regions, and carrying out the detection of the feature points in the sub-region. According to the method, a Voronoi graph is used for dynamically dividing a feature point dense area, feature points with good quality are selected, redundant feature points are reduced, and then feature point extraction and processing in an SLAM system are achieved. According to the method provided by the invention, the quality of the feature points can be ensured while uniform distribution of the feature points is ensured, the feature points with poor effects are abandoned, calculation for screening areas without the feature points is skipped, the calculation burden in processing scenes with poor texturing property is effectively saved, and the calculation efficiency is improved. And relatively high real-time performance and robustness are still kept in calculation of a global environment.

Description

Technical Field

[0001] The present invention belongs to the technical field of industrial SLAM (Simultaneous Localization and Mapping) robots, and particularly relates to a method for quickly extracting and processing SLAM feature points based on Voronoi diagram partitioning. Background Art

[0002] In the maintenance operation environment with narrow space, insufficient light, and uneven ground, maintenance robots have natural advantages in replacing human work. However, the currently widely used SLAM technology in maintenance robots cannot well adapt to large, uniformly colored, and poor-texture planar areas. When processing image feature points, it will calculate blank areas without feature points in the image layer, and at the same time, it cannot adaptively adjust the reasonable number of detection areas required for each layer of the image. This situation will waste computing costs and time, affect the rate of real-time localization and map construction, and lead to a decline in work efficiency. Summary of the Invention

[0003] The technical problem of the present invention is that in the existing SLAM technology during the extraction and processing of feature points, it is impossible to reasonably divide the feature point area and waste computing resources for blank areas without feature points when the environmental texture is poor and the possible feature points are uneven.

[0004] In order to achieve the above technical features, the object of the present invention is achieved as follows: A method for quickly extracting and processing SLAM feature points based on dynamic Voronoi diagram partitioning, comprising the following steps: Step 1: Collect environmental image data and construct an image pyramid composed of multiple layers of images for each frame of image according to a ratio. Use an RGB-D camera to collect data as the input of environmental image information; Step 2: According to the image pyramid information obtained in Step 1, and based on the total number of feature points to be extracted N , evenly allocate an appropriate number of initial Voronoi diagram seed points for each layer of the image according to a ratio; Step 3: Utilize the initial seed point information in the image pyramid in Step 2, and perform uniform Voronoi diagram partitioning on each layer of the image in the image pyramid; Step 4: For the divided Voronoi regions, sequentially extract feature points using the FAST algorithm. The feature points in each region are extracted following the initial threshold. For relatively sparse regions, appropriately lower the requirements and use the lowest threshold for extraction. Regions that cannot be extracted even with the lowest threshold are marked as blank regions; Step 5: Based on the extraction of the feature points of the sub-Voronoi regions in Step 4, the feature points are screened. The Voronoi regions with a relatively dense number of feature points are further divided, and the feature point with the largest response value within the divided region is retained. For the labeled "inactive blank regions", the screening and calculation are directly skipped; Step 6: Based on the rapid extraction and screening of the feature points in Steps 4 - 5, the feature point extraction and processing process of the entire SLAM system is completed, and it is delivered to the feature point matching stage.

[0005] Preferably, in Step 1, on the ROS platform of Ubuntu 18.04, by using an RGB-D depth camera, the environmental information is collected in real time. The collected image is used as the information input and input to the ORB-SLAM algorithm in the ROS system. On this basis, the OpenCV library is used to process the image. According to the current frame of the input collected information image, it is scaled down proportionally to generate an image pyramid with 6 layers.

[0006] Preferably, during the construction of the 6-layer image pyramid, the original image frame is scaled proportionally to build an image pyramid with 6 layers. The bottom layer L0 is the original image, and the L1 layer is the original image scaled once according to the scaling factor. The L2 layer is the result of scaling the previous L1 layer once according to the scaling factor , and so on until it is scaled to the sixth layer L5.

[0007] Preferably, according to the characteristics of the image pyramid, the image resolution decreases in turn. According to the total number of feature points N to be extracted, according to the scaling factor , the number of feature points for each layer is obtained according to the following calculation method: ; ; where N 0 , N 1 , N 2 , N 3 , N 4 , N 5 are the number of feature points allocated to each layer of the image N i ; S is the area of the 0th layer image, m is the total number of layers of the image pyramid, i represents which layer.

[0008] Preferably, in step 2, according to the number of feature points required for each image layer and the image area, the number of seed points required for the Voronoi diagram is evenly distributed. The calculation method for selecting the number of seed points follows: ; In the formula, Q represents the number of sub-regions to be divided in the image, represents the i th layer image area. Based on the characteristics of the Voronoi diagram, one seed point determines one sub-region. Therefore, the number of seed points required is also Q , and the seed points are evenly distributed in the image.

[0009] Preferably, in the process of dividing the Voronoi diagram in step 3, for each pair of seed points, the edge of the Voronoi diagram is a straight line that perpendicularly bisects the line connecting the two points. At this time, the distance from any point on the boundary to the two nearest seed points is equal. Based on this, the mathematical form of the Voronoi diagram is as follows: ; In the formula, p i is a site with coordinates ([[]] , ), x is any point in the Euclidean space with coordinates (x 1 , x 2 ), is the distance from point x to p i , and it is also the Euclidean distance:

[0010] The set composed of the regions controlled by all sites is the Voronoi diagram, which is defined as: .

[0011] The present invention has the following beneficial effects: 1. The Voronoi diagram division method of the present invention pre-selects and distributes seed points evenly for each layer of the image pyramid, and uses the selected initial seed points to divide the image into more sub-Voronoi regions, so that the entire image is segmented into relatively uniform smaller regions, improving the uniform distribution of subsequent extracted feature points.

[0012] 2. The rule logic of the method of the present invention when extracting FAST corner points is to preferentially maintain the extraction using the initial threshold, initially reducing the feature points with small Harris response values and reducing redundancy. For areas with extremely few feature points, the lowest threshold is used for extraction to improve the overall uniformity of feature points. When the minimum threshold still cannot be used for extraction within the area, the sub-area is marked as an "inactive blank area" to reduce subsequent calculations and operations.

[0013] 3. During the feature point screening process of the present invention, only the sub-Voronoi regions with feature points are calculated. For the marked "inactive blank areas", the calculation is directly skipped, reducing the calculation burden, shortening the processing time, and improving the efficiency. Especially in an environment with extremely uneven feature point distribution and poor texture, the effect is more obvious, which can greatly shorten the calculation time. At the same time, for a uniform scene, it still has good performance and efficiency.

[0014] 4. In the dynamic division of the Voronoi diagram of the present invention, in areas with dense feature points, it continues to be divided into smaller sub-Voronoi regions, and only the feature point with the largest response value within the sub-region is retained, better ensuring the quantity and quality of feature points in the dense region, reducing the subsequent matching calculation amount and redundancy, and improving the matching and mapping accuracy and speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The present invention will be further described below in conjunction with the drawings and embodiments.

[0016] Figure 1 It is a schematic diagram of 6-layer scaling of the image pyramid according to an embodiment of the present invention.

[0017] Figure 2 It is a schematic diagram of the principle of image pyramid scaling according to an embodiment of the present invention.

[0018] Figure 3 It is a schematic diagram of the principle of seed point allocation according to an embodiment of the present invention.

[0019] Figure 4 It is a schematic diagram of the principle of Voronoi diagram division according to an embodiment of the present invention.

[0020] Figure 5 It is a schematic diagram of Voronoi diagram division according to an embodiment of the present invention.

[0021] Figure 6 It is a schematic diagram of the Voronoi diagram according to an embodiment of the present invention.

[0022] Figure 7 It is a Voronoi diagram with uniform image distribution according to an embodiment of the present invention.

[0023] Figure 8 It is a flow chart of feature point extraction and screening within the Voronoi region according to an embodiment of the present invention.

[0024] Figure 9 It is the schematic diagram of feature point extraction and processing within the image area of the embodiment of the present invention. Detailed implementation manners

[0025] The following further describes the implementation manners of the present invention with reference to the accompanying drawings.

[0026] Embodiment 1: As Figures 1-9 , the present invention provides a fast SLAM feature point extraction and processing method based on dynamic division of Voronoi diagrams. The fast SLAM feature point extraction and processing method is based on constructing an image pyramid for each frame of image, reasonably and evenly selecting initial seed points of the Voronoi diagram, adaptively dividing each layer of the image by using the selected seed points, detecting feature points within the divided sub-regions and performing preliminary screening of the feature points and regions, dynamically dividing the feature point dense regions by using the Voronoi diagram, selecting feature points with good quality, reducing redundant feature points, ensuring the quality of the feature points while ensuring the uniform distribution of the feature points, discarding feature points with poor effects and skipping the calculation of regions where no feature points exist (Voronoi regions marked as inactive regions).

[0027] Embodiment 2: A fast SLAM feature point extraction and processing method based on dynamic division of Voronoi diagrams specifically includes the following steps: Step 1: Collect environmental image data and construct an image pyramid consisting of 6 layers of images for each frame of image according to a ratio. Use an RGB-D camera to collect data as the input of environmental image information; Step 2: According to the image pyramid information obtained in Step 1, and based on the total number of feature points N to be extracted, evenly allocate an appropriate number of initial seed points of the Voronoi diagram to each layer of the image according to a ratio; Step 3: Use the initial seed point information in the image pyramid in Step 2 and perform uniform Voronoi diagram division on each layer of the image in the image pyramid; Step 4: For the divided Voronoi regions, sequentially extract feature points using the FAST algorithm. The feature points within each region are extracted following the initial threshold. For relatively sparse regions, the requirements are appropriately reduced, and the lowest threshold is used for extraction. Regions where the lowest threshold cannot be used for extraction are marked as blank regions; Step 5: Based on the extraction of feature points in the sub-Voronoi regions in Step 4, perform screening of the feature points. Further divide the Voronoi regions with a relatively dense number of feature points, and retain the feature point with the largest response value within the divided region. For the marked "inactive blank regions", directly skip the screening and calculation; Step 6: Based on the rapid extraction and screening of feature points in the previous steps, complete the feature point extraction and processing process of the entire SLAM system, and send it to the feature point matching stage.

[0028] Further, in Step 1, on the ROS platform of Ubuntu 18.04, by using an RGB-D depth camera, the environmental information is collected in real time. The collected images are used as information input and input to the improved ORB-SLAM algorithm in the ROS system. On this basis, the OpenCV library is used to process the images. According to the current frame of the input collected information image, it is scaled down proportionally to generate an image pyramid with 6 layers.

[0029] Further, in Step 2, according to the total number of feature points to be extracted, the initial seed points of the Voronoi diagram are reasonably and adaptively evenly distributed to each layer of the image.

[0030] Further, in Step 3, for each layer of pyramid images with relatively uniform seed point information, the images are divided into relatively uniform sub-Voronoi regions according to the Voronoi diagram division, ensuring that the subsequent feature point extraction is as uniform as possible.

[0031] Further, in Step 4, for the divided Voronoi regions, the FAST algorithm is used to extract feature points in turn. The points that meet the set threshold in each region are retained as feature points. For relatively sparse regions, the requirements are appropriately reduced and the lowest threshold is used for extraction to ensure the uniform distribution of feature points. For regions where the response value is less than the lowest threshold, they are regarded as "inactive blank regions".

[0032] Further, in Step 5, according to the detected Voronoi regions, for regions with relatively dense feature points, the Voronoi diagram division method is used to further dynamically divide the region to continue obtaining sub-Voronoi regions, and the feature point with the largest response value in the sub-region is retained. For the marked blank regions, the calculation is directly skipped when screening feature points, thereby saving the calculation amount and reducing the calculation pressure.

[0033] Further, in Step 6, based on the uniformly and well-quality feature points quickly extracted and screened in Steps 4 - 5, they are passed into the feature point matching stage.

[0034] Example 3: Regarding the environmental images collected by the existing depth camera as the data source for data collection, based on the Ubuntu 18.04 platform combined with the OpenCV library, the collected images are processed in the next step.

[0035] Such as Figures 1-2As shown, in the embodiment, an image pyramid is constructed for the current frame, and the original image frame is scaled proportionally according to the scaling factor to construct an image pyramid with 6 layers. In the figure, the bottom layer L0 is the original image, and the L1 layer is the original image scaled once according to the scaling factor . The L2 layer is the result of scaling the previous L1 layer once according to the scaling factor , and so on until the sixth layer L5 is scaled.

[0036] Preferably, according to the characteristics of the image pyramid, the image resolution decreases successively. According to the total number of feature points N to be extracted, and according to the scaling factor , the number of feature points for each layer is obtained according to the following calculation method: ; ; Among them, N 0 , N 1 , N 2 , N 3 , N 4 , N 5 are the number of feature points assigned to each layer of the image N i ; S is the area of the image of the 0th layer, m is the total number of layers of the image pyramid, i represents which layer.

[0037] As Figure 3 shown, according to the number of feature points required for each image layer and the image area, the number of seed points required for the Voronoi diagram is evenly distributed. The selection of the number of seed points follows the following calculation method: ; Among them Q represents the number of sub-regions to be divided in the image, represents the area of the image of the i-th layer. Based on the characteristics of the Voronoi diagram, one seed point determines one sub-region. Therefore, the number of seed points required is also Q , and the seed points are evenly distributed in the image to ensure that the image can obtain a relatively evenly distributed Voronoi region and ensure that the feature points extracted subsequently are as even as possible.

[0038] Preferably, as Figures 4-6As shown, this is the principle of Voronoi diagram partitioning. For each pair of seed points, the edge of the Voronoi diagram is a straight line that perpendicularly bisects the line segment connecting the two points, which means that any point on the boundary is equidistant from the two nearest seed points. The mathematical form of the Voronoi diagram is as follows: ; where p i is a site with coordinates ([[]] , ), x is an arbitrary point in the Euclidean space with coordinates (x 1 , x 2 ), is the distance from point x to p i , which is also the Euclidean distance: ; The set composed of the regions controlled by all sites is the Voronoi diagram, defined as: ; As Figure 7 shown, given the distribution of seed points in the image, based on the principle of Voronoi diagram partitioning, the image can be partitioned to obtain relatively evenly distributed sub-Voronoi regions. The subsequent process of detecting feature points is carried out for the partitioned Voronoi regions.

[0039] Preferably, as shown in the schematic diagram of feature point detection in the Voronoi region in Figure 8 , based on the partitioned regions, the known FAST algorithm is used to sequentially extract the feature points in each region, and the points that meet the set initial threshold in each region are retained as feature points. In regions where the feature points are relatively sparse, the requirements are appropriately reduced and the lowest threshold is used for extraction to ensure the even distribution of feature points. For regions where the Harris response value is less than the lowest threshold, they are regarded as "inactive blank regions".

[0040] Furthermore, according to the detected Voronoi regions, for regions with relatively dense feature points, the Voronoi diagram partitioning method is used to further dynamically partition the region to continue obtaining sub-Voronoi regions, and the number of sub-regions obtained in this step does not exceed 4, and the feature point with the largest response value in the sub-region is retained. For relatively sparse regions, the feature points with relatively large response values in the region are retained to ensure that there are sufficient feature point distributions in the sparse regions as much as possible, making the feature point distribution of the entire image uniform. For the labeled "inactive blank regions", the calculation is directly skipped when screening feature points, thereby saving the calculation amount and reducing the calculation pressure. For scenes with poor texture, the calculation time can be greatly shortened.

[0041] The entire process of feature point detection and processing is asFigure 9 The flowchart shown. After being processed by the improved feature point extraction and screening algorithm, the obtained ORB feature points with uniform distribution and good quality are used for the subsequent feature point matching process.

Claims

1. A fast SLAM feature point extraction and processing method based on dynamic partitioning of Voronoi diagram, characterized in that: The following steps are involved: Step 1: Collect environmental image data and construct an image pyramid consisting of multiple layers of images in proportion for each frame of image. Use an RGB-D camera to collect data as input of environmental image information. Step 2: Based on the image pyramid information obtained in step 1, the total number of feature points to be extracted N , evenly distribute the appropriate number of initial seed points of the Voronoi diagram to each layer of the image in proportion; Step 3: Using the initial seed point information in the image pyramid in step 2, evenly divide each layer of the image in the image pyramid into Voronoi diagrams; Step 4: For the divided Voronoi regions, use the FAST algorithm to extract feature points in sequence. The feature points in each region are extracted according to the initial threshold. The requirements for relatively sparse areas are appropriately lowered, and the lowest threshold is used for extraction. The areas that cannot be extracted using the lowest threshold are marked as blank areas. Step 5: Based on the extraction of feature points in the sub-Voronoi region in step 4, the feature points are screened, and the Voronoi region with dense feature points is further divided. The feature points with the largest response value in the divided region are retained. For the marked "inactive blank area", the screening and calculation are directly skipped; Step 6: Based on the rapid extraction and screening of feature points in steps 4 and 5, the feature point extraction and processing process of the entire SLAM system is completed and transmitted to the feature point matching stage.

2. A fast SLAM feature point extraction and processing method based on dynamic partitioning of Voronoi diagram according to claim 1, characterized in that, In the step 1, on the ROS platform of Ubuntu 18.04, the environmental information is collected in real time by using an RGB-D depth camera. The collected image is used as information input and input into the ORB-SLAM algorithm in the ROS system. On this basis, the image is processed using the OpenCV library. According to the current frame of the input collected information image, it is scaled down to generate an image pyramid with 6 layers.

3. A fast SLAM feature point extraction and processing method based on dynamic partitioning of Voronoi diagram according to claim 2, characterized in that, In the process of building the 6-layer image pyramid, the original image frame is scaled according to the scaling factor Proportional scaling is used to construct an image pyramid with 6 layers. The bottom layer L0 is the original image, and the L1 layer is the original image, which is scaled according to the scaling factor. Once the scaling is performed, the L2 layer is the previous L1 layer according to the scaling factor The result of scaling once, and so on, until scaling to the sixth layer L5.

4. A fast SLAM feature point extraction and processing method based on dynamic partitioning of Voronoi diagram according to claim 3, characterized in that, According to the characteristics of the image pyramid, the image resolution is reduced in sequence, and the total number of feature points N to be extracted is calculated according to the scaling factor , the number of feature points in each layer is obtained according to the following calculation method: ； ； In the formula, N 0. N 1. N 2. N 3. N 4. N 5, the number of feature points assigned to each layer of image N i ; S is the area of ​​the 0th layer image, m is the total number of layers of the image pyramid, i Represents the layer.

5. A fast SLAM feature point extraction and processing method based on dynamic partitioning of Voronoi diagram according to claim 4, characterized in that, In step 2, the number of seed points required for the Voronoi diagram is evenly distributed according to the number of feature points required for each image layer and the image area. The calculation method for selecting the number of seed points is as follows: ； In the formula, Q Represents the number of sub-regions that need to be divided in the image. Representative i Layer image area; Based on the characteristics of the Voronoi diagram, one seed point determines one sub-region, so the number of seed points required is also Q , and evenly distribute the seed points in the image.

6. A fast SLAM feature point extraction and processing method based on dynamic partitioning of Voronoi diagram according to claim 5, characterized in that: In the Voronoi diagram partitioning process in step 3, for each pair of seed points, the edge of the Voronoi diagram is the straight line that perpendicularly bisects the line between the two points. At this time, the distance from any point along the boundary to the two nearest seed points is equal. Based on this, the mathematical form of the Voronoi diagram is as follows: ； In the formula, p i is a site with coordinates ( , ), x is any point in Euclidean space with coordinates (x1,x2), It is point x to p i The distance is also the Euclidean distance: The set of regions controlled by all sites is the Voronoi diagram, defined as: 。

Images