A fast and uniform feature point extraction method based on image partitioning

Abstract

The present invention discloses a method for fast and uniform feature point extraction based on image partitioning, which specifically includes a method for fast and uniform feature point extraction based on image partitioning and an evaluation criterion for uniform distribution of image feature points; in the feature point extraction method of the present invention, first, the image is partitioned according to the number of feature points to be extracted, and then FAST corner features are extracted in parallel for multiple image blocks at the same time; finally, while ensuring low computational time consumption, feature points evenly distributed on the image are obtained, these feature points are non-redundant, and the image feature information is fully retained; the evaluation criterion proposed by the present invention, by designing an optimal distribution model and then comparing and calculating the actual distribution situation with the model to obtain a uniformity distribution coefficient, this method not only considers whether the feature points are evenly distributed on the entire image, but also considers whether the feature points are evenly distributed among each other; this is a feature point uniformity evaluation criterion that considers both global uniformity and local uniformity at the same time.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision, and particularly relates to a method for quickly and evenly extracting feature points based on image partitioning. Background Art

[0002] Simultaneous Localization and Mapping (SLAM) are two fundamental problems in the field of robot navigation and control research. The SLAM technology in the field of robot research is one of the key technologies to jointly solve these two problems. Currently, the SLAM technology is one of the key technologies in the fields of robots, autonomous driving, augmented reality, etc., and is a basic technology for intelligent mobile platforms to perceive changes in the surrounding environment. Since images or videos can provide rich environmental information, most of the SLAM technology research focuses on visual algorithms (VSLAM). In VSLAM, image feature point extraction is one of the cores of SLAM, which is related to subsequent localization and mapping. At the same time, it has a wide range of applications in the fields of image stitching, target tracking, face recognition, 3D reconstruction, etc. However, the existing image feature point extraction methods have uneven extraction results, information redundancy, and cannot provide rich spatial information. Summary of the Invention

[0003] The purpose of the present invention is to provide a method for quickly and evenly extracting feature points based on image partitioning, which solves the problems of uneven extraction results, information redundancy, and inability to provide rich spatial information in the existing image feature point extraction methods.

[0004] The technical solution adopted by the present invention is a method for quickly and evenly extracting feature points based on image partitioning, which specifically includes a method for quickly and evenly extracting feature points based on image partitioning and an evaluation criterion for uniform distribution of image feature points.

[0005] The characteristics of the present invention also lie in:

[0006] Among them, the feature point extraction method specifically includes the following steps:

[0007] Step 1: Divide the image into rectangular regions with the same number as the number of target feature points to be extracted according to the number of target feature points to be extracted;

[0008] Step 2: Construct an image pyramid;

[0009] Step 3: Perform parallel feature point detection on the image sub-block pyramid;

[0010] Step 4: Map all the extracted feature points onto the original image;

[0011] Step 5: For the image blocks where no feature points are extracted, perform compensatory extraction in the form of a quadtree;

[0012] Specifically, in step 1, if the number of target extraction feature points is T, the target image is divided into T image patches of size i×j, where i and j represent the width and height of the image patch, and the calculation methods of i and j are shown in formula (1), where W and H are the width and height of the target image respectively:

[0013]

[0014] Specifically, in step 2: The target image uses a scaling factor of 0.8 to construct an image pyramid with a maximum number of layers of 3, and the image pyramid shown as follows is obtained. The 0th layer represents the original target image; Figure 3 shown as follows, the 0th layer represents the original target image;

[0015] In step 3, parallel feature point detection is performed on the image sub-block pyramid, and the specific steps are as follows:

[0016] Step 3.1: The divided image patches form an image sub-block pyramid in the image pyramid;

[0017] Step 3.2: For each image sub-block pyramid, FAST feature points are extracted one by one from the bottom layer upwards, and when the first feature point is detected, the extraction task of this area is immediately ended;

[0018] Step 3.3: For the T image sub-block pyramids obtained in step 3.1, the method in step 3.2 is used to perform parallel extraction on multiple image sub-block pyramids simultaneously;

[0019] In step 5, for the image patches where no feature points are extracted, compensation extraction is performed in the form of a quadtree, and the specific steps are as follows;

[0020] Step 5.1: The image patches on the original target image are sorted from left to right and from top to bottom;

[0021] Step 5.2: Traverse each image patch according to the sorting. For the image patch where no feature points are detected, find the next image patch with feature points extracted from this image patch, divide this image patch into 4 equal new image regions, and then use the method in step 3 to extract feature points from the newly divided 4 image regions, and at most 4 evenly distributed feature points are obtained. Finally, these feature points are mapped onto the original image;

[0022] The evaluation criteria for the uniform distribution of image feature points are specifically implemented according to the following steps:

[0023] Step 1: Design a template: If there are n feature points on an image with a resolution of W×H, the template situation is that n points are evenly distributed on the entire image, and the distance d between any feature point and its nearest neighbor feature point is calculated according to formula (2), where d = 2*r, H represents the pixel height of the image, and W represents the pixel width of the image:

[0024]

[0025] Step 2: Traverse each feature point actually extracted, find the nearest neighbor feature point for each feature point, calculate the pixel distance between them, and then use formula (3) to calculate the feature point uniform distribution coefficient C, x i represents the pixel distance between the i-th feature point and its nearest neighbor feature point. The smaller the value of C, the better the uniformity, and the closer it is to 1, the worse the uniformity:

[0026]

[0027] The beneficial effects of the present invention are:

[0028] The fast and uniform feature point extraction method based on image division extracted by the present invention first divides the image according to the number of feature points to be extracted for the target, and then simultaneously extracts FAST corner features for multiple image blocks in parallel. Finally, while ensuring low computational time consumption, feature points evenly distributed on the image are obtained. These feature points have no redundancy and fully retain the image feature information. The image feature point uniform distribution evaluation criterion proposed by the present invention designs an optimal distribution model, and then compares the actual distribution situation with the model to calculate the uniformity distribution coefficient. This method not only considers whether the feature points are evenly distributed on the entire image, but also considers whether the feature points are evenly distributed among each other. This is a feature point uniformity evaluation criterion that considers both global uniformity and local uniformity. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 is the flowchart of feature point extraction provided in the embodiment of a fast and uniform feature point extraction method based on image division of the present invention;

[0030] Figure 2 is the schematic diagram of uniform feature point extraction provided in the embodiment of a fast and uniform feature point extraction method based on image division of the present invention;

[0031] Figure 3 is the schematic diagram of image pyramid provided in the embodiment of a fast and uniform feature point extraction method based on image division of the present invention;

[0032] Figure 4 is the schematic diagram of feature point uniform distribution template provided in the embodiment of a fast and uniform feature point extraction method based on image division of the present invention;

[0033] Figure 5 is the feature point extraction result obtained by using the method of the present application, the ORB feature extraction method, and the quadtree uniform extraction method in the embodiment of a fast and uniform feature point extraction method based on image division of the present invention. Detailed implementation manners

[0034] The present invention will be described in detail below in conjunction with the accompanying drawings and specific implementation manners.

[0035] The present invention provides a method for fast and uniform feature point extraction based on image partitioning, which is mainly divided into two parts. The first part is the method for fast and uniform feature point extraction based on image partitioning, and the second part is the evaluation criterion for uniform distribution of image feature points.

[0036] Among them, the method for fast and uniform feature point extraction based on image partitioning is specifically implemented according to the following steps:

[0037] The grid uniformization feature detection method based on the number of target features proposed in this application uses the number of target features to partition the image into grids. At the same time, it combines the idea of quadtree uniformization and uses multi-threading to speed up during feature extraction. The main process of the algorithm is as Figure 1 shown;

[0038] Step 1: According to the number of feature points to be extracted for the target, divide the image into rectangular regions with the same number as the number of target feature points;

[0039] Step 2: Construct an image pyramid. The target image uses a scaling factor of 0.8 to construct an image pyramid with a maximum number of layers of 3, and the image pyramid as shown in Figure 3 is obtained. Layer 0 represents the original target image;

[0040] Step 3: Map each image block after image partitioning into the image pyramid, and each image block obtains an independent block image pyramid. As shown in Figure 2 , use parallel detection means to extract FAST feature points from the bottom up for the block image pyramids in multiple regions at the same time. When the first feature point is detected in a certain region, immediately end the extraction task of this region;

[0041] Step 4: Map all the extracted feature points onto the original image;

[0042] Step 5: Sort the original image blocks from left to right and from top to bottom. For the image blocks where no feature points are detected, find the next image block with feature points extracted for this image block, divide this image block into 4 equal image regions, and then use the method in Step 3 to extract feature points for the newly divided 4 image regions, and at most 4 uniform feature points can be obtained. Finally, map these feature points onto the original image, and the final result is as shown in Figure 2 ;

[0043] In the above method, in step 3, a target feature point is extracted for each divided image region, but there may be a situation where there is no feature point in this region; in step 5, aiming at the deficiency in step 3, the idea of quadtree homogenization is used to perform a compensatory feature point extraction to ensure that the actually obtained number of feature points is as close as possible to the target point number.

[0044] The evaluation criterion for the uniform distribution of image feature points is specifically implemented according to the following steps:

[0045] If n feature points are to be evenly distributed on an image, the best solution is that n circles with a radius of r cover the entire image, and the feature points are distributed at the center points of each circle. This kind of feature point distribution situation is called template distribution. The template schematic diagram is shown in Figure 5 ; The evaluation criterion for the uniform distribution of image feature points in this application is specifically as follows:

[0046] Step 1, design the template: If there are n feature points on an image with a resolution of W×H, the template situation is that n points are evenly distributed on the entire image, and the distance between any feature point and its nearest neighbor feature point is d. The calculation of d is based on formula (2), and the template distribution is as Figure 4 shown, where d = 2*r, H represents the pixel height of the image, and W represents the pixel width of the image;

[0047]

[0048] Step 2, traverse each actually extracted feature point, find the nearest neighbor feature point for each feature point, calculate the pixel distance between them, and then use formula (3) to calculate the feature point uniform distribution coefficient C. x i represents the pixel distance between the i-th feature point and its nearest neighbor feature point. The smaller the value of C, the better the uniformity, and the closer it is to 1, the worse the uniformity.

[0049]

[0050] This application calculates the feature point uniform distribution coefficient based on the difference between the actual feature point distribution situation and the best distribution situation (template situation). This evaluation method takes into account both the uniform distribution of feature points on the entire image and the uniform distribution situation between feature points.

[0051] All experiments used the NVIDIA Jetson TX2 embedded computer module as the computing platform. The ORB feature extraction method, the quadtree homogenization method, and the method of the present invention were used to extract image feature points respectively. First, a comparative analysis of the time efficiency was carried out, and then the uniform feature point evaluation criteria of the present invention were used to compare and analyze the feature points extracted by the three methods. The experiments used four video sequences, namely sequence 14, sequence 21, sequence 30, and sequence 47 in the monocular visual odometry dataset of TUM for experimental verification. First, the dataset was corrected for distortion using the parameters provided by the dataset to obtain undistorted images with a resolution of 640*480, and then relevant experiments on feature extraction and matching were carried out. Sequence 14 is an indoor scene with strong exposure effects, which will directly affect the uniformity of feature point distribution. Sequence 21 is an outdoor forest scene, sequence 30 is an outdoor campus scene, and sequence 47 is a block scene. Compared with sequence 14, the feature distribution of these three sequences is relatively uniform.

[0052] Figure 5 When the number of target extracted feature points is 300, the results of the three methods for extracting feature points in the indoor scene, block scene, and forest scene are shown. The ORB feature point extraction method has the worst uniformity, with redundant feature points and unable to accurately describe the image spatial information. The feature points obtained by the quadtree homogenization method are evenly distributed throughout the image, but there is a phenomenon of local denseness. The extraction result of the method of the present invention is the best, with evenly distributed feature points, fully highlighting the image feature information.

[0053] Table 1 shows the evaluation of the three feature extraction methods using the uniformity evaluation method based on the distance between feature points of the present invention. First, 50, 100, and 300 feature points were extracted from the datasets of 4 types of scenes respectively, and then the uniformity coefficient per frame was calculated. Among them, the method of the present invention performs the best, the quadtree homogenization method ranks second, and the ORB feature extraction method without any uniformity processing has the worst effect, which is consistent with the results in Figure 5 . The uniformity coefficients of the three methods for extracting feature points on sequence 14 are slightly higher than those obtained in other sequence scenes because there are problems of overexposure or underexposure in many images in sequence 14, which will cause information loss in the image areas affected by exposure and unable to extract feature points, resulting in the phenomenon of non-uniform distribution of feature points.

[0054] Table 1 Uniformity Coefficient

[0055]

[0056]

[0057] Table 2 shows the average processing time per frame obtained when testing video sequences in 4 scenarios using 3 methods respectively with the number of target extraction feature points being 50, 100, and 300. The method of the present invention has the shortest average time consumption, and the average time consumption of ORB feature extraction is the longest. ORB feature extraction first extracts FAST feature points and then uses Harries to select feature points, which takes the longest time. The quadtree homogenization method needs to dynamically divide the image area and select the optimal points within the image area, also taking a lot of time. FUFP divides the image according to the number of target feature points and then adopts a parallel extraction method, consuming the shortest time.

[0058] Table 2 Average time consumption per frame for feature extraction (ms)

[0059]

Claims

1. A fast and uniform feature point extraction method based on image partitioning, characterized in that, it specifically includes a fast and uniform feature point extraction method based on image partitioning and an evaluation criterion for uniform distribution of image feature points; The feature point extraction method specifically includes the following steps: Step 1, divide the image into rectangular regions with the same number as the target feature points according to the number of target feature points to be extracted; Step 2, construct an image pyramid; Step 3, perform parallel feature point detection on the image sub-block pyramid; Step 4, map all the extracted feature points to the original image; Step 5, for the image blocks where no feature points are extracted, perform compensatory extraction in the form of a quadtree; In Step 5, for the image blocks where no feature points are extracted, perform compensatory extraction in the form of a quadtree. The specific steps are as follows; Step 5.1, sort the image blocks on the original target image from left to right and from top to bottom; Step 5.2, traverse each image block according to the sorting. For the image blocks where no feature points are detected, find the next image block with feature points extracted for this image block, divide this image block into 4 equal new image regions, then use the method in Step 3 to extract feature points from the 4 newly divided image regions, and at most get 4 uniform feature points. Finally, map these feature points to the original image; The evaluation criterion for uniform distribution of image feature points is specifically implemented according to the following steps: Step 1, design template: If there are W×H feature points on an image with a resolution of n , and the template situation is that n points are evenly distributed on the entire image, and the distance between any feature point and its nearest neighbor feature point is d, d . The calculation of is based on formula (2), where H represents the pixel height of the image, and W represents the pixel width of the image: （2）； Step 2: Traverse each feature point actually extracted, find the nearest neighbor of each feature point, calculate the pixel distance between them, and then use formula (3) to calculate the feature point uniform distribution coefficient C, x i Denote i the pixel distance between the C th feature point and its nearest neighbor feature point. The smaller the value of （3）。 2. The fast and uniform feature point extraction method based on image partitioning according to claim 1, characterized in that, The specific content of step 1 is that if the number of target extraction feature points is T , then the target image is divided into T image blocks of size i×j , i and j represent the width and height of the image block, and the calculation methods of ，i and j are shown in formula (1), where W, H are the width and height of the target image respectively: （1）。 3. The fast and uniform feature point extraction method based on image partitioning according to claim 1, characterized in that, Step 2 is specifically: for the target image, use a scaling factor of 0.8 to construct an image pyramid with a maximum number of layers of 3.

4. The fast and uniform feature point extraction method based on image partitioning according to claim 1, characterized in that, The parallel feature point detection of the image sub-block pyramid in Step 3 is specifically as follows: Step 3.1, the divided image blocks form an image sub-block pyramid in the image pyramid; Step 3.2, extract FAST feature points one by one from the bottom up for each image sub-block pyramid. When the first feature point is detected, immediately end the extraction task for this region; Step 3.

3. For the T image sub-block pyramids obtained in Step 3.1, the method in Step 3.2 is used to extract them in parallel for multiple image sub-block pyramids simultaneously.

Images