A visual odometry dynamic feature point detection and suppression method

Abstract

The present application relates to a kind of visual odometry dynamic feature point detection and suppression method, comprising: obtaining target image containing ground road and sky, and identifying the segmentation position between ground road and sky;Target image is divided into upper image and lower image based on segmentation position;First feature point is extracted based on upper image, the first motion speed of first feature point under world coordinate system is obtained, and the first average of the first motion speed of all first feature points;Second feature point based on lower image, and the second motion speed of second feature point under world coordinate system is obtained;For each second feature point in lower image, the speed difference Δv i Between its corresponding second motion speed and first average is calculated;Speed difference Δv i With preset threshold δ is compared, if Δv i

Description

TECHNICAL FIELD

[0001] The present application relates to the field of automatic driving, and in particular to a visual odometry dynamic feature point detection and suppression method. BACKGROUND

[0002] Visual odometry (VO) is a process that estimates its own motion by using a single camera or multiple cameras as input, and its application fields include automatic driving, robots, unmanned aerial vehicles, augmented reality (AR), etc. The concept of VO was created by Nister in a landmark paper in 2004. This term is very similar to wheel odometry, which incrementally estimates the motion of a vehicle by integrating the number of turns the wheels have made. Similarly, visual odometry uses a board camera to detect image motion changes to incrementally estimate the pose of the carrier. In order to make visual odometry more effective, there must be enough environmental images, and static images with enough texture can extract motion features. In addition, it is necessary to capture continuous images to overlap the scene.

[0003] Compared with vehicle wheel odometry, the advantage of visual odometry is that it is not affected by the sliding of the wheels on uneven ground or other bad conditions. Since modern robots, cars, etc. are generally equipped with camera sensors, visual odometry does not significantly increase the additional cost. In addition, visual odometry provides more accurate trajectory estimation, with a relative position error range of 0.1% to 2%. This makes visual odometry a useful supplement to wheel odometry, as well as other navigation systems such as the Global Positioning System (GPS), Inertia Measurement Unit (IMU), and radar ranging system. In environments where GPS fails, such as urban high-rise buildings, tunnels, underwater, or space, visual odometry is very important.

[0004] The main way of visual odometry is divided into feature point method and direct method. Among them, the feature point method currently occupies the mainstream, which can work when the noise is large and the camera motion is fast, but the map is sparse feature point; the direct method does not need to extract features, which can establish a dense map, but has the defects of large amount of calculation and poor robustness. In addition, in the texture-rich, pose estimation will also bring a larger error, which will even cause the visual odometry system to diverge and not work normally. For example, Chinese patent CN 109813334 A discloses a real-time high-precision vehicle mileage calculation method based on binocular vision, in which the scheme extracts and matches features by using directly obtained adjacent frame images, which is easily affected by dynamic vehicles and other objects, thereby easily causing the output precision of the visual odometry to be greatly affected. SUMMARY

[0005] The present application aims to provide a visual odometry dynamic feature point detection and suppression method.

[0006] To achieve the above-mentioned purposes, the present application provides a visual odometry dynamic feature point detection and suppression method, comprising:

[0007] S1. Obtain a target image containing a ground road and a sky, and identify a segmentation position between the ground road and the sky based on the target image;

[0008] S2. Divide the target image into an upper image and a lower image based on the segmentation position;

[0009] S3. Extract first feature points in the upper image based on the upper image, and obtain a first motion speed of the first feature points in a world coordinate system and a first average value of the first motion speeds of all the first feature points in the world coordinate system;

[0010] S4. Extract second feature points in the lower image based on the lower image, and obtain a second motion speed v i of the second feature points in the world coordinate system;

[0011] S5. For each second feature point in the lower image, calculate a speed difference Δv i between the corresponding second motion speed v i and the first average value;

[0012] S6. Compare the speed difference Δv i with a preset threshold value δ, when Δv i < δ, it is determined that the second feature point is static and is retained, when Δv i > δ, it is determined that the second feature point is dynamic and is deleted;

[0013] S7. Update the visual odometry by using the retained second feature points.

[0014] According to an aspect of the present application, in step S1, the step of identifying the segmentation position between the ground road and the sky based on the target image comprises:

[0015] S11. Convert the target image into a gray image;

[0016] S12. Detect and identify the road lane features in the target image by using an edge detection operator; wherein the road lane features are lane lines; ​

[0017] S13. Obtain a split position between the ground road and the sky based on the feature point of the distal end of the lane line.

[0018] According to an aspect of the present application, in step S12, in the step of detecting and identifying the road lane features in the target image by using the edge detection operator, at least two lane lines with intervals are obtained.

[0019] According to an aspect of the present application, in step S13, in the step of obtaining the split position between the ground road and the sky based on the feature point of the distal end of the lane line, the step includes:

[0020] taking one of the obtained lane lines as a reference lane line, and based on the reference lane line, obtaining a first lane line feature point pair with the maximum interval distance at the proximal end and a second lane line feature point pair with the minimum interval distance at the distal end of the two lane lines in the direction from the proximal end to the distal end;

[0021] calculating a parallelism error between the first lane line feature point pair and the second lane line feature point pair; and if the parallelism error is less than a preset threshold, obtaining the split position of the target image based on the line connecting the second lane line feature point pair.

[0022] According to an aspect of the present application, in the step of calculating the parallelism error between the first lane line feature point pair and the second lane line feature point pair, if the parallelism error is greater than a preset threshold, the split position is obtained based on the lane line feature point closer to the distal end in the second lane line feature point pair.

[0023] According to an aspect of the present application, in step S2, in the step of dividing the target image into an upper image and a lower image based on the split position, a split line is generated on the target image based on the split position, and the upper image and the lower image are divided based on the split line as a boundary.

[0024] According to an aspect of the present application, the split line is a horizontal split line.

[0025] According to an aspect of the present application, in step S5, the speed difference Δv i is represented as:

[0026] According to one scheme of the present application, the present application fully taps and utilizes the objective actual rule that in the application of urban vehicle-mounted visual odometer, dynamic interference feature points mostly come from the ground road rather than the urban sky, takes the urban sky as a "background board", and only through an algorithm, effective detection and good suppression of image dynamic feature points can be realized, and the precision and reliability of the vehicle-mounted visual odometer are extremely effectively improved.

[0027] According to one scheme of the present application, the present application can realize detection and identification of urban road dynamic feature points without increasing external sensors or devices such as laser radars and millimeter wave radars, is easy to use, and has low implementation cost. BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1 is a step block diagram of a visual odometer dynamic feature point detection and suppression method according to one embodiment of the present application;

[0029] Figure 2 is a target image obtained according to one embodiment of the present application;

[0030] Figure 3 is a schematic diagram of different feature point pairs in a target image according to one embodiment of the present application;

[0031] Figure 4 is a schematic diagram of a segmentation position obtained based on a far-end feature point pair in a target image according to one embodiment of the present application. DETAILED DESCRIPTION

[0032] In order to more clearly illustrate the technical solutions of the embodiments of the present application, further detailed description will be made below.

[0033] As shown in Figure 1 , according to one embodiment of the present application, a visual odometer dynamic feature point detection and suppression method of the present application comprises:

[0034] S1. obtaining a target image containing a ground road and a sky, and identifying a segmentation position between the ground road and the sky based on the target image;

[0035] S2. dividing the target image into an upper image and a lower image based on the segmentation position;

[0036] S3. extracting first feature points in the upper image, obtaining a first motion speed of the first feature points in a world coordinate system, and obtaining a first average value of the first motion speeds of all the first feature points in the world coordinate system

[0037] S4. extracting second feature points in the lower image, and obtaining a second motion speed v of the second feature points in the world coordinate systemi ;

[0038] S5. For each second feature point in the lower image, calculate the speed difference Δv between its corresponding second motion speed v i and the first average value ; i ;

[0039] S6. Compare the speed difference Δv i with a preset threshold value δ, when Δv i < δ, then determine that the second feature point is static and is retained, when Δv i > δ, then determine that the second feature point is dynamic and is deleted.

[0040] S7. Update the visual odometer using the retained second feature points.

[0041] As shown in Figure 1 , according to an embodiment of the present application, in the step S1, in the step of acquiring the target image containing the ground road and the sky, a monocular camera is used to acquire the target image, wherein the monocular camera is installed on the front side of the vehicle for continuous shooting of the environment in front.

[0042] In combination with Figure 1 , Figure 2 , Figure 3 and Figure 4 , according to an embodiment of the present application, in the step S1, in the step of identifying the segmentation position between the ground road and the sky based on the target image, the edge detection of the intersection position of the sky and the ground road in the target image can be completed using the Robert edge detection operator, or the Sobel edge detection operator, or the Granient edge detection operator, or the Krshc edge detection operator, or the Robisnon edge detection operator, or the Prewitt edge detection operator, or the Laplaican edge detection operator, or the Canny edge detection operator, and the specific steps include:

[0043] S11. Convert the target image into a gray image.

[0044] S12. Use the edge detection operator to detect and identify the lane features of the road lane in the target image; in this embodiment, the lane features can be implemented by lane lines; in this embodiment, at least two lane lines with intervals are acquired.

[0045] S13. Obtain the segmentation position between the ground road and the sky based on the feature points of the far end of the lane line. In this embodiment, the segmentation line can be generated in sequence by the obtained segmentation position to distinguish the sky and the ground road. In the step of obtaining the segmentation position between the ground road and the sky based on the feature points of the far end of the lane line, the step comprises:

[0046] Take one of the obtained lane lines as a reference lane line, and in the direction from the near end to the far end, obtain the first lane line feature point pair i1, j1, g1, h1 with the maximum interval distance between the two lane lines based on the reference lane line, and obtain the second lane line feature point pair i'1, j'1, h'1, g'1 with the minimum interval distance between the two lane lines, as shown in Figure 3 .

[0047] Calculate the parallel error between the first lane line feature point pair and the second lane line feature point pair; wherein, according to the first lane line feature point pair (i1, j1), (g1, h1) corresponding to the two lane lines with the maximum interval distance at the near end, the slope k of the straight line connecting the feature point pair can be calculated as: max_dist

[0048]

[0049] Similarly, according to the second lane line feature point pair (i'1, j'1), (h'1, g'1) corresponding to the two lane lines with the minimum interval distance at the far end, the slope k of the straight line connecting the feature point pair can be calculated as: min_dist

[0050]

[0051] Calculate the difference between the slope k and the slope k, to obtain the parallel error ε between the first lane line feature point pair and the second lane line feature point pair: max_dist min_dist

[0052] ε = |k max_dist -k min_dist |.

[0053] In this embodiment, when there are multiple feature point pairs between the two lane lines, the slopes can be calculated respectively and then averaged to obtain the average slope. Details are not described herein.

[0054] In this embodiment, if the parallel error ε is less than a preset threshold T ε , the segmentation position of the target image is obtained based on the connecting line of the second lane line feature point pair, wherein the segmentation position of the target image is obtained by the connecting line of the second lane line feature point pair and its extension, as shown in Figure 4 . ​​​​

[0055] In the embodiment, if the parallel error ε is greater than the preset threshold T ε or the feature point pair between the two lane lines cannot be found, one of the lane lines or the feature point of the far end of the two lane lines is selected to extend the horizontal line to obtain the segmentation position of the target image.

[0056] In another embodiment, according to the installation mode of the monocular camera on the vehicle and the image field of view, the intersection position between the sky and the ground road in the target image is determined by experience and combined with manual observation or manual input of the feature position.

[0057] As shown in Figure 2 , according to an embodiment of the present application, in step S2, in the step of dividing the target image into an upper image and a lower image based on the segmentation position, a segmentation line is generated on the target image based on the segmentation position, and the upper image and the lower image are divided based on the segmentation line as a boundary.

[0058] As shown in Figure 1 and Figure 2 , according to an embodiment of the present application, in step S3, in the step of extracting the first feature point in the upper image, since the buildings such as buildings, overpasses, street lamps, traffic signs, and trees are usually static, they can be used as a reference. Therefore, the first feature point can be extracted based on the static objects.

[0059] As shown in Figure 1 and Figure 2 , according to an embodiment of the present application, in step S4, in the step of extracting the second feature point in the lower image, since the dynamic feature points that cause interference are mostly located on the road, such as vehicles, motorcycles, bicycles, pedestrians, etc. Therefore, if the target image contains the above objects, the corresponding second feature point can be extracted.

[0060] As shown in Figure 2 , according to an embodiment of the present application, the segmentation line is a horizontal segmentation line. In this embodiment, in the process of calculating the parallel error ε between the first lane line feature point pair and the second lane line feature point pair, the parallel error between the line connecting the first lane line feature point pair and the edge of the target image is further calculated, and the horizontal angle of the segmentation position is corrected based on the obtained parallel error, to further ensure the horizontal angle of the segmentation line.

[0061] According to an embodiment of the present application, based on the obtained segmentation line, a transition region is divided on both sides of the segmentation line with a preset width, so as to further delete feature points in the transition region. Through the above setting, the influence of feature points around the segmentation line on the detection accuracy can be further effectively avoided, and the detection accuracy of the present application is further ensured.

[0062] Through the above setting, the present application can more accurately distinguish the upper and lower two parts of the target image based on the segmentation of the target image by the distal end of the lane line, which is beneficial to improving the detection accuracy of the present application. In addition, by using the lane line to divide the image, the dynamic feature points can be more effectively ensured to be divided in the lower region of the target image, and the detection accuracy of the present application is further effectively improved.

[0063] According to an embodiment of the present application, in step S5, the speed difference Δv i is represented as:

[0064] According to an embodiment of the present application, in step S7, in the step of updating the visual odometry by using the reserved second feature points, the updating of the visual odometry data can be realized by taking the second feature points reserved in the foregoing steps as the input of the visual odometry.

[0065] The present application fully excavates and utilizes the objective laws and actual phenomena in the application of vehicle-mounted visual odometry, and can effectively detect and well suppress the dynamic feature points of the image through the algorithm, without increasing external sensors such as laser radars or hardware devices, and has very low application cost. Compared with other dynamic feature point detection and suppression algorithms, the present application is simpler, easier and more effective, and can significantly eliminate the dynamic feature interference in the application of vehicle-mounted visual odometry, and improve the accuracy and reliability of visual positioning.

[0066] The above is only an example of a specific scheme of the present application, and for the devices and structures not described in detail, it should be understood that the general devices and general methods in the art are used to implement them.

[0067] The above only describes one scheme of the present application and is not used to limit the present application. For those skilled in the art, the present application can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A visual odometry dynamic feature point detection and suppression method, characterized in that, The method comprises: S1. obtaining a target image containing a ground road and a sky, and identifying a segmentation position between the ground road and the sky based on the target image; S2. dividing the target image into an upper image and a lower image based on the segmentation position; S3. extracting first feature points in the upper image, and obtaining a first motion speed of the first feature points in a world coordinate system and a first average value of the first motion speeds of all the first feature points in the world coordinate system S4. Extracting a second feature point in the lower image based on the second feature point, and obtaining a second motion speed v of the second feature point in a world coordinate system i ; S5. For each of the second feature points in the lower image, calculate its corresponding second motion velocity v i the velocity difference Δv between the first average value i ; S6. comparing the speed difference Δv i with a preset threshold value δ, when Δv i < δ, it is determined that the second feature point is static and is retained, when Δv i > δ, it is determined that the second feature point is dynamic and is deleted; S7. updating the visual odometer by using the reserved second feature points.

2. The visual odometry dynamic feature point detection and suppression method according to claim 1, characterized in that, In step S1, in the step of identifying the segmentation position between the ground road and the sky based on the target image, the step comprises: S11. converting the target image into a gray image; S12. detecting and identifying the road lane feature in the target image by using an edge detection operator; wherein the road lane feature is a lane line; S13. obtaining the segmentation position between the ground road and the sky based on the feature points at the far end of the lane line.

3. The visual odometry dynamic feature point detection and suppression method according to claim 2, characterized in that, In step S12, in the step of detecting and identifying the road lane feature in the target image by using an edge detection operator, at least two lane lines with intervals are obtained.

4. The visual odometry dynamic feature point detection and suppression method according to claim 3, characterized in that, In step S13, in the step of obtaining the segmentation position between the ground road and the sky based on the feature points at the far end of the lane line, the step comprises: taking one of the obtained lane lines as a reference lane line, and based on the reference lane line, obtaining a first lane line feature point pair with the maximum interval distance at the near end and a second lane line feature point pair with the minimum interval distance at the far end of the two lane lines in the direction from the near end to the far end; calculating the parallelism error between the first lane line feature point pair and the second lane line feature point pair; wherein if the parallelism error is less than a preset threshold, the segmentation position of the target image is obtained based on the line connecting the second lane line feature point pair.

5. The visual odometry dynamic feature point detection and suppression method according to claim 4, characterized in that, In the step of calculating the parallelism error between the first lane line feature point pair and the second lane line feature point pair, if the parallelism error is greater than a preset threshold, the segmentation position is obtained based on the lane line feature point at the far end of the second lane line feature point pair.

6. The visual odometry dynamic feature point detection and suppression method according to claim 5, characterized in that, In step S2, in the step of dividing the target image into an upper image and a lower image based on the segmentation position, a segmentation line is generated on the target image based on the segmentation position, and the upper image and the lower image are divided based on the segmentation line as a boundary.

7. The visual odometry dynamic feature point detection and suppression method according to claim 6, characterized in that, The segmentation line is a horizontal segmentation line.

8. The visual odometry dynamic feature point detection and suppression method according to claim 7, characterized in that, In step S5, the speed difference Δv i is represented as:

Images