An adjacent two frame image ORB feature point matching method for visual SLAM

Abstract

The present disclosure belongs to the technical field of image processing, and discloses a method for matching ORB feature points of adjacent two frames of images for visual SLAM, comprising: extracting ORB feature points from adjacent two frames of images, performing coarse matching by using a brute force matching method, obtaining N pairs of matching pairs, obtaining the coordinates of each feature point successfully matched on the adjacent two frames of images in the image, including the point on the previous frame (from.x, to.y) and the point on the next frame (to.x, to.y); obtaining the area of the matrix obtained by drawing a perpendicular line to the image from the feature point, the area including S from and S to , obtaining the area ratio of the matrix in the image, the area of the image including S, the area ratio including and taking the absolute value of the difference of the area ratio of each pair of points after brute force matching in the image as |S d |. The method for matching ORB feature points of adjacent two frames of images for visual SLAM solves the problem of a large number of false matches generated by traditional matching methods.

Description

TECHNICAL FIELD

[0001] The present disclosure belongs to the technical field of image processing, and particularly relates to a method for matching ORB feature points of adjacent two frames of images for visual SLAM. BACKGROUND

[0002] Simultaneous Localization And Mapping (SLAM) refers to that a robot completes construction of a map of a surrounding unknown environment by means of an environment perception sensor carried by the robot without any prior knowledge. A technique for completing the map construction by using a visual sensor is referred to as visual SLAM. Image matching is a basis for image analysis and processing, that is, a given image is processed according to a specific purpose by using a computer and corresponding mathematical theory. Feature extraction and matching are key technologies in a SLAM system based on feature point method.

[0003] At present, most of the existing methods based on Hamming distance, such as brute force matching, cross matching and KNN matching, have low precision and low code execution efficiency, and a large number of false matches are retained. SUMMARY

[0004] In view of the deficiencies of the prior art, the purpose of the present disclosure is to provide a method for matching ORB feature points of adjacent two frames of images for visual SLAM, which solves the problems mentioned in the background art.

[0005] According to one aspect of the present disclosure, a method for matching ORB feature points of adjacent two frames of images for visual SLAM is provided, comprising: extracting ORB feature points from adjacent two frames of images, performing coarse matching by using a brute force matching method, obtaining N pairs of matching points, obtaining coordinates of each feature point successfully matched on the image on which the feature point is located, including a point on a previous frame (from.x, to.y) and a point on a next frame (to.x, to.y);

[0006] obtaining an area of a matrix obtained by drawing a perpendicular line to the image from the feature point, the area including S from and S to , obtaining an area ratio of the matrix to the image on which the matrix is located, the area of the image including S, and the area ratio including and determining a difference between the area ratio of each pair of points after the brute force matching to the image on which the pair of points is located, and taking an absolute value as |S d |;

[0007] determining a difference between the vertical coordinates of each pair of points after the brute force matching, the vertical coordinates including from.y and to.y, removing the maximum and minimum outliers, taking an absolute value, summing, and obtaining an average value |H d |;

[0008] |S d | is less than a set threshold value δ, and an absolute value of a height difference of a longitudinal coordinate of each pair of points connected by the brute force matching |from.y-to.y| is less than an average value |H d | of the height difference of the longitudinal coordinate of each pair of points connected by the brute force matching.

[0009] Preferably, the δ is 0.04.

[0010] Preferably, the |H d | value is automatically calculated and adjusted with the change of the image, and the |H d | value is an adaptive threshold value.

[0011] Preferably, the each pair of points connected by the brute force matching must simultaneously satisfy |from.y-to.y|<|H d | and |S d |<δ to be retained.

[0012] According to another aspect of the present application, the present application provides an image processing method, comprising:

[0013] receiving target encoding data, the target encoding data being obtained by encoding a plurality of images by a collection end server;

[0014] a plurality of sub-encoding data corresponding to the plurality of images one by one, the plurality of images being obtained by the collection end server from each sending end respectively, the images obtained from each sending end being collected from an image source device connected to the sending end, and the sub-encoding data being provided with a target identifier;

[0015] splitting the corresponding sub-encoding data from the target encoding data according to the target identifier;

[0016] sending the sub-encoding data to a corresponding receiving end for decoding to obtain the images, and displaying the images by a display device connected to the receiving end.

[0017] Preferably, before the sub-encoding data is sent to the corresponding receiving end for decoding to obtain the images, the method further comprises: determining the receiving end corresponding to the target identifier.

[0018] Preferably, splitting the corresponding sub-encoding data from the target encoding data based on the target identifier comprises:

[0019] determining a position of the target identifier in the target encoding data;

[0020] storing data associated with the position in the target encoding data into a data packet; and

[0021] determining the data packet as the sub-encoding data.

[0022] According to still another aspect of the present application, the present application provides an image processing system, comprising:

[0023] a plurality of first image source devices, respectively configured to generate a first image, thereby obtaining a plurality of first images;

[0024] a plurality of first sending ends, respectively connected with the plurality of first image source devices, configured to send the first images;

[0025] a collection end server, connected with the plurality of first sending ends, configured to splice the plurality of first images received thereby to obtain a target image, and encode the target image to obtain first encoding data;

[0026] a first receiving end, connected with the collection end server, configured to receive the first encoding data and decode the first encoding data to obtain the target image;

[0027] a display device, connected with the first receiving end, configured to display the target image.

[0028] According to still another aspect of the present application, the present application provides an image processing device, comprising:

[0029] an acquisition unit, configured to acquire a first image of a plurality of first image source devices sent by a plurality of first sending ends respectively, thereby obtaining a plurality of first images, wherein the plurality of first sending ends correspond to the plurality of first image source devices one by one;

[0030] a splicing unit, configured to splice the plurality of first images respectively, thereby obtaining a target image;

[0031] an encoding unit, configured to encode the target image respectively, thereby obtaining first encoding data;

[0032] a sending unit, configured to send the first encoding data to a first receiving end respectively, wherein the first encoding data is decoded by the first receiving end to obtain the target image, and the target image is displayed by a display device connected with the first receiving end

[0033] According to still another aspect of the present application, the present application provides a computer readable storage medium, comprising a stored program, wherein the program controls a device where the computer readable storage medium is located to execute the method when the program is running.

[0034] The present disclosure has the following beneficial effects:

[0035] The method for matching ORB feature points of adjacent two image frames for visual SLAM eliminates a large number of false matching pairs caused by traditional image feature point matching methods such as brute force matching by setting a constraint condition, and achieves a good matching accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced below. Obviously, for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0037] Figure 1 is a method flowchart of the present disclosure embodiment; DETAILED DESCRIPTION

[0038] The technical solutions in the embodiments of the present disclosure will be described clearly and completely below in combination with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present disclosure.

[0039] A method for matching ORB feature points of adjacent two image frames for visual SLAM, comprising: extracting ORB feature points from adjacent two image frames, i.e. first extracting FAST corner points and calculating binary descriptors, giving FAST corner points rotation consistency by using a gray centroid method, and performing coarse matching by a brute force matching method, i.e. obtaining N pairs of matching pairs by directly comparing the distance of the binary descriptors of the ORB feature points, obtaining the coordinates of each feature point successfully matched on the image in the adjacent two image frames, including the point from on the previous frame and the point to (to.x, to.y) on the next frame;

[0040] Obtaining the area of the matrix obtained by making the feature point perpendicular to the two sides of the image, the area including S from and S to , obtaining the area ratio of the matrix in the image, the area of the image including S, and the area ratio including and Making the area ratio of each pair of points after brute force matching different, and taking the absolute value as |S d |;

[0041] Making the difference of the vertical coordinates of each pair of points after brute force matching, the vertical coordinates including from.y and to.y, removing the maximum and minimum outliers, taking the absolute value and summing, and obtaining the average value |H d |;

[0042] Let |Sd | less than the set threshold value δ, and the absolute value of the height difference of the longitudinal coordinates of each pair of points connected by the brute-force matching |from.y-to.y| is less than the average value |H d .

[0043] δ is taken as 0.04. The visual SLAM processing changes motion between two adjacent frames, and the position of the point on the two frame images does not change too much. In order to eliminate the false matching of the brute-force matching, S d | is taken as δ=0.04.

[0044] |H d | is automatically calculated and adjusted with the change of the image. d | is an adaptive threshold value.

[0045] Each pair of matched pairs matched by the brute-force matching must satisfy |from.y-to.y|<|H d | and |S d |<δ at the same time.

[0046] As shown in Figure 1 , first, the input adjacent two frame images are subjected to ORB feature point extraction, and then the brute-force matching is used to obtain the coarse matching, and a total of N pairs of matching pairs are obtained. The points of the previous frame are denoted as from, and the points of the next frame are denoted as to. The coordinates of the points in the two frames on the image are calculated as (from.x, from.y) and (to.x, to.y) respectively.

[0047] The areas of the matrices obtained by the vertical lines of the points to the two sides of the image are denoted as S from and S to , and the area ratios of the matrices of the points to the images are denoted as and , wherein S from =from.x*from.y, and S to =to.x*to.y. S refers to the image area, and the image sizes in the video frames are the same by default.

[0048] Each matching pair has two matrices, and the area ratios of the two matrices to the images are taken as the difference and the absolute value to obtain denoted as |S d |.

[0049] The height difference of the longitudinal coordinates of the two feature points of each matching pair is calculated and denoted as from.y-to.y. There are N data in total, and the maximum and minimum outliers are removed. The absolute values of N-2 from.y-to.y are taken and summed to obtain Sum, and the average value |H d | is obtained by using the formula Sum / (N-2).

[0050] After the above steps are performed, a constraint condition is set, i.e., |S d |<0.04 and |from.y-to.y|<|H d |. All matching pairs must satisfy both conditions at the same time to be retained, otherwise discarded.

[0051] Through the above matching method, a large number of false matching pairs caused by traditional image feature point matching methods such as brute force matching can be eliminated, a good matching precision is achieved, and the code execution efficiency is high and the running speed is fast.

[0052] In the description of the present specification, the description referring to the terms "one embodiment", "an example", "a specific example" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present disclosure. In the present specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0053] The basic principles, main features and advantages of the present disclosure are shown and described above. It should be understood by those skilled in the art that the present disclosure is not limited by the above embodiments, and the above embodiments and descriptions in the specification are only illustrative of the principles of the present disclosure. Without departing from the spirit and scope of the present disclosure, various changes and improvements of the present disclosure can be made, and these changes and improvements all fall within the scope of the claimed present disclosure.

Claims

1. A method for matching ORB feature points of adjacent two frames of images for visual SLAM, characterized in that, The method comprises the following steps: ORB feature points are extracted from two adjacent images, and a brute force matching method is used for coarse matching to obtain N matching pairs, and the coordinates of each feature point successfully matched on the two adjacent images are obtained, including the coordinates of the point on the previous frame (from.x, to.y) and the coordinates of the point on the next frame (to.x, to.y); The area of the matrix obtained by drawing a perpendicular line to the image from the feature point, the area including S from and S to , and the area ratio of the matrix to the image, the area of the image including S, the area ratio including and The area ratio of each pair of points after the violent matching to the image is taken as a difference, and the absolute value is |S d |; After the violent matching, the difference of the longitudinal coordinates of each pair of points is calculated, the longitudinal coordinates including from.y and to.y, the maximum and minimum outliers are removed, the absolute values are added, the average value |H is obtained d |; Let |S d | is less than a set threshold value δ, and the absolute value of the height difference of the longitudinal coordinates of each pair of points connected through brute force matching |from.y-to.y| is less than the average value |H d | 2. The method for matching ORB feature points of adjacent two image frames for visual SLAM according to claim 1, characterized in that, The δ is 0.

04.

3. The method of claim 1, wherein the method further comprises: The |H d | value is automatically computed and adjusted as the image changes, and the |H d | value is an adaptive threshold.

4. The method of claim 1, wherein the method further comprises: The each pair of matched pairs matched by the brute force must satisfy |from.y-to.y| < |H d | and |S d | < δ will be retained.

5. An image processing method characterized by, The method comprises the following steps: Receiving target encoding data, the target encoding data being obtained by encoding a plurality of images by a collection end server; A plurality of sub-encoding data corresponding to the plurality of images, the plurality of images being obtained by the collection end server from a plurality of sending ends respectively, the images obtained from the sending ends being collected from image source devices connected to the sending ends, and the sub-encoding data being provided with a target identifier; Splitting the corresponding sub-encoding data from the target encoding data according to the target identifier; Sending the sub-encoding data to a corresponding receiving end for decoding to obtain the images, and displaying the images by a display device connected to the receiving end.

6. The method of claim 5, wherein, Before the step of sending the sub-encoding data to the corresponding receiving end for decoding to obtain the images, the method further comprises the step of determining the receiving end corresponding to the target identifier.

7. The method of claim 5, wherein, The step of splitting the corresponding sub-encoding data from the target encoding data according to the target identifier comprises the following steps: Determining the position of the target identifier in the target encoding data; Storing data associated with the position in the target encoding data into a data packet; and Determining the data packet as the sub-encoding data.

8. An image processing system, characterized by The method comprises the following steps: A plurality of first image source devices are provided for generating a plurality of first images respectively; A plurality of first sending ends are provided for sending the first images respectively; A collection end server is provided for splicing a plurality of first images received by the plurality of first sending ends to obtain a target image, and encoding the target image to obtain first encoding data; A first receiving end is provided for receiving the first encoding data and decoding the first encoding data to obtain the target image; A display device is provided for displaying the target image.

9. An image processing apparatus characterized by comprising: The method comprises the following steps: An acquisition unit is provided for acquiring a plurality of first images generated by a plurality of first image source devices respectively, wherein the plurality of first sending ends correspond to the plurality of first image source devices one by one; A splicing unit is provided for splicing the plurality of first images respectively to obtain a target image; An encoding unit is provided for encoding the target image respectively to obtain first encoding data. A sending unit is configured to send the first encoded data to a first receiving end respectively, wherein the first encoded data is decoded by the first receiving end to obtain the target image, and the target image is displayed by a display device connected to the first receiving end.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium includes a stored program, and the program, when executed, controls a device in which the computer readable storage medium is located to perform the method of any one of claims 1 to 7.

Images